XProc: An XML Pipeline Language

A step can bind a document or a sequence of documents to a port in three ways: by source, by URI, or by providing it “here”, as the content of the element establishing the binding. A document must be specified in exactly one of these ways.

The scope of a port name is the component on which it is defined. The names of all input and output ports on any component must be unique.

The scope of step names is the flow graph .

In order for step container components (such as p:for-each and p:choose) to function as “black boxes” the way other components do, it is necessary to make sure that they are wholly self-contained. This means that any inputs, outputs, and parameters that are used within the subpipeline must be declared:

<p:pipeline xmlns:p="http://www.w3.org/2006/08/pipeline">
<p:declare-input port="document"/>
<p:declare-parameter name="makeHTML" required="yes"/>

<!-- for the sake of convenience, we assume these steps take no
     inputs and produce a single output on a port named "result" -->
<p:step name="gen-fo" component="ex:generate-fo-stylesheet"/>
<p:step name="gen-html" component="ex:generate-html-stylesheet"/>

<p:choose name="choose-result">
  <p:declare-input port="document" source="!document"/>
  <p:declare-input port="fo-style" source="gen-fo!result"/>
  <p:declare-input port="html-style" source="gen-html!result"/>
  <p:declare-parameter name="makeHTML" select="$makeHTML"/>

  <p:when test="$makeHTML = '1'">
    <p:step name="makeHTML" component="p:xslt">
      <p:input port="document" source="chose-result!document"/>
      <p:input port="stylesheet" source="choose-result!html-style"/>
    </p:step>
    <p:step name="writeHTML" component="p:serialize">
      <p:input port="document" source="makeHTML!result"/>
    </p:step>
  </p:when>

  <p:otherwise>
    <p:step name="makeFO" component="p:xslt">
      <p:input port="document" source="chose-result!document"/>
      <p:input port="stylesheet" source="choose-result!fo-style"/>
    </p:step>
    <p:step name="writePDF" component="p:fo-to-pdf">
      <p:input port="document" source="makeFO!result"/>
    </p:step>
  </p:otherwise>
</p:choose>

</p:pipeline>

However, in practice, forcing authors to declare all of the inputs and parameters to each component is tedious and error-prone. What's more, the pipeline processor can actually determine what additional declarations are necessary simply by examining the source declarations on the steps contained within the step.

Therefore, we allow the syntactic shortcut of simply referring to in-scope sources and parameters directly.

<p:pipeline xmlns:p="http://www.w3.org/2006/08/pipeline">
<p:declare-input port="document"/>
<p:declare-parameter name="makeHTML" required="yes"/>

<!-- for the sake of convenience, we assume these steps take no
     inputs and produce a single output on a port named "result" -->
<p:step name="gen-fo" component="ex:generate-fo-stylesheet"/>
<p:step name="gen-html" component="ex:generate-html-stylesheet"/>

<p:choose name="choose-result">
  <p:when test="$makeHTML = '1'">
    <p:step name="makeHTML" component="p:xslt">
      <p:input port="document" source="!document"/>
      <p:input port="stylesheet" source="gen-html!result"/>
    </p:step>
    <p:step name="writeHTML" component="p:serialize">
      <p:input port="document" source="makeHTML!result"/>
    </p:step>
  </p:when>

  <p:otherwise>
    <p:step name="makeFO" component="p:xslt">
      <p:input port="document" source="!document"/>
      <p:input port="stylesheet" source="gen-fo!result"/>
    </p:step>
    <p:step name="writePDF" component="p:fo-to-pdf">
      <p:input port="document" source="makeFO!result"/>
    </p:step>
  </p:otherwise>
</p:choose>

</p:pipeline>

These shortcuts are purely syntactic. The processor is required to behave as if appropriate, unique declarations with the corresponding remapping of names had been present.

[Definition: An element from the XProc namespace may have any attribute not from the XProc namespace, provided that the expanded-QName of the attribute has a non-null namespace URI. These attributes are called extension attributes.] The presence of an extension attribute must not cause the flow graph to differ from the flow graph that any other conformant XProc processor might produce. They must not cause the processor to fail to signal an error that a conformant processor is required to signal. This means that an extension attribute must not change the effect of any XProc element except to the extent that the effect is implementation-defined or implementation-dependent.

A processor which encounters an extension attribute that it does not recognize must behave as if the attribute was not present.

[Definition: Outside the context of a “here document”, any element not in the XProc namespace is an extension element.] The presence of an extension element must not cause the flow graph to differ from the flow graph that any other conformant XProc processor might produce. They must not cause the processor to fail to signal an error that a conformant processor is required to signal. This means that an extension element must not change the effect of any XProc element except to the extent that the effect is implementation-defined or implementation-dependent.

Inside the context of a here document , all content is considered quoted so neither extension elements nor XProc elements are said to occur.

A processor which encounters an extension element that it does not recognize must behave as if neither the element nor its attributes, nor any of its content was present.

A p:pipeline represents a pipeline . Its children declare the inputs, outputs, and parameters that the pipeline exposes and the subpipeline that constitutes its definition.

<p:pipeline
  name? = QName>
   (p:declare-input*,
    p:declare-output*,
    p:declare-parameter*,
    p:import*,
    p:declare-component*,
    subpipeline)
</p:pipeline>

If specified, the name must be unique across all available pipelines. If a p:pipeline occurs as the child of a p:pipeline-library element, it must be named.

<p:pipeline name="buildspec">
  <p:declare-input port="document"/>
  <p:declare-input port="stylesheet"/>
  <p:declare-output port="result"/>
  <p:declare-parameter name="validate"/>
  …
</p:pipeline>

A p:declare-input declares an input port.

<p:declare-input
  port = QName
  sequence? = yes|no />

The port attribute defines the name of the port. It is a static error two define two ports with the same name.

An input declaration can indicate if a sequence of documents is allowed to appear on the declared port. If sequence is specified with the value “yes”, then a sequence is allowed. If the sequence is not specified, or has the value “no”, then it is a dynamic error for a sequence of more than one document to appear on the declared port.

The declaration may be accompanied by a binding (or default binding) for the input. This binding can be accomplished by source :

<p:declare-input
  port = QName
  step = step name
  source = port name
  select? = xpath expression
  sequence? = yes|no />

by URI :

<p:declare-input
  port = QName
  href = URI
  select? = xpath expression
  sequence? = yes|no />

or by here document :

<p:declare-input
  port = QName
  select? = xpath expression
  sequence? = yes|no>
   heredocument
</p:declare-input>

If a by source binding is used, the port selected must be an output port on a step which is not a descendant of the step on which the p:declare-input appears or it must be a port declared with p:declare-input on some ancestor (e.g., an enclosing p:choose or p:for-each) of the step.

If a binding is provided, a select expression may also be provided. If provided, the specified XPath select expression is used to filter the document(s) that are read. Each matching node or set of nodes is wrapped in a document and provided to the input port. See Section 4.2.6, “p.input Element”.

A p:declare-parameter declares a parameter.

<p:declare-parameter
  name = token />

Components must declare the parameters that they accept.

The name attribute must be a single asterisk (*), a QName, or a string of the form *: NCName or NCName :*.

If name is a QName, then the declaration may include a default value using either content or a select attribute as per p:param.

A p:declare-output identifies an output port.

<p:declare-output
  port = QName
  sequence? = yes|no />

The port attribute defines the name of the port. It is a static error to declare two ports with the same name.

An output declaration can indicate if a sequence of documents is allowed to appear on the declared port. If sequence is specified with the value “yes”, then a sequence is allowed. If the sequence is not specified, or has the value “no”, then it is a dynamic error if the component produces a sequence of more than one document on the declared port.

On a step container, the declaration must be accompanied by a binding for the output. This binding can be accomplished by source :

<p:declare-output
  port = QName
  step = step name
  source = port name
  sequence? = yes|no />

by URI :

<p:declare-output
  port = QName
  href = URI
  sequence? = yes|no />

or by here document :

<p:declare-output
  port = QName
  sequence? = yes|no>
   heredocument
</p:declare-output>

If a by source binding is used, the port selected must be an output port on an step which is a descendant of the step on which the p:declare-output appears.

A p:step represents a component in a pipeline.

<p:step
  component = QName
  name = QName>
   (p:input*,
    p:import-param*,
    p:parameter*)
</p:step>

The component attribute identifies the component to be instantiated. It is a static error if the name is not unique in the current scope, if the specified component is not known to the processor, or if the specified inputs, outputs, and parameters do not match the component's signature .

A p:input identifies input for a component.

Inputs identify their source in exactly one of three ways, by source :

<p:input
  port = NCName
  step = step name
  source = port name
  select? = xpath expression />

by URI:

<p:input
  port = NCName
  href = URI
  select? = xpath expression />

or by here document :

<p:input
  port = NCName
  select? = xpath expression>
   heredocument
</p:input>

It is a static error if more than one of source, uri or here document is specified.

The port attribute identifies the input port of the component that will read from the specified source. It is a static error if the name given does not match the name of an input port for the component or if more than one input is specified for any given port.

The select expression, if specified, applies the specified XPath select expression to the document(s) that are read. Each matching node or set of nodes is wrapped in a document and provided to the input port. In other words,

<p:input port="document" href="http://example.org/input.html"/>

provides a single document, but

<p:input port="document" href="http://example.org/input.html" select="//html:div"/>

provides a sequence of zero or more documents, one for each matching html:div in http://example.org/input.html.

<p:input port="document" source="stepname!portname" select="//html:div"/>

provides a sequence of zero or more documents, one for each matching html:div in the document (or each of the documents) that is read from the portname port of the step named stepname.

A p:param associates a particular value with a parameter.

The value of the parameter can be specified in several ways; as the content of the p:param element:

<p:param
  name = QName>
   any content
</p:param>

In this case, the (XPath 1.0) string value of the content becomes the value of the parameter.

With a value attribute:

<p:param
  name = QName
  value = string />

Or as an XPath expression. In this case, the context in which the expression is evaulated must be specified in exactly one of three ways, by source :

<p:param
  name = Qname
  step = step name
  source = port name
  select? = xpath expression />

by URI:

<p:param
  name = QName
  href = URI
  select? = xpath expression />

or by here document :

<p:param
  name = QNName
  select? = xpath expression>
   heredocument
</p:param>

The (XPath 1.0) string value of the result of evaluating the expression becomes the value of the parameter.

An p:import-param provides a set of in-scope parameters to a component.

<p:import-param
  name = token />

All in-scope parameters which match the name are made available to the component as if they had been specified with individual p:param elements.

The name attribute must be a single asterisk (*), a QName, or a string of the form *: NCName or NCName :*.

A p:for-each represents a for-each .

<p:for-each
  name = NCName>
   (p:declare-input,
    p:declare-output*,
    p:declare-parameter*,
    subpipeline)
</p:for-each>

Exactly one input must be declared and it must include a binding for the port it declares. If outputs are declared, they must also include a binding. The processor will provide each document read through that binding to the subpipeline that the p:for-each contains, one at a time. For each declared output, the processor will collect all the documents that are produced for that output from all the iterations, in order, into a sequence. The result of the p:for-each is that set of document sequences.

Example 2, “A Sample For-Each” shows an example of a p:for-each in action.

<p:for-each name="chapters">
  <p:declare-input port="chap" href="http://example.org/docbook.xml" select="//chapter"/>
  <p:declare-output port="html" source="xform-to-html!result"/>
  <p:declare-output port="fo" source="xform-to-fo!result"/>
  <p:step name="xform-to-fo" component="p:xslt">
    <p:input name="document" source="chapters!chap"/>
    <p:input name="stylesheet" href="fo/docbook.xsl"/>
  </:step>
  <p:step name="xform-to-html" component="p:xslt">
    <p:input name="document" source="chapters!chap"/>
    <p:input name="stylesheet" href="html/docbook.xsl"/>
  </:step>
</p:for-each>

The //chapters of the DocBook document are selected. Each chapter is transformed into HTML and XSL FO using an XSLT step. The resulting HTML and FO documents are aggregated together and appear on the html and fo ports, respectively, of the chapters step.

It is a static error if there is not exactly one p:declare-input child of p:for-each, if the declared input does not specify a binding, or if it specifies a binding to a step inside the p:for-each.

It is a static error if any declared output does not specify a binding.

A p:viewport represents a viewport .

<p:viewport
  name = NCName>
   (p:declare-input,
    p:declare-output,
    p:declare-parameter*,
    subpipeline)
</p:viewport>

Exactly one input must be declared and it must include both a binding and a select expression. Exactly one output must be declared and it must include a binding. The processor will provide a document that contains each set of nodes that matches the specified select expression through the input binding to the subpipeline that the p:viewport contains, one at a time. What appears on the output from the p:viewport will be a copy of the input document except that where each matching node or set of nodes appears, the result of applying the subpipeline to those nodes will be output.

It is a dynamic error if the input source is a sequence of more than one document or if the output from any iteration is a sequence of more than one document.

Example 2, “A Sample For-Each” shows an example of a p:for-each in action.

<p:viewport name="encdivs">
  <p:declare-input port="div" source="step!port" select="//h:div[@class='enc']"/>
  <p:declare-output port="html" source="encrypt!result"/>
  <p:step name="encrypt" component="p:encrypt-document">
    <p:input name="document" source="encdivs!div"/>
  </:step>
</p:viewport>

The //h:div[@class='enc']s of the document are selected. Each selected div is encrypted and the resulting encrypted version replaces the original div. The result of the step is a copy of the input document with each selected div encrypted.

It is a static error if there is not exactly one p:declare-input child and exactly one p:declare-output child of p:viewport, if the declared ports do not specify a binding, or if the input port specifies a binding to a step inside the p:viewport.

A p:choose represents a choose .

<p:choose
  name = NCName
  source? = source specification>
   (p:when*,
    p:otherwise?)
</p:choose>

Where p:when specifies a conditional branch.

<p:when
  test = expression
  source? = source specification>
   (p:declare-output*,
    p:declare-parameter*,
    subpipeline)
</p:when>

And p:otherwise specifies the default branch.

<p:otherwise>
   (p:declare-output*,
    p:declare-parameter*,
    subpipeline)
</p:otherwise>

The test expression is evaluated for each p:when in turn. The first p:when for which the expression evaluates to “true” is selected, all other p:when elements (and the p:otherwise) are ignored. If no p:when has a test expression which evaluates to “true”, then the p:otherwise is selected and all p:when elements are ignored.

The context for each test expression is the document specified by the source attribute on the corresponding p:when. If no source attribute is specified on the p:when, then the context is the document specified by the source attribute on the p:choose. It is a static error if no source is specified in either place.

All of the p:when branches and the p:otherwise must declare the same number of output ports with the same names. It is a static error if they do not.

The result of the p:choose is the result of the selected branch. It is a dynamic error if no p:when is selected and no p:otherwise is specified.

A p:group is a wrapper for a subpipeline .

<p:group
  name = NCName>
   (p:declare-output*,
    p:declare-parameter*,
    subpipeline)
</p:group>

The result of a p:group is its declared outputs.

A p:try represents a try/catch .

<p:try
  name = NCName>
   (p:group,
    p:catch)
</p:try>

Where p:group is any pipeline group and p:catch surrounds the error recovery behavior:

<p:catch>
   (p:declare-output*,
    p:declare-parameter*,
    subpipeline)
</p:catch>

The p:try component evaluates the p:group. If the group evaluates without signaling an error, that is the result of the p:try component.

However, if any component in that group signals an error, then the group is abandoned (any accumulated output is discarded) and the p:catch subpipeline is evaluated. In that case, the result of the p:catch is the result of the p:try component.

If any component in the p:catch subpipeline signals an error, the p:try fails.

Within the p:catch block, the special input port !#error is defined. The document(s) on that port constitute the error messages received from the component which failed. Note that the order of the messages on that port is undefined. Note also that the failure of one component may cause others to fail and the component which signaled the error may not be the only or even the first component that failed.

Both the p:group and the p:catch must declare the same number of output ports with the same names. It is a static error if they do not.

A p:declare-component provides the name and signature of an implementation-dependent component. It declares the inputs, outputs, and parameters of the component.

<p:declare-component
  name = QName>
   (p:declare-input*,
    p:declare-output*,
    p:declare-parameter*)
</p:declare-component>

It is a static error if a pipeline step refers to a component that is not recognized by the processor. It is not an error to declare such a component, only to use it.

Exactly one input declaration of a p:declare-component may use the name “*” to indicate that the component accepts an arbitrary number of inputs.

A p:pipeline-library contains one or more component declarations and/or pipelines. It declares components that pipelines can import.

<p:pipeline-library>
   (p:import*,
    p:declare-component*,
    p:pipeline*)
</p:pipeline-library>

<p:pipeline-library>
  <p:declare-component name="extension-component">…</p:declare-component>
  <p:pipeline name="xinclude-and-validate">…</p:pipeline>
  <p:pipeline name="validate-and-transform">…</p:pipeline>
  …
</p:pipeline>

An p:import loads a pipeline or pipeline library, making it available in the current context.

<p:import
  href = URI />

An import statement loads the specified URI and makes any pipelines declared within it available to the current pipeline.

It is a dynamic error if the URI cannot be retrieved or if, once retrieved, it does not point to a p:pipeline-library or p:pipeline. If it points to a p:pipeline, it is a dynamic error if the pipeline does not have a name.