PROV-AQ: Provenance Access and Query

1.1 Concepts

In defining the specification below, we make use of the following concepts.

Provenance information

refers to provenance represented in some fashion.

Provenance-URI

a URI denoting some provenance information.

Constrained resource

an aspect, version or instance of a resource, about which one may wish to present some provenance information. For example, a weather report for a given date may be an aspect of a resource that is maintained as the current weather report. A constrained resource is itself a resource, and may have it's own URI different from that of the original. See also [PROV-DM], and [WEBARCH] section 2.3.2.

Target-URI

a URI denoting a resource (including any constrained resource), which identifies that resource for the purpose of finding and expressing provenance information associated with it (see section 1.2 for discussion)

Provenance service

a service that provides provenance information given a target-URI.

Service-URI

the URI of a provenance service.

Resource

a resource in the general sense of "whatever might be identified by a URI", as described by the Architecture of the World Wide Web [WEBARCH], section 2.2. A resource may be associated with multiple instances or views (constrained resources) with differing provenance.

1.2 Provenance and resources

Fundamentally, provenance information is about resources. In general, resources may vary over time and context. E.g., a resource describing the weather in London changes from day-to-day, or a listing of restaurants near you will vary depending on your location. Provenance information, to be useful, must be persistent and not itself dependent on context. Yet we may still want to make provenance assertions about dynamic or context-dependent resources (e.g. a weather forecast for London on a particular day may have been derived from a particular set of Meteorological Office data).

Provenance descriptions of dynamic and context-dependent resources are possible through a notion of constrained resources. A constrained resource is simply a resource (in the sense defined by [WEBARCH], section 2.2) that is a contextualized view or instance of some other resource. For example, a W3C specification typically undergoes several public revisions before it is finalized. A URI that refers to the "current" revision might be thought of as denoting the specification throughout its lifetime. Each individual revision would also have its own target-URI denoting the specification at that particular stage in its development. Using these, we can make provenance assertions that a particular revision was published on a particular date, and was last modified by a particular editor. Target-URIs may use any URI scheme, and are not required to be dereferencable.

Requests for provenance about a resource may return provenance information that uses one or more target-URIs to refer to versions of that resource. Some given provenance information may use multiple target-URIs if there are assertions referring to the same underlying resource in different contexts. For example, provenance information describing a W3C document might include information about all revisions of the document using statements that use the different target-URIs of the various revisions.

In summary, provenance information may be not universally applicable to a resource, but may be expressed with respect to that resource in a restricted context (e.g. at a particular time). This restriction is itself just another resource (e.g. the weather forecast for a give date as opposed to the current weather forecast), with its own URI for referring to it within provenance information.

1.3 Interpreting provenance information

Any given provenance information may contain information about several resources, referring to them using their various target-URIs. Thus, when interpreting provenance information, it is important to be aware that statements about several resources may be present, and to be accordingly selective when using the information provided. (In some exceptional cases, it may be that the provenance information returned does not contain any information relating to a specific associated resource.)

Provenance information is not of itself guaranteed to be authoritative or correct. Trust in provenance information must be determined separately from trust in the original resource. Just as in the web at large, it is a user's responsibility to determine an appropriate level of trust in any other resource; e.g. based on the domain that serves it, or an associated digital signature. (See also section 7. Security considerations.)

3.1 Resource accessed by HTTP

For a resource accessible using HTTP, provenance information may be indicated using an HTTP Link header field, as defined by Web Linking (RFC 5988) [LINK-REL]. The Link header field is included in the HTTP response to a GET or HEAD operation (other HTTP operations are not excluded, but are not considered here).

A provenance link relation type for referencing provenance information is registered according to the template in section 6. IANA considerations, and may be used as shown:

Link: provenance-URI; rel="provenance"; anchor="target-URI"

If no anchor parameter is provided then the target-URI is assumed to be the URI of the resource, used in the corresponding HTTP request.

At this time, the meaning of these links returned with other HTTP response codes is not defined: future revisions of this specification may define interpretations for these.

An HTTP response may include multiple provenance link header fields, indicating a number of different provenance resources that are known to the responding server, each providing provenance information about the accessed resource.

The presence of a provenance link in an HTTP response does not preclude the possibility that other publishers may offer provenance information about the same resource. In such cases, discovery of the additional provenance information must use other means (e.g. see section 4. Provenance services).

Provenance resources indicated in this way are not guaranteed to be authoritative. Trust in the linked provenance information must be determined separately from trust in the original resource. Just as in the web at large, it is a user's responsibility to determine an appropriate level of trust in any other linked resource; e.g. based on the domain that serves it, or an associated digital signature. (See also section 7. Security considerations.)

Link: provenance-service-URI; anchor="target-URI"; rel="provenance-service"

3.2 Resource represented as HTML

For a document presented as HTML or XHTML, without regard for how it has been obtained, provenance information may be associated with a resource by adding a <link> element to the HTML <head> section. Two new link relation types for referencing provenance information are registered according to the template in section 6. IANA considerations, and may be used as shown:

  <html xmlns="http://www.w3.org/1999/xhtml">
     <head>
        <link rel="provenance" href="provenance-URI">
        <link rel="anchor" href="target-URI">
        <title>Welcome to example.com</title>
     </head>
     <body>
        ...
     </body>
  </html>

The provenance-URI given by the provenance link element identifies the provenance-URI for the document.

The target-URI given by the anchor link element specifies an identifier for the document that may be used within the provenance information when referring to the document.

An HTML document header may include multiple provenance link elements, indicating a number of different provenance sources that are known to the creator of the document, each of which may provide provenance information about the document.

If no "anchor" link element is provided then the target-URI is assumed to be the URI of the document. It is recommended that this convention be used only when the document is static and has a stable URI that is reasonably expected to be available to anyone accessing the document (e.g. when delivered from a web server, or as part of a MIME structure containing content identifiers [RFC2392]).

  <html xmlns="http://www.w3.org/1999/xhtml">
     <head>
        <link rel="provenance-service" href="service-URI">
        <link rel="anchor" href="target-URI">
        <title>Welcome to example.com</title>
     </head>
     <body>
        ...
     </body>
  </html>

3.3 Resource represented as RDF

If a resource is represented as RDF (in any of its recognized syntaxes, including RDFa), it may contain references to its own provenance using additional RDF statements. For this purpose three new RDF properties, prov:hasProvenance, prov:hasAnchor, and prov:hasProvenanceService, are defined as follows. (These terms may be used to indicate provenance of other resources too, but discussion of such usage is beyond the scope of this section.)

The RDF property prov:hasProvenance is defined as a relation between two resources, where the object of the property is a resource that provides provenance information about the subject resource. Multiple prov:hasProvenance assertions may be made about a subject resource. This property corresponds to a provenance link relation used with an HTTP Link header field, or HTML <link> element.

Property prov:hasAnchor specifies a target-URI used in the provenance information to refer to the containing RDF document. This corresponds to use of the anchor parameter in an HTTP provenance Link header field, or an anchor link relation in an HTML <link> element, which similarly indicate a URI used by the provenance information to refer to the described document.

Property prov:hasProvenanceService specifies a service-URI associated with the RDF document for possible access to provenance information. This property corresponds to a provenance-service link relation used with an HTTP Link header field, or HTML <link> element.

@prefix prov: <http://www.w3.org/ns/prov#>

<> dcterms:title      "Welcome to example.com" ;
   prov:hasAnchor     <http://example.com/data/resource.rdf> ;
   prov:hasProvenance <http://example.com/provenance/resource.rdf> ;
   prov:hasProvenanceService <http://example.com/provenance-service/> .
    :
   (RDF data)
    :
    

3.4 Arbitrary data

If a resource is represented using a data format other than HTML or RDF, and no URI for the resource is known, provenance discovery becomes trickier to achieve. This specification does not define a specific mechanism for such arbitrary resources, but this section discusses some of the options that might be considered.

For formats which have provision for including metadata within the file (e.g. JPEG images, PDF documents, etc.), use the format-specific metadata to include a target-URI, provenance-URI and/or service-URI. Format-specific metadata provision might also be used to include provenance information directly in the resource.

Use a generic packaging format that can combine an arbitrary data file with a separate metadata file in a known format, such as RDF. At this time, it is not clear what format that should be, but some possible candidates are:

• MIME multipart/related [RFC2387]: both email and HTTP are based on MIME or MIME-derivatives, so this has the advantage of working well with the network transfer mechanisms discussed in the motivating scenarios considered.
• Composite object-packaging work from the digital library community, of which there are several (e.g., ORE, METS, MPEG-21 DIDL, BagIt) to name a handful.
• Packaging formats along the lines of those used for shipping Java web applications (basically, a ZIP file with a manifest and some imposed structure)
• Ongoing work in the research community (e.g. Why Linked Data is Not Enough for Scientists, ePub, etc.) to encapsulate data, code, annotations and metadata into a common exchangeable format.

4.1 HTTP GET

This protocol combines the target-URI with the service-URI to formulate an HTTP GET request, according to the following convention:

  GET /provenance/service?target=http://www.example.com/entity HTTP/1.1
  Host: example.com

The embedded target-URI (http://www.example.com/entity) identifies the resource for which provenance information is to be returned. Any server that implements this protocol and receives a request URI in this form should return provenance information for the resource-URI embedded in the query component where that URI is the result of percent-decoding the value associated with the provenance-resource key. The embedded URI must be an absolute URI and the server must respond with a 400 Bad Request if it is not. If the supplied resource-URI includes a fragment identifier, the '#' must be %-encoded as %23 when constructing the provenance-URI value; similarly, any '&' character in the resource-URI must be %-encoded as %26.

If the provenance information identified in the request does not exist in the server, a 404 Not Found response code should be returned.

The format of returned provenance information is not defined here, but may be established through content type negotiation using Accept: header fields in the HTTP request. A provenance service should be capable of returning RDF using the vocabulary defined by [PROV-O], in any standard RDF serialization (e.g. RDF/XML), or any other standard serialization of the Provenance Model specification [PROV-DM]. Services must identify the Content-Type of the information returned.

4.2 Provenance service discovery

This specification does not define any specific mechanism for discovering provenance services. Applications may use any appropriate mechanism, including but not limited to: prior configuration, search engines, service registries, etc.

To facilitate service discovery, we recommend that RDF publication of service descriptions uses the provenance service type prov:ProvenanceService, defined by the provenance ontology [PROV-O]. The RDF service description example below in section 4.3 Provenance service description shows this use.

@@TODO: sync up term definitions with provenance ontology specification.

4.3 Provenance service description

Dereferencing a provenance service URI should yield a provenance service description. This is to be compatible with the constraints of [REST-APIs]. The provenance service description should be available as RDF (in any of its common serializations, and determined through HTTP content negotiation), and it should contain RDF statements of the form:

  <service-URI> a prov:ProvenanceService ;
    prov:provenanceUriTemplate "service-URI?target={+uri}" .

where service-URI is the URI of the provenance service. Note that the object of the prov:provenanceUriTemplate statement is a literal text value, not a URI.

A client may retrieve this service description and extract the associated value for prov:provenance-URI-template. This value is a string containing a URI template [URI-template] (level 2). A URI for the desired provenance information is obtained by expanding the URI template with the variable uri set to the resource-URI for which provenance is required. If the target-URI contains '#' or '&' these must be %-escaped as %23 or %26 respectively before template expansion.

@@TODO: sync up term definitions with provenance ontology specification.

5.1 Using SPARQL for provenance queries

Simply identifying and retrieving provenance information as a resource on the Web may not always meet the requirements of a particular application or service, e.g.:

• the resource for which provenance information is required is not identified by a known URI
• the provenance information for a resource is not directly identified by a known URI
• a requirement to access provenance information for a number of distinct but related resources in a single atomic operation
• etc.

A provenance query service provides an alternative way to access provenance information and/or provenance-URIs. An application will need a URI for the provenance query service, and some relevant information about the resource whose provenance is to be accessed.

The details of a provenance query service is an implementation choice, but for interoperability between different providers and users we recommend use of SPARQL [RDF-SPARQL-PROTOCOL] [RDF-SPARQL-QUERY]. The query service URI would then be the URI of a SPARQL protocol service (often referred to as a "SPARQL endpoint"). The following subsections provide examples for what are considered to be some plausible common scenarios for using SPARQL, and are not intended to cover all possibilities.

A SPARQL protocol service description may be published using the SPARQL 1.1 Service Description vocabulary [SPARQL-SD].

The following subsections illustrate use cases for querying a SPARQL-based provenance query service.

@prefix prov: <http://www.w3c.org/ns/prov#>
SELECT ?provenance_uri WHERE
{
  <http://example.org/resource> prov:hasProvenance ?provenance_uri
}
          

@prefix prov: <http://www.w3c.org/ns/prov#>
@prefix prism: <http://prismstandard.org/namespaces/basic/2.0/>
SELECT ?provenance_uri WHERE
{
  [ prism:doi "1234.5678" ] prov:hasProvenance ?provenance_uri
}
          

@prefix prov: <http://www.w3c.org/ns/prov#>
SELECT ?generationStartTime WHERE {
    <http://example.org/resource> prov:wasGeneratedBy ?activity .
    ?activity prov:startedAtTime ?generationStartTime .
}
          

5.2 Incremental Provenance Retrieval

Provenance information may be large. While this specification does not define how to implement scalable provenance systems, it does allow for publishers to make available provenance in an incremental fashion. We now discuss two possibilities for incremental provenance retrieval.

6.1 Registration template for link relation: "provenance"

Relation Name:

provenance

Description:

the resource identified by target IRI of the link provides provenance information about the resource identified by the context link

Reference:

@@this spec, @@provenance-model-spec

Notes:

...

Application Data:

...

6.2 Registration template for link relation: "anchor"

Relation Name:

anchor

Description:

when used in conjunction with a "provenance" link, the resource identified by target IRI of the link is one for which provenance information may be provided. This may be used, for example, to isolate relevant information from a referenced document that contains provenance information for several entities.

Reference:

@@this spec, @@provenance-model-spec

Notes:

...

Application Data:

...

6.3 Registration template for link relation: "provenance-service"

Relation Name:

provenance-service

Description:

the resource identified by target URI of the link is an provenance service per section 4. Provenance services of this specification.

Reference:

@@this spec, @@provenance-model-spec

Notes:

...

Application Data:

...