This document defines the procedures and rules to be applied when converting tabular data into RDF. Tabular data may be complemented with metadata annotations that describe its structure, the meaning of its content and how it may form part of a collection of interrelated tabular data. This document specifies the effect of this metadata on the resulting RDF.
Status of This Document
This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.
The CSV on the Web Working Group was chartered to produce a recommendation "Access methods for CSV Metadata" as well as recommendations for "Metadata vocabulary for CSV data" and "Mapping mechanism to transforming CSV into various formats (e.g., RDF, JSON, or XML)". This document aims to satisfy the RDF variant of the mapping recommendation.
This document has been reviewed by W3C Members, by software developers, and by other W3C
groups and interested parties, and is endorsed by the Director as a W3C Recommendation.
It is a stable document and may be used as reference material or cited from another
document. W3C's role in making the Recommendation is to draw attention to the
specification and to promote its widespread deployment. This enhances the functionality
and interoperability of the Web.
This document describes the processing of tabular data to create an RDF subject-predicate-object triples [rdf11-concepts]. Since RDF is an abstract syntax, these triples MAY be serialized in a concrete RDF syntax such as N-Triples [n-triples], Turtle [turtle], RDFa [rdfa-primer], JSON-LD [json-ld], or TriG [trig]. The RDF serializations offered by a conversion application is implementation defined.
The conversion procedure described in this specification operates on the annotated tabular data model. This specification does not specify the processes needed to convert CSV-encoded data into tabular data form. Please refer to [tabular-data-model] for details of parsing tabular data.
Conversion applications MUST provide at least two modes of operation: standard and minimal.
Standard mode conversion frames the information gleaned from the cells of the tabular data with details of the rows, tables, and a group of tables within which that information is provided.
Minimal mode conversion includes only the information gleaned from the cells of the tabular data.
There is no requirement on conversion applications to check the semantic consistency of the data during the conversion, nor to validate the triples against RDF schema. Downstream applications SHOULD be aware of the potential for inconsistencies and take appropriate action.
2. Conformance
As well as sections marked as non-normative, all authoring guidelines, diagrams, examples,
and notes in this specification are non-normative. Everything else in this specification is
normative.
The key words MAY, MUST, and SHOULD are
to be interpreted as described in [RFC2119].
Tabular dataMUST conform to the description from [tabular-data-model]. In particular note that each rowMUST contain the same number of cells (although some of these cells may be empty).
Note
Not all CSV-encoded data can be parsed into a tabular data model. An algorithm for parsing CSV-based files is described in [tabular-data-model].
This specification makes use of the compact IRI Syntax; please refer to the Compact IRIs from [json-ld].
This specification makes use of the following namespaces:
csvw:
http://www.w3.org/ns/csvw#
rdf:
http://www.w3.org/1999/02/22-rdf-syntax-ns#
xsd:
http://www.w3.org/2001/XMLSchema#
3. Typographical conventions
The following typographic conventions are used in this specification:
markup
Markup (elements, attributes, properties), machine processable values (string, characters, media types), property name, or a file name is in red-orange monospace font.
variable
A variable in pseudo-code or in an algorithm description is in italics.
definition
A definition of a term, to be used elsewhere in this or other specifications, is in bold and italics.
A references to a definition in this document, when the reference itself is also a markup, is underlined, red-orange monospace font, and is also an active link to the definition itself.
A reference to a definition in another document, when the reference itself is also a markup, is underlined, in italics red-orange monospace font, and is also an active link to the definition itself.
A document reference (normative or informative) is enclosed in square brackets and links to the references section.
Note
Notes are in light green boxes with a green left border and with a "Note" header in green. Notes are normative or informative depending on the whether they are in a normative or informative section, respectively.
Example 1
Examples are in light khaki boxes, with khaki left border, and with a
numbered "Example" header in khaki. Examples are always informative.
The content of the example is in monospace font and may be syntax colored.
4. Converting Tabular Data to RDF
The procedures for converting tabular data into RDF are described below for both standard and minimal modes.
A literal node is defined in [rdf11-concepts] as a node within an RDF graph that provides values such as strings, numbers, and dates.
node
A node is defined in [rdf11-concepts] as a subject or an object of an RDF triple. When in subject position, it can be either a blank node or identified with a URL; when in object position, it can be a blank node, a literal, or identified with a URL.
A conformant RDF conversion application MUST emit triples conforming to those described in this algorithm according to the chosen mode of conversion: standard or minimal.
Unless specified otherwise, the steps in the algorithm defined herein apply to both standardandminimal modes.
The [tabular-data-model] specifies that string values within tabular data (such as column titles or cell string values) MUST contain only Unicode characters. No Unicode normalization (as specified in [UAX15]) is applied to these string values during the conversion to RDF.
Note
If a CSV file is originally encoded as UTF-8, it should not go through Unicode normalization during parsing, nor in conversion to RDF. This can result in RDF literals that are not in Normal Form C as they should be according to [rdf11-concepts].
Else, predicateP is constructed by appending the value of the name annotation for the column associated with the cell to the the tabular data file URL as a fragment identifier.
If the value URL for the current cell is not null, then value URL identifies a nodeVurl that is related the current subject using the predicateP; emit the following triple:
Else, if the cell value is a list and the cellordered annotation is true, then the cell value provides an ordered sequence of literal nodes for inclusion within the RDF output using an instance of rdf:ListVlist as defined in [rdf-schema]. This instance is related to the subject using the predicateP; emit the triples defining list Vlist plus the following triple:
Else, if the cell value is a list, then the cell value provides an unordered sequence of literal nodes for inclusion within the RDF output, each of which is related to the subject using the predicateP. For each value provided in the sequence, add a literal nodeVliteral; emit the following triple:
Else, if the cell value is not null, then the cell value provides a single literal nodeVliteral for inclusion within the RDF output that is related the current subject using the predicateP; emit the following triple:
In the case when a cell value does not have a datatype, the conversion SHOULD default to string.
Note
In the case where a sequence of values is provided, each value in the list has its own datatype; the datatype may be different for different items in the sequence.
Choice depends on whether or not the value has an associated language
normalizedString
xsd:normalizedString
token
xsd:token
language
xsd:language
Name
xsd:Name
NMTOKEN
xsd:NMTOKEN
xml
rdf:XMLLiteral
html
rdf:HTML
json
csvw:JSON
csvw:JSON is a sub-class of xsd:string
time
xsd:time
Note
A datatype's format annotation is irrelevant to the conversion procedure defined in this specification; the cell value has already been parsed from the contents of the cell according to the format annotation.
Cell errorsMUST be recorded by applications where the contents of a cell cannot be parsed or validated (see Parsing Cells and Validating Tables in [tabular-data-model] respectively). In cases where cell errors are recorded, applications may attempt to determine the appropriate RDF datatype IRI during the subsequent conversion process according to local rules.
According to [rdf11-concepts] language tags cannot be combined with other xsd datatypes. If a cell has any datatype other than string, the value of langMUST be ignored. Also, all literals have a datatype; however, specific serializations, like [turtle], MAY provide a special syntax for literals with datatype xsd:string or rdf:langString.
5. Inclusion of provenance information
This section is non-normative.
In addition to the namespaces defined above, the following namespace is used in this section:
prov:
http://www.w3.org/ns/prov#
Conversion applications MAY include provenance information in the RDF output describing how and when the output was created; e.g., using terms from the PROV Ontology [prov-o]. Information that may be of interest to downstream applications includes:
the source tabular data file;
the metadata description file(s) used;
when the conversion to RDF occurred; and
the conversion application used.
In order to facilitate providing such information, this specification introduces two instances of prov:Role:
csvw:csvEncodedTabularData
Defines the role of the source tabular data file.
csvw:tabularMetadata
Defines the role of the metadata description file.
An illustrative example of provenance information is provided below in [turtle] syntax, the conversion application used is identified as http://example.org/my-csv2rdf-application:
Conversion applications may have other means to create annotated tables, e.g. through some application specific APIs. In such cases the exact format for non-core annotations or notes may be different. Specifications for such annotation processes should specify how these annotations are converted into RDF.
Given a subject, property and value in normalized form:
If value is an array, generate RDF by running this algorithm using subject, property using each array member as value.
If value is an object containing @value, create an RDF Literal lit using the string value of @value and language from @language, or datatype from @type if present, expanding @type as necessary using the procedure outlined for property, and emit the following triple:
For every value of @type, either a term defined in the [csvw-context], a prefixed name, or an absolute URL; establish a new node Ti by expanding the value to an absolute URL by replacing a term with the URI from the term definition in [csvw-context] or a prefixed name with its expanded value. For each Ti, emit the following triple:
For every key and val from value that does not start with @ (U+0040) generate RDF by running this algorithm using S for subject, key for property and val for value.
Else, establish lit as an RDF Literal as follows:
If value is true or false, create an RDF Literal lit using the strings "true" or "false", accordingly with datatype xsd:boolean.
Else, if value is a JSON number with a non-zero fractional part, create an RDF Literal lit using the canonical representation for value with datatype xsd:double.
Else, if value is a JSON number with no non-zero fractional part, create an RDF Literal lit using the canonical representation for value with datatype xsd:integer.
Otherwise, create an RDF Literal lit using the canonical representation for value with datatype xsd:string.
In addition to the namespaces defined above, the examples provided here make use of the following namespaces:
dc:
http://purl.org/dc/terms/
foaf:
http://xmlns.com/foaf/0.1/
oa:
http://www.w3.org/ns/oa#
org:
http://www.w3.org/ns/org#
schema:
http://schema.org/
Furthermore, these examples also make use of the Turtle syntax @base declaration (as defined in [turtle]). Where a single tabular data file is used in the example, the @base declaration is set to the URL of that tabular data file.
Note
Each of the examples expresses more complex conversions - it is recommended that readers of this specification work through the examples in sequential order.
7.1 Simple example
This example comprises a single annotated table containing information attributes about countries; country code, position (latitude, longitude) and name. Whilst the input tabular data file, published at http://example.org/countries.csv, includes a header line, no further metadata annotations are given. The tabular data file is provided below:
Example 3: http://example.org/countries.csv
countryCode,latitude,longitude,name
AD,42.5,1.6,Andorra
AE,23.4,53.8,"United Arab Emirates"
AF,33.9,67.7,Afghanistan
The annotated table generated from parsing the tabular data file is shown below and provides the basis for the conversion to RDF.
Annotations for the resulting tableT, with 4 columns and 3 rows, are shown below:
id
core annotations
url
columns
rows
T
http://example.org/countries.csv
C1, C2, C3, C4
R1, R2, R3
Annotations for the columns, rows and cells in tableT are shown in the tables below:
Minimal mode output for this example is provided in [turtle] syntax below:
Example 4: http://example.org/countries-minimal.ttl
@base<http://example.org/countries.csv> .
_:8228a149-8efe-448d-b15f-8abf92e7bd17<#countryCode>"AD";<#latitude>"42.5";<#longitude>"1.6";<#name>"Andorra".
_:ec59dcfc-872a-4144-822b-9ad5e2c6149c<#countryCode>"AE";<#latitude>"23.4";<#longitude>"53.8";<#name>"United Arab Emirates".
_:e8f2e8e9-3d02-4bf5-b4f1-4794ba5b52c9<#countryCode>"AF";<#latitude>"33.9";<#longitude>"67.7";<#name>"Afghanistan".
Note
The about URL annotation has not been set for cells in table T ({ "url": "http://example.org/countries.csv"}); cells in a given row where about URL has not been specified are assumed to refer to the same subject. This unspecified subject is treated as a blank node.
Given that the property URL is null for cells in table T ({ "url": "http://example.org/countries.csv"}), the property URL defaults to the URI Template (see [RFC6570]) #{[column-name]}, where [column-name] is the value of the name annotation of the column associated with the cell. For example, the value of the property URL annotation for all cells in columnC1 ("name": "countryCode") is http://example.org/countries.csv#countryCode.
The type of both table and group of tables objects is explicitly stated; csvw:TableGroup and csvw:Table respectively.
The csvw:url property provides reference to the original tabular data file and to specific rows therein - noting the need to escape the Turtle-syntax reserved character = (U+003D) within the fragment identifier.
The row number is provided for each row using csvw:rownum property.
A subject and row are related using the csvw:describes property.
7.2 Example with single table and rich annotations
This example is based on Use Case #11 - City of Palo Alto Tree Data and comprises a single annotated table describing an inventory of tree maintenance operations. The input tabular data file, published at http://example.org/tree-ops-ext.csv, and the associated metadata description http://example.org/tree-ops-ext.csv-metadata.json are provided below:
Example 6: http://example.org/tree-ops-ext.csv
GID,On Street,Species,Trim Cycle,Diameter at Breast Ht,Inventory Date,Comments,Protected,KML
1,ADDISON AV,Celtis australis,Large Tree Routine Prune,11,10/18/2010,,,"<Point><coordinates>-122.156485,37.440963</coordinates></Point>"
2,EMERSON ST,Liquidambar styraciflua,Large Tree Routine Prune,11,6/2/2010,,,"<Point><coordinates>-122.156749,37.440958</coordinates></Point>"
6,ADDISON AV,Robinia pseudoacacia,Large Tree Routine Prune,29,6/1/2010,cavity or decay; trunk decay; codominant leaders; included bark; large leader or limb decay; previous failure root damage; root decay; beware of BEES,YES,"<Point><coordinates>-122.156299,37.441151</coordinates></Point>"
Example 7: http://example.org/tree-ops-ext.csv-metadata.json
{"@context":["http://www.w3.org/ns/csvw",{"@language":"en"}],"@id":"http://example.org/tree-ops-ext","url":"tree-ops-ext.csv","dc:title":"Tree Operations","dcat:keyword":["tree","street","maintenance"],"dc:publisher":[{"schema:name":"Example Municipality","schema:url":{"@id":"http://example.org"}}],"dc:license":{"@id":"http://opendefinition.org/licenses/cc-by/"},"dc:modified":{"@value":"2010-12-31","@type":"xsd:date"},"notes":[{"@type":"oa:Annotation","oa:hasTarget":{"@id":"http://example.org/tree-ops-ext"},"oa:hasBody":{"@type":"oa:EmbeddedContent","rdf:value":"This is a very interesting comment about the table; it's a table!","dc:format":{"@value":"text/plain"}}}],"dialect":{"trim":true},"tableSchema":{"columns":[{"name":"GID","titles":["GID","Generic Identifier"],"dc:description":"An identifier for the operation on a tree.","datatype":"string","required":true,"suppressOutput":true},{"name":"on_street","titles":"On Street","dc:description":"The street that the tree is on.","datatype":"string"},{"name":"species","titles":"Species","dc:description":"The species of the tree.","datatype":"string"},{"name":"trim_cycle","titles":"Trim Cycle","dc:description":"The operation performed on the tree.","datatype":"string","lang":"en"},{"name":"dbh","titles":"Diameter at Breast Ht","dc:description":"Diameter at Breast Height (DBH) of the tree (in feet), measured 4.5ft above ground.","datatype":"integer"},{"name":"inventory_date","titles":"Inventory Date","dc:description":"The date of the operation that was performed.","datatype":{"base":"date","format":"M/d/yyyy"}},{"name":"comments","titles":"Comments","dc:description":"Supplementary comments relating to the operation or tree.","datatype":"string","separator":";"},{"name":"protected","titles":"Protected","dc:description":"Indication (YES / NO) whether the tree is subject to a protection order.","datatype":{"base":"boolean","format":"YES|NO"},"default":"NO"},{"name":"kml","titles":"KML","dc:description":"KML-encoded description of tree location.","datatype":"xml"}],"primaryKey":"GID","aboutUrl":"http://example.org/tree-ops-ext#gid-{GID}"}}
The annotated table generated from parsing the tabular data file and associated metadata is shown below and provides the basis for the conversion to RDF.
Core annotations for the resulting tableT, with 9 columns and 3 rows, are shown below:
"cavity or decay; trunk decay; codominant leaders; included bark; large leader or limb decay; previous failure root damage; root decay; beware of BEES"
"cavity or decay", "trunk decay", "codominant leaders", "included bark", "large leader or limb decay", "previous failure root damage", "root decay", "beware of BEES"
The lists of values from cells in columnC7 ("name": "comments") are assumed to be unordered as the boolean ordered annotation, which defaults to false, has not be set within the metadata description.
Minimal mode output for this example is provided in [turtle] syntax below:
Example 8: http://example.org/tree-ops-ext-minimal.ttl
@base<http://example.org/tree-ops-ext.csv> .@prefix rdf:<http://www.w3.org/1999/02/22-rdf-syntax-ns#> .@prefix xsd:<http://www.w3.org/2001/XMLSchema#> .<http://example.org/tree-ops-ext#gid-1><#on_street>"ADDISON AV";<#species>"Celtis australis";<#trim_cycle>"Large Tree Routine Prune"@en;<#dbh>11;<#inventory_date>"2010-10-18"^^xsd:date ;<#protected>false;<#kml>"<Point><coordinates>-122.156485,37.440963</coordinates></Point>"^^rdf:XMLLiteral.<http://example.org/tree-ops-ext#gid-2><#on_street>"EMERSON ST";<#species>"Liquidambar styraciflua";<#trim_cycle>"Large Tree Routine Prune"@en;<#dbh>11;<#inventory_date>"2010-06-02"^^xsd:date ;<#protected>false;<#kml>"<Point><coordinates>-122.156749,37.440958</coordinates></Point>"^^rdf:XMLLiteral.<http://example.org/tree-ops-ext#gid-6><#on_street>"ADDISON AV";<#species>"Robinia pseudoacacia";<#trim_cycle>"Large Tree Routine Prune"@en;<#dbh>29;<#inventory_date>"2010-06-01"^^xsd:date ;<#comments>"cavity or decay","trunk decay","codominant leaders","included bark","large leader or limb decay","previous failure root damage","root decay","beware of BEES";<#protected>true;<#kml>"<Point><coordinates>-122.156299,37.441151</coordinates></Point>"^^rdf:XMLLiteral.
Note
The subject described by each row is explicitly defined using the about URL annotation; e.g. the subject of rowR1 is http://example.org/tree-ops-ext#gid-1.
The datatype annotation is set on columnsC5, C6, C8 and C9 ({ "name": "dbh"}, { "name": "inventory_date" }, { "name": "protected" } and { "name": "kml" }); as integer, date, boolean and xml respectively. The datatype property is inherited by all cells in each of those columns, therefore the RDF output for those cells includes the appropriate datatype IRI.
CellsC1.7 and C2.7 (rowsR1 and R2; column, { "name": "comments" }) have nullvalues - no output is included for these cells.
CellC3.7 (rowR3; column, { "name": "comments" }) contains an unordered sequence of values; the set of values are included as a simple set of triples as opposed to an instance of rdf:List as the ordered annotation has defaulted to false.
Example 9: http://example.org/tree-ops-ext-standard.ttl
@base<http://example.org/tree-ops-ext.csv> .@prefix csvw:<http://www.w3.org/ns/csvw#> .@prefix dc:<http://purl.org/dc/terms/> .@prefix dcat:<http://www.w3.org/ns/dcat#> .@prefix oa:<http://www.w3.org/ns/oa#> .@prefix rdf:<http://www.w3.org/1999/02/22-rdf-syntax-ns#> .@prefix schema:<http://schema.org/> .@prefix xsd:<http://www.w3.org/2001/XMLSchema#> .
_:68fc08e5-56a0-47e2-a784-3a644d8257c4 a csvw:TableGroup;
csvw:table <http://example.org/tree-ops-ext> .<http://example.org/tree-ops-ext> a csvw:Table ;
csvw:url <http://example.org/tree-ops-ext.csv> ;
dc:title "Tree Operations"@en;
dcat:keyword "tree"@en,"street"@en,"maintenance"@en;
dc:publisher [
schema:name "Example Municipality"@en;
schema:url <http://example.org>];
dc:license <http://opendefinition.org/licenses/cc-by/> ;
dc:modified "2010-12-31"^^xsd:date ;
csvw:note [
a oa:Annotation;
oa:hasTarget <http://example.org/tree-ops-ext> ;
oa:hasBody [
a oa:EmbeddedContent;
rdf:value "This is a very interesting comment about the table; it's a table!"@en;
dc:format "text/plain"]];
csvw:row [ a csvw:Row;
csvw:rownum 1;
csvw:url <#row=2>;
csvw:describes <http://example.org/tree-ops-ext#gid-1>],[ a csvw:Row;
csvw:rownum 2;
csvw:url <#row=3>;
csvw:describes <http://example.org/tree-ops-ext#gid-2>],[ a csvw:Row;
csvw:rownum 3;
csvw:url <#row=4>;
csvw:describes <http://example.org/tree-ops-ext#gid-6>].<http://example.org/tree-ops-ext#gid-1><#on_street>"ADDISON AV";<#species>"Celtis australis";<#trim_cycle>"Large Tree Routine Prune"@en;<#dbh>11;<#inventory_date>"2010-10-18"^^xsd:date ;<#protected>false;<#kml>"<Point><coordinates>-122.156485,37.440963</coordinates></Point>"^^rdf:XMLLiteral.<http://example.org/tree-ops-ext#gid-2><#on_street>"EMERSON ST";<#species>"Liquidambar styraciflua";<#trim_cycle>"Large Tree Routine Prune"@en;<#dbh>11;<#inventory_date>"2010-06-02"^^xsd:date ;<#protected>false;<#kml>"<Point><coordinates>-122.156749,37.440958</coordinates></Point>"^^rdf:XMLLiteral.<http://example.org/tree-ops-ext#gid-6><#on_street>"ADDISON AV";<#species>"Robinia pseudoacacia";<#trim_cycle>"Large Tree Routine Prune"@en;<#dbh>29;<#inventory_date>"2010-06-01"^^xsd:date ;<#comments>"cavity or decay","trunk decay","codominant leaders","included bark","large leader or limb decay","previous failure root damage","root decay","beware of BEES";<#protected>true;<#kml>"<Point><coordinates>-122.156299,37.441151</coordinates></Point>"^^rdf:XMLLiteral.
Note
TableT ({ "url": "http://example.org/tree-ops-ext.csv"}) has been explicitly identified: { "@id": "<http://exmple.org/tree-ops-ext>"}.
Non-core annotations and notes specified for tableT ({ "url": "http://example.org/tree-ops-ext.csv"}) are included in the output.
As the metadata description file http://example.org/tree-ops-ext.csv-metadata.json defines a default language within the context ("@context": ["http://www.w3.org/ns/csvw", {"@language": "en"}]), all non-core annotations of type string (e.g. dc:title, dcat:keyword, dc:publisher, dc:license and dc:modified) are expressed in the RDF output using the the appropriate language tag.
7.3 Example with single table and using virtual columns to produce multiple subjects per row
This example uses a single annotated table describing a listing of music events. Each row from the tabular data file corresponds to three resources; the music event itself, the location where that event occurs and the offer to sell tickets for that event. The goal is to convert the CSV content into schema.org markup that a search engine such as Google can use to index music events. Details of how Google expects this information to be structured can be found here.
The input tabular data file, published at http://example.org/events-listing.csv, and the associated metadata description http://example.org/events-listing.csv-metadata.json are provided below:
The CSV to RDF translation is limited to providing one statement, or triple, per column in the table. The target schema.org markup requires 10 statements to describe each event. As the base tabular data file contains 5 columns, an additional 5 virtual columns have been added in order to provide for the full complement of statements—including the relationships between the 3 resources (event, location, and offer) described by each row of the table. Note that the virtual annotation is set to true for these virtual columns.
Furthermore, note that no attempt is made to reconcile between locations or offers that may be associated with more than one event; every row in the table will create both a new location resource and offer resource in addition to the event resource. If considered necessary, applications such as OpenRefine may be used to identify and reconcile duplicate location resources once the RDF output has been generated.
The annotated table generated from parsing the tabular data file and associated metadata is shown below and provides the basis for the conversion to RDF.
Annotations for the resulting tableT, with 10 columns and 2 rows, are shown below:
id
core annotations
url
columns
rows
T
http://example.org/events-listing.csv
C1, C2, C3, C4, C5, C6, C7, C8, C9, C10
R1, R2
Annotations for the columns, rows and cells in tableT are shown in the tables below:
ColumnsC6, C7 and C8 ({ "name": "type_event"}, { "name": "type_place"} and { "name": "type_offer"}) define the semantic types of the resources described by each row: schema:MusicEvent, schema:Place and schema:Offer respectively.
ColumnC9 ({ "name": "location"}) uses the about URL, property URL and value URL to assert the relationship between the event and location resources.
Example 13: http://example.org/events-listing-standard.ttl
@base<http://example.org/events-listing.csv> .@prefix csvw:<http://www.w3.org/ns/csvw#> .@prefix schema:<http://schema.org/> .@prefix xsd:<http://www.w3.org/2001/XMLSchema#> .
_:95cc7970-ce99-44b0-900c-e2c2c028bbd3 a csvw:TableGroup;
csvw:table [ a csvw:Table;
csvw:url <http://example.org/events-listing.csv> ;
csvw:row [ a csvw:Row;
csvw:rownum 1;
csvw:url <#row=2>;
csvw:describes <#event-1>,<#place-1>,<#offer-1>],[ a csvw:Row;
csvw:rownum 2;
csvw:url <#row=3>;
csvw:describes <#event-2>,<#place-2>,<#offer-2>]].<#event-1> a schema:MusicEvent;
schema:name "B.B. King";
schema:startDate "2014-04-12T19:30:00"^^xsd:dateTime ;
schema:location <#place-1>;
schema:offers <#offer-1>.<#place-1> a schema:Place;
schema:name "Lupo’s Heartbreak Hotel";
schema:address "79 Washington St., Providence, RI".<#offer-1> a schema:Offer;
schema:url "https://www.etix.com/ticket/1771656"^^xsd:anyURI .<#event-2> a schema:MusicEvent;
schema:name "B.B. King";
schema:startDate "2014-04-13T20:00:00"^^xsd:dateTime ;
schema:location <#place-2>;
schema:offers <#offer-2>.<#place-2> a schema:Place;
schema:name "Lynn Auditorium";
schema:address "Lynn, MA, 01901".<#offer-2> a schema:Offer;
schema:url "http://frontgatetickets.com/venue.php?id=11766"^^xsd:anyURI .
Note
The resources described by each row are explicitly defined using the about URL annotation this case three resources per row (event, location, and offer). The relationship between the row and each subject resource is asserted using the csvw:describes property; e.g. for rowR1, a blank node, we state the following triples:
subject
R1
predicate
csvw:describes
object
t1:event-1
subject
R1
predicate
csvw:describes
object
t1:place-1
subject
R1
predicate
csvw:describes
object
t1:offer-1
7.4 Example with group of tables comprising four interrelated tables
This example is based on Use Case #4 - Publication of public sector roles and salaries and uses four annotated tables published as a group of tables. Information about senior roles and junior roles within a government department or organization are published in CSV format by each department. These are validated against a centrally published schema to ensure that all the data published by departments is consistent. Additionally, lists of organizations and professions are also published centrally, providing controlled vocabularies against which departmental submissions are validated.
Information published about junior and senior roles provides summary information for each post within the government department or organization. Whilst the junior role information is anonymous, providing only an indication of the number of full-time-equivalent (FTE) staff occupying a given post, the senior role information specifies the named individual occupying each post. As such, each row from the tabular data file describing senior roles corresponds to two resources; the post and the person occupying that post.
This example is concerned only with converting the information provided by each government department or organization rather than the centrally published information listing organizations and professions.
The input tabular data files and associated metadata descriptions are provided below:
Example 14: http://example.org/gov.uk/data/organizations.csv
Organization Unique Reference,Organization Name,Department Reference
hefce.ac.uk,Higher Education Funding Council for England,bis.gov.uk
bis.gov.uk,"Department for Business, Innovation and Skills",xx
Example 15: http://example.org/gov.uk/data/professions.csv
Profession
Finance
Information Technology
Operational Delivery
Policy
Example 16: http://example.org/senior-roles.csv
Post Unique Reference,Name,Grade,Job Title,Reports to Senior Post,Profession,Organization Reference
90115,Steve Egan,SCS1A,Deputy Chief Executive,90334,Finance,hefce.ac.uk
90334,Sir Alan Langlands,SCS4,Chief Executive,xx,Policy,hefce.ac.uk
Example 17: http://example.org/junior-roles.csv
Reporting Senior Post,Grade,Payscale Minimum (£),Payscale Maximum (£),Generic Job Title,Number of Posts (FTE),Profession,Organization Reference
90115,4,17426,20002,Administrator,8.67,Operational Delivery,hefce.ac.uk
90115,5,19546,22478,Administrator,0.5,Operational Delivery,hefce.ac.uk
Example 21: http://example.org/gov.uk/schema/senior-roles.json
{"@id":"http://example.org/gov.uk/schema/senior-roles.json","@context":"http://www.w3.org/ns/csvw","columns":[{"name":"ref","titles":"Post Unique Reference","datatype":"string","required":true,"propertyUrl":"dc:identifier"},{"name":"name","titles":"Name","datatype":"string","aboutUrl":"http://example.org/organization/{organizationRef}/person/{_row}","propertyUrl":"foaf:name"},{"name":"grade","titles":"Grade","datatype":"string","propertyUrl":"http://example.org/gov.uk/def/grade"},{"name":"job","titles":"Job Title","datatype":"string","propertyUrl":"http://example.org/gov.uk/def/job"},{"name":"reportsTo","titles":"Reports to Senior Post","datatype":"string","null":"xx","propertyUrl":"org:reportsTo","valueUrl":"http://example.org/organization/{organizationRef}/post/{reportsTo}"},{"name":"profession","titles":"Profession","datatype":"string","propertyUrl":"http://example.org/gov.uk/def/profession"},{"name":"organizationRef","titles":"Organization Reference","datatype":"string","propertyUrl":"org:postIn","valueUrl":"http://example.org/organization/{organizationRef}","required":true},{"name":"post_holder","virtual":true,"propertyUrl":"org:heldBy","valueUrl":"http://example.org/organization/{organizationRef}/person/{_row}"}],"primaryKey":"ref","aboutUrl":"http://example.org/organization/{organizationRef}/post/{ref}","foreignKeys":[{"columnReference":"reportsTo","reference":{"schemaReference":"http://example.org/gov.uk/schema/senior-roles.json","columnReference":"ref"}},{"columnReference":"profession","reference":{"schemaReference":"http://example.org/gov.uk/schema/professions.json","columnReference":"name"}},{"columnReference":"organizationRef","reference":{"schemaReference":"http://example.org/gov.uk/schema/organizations.json","columnReference":"ref"}}]}
Example 22: http://example.org/gov.uk/schema/junior-roles.json
{"@id":"http://example.org/gov.uk/schema/junior-roles.json","@context":"http://www.w3.org/ns/csvw","columns":[{"name":"reportsToSenior","titles":"Reporting Senior Post","datatype":"string","propertyUrl":"org:reportsTo","valueUrl":"http://example.org/organization/{organizationRef}/post/{reportsToSenior}","required":true},{"name":"grade","titles":"Grade","datatype":"string","propertyUrl":"http://example.org/gov.uk/def/grade"},{"name":"min_pay","titles":"Payscale Minimum (£)","datatype":"integer","propertyUrl":"http://example.org/gov.uk/def/min_pay"},{"name":"max_pay","titles":"Payscale Maximum (£)","datatype":"integer","propertyUrl":"http://example.org/gov.uk/def/max_pay"},{"name":"job","titles":"Generic Job Title","datatype":"string","propertyUrl":"http://example.org/gov.uk/def/job"},{"name":"number","titles":"Number of Posts (FTE)","datatype":"number","propertyUrl":"http://example.org/gov.uk/def/number_of_posts"},{"name":"profession","titles":"Profession","datatype":"string","propertyUrl":"http://example.org/gov.uk/def/profession"},{"name":"organizationRef","titles":"Organization Reference","datatype":"string","propertyUrl":"org:postIn","valueUrl":"http://example.org/organization/{organizationRef}","required":true}],"foreignKeys":[{"columnReference":"reportsToSenior","reference":{"schemaReference":"http://example.org/gov.uk/schema/senior-roles.json","columnReference":"ref"}},{"columnReference":"profession","reference":{"schemaReference":"http://example.org/gov.uk/schema/professions.json","columns":"name"}},{"columnReference":"organizationRef","reference":{"schemaReference":"http://example.org/gov.uk/schema/organizations.json","columns":"ref"}}]}
Note
This example makes extensive use of the example.org domain. As described in [RFC6761], this domain is used for illustrative examples within documentation. In reality, the resources described here with the URL path /gov.uk would be centrally published by the UK Government at, say, the domain data.gov.uk.
Given that these resources are centrally published with an aspiration for reuse, the schema descriptions have been factored out into separate resources. As such, the top-level metadata description resource metadata.json simply provides the list of tables and binds each of them to the appropriate schema that is defined elsewhere.
Finally, note that because the centrally published metadata descriptions are intended to be reused across many government departments and organizations, extra consideration has been given to defining URIs for the person and post resources defined in each row of the senior roles tabular data and subsequently referenced from the junior roles tabular data. To ensure that naming clashes are avoided, the unique reference for the organization to which the person or post belongs has been included in a path segment of the identifier. For example, the URI template propertyaboutUrl used to identify the senior post is specified as http://example.org/organization/{organizationRef}/post/{ref}, thus yielding the URI http://example.org/organization/hefce.ac.uk/post/90115 for the post described in the first row of the senior roles tabular data.
The group of tables generated from parsing the tabular data files and associated metadata is shown below and provides the basis for the conversion to RDF.
Annotations for the group of tablesG and the four tablesTa, Tb, Tc, and Td are shown below:
In this example, output for the centrally published lists of organizations and professions, tables Ta and Tb (http://example.org/gov.uk/data/organizations.csv and http://example.org/gov.uk/data/professions.csv respectively), are not required; only information from the departmental submissions is to be translated to RDF. Note the suppress output annotation on this table.
The following foreign keys are defined:
id
columns in table
columns in referenced table
Fa1
Ca3
Ca1
Fc1
Cc5
Cc1
Fc2
Cc6
Cb1
Fc3
Cc7
Ca1
Fd1
Cd1
Cc1
Fd2
Cd7
Cb1
Fd3
Cd8
Ca1
Annotations for the columns, rows and cells in tableT are shown in the tables below:
ColumnCc8, with the virtual annotation specified as true, is used to relate the person resource, whose name is provided in columnCc2, to the associated post resource within the current row of tableTc ({ "url": "http://example.org/senior-roles.csv" }).
Notice that value URL is not specified for cellsCa2.3 and Cc2.5 because in each case the cell value is null and the virtual annotation of columnCb5 is not defined.
Minimal mode output for this example is provided in [turtle] syntax below:
Example 23: http://example.org/public-sector-roles-and-salaries-minimal.ttl
Output for tablesTa and Tb ({ "url": "http://example.org/gov.uk/data/organizations.csv" } and { "url": "http://example.org/gov.uk/data/professions.csv" }) are not included as the suppress output annotation is true.
ColumnsCc5 and Cd1 ({ "name": "reportsTo" } and { "name": "reportsToSenior" }) use the about URL, property URL and value URL annotations to assert the relationship between the post described by a given row and the senior post to which it reports.
Similarly, columnsCc7 and Cd8 (both with { "name": "organizationRef" }) use the about URL, property URL and value URL annotations to assert the relationship between the post described by a given row and the organization to which it belongs.
Finally, note that two resources are created for each row within tableTc ({ "url": "http://example.org/senior-roles.csv" }): the person and the post they occupy. The relationship between these resources is specified via virtualcolumnCc8 ({ "name": "post_holder" }) using the about URL, property URL and value URL annotations.
Table groupG was explicitly defined, but has not been explicitly identified; the table group and table resources are treated as blank nodes.
The person and post resources described by each row of tableTc ({ "url": "http://example.org/senior-roles.csv"}) are explcitly defined using the aboutUrl property; therefore, say, for rowRc1 we state the following triples:
Conversely, the aboutUrl property has not been defined for resources described by each row of tableTd ({ "url": "http://example.org/junior-roles.csv"}); therefore blank nodes are used, e.g. for rowRd1 we state the following triple:
subject
Rd1
predicate
csvw:describes
object
_:d8b8e40c-8c74-458b-99f7-64d1cf5c65f2
A. Relationship to RDB Direct Mapping
This section is non-normative.
The "Direct Mapping of Relational Data to RDF" W3C Recommendation [rdb-direct-mapping] defines a simple transformation (referred to as direct mapping) from a relational representation of data to RDF. The direct mapping takes as input a relational database (data and schema), and generates an RDF graph called the direct graph. Tables in a relational databases bear a strong resemblance to tabular data as defined in [tabular-data-model]; this section highlights the similarities and differences between the direct mapping and the tabular data mapping defined by this document. The following statements summarize the relationships:
It is possible, for a single table in a relational database, to provide an annotation to the corresponding tabular data (i.e., the CSV export of the table from the database) such that the output of the direct mapping is semantically equivalent to the output of the (minimal) tabular data mapping. See the example with a single table.
If the database consists of several interrelated tables it is not always possible to provide an annotation to the corresponding tabular data to achieve the same level of semantic equivalence. Examples are provided indicating where tabular data mappingcan and cannot match direct mapping: the example with two tables and the example with three tables respectively.
The rest of this section will provide some examples. These are somewhat simplified forms of the examples as used in the [rdb-direct-mapping] document.
A.1 Single table case
Consider a simple table in a relational database, called People:
PK
ID
fname
addr
7
Bob
18
A relational table always has a schema that the direct mapping makes use of. In this example, the schema defines that:
The table has three columns named ID, fname, and addr.
Values in ID are the primary keys in the table, and the corresponding cells contain integers.
where http://foo.example.org/DB/ is the URL for the database containing the People table.
When exporting the table into CSV the column names would naturally be mapped on the titles of the respective columns. The (minimal) tabular data mapping yields:
where http://foo.example.org/CSV/People is the URL of the CSV file exported from the relational database.
Comparing the two conversion results:
The tabular data mapping does not have the information that the first column provides unique identifiers for the rows (i.e., that it is a primary key); consequently, a blank node must be used for the common row subject.
The tabular data mapping does not have information on the data types and, therefore, cannot presume that the first and the third columns contain integers.
The direct mapping adds a triple with predicate rdf:type to the direct graph. However, by default (e.g., without additional annotation), the tabular data mapping has insufficient information to make such an assertion.
It is, however, possible to add annotation to the tabular data so that the two graphs would be semantically equivalent. Indeed, consider the following metadata using the definitions of the [tabular-metadata] specification:
The metadata adds annotations on datatypes, adds the RDF typing triple explicitly (using a virtual column), and uses the aboutUrl URL template property to provide the common subject. Essentially, the metadata provides the information that the direct mapping retrieves from the table schema.
Note
The [tabular-metadata] specification includes an annotation for primary keys. However, that annotation is used for validation and does not influence the tabular data mapping, hence it is not included in the example metadata.
Note
A processor exporting the table into a CSV file may be able to automatically generate the metadata to complement the CSV file.
A.2 Simple, two table case
Consider a two interrelated simple tables in a relational database, called People and Addresses, respectively:
PK
→ Addresses(ID)
ID
fname
addr
7
Bob
18
PK
ID
city
state
18
Cambridge
MA
Beyond what is already specified for the People table the corresponding relational schema also specifies that:
The Addresses table has three columns named ID, city, and state.
Values of ID are primary keys in the Addresses table, and the corresponding cells contain integers.
Cells in both the city and state columns in the Addresses table contain strings.
Cells in the addr column of the People table are foreign keys that reference the ID field of the Addresses table.
Consider a case with three, interrelated simple tables in a relational database, called People, Addresses, and Departments, respectively:
PK
→ Addresses(ID)
→ Department(name,city)
ID
fname
addr
deptName
deptCity
7
Bob
18
accounting
Cambridge
PK
ID
city
state
18
Cambridge
MA
PK
Unique Key
ID
name
city
23
accounting
Cambridge
The corresponding relational schema specifies that:
For the People table
The table has five columns, named ID, name, addr, deptName, and deptCity.
Values in ID are the primary keys in the table, and the corresponding cells contain integers.
Cells in the name, deptName, and deptCity columns contain strings.
Cells in the addr columns contains integers.
For the Addresses table
The table has three columns, named ID, city, and state.
Values in ID are the primary keys in the table, and the corresponding cells contain integers.
Cells in both the city and state columns contain strings.
For the Departments table
The table has three columns, named ID, name, and city.
Values in ID are the primary keys in the table, and the corresponding cells contain integers.
Cells in both the name and city columns contain strings.
The combination of name and city are unique keys.
The cells in the addr column of the People table are foreign keys that reference the ID field of the Addresses table.
The cells in the deptName and deptCity columns in the People table are combined foreign keys referencing candidate keys for the name and city pairs of the Departments table.
To generate the correct object URI in the highlighted statement of the direct graph (above) the processor has to:
Extract the values of columns deptName and deptCity for the current row in the People table in order to determine the value of the compound unique key to the Departments table for that row.
Find the associated row in the Departments table that matches that compound unique key value and determine the subject for that row.
This can be done because the direct mapping processor has simultaneous access to several tables within the same relational database. It is therefore straightforward to access all the tables in parallel and establish the necessary relationships to generate the triples.
However, this combination cannot be handled by the tabular data mapping. The situation for tabular data is indeed different: tables are typically generated from single and, potentially, very large CSV files, meaning that a tabular data mapping processor cannot be expected to handle several tables in parallel. That is the reason why the [tabular-data-model] does not include features that would require such parallel access. As a consequence, the output of the direct mapping for such tables cannot be reproduced by the tabular data mapping.
Note
Note that the [tabular-data-model] includes an annotation for transformations; implementations may include scripts or templates to transform the output of the tabular data mapping and generate the required RDF graph.
B. Acknowledgements
At the time of publication, the following individuals had participated in the Working Group, in the order of their first name:
Adam Retter,
Alf Eaton,
Anastasia Dimou,
Andy Seaborne,
Axel Polleres,
Christopher Gutteridge,
Dan Brickley,
Davide Ceolin,
Eric Stephan,
Erik Mannens,
Gregg Kellogg,
Ivan Herman,
Jeni Tennison,
Jeremy Tandy,
Jürgen Umbrich,
Rufus Pollock,
Stasinos Konstantopoulos,
William Ingram, and
Yakov Shafranovich.
C. Changes since previous versions
C.1 Changes since candidate recommendation of 16 July 2015
C.2 Changes since the working draft of 16 April 2015
Added an appendix describing the relationships to the "Direct Mapping to RDF Specification" [rdb-direct-mapping].
The section on datatype has been changed to allow for the reference to externally defined datatypes (as datatype IRI-s in RDF Literals) using the new @id annotation in the model.
Removed the separate section on "Mapping Core Tabular Data" (to be in line with the latest version of the tabular model specification).
Reformulated the mappings in terms of table "annotations" (i.e., based on the tabular model) rather than table "properties" (i.e., not based on the specificities of the metadata vocabulary).
Provided a set of standard mappings from the general datatypes and the corresponding JSON types.