The D2RML Information Sources (D2RML‑IS) Specification

Document Conventions

Data Sources

A data source is an information source which represents one or more data blocks, typically a single data block in a particular a file, that are obtained every time they are requested from the data source. A specific data source is a subclass of dris:DataSource, which is the abstract class of all data sources.

An instance of a data source should contain the necessary information for interpreting the data blocks it provides. If the file format of the data block is important and cannot be determined correctly from a file extension, it may be specified using the dris:fileFormat property; appropriate values for this property include e.g. those defined in the file formats and FF vocabularies. The character encoding of a textual data block may be specified using a dris:characterEncoding property; appropriate values for this property include e.g. those defined in the CENC vocabulary. If access to the data block is secured by a password, the password may be specified by a dris:fileCredentials property.

The current version of D2RML-IS defines two data sources: file sources, which represent local files, and HTTP sources, which represent remote files or data obtained though an HTTP request.

<#CSVFile>
   a dris:FileSource ;
   dris:path "c:/data/part-1.csv" ;
   dris:path "c:/data/part-2.csv" ;
   dris:characterEncoding cenc:Windows-1253 .
					

<#HTTPSource>
   a dris:HTTPSource ;
   dris:uri "http://www.example.com/data/countries" .
					

<#HTTPSource>
   a dris:HTTPSource ;
   dris:httpRequest [
      http:absoluteURI "https://www.example.com/api/analyze?type=image&language=en" ;
      http:methodName "POST" ;
      http:headers ( 
        [ 
          http:fieldName "Content-Type" ;
          http:fieldValue "application/json" ; 
        ]
        [ 
          http:fieldName "APIKey" ;
          http:fieldValue "A$2@3KZa" ; 
        ] 
      ) ;
      http:body [ 
        a cnt:ContentAsText ;
        cnt:chars "{\"url\" : \"http://www.example.com/data/image.jpg\" }" ; 
      ] ;
   ] .				
					

<#HTTPSource>
   a dris:HTTPSource ;
   dris:uri "http://www.example.com/data/countries/{COUNTRY}" .
   drop:parameter [
      drop:name "COUNTRY" ;
   ] .
					

<#RemoteZipFile>  
   a dris:HTTPSource ;
   dris:uri "http://www.example.com/data/all.zip" ;
   dris:fileCredentials [
      dris:password "A$2@3KZa" 
   ] .

<#CompaniesSource>  
   a dris:FileSource ;
   dris:containerSource <#RemoteZipFile> ;
   dris:path "content/companies.csv" .
   
<#PersonsSource>  
   a dris:FileSource ;
   dris:containerSource <#RemoteZipFile> ;
   dris:path "content/persons.csv" .
					

<#CompaniesSource>  
   a dris:HTTPSource ;
   dris:uri "http://www.example.com/data/all.zip" ;
   dris:filePassword "A$2@3KZa" .
   dris:inContainerPath "content/companies.csv" .
					

Service Sources

A service source is an information source that represents a data repository, from which data blocks are eventually obtained by issuing a specific request to the service source, usually in the form of a query in a language that the service source understands. Usually, the data blocks obtained from a service source have a specific format that depends on the type of the source. A specific service source is a subclass of dris:ServiceSource which is the abstract class of all service sources. The descriptions of a service source contains the necessary information for establishing the connection to the service source, but not the queries by which the data blocks are obtained.

The current version of D2RML-IS defines two service sources: SPARQL endpoints, which represent SPARQL endpoints answering SPARQL queries, and RDBMSs, which represent relational database management systems answering SQL queries.

<#WikidataEndpoint>
   a dris:SPARQLEndpoint ;
   dris:uri "https://query.wikidata.org/bigdata/namespace/wdq/sparql" .
					

<#MySQLDatabase>
   a dris:RDBMSSource ;
   dris:rdbmsType dris:MySQL ;
   dris:host "http://database.org" ;
   dris:port 3006 ;
   dris:databaseName "companies" ;
   dris:credentials [
      a dris:StandardUserCredentials ;
      dris:username "root" ;
      dris:password "r@@Tx#" 
   ] .
					

<#MSAccessDatabase>
   a dris:RDBMSSource ;
   dris:rdbmsType dris:MicrosoftAccess ;
   dris:path "c:/data/database.mdb" ;
					

String Sources

An information source may also be a string source. A string source represents a single data block that is a directly provided user-defined string. It is an instance of dris:StringSource, and the string is provided through the dris:string property, which typically will make use of a parameter.

<#StringSource>
   a dris:StringSource ;
   dris:string "name;surname\nJohn;Smith" .
				

<#StringSource>
   a dris:StringSource ;
   dris:string "{@@VALUE@@}" ;
   drop:parameter [
      drop:name "VALUE" 
   ] .
				

Standard User Credentials

standard user credentials allow the identification of a user by means of a username and password specified by the dris:username and dris:password properties respectively.

Enumerate Request Iterators

An enumerate request iterator is a request iterator that takes successively all values from a predefined list of string values. The list of values is provided by the dris:values property.

<#Source>
a dris:HTTPSource ;
dris:httpRequest [ 
  http:absoluteURI "https://www.museum.org/items/{@@id@@}" ;
  http:methodName "GET" ;
] ;
drop:parameter [ 
  a dris:EnumerateRequestIterator;
  drop:name "id" ;
  dris:values ( "photo-49" "photo-34" "document-19" "document-92" ) ;
] .
				

Simple Count Request Iterators

An simple count request iterator is a request iterator that takes successively all values from an initial value up to a maximum value by the specified increment.

<#Source>
a dris:HTTPSource ;
dris:httpRequest [ 
  http:absoluteURI "https://www.museum.org/items?page={@@page@@}&size=20" ;
  http:methodName "GET" ;
] ;
drop:parameter [ 
  a dris:SimpleCountRequestIterator;
  drop:name "id" ;
  dris:initialCount 1 ;
  dris:maxCount 100 ;
  dris:increment 20 ;
] .
				

Simple Key Request Iterators

A simple key request iterator is a request iterator that takes successively some values. Each time the value is obtained from an element of the current data returned by the information source. In order for this to be possible, the data should be interpreted.

If an HTTP source returns paginated results e.g. in the form

{
"itemsCount": 20,
"totalResults": 567,
"nextCursor": "K2FSDA48DSA93GH6B97F5D767KD3B08HJ",
"items": [
  ...
]
}
				

where the value of nextCursor is a value that should be used in the URL to retrieve the next page, all pages can be obtained iteratively by using the following request iterator

<#Source>
a dris:HTTPSource ;
dris:httpRequest [
  http:absoluteURI "http://www.example.org/search?cursor={@@cursor@@}&rows=20" ;
  http:methodName "GET" ;
] ;
drop:parameter [ 
  a dris:SimpleKeyRequestIterator ;
  drop:name "cursor" ;
  dris:initialValue "*" ;
  dr:column "$.nextCursor" ;
  dr:columnFormulation dris:JSONPath ; 
] .