The Virtuoso Sponger is a middleware component of Virtuoso that generates RDF Linked Data from a variety of data sources. The sponger is transparently integrated into the Virtuoso SPARQL Query Processor, where it serves as part of the URI/IRI dereferencing functionality. It is also optionally used by the Virtuoso Content Crawler.
|
| Figure: 16.6.1.1.1. Virtuoso Sponger |
A majority of the worlds data naturally resides in non RDF form at the current time. The Sponger delivers middleware that accelerates the bootstrap of the Semantic Data Web by generating RDF from non RDF data sources, unobtrusively.
When an RDF aware client requests data from a network accessible resource via the Sponger the following events occur:
The imported data forms a local cache and its invalidation rules conform to those of traditional HTTP clients (Web Browsers). Thus, expiration time is determined based on subsequent data fetches of the same resource (note: the first data load will record the 'expires' header) with current time compared to expiration time stored in the local cache. If HTTP 'expires' header data isn't returned by the source data server, then the "Sponger" will derive it's own invalidation time frame by evaluating the 'date' header and 'last-modified' HTTP headers. Irrespective of path taken, local cache invalidation is driven by an assessment of current time relative to recorded expiration time.
Designed with a pluggable architecture, the Sponger's core functionality is provided by Cartridges. Each cartridge includes Data Extractors which extract data from one or more data sources, and Ontology Mappers which map the extracted data to one or more ontologies/schemas, and route to producing RDF Linked Data.
The Schema Mappers are typically XSLT (e.g. GRDDL and other OpenLink Mapping Schemes) or Virtuoso PL based. The Metadata Extractors may be developed in Virtuoso PL, C/C++, Java, or any other language that can be integrated into the Virtuoso via it's server extensions APIs.
The Sponger also includes a pluggable name resolution mechanism that enables the development of Custom Resolvers for naming schemes (e.g. URNs) associated with protocols beyond HTTP. Examples of custom resolvers include:
The Sponger is comprised of cartridges which are themselves comprised of an entity extractor and an ontology mapper. Entities extracted from non-RDF resources are used as the basis for generating structured data by mapping them to a suitable ontology. A cartridge is invoked through its cartridge hook, a Virtuoso/PL procedure entry point and binding to the cartridge's entity extractor and ontology mapper.
These Cartridges handle open formats: typically community-developed, openly-documented, and freely-licensed data structures.
These Cartridges handle closed formats: typically proprietary, sometimes undocumented, possibly licensed to no-one except the format originator. Sometimes data may not be parsed as desired or expected, as many of these Cartridges have required reverse-engineering of the data format in question.
These Cartridges submit the Resource identified by the URI to third-party Web Services for processing. Returned RDF supplements the RDF generated by other Cartridges, such as those in the preceding tables. Locally generated RDF may also be submitted to the third-party services, instead of or in addition to the original Resource itself.
| Cartridge | Sample URI | Resource Description | Linked Data Graph |
|---|---|---|---|
| BBC | example | HTML Representation | Data Explorer View |
| CNET | example | HTML Representation | Data Explorer View |
| DBpedia | example | HTML Representation | Data Explorer View |
| eBay | example | HTML Representation | Data Explorer View |
| FriendFeed | example | HTML Representation | Data Explorer View |
| Get Glue | example | HTML Representation | Data Explorer View |
| LOD | example | HTML Representation | Data Explorer View |
| New York Times | example | HTML Representation | Data Explorer View |
| OpenCalais | example | HTML Representation | Data Explorer View |
| Technorati | example | HTML Representation | Data Explorer View |
| Twine | example | HTML Representation | Data Explorer View |
| example | HTML Representation | Data Explorer View | |
| World Bank | example | HTML Representation | Data Explorer View |
| Zemanta | example | HTML Representation | Data Explorer View |
Basic
Sponger cartridges are invoked as follows:
When the SPARQL processor dereferences a URI, it plays the role of an HTTP user agent (client) that makes a content type specific request to an HTTP server via the HTTP request's Accept headers. The following then occurs:
Meta
Virtuoso also supports another cartridge type - a 'meta-cartridge'. Meta-cartridges act as post-processors in the cartridge pipeline, augmenting entity descriptions in an RDF graph with additional information gleaned from 'lookup' data sources and web services.
The "Sponger" is an example of a new class of tools for converting non-RDF data into RDF. Such tools are known as RDFizers. Introduced in Virtuoso Universal Server 5.0, the Sponger is packaged in an easily extensible framework, with tight integration to the Virtuoso RDF Quad Store.
The Sponger provides built-in RDF middleware for transforming non-RDF data into RDF "on the fly". Its goal is to use non-RDF Web data sources as input, e.g. (X)HTML Web Pages, (X)HTML Web pages hosting microformats, and even Web services such as those from Google, Del.icio.us, Flickr etc., and create RDF as output. The implication of this facility is that you can use non-RDF data sources as Semantic Web data sources.
How is it used?
The Sponger can be invoked via the following mechanisms:
File metadata extraction by ODS-Briefcase details you can find at the Virtuoso Sponger whitepaper.
SPARQL Query Processor IRI Dereferencing
The Sponger is transparently integrated into the Virtuoso SPARQL query processor, where it supports IRI dereferencing.
Given the distributed nature of RDF data, it is quite possible when executing a SPARQL query that some of the referenced data resides outside the local quad store. To cope with this scenario, the Virtuoso SPARQL query processor can be instructed to retrieve the external data and cache it in local quad storage. This feature is exposed through a set of Virtuoso SPARQL extensions known as "IRI dereferencing". Essentially these enable downloading and local storage of selected triples either from one or more named graphs or based on a proximity search from a starting URI for entities matching the select criteria and also related by the specified predicates, up to a given depth. Because the SPARQL processor understands only RDF data (serialized as RDF/XML, Turtle, N3), it utilizes the Sponger RDF mapper functionality when dereferencing web or file resources which don't naturally contain RDF data.
RDF Proxy Service
The Sponger's functionality is also exposed via an in-built REST style Web service. This web service takes a target URL and either returns the content "as is" or tries to transform (by sponging) to RDF. Thus, the proxy service can be used as a 'pipe' for RDF browsers to browse non-RDF sources.
When the rdf_mappers package is installed, Virtuoso reserves the path '/about/[id|data|rdf|html]/http/' for Sponger Proxy URI Service. For example, if a Virtuoso installation on host example.com listens for HTTP requests on port 8080 then client applications should use a 'service endpoint' string equal to 'http://example.com:8080/about/[id|data|rdf|html]/http/'. If the rdf_mappers package is not installed, then the service uses the path '/proxy/rdf/'.
Note: The old Sponger Proxy URI Service pattern '/proxy/' is now deprecated.
Example:
The following URLs return information about musician John Cale, gleaned from the MusicBrainz music metadatabase, rendered as RDF or HTML respectively. (The sponged data is available in the HTML rendering through the foaf:primaryTopic property.)
The Sponger forms part of the extensible RDF framework built into Virtuoso Universal Server. A key component of the Sponger's pluggable architecture is its support for Sponger Cartridges, which themselves are comprised of an Entity Extractor and an Ontology Mapper. Virtuoso bundles numerous pre-written cartridges for RDF data extraction from a wide range of data sources. However, developers are free to develop their own custom cartridges. This programmer's guide describes how.
The guide is a companion to the Virtuoso Sponger whitepaper. The latter describes the Sponger in depth, its architecture, configuration, use and integration with other Virtuoso facilities such as the Open Data Services (ODS) application framework. This guide focuses solely on custom cartridge development.
For configuring CURIEs used by the Sponger which is exposed via sponger clients such as "description.vsp" - the VSP based information resource description utility, you can use the xml_set_ns_decl function.
Here is sample example to add curie pattern:
-- Example link: http://linkeddata.uriburner.com/about/rdf/http://twitter.com/guykawasaki/status/1144945513#this
XML_SET_NS_DECL ('uriburner',
'http://linkeddata.uriburner.com/about/rdf/http://',
2);
The Sponger is comprised of cartridges which are themselves comprised of an entity extractor and an ontology mapper. Entities extracted from non-RDF resources are used as the basis for generating structured data by mapping them to a suitable ontology. A cartridge is invoked through its cartridge hook, a Virtuoso/PL procedure entry point and binding to the cartridge's entity extractor and ontology mapper.
Entity Extractor
When an RDF aware client requests data from a network accessible resource via the Sponger the following events occur:
Extraction Pipeline
Depending on the file or format type detected at ingest, the Sponger applies the appropriate entity extractor. Detection occurs at the time of content negotiation instigated by the retrieval user agent. The normal extraction pipeline processing is as follows:
Ontology Mapper
Sponger ontology mappers peform the the task of generating RDF instance data from extracted entities (non-RDF) using ontologies associated with a given data source type. They are typically XSLT (using GRDDL or an in-built Virtuoso mapping scheme) or Virtuoso/PL based. Virtuoso comes preconfigured with a large range of ontology mappers contained in one or more Sponger cartridges.
Cartridge Registry
To be recognized by the SPARQL engine, a Sponger cartridge must be registered in the Cartridge Registry by adding a record to the table DB.DBA.SYS_RDF_MAPPERS, either manually via DML, or more easily through Conductor, Virtuoso's browser-based administration console, which provides a UI for adding your own cartridges. (Sponger configuration using Conductor is described in detail later.) The SYS_RDF_MAPPERS table definition is as follows:
create table "DB"."DBA"."SYS_RDF_MAPPERS"
(
"RM_ID" INTEGER IDENTITY, -- cartridge ID. Determines the order of the cartridge's invocation in the Sponger processing chain
"RM_PATTERN" VARCHAR, -- a REGEX pattern to match the resource URL or MIME type
"RM_TYPE" VARCHAR, -- which property of the current resource to match: "MIME" or "URL"
"RM_HOOK" VARCHAR, -- fully qualified Virtuoso/PL function name
"RM_KEY" LONG VARCHAR, -- API specific key to use
"RM_DESCRIPTION" LONG VARCHAR, -- cartridge description (free text)
"RM_ENABLED" INTEGER, -- a 0 or 1 integer flag to exclude or include the cartridge from the Sponger processing chain
"RM_OPTIONS" ANY, -- cartridge specific options
"RM_PID" INTEGER IDENTITY,
PRIMARY KEY ("RM_PATTERN", "RM_TYPE")
);
The Virtuoso SPARQL processor supports IRI dereferencing via the Sponger. If a SPARQL query references non-default graph URIs, the Sponger goes out (via HTTP) to sponge the data source URIs and inserts the extracted RDF data into the local RDF quad store. The Sponger invokes the appropriate cartridge for the data source type to produce RDF instance data. If none of the registered cartridges are capable of handling the received content type, the Sponger will attempt to obtain RDF instance data via the in-built WebDAV metadata extractor.
Sponger cartridges are invoked as follows:
When the SPARQL processor dereferences a URI, it plays the role of an HTTP user agent (client) that makes a content type specific request to an HTTP server via the HTTP request's Accept headers. The following then occurs:
Meta-Cartridges
The above describes the RDF generation process for 'primary' Sponger cartridges. Virtuoso also supports another cartridge type - a 'meta-cartridge'. Meta-cartridges act as post-processors in the cartridge pipeline, augmenting entity descriptions in an RDF graph with additional information gleaned from 'lookup' data sources and web services. Meta-cartridges are described in more detail in a later section.
|
| Figure: 16.6.1.3.3.1. Meta-Cartridges |
Virtuoso supplies a number of prewritten cartridges for extracting RDF data from a variety of popular Web resources and file types. The cartridges are bundled as part of the rdf_mappers VAD (Virtuoso Application Distribution). Appendix B of the Virtuoso Sponger whitepaper briefly outlines the cartridges contained in the VAD.
To see which cartridges are available, look at the 'RDF Cartridges' screen in Conductor. This can be reached through the 'RDF' > 'RDF Cartridges' tabbed menu items.
|
| Figure: 16.6.1.3.4.1. RDF Cartridges |
To check which version of the rdf_mappers VAD is installed, or to upgrade it, refer to Conductor's 'VAD Packages' screen, reachable through the 'System Admin' > 'Packages' menu items.
The latest VADs for the closed source releases of Virtuoso can be downloaded from the downloads area on the OpenLink website. Select either the 'DBMS (WebDAV) Hosted' or 'File System Hosted' product format from the 'Distributed Collaborative Applications' section, depending on whether you want the Virtuoso application to use WebDAV or native filesystem storage. VADs for Virtuoso Open Source edition (VOS) are available for download from the VOS Wiki.
For developers wanting example cartridge code, the most authoritative reference is the rdf_mappers VAD source code itself. This is included as part of the VOS distribution. After downloading and unpacking the sources, the script used to create the cartridges, and the associated stylesheets can be found in:
Alternatively, you can look at the actual cartridge implementations installed in your Virtuoso instance by inspecting the cartridge hook function used by a particular cartridge. This is easily identified from the 'Cartridge name' field of Conductor's 'RDF Cartridges' screen, after selecting the cartridge of interest. The hook function code can be viewed from the 'Schema Objects' screen under the 'Database' menu, by locating the function in the 'DB' > 'Procedures' folder. Stylesheets used by the cartridges are installed in the WebDAV folder DAV/VAD/rdf_mappers/xslt. This can be explored using Conductor's WebDAV interface. The actual rdf_mappers.sql file installed with your system can also be found in the DAV/VAD/rdf_mappers folder.
Virtuoso comes well supplied with a variety of Sponger cartridges and GRDDL filters. When then is it necessary to write your own cartridge?
In the main, writing a new cartridge should only be necessary to generate RDF from a REST-style Web service not supported by an existing cartridge, or to customize the output from an existing cartridge to your own requirements. Apart from these circumstances, the existing Sponger infrastructure should meet most of your needs. This is particularly the case for document resources.
We use the term document resource to identify content which is not being returned from a Web service. Normally it can broadly be conceived as some form of document, be it a text based entity or some form of file, for instance an image file.
In these cases, the document either contains RDF, which can be extracted directly, or it holds metadata in a supported format which can be transformed to RDF using an existing filter.
The following cases should all be covered by the existing Sponger cartridges:
GRDDL (Gleaning Resource Descriptions from Dialects of Languages) is mechanism for deriving RDF data from XML documents and in particular XHTML pages. Document authors may associate transformation algorithms, typically expressed in XSLT, with their documents to transform embedded metadata into RDF.
The rdf_mappers VAD installs a number of GRDDL filters for transforming popular microformats (such as RDFa, eRDF or hCalendar) into RDF. The available filters can be viewed, or configured, in Conductor's 'GRDDL Filters for XHTML' screen. Navigate to the 'RDF Cartridges' screen using the 'RDF' > 'RDF Cartridges' menu items, then SELECT the 'GRDDL Mappings' tab to display the 'GRDDL Filters for XHTML' screen. GRDDL filters are held in the WebDAV folder /DAV/VAD/rdf_cartridges/xslt/ alongside other XSLT templates. The Conductor interface allows you to add new GRDDL filters should you so wish.
For an introduction to GRDDL, try the GRDDL Primer. To underline GRDDL's utility, the primer includes an example of transforming Excel spreadsheet data, saved as XML, into RDF.
A comprehensive list of stylesheets for transforming HTML and non-HTML XML dialects is maintained on the ESW Wiki. The list covers a range of microformats, syndication formats and feedlists.
To see which Web Services are already catered for, view the list of cartridges in Conductor's 'RDF Cartridges' screen.
The Sponger is fully extensible by virtue of its pluggable cartridge architecture. New data formats can be sponged by creating new cartridges. While OpenLink is active in adding cartridges for new data sources, you are free to develop your own custom cartridges. Entity extractors can be built using Virtuoso PL, C/C++, Java or any other external language supported by Virtuoso's Server Extension API. Of course, Virtuoso's own entity extractors are written in Virtuoso PL.
Cartridge Hook Function
Every Virtuoso PL hook function used to plug a custom Sponger cartridge into the Virtuoso SPARQL engine must have a parameter list with the following parameters (the names of the parameters are not important, but their order and presence are):
Return Value
If the hook procedure returns zero the next cartridge will be tried. If the result is negative the sponging process stops, instructing the SPARQL engine that nothing was retrieved. If the result is positive the process stops, this time instructing the SPARQL engine that RDF data was successfully retrieved.
If your cartridge should need to test whether other cartridges are configured to handle a particular data source, the following extract taken from the RDF_LOAD_CALAIS hook procedure illustrates how you might do this:
if (xd is not null)
{
-- Sponging successful. Load sponged data in Virtuoso quad store
DB.DBA.RM_RDF_LOAD_RDFXML (xd, new_origin_uri, coalesce (dest, graph_iri));
flag := 1;
}
declare ord any;
ord := (SELECT RM_ID FROM DB.DBA.SYS_RDF_MAPPERS WHERE
RM_HOOK = 'DB.DBA.RDF_LOAD_CALAIS');
for SELECT RM_PATTERN FROM DB.DBA.SYS_RDF_MAPPERS WHERE
RM_ID > ord and RM_TYPE = 'URL' and RM_ENABLED = 1 ORDER BY RM_ID do
{
if (regexp_match (RM_PATTERN, new_origin_uri) is not null)
-- try next candidate cartridge
flag := 0;
}
return flag;
Specifying the Target Graph
Two cartridge hook function parameters contain graph IRIs, graph_iri and dest. graph_iri identifies an input graph being crawled. dest holds the IRI specified in any input:grab-destination pragma defined to control the SPARQL processor's IRI dereferencing. The pragma overrides the default behaviour and forces all retrieved triples to be stored in a single graph, irrespective of their graph of origin.
So, under some circumstances depending on how the Sponger has been invoked and whether it is being used to crawl an existing RDF graph, or derive RDF data from a non-RDF data source, dest may be null.
Consequently, when loading sponged RDF data into the quad store, cartridges typically specify the graph to receive the data using the coalesce function which returns the first non-null parameter. e.g.
DB.DBA.RDF_LOAD_RDFXML (xd, new_origin_uri, coalesce (dest, graph_iri));
Here xd is an RDF/XML string holding the sponged RDF.
Specifying & Retrieving Cartridge Specific Options
The hook function prototype allows cartridge specific data to be passed to a cartridge through the RM_OPTIONS parameter, a Virtuoso/PL vector which acts as a heterogeneous array.
In the following example, two options are passed, 'add-html-meta' and 'get-feeds' with both values set to 'no'.
insert soft DB.DBA.SYS_RDF_MAPPERS (
RM_PATTERN, RM_TYPE, RM_HOOK, RM_KEY, RM_DESCRIPTION, RM_OPTIONS
)
values (
'(text/html)|(text/xml)|(application/xml)|(application/rdf.xml)',
'MIME', 'DB.DBA.RDF_LOAD_HTML_RESPONSE', null, 'xHTML',
vector ('add-html-meta', 'no', 'get-feeds', 'no')
);
The RM_OPTIONS vector can be handled as an array of key-value pairs using the get_keyword function. get_keyword performs a case sensitive search for the given keyword at every even index of the given array. It returns the element following the keyword, i.e. the keyword value.
Using get_keyword, any options passed to the cartridge can be retrieved using an approach similar to that below:
create procedure DB.DBA.RDF_LOAD_HTML_RESPONSE (
in graph_iri varchar, in new_origin_uri varchar, in dest varchar,
inout ret_body any, inout aq any, inout ps any, inout _key any,
inout opts any )
{
declare get_feeds, add_html_meta;
...
get_feeds := add_html_meta := 0;
if (isarray (opts) and 0 = mod (length(opts), 2))
{
if (get_keyword ('get-feeds', opts) = 'yes')
get_feeds := 1;
if (get_keyword ('add-html-meta', opts) = 'yes')
add_html_meta := 1;
}
...
API Keys
Certain web services require applications to provide an API key to use the service. Flickr is one such service. Developers must register to obtain a key. See for instance http://developer.yahoo.com/flickr/. In order to cater for services which require an application key, the Cartridge Registry SYS_RDF_MAPPERS table includes an RM_KEY column to store any key required for a particular service. This value is passed to the service's cartridge through the _key parameter of the cartridge hook function.
Alternatively a cartridge can store a key value in the virtuoso.ini configuration file and retrieve it in the hook function.
The next example shows an extract from the Flickr cartridge hook function DB.DBA.RDF_LOAD_FLICKR_IMG and the use of an API key. Also, commented out, is a call to cfg_item_value() which illustrates how the API key could instead be stored and retrieved from the SPARQL section of the virtuoso.ini file.
create procedure DB.DBA.RDF_LOAD_FLICKR_IMG (
in graph_iri varchar, in new_origin_uri varchar, in dest varchar,
inout _ret_body any, inout aq any, inout ps any, inout _key any,
inout opts any )
{
declare xd, xt, url, tmp, api_key, img_id, hdr, exif any;
declare exit handler for sqlstate '*'
{
return 0;
};
tmp := sprintf_inverse (new_origin_uri,
'http://farm%s.static.flickr.com/%s/%s_%s.%s', 0);
img_id := tmp[2];
api_key := _key;
--cfg_item_value (virtuoso_ini_path (), 'SPARQL', 'FlickrAPIkey');
if (tmp is null or length (tmp) <> 5 or not isstring (api_key))
return 0;
url := sprintf('http://api.flickr.com/services/rest/?method=flickr.photos.getInfo&photo_id=%s&api_key=%s',img_id, api_key);
tmp := http_get (url, hdr);
XSLT - The Fulchrum
XSLT is the fulchrum of all OpenLink supplied cartridges. It provides the most convenient means of converting structured data extracted from web content by a cartridge's Entity Extractor into RDF.
Virtuoso's XML Infrastructure & Tools
Virtuoso's XML support and XSLT support are covered in detail in the on-line documentation. Virtuoso includes a highly capable XML parser and supports XPath, XQuery, XSLT and XML Schema validation.
Virtuoso supports extraction of XML documents from SQL datasets. A SQL long varchar, long xml or xmltype column in a database table can contain XML data as text or in a binary serialized format. A string representing a well-formed XML entity can be converted into an entity object representing the root node.
While Sponger cartridges will not normally concern themselves with handling XML extracted from SQL data, the ability to convert a string into an in-memory XML document is used extensively. The function xtree_doc(string) converts a string into such a document and returns a reference to the document's root. This document together with an appropriate stylesheet forms the input for the transformation of the extracted entities to RDF using XSLT. The input string to xtree_doc generally contains structured content derived from a web service.
Virtuoso XSLT Support
Virtuoso implements XSLT 1.0 transformations as SQL callable functions. The xslt() Virtuoso/PL function applies a given stylesheet to a given source XML document and returns the transformed document. Virtuoso provides a way to extend the abilities of the XSLT processor by creating user defined XPath functions. The functions xpf_extension() and xpf_extension_remove() allow addition and removal of XPath extension functions.
General Cartridge Pipeline
The broad pipeline outlined here reflects the steps common to most cartridges:
The MusicBrainz cartridge typifies this approach. MusicBrainz is a community music metadatabase which captures information about artists, their recorded works, and the relationships between them. Artists always have a unique ID, so the URL http://musicbrainz.org/artist/4d5447d7-c61c-4120-ba1b-d7f471d385b9.html takes you directly to entries for John Lennon.
If you were to look at this page in your browser, you would see that the information about the artist contains no RDF data. However, the cartridge is configured to intercept requests to URLs of the form http://musicbrainz.org/([^/]*)/([^.]*) and redirect to the cartridge to sponge all the available information on the given artist, release, track or label.
The cartridge extracts entities by redirecting to the MusicBrainz XML Web Service using as the basis for the initial query the item ID, e.g. an artist or label ID, extracted from the original URL. Stripped to its essentials, the core of the cartridge is:
webservice_uri := sprintf ('http://musicbrainz.org/ws/1/%s/%s?type=xml&inc=%U',
kind, id, inc);
content := RDF_HTTP_URL_GET (webservice_uri, '', hdr, 'GET', 'Accept: */*');
xt := xtree_doc (content);
...
xd := DB.DBA.RDF_MAPPER_XSLT (registry_get ('_rdf_mappers_path_') || 'xslt/mbz2rdf.xsl', xt);
...
xd := serialize_to_UTF8_xml (xd);
DB.DBA.RM_RDF_LOAD_RDFXML (xd, new_origin_uri, coalesce (dest, graph_iri));
In the above outline, RDF_HTTP_URL_GET sends a query to the MusicBrainz web service, using query parameters appropriate for the original request, and retrieves the response using HTTP GET.
The returned XML is parsed into an in-memory parse tree by xtree_doc. Virtuoso/PL function RDF_MAPPER_XSLT is a simple wrapper around the function xslt which sets the current user to dba before returning an XML document transformed by an XSLT stylesheet, in this case mbz2rdf.xsl. Function serialize_to_UTF8_xml changes the character set of the in-memory XML document to UTF8. Finally, RM_RDF_LOAD_RDFXML is a wrapper around RDF_LOAD_RDFXML which parses the content of an RDF/XML string into a sequence of RDF triples and loads them into the quad store. XSLT stylesheets are usually held in the DAV/VAD/rdf_mappers/xslt folder of Virtuoso's WebDAV store. registry_get('rdf_mappers_path') returns the RDF Mappers VAD path, 'DAV/VAD/rdf_mappers', from the Virtuoso registry.
Error Handling with Exit Handlers
Virtuoso condition handlers determine the behaviour of a Virtuoso/PL procedure when a condition occurs. You can declare one or more condition handlers in a Virtuoso/PL procedure for general SQL conditions or specific SQLSTATE values. If a statement in your procedure raises an SQLEXCEPTION condition and you declared a handler for the specific SQLSTATE or SQLEXCEPTION condition the server passes control to that handler. If a statement in your Virtuoso/PL procedure raises an SQLEXCEPTION condition, and you have not declared a handler for the specific SQLSTATE or the SQLEXCEPTION condition, the server passes the exception to the calling procedure (if any). If the procedure call is at the top-level, then the exception is signaled to the calling client.
A number of different condition handler types can be declared (see the Virtuoso reference documentation for more details.) Of these, exit handlers are probably all you will need. An example is shown below which handles any SQLSTATE. Commented out is a debug statement which outputs the message describing the SQLSTATE.
create procedure DB.DBA.RDF_LOAD_SOCIALGRAPH (in graph_iri varchar, ...)
{
declare qr, path, hdr any;
...
declare exit handler for sqlstate '*'
{
-- dbg_printf ('%s', __SQL_MESSAGE);
return 0;
};
...
-- data extraction and mapping successful
return 1;
}
Exit handlers are used extensively in the Virtuoso supplied cartridges. They are useful for ensuring graceful failure when trying to convert content which may not conform to your expectations. The RDF_LOAD_FEED_SIOC procedure (which is used internally by several cartridges) shown below uses this approach:
-- /* convert the feed in rss 1.0 format to sioc */
create procedure DB.DBA.RDF_LOAD_FEED_SIOC (in content any, in iri varchar, in graph_iri varchar, in is_disc int := '')
{
declare xt, xd any;
declare exit handler for sqlstate '*'
{
goto no_sioc;
};
xt := xtree_doc (content);
xd := DB.DBA.RDF_MAPPER_XSLT (
registry_get ('_rdf_mappers_path_') || 'xslt/feed2sioc.xsl', xt,
vector ('base', graph_iri, 'isDiscussion', is_disc));
xd := serialize_to_UTF8_xml (xd);
DB.DBA.RM_RDF_LOAD_RDFXML (xd, iri, graph_iri);
return 1;
no_sioc:
return 0;
}
Loading RDF into the Quad Store
RDF_LOAD_RDFXML & TTLP
The two main Virtuoso/PL functions used by the cartridges for loading RDF data into the Virtuoso quad store are DB.DBA.TTLP and DB.DBA.RDF_LOAD_RDFXML. Multithreaded versions of these functions, DB.DBA.TTLP_MT and DB.DBA.RDF_LOAD_RDFXML_MT, are also available.
RDF_LOAD_RDFXML parses the content of an RDF/XML string as a sequence of RDF triples and loads then into the quad store. TTLP parses TTL (Turtle or N3) and places its triples into quad storage. Ordinarily, cartridges use RDF_LOAD_RDFXML. However there may be occasions where you want to insert statements written as TTL, rather than RDF/XML, in which case you should use TTLP.
Attribution
Many of the OpenLink supplied cartridges actually use RM_RDF_LOAD_RDFXML to load data into the quad store. This is a thin wrapper around RDF_LOAD_RDFXML which includes in the generated graph an indication of the external ontologies being used. The attribution takes the form:
<ontologyURI> a opl:DataSource . <spongedResourceURI> rdfs:isDefinedBy <ontologyURI> . <ontologyURI> opl:hasNamespacePrefix "<ontologyPrefix>" .
where prefix opl: denotes the ontology http://www.openlinksw.com/schema/attribution#.
Deleting Existing Graphs
Before loading sponged RDF data into a graph, you may want to delete any existing graph with the same URI. To do so, select the 'RDF' > 'List of Graphs' menu commands in Conductor, then use the 'Delete' command for the appropriate graph. Alternatively, you can use one of the following SQL commands:
SPARQL CLEAR GRAPH -- or DELETE FROM DB.DBA.RDF_QUAD WHERE G = DB.DBA.RDF_MAKE_IID_OF_QNAME (graph_iri)
Proxy Service Data Expiration
When the Proxy Service is invoked by a user agent, the Sponger records the expiry date of the imported data in the table DB.DBA.SYS_HTTP_SPONGE. The data invalidation rules conform to those of traditional HTTP clients (Web browsers). The data expiration time is determined based on subsequent data fetches of the same resource. The first data retrieval records the 'expires' header. On subsequent fetches, the current time is compared to the expiration time stored in the local cache. If HTTP 'expires' header data isn't returned by the source data server, the Sponger will derive its own expiration time by evaluating the 'date' header and 'last-modified' HTTP headers.
After extracting entities from a web resource and converting them to an in-memory XML document, the entities must be transformed to the target ontology using XSLT and an appropriate stylesheet. A typical call sequence would be:
xt := xtree_doc (content);
...
xd := DB.DBA.RDF_MAPPER_XSLT (registry_get ('_rdf_mappers_path_') || 'xslt/mbz2rdf.xsl', xt);
Because of the wide variation in the data mapped by cartridges, it is not possible to present a typical XSL stylesheet outline. The Examples section presented later includes detailed extracts from the MusicBrainz? cartridge's stylesheet which provide a good example of how to map to an ontology. Rather than attempting to be an XSLT tutorial, the material which follows offers some general guidelines.
Passing Parameters to the XSLT Processor
Virtuoso's XSLT processor will accept default values for global parameters from the optional third argument of the xslt() function. This argument, if specified, must be a vector of parameter names and values of the form vector(name1, value1,... nameN, valueN), where name1 ... nameN must be of type varchar, and value1 ... valueN may be of any Virtuoso datatype, but may not be null.
This extract from the Crunchbase cartridge shows how parameters may be passed to the XSLT processor. The function RDF_MAPPER_XSLT (in xslt varchar, inout xt any, in params any := null) passes the parameters vector directly to xslt().
xt := DB.DBA.RDF_MAPPER_XSLT (
registry_get ('_rdf_mappers_path_') || 'xslt/crunchbase2rdf.xsl', xt,
vector ('baseUri', coalesce (dest, graph_iri), 'base', base, 'suffix', suffix)
);
The corresponding stylesheet crunchbase2rdf.xsl retrieves the parameters baseUri, base and suffix as follows:
... <xsl:output method="xml" indent="yes" /> <xsl:variable name="ns">http://www.crunchbase.com/</xsl:variable> <xsl:param name="baseUri" /> <xsl:param name="base"/> <xsl:param name="suffix"/> <xsl:template name="space-name"> ...
An RDF Description Template
Defining A Generic Resource Description Wrapper
Many of the OpenLink cartridges create a resource description formed to a common "wrapper" template which describes the relationship between the (usually) non-RDF source resource being sponged and the RDF description generated by the Sponger. The wrapper is appropriate for resources which can broadly be conceived as documents. It provides a generic minimal description of the source document, but also links to the much more detailed description provided by the Sponger. So, instead of just emitting a resource description, the Sponger factors the container into the generated graph constituting the RDF description.
The template is depicted below:
|
| Figure: 16.6.1.3.6.1. Template |
To generate an RDF description corresponding to the wrapper template, a stylesheet containing the following block of instructions is used. This extract is taken from the eBay cartridge's stylesheet, ebay2rdf.xsl. Many of the OpenLink cartridges follow a similar pattern.
<xsl:param name="baseUri"/>
...
<xsl:variable name="resourceURL">
<xsl:value-of select="$baseUri"/>
</xsl:variable>
...
<xsl:template match="/">
<rdf:RDF>
<rdf:Description rdf:about="{$resourceURL}">
<rdf:type rdf:resource="Document"/>
<rdf:type rdf:resource="Document"/>
<rdf:type rdf:resource="Container"/>
<sioc:container_of rdf:resource="{vi:proxyIRI ($resourceURL)}"/>
<foaf:primaryTopic rdf:resource="{vi:proxyIRI ($resourceURL)}"/>
<dcterms:subject rdf:resource="{vi:proxyIRI ($resourceURL)}"/>
</rdf:Description>
<rdf:Description rdf:about="{vi:proxyIRI ($resourceURL)}">
<rdf:type rdf:resource="Item"/>
<sioc:has_container rdf:resource="{$resourceURL}"/>
<xsl:apply-templates/>
</rdf:Description>
</rdf:RDF>
</xsl:template>
...
Using SIOC as a Generic Container Model
The generic resource description wrapper just described uses SIOC to establish the container/contained relationship between the source resource and the generated graph. Although the most important classes for the generic wrapper are obviously Container and Item, SIOC provides a generic data model of containers, items, item types, and associations between items which can be combined with other vocabularies such as FOAF and Dublin Core.
SIOC defines a number of other classes, such as User, UserGroup, Role, Site, Forum and Post. A separate SIOC types module (T-SIOC) extends the SIOC Core ontology by defining subclasses and subproperties of SIOC terms. Subclasses include: AddressBook, BookmarkFolder, Briefcase, EventCalendar, ImageGallery, Wiki, Weblog, BlogPost, Wiki plus many others.
OpenLink Data Spaces (ODS) uses SIOC extensively as a data space "glue" ontology to describe the base data and containment hierarchy of all the items managed by ODS applications (Data Spaces). For example, ODS-Weblog is an application of type sioc:Forum. Each ODS-Weblog application instance contains blogs of type sioct:Weblog. Each blog is a sioc:container_of posts of type sioc:Post.
Generally, when deciding how to describe resources handled by your own custom cartridge, SIOC provides a useful framework for the description which complements the SIOC-based container model adopted throughout the ODS framework.
Naming Conventions for Sponger Generated Descriptions
As can be seen from the stylesheet extract just shown, the URI of the resource description generated by the Sponger to describe the sponged resource is given by the function {vi:proxyIRI ($resourceURL)} where resourceURL is the URL of the original resource being sponged. proxyIRI is an XPath extension function defined in rdf_mappers.sql as
xpf_extension ('http://www.openlinksw.com/virtuoso/xslt/:proxyIRI', 'DB.DBA.RDF_SPONGE_PROXY_IRI');
which maps to the Virtuoso/PL procedure DB.DBA.RDF_SPONGE_PROXY_IRI. This procedure in turn generates a resource description URI which typically takes the form: http://<hostName:port>/about/html/http/<resourceURL>#this
Once you have developed a cartridge, you must register it in the Cartridge Registry to have the SPARQL processor recognize and use it. You should have compiled your cartridge hook function first by issuing a "create procedure DB.DBA.RDF_LOAD_xxx ..." command through one of Virtuoso's SQL interfaces. You can create the required Cartridge Registry entry either by adding a row to the SYS_REF_MAPPERS table directly using SQL, or by using the Conductor UI.
Using SQL
If you choose register your cartridge using SQL, possibly as part of a Virtuoso/PL script, the required SQL will typically mirror one of the following INSERT commands.
Below, a cartridge for OpenCalais is being installed which will be tried when the MIME type of the data being sponged is one of text/plain, text/xml or text/html. (The definition of the SYS_RDF_MAPPERS table was introduced earlier in section 'Cartridge Registry'.)
insert soft DB.DBA.SYS_RDF_MAPPERS ( RM_PATTERN, RM_TYPE, RM_HOOK, RM_KEY, RM_DESCRIPTION, RM_ENABLED) values ( '(text/plain)|(text/xml)|(text/html)', 'MIME', 'DB.DBA.RDF_LOAD_CALAIS', null, 'Opencalais', 1);
As an alternative to matching on the content's MIME type, candidate cartridges to be tried in the conversion pipeline can be identified by matching the data source URL against a URL pattern stored in the cartridge's entry in the Cartridge Registry.
insert soft DB.DBA.SYS_RDF_MAPPERS ( RM_PATTERN, RM_TYPE, RM_HOOK, RM_KEY, RM_DESCRIPTION, RM_OPTIONS) values ( '(http://api.crunchbase.com/v/1/.*)|(http://www.crunchbase.com/.*)', 'URL', 'DB.DBA.RDF_LOAD_CRUNCHBASE', null, 'CrunchBase', null);
The value of RM_ID to set depends on where in the cartridge invocation order you want to position a particular cartridge. RM_ID should be set lower than 10028 to ensure the cartridge is tried before the ODS-Briefcase (WebDAV) metadata extractor, which is always the last mapper to be tried if no preceding cartridge has been successful.
UPDATE DB.DBA.SYS_RDF_MAPPERS SET RM_ID = 1000 WHERE RM_HOOK = 'DB.DBA.RDF_LOAD_BIN_DOCUMENT';
Using Conductor
Cartridges can be added manually using the 'Add' panel of the 'RDF Cartridges' screen.
|
| Figure: 16.6.1.3.6.1. RDF Cartridges |
|
| Figure: 16.6.1.3.6.2. RDF Cartridges |
Installing Stylesheets
Although you could place your cartridge stylesheet in any folder configured to be accessible by Virtuoso, the simplest option is to upload them to the DAV/VAD/rdf_mappers/xslt folder using the WebDAV browser accessible from the Conductor UI.
|
| Figure: 16.6.1.3.6.3. WebDAV browser |
Should you wish to locate your stylesheets elsewhere, ensure that the DirsAllowed setting in the virtuoso.ini file is configured appropriately.
To illustrate some of the material presented so far, we'll delve deeper into the MusicBrainz cartridge mentioned earlier.
MusicBrainz XML Web Service
The cartridge extracts data through the MusicBrainz XML Web Service using, as the basis for the initial query, an item type and MBID (MusicBrainz ID) extracted from the original URI submitted to the RDF proxy. A range of item types are supported including artist, release and track.
Using the album "Imagine" by John Lennon as an example, a standard HTML description of the album (which has an MBID of f237e6a0-4b0e-4722-8172-66f4930198bc) can be retrieved direct from MusicBrainz using the URL:
http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html
Alternatively, information can be extracted in XML form through the web service. A description of the tracks on the album can be obtained with the query:
http://musicbrainz.org/ws/1/release/f237e6a0-4b0e-4722-8172-66f4930198bc?type=xml&inc=tracks
The XML returned by the web service is shown below (only the first two tracks are shown for brevity):
<?xml version="1.0" encoding="UTF-8"?>
<metadata xmlns="http://musicbrainz.org/ns/mmd-1.0#"
xmlns:ext="http://musicbrainz.org/ns/ext-1.0#">
<release id="f237e6a0-4b0e-4722-8172-66f4930198bc" type="Album Official" >
<title>Imagine</title>
<text-representation language="ENG" script="Latn"/>
<asin>B0000457L2</asin>
<track-list>
<track id="b88bdafd-e675-4c6a-9681-5ea85ab99446">
<title>Imagine</title>
<duration>182933</duration>
</track>
<track id="b38ce90d-3c47-4ccd-bea2-4718c4d34b0d">
<title>Crippled Inside</title>
<duration>227906</duration>
</track>
. . .
</track-list>
</release>
</metadata>
Although, as shown above, MusicBrainz defines its own XML Metadata Format to represent music metadata, the MusicBrainz sponger converts the raw data to a subset of the Music Ontology, an RDF vocabulary which aims to provide a set of core classes and properties for describing music on the Semantic Web. Part of the subset used is depicted in the following RDF graph (representing in this case a John Cale album).
|
| Figure: 16.6.1.3.6.1. RDF graph |
With the prefix mo: denoting the Music Ontology at http://purl.org/ontology/mo/, it can be seen that artists are represented by instances of class mo:Artist, their albums, records etc. by instances of class mo:Release and tracks on these releases by class mo:Track. The property foaf:made links an artist and his/her releases. Property mo:track links a release with the tracks it contains
RDF Output
An RDF description of the album can be obtained by sponging the same URL, i.e. by submitting it to the Sponger's proxy interface using the URL:
http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html
The extract below shows part of the (reorganized) RDF output returned by the Sponger for "Imagine". Only the album's title track is included.
<?xml version="1.0" encoding="utf-8" ?> <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"> <rdf:Description rdf:about="http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html"> <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Document"/> </rdf:Description> <rdf:Description rdf:about="http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html"> <foaf:primaryTopic xmlns:foaf="http://xmlns.com/foaf/0.1/" rdf:resource="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html#this"/> </rdf:Description> <rdf:Description rdf:about="http://purl.org/ontology/mo/"> <rdf:type rdf:resource="http://www.openlinksw.com/schema/attribution#DataSource"/> </rdf:Description> ... <rdf:Description rdf:about="http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html"> <rdfs:isDefinedBy rdf:resource="http://purl.org/ontology/mo/"/> </rdf:Description> ... <!-- Record description --> <rdf:Description rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html#this"> <rdf:type rdf:resource="http://purl.org/ontology/mo/Record"/> </rdf:Description> <rdf:Description rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html#this"> <dc:title xmlns:dc="http://purl.org/dc/elements/1.1/">Imagine</dc:title> </rdf:Description> <rdf:Description rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html#this"> <mo:release_status xmlns:mo="http://purl.org/ontology/mo/" rdf:resource="http://purl.org/ontology/mo/official"/> </rdf:Description> <rdf:Description rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html#this"> <mo:release_type xmlns:mo="http://purl.org/ontology/mo/" rdf:resource="http://purl.org/ontology/mo/album"/> </rdf:Description> <!-- Title track description --> <rdf:Description rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html#this"> <mo:track xmlns:mo="http://purl.org/ontology/mo/" rdf:resource="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/track/b88bdafd-e675-4c6a-9681-5ea85ab99446.html#this"/> </rdf:Description> <rdf:Description rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/track/b88bdafd-e675-4c6a-9681-5ea85ab99446.html#this"> <rdf:type rdf:resource="http://purl.org/ontology/mo/Track"/> </rdf:Description> <rdf:Description rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/track/b88bdafd-e675-4c6a-9681-5ea85ab99446.html#this"> <dc:title xmlns:dc="http://purl.org/dc/elements/1.1/">Imagine</dc:title> </rdf:Description> <rdf:Description rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/track/b88bdafd-e675-4c6a-9681-5ea85ab99446.html#this"> <mo:track_number xmlns:mo="http://purl.org/ontology/mo/">1</mo:track_number> </rdf:Description> <rdf:Description rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/track/b88bdafd-e675-4c6a-9681-5ea85ab99446.html#this"> <mo:duration xmlns:mo="http://purl.org/ontology/mo/" rdf:datatype="http://www.w3.org/2001/XMLSchema#integer">182933</mo:duration> </rdf:Description> </rdf:RDF>
Cartridge Hook Function
The cartridge's hook function is listed below. It is important to note that MusicBrainz supports a variety of query types, each of which returns a different set of information, depending on the item type being queried. Full details can be found on the MusicBrainz? site. The sponger cartridge is capable of handling all the query types supported by MusicBrainz? and is intended to be used in a drill-down scenario, as would be the case when using an RDF browser such as the OpenLink Data Explorer (ODE). This example focuses primarily on the types release and track.
create procedure DB.DBA.RDF_LOAD_MBZ (
in graph_iri varchar, in new_origin_uri varchar, in dest varchar,
inout _ret_body any, inout aq any, inout ps any, inout _key any,
inout opts any)
{
declare kind, id varchar;
declare tmp, incs any;
declare uri, cnt, hdr, inc, xd, xt varchar;
tmp := regexp_parse ('http://musicbrainz.org/([^/]*)/([^\.]+)', new_origin_uri, 0);
declare exit handler for sqlstate '*'
{
-- dbg_printf ('%s', __SQL_MESSAGE);
return 0;
};
if (length (tmp) < 6)
return 0;
kind := subseq (new_origin_uri, tmp[2], tmp[3]);
id := subseq (new_origin_uri, tmp[4], tmp[5]);
incs := vector ();
if (kind = 'artist')
{
inc := 'aliases artist-rels label-rels release-rels track-rels url-rels';
incs :=
vector (
'sa-Album', 'sa-Single', 'sa-EP', 'sa-Compilation', 'sa-Soundtrack',
'sa-Spokenword', 'sa-Interview', 'sa-Audiobook', 'sa-Live', 'sa-Remix', 'sa-Other'
, 'va-Album', 'va-Single', 'va-EP', 'va-Compilation', 'va-Soundtrack',
'va-Spokenword', 'va-Interview', 'va-Audiobook', 'va-Live', 'va-Remix', 'va-Other'
);
}
else if (kind = 'release')
inc := 'artist counts release-events discs tracks artist-rels label-rels release-rels track-rels url-rels track-level-rels labels';
else if (kind = 'track')
inc := 'artist releases puids artist-rels label-rels release-rels track-rels url-rels';
else if (kind = 'label')
inc := 'aliases artist-rels label-rels release-rels track-rels url-rels';
else
return 0;
if (dest is null)
DELETE FROM DB.DBA.RDF_QUAD WHERE G = DB.DBA.RDF_MAKE_IID_OF_QNAME (graph_iri);
DB.DBA.RDF_LOAD_MBZ_1 (graph_iri, new_origin_uri, dest, kind, id, inc);
DB.DBA.TTLP (sprintf ('<%S> <http://xmlns.com/foaf/0.1/primaryTopic> <%S> .\n<%S> a <http://xmlns.com/foaf/0.1/Document> .',
new_origin_uri, DB.DBA.RDF_SPONGE_PROXY_IRI (new_origin_uri), new_origin_uri),
'', graph_iri);
foreach (any inc1 in incs) do
{
DB.DBA.RDF_LOAD_MBZ_1 (graph_iri, new_origin_uri, dest, kind, id, inc1);
}
return 1;
};
The hook function uses a subordinate procedure RDF_LOAD_MBZ_1:
create procedure DB.DBA.RDF_LOAD_MBZ_1 (in graph_iri varchar, in new_origin_uri varchar,
in dest varchar, in kind varchar, in id varchar, in inc varchar)
{
declare uri, cnt, xt, xd, hdr any;
uri := sprintf ('http://musicbrainz.org/ws/1/%s/%s?type=xml&inc=%U', kind, id, inc);
cnt := RDF_HTTP_URL_GET (uri, '', hdr, 'GET', 'Accept: */*');
xt := xtree_doc (cnt);
xd := DB.DBA.RDF_MAPPER_XSLT (registry_get ('_rdf_mappers_path_') || 'xslt/mbz2rdf.xsl', xt,
vector ('baseUri', new_origin_uri));
xd := serialize_to_UTF8_xml (xd);
DB.DBA.RM_RDF_LOAD_RDFXML (xd, new_origin_uri, coalesce (dest, graph_iri));
};
XSLT Stylesheet
The key sections of the MusicBrainz XSLT template relevant to this example are listed below. Only the sections relating to an artist, his releases, or the tracks on those releases, are shown.
<!DOCTYPE xsl:stylesheet [
<!ENTITY xsd "http://www.w3.org/2001/XMLSchema#">
<!ENTITY rdf "http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<!ENTITY rdfs "http://www.w3.org/2000/01/rdf-schema#">
<!ENTITY mo "http://purl.org/ontology/mo/">
<!ENTITY foaf "http://xmlns.com/foaf/0.1/">
<!ENTITY mmd "http://musicbrainz.org/ns/mmd-1.0#">
<!ENTITY dc "http://purl.org/dc/elements/1.1/">
]>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:vi="http://www.openlinksw.com/virtuoso/xslt/"
xmlns:rdf=""
xmlns:rdfs=""
xmlns:foaf=""
xmlns:mo=""
xmlns:mmd=""
xmlns:dc=""
>
<xsl:output method="xml" indent="yes" />
<xsl:variable name="base" select="'http://musicbrainz.org/'"/>
<xsl:variable name="uc">ABCDEFGHIJKLMNOPQRSTUVWXYZ</xsl:variable>
<xsl:variable name="lc">abcdefghijklmnopqrstuvwxyz</xsl:variable>
<xsl:template match="/mmd:metadata">
<rdf:RDF>
<xsl:apply-templates />
</rdf:RDF>
</xsl:template>
...
<xsl:template match="mmd:artist[@type='Person']">
<mo:MusicArtist rdf:about="{vi:proxyIRI (concat($base,'artist/',@id,'.html'))}">
<foaf:name><xsl:value-of select="mmd:name"/></foaf:name>
<xsl:for-each select="mmd:release-list/mmd:release|mmd:relation-list[@target-type='Release']/mmd:relation/mmd:release">
<foaf:made rdf:resource="{vi:proxyIRI (concat($base,'release/',@id,'.html'))}"/>
</xsl:for-each>
</mo:MusicArtist>
<xsl:apply-templates />
</xsl:template>
<xsl:template match="mmd:release">
<mo:Record rdf:about="{vi:proxyIRI (concat($base,'release/',@id,'.html'))}">
<dc:title><xsl:value-of select="mmd:title"/></dc:title>
<mo:release_type rdf:resource="{translate (substring-before (@type, ' '),
$uc, $lc)}"/>
<mo:release_status rdf:resource="{translate (substring-after (@type, ' '), $uc,
$lc)}"/>
<xsl:for-each select="mmd:track-list/mmd:track">
<mo:track rdf:resource="{vi:proxyIRI (concat($base,'track/',@id,'.html'))}"/>
</xsl:for-each>
</mo:Record>
<xsl:apply-templates select="mmd:track-list/mmd:track"/>
</xsl:template>
<xsl:template match="mmd:track">
<mo:Track rdf:about="{vi:proxyIRI (concat($base,'track/',@id,'.html'))}">
<dc:title><xsl:value-of select="mmd:title"/></dc:title>
<mo:track_number><xsl:value-of select="position()"/></mo:track_number>
<mo:duration rdf:datatype="integer">
<xsl:value-of select="mmd:duration"/>
</mo:duration>
<xsl:if test="artist[@id]">
<foaf:maker rdf:resource="{vi:proxyIRI (concat ($base, 'artist/',
artist/@id, '.html'))}"/>
</xsl:if>
<mo:musicbrainz rdf:resource="{vi:proxyIRI (concat ($base, 'track/', @id, '.html'))}"/>
</mo:Track>
</xsl:template>
...
<xsl:template match="text()"/>
</xsl:stylesheet>
Used to extract RDF from a Web Data Source the Virtuoso Sponger Cartridge RDF Extractor consumes services from: Virtuoso PL, C/C++, Java based RDF Extractors
The RDF mappers provide a way to extract metadata from non-RDF documents such as HTML pages, images Office documents etc. and pass to SPARQL sponger (crawler which retrieve missing source graphs). For brevity further in this article the "RDF mapper" we simply will call "mapper".
The mappers consist of PL procedure (hook) and extractor, where extractor itself can be built using PL, C or any external language supported by Virtuoso server.
Once the mapper is developed it must be plugged into the SPARQL engine by adding a record in the table DB.DBA.SYS_RDF_MAPPERS.
If a SPARQL query instructs the SPARQL processor to retrieve target graph into local storage, then the SPARQL sponger will be invoked. If the target graph IRI represents a dereferenceable URL then content will be retrieved using content negotiation. The next step is the content type to be detected:
PL hook requirements:
Every PL function used to plug a mapper into SPARQL engine must have following parameters in the same order:
Note: the names of the parameters are not important, but their order and presence are!
Example Implementation:
In the example script bellow we implement a basic mapper, which maps a text/plain mime type to an imaginary ontology, which extends the class Document from FOAF with properties 'txt:UniqueWords' and 'txt:Chars', where the prefix 'txt:' we specify as 'urn:txt:v0.0:'.
use DB;
create procedure DB.DBA.RDF_LOAD_TXT_META
(
in graph_iri varchar,
in new_origin_uri varchar,
in dest varchar,
inout ret_body any,
inout aq any,
inout ps any,
inout ser_key any
)
{
declare words, chars int;
declare vtb, arr, subj, ses, str any;
declare ses any;
-- if any error we just say nothing can be done
declare exit handler for sqlstate '*'
{
return 0;
};
subj := coalesce (dest, new_origin_uri);
vtb := vt_batch ();
chars := length (ret_body);
-- using the text index procedures we get a list of words
vt_batch_feed (vtb, ret_body, 1);
arr := vt_batch_strings_array (vtb);
-- the list has 'word' and positions array, so we must divide by 2
words := length (arr) / 2;
ses := string_output ();
-- we compose a N3 literal
http (sprintf ('<%s> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Document> .\n', subj), ses);
http (sprintf ('<%s> <urn:txt:v0.0:UniqueWords> "%d" .\n', subj, words), ses);
http (sprintf ('<%s> <urn:txt:v0.0:Chars> "%d" .\n', subj, chars), ses);
str := string_output_string (ses);
-- we push the N3 text into the local store
DB.DBA.TTLP (str, new_origin_uri, subj);
return 1;
};
DELETE FROM DB.DBA.SYS_RDF_MAPPERS WHERE RM_HOOK = 'DB.DBA.RDF_LOAD_TXT_META';
INSERT SOFT DB.DBA.SYS_RDF_MAPPERS (RM_PATTERN, RM_TYPE, RM_HOOK, RM_KEY, RM_DESCRIPTION)
VALUES ('(text/plain)', 'MIME', 'DB.DBA.RDF_LOAD_TXT_META', null, 'Text Files (demo)');
-- here we set order to some large number so don't break existing mappers
update DB.DBA.SYS_RDF_MAPPERS
SET RM_ID = 2000
WHERE RM_HOOK = 'DB.DBA.RDF_LOAD_TXT_META';
To test the mapper we just use /sparql endpoint with option 'Retrieve remote RDF data for all missing source graphs' to execute:
SELECT *
FROM <URL-of-a-txt-file>
WHERE { ?s ?p ?o }
It is important that the SPARQL_UPDATE role to be granted to "SPARQL" account in order to allow local repository update via sponge feature.
Authentication in Sponger
To enable usage of user defined authentication, there are added more parameters to the /proxy/rdf and /sparql endpoints. So to use it, the RDF browser and iSPARQL should send following url parameters:
'login=<account name>'
get-login=<account name>
The table DB.DBA.SYS_RDF_MAPPERS is used as registry for registering RDF mappers.
create table DB.DBA.SYS_RDF_MAPPERS (
RM_ID integer identity, -- mapper ID, designate order of execution
RM_PATTERN varchar, -- a REGEX pattern to match URL or MIME type
RM_TYPE varchar default 'MIME', -- what property of the current resource to match: MIME or URL are supported at present
RM_HOOK varchar, -- fully qualified PL function name e.q. DB.DBA.MY_MAPPER_FUNCTION
RM_KEY long varchar, -- API specific key to use
RM_DESCRIPTION long varchar, -- Mapper description, free text
RM_ENABLED integer default 1, -- a flag 0 or 1 integer to include or exclude the given mapper from processing chain
primary key (RM_TYPE, RM_PATTERN))
;
The current way to register/update/unregister a mapper is just a DML statement e.g. NSERT/UPDATE/DELETE.
Execution order and processingWhen SPARQL retrieves a resource with unknown content it will look in the mappers registry and will loop over every record having RM_ENABLED flag true. The sequence of look-up is based on ordering by RM_ID column. For every record it will either try matching the MIME type or URL against RM_PATTERN value and if there is match the function specified in RM_HOOK column will be called. If the function doesn't exists or signal an error the SPARQL will look at next record.
When it stops looking? It will stop if value returned by mapper function is positive or negative number, if the return is negative processing stops with meaning no RDF was supplied, if return is positive the meaning is that RDF data was extracted, if zero integer is returned then SPARQL will look for next mapper. The mapper function also can return zero if it is expected next mapper in the chain to get more RDF data.
If none of the mappers matches the signature (MIME type nor URL) the built-in WebDAV metadata extractor will be called.
Extension functionThe mapper function is a PL stored procedure with following signature:
THE_MAPPER_FUNCTION_NAME (
in graph_iri varchar,
in origin_uri varchar,
in destination_uri varchar,
inout content varchar,
inout async_notification_queue any,
inout ping_service any,
inout keys any
)
{
-- do processing here
-- return -1, 0 or 1 (as explained above in Execution order and processing section)
}
;
Parameters
Return value
The Virtuoso supply as a rdf_mappers_dav VAD package a cartridge for extracting RDF data from certain popular Web resources and file types. It can be installed (if not already) using VAD_INSTALL function, see the VAD chapter in documentation on how to do that.
HTTP-in-RDF
Maps the HTTP request response to HTTP Vocabulary in RDF, see http://www.w3.org/2006/http#.
This mapper is disabled by default. If it's enabled , it must be first in order of execution.
Also it always will return 0, which means any other mapper should push more data.
HTML
This mapper is composite, it looking for metadata which can specified in a HTML pages as follows:
<link rel="meta" type="application/rdf+xml"
The HTML page mapper will look for RDF data in order as listed above, it will try to extract metadata on each step and will return positive flag if any of the above step give a RDF data. In case where page URL matches some of other RDF mappers listed in registry it will return 0 so next mapper to extract more data. In order to function properly, this mapper must be executed before any other specific mappers.
Flickr URLs
This mapper extracts metadata of the Flickr images, using Flickr REST API. To function properly it must have configured key. The Flickr mapper extracts metadata using: CC license, Dublin Core, Dublin Core Metadata Terms, GeoURL, FOAF, EXIF: http://www.w3.org/2003/12/exif/ns/ ontology.
Amazon URLs
This mapper extracts metadata for Amazon articles, using Amazon REST API. It needs a Amazon API key in order to be functional.
eBay URLs
Implements eBay REST API for extracting metadata of eBay articles, it needs a key and user name to be configured in order to work.
Open Office (OO) documents
The OO documents contains metadata which can be extracted using UNZIP, so this extractor needs Virtuoso unzip plugin to be configured on the server.
Yahoo traffic data URLs
Implements transformation of the result of Yahoo traffic data to RDF.
iCal files
Transform iCal files to RDF as per http://www.w3.org/2002/12/cal/ical# .
Binary content, PDF, PowerPoint
The unknown binary content, PDF and MS PowerPoint files can be transformed to RDF using Aperture framework (http://aperture.sourceforge.net/). This mapper needs Virtuoso with Java hosting support, Aperture framework and MetaExtractor.class installed on the host system in order to work.
The Aperture framework & MetaExtractor.class must be installed on the system before to install the RDF mappers package. If the package is already installed, then to activate this mapper you can just re-install the VAD.
Setting-up Virtuoso with Java hosting to run Aperture framework
JavaClasspath = lib:aperture/DFKIUtils2.jar:aperture/JempBox-0.2.0.jar:aperture/activation-1.0.2-upd2.jar:aperture/aduna-commons-xml-2.0.jar: aperture/ant-compression-utils-1.7.1.jar:aperture/aperture-1.2.0.jar:aperture/aperture-examples-1.2.0.jar:aperture/aperture-test-1.2.0.jar: aperture/applewrapper-0.2.jar:aperture/bcmail-jdk14-132.jar:aperture/bcprov-jdk14-132.jar:aperture/commons-codec-1.3.jar:aperture/commons-httpclient-3.1.jar: aperture/commons-lang-2.3.jar:aperture/demork-2.1.jar:aperture/flickrapi-1.0.jar:aperture/fontbox-0.2.0-dev.jar:aperture/htmlparser-1.6.jar: aperture/ical4j-1.0-beta4.jar:aperture/infsail-0.1.jar:aperture/jacob-1.10.jar:aperture/jai_codec-1.1.3.jar:aperture/jai_core-1.1.3.jar:aperture/jaudiotagger-1.0.8.jar: aperture/jcl104-over-slf4j-1.5.0.jar:aperture/jpim-0.1-aperture-1.jar:aperture/junit-3.8.1.jar:aperture/jutf7-0.9.0.jar:aperture/mail-1.4.jar: aperture/metadata-extractor-2.4.0-beta-1.jar:aperture/mstor-0.9.11.jar:aperture/nrlvalidator-0.1.jar:aperture/openrdf-sesame-2.2.1-onejar-osgi.jar: aperture/osgi.core-4.0.jar:aperture/pdfbox-0.7.4-dev-20071030.jar:aperture/poi-3.0.2-FINAL-20080204.jar:aperture/poi-scratchpad-3.0.2-FINAL-20080204.jar: aperture/rdf2go.api-4.6.2.jar:aperture/rdf2go.impl.base-4.6.2.jar:aperture/rdf2go.impl.sesame20-4.6.2.jar:aperture/rdf2go.impl.util-4.6.2.jar: aperture/slf4j-api-1.5.0.jar:aperture/slf4j-jdk14-1.5.0.jar:aperture/unionsail-0.1.jar:aperture/winlaf-0.5.1.jar
To check the cartridge has been configured, connect with Virtuoso's ISQL tool:
SQL> SELECT udt_is_available('APERTURE.DBA.MetaExtractor');
SQL> SELECT APERTURE.DBA."MetaExtractor"().getMetaFromFile ('some_pdf_in_server_working_dir.pdf', 0);
... some RDF data should be returned ...
You should now be able to sponge all document types supported by the Aperture framework, (using one of the standard Sponger invocation mechanisms, for instance with a URL of the form http://localhost:8890/about/rdf/http://targethost/targetfile.pdf), subject to the MIME type pattern filters configured for the cartridge in the Conductor UI. By default the Aperture cartridge is registered to match MIME types (application/octet-stream)|(application/pdf)|(application/mspowerpoint). To sponge all the MIME types Aperture is capable of handling, changed the MIME type pattern to 'application/.*'.
Important: The installation guidelines presented above have been verified on Mac OS X with Aperture 1.2.0. Some adjustment may be needed for different operating systems or versions of Aperture.
Examples & tutorials
How to write own RDF mapper? Look at Virtuoso tutorial on this subject http://demo.openlinksw.com/tutorial/rdf/rd_s_1/rd_s_1.vsp .
So far the discussion has centered on 'primary' cartridges. However, Virtuoso supports an alternative type of cartridge, a 'meta-cartridge'. The way a meta-cartridge operates is essentially the same as a primary cartridge, that is it has a cartridge hook function with the same signature and its inserts data into the quad store through entity extraction and ontology mapping as before. Where meta-cartridges differ from primary cartridges is in their intent and their position in the cartridge invocation pipeline.
The purpose of meta-cartridges is to enrich graphs produced by other (primary) cartridges. They serve as general post-processors to add additional information about selected entities in an RDF graph. For instance, a particular meta-cartridge might be designed to search for entities of type 'umbel:Country' in a given graph, and then add additional statements about each country it finds, where the information contained in these statements is retrieved from the web service targeted by the meta-cartridge. One such example might be a 'World Bank' meta-cartridge which adds information relating to a country's GDP, its exports of goods and services as a percentage of GDP etc; retrieved using the World Bank web service API. In order to benefit from the World Bank meta-cartridge, any primary cartridge which might generate instance data relating to countries should ensure that each country instance it handles is also described as being of rdf:type 'umbel:Country'. Here, the UMBEL (Upper Mapping and Binding Exchange Layer) ontology is used as a data-source-agnostic classification system. It provides a core set of 20,000+ subject concepts which act as "a fixed set of reference points in a global knowledge space". The use of UMBEL in this way serves to decouple meta-cartridges from primary cartridges and data source specific ontologies.
Virtuoso includes two default meta-cartridges which use UMBEL and OpenCalais to augment source graphs.
Registration
Meta-cartridges must be registered in the RDF_META_CARTRIDGES table, which fulfills a role similar to the SYS_RDF_MAPPERS table used by primary cartridges. The structure of the table, and the meaning and use of its columns, are similar to SYS_RDF_MAPPERS. The meta-cartridge hook function signature is identical to that for primary cartridges.
The RDF_META_CARTRIDGES table definition is as follows:
create table DB.DBA.RDF_META_CARTRIDGES (
MC_ID INTEGER IDENTITY, -- meta-cartridge ID. Determines the order of the
meta-cartridge's invocation in the Sponger
processing chain
MC_SEQ INTEGER IDENTITY,
MC_HOOK VARCHAR, -- fully qualified Virtuoso/PL function name
MC_TYPE VARCHAR,
MC_PATTERN VARCHAR, -- a REGEX pattern to match resource URL or
MIME type
MC_KEY VARCHAR, -- API specific key to use
MC_OPTIONS ANY, -- meta-cartridge specific options
MC_DESC LONG VARCHAR, -- meta-cartridge description (free text)
MC_ENABLED INTEGER -- a 0 or 1 integer flag to exclude or include
meta-cartridge from Sponger processing chain
);
(At the time of writing there is no Conductor UI for registering meta-cartridges, they must be registered using SQL. A Conductor interface for this task will be added in due course.)
Invocation
Meta-cartridges are invoked through the post-processing hook procedure RDF_LOAD_POST_PROCESS which is called, for every document retrieved, after RDF_LOAD_RDFXML loads sponged data into the quad store.
Cartridges in the meta-cartridge registry (RDF_META_CARTRIDGES) are configured to match a given MIME type or URI pattern. Matching meta-cartridges are invoked in order of their MC_SEQ value. Ordinarily a meta-cartridge should return 0, in which case the next meta-cartridge in the post-processing chain will be invoked. If it returns 1 or -1, the post-processing stops and no further meta-cartridges are invoked.
The order of processing by the Sponger cartridge pipeline is thus:
Notice that meta-cartridges may be invoked even if primary cartridges are not.
Note
The example which follows builds on a Freebase Sponger cartridge developed prior to the announcement of Freebase's support for generating Linked Data through the endpoint http://rdf.freebase.com/ . The OpenLink cartridge has since evolved to reflect these changes. A snapshot of the Freebase cartridge and stylesheet compatible with this example can be found in Appendix C.
Freebase is an open community database of the world's information which serves facts and statistics rather than articles. Its designers see this difference in emphasis from article-oriented databases as beneficial for developers wanting to use Freebase facts in other websites and applications.
Virtuoso includes a Freebase cartridge in the rdf_mappers VAD. The aim of the example cartridge presented here is to provide a lightweight meta-cartridge that is used to conditionally add triples to graphs generated by the Freebase cartridge, if Freebase is describing a U.S. senator.
New York Times Campaign Finance (NYTCF) API
The New York Times Campaign Finance (NYTCF) API allows you to retrieve contribution and expenditure data based on United States Federal Election Commission filings. You can retrieve totals for a particular presidential candidate, see aggregates by ZIP code or state, or get details on a particular donor.
The API supports a number of query types. To keep this example from being overly long, the meta-cartridge supports just one of these - a query for the candidate details. An example query and the resulting output follow:
Query:
http://api.nytimes.com/svc/elections/us/v2/president/2008/finances/candidates/obama,barack.xml?api-key=xxxx
Result:
<result_set>
<status>OK</status>
<copyright>
Copyright (c) 2008 The New York Times Company. All Rights Reserved.
</copyright>
<results>
<candidate>
<candidate_name>Obama, Barack</candidate_name>
<committee_id>C00431445</committee_id>
<party>D</party>
<total_receipts>468841844</total_receipts>
<total_disbursements>391437723.5</total_disbursements>
<cash_on_hand>77404120</cash_on_hand>
<net_individual_contributions>426902994</net_individual_contributions>
<net_party_contributions>150</net_party_contributions>
<net_pac_contributions>450</net_pac_contributions>
<net_candidate_contributions>0</net_candidate_contributions>
<federal_funds>0</federal_funds>
<total_contributions_less_than_200>222694981.5</total_contributions_less_than_200>
<total_contributions_2300>76623262</total_contributions_2300>
<net_primary_contributions>46444638.81</net_primary_contributions>
<net_general_contributions>30959481.19</net_general_contributions>
<total_refunds>2058240.92</total_refunds>
<date_coverage_from>2007-01-01</date_coverage_from>
<date_coverage_to>2008-08-31</date_coverage_to>
</candidate>
</results>
</result_set>
Sponging Freebase
Using OpenLink Data Explorer
The following instructions assume you have the OpenLink Data Explorer (ODE) browser extension installed in your browser.
An HTML description of Barack Obama can be obtained directly from Freebase by pasting the following URL into your browser: http://www.freebase.com/view/en/barack_obama
To view RDF data sponged from this page, select 'Linked Data Sources' from the browser's 'View' menu. An OpenLink Data Explorer interface will load in a new tab.
Clicking on the 'Barack Obama' link under the 'Person' category displayed by ODE sponges RDF data using the Freebase cartridge. Click the 'down arrow' adjacent to the 'Barack Obama' link to explore the retrieved data.
Assuming your Virtuoso instance is running on port 8890 on localhost, the list of data caches displayed by ODE should include: http://localhost:8890/about/html/http/www.freebase.com/view/en/barack_obama#this
The information displayed in the rest of the page relates to the entity instance identified by this URI. The prefix http://localhost:8890/about/html/http/ prepended to the original URI indicates that the Sponger Proxy Service has been invoked. The Sponger creates an associated entity instance (identified by the above URI with the #this suffix) which holds sponged information about the original entity.
Using the Command Line
As an alternative to ODE, you can sponge from the command line with the command:
curl -H "Accept: text/xml" "http://localhost:8890/about/html/http/www.freebase.com/view/en/barack_obama"
To view the results, you can use Conductor's browser-based SPARQL interface (e.g. http://localhost:8890/sparql) to query the resulting graph generated by the Sponger, http://www.freebase.com/view/en/barack_obama.
Installing the Meta-Cartridge
To register the meta-cartridge, a procedure similar to the following can be used:
create procedure INSTALL_RDF_LOAD_NYTCF ()
{
-- delete any previous NYTCF cartridge installed as a primary cartridge
DELETE FROM SYS_RDF_MAPPERS WHERE RM_HOOK = 'DB.DBA.RDF_LOAD_NYTCF';
-- register in the meta-cartridge post-processing chain
INSERT SOFT DB.DBA.RDF_META_CARTRIDGES (MC_PATTERN, MC_TYPE, MC_HOOK,
MC_KEY, MC_DESC, MC_OPTIONS)
VALUES (
'http://www.freebase.com/view/.*',
'URL', 'DB.DBA.RDF_LOAD_NYTCF', '2c1d95a62e5fxxxxx', 'Freebase NYTCF',
vector ());
};
Looking at the list of cartridges in Conductor's 'RDF Cartridges' screen, you will see that the Freebase cartridge is configured by default to sponge URIs which match the pattern "http://www.freebase.com/view/.*" The meta-cartridge is configured to match on the same URI pattern.
To use the Campaign Finance API, you must register and request an API key. The script above shows an invalid key. Replace it with your own key before executing the procedure.
NYTCF Meta-Cartridge Functions
The meta-cartridge function definitions are listed below. They can be executed by pasting them into Conductor's iSQL interface.
-- New York Times: Campaign Finance Web Service
-- See http://developer.nytimes.com/docs/campaign_finance_api
-- DB.DBA.RDF_NYTCF_LOOKUP is in effect a lightweight lookup cartridge that is used
-- to conditionally add triples to graphs generated by the Wikipedia and
-- Freebase cartridges. These cartridges call on RDF_NYTCF_LOOKUP when
-- handling an entity of rdf:type yago:Congressman109955781. The NYTCF lookup
-- cartridge (aka a metacartridge) is used to return campaign finance data
-- for the candidate in question retrieved from the New York Times Campaign
-- Finance web service.
create procedure DB.DBA.RDF_NYTCF_LOOKUP(
in candidate_id any, -- id of candidate
in graph_iri varchar, -- graph into which the additional campaign finance triples should be loaded
in api_key varchar -- NYT finance API key
)
{
declare version, campaign_type, year any;
declare nyt_url, hdr, tmp any;
declare xt, xd any;
-- Common parameters - The NYT API only supports the following values at present:
version := 'v2';
campaign_type := 'president';
year := '2008';
-- Candidate summaries
-- nyt_url := sprintf('http://api.nytimes.com/svc/elections/us/%s/%s/%s/finances/totals.xml?api-key=%s',
-- version, campaign_type, year, api_key);
-- Candidate details
nyt_url := sprintf('http://api.nytimes.com/svc/elections/us/%s/%s/%s/finances/candidates/%s.xml?api-key=%s',
version, campaign_type, year, candidate_id, api_key);
tmp := http_client_ext (nyt_url, headers=>hdr, proxy=>connection_get ('sparql-get:proxy'));
if (hdr[0] not like 'HTTP/1._ 200 %')
signal ('22023', trim(hdr[0], '\r\n'), 'DB.DBA.RDF_LOAD_NYTCF_LOOKUP');
xd := xtree_doc (tmp);
-- baseUri specifies what the generated RDF description is about
-- <rdf:Description rdf:about="{baseUri}">
-- Example baseUri's:
-- http://localhost:8890/about/rdf/http://www.freebase.com/view/en/barack_obama#this
-- http://localhost:8890/about/rdf/http://www.freebase.com/view/en/hillary_rodham_clinton#this
declare path any;
declare lang, k, base_uri varchar;
if (graph_iri like 'http://rdf.freebase.com/ns/%.%')
base_uri := graph_iri;
else
{
path := split_and_decode (graph_iri, 0, '%\0/');
k := path [length(path) - 1];
lang := path [length(path) - 2];
base_uri := sprintf ('http://rdf.freebase.com/ns/%U.%U', lang, k);
}
xt := DB.DBA.RDF_MAPPER_XSLT (registry_get ('_rdf_mappers_path_') || 'xslt/nytcf2rdf.xsl', xd,
vector ('baseUri', base_uri));
xd := serialize_to_UTF8_xml (xt);
DB.DBA.RDF_LOAD_RDFXML (xd, '', graph_iri);
}
;
create procedure DB.DBA.RDF_MQL_RESOURCE_IS_SENATOR (
in fb_graph_uri varchar -- URI of graph containing Freebase resource
)
{
-- Check if the resource described by Freebase is a U.S. senator. Only then does it make sense to query for campaign finance
-- data from the NYT data space.
--
-- To test for senators, we start by looking for two statements in the Freebase cartridge output, similar to:
--
-- <rdf:Description rdf:about="http://localhost:8890/about/rdf/http://www.freebase.com/view/en/hillary_rodham_clinton#this">
-- <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Person"/>
-- <rdfs:seeAlso rdf:resource="http://en.wikipedia.org/wiki/Hillary_Rodham_Clinton"/>
-- ...
-- where the graph generated by the Sponger will be <http://www.freebase.com/view/en/hillary_rodham_clinton>
--
-- To test whether a resource is a senator:
-- 1) Check whether the Freebase resource is of rdf:type foaf:Person
-- 2) Extract the person_name from the Wikipedia URI referenced by rdfs:seeAlso
-- 3) Use the extracted person_name to build a URI to DBpedia's description of the person.
-- 4) Query the DBpedia description to see if the person is of rdf:type yago:Senator110578471
declare xp, xt, tmp any;
declare qry varchar; -- SPARQL query
declare qry_uri varchar; -- query URI
declare qry_res varchar; -- query result
declare dbp_resource_name varchar; -- Equivalent resource name in DBpedia
declare fb_resource_uri varchar; -- Freebase resource URI
declare path any;
declare lang, k varchar;
declare exit handler for sqlstate '*' {
return 0;
};
if (fb_graph_uri like 'http://rdf.freebase.com/ns/%.%')
fb_resource_uri := fb_graph_uri;
else
{
path := split_and_decode (fb_graph_uri, 0, '%\0/');
if (length (path) < 2)
return 0;
k := path [length(path) - 1];
lang := path [length(path) - 2];
fb_resource_uri := sprintf ('http://rdf.freebase.com/ns/%U.%U', lang, k);
}
-- 1) Check whether the Freebase resource is a politician from united_states
{
declare stat, msg varchar;
declare mdata, rset any;
qry := sprintf ('sparql ask from <%s> where { <%s> <http://rdf.freebase.com/ns/people.person.profession> <http://rdf.freebase.com/ns/en.politician> ; <http://rdf.freebase.com/ns/people.person.nationality> <http://rdf.freebase.com/ns/en.united_states> . }', fb_graph_uri, fb_resource_uri);
exec (qry, stat, msg, vector(), 1, mdata, rset);
if (length(rset) = 0 or rset[0][0] <> 1)
return 0;
}
return 1;
}
;
create procedure DB.DBA.RDF_LOAD_NYTCF_META (in graph_iri varchar, in new_origin_uri varchar, in dest varchar,
inout _ret_body any, inout aq any, inout ps any, inout _key any, inout opts any)
{
declare candidate_id, candidate_name any;
declare api_key any;
declare indx, tmp any;
declare ord int;
declare exit handler for sqlstate '*'
{
return 0;
};
if (not DB.DBA.RDF_MQL_RESOURCE_IS_SENATOR (new_origin_uri))
return 0;
-- TO DO: hardcoded for now
-- Need a mechanism to specify API key for meta-cartridges
-- Could retrieve from virtuoso.ini?
api_key := _key;
-- NYT API supports a candidate_id in one of two forms:
-- candidate_id ::= {candidate_ID} | {last_name [,first_name]}
-- first_name is optional. If included, there should be no space after the comma.
--
-- However, because this meta cartridge supplies additional triples for the
-- Wikipedia or Freebase cartridges, only the second form of candidate_id is
-- supported. i.e. We extract the candidate name, rather than a numeric
-- candidate_ID (FEC committee ID) from the Wikipedia or Freebase URL.
--
-- It's assumed that the source URI includes the candidate's first name.
-- If it is omitted, the NYT API will return information about *all* candidates
-- with that last name - something we don't want.
indx := strstr(graph_iri, 'www.freebase.com/view/en/');
if (indx is not null)
{
-- extract candidate_id from Freebase URI
tmp := sprintf_inverse(subseq(graph_iri, indx), 'www.freebase.com/view/en/%s', 0);
if (length(tmp) <> 1)
return 0;
candidate_name := tmp[0];
}
else
{
indx := strstr(graph_iri, 'wikipedia.org/wiki/');
if (indx is not null)
{
-- extract candidate_id from Wikipedia URI
tmp := sprintf_inverse(subseq(graph_iri, indx), 'wikipedia.org/%s', 0);
if (length(tmp) <> 1)
return 0;
candidate_name := tmp[0];
}
else
{
tmp := sprintf_inverse(graph_iri, 'http://%s.freebase.com/ns/%s/%s', 0);
if (length (tmp) <> 3)
tmp := sprintf_inverse(graph_iri, 'http://%s.freebase.com/ns/%s.%s', 0);
if (length (tmp) <> 3)
return 0;
candidate_name := tmp[2];
}
}
-- split candidate_name into its component parts
-- candidate_name is assumed to be firstname_[middlename_]*lastname
-- e.g. hillary_rodham_clinton (Freebase), Hillary_clinton (Wikipedia)
{
declare i, _end, len int;
declare names, tmp_name varchar;
names := vector ();
tmp_name := candidate_name;
len := length (tmp_name);
while (1)
{
_end := strchr(tmp_name, '_');
if (_end is not null)
{
names := vector_concat (names, vector(subseq(tmp_name, 0, _end)));
tmp_name := subseq(tmp_name, _end + 1);
}
else
{
names := vector_concat(names, vector(tmp_name));
goto done;
}
}
done:
if (length(names) < 2)
return 0;
-- candidate_id ::= lastname,firstname
candidate_id := sprintf('%s,%s', names[length(names)-1], names[0]);
}
DB.DBA.RDF_NYTCF_LOOKUP(candidate_id, coalesce (dest, graph_iri), api_key);
return 0;
}
;
NYTCF Meta-Cartridge Stylesheet
The XSLT stylesheet, nyctf2rdf.xsl, used by the meta-cartridge to transform the base Campaign Finance web service output to RDF is shown below. RDF_NYCTF_LOOKUP() assumes the stylesheet is located alongside the other stylesheets provided by the rdf_mappers VAD in the Virtuoso WebDAV folder DAV/VAD/rdf_mappers/xslt. You should create nyctf2rdf.xsl here from the following listing. The WebDAV Browser interface in Conductor provides the easiest means to upload the stylesheet.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE xsl:stylesheet [
<!ENTITY rdf "http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<!ENTITY nyt "http://www.nytimes.com/">
]>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:vi="http://www.openlinksw.com/virtuoso/xslt/"
xmlns:rdf=""
xmlns:nyt=""
>
<xsl:output method="xml" indent="yes" />
<xsl:template match="/result_set/status">
<xsl:if test="text() = 'OK'">
<xsl:apply-templates mode="ok" select="/result_set/results/candidate"/>
</xsl:if>
</xsl:template>
<xsl:template match="candidate" mode="ok">
<rdf:Description rdf:about="{vi:proxyIRI($baseUri)}">
<nyt:candidate_name><xsl:value-of select="candidate_name"/></nyt:candidate_name>
<nyt:committee_id><xsl:value-of select="committee_id"/></nyt:committee_id>
<nyt:party><xsl:value-of select="party"/></nyt:party>
<nyt:total_receipts><xsl:value-of select="total_receipts"/></nyt:total_receipts>
<nyt:total_disbursements>
<xsl:value-of select="total_disbursements"/>
</nyt:total_disbursements>
<nyt:cash_on_hand><xsl:value-of select="cash_on_hand"/></nyt:cash_on_hand>
<nyt:net_individual_contributions>
<xsl:value-of select="net_individual_contributions"/>
</nyt:net_individual_contributions>
<nyt:net_party_contributions>
<xsl:value-of select="net_party_contributions"/>
</nyt:net_party_contributions>
<nyt:net_pac_contributions>
<xsl:value-of select="net_pac_contributions"/>
</nyt:net_pac_contributions>
<nyt:net_candidate_contributions>
<xsl:value-of select="net_candidate_contributions"/>
</nyt:net_candidate_contributions>
<nyt:federal_funds><xsl:value-of select="federal_funds"/></nyt:federal_funds>
<nyt:total_contributions_less_than_200>
<xsl:value-of select="total_contributions_less_than_200"/>
</nyt:total_contributions_less_than_200>
<nyt:total_contributions_2300>
<xsl:value-of select="total_contributions_2300"/>
</nyt:total_contributions_2300>
<nyt:net_primary_contributions>
<xsl:value-of select="net_primary_contributions"/>
</nyt:net_primary_contributions>
<nyt:net_general_contributions>
<xsl:value-of select="net_general_contributions"/>
</nyt:net_general_contributions>
<nyt:total_refunds><xsl:value-of select="total_refunds"/></nyt:total_refunds>
<nyt:date_coverage_from rdf:datatype="date">
<xsl:value-of select="date_coverage_from"/>
</nyt:date_coverage_from>
<nyt:date_coverage_to rdf:datatype="date">
<xsl:value-of select="date_coverage_to"/>
</nyt:date_coverage_to>
</rdf:Description>
</xsl:template>
<xsl:template match="text()|@*"/>
</xsl:stylesheet>
The stylesheet uses the prefix nyt: (http://www.nytimes.com) for the predicates of the augmenting triples. This has been used purely for illustration - you may prefer to define your own ontology for RDF data derived from New York Times APIs.
Testing the Meta-Cartridge
After creating the required Virtuoso/PL functions and installing the stylesheet, you should be able to test the meta-cartridge by sponging a Freebase page as described earlier using ODE or the command line. For instance:
You should see campaign finance data added to the graph created by the Sponger in the form of triples with predicates starting http://www.nytimes.com/xxx, e.g. http://www.nytimes.com/net_primary_contribution.
How The Meta-Cartridge Works
The comments in the meta-cartridge code detail how the cartridge works. In brief:
Given the URI of the graph being created by the Freebase cartridge, RDF_MQL_RESOURCE_IS_SENATOR checks if the resource described by Freebase is a U.S. senator. Only then does it make sense to query for campaign finance data from the NYTCF data space.
To test for senators, the procedure starts by looking for two statements in the Freebase cartridge output similar to:
<rdf:Description rdf:about="http://localhost:8890/about/rdf/http://www.freebase.com/view/en/barack_obama#this"> <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Person"/> <rdfs:seeAlso rdf:resource="http://en.wikipedia.org/wiki/Barack_Obama"/> ...
where the graph generated by the Sponger will be
<http://www.freebase.com/view/en/barack_obama>
To test whether a resource is a senator, RDF_MQL_RESOURCE_IS_SENATOR
Only if this is the case is the RDF_NYTCF_LOOKUP routine called to query for and return campaign finance data for the candidate. The form of the query and the resulting XML output from the Campaign Finance service were presented earlier.
tmp := sprintf_inverse (new_origin_uri, 'http://farm%s.static.flickr.com/%s/%s_%s.%s', 0); img_id := tmp[2];
request_hdr := headers[0]; response_hdr := headers[1]; host := http_request_header (request, 'Host'); tmp := split_and_decode (request_hdr[0], 0, '\0\0 '); http_method := tmp[0]; url := tmp[1]; protocol_version := substring (tmp[2], 6, 8); tmp := rtrim (response_hdr[0], '\r\n'); tmp := split_and_decode (response_hdr[0], 0, '\0\0 ');
url := sprintf('http://api.flickr.com/services/rest/?i"??
method=flickr.photos.getInfo&photo_id=%s&api_key=%s', img_id, api_key);
tmp := http_get (url, hdr);
if (hdr[0] not like 'HTTP/1._ 200 %')
signal ('22023', trim(hdr[0], '\r\n'), 'RDFXX');
xd := xtree_doc (tmp);
DB.DBA.RDF_HTTP_URL_GET
A wrapper around http_get. Retrieves a URL using the specified HTTP method (defaults to GET). The function can handle proxies, redirects (up to fifteen) and HTTPS.
uri := sprintf ('http://musicbrainz.org/ws/1/%s/%s?type=xml&inc=%U',
kind, id, inc);
cnt := RDF_HTTP_URL_GET (uri, '', hdr, 'GET', 'Accept: */*');
xt := xtree_doc (cnt);
xd := DB.DBA.RDF_MAPPER_XSLT (registry_get ('_rdf_mappers_path_') || 'xslt/mbz2rdf.xsl', xt, vector ('baseUri', new_origin_uri));
content := RDF_HTTP_URL_GET (rdf_url, new_origin_uri, hdr, 'GET', 'Accept: application/rdf+xml, text/rdf+n3, */*'); ret_content_type := http_request_header (hdr, 'Content-Type', null, null);
json_parse: Parses JSON content into a tree.
url := sprintf ('http://www.freebase.com/api/service/mqlread?queries=%U', qr);
content := http_get (url, hdr);
tree := json_parse (content);
tree := get_keyword ('ROOT', tree);
tree := get_keyword ('result', tree);
-- Writing N3 to a string output stream using function http(), parsing the N3 into a graph, then loading the graph into the quad store.
ses := string_output ();
http ('@prefix opl: <http://www.openlinksw.com/schema/attribution#> .\n', ses);
http ('@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .\n', ses);
...
DB.DBA.TTLP (ses, base, graph);
DB.DBA.RDF_LOAD_RDFXML (strg, base, graph);
ses := string_output ();
cnt := http_get (sprintf ('http://download.finance.yahoo.com/d/quotes.csv?s=%U&f=nsbavophg&e=.csv',
symbol));
arr := rdfm_yq_parse_csv (cnt);
http ('<quote stock="NASDAQ">', ses);
foreach (any q in arr) do
{
http_value (q[0], 'company', ses);
http_value (q[1], 'symbol', ses);
...
}
http ('</quote>', ses);
content := string_output_string (ses);
xt := xtree_doc (content);
content := RDF_HTTP_URL_GET (uri, '', hdr, 'GET', 'Accept: */*'); xt := xtree_doc (content);
profile := cast (xpath_eval ('/html/head/@profile', xt) as varchar);
tmp := http_get (url);
xd := xtree_doc (tmp);
xt := DB.DBA.RDF_MAPPER_XSLT (
registry_get ('_rdf_mappers_path_') || 'xslt/atom2rdf.xsl',
xd, vector ('baseUri', coalesce (dest, graph_iri)));
xt := DB.DBA.RDF_MAPPER_XSLT (
registry_get ('_rdf_mappers_path_') || 'xslt/crunchbase2rdf.xsl',
xt, vector ('baseUri', coalesce (dest, graph_iri), 'base', base,
'suffix', suffix));
xd := serialize_to_UTF8_xml (xt);
DB.DBA.RM_RDF_LOAD_RDFXML (xd, new_origin_uri, coalesce (dest, graph_iri));
content := RDF_HTTP_URL_GET (uri, '', hdr, 'GET', 'Accept: */*');
xt := xtree_doc (content);
xd := DB.DBA.RDF_MAPPER_XSLT (
registry_get ('_rdf_mappers_path_') || 'xslt/mbz2rdf.xsl',
xt, vector ('baseUri', new_origin_uri));
xd := serialize_to_UTF8_xml (xd);
DB.DBA.RM_RDF_LOAD_RDFXML (xd, new_origin_uri, coalesce (dest, graph_iri));
sess := string_output ();
...
http (sprintf ('<http://dbpedia.org/resource/%s>
<http://xbrlontology.com/ontology/finance/stock_market#hasCompetitor>
<http://dbpedia.org/resource/%s> .\n',
symbol, x), sess);
http (sprintf ('<http://dbpedia.org/resource/%s>
<http://www.w3.org/2000/01/rdf-schema#isDefinedBy>
<http://finance.yahoo.com/q?s=%s> .\n',
x, x), sess);
content := string_output_string (sess);
DB.DBA.TTLP (content, new_origin_uri, coalesce (dest, graph_iri));
dbg_obj_print ('try all grddl mappings here');
PingtheSemanticWeb (PTSW) is a repository for RDF documents. The PTSW web service archives the location of recently created or updated RDF documents on the Web. It is intended for use by crawlers or other types of software agents which need to know when and where the latest updated RDF documents can be found. They can request a list of recently updated documents as a starting location to crawl the Semantic Web.
You may find this service useful for publicizing your own RDF content. Content authors can notify PTSW that an RDF document has been created or updated by pinging the service with the URL of the document. The Sponger supports this facility through the async_queue and ping_service parameters of the cartridge hook function, where the ping_service parameter contains the ping service URL as configured in the SPARQL section of the virtuoso.ini file:
[SPARQL] ... PingService = http://rpc.pingthesemanticweb.com/ ...
The configured ping service can be called using an asynchronous request and the RDF_SW_PING procedure as illustrated below.
create procedure DB.DBA.RDF_LOAD_HTML_RESPONSE (
in graph_iri varchar, in new_origin_uri varchar, in dest varchar,
inout ret_body any, inout async_queue any, inout ping_service any,
inout _key any, inout opts any )
{
...
if ( ... and async_queue is not null)
aq_request (async_queue, 'DB.DBA.RDF_SW_PING',
vector (ping_service, new_origin_uri));
For more details refer to section Asynchronous Execution and Multithreading in Virtuoso/PL
A list of the main namespaces / ontologies used by OpenLink-provided Sponger cartridges is given below. Some of these ontologies may prove useful when creating your own cartridges.
Snapshots of the Freebase cartridge and stylesheet compatible with the meta-cartridge example presented earlier in this document can be found below.
DB.DBA.RDF_LOAD_MQL:
--no_c_escapes-
create procedure DB.DBA.RDF_LOAD_MQL (in graph_iri varchar, in new_origin_uri varchar, in dest varchar,
inout _ret_body any, inout aq any, inout ps any, inout _key any, inout opts any)
{
declare qr, path, hdr any;
declare tree, xt, xd, types any;
declare k, cnt, url, sa varchar;
hdr := null;
sa := '';
declare exit handler for sqlstate '*'
{
--dbg_printf ('%s', __SQL_MESSAGE);
return 0;
};
path := split_and_decode (new_origin_uri, 0, '%\0/');
if (length (path) < 1)
return 0;
k := path [length(path) - 1];
if (path [length(path) - 2] = 'guid')
k := sprintf ('"id":"/guid/%s"', k);
else
{
if (k like '#%')
k := sprintf ('"id":"%s"', k);
else
{
sa := DB.DBA.RDF_MQL_GET_WIKI_URI (k);
k := sprintf ('"key":"%s"', k);
}
}
qr := sprintf ('{"ROOT":{"query":[{%s, "type":[]}]}}', k);
url := sprintf ('http://www.freebase.com/api/service/mqlread?queries=%U', qr);
cnt := http_get (url, hdr);
tree := json_parse (cnt);
xt := get_keyword ('ROOT', tree);
if (not isarray (xt))
return 0;
xt := get_keyword ('result', xt);
types := vector ();
foreach (any tp in xt) do
{
declare tmp any;
tmp := get_keyword ('type', tp);
types := vector_concat (types, tmp);
}
--types := get_keyword ('type', xt);
DELETE FROM DB.DBA.RDF_QUAD WHERE g = iri_to_id(new_origin_uri);
foreach (any tp in types) do
{
qr := sprintf ('{"ROOT":{"query":{%s, "type":"%s", "*":[]}}}', k, tp);
url := sprintf ('http://www.freebase.com/api/service/mqlread?queries=%U', qr);
cnt := http_get (url, hdr);
--dbg_printf ('%s', cnt);
tree := json_parse (cnt);
xt := get_keyword ('ROOT', tree);
xt := DB.DBA.MQL_TREE_TO_XML (tree);
--dbg_obj_print (xt);
xt := DB.DBA.RDF_MAPPER_XSLT (registry_get ('_rdf_mappers_path_') || 'xslt/mql2rdf.xsl', xt,
vector ('baseUri', coalesce (dest, graph_iri), 'wpUri', sa));
sa := '';
xd := serialize_to_UTF8_xml (xt);
-- dbg_printf ('%s', xd);
DB.DBA.RM_RDF_LOAD_RDFXML (xd, new_origin_uri, coalesce (dest, graph_iri));
}
return 1;
}
mql2rdf.xsl:
<?xml version="1.0" encoding="UTF-8"?>
<!--
-
- $Id: rdfandsparql.xml,v 1.235 2009/11/16 14:04:18 rtsekova Exp $
-
- This file is part of the OpenLink Software Virtuoso Open-Source (VOS)
- project.
-
- Copyright (C) 1998-2008 OpenLink Software
-
- This project is free software; you can redistribute it and/or modify it
- under the terms of the GNU General Public License as published by the
- Free Software Foundation; only version 2 of the License, dated June 1991.
-
- This program is distributed in the hope that it will be useful, but
- WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- General Public License for more details.
-
- You should have received a copy of the GNU General Public License along
- with this program; if not, write to the Free Software Foundation, Inc.,
- 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
-->
<!DOCTYPE xsl:stylesheet [
<!ENTITY rdf "http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<!ENTITY bibo "http://purl.org/ontology/bibo/">
<!ENTITY xsd "http://www.w3.org/2001/XMLSchema#">
<!ENTITY foaf "http://xmlns.com/foaf/0.1/">
<!ENTITY sioc "http://rdfs.org/sioc/ns#">
]>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:vi="http://www.openlinksw.com/virtuoso/xslt/"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
xmlns:sioc=""
xmlns:bibo=""
xmlns:foaf=""
xmlns:skos="http://www.w3.org/2004/02/skos/core#"
xmlns:dcterms= "http://purl.org/dc/terms/"
xmlns:mql="http://www.freebase.com/">
<xsl:output method="xml" indent="yes" />
<xsl:param name="baseUri" />
<xsl:param name="wpUri" />
<xsl:variable name="ns">http://www.freebase.com/</xsl:variable>
<xsl:template match="/">
<rdf:RDF>
<xsl:if test="/results/ROOT/result/*">
<rdf:Description rdf:about="{$baseUri}">
<rdf:type rdf:resource="Document"/>
<rdf:type rdf:resource="Document"/>
<rdf:type rdf:resource="Container"/>
<sioc:container_of rdf:resource="{vi:proxyIRI($baseUri)}"/>
<foaf:primaryTopic rdf:resource="{vi:proxyIRI($baseUri)}"/>
<dcterms:subject rdf:resource="{vi:proxyIRI($baseUri)}"/>
</rdf:Description>
<rdf:Description rdf:about="{vi:proxyIRI($baseUri)}">
<rdf:type rdf:resource="Item"/>
<sioc:has_container rdf:resource="{$baseUri}"/>
<xsl:apply-templates select="/results/ROOT/result/*"/>
<xsl:if test="$wpUri != ''">
<rdfs:seeAlso rdf:resource="{$wpUri}"/>
</xsl:if>
</rdf:Description>
</xsl:if>
</rdf:RDF>
</xsl:template>
<xsl:template match="*[starts-with(.,'http://') or starts-with(.,'urn:')]">
<xsl:element namespace="{$ns}" name="{name()}">
<xsl:attribute name="rdf:resource">
<xsl:value-of select="vi:proxyIRI (.)"/>
</xsl:attribute>
</xsl:element>
</xsl:template>
<xsl:template match="*[starts-with(.,'/')]">
<xsl:if test="local-name () = 'type' and . like '%/person'">
<rdf:type rdf:resource="Person"/>
</xsl:if>
<xsl:if test="local-name () = 'type'">
<sioc:topic>
<skos:Concept rdf:about="{vi:proxyIRI (concat ($ns, 'view', .))}"/>
</sioc:topic>
</xsl:if>
<xsl:element namespace="{$ns}" name="{name()}">
<xsl:attribute name="rdf:resource">
<xsl:value-of select="vi:proxyIRI(concat ($ns, 'view', .))"/>
</xsl:attribute>
</xsl:element>
</xsl:template>
<xsl:template match="*[* and ../../*]">
<xsl:element namespace="{$ns}" name="{name()}">
<xsl:attribute name="rdf:parseType">Resource</xsl:attribute>
<xsl:apply-templates select="@*|node()"/>
</xsl:element>
</xsl:template>
<xsl:template match="*">
<xsl:if test="* or . != ''">
<xsl:choose>
<xsl:when test="name()='image'">
<foaf:depiction rdf:resource="{vi:mql-image-by-name (.)}"/>
</xsl:when>
<xsl:otherwise>
<xsl:element namespace="{$ns}" name="{name()}">
<xsl:if test="name() like 'date_%'">
<xsl:attribute name="rdf:datatype">dateTime</xsl:attribute>
</xsl:if>
<xsl:apply-templates select="@*|node()"/>
</xsl:element>
</xsl:otherwise>
</xsl:choose>
</xsl:if>
</xsl:template>
</xsl:stylesheet>
This section contains examples of Web resources which can be transformed by RDF Cartridges. It also states where additional setup for given cartridges is needed i.e. keys account names etc.
Service based:
needs: api key example: http://www.amazon.com/gp/product/0553383043
needs: account, api-key example: http://cgi.ebay.com/RARE-DAY-IN-FAIRY-LAND-ELEPHANT-FOLIO-20-FULL-COLOR_W0QQitemZ140209597189QQihZ004QQcategoryZ29223QQssPageNameZWDVWQQrdZ1QQcmdZViewItem
example: http://musicbrainz.org/release/37e955d4-a53c-45aa-a812-1b23b88dbc13.html
example: http://www.freebase.com/view/en/beta_ursae_majoris
needs: api-key, secret, persistent-session-id example: http://www.facebook.com/profile.php?id=841100003
example: http://finance.yahoo.com/q?s=AAPL
example: http://local.yahooapis.com/MapsService/V1/trafficData?appid=YahooDemo&street=701+First+Street&city=Sunnyvale&state=CA
example: https://bugzilla.mozilla.org/show_bug.cgi?id=251714
needs: unzip plugin
needs: php plugin & dbpedia extractor example: http://wikipedia.org/wiki/London
GRDDL
example: http://www.google.com/base/feeds/snippets/17891817243016304554
URN handlers
example: urn:lsid:ubio.org:namebank:12292
needs: hslookup plugin, relevant html, pdf, xml etc. mappers enabled example: doi:10.1038/35057062
example: oai:dcmi.ischool.washington.edu:article/8
The Virtuoso SPARQL engine (called for brevity just SPARQL bellow) supports IRI Dereferencing, however it understands only RDF data, that is it can retrieve only files containing RDF/XML, turtle or N3 serialized RDF data, if format is unknown it will try mapping with built-in WebDAV metadata extractor. In order to extend this feature with dereferencing web or file resources which naturally don't have RDF data (like PDF, JPEG files for example) is provided a special mechanism in SPARQL engine. This mechanism is called RDF mappers for translation of non-RDF data files to RDF.
In order to instruct the SPARQL to call a RDF mapper it needs to be registered and it will be called for a given URL or MIME type pattern. In other words, when unknown for SPARQL format is received during URL dereferencing process, it will look into a special registry (a table) to match either the MIME type or IRI using a regular expression, if match is found the mapper function will be called.
Sponger functionality is also exposed via Virtuoso's "/proxy/rdf/" endpoint, as an in-built REST style Web service available in any Virtuoso standard installation. This web service takes a target URL and either returns the content "as is" or tries to transform (by sponging) to RDF. Thus, the proxy service can be used as a 'pipe' for RDF browsers to browse non-RDF sources.
For more information see RDF Sponger Proxy service
To clear cache on all values of HS_LOCAL_IRI of the SYS_HTTP_SPONGE table use: SPARQL clear graph <A-Named-Graph>;
Requires Virtuoso 6.0 TP1 or higher for use.
If you have an existing Virtuoso 6.x installation, and your Quad Store has greater than 10K worth of triples, please perform the following steps:
drop index RDF_QUAD_OPGS; drop index RDF_QUAD_POGS; drop index RDF_QUAD_GPOS; drop index RDF_QUAD_OGPS; checkpoint; create table R2 (G iri_id_8, S iri_id_8, P iri_id_8, O any, primary key (S, P, O, G)); alter index R2 on R2 partition (S int (0hexffff00)); log_enable (2); INSERT INTO R2 (G, S, P, O) select G, S, P, O FROM rdf_quad; DROP TABLE RDF_QUAD; ALTER TABLE r2 rename RDF_QUAD; checkpoint; create bitmap index RDF_QUAD_OPGS on RDF_QUAD (O, P, G, S) partition (O varchar (-1, 0hexffff)); create bitmap index RDF_QUAD_POGS on RDF_QUAD (P, O, G, S) partition (O varchar (-1, 0hexffff)); create bitmap index RDF_QUAD_GPOS on RDF_QUAD (G, P, O, S) partition (O varchar (-1, 0hexffff)); checkpoint;
Note this step may take sometime depending on how many triples are already in your Quad Store.
|
| Figure: 16.6.1.5.3.1. Install the FCT package |
grant SELECT on DB.DBA.RDF_QUAD to "SPARQL";
|
| Figure: 16.6.1.5.3.1. Install the RDF Mappers package |
|
| Figure: 16.6.1.5.3.1. FCT HTML interface |
RDF_OBJ_FT_RULE_ADD (null, null, 'All'); VT_INC_INDEX_DB_DBA_RDF_OBJ ();
urilbl_ac_init_db()
s_rank()
http://cname/about/html/http/news.cnet.com
|
| Figure: 16.6.1.5.4.1. Sponge data |
|
| Figure: 16.6.1.5.4.1. Facet Browser Search |
|
| Figure: 16.6.1.5.4.1. Facet Browser Search Results |
|
| Figure: 16.6.1.5.4.1. Results of the classes/properties |
|
| Figure: 16.6.1.5.4.1. Exclude Type(s) |
|
| Figure: 16.6.1.5.4.1. Results of Excluded Type(s) |
$ more cnet.xml
<?xml version="1.0"?>
<query xmlns="http://openlinksw.com/services/facets/1.0" inference="" same-as="">
<text>CNET</text>
<view type="text" limit="20" offset=""/>
</query>
$ curl -H "Content-Type: text/xml" -d @cnet.xml http://cname/fct/service
<fct:facets xmlns:fct="http://openlinksw.com/services/facets/1.0/">
<fct:sparql> SELECT ?s1 as ?c1, (bif:search_excerpt (bif:vector ('CNET'), ?o1)) as ?c2, ?sc, ?rank WHERE {{{ SELECT ?s1, (?sc * 3e-1) as ?sc, ?o1, (sql:rnk_scale (<LONG::IRI_RANK> (?s1))) as ?rank WHERE { ?s1 ?s1textp ?o1 . ?o1 bif:contains '"CNET"' option (score ?sc) . } ORDER BY DESC (?sc * 3e-1 + sql:rnk_scale (<LONG::IRI_RANK> (?s1))) LIMIT 20 OFFSET 0 }}}</fct:sparql>
<fct:time>16</fct:time>
<fct:complete>yes</fct:complete>
<fct:timeout>0</fct:timeout>
<fct:db-activity> 131R rnd 36R seq 0P disk 0B / 0 messages</fct:db-activity>
<fct:result type="text">
<fct:row>
<fct:column datatype="trank">4.5</fct:column>
<fct:column datatype="erank">5.881291583872905e-014</fct:column>
<fct:column datatype="url" shortform="http://news.com">http://news.com</fct:column>
<fct:column>Technology News - CNET News</fct:column>
<fct:column><span class="srch_xerpt"><b>CNET</b> News.</span></fct:column>
</fct:row>
<fct:row>
<fct:column datatype="trank">4.5</fct:column>
<fct:column datatype="erank">5.881291583872905e-014</fct:column>
<fct:column datatype="url" shortform="http://news.cnet.com/2547-1_3-0-20.xml">http://news.cnet.com/2547-1_3-0-20.xml</fct:column>
<fct:column>CNET News.com</fct:column>
<fct:column><span class="srch_xerpt"><b>CNET</b> News.</span></fct:column>
</fct:row>
<fct:row>
<fct:column datatype="trank">4.5</fct:column>
<fct:column datatype="erank">5.881291583872905e-014</fct:column>
<fct:column datatype="url" shortform="http://news.cnet.com">http://news.cnet.com</fct:column>
<fct:column>Technology News - CNET News</fct:column>
<fct:column><span class="srch_xerpt"><b>CNET</b> News.</span></fct:column>
</fct:row>
<fct:row>
<fct:column datatype="trank">3.9</fct:column>
<fct:column datatype="erank">5.881291583872905e-014</fct:column>
<fct:column datatype="url" shortform="http://news.com">http://news.com</fct:column>
<fct:column>Technology News - CNET News</fct:column>
<fct:column><span class="srch_xerpt">Technology News <b>CNET</b> News.</span></fct:column>
</fct:row>
<fct:row>
<fct:column datatype="trank">3.9</fct:column>
<fct:column datatype="erank">5.881291583872905e-014</fct:column>
<fct:column datatype="url" shortform="http://news.cnet.com">http://news.cnet.com</fct:column>
<fct:column>Technology News - CNET News</fct:column>
<fct:column><span class="srch_xerpt">Technology News <b>CNET</b> News.</span></fct:column>
</fct:row>
<fct:row>
<fct:column datatype="trank">3</fct:column>
<fct:column datatype="erank">5.881291583872905e-014</fct:column>
<fct:column datatype="url" shortform="http://news.com">http://news.com</fct:column>
<fct:column>Technology News - CNET News</fct:column>
<fct:column><span class="srch_xerpt">Tech news and business reports by <b>CNET</b> News.</span></fct:column>
</fct:row>
<fct:row>
<fct:column datatype="trank">3</fct:column>
<fct:column datatype="erank">5.881291583872905e-014</fct:column>
<fct:column datatype="url" shortform="http://news.cnet.com/2547-1_3-0-20.xml">http://news.cnet.com/2547-1_3-0-20.xml</fct:column>
<fct:column>CNET News.com</fct:column>
<fct:column><span class="srch_xerpt">Tech news and business reports by <b>CNET</b> News.</span></fct:column>
</fct:row>
<fct:row>
<fct:column datatype="trank">3</fct:column>
<fct:column datatype="erank">5.881291583872905e-014</fct:column>
<fct:column datatype="url" shortform="http://news.cnet.com">http://news.cnet.com</fct:column>
<fct:column>Technology News - CNET News</fct:column>
<fct:column><span class="srch_xerpt">Tech news and business reports by <b>CNET</b> News.</span></fct:column>
</fct:row>
<fct:row>
<fct:column datatype="trank">3</fct:column>
<fct:column datatype="erank">5.881291583872905e-014</fct:column>
<fct:column datatype="url" shortform="http://news.com#6">http://news.com#6</fct:column>
<fct:column>There's an electric car in your future</fct:column>
<fct:column><span class="srch_xerpt">... <b>CNET</b> Car Tech posts photos of electric cars expected to come out by 2011.</span></fct:column>
</fct:row>
<fct:row>
<fct:column datatype="trank">3</fct:column>
<fct:column datatype="erank">5.881291583872905e-014</fct:column>
<fct:column datatype="url" shortform="http://news.cnet.com/2547-1_3-0-20.xml#9">http://news.cnet.com/2547-1_3-0-20.xml#9</fct:column>
<fct:column>There's an electric car in your future</fct:column>
<fct:column><span class="srch_xerpt">... <b>CNET</b> Car Tech posts photos of electric cars expected to come out by 2011.</span></fct:column>
</fct:row>
<fct:row>
<fct:column datatype="trank">3</fct:column>
<fct:column datatype="erank">5.881291583872905e-014</fct:column>
<fct:column datatype="url" shortform="http://news.cnet.com#9">http://news.cnet.com#9</fct:column>
<fct:column>There's an electric car in your future</fct:column>
<fct:column><span class="srch_xerpt">... <b>CNET</b> Car Tech posts photos of electric cars expected to come out by 2011.</span></fct:column>
</fct:row>
<fct:row>
<fct:column datatype="trank">3</fct:column>
<fct:column datatype="erank">5.881291583872905e-014</fct:column>
<fct:column datatype="url" shortform="http://news.com#6">http://news.com#6</fct:column>
<fct:column>There's an electric car in your future</fct:column>
<fct:column><span class="srch_xerpt">... <b>CNET</b> Car Tech posts photos of electric cars expected to come out by 2011.</span></fct:column>
</fct:row>
<fct:row>
<fct:column datatype="trank">3</fct:column>
<fct:column datatype="erank">5.881291583872905e-014</fct:column>
<fct:column datatype="url" shortform="http://news.cnet.com/2547-1_3-0-20.xml#9">http://news.cnet.com/2547-1_3-0-20.xml#9</fct:column>
<fct:column>There's an electric car in your future</fct:column>
<fct:column><span class="srch_xerpt">... <b>CNET</b> Car Tech posts photos of electric cars expected to come out by 2011.</span></fct:column>
</fct:row>
<fct:row>
<fct:column datatype="trank">3</fct:column>
<fct:column datatype="erank">5.881291583872905e-014</fct:column>
<fct:column datatype="url" shortform="http://news.cnet.com#9">http://news.cnet.com#9</fct:column>
<fct:column>There's an electric car in your future</fct:column>
<fct:column><span class="srch_xerpt">... <b>CNET</b> Car Tech posts photos of electric cars expected to come out by 2011.</span></fct:column>
</fct:row>
</fct:result>
</fct:facets>
|
| Figure: 16.6.1.5.4.1. Query Facet Browser Web service endpoint |
|
| Figure: 16.6.1.5.4.1. Select a URI from the list of available Labels |
|
| Figure: 16.6.1.5.4.1. Enter URI |
|
| Figure: 16.6.1.5.4.1. Obtain a description of the URI |
VT_INC_INDEX_DB_DBA_RDF_OBJ ()
|
| Figure: 16.6.1.5.5.1. Usage Statistics |
|
| Figure: 16.6.1.5.5.1. Usage Statistics |
dbpedia-owl:Artist
|
| Figure: 16.6.1.5.5.1. Usage Statistics |
Displaying types of e1 where: e1 has any property whose value contains "Michael Jackson". e1 is a dbpedia-owl:Artist . Drop
|
| Figure: 16.6.1.5.5.1. Usage Statistics |
Displaying values and text summaries associated with pattern e1 where: e1 has any property whose value contains "Michael Jackson". e1 is a dbpedia-owl:Artist . Drop
dbpedia:Michael_Jackson
|
| Figure: 16.6.1.5.5.1. Usage Statistics |
|
| Figure: 16.6.1.5.5.1. Usage Statistics |
SPARQL
SELECT ?g count (*)
where
{
graph ?g { <URI> ?p ?o }
}
group by ?g
order by desc 2
limit 20
|
| Figure: 16.6.1.5.5.1. Usage Statistics |
SPARQL
SELECT ?g count (*)
where
{
graph ?g { ?s ?p <URI> }
}
group by ?g
order by desc 2
limit 20
|
| Figure: 16.6.1.5.5.1. Usage Statistics |
SPARQL
SELECT ?syn ( sql:rnk_scale (<LONG::IRI_RANK> (?syn)))
where
{
{ SELECT ?s ?syn
where
{
{?syn owl:sameAs ?s } union {?s owl:sameAs ?syn}
}
}
option (transitive, t_distinct, t_min (0), T_in (?s), t_out (?syn)) . filter (!isliteral (?syn) && ?s = <URI> )
}
order by desc 2
limit 20
|
| Figure: 16.6.1.5.5.1. Usage Statistics |
SPARQL
SELECT distinct ?syn ?p ?o (sql:rnk_scale (<LONG::IRI_RANK> (?syn)))
where
{ <URI> ?p ?o . filter (0 != (<LONG::bif:rdf_is_sub> ("b3sifp", ?p, lod:ifp_like, 3))) .
?syn ?p ?o .
}
order by desc 4
limit 20
|
| Figure: 16.6.1.5.5.1. Usage Statistics |
The Virtuoso Facets web service is a general purpose RDF query facility for facet based browsing. It takes an XML description of the view desired and generates the reply as an XML tree containing the requested data. The user agent or a local web page can use XSLT for rendering this for the end user. The selection of facets and values is represented as an XML tree. The rationale for this is the fact that such a representation is easier to process in an application than the SPARQL source text or a parse tree of SPARQL and more compactly captures the specific subset of SPARQL needed for faceted browsing. The web service returns the SPARQL source text also, thus this can serve as a basis for and-crafted queries.
The top element of the tree is <query>, it must be in namespace "http://openlinksw.com/services/facets/1.0/".
This has the following attributes:
The result is a tree of the form:
<facets xmlns="http://openlinksw.com/services/facets/1.0/"> <result><row><column datatype="..." shortform="..." xml:lang="..">...</column></row></result> <time>msecs</time> <complete>yes or no</complete> <db-activity>resource use string</db-activity> <sparql>sparql statement text</sparql> </facets>
By convention, the first column is the subject selected by the view element, typically a URI, the second a label of the URI and the third, if present, is either a count or a search summary.
The first column's text child is the text form of the value. The column element has the following attributes qualifying this further:
The query has the top level element <query>. The child elements of this represent conditions pertaining to a single subject. A join is expressed with the property or property-of element. This has in turn children which state conditions on a property of the first subject. property and property-of elements can be nested to an arbitrary depth and many can occur inside one containing element. In this way, tree-shaped structures of joins can be expressed.
Expressing more complex relationships, such as intermediate grouping, subqueries, arithmetic or such requires writing the query in SPARQL. The XML format is a shorthand for easy automatic composition of queries needed for showing facets, not a replacement for SPARQL.
A facet query contains a single view element. This specifies which subject of the joined subjects is shown. Its attributes specify the manner of viewing, e.g. list of distinct values, distinct values with occurrence counts, properties or classes of the selected subjects etc.
The top query element or any property or property-of element can have the following types of children:
<text property="iri">text pattern</text>
The subject has an O that matches the text pattern. If property is given, the text pattern must occur in a value of this property. If not specified, any property will do. The value "none" for property is the same as not specifying a property. This is restricted to occurring directly under the top level query element.
<class iri="iri" inference="ctx_name" />
The S must be an instance of this class. If inference is specified then option (input:inference "ctx_name" is added and applies to this pattern alone.
<property iri="iri" same_as="yes" inference="ctx_name">
The child elements of this are conditions that apply to the value of this property of the S that is in scope in the enclosing <query> or <property> element. If same_as is present, then option (input:same-as "yes") is added to the triple pattern which specifies this property. If inference is present, then option (input:inference "ctx_name") is added to the triple pattern for the property.
<property-of iri="iri" same_as="yes" inference="ctx_name" >
The child elements of this are conditions that apply to an S which has property "iri" whose object is the S in scope in the enclosing <query> or <property> element. The options are otherwise the same as with property.
<value datatype="type" xml:lang="lng" op="= | < | > | >= | <=">value </value>
When this occurs inside <property> or <property-of> this means that the property in scope has the specified relation to the value. type and language can be used for XML typed or language tagged literals. The "uri" type means that the value is a qualified name of a URI. If this occurs directly under the <query> element, this means that the query starts with a fixed subject. If this is so, then there must be property or propertyof elements or the view element must specify properties or classes, list is not allowed as a view type. This is so because the query must have at least one triple pattern.
<view type="view" limit="n" offset="n" >
This may occur once inside a <query> element but may occur either at top level or inside property or property-of elements. This specifies what which subject is presented in the result set.
The type can be:
SPARQL
SELECT ?p count (*) { ?this_s ?p ?any_o ...}
GROUP BY ?p
ORDER BY DESC 2
LIMIT l OFFSET 0
SPARQL
SELECT ?p count (*) { ?any_s ?p ?this_s ... }
GROUP BY ?p
ORDER BY DESC 2
LIMIT L OFFSET 0
SPARQL
SELECT ?c count (*)
WHERE { ?xx a ?c ... }
GROUP BY ?c
ORDER BY DESC 2
LIMIT l OFFSET 0
SPARQL
SELECT DISTINCT ?s (bif:search_excerpt (sql:search_terms (""pattern"), ?o)) ...
LIMIT l OFFSET 0
SPARQL SELECT DISTINCT ?s long::sql:fct_label (?s) ... LIMIT l OFFSET 0
SPARQL SELECT ?s COUNT (*) .... GROUP BY ?s ORDER BY DESC 2
SPARQL SELECT (sql:subseq (?s, 0, 1)) count (*) ... GROUP BY (sql:subseq (?s, 0, 1)) ORDER BY 1
SPARQL SELECT DISTINCT ?lat ?long ?s WHERE ?s geo:lat ?lat . ?s geo:long ?long . ... }
SPARQL SELECT sql::year (?s) count (*) ... GROUP BY (bif:year (?s)) ORDER BY 1 OFFSET 0 LIMIT l
SPARQL SELECT sql::round_month (?s) count (*) ... GROUP BY (sql:round_month (?s)) ORDER BY 1 OFFSET 0 LIMIT l
SPARQL SELECT sql::round_week (?s) COUNT (*) ... GROUP BY (sql:round_week (?s)) ORDER BY 1 OFFSET 0 LIMIT l
SPARQL describe ?s ... OFFSET 0 LIMIT l
The following types of customization will be generally useful:
The source code is divided in two SQL files and a number of XSLT sheets. The file facet.sql has the code for the web service. The facet_view.sql file contains the procedures for the sample HTML interface.
Note: in all examples the default namespace xmlns="http://openlinksw.com/services/facets/1.0/" is omitted for brevity.
For people called Mike:
<query> <text>Mike</text> <view type="text"/> </query>
To open the list of people who Mike knows:
<query> <text>Mike</text> <view type="properties"/> </query>
To show the list of subjects Mike knows:
<query>
<text>Mike</text>
<property iri="foaf:knows>
<view type="list" />
</property>
</query>
To show the properties of people Mike knows:
<query>
<text>Mike</text>
<property iri="foaf:knows>
<view type="properties" />
</property>
</query>
To show the names:
<query>
<text>Mike</text>
<property iri="foaf:knows>
<property iri="foaf:name>
<view type="list" />
</property>
</property>
</query>
To specify one named Joe:
<query>
<text>Mike</text>
<property iri="foaf:knows>
<property iri="foaf:name>
<value>Joe</value>
</property>
<view type="properties" />
</property>
</query>
This lists the properties of the friends of Mike that are called Joe.
To show the Mikes that know a Joe, one would change the shown variable in the navigation and get:
<query>
<text>Mike</text>
<property iri="foaf:knows>
<property iri="foaf:name>
<value>Joe</value>
</property>
</property>
<view type="text" />
</query>
This would be the search summaries of subjects with Mike in some field that know a subject with name Joe.
Now to specify that Mike must be a member of a discussion board:
<query>
<text>Mike</text>
<property iri="foaf:knows>
<property iri="foaf:name>
<value>Joe</value>
</property>
</property>
<view type="property-in" />
</query>
This lists the properties of triples whom object is Mike. Pick sioc:member_of
<query>
<text>Mike</text>
<property iri="foaf:knows>
<property iri="foaf:name>
<value>Joe</value>
</property>
</property>
<property-of iri="sioc:member_of>
<view type="list" />
</property-of>
</query>
This would show things where Mike is a member. To specify that the thing must be a forum:
<query>
<text>Mike</text>
<property iri="foaf:knows>
<property iri="foaf:name>
<value>Joe</value>
</property>
</property>
<property-of iri="sioc:member_of>
<view type="classes" />
</property-of>
</query>
This shows classes of things where Mike is a member Clicking on sioc:Forum gives:
<query>
<text>Mike</text>
<property iri="foaf:knows>
<property iri="foaf:name>
<value>Joe</value>
</property>
</property>
<property-of iri="sioc:member_of>
<class iri="sioc:Forum" />
<view type="classes"/>
</property-of>
</query>
The view stays with classes, but now scoped to the classes of things where Mike is a member that are instances of sioc:Forum.
To go look at the list of Mikes with the added restriction, click the shown variable in the navigation and set it to s1.
<query>
<text>Mike</text>
<property iri="foaf:knows>
<property iri="foaf:name>
<value>Joe</value>
</property>
</property>
<property-of iri="sioc:member_of>
<class iri="sioc:Forum" />
</property-of>
<view type="list"/>
</query>
To say that Joe must also have a geekCode, One clicks the shown variable and sets it to s2 and the view to properties.
<query>
<text>Mike</text>
<property iri="foaf:knows>
<property iri="foaf:name>
<value>Joe</value>
</property>
<view type="properties"/>
</property>
<property-of iri="sioc:member_of>
<class iri="sioc:Forum" />
</property-of>
</query>
Pick geekCode
<query>
<text>Mike</text>
<property iri="foaf:knows>
<property iri="foaf:name>
<value>Joe</value>
</property>
<property iri="geekCode">
<view type="list"/>
</property>
</property>
<property-of iri="sioc:member_of>
<class iri="sioc:Forum" />
</property-of>
</query>
We specify no restriction on the geekCode. Click the shown variable to take the focus back to Mike.
<query>
<text>Mike</text>
<property iri="foaf:knows>
<property iri="foaf:name>
<value>Joe</value>
</property>
<property iri="geekCode"></property>
</property>
<property-of iri="sioc:member_of>
<class iri="sioc:Forum" />
</property-of>
<view type="text"/>
</query>
The Virtuoso Facets web service provide following REST interface:
Service description:
Error conditions:
The all error conditions are reported via 'Error explanation'
Files:
The facet_svc.sql contains web service code and virtual directory mapping, and it uses fct_req.xsl & fct_resp.xsl as request & response filters.
Example:
Using CURL program
curl -H "Content-Type: text/xml" -d @post.xml http://lod.openlinksw.com/fct/service
Where 'post.xml' document contains query document:
<?xml version="1.0"?> <query xmlns="http://openlinksw.com/services/facets/1.0" inference="" same-as=""> <text> Seattle Mariners traveled all the way to Japan to watch</text> <view type="text" limit="20" offset=""/> </query>
Produces following response:
<fct:facets xmlns:fct="http://openlinksw.com/services/facets/1.0/">
<fct:sparql> SELECT distinct ?s1 as ?c1, (bif:search_excerpt (bif:vector ('THE', 'MARINERS', 'WAY', 'SEATTLE', 'WATCH', 'ALL', 'TO', 'JAPAN', 'TRAVELED'), ?o1)) as ?c2 WHERE { ?s1 ?s1textp ?o1 . FILTER (bif:contains (?o1, '(THE AND MARINERS AND WAY AND SEATTLE AND WATCH AND ALL AND TO AND JAPAN AND TRAVELED)')) . } LIMIT 20 OFFSET 0 </fct:sparql>
<fct:time>116</fct:time>
<fct:complete>yes</fct:complete>
<fct:db-activity> 134R rnd 9.488KR seq 0P disk 8.966MB / 602 messages</fct:db-activity>
<fct:result>
<fct:row>
<fct:column datatype="url" shortform="http://bobdupuy.mlbl...ld_baseball__6.html">http://bobdupuy.mlblogs.com/bobdupuy/2006/03/world_baseball__6.html></fct:column>
<fct:column />
<fct:column><span class="srch_xerpt">... While Chuck Armstrong president of <b>the</b> <b>Seattle</b> <b>Mariners</b> <b>traveled</b> <b>all</b> <b>the</b> <b>way</b> <b>to</b> <b>Japan</b> <b>to</b> <b>watch</b> Ichiro... for <b>the</b> advancing <b>Japan</b> team last week <b>the</b> star from <b>the</b> <b>Seattle</b> roster so far in Round 1 has without a doubt... leading <b>the</b> Dominican <b>to</b> its...</span></fct:column>
</fct:row>
<fct:row>
<fct:column datatype="url" shortform="http://bobdupuy.mlbl...ld_baseball__6.html">http://bobdupuy.mlblogs.com/bobdupuy/2006/03/world_baseball__6.html></fct:column>
<fct:column />
<fct:column><span class="srch_xerpt">Orlando While Chuck Armstrong president of <b>the</b> <b>Seattle</b> <b>Mariners</b> <b>traveled</b> <b>all</b> <b>the</b> <b>way</b> <b>to</b> <b>Japan</b> <b>to</b> <b>watch</b>... perform for <b>the</b> advancing <b>Japan</b> team last week <b>the</b> star from <b>the</b> <b>Seattle</b> roster so far in Round 1 has without...</span></fct:column>
</fct:row>
</fct:result>
</fct:facets>
The Virtuoso APIs for FCT REST services are Virtuoso Stored Procedures that enable faceted browsing over Linked Data hosted in the RDF Quad Store. This also includes Linked Data that is progressively added to the Quad Store via URI de-referencing.
They enable the use Virtuoso's VSP/VSPX technology to produce (X)HTML-based Linked Data explorer pages that are endowed with high-performance (in-process) faceted browsing capability.
You can use this API with Virtuoso SQL calls that provide data to your VSP/VSPX, ASP.NET, PHP, etc., -based interfaces using ODBC, JDBC, ADO.NET, or XMLA connectivity (SPASQL) to Virtuoso.
API Definition
CREATE PROCEDURE
fct_exec
(
IN tree ANY ,
IN timeout INT
)
{
DECLARE start_time,
view3,
inx,
n_rows INT ;
DECLARE sqls,
msg,
qr,
qr2,
act,
query VARCHAR ;
DECLARE md,
res,
results,
more ANY ;
DECLARE tmp ANY ;
DECLARE offs,
lim INT ;
SET result_timeout = _min
(
timeout,
ATOI
(
registry_get ('fct_timeout_max')
)
)
;
offs := xpath_eval ('//query/view/@offset', tree);
lim := xpath_eval ('//query/view/@limit', tree);
-- db_activity ();
results := vector (null, null, null);
more := vector ();
IF
(
xpath_eval
(
'//query[@view3="yes"]//view[@type="text"]',
tree
)
IS NOT NULL
)
{
more := VECTOR ('classes', 'properties');
}
sqls := '00000';
qr := fct_query
(
xpath_eval ('//query', tree, 1)
)
;
query := qr;
-- dbg_obj_print (qr);
qr2 := fct_xml_wrap (tree, qr);
start_time := msec_time ();
dbg_printf('query: %s', qr2);
EXEC
(
qr2,
sqls,
msg,
vector (),
0,
md,
res
)
;
n_rows := row_count ();
act := db_activity ();
SET result_timeout = 0;
IF (
sqls <> '00000'
AND
sqls <> 'S1TAT'
)
SIGNAL (sqls, msg);
IF (
NOT ISARRAY (res)
OR
0 = length (res)
OR
NOT ISARRAY (res[0])
OR
0 = length (res[0])
)
results[0] := xtree_doc ('<result/>');
ELSE
results[0] := res[0][0];
inx := 1;
FOREACH (VARCHAR tp IN more) DO
{
tree := XMLUpdate (
tree,
'/query/view/@type',
tp,
'/query/view/@limit',
'40',
'/query/view/@offset',
'0'
)
;
qr := fct_query (xpath_eval ('//query', tree, 1));
qr2 := fct_xml_wrap (tree, qr);
sqls := '00000';
SET result_timeout = _min (
timeout,
ATOI
(
registry_get ('fct_timeout_max')
)
)
;
EXEC (
qr2,
sqls,
msg,
vector (),
0,
md,
res
);
n_rows := row_count ();
act := db_activity ();
SET result_timeout = 0;
IF ( sqls <> '00000'
AND
sqls <> 'S1TAT'
)
SIGNAL (sqls, msg);
IF (
ISARRAY (res)
AND
LENGTH (res)
AND
ISARRAY (res[0])
AND
LENGTH (res[0])
)
{
tmp := res[0][0];
tmp := XMLUpdate (tmp, '/result/@type', tp);
results[inx] := tmp;
}
inx := inx + 1;
}
res := XMLELEMENT
(
"facets",
XMLELEMENT
( "sparql", query ),
XMLELEMENT
( "time", msec_time () - start_time ),
XMLELEMENT
(
"complete",
CASE WHEN sqls = 'S1TAT'
THEN 'no'
ELSE 'yes'
END
),
XMLELEMENT
(
"timeout",
_min
(
timeout * 2,
ATOI
(
registry_get
( 'fct_timeout_max' )
)
)
),
XMLELEMENT
("db-activity", act),
XMLELEMENT
("processed", n_rows),
XMLELEMENT
(
"view",
XMLATTRIBUTES
(
offs AS "offset",
lim AS "limit"
)
),
results[0],
results[1],
results[2]
);
---- for debugging:
--string_to_file ('ret.xml', serialize_to_UTF8_xml (res), -2);
-- dbg_obj_print (res);
RETURN res;
}
;
The following example shows how to use the fct_exec APi in vsp page to perform a "text" search for the word "Mike" assuming this exists in your Virtuoso RDF store (if not amend the query in the fct_example.vsp code sample below to search for text known to exist).
<?vsp
declare txt, reply, tree any;
declare timeout int;
tree := xtree_doc ('
<query>
<text>Mike</text>
<view type="text"/>
</query>
');
timeout := 3000;
reply := fct_exec (tree, timeout);
txt := string_output ();
http_value (xslt ('virt://WS.WS.SYS_DAV_RES.RES_FULL_PATH.RES_CONTENT:/DAV/fct_example.xsl',
reply,
vector ()),
null, txt);
http (txt);
?>
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="html" encoding="ISO-8859-1"/>
<xsl:variable name="rowcnt" select="count(/facets/result/row)"/>
<xsl:template match="facets">
<div id="res">
<xsl:if test="/facets/complete = 'yes' and /facets/processed = 0 and $rowcnt = 0">
<div class="empty_result">
Nothing found.
</div>
</xsl:if>
<xsl:for-each select="/facets/result">
<xsl:call-template name="render-result"/>
</xsl:for-each>
</div>
<!-- #res -->
</xsl:template>
<xsl:template name="render-result">
<table class="result" border="1">
<thead>
<tr>
<th>Entity</th>
<th>Title</th>
<th>Text excerpt</th>
</tr>
</thead>
<tbody>
<xsl:for-each select="row">
<tr>
<td class="rnk">
<xsl:for-each select="column[@datatype='trank' or @datatype='erank']">
<xsl:choose>
<xsl:when test="./@datatype='trank'">Text Rank:</xsl:when>
<xsl:when test="./@datatype='erank'">Entity Rank:</xsl:when>
</xsl:choose>
<xsl:value-of select="."/>
<br/>
</xsl:for-each>
</td>
<xsl:for-each select="column">
<xsl:choose>
<xsl:when test="'url' = ./@datatype">
<td>
<a>
<xsl:attribute name="href">http://lod.openlinksw.com/describe/?url=<xsl:value-of select="urlify (.)"/></xsl:attribute>
<xsl:attribute name="title"><xsl:value-of select="."/></xsl:attribute>
<xsl:choose>
<xsl:when test="'' != ./@shortform">
<xsl:value-of select="./@shortform"/>
</xsl:when>
<xsl:when test="'erank' = ./@datatype or 'trank' = ./@datatype">rank</xsl:when>
<xsl:otherwise>
<xsl:value-of select="."/>
</xsl:otherwise>
</xsl:choose>
</a>
</td>
</xsl:when>
<xsl:when test="'erank' = ./@datatype or 'trank' = ./@datatype"/>
<xsl:when test="'srch_xerpt' = ./span/@class">
<td>
<xsl:value-of select="."/>
</td>
</xsl:when>
<xsl:otherwise/>
</xsl:choose>
</xsl:for-each>
</tr>
</xsl:for-each>
</tbody>
</table>
</xsl:template>
<xsl:template match="@* | node()">
<xsl:copy>
<xsl:apply-templates select="@* | node()"/>
</xsl:copy>
</xsl:template>
</xsl:stylesheet>
|
| Figure: 16.6.1.6.3.1. Facet API Example |
The facet web service is also available via SOAP protocol.
The request message contains single element 'query' with syntax explained earlier. Also the SOAPAction HTTP header should be '#query' . After successful evaluation of the query, the service will return a SOAP envelope containing in the Body element single 'facets' element described above.
Example:
This example shows execution of same command as in example for REST interface here it using SOAP:
Request message:
<SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP:Body>
<query xmlns="http://openlinksw.com/services/facets/1.0/" inference="" same-as="">
<text>Seattle Mariners traveled all the way to Japan to watch</text>
<view type="text" limit="20" offset="0"/>
</query>
</SOAP:Body>
</SOAP:Envelope>
Response message:
<SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP:Body>
<fct:facets xmlns:fct="http://openlinksw.com/services/facets/1.0/">
<fct:sparql>SELECT distinct ?s1 as ?c1, (bif:search_excerpt (bif:vector ('THE', 'MARINERS', 'WAY', 'SEATTLE', 'WATCH', 'ALL', 'TO', 'JAPAN', 'TRAVELED'), ?o1)) as ?c2 where { ?s1 ?s1textp ?o1 . filter (bif:contains (?o1, '(THE AND MARINERS AND WAY AND SEATTLE AND WATCH AND ALL AND TO AND JAPAN AND TRAVELED)')) . } LIMIT 20 OFFSET 0</fct:sparql>
<fct:time>114</fct:time>
<fct:complete>yes</fct:complete>
<fct:db-activity> 134R rnd 9.488KR seq 0P disk 8.966MB / 602 messages</fct:db-activity>
<fct:result>
<fct:row>
<fct:column datatype="url" shortform="http://bobdupuy.mlbl...ld_baseball__6.html">http://bobdupuy.mlblogs.com/bobdupuy/2006/03/world_baseball__6.html</fct:column>
<fct:column/>
<fct:column><span class="srch_xerpt">... While Chuck Armstrong president of <b>the</b> <b>Seattle</b> <b>Mariners</b> <b>traveled</b> <b>all</b> <b>the</b> <b>way</b> <b>to</b> <b>Japan</b> <b>to</b> <b>watch</b> Ichiro... for <b>the</b> advancing <b>Japan</b> team last week <b>the</b> star from <b>the</b> <b>Seattle</b> roster so far in Round 1 has without a doubt... leading <b>the</b> Dominican <b>to</b> its...</span></fct:column>
</fct:row>
<fct:row>
<fct:column datatype="url" shortform="http://bobdupuy.mlbl...ld_baseball__6.html">http://bobdupuy.mlblogs.com/bobdupuy/2006/03/world_baseball__6.html</fct:column>
<fct:column/>
<fct:column><span class="srch_xerpt">Orlando While Chuck Armstrong president of <b>the</b> <b>Seattle</b> <b>Mariners</b> <b>traveled</b> <b>all</b> <b>the</b> <b>way</b> <b>to</b> <b>Japan</b> <b>to</b> <b>watch</b>... perform for <b>the</b> advancing <b>Japan</b> team last week <b>the</b> star from <b>the</b> <b>Seattle</b> roster so far in Round 1 has without...</span></fct:column>
</fct:row>
</fct:result>
</fct:facets>
</SOAP:Body>
</SOAP:Envelope>
use DB;
CREATE TABLE DB.DBA.ProductFeature (
nr integer primary key,
label varchar(100) not null,
comment varchar(1500) not null,
publisher integer not null,
publishDate date not null
)
;
grant SELECT on DB.DBA.ProductFeature to public
;
CREATE TABLE DB.DBA.ProductType (
nr integer primary key,
label varchar(100) not null,
comment varchar(1500) not null,
parent integer,
publisher integer not null,
publishDate date not null
)
;
grant SELECT on DB.DBA.ProductType to public
;
CREATE TABLE DB.DBA.Producer (
nr integer primary key,
label varchar(100) not null,
comment varchar(1500) not null,
homepage varchar(100) not null,
country char(2) not null,
publisher integer not null,
publishDate date not null
)
;
grant SELECT on DB.DBA.Producer to public
;
create index producer_homepage on DB.DBA.Producer (homepage)
;
CREATE TABLE DB.DBA.Product (
nr integer primary key,
label varchar(100) not null,
comment varchar not null,
producer integer not null,
propertyNum1 integer,
propertyNum2 integer,
propertyNum3 integer,
propertyNum4 integer,
propertyNum5 integer,
propertyNum6 integer,
propertyTex1 varchar(200),
propertyTex2 varchar(200),
propertyTex3 varchar(200),
propertyTex4 varchar(200),
propertyTex5 varchar(200),
propertyTex6 varchar(200),
publisher integer not null,
publishDate date not null
)
;
grant SELECT on DB.DBA.Product to public
;
create index product_lbl on DB.DBA.Product (label)
;
create unique index product_producer_nr on DB.DBA.Product (producer, nr)
;
create index product_pn1 on DB.DBA.Product (propertyNum1)
;
create index product_pn2 on DB.DBA.Product (propertyNum2)
;
create index product_pn3 on DB.DBA.Product (propertyNum3)
;
create text index on DB.DBA.Product (label) with key nr
;
CREATE TABLE DB.DBA.ProductTypeProduct (
product integer not null,
productType integer not null,
PRIMARY KEY (product, productType)
)
;
grant SELECT on DB.DBA.ProductTypeProduct to public
;
create index ptype_inv on DB.DBA.ProductTypeProduct (productType, product)
;
CREATE TABLE DB.DBA.ProductFeatureProduct (
product integer not null,
productFeature integer not null,
PRIMARY KEY (product, productFeature)
)
;
grant SELECT on DB.DBA.ProductFeatureProduct to public
;
create index pfeature_inv on DB.DBA.ProductFeatureProduct (productFeature, product)
;
CREATE TABLE DB.DBA.Vendor (
nr integer primary key,
label varchar(100) not null,
comment varchar not null,
homepage varchar(100) not null,
country char(2) not null,
publisher integer not null,
publishDate date not null
)
;
grant SELECT on DB.DBA.Vendor to public
;
create index vendor_country on DB.DBA.Vendor (country)
;
create index vendor_homepage on DB.DBA.Vendor (homepage)
;
CREATE TABLE DB.DBA.Offer (
nr integer primary key,
product integer not null,
producer integer,
vendor integer not null,
price double precision not null,
validFrom date not null,
validTo date not null,
deliveryDays integer not null,
offerWebpage varchar(100) not null,
publisher integer not null,
publishDate date not null
)
;
grant SELECT on DB.DBA.Offer to public
;
create index offer_product on DB.DBA.Offer (product, deliveryDays)
;
create unique index offer_producer_product on DB.DBA.Offer (producer, product, nr)
;
create index offer_validto on DB.DBA.Offer (validTo)
;
create index offer_vendor_product on DB.DBA.Offer (vendor, product)
;
create index offer_webpage on DB.DBA.Offer (offerWebpage)
;
CREATE TABLE DB.DBA.Person (
nr integer primary key,
name varchar(30) not null,
mbox_sha1sum char(40) not null,
country char(2) not null,
publisher integer not null,
publishDate date not null
)
;
grant SELECT on DB.DBA.Person to public
;
CREATE TABLE DB.DBA.Review (
nr integer primary key,
product integer not null,
producer integer,
person integer not null,
reviewDate date not null,
title varchar(200) not null,
text long varchar not null,
textlang char(2) not null,
rating1 integer,
rating2 integer,
rating3 integer,
rating4 integer,
publisher integer not null,
publishDate date not null
)
;
grant SELECT on DB.DBA.Review to public
;
create unique index review_product on DB.DBA.Review (product, producer, nr)
;
create unique index review_producer_product on DB.DBA.Review (producer, product, nr)
;
create bitmap index review_textlang on DB.DBA.Review (textlang)
;
DB.DBA.XML_SET_NS_DECL ('foaf', 'http://xmlns.com/foaf/0.1/', 2)
;
DB.DBA.XML_SET_NS_DECL ('dc', 'http://purl.org/dc/elements/1.1/', 2)
;
DB.DBA.XML_SET_NS_DECL ('xsd', 'http://www.w3.org/2001/XMLSchema-datatypes/', 2)
;
DB.DBA.XML_SET_NS_DECL ('rev', 'http://purl.org/stuff/rev#', 2)
;
DB.DBA.XML_SET_NS_DECL ('bsbm', 'http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/', 2)
;
DB.DBA.XML_SET_NS_DECL ('bsbm-inst', 'http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/instances/', 2)
;
SPARQL drop quad map bsbm:SingleGraphView
;
SPARQL create iri class bsbm:ProductFeature-iri "http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/instances/ProductFeature%d" (in nr integer not null)
;
SPARQL create iri class bsbm:ProductType-iri "http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/instances/ProductType%d" (in nr integer not null)
;
SPARQL create iri class bsbm:Producer-iri "http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/instances/dataFromProducer%d/Producer%d" (in nr_ integer not null, in nr integer not null)
;
SPARQL create iri class bsbm:Product-iri "http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/instances/dataFromProducer%d/Product%d" (in producer integer not null, in nr integer not null)
;
SPARQL create iri class bsbm:Vendor-iri "http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/instances/dataFromVendor%d/Vendor%d" (in nr_ integer not null, in nr integer not null)
;
SPARQL create iri class bsbm:Offer-iri "http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/instances/dataFromVendor%d/Offer%d" (in vendor integer not null, in nr integer not null)
;
SPARQL create iri class bsbm:StdInst-iri "http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/instances/StandardizationInstitution%d" (in publisher integer not null)
;
SPARQL create iri class bsbm:Person-iri "http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/instances/dataFromRatingSite%d/Person%d" (in publisher integer not null, in nr integer not null)
;
SPARQL create iri class bsbm:Review-iri "http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/instances/dataFromRatingSite%d/Review%d" (in site integer, in nr integer not null)
;
SPARQL create iri class bsbm:ISO3166-country-iri "http://downlode.org/rdf/iso-3166/countries#%s" (in code varchar not null)
;
SPARQL create iri class bsbm:homepage-iri "%s" (in homepage varchar not null) option (returns "http://%s")
;
SPARQL create iri class bsbm:RatingSite-iri "http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/instances/dataFromRatingSite%d/RatingSite%d" (in nr_ integer not null, in nr integer not null)
;
SPARQL
alter quad storage virtrdf:DefaultQuadStorage
FROM DB.DBA.ProductFeature as pfeature
FROM DB.DBA.ProductType as ptype
FROM DB.DBA.Producer as producer
FROM DB.DBA.Product as product text literal product.label
FROM DB.DBA.ProductTypeProduct as ptypeproduct
FROM DB.DBA.ProductFeatureProduct as pfeatureproduct
FROM DB.DBA.Vendor as vendor
FROM DB.DBA.Offer as offer
FROM DB.DBA.Person as person
FROM DB.DBA.Review as review
where (^{product.}^.nr = ^{ptypeproduct.}^.product)
where (^{product.}^.nr = ^{pfeatureproduct.}^.product)
{
create bsbm:SingleGraphView as graph <BSBM> option (exclusive)
{
bsbm:Product-iri (product.producer, product.nr)
a bsbm:Product ;
rdfs:label product.label ;
rdfs:comment product.comment ;
bsbm:producer bsbm:Producer-iri (product.producer, product.producer) ;
bsbm:productPropertyTextual1 product.propertyTex1 ;
bsbm:productPropertyTextual2 product.propertyTex2 ;
bsbm:productPropertyTextual3 product.propertyTex3 ;
bsbm:productPropertyTextual4 product.propertyTex4 ;
bsbm:productPropertyTextual5 product.propertyTex5 ;
bsbm:productPropertyTextual6 product.propertyTex6 ;
bsbm:productPropertyNumeric1 product.propertyNum1 ;
bsbm:productPropertyNumeric2 product.propertyNum2 ;
bsbm:productPropertyNumeric3 product.propertyNum3 ;
bsbm:productPropertyNumeric4 product.propertyNum4 ;
bsbm:productPropertyNumeric5 product.propertyNum5 ;
bsbm:productPropertyNumeric6 product.propertyNum6 ;
rdf:type bsbm:ProductType-iri (ptypeproduct.productType) ;
bsbm:productFeature bsbm:ProductFeature-iri (pfeatureproduct.productFeature) ;
dc:publisher bsbm:Producer-iri (product.publisher, product.publisher) ;
dc:date product.publishDate .
bsbm:ProductType-iri (ptype.nr)
a bsbm:ProductType ;
rdfs:label ptype.label ;
rdfs:comment ptype.comment ;
rdfs:subClassOf bsbm:ProductType-iri (ptype.parent) ;
dc:publisher bsbm:StdInst-iri (ptype.publisher) ;
dc:date ptype.publishDate .
bsbm:ProductFeature-iri (pfeature.nr)
a bsbm:ProductFeature ;
rdfs:label pfeature.label ;
rdfs:comment pfeature.comment ;
dc:publisher bsbm:StdInst-iri (pfeature.publisher) ;
dc:date pfeature.publishDate .
bsbm:Producer-iri (producer.nr, producer.nr)
a bsbm:Producer ;
rdfs:label producer.label ;
rdfs:comment producer.comment ;
foaf:homepage bsbm:homepage-iri (producer.homepage) ;
bsbm:country bsbm:ISO3166-country-iri (producer.country) ;
dc:publisher bsbm:Producer-iri (producer.nr, producer.nr) ;
dc:date producer.publishDate .
bsbm:Vendor-iri (vendor.nr, vendor.nr)
a bsbm:Vendor ;
rdfs:label vendor.label ;
rdfs:comment vendor.comment ;
foaf:homepage bsbm:homepage-iri (vendor.homepage) ;
bsbm:country bsbm:ISO3166-country-iri (vendor.country) ;
dc:publisher bsbm:Vendor-iri (vendor.publisher, vendor.publisher) ;
dc:date vendor.publishDate .
bsbm:Offer-iri (offer.vendor, offer.nr)
a bsbm:Offer ;
bsbm:product bsbm:Product-iri (offer.producer, offer.product) ;
bsbm:vendor bsbm:Vendor-iri (offer.vendor, offer.vendor) ;
bsbm:vendor bsbm:Vendor-iri (offer.vendor, offer.vendor) ;
bsbm:price offer.price ;
bsbm:validFrom offer.validFrom ;
bsbm:validTo offer.validTo ;
bsbm:deliveryDays offer.deliveryDays ;
bsbm:offerWebpage bsbm:homepage-iri (offer.offerWebpage) ;
dc:publisher bsbm:Vendor-iri (offer.publisher, offer.publisher) ;
dc:date offer.publishDate .
bsbm:Person-iri (person.publisher, person.nr)
a foaf:Person ;
foaf:name person.name ;
foaf:mbox_sha1sum person.mbox_sha1sum ;
bsbm:country bsbm:ISO3166-country-iri (person.country) ;
dc:publisher bsbm:RatingSite-iri (person.publisher, person.publisher) ;
dc:date person.publishDate .
bsbm:Review-iri (review.publisher, review.nr)
a rev:Review ;
bsbm:reviewFor bsbm:Product-iri (review.producer, review.product) ;
bsbm:producer bsbm:Producer-iri (review.producer, review.producer) ;
rev:reviewer bsbm:Person-iri (review.publisher, review.person) ;
bsbm:reviewDate review.reviewDate ;
dc:title review.title ;
rev:text review.text lang review.textlang ;
bsbm:rating1 review.rating1 ;
bsbm:rating2 review.rating2 ;
bsbm:rating3 review.rating3 ;
bsbm:rating4 review.rating4 ;
dc:publisher bsbm:RatingSite-iri (review.publisher, review.publisher) ;
dc:date review.publishDate .
}
}
;
RDF Views map relational data into RDF and allow customizing RDF representation of locally stored RDF data. To let SPARQL clients access relational data as well as physical RDF graphs in a single query, we introduce a declarative Meta Schema Language for mapping SQL Data to RDF Ontologies. As a result, all types of clients can efficiently access all data stored on the server. The mapping functionality dynamically generates RDF Data Sets for popular ontologies such as SIOC, SKOS, FOAF, and ATOM/OWL without disruption to the existing database infrastructure of Web 1.0 or Web 2.0 solutions. RDF views are also suitable for declaring custom representation for RDF triples, e.g. property tables, where one row holds many single-valued properties.
The Virtuoso RDF Views meta schema is a built-in feature of Virtuoso's SPARQL to SQL translator. It recognizes triple patterns that refer to graphs for which an alternate representation is declared and translates these into SQL accordingly. The main purpose of this is evaluating SPARQL queries against existing relational databases. There exists previous work from many parties for rendering relational data as RDF and opening it to SPARQL access. We can mention D2RQ, SPASQL, Squirrel RDF, DBLP and others. The Virtuoso effort differs from these mainly in the following:
Since most of the data that is of likely use for the emerging semantic web is stored in relational databases, the argument for exposing this to SPARQL access is clear. We note that historically, SQL access to relational data has essentially never been given to the public outside of the organization. If programmatic access to corporate IS has been available to partners or the public, it has been through dynamic web pages or more recently web services. There are reasons of performance, security, maintainability and so forth for this.
The culture of the emerging semantic web is however taking a different turn. Since RDF and OWL offer a mergeable and queryable model for heterogeneous data, it is more meaningful and maintainable to expose selected data for outside query than it would be with SQL. Advances in hardware make this also less of a performance issue than it would have been in the client-server database era.
In the context of Virtuoso, since Virtuoso is originally a virtual/federated database, incorporating SPARQL to relational mapping is an evident extension of the product's mission as a multi-protocol, multi-platform connector between information systems.
In the simplest sense, any relational schema can be rendered into RDF by converting all primary keys and foreign keys into IRI's, assigning a predicate IRI to each column, and an rdf:type predicate for each row linking it to a RDF class IRI corresponding to the table. Then a triple with the primary key IRI as subject, the column IRI as predicate and the column's value as object is considered to exist for each column that is neither part of a primary or foreign key.
Strictly equating a subject value to a row and each column to a predicate is often good but is too restrictive for the general case.
Thus in the most common case the RDF meta schema should consist of independent transformations; the domain of each transformation is a result-set of some SQL SELECT statement and range is a set of triples. The SELECT that produce the domain is quite simple: it does not use aggregate functions, joins and sorting, only inner joins and WHERE conditions. There is no need to support outer joins in the RDF meta schema because NULLs are usually bad inputs for functions that produce IRIs. In the rare cases when NULLs are OK for functions, outer joins can be encapsulated in SQL views. The range of mapping can be described by a SPARQL triple pattern: a pattern field is a variable if it depends on table columns, otherwise it is a constant. Values of variables in the pattern may have additional restrictions on datatypes, when datatypes of columns are known.
This common case of an RDF meta schema is implemented in Virtuoso, with one adjustment. Virtuoso stores quads, not triples, using the graph field (G) to indicate that a triple belongs to some particular application or resource. A SPARQL query may use quads from different graphs without large difference between G and the other three fields of a quad. E.g., variable ?g in expression GRAPH ?g {...} can be unbound. SPARQL has special syntax for "graph group patterns" that is convenient for sets of triple patterns with a common graph, but it also has shorthands for common subject and predicate, so the difference is no more than in syntax. There is only one feature that is specific for graphs but not for other fields: the SPARQL compiler can create restrictions on graphs according to FROM and FROM NAMED clauses.
Virtuoso RDF Views should offer the same flexibility with the graphs as SPARQL addressing physical triples. A transformation cannot always be identified by the graph used for ranges because graph may be composed from SQL data. The key element of the meta schema is a "quad map pattern". A simple quad map pattern fully defines one particular transformation from one set of relational columns into triples that match one SPARQL graph pattern. The main part of quad map pattern is four declarations of "quad map values", each declaration specifies how to calculate the value of the corresponding triple field from the SQL data. The pattern also lists boolean SQL expressions that should be used to filter out unwanted rows of source data (and to join multiple tables if source columns belong to different tables). There are also quad map patterns that group together similar quad patterns but do not specify any real transformation or even prevent unwanted transformations from being used, they are described in "Grouping Map Patterns" below.
Quad map values refer to schema elements of two further types: "IRI classes" and "literal classes".
An IRI class declares that a column or set of columns gets converted into a IRI in a certain way. The conversion of this sort can be declared revertible (bijection) so an IRI can be parsed into original SQL values; this is useful when some equality of an IRI constant and a calculated IRI can be replaced with an equality of a parse result of a constant and an SQL column that is index criteria or simply faster. In addition, the SPARQL optimizer will eliminate redundant conversions if one IRI class is explicitly declared as a subclass of another. The most flexible declaration for conversion consists of specifying functions that assemble and disassemble from IRI into its constituent parts. This is overkill for typical conversions so it is possible to specify only one sprintf-style format string such that sprintf() SQL function will print an IRI using this format and sprintf_inverse() will be able to parse it back.
The use of sprintf_inverse() assumes that the format does not contain fragments like '%s%s' that make it impossible to separate parts of IRI from each other.
In the following, we shall map the Virtuoso users and user roles system tables into the SIOC ontology.
create iri class oplsioc:user_iri "http://myhost/sys/user?id=%d" (in uid integer not null) . create iri class oplsioc:group_iri "http://myhost/sys/group?id=%d" (in gid integer not null) . create iri class oplsioc:membership_iri "http://myhost/sys/membership?super=%d&sub=%d" (in super integer not null, in sub integer not null) . create iri class oplsioc:dav_iri "http://myhost%s" (in path varchar) .
These IRI classes are used for mapping data from the DB.DBA.SYS_USERS and DB.DBA.SYS_ROLE_GRANTS system tables that are defined in Virtuoso as follows:
create table DB.DBA.SYS_USERS ( U_ID integer not null unique, U_NAME char (128) not null primary key, U_IS_ROLE integer default 0, U_FULL_NAME char (128), U_E_MAIL char (128) default ", U_ACCOUNT_DISABLED integer default 1, U_DAV_ENABLE integer default 0, U_SQL_ENABLE integer default 1, U_HOME varchar (128), . . . );
Single record in DB.DBA.SYS_USERS corresponds to a plain user or a group (role). Users and roles are collectively named "grantees". Thus a role may be granted to another role or to a user account. A role grant may be direct (explicit) or assigned by recursion.
create table SYS_ROLE_GRANTS ( GI_SUPER integer, GI_SUB integer, GI_DIRECT integer default 1, . . . primary key (GI_SUPER, GI_SUB, GI_DIRECT));
One IRI class usually corresponds to one ontology class, because similar things are usually called similarly. One may wish to use identifiers of ontology classes as identifiers of related IRI classes, to not remember double number of names, e.g. create IRI class mybank:XpressXfer for subjects that will have rdf:type property mybank:XpressXfer made by mapping. That is technically possible but proven to become inconvenient and misleading as application evolves. While RDF types tend to persist, IRI classes may change over time or same subject may get more than one name via more than one IRI class, say, for exports to different systems. It is found to be more convenient to compose names of IRI classes by adding some common prefixes or suffixes to RDF classes (or to table names), say, write create IRI class mybank:XpressXfer_iri.
A "literal class" declares that a column or set of columns gets converted into a literal instead of an IRI. More precisely, the result of conversion can be IRI_ID so it represents an IRI, but in current version of Virtuoso this is supported only for some internal built-in literal classes, not for classes declared by the user. So for user-defined literal class the result of the conversion is an RDF literal even if it is a string representation of a valid IRI.
In any case, a literal class can be used only in quad map values of O fields, because Virtuoso does not support literal values as subjects.
A special case of literal class is the identity class that converts a value from varchar column into an untyped literal and value from column of any other SQL datatype into a typed literal with type from XMLSchema set, i.e. xsd:integer, xsd:dateTime and so on. Columns of types ANY and IRI_ID are not supported.
The SPARQL optimizer knows that RDF literal types are pairwise disjoint so literal classes that produce literals of different types are known to be pairwise disjoint. The optimizer will replace a join on two disjoint literal classes with an empty statement, to simplify the resulting query.
The following declaration of quad map pattern is self-explanatory. The line for object uses identity literal class so there's no need to specify its name.
graph <http://myhost/sys> subject oplsioc:user_iri (DB.DBA.SYS_USERS.U_ID) predicate foaf:email object DB.DBA.SYS_USERS.U_E_MAIL
The description language also supports SPARQL-style notation that contains less keywords and eliminates duplicate graphs, subjects and predicates. The following add two patterns with constant graph IRI <http://myhost/sys> and subjects are made from column DB.DBA.SYS_USERS.U_ID by oplsioc:user_iri.
graph <http://myhost/sys>
{
oplsioc:user_iri (DB.DBA.SYS_USERS.U_ID)
a sioc:user ;
oplsioc:name DB.DBA.SYS_USERS.U_FULL_NAME .
}
In real applications, quad map patterns should be named, for schema manipulation and keeping debug info readable. Thus it is much better to rewrite the previous example as
create virtrdf:SysUsers as graph <http://myhost/sys>
{
oplsioc:user_iri (DB.DBA.SYS_USERS.U_ID)
a sioc:user
as virtrdf:SysUserType-User;
oplsioc:name DB.DBA.SYS_USERS.U_FULL_NAME
as virtrdf:SysUsersFullName .
}
Using these names, one may later write, say, drop quad map virtrdf:SysUserType-User.
One name, virtrdf:DefaultQuadMap is reserved. It is an internal quad map pattern used to access "native-form" quads from DB.DBA.RDF_QUAD:
create virtrdf:DefaultQuadMap as graph rdfdf:default-iid-nonblank (DB.DBA.RDF_QUAD.G) subject rdfdf:default-iid (DB.DBA.RDF_QUAD.S) predicate rdfdf:default-iid-nonblank (DB.DBA.RDF_QUAD.P) object rdfdf:default (DB.DBA.RDF_QUAD.O)
IRI classes from rdfdf:... namespace are also reserved.
The previous example actually contains three map patterns, not two. The name virtrdf:SysUsers refers to a "group map pattern" that does not define any real transformation of relational data into RDF but helps organize quad map patterns into a tree. Group may contain both quad map patterns and other groups. A group can be manipulated as a whole, e.g. drop quad map virtrdf:SysUsers will remove all three map patterns.
"Quad Storage" is a named set of quad map patterns. The declaration define input:storage storage-name states that a SPARQL query will be executed using only quad patterns of the given quad storage. Declarations of IRI classes, literal classes and quad patterns are shared between all quad storages of an RDF meta schema but every quad storage contains only a subset of all available quad patterns. Two quad storages are always defined:
Three statements for manipulating storages are
A map pattern can be created only as a part of create quad storage or alter quad storage statement, so initially it is used by exactly one storage. It can be imported to some other storage using directive create map-id using storage source-storage. E.g., declarations of many storages create virtrdf:DefaultQuadMap using storage virtrdf:DefaultQuadStorage.
Only a "top-level" quad map pattern (standalone or a whole group with descendants) can be imported, member of a group can not. The import directive also can not be a part of some group declaration.
The directive drop quad map map-name removes a map from one storage when it appears inside alter quad storage statement. Otherwise it removes the map from all storages. There exists garbage collection for quad map patterns, so any unused map is immediately deleted. A group is deleted with all its descendants.
When a SPARQL query is compiled into SQL using a quad storage, every triple pattern should become a subquery that retrieves data from relational tables. This subquery is an UNION ALL of joins generated from appropriate quad map patterns. The complete SQL query is composed from these basic subqueries. Thus the first operation of the SQL generation for a triple pattern is searching for quad map patterns that may in principle produce triples that match the triple pattern.
The more restrictions contained in the triple pattern the fewer quad map patterns will be used. A triple pattern graph ?g { ?s ?p ?o } is common enough to invoke all data transformations of the storage. A triple pattern graph <g> { ?s <p> <o> } will usually intersect with the range of only one quad map. Sometimes it is possible to prove that the storage can not contain any data that matches the given triple pattern, hence zero number of members of UNION ALL will result in constantly empty result-set.
The search for quad maps for a given pair of triple pattern and quad map storage is quite simple. The storage is treated as a tree of map patterns where quad map patterns are leafs, grouping patterns are inner nodes and the whole storage is also treated as a grouping pattern that specify no fields and contains all top-level map patterns of the storage.
The tree is traversed from the root, left to right, non-leaf vertex are checked before their children. The check of a vertex consists of up to four field checks, for G, S, P and O. Every field check compares the field definition in the vertex and the corresponding field in the triple pattern, G and G, S and S and so on. Note that a non-leaf vertex defines less than four of its fields, e.g., the root vertex does not define any of its fields and top-level graph map { ... } defines only graph. Checks are performed only for defined fields and return one of three values: "failed", "passed", "full match", according to the following rules:
| Field of vertex | Field in triple pattern | Result |
|---|---|---|
| constant | same constant | full match |
| constant | different constant | failed |
| constant | variable of same type | passed |
| constant | variable of different type | failed |
| quad map value | constant of same type | full match |
| quad map value | constant of different type | failed |
| quad map value of type X | variable, X or subtype of X | full match |
| quad map value of type X | variable, supertype of X | passed |
| quad map value of type X | variable, type does not intersect with X | failed |
If any of the checks fails, the vertex and all its children are excluded from the rest of processing. Otherwise, if all four fields are defined for the quad map pattern, the map is added to the list of matching map patterns. The difference between "passed" and "full match" is significant only if the map is declared with option (exclusive) If all performed checks return "full match" and option (exclusive) is set then the traverse of the tree is stopped as soon as all children of the vertex are traversed. The most typical use of this option is when the application developer is sure that all triples of a graph belong to his application and they come from his own quad map patterns, not from DB.DBA.RDF_QUAD. This is to prevent the SPARQL compiler from generating redundant subqueries accessing DB.DBA.RDF_QUAD. The declaration may look like
create quad storage <mystorage>
{
graph <mygraph> option (exclusive) { . . . }
create virtrdf:DefaultQuadMap
using storage virtrdf:DefaultQuadStorage .
}
Exclusive patterns make the order of declarations important, because an exclusive declaration may "throw a shadow" on declarations after it. Consider a database that have a special table RDF_TYPE that caches all RDF types of all subjects in all graphs. Consider two declarations: all triples from graph <http://myhost/sys> and all triples with rdf:type predicate, both exclusive:
graph <http://myhost/sys> option (exclusive)
{
. . . # mapping of DB.DBA.SYS_USERS as in previous examples.
}
graph rdfdf:default-iid-nonblank (DB.DBA.RDF_TYPE.G)
subject rdfdf:default-iid (DB.DBA.RDF_TYPE.S)
predicate rdf:type
object rdfdf:default (DB.DBA.RDF_TYPE.O)
option (exclusive)
The order of these declarations dictates that triple pattern
graph <http://myhost/sys> {?s rdf:type ?o}
is compiled using only quad map patterns of the graph declaration, ignoring second declaration (and of course ignoring default mapping rule, if any). An explicit option (order N) at the end of quad map pattern will tweak the priority. By default, order will grow from 1000 for the first declaration in the statement to 1999 for the last, explicit configuration is especially useful to make order persistent to alter storage statements.
The option (exclusive) trick is ugly, low-level and prone to cause compilation errors after altering storage declarations. When misused, it is as bad as "red cut" in PROLOG, but one must use this trick to build scalable storages.
The option (exclusive) helps the SPARQL compiler to prepare better SQL queries, but sometimes it is "too exclusive". For instance, if a grouping quad map pattern specify only quad map value for graph and no other fields then making it exclusive prohibits the use of all declarations of the storage after that one. Sometimes it is better to notify compiler that quads made by the given quad map pattern are supposed to be different from all quads made by declarations listed after the given one.
Consider an application that exports users' personal data as graphs whose IRIs looks like http://www.example.com/DAV/home/username/RDF/personal/; the application makes a query and a triple pattern is proven to be restrictive enough to filter out all quads that are not similar to quads generated by the given quad map pattern (say, the graph is constant http://www.example.com/DAV/home/JohnSmith/RDF/personal/). The application do not hope to find any quads that match the pattern but made by other applications, because graphs named like in the pattern are supposed to be solely for this single purpose; if, say, DB.DBA.RDF_QUAD occasionally contains some quads with graph equal to http://www.example.com/DAV/home/JohnSmith/RDF/personal/ then they can be ignored.
Under this circumstances, the quad map pattern may have option (soft exclusive). That grants a permission to the compiler to ignore rest of storage as soon as it is proven that the triple pattern can not access quads that does not match the pattern. So if that is proven then the pattern is exclusive and it makes the query faster; when unsure, the compiler work like there is no option at all.
The option (exclusive) can be used as a security measure, option (soft exclusive) can not. Say, if an financial application exports its data as a single graph http://www.example.com/front-office/cash/ using exclusive then the query that explicitly refers to that graph will never access any quads written by the attacker into DB.DBA.RDF_QUAD using same graph IRI. The use of soft exclusive gives no such protection. From the compiler's perspective, the option (soft exclusive) is a hint that may be ignored, not an unambiguous order.
There is one exception from the rules described above. This exception is for virtrdf:DefaultQuadStorage only. If a graph variable of a quad map pattern is not bound and no source graph specified by FROM clauses then quad maps for specific constant graphs are ignored. In other words, if a default quad storage contains quad maps for specific graphs then the query in that storage should explicitly specify the graph in order to use a map for graph. This rule will not work if the default quad map is removed from the virtrdf:DefaultQuadStorage. This rule relates to the default storage itself, not to the containing patterns; copying some or all patterns into other storage will not reproduce there this special effect.
Quad map patterns of an application usually share a common set of source tables and quad map values of one pattern usually share either a single table or very small number of joined tables. Join and filtering conditions are also usually repeated in different patterns. It is not necessary to type table descriptions multiple times, they are declare once in the beginning of storage declaration statement and shared between all quad map declarations inside the statement. Names of aliases can be used instead of table names in quad map values.
FROM DB.DBA.SYS_USERS as user WHERE (^{user.}^.U_IS_ROLE = 0)
FROM DB.DBA.SYS_USERS as group WHERE (^{group.}^.U_IS_ROLE = 1)
FROM DB.DBA.SYS_USERS as account
FROM user as active_user
WHERE (^{active_user.}^.U_ACCOUNT_DISABLED = 0)
FROM DB.DBA.SYS_ROLE_GRANTS as grant
WHERE (^{grant.}^.GI_SUPER = ^{account.}^.U_ID)
WHERE (^{grant.}^.GI_SUB = ^{group.}^.U_ID)
WHERE (^{grant.}^.GI_SUPER = ^{user.}^.U_ID)
This declares five distinct aliases for two distinct tables, and six filtering conditions. Every condition is an SQL expression with placeholders where a reference to the table should be printed. The SPARQL compiler will not try to parse texts of these expressions (except dummy search for placeholders), so any logical expressions are acceptable. When a quad map pattern declaration refers to some aliases, the WHERE clause of the generated SQL code will contain a conjunction of all distinct texts of "relevant" conditions. A condition is relevant if every alias inside the condition is used in some quad map value of the map pattern, either directly or via clause like from user as active_user. (user is a "base alias" for active_user).
Consider a group of four declarations.
graph <http://myhost/sys>
{
oplsioc:user_iri (active_user.U_ID)
a oplsioc:active-user .
oplsioc:membership_iri (grant.GI_SUPER, grant.GI_SUB).
oplsioc:is_direct
grant.GI_DIRECT ;
oplsioc:member-e-mail
active_user.U_E_MAIL
where (^{active_user.}^.U_E_MAIL like 'mailto:%').
ldap:account-ref (account.U_NAME)
ldap:belongs-to
ldap:account-ref (group.U_NAME) option (using grant).
}
The first declaration will extend <http://myhost/sys> graph with one imaginary triples { user a oplsioc:active-user } for every account record that is not a role and not disabled. The second declaration deals with membership records. A membership is a pair of a grantee ("super") and a granted role ("sub") stored as a row in DB.DBA.SYS_ROLE_GRANTS).
The second declaration states that every membership has oplsioc:is_direct property with value from GI_DIRECT column of that table (roles may be granted to other roles and users, so permissions are "direct" or "recursive").
The third declaration declares oplsioc:member-e-mail property of memberships. The value is a literal string from DB.DBA.SYS_USERS.U_E_MAIL, if the grantee is active (not disabled) and is not a role and its e-mail address starts with 'mailto:'. The join between DB.DBA.SYS_ROLE_GRANTS and DB.DBA.SYS_USERS is made by equality (GI_SUPER = U_ID) because the alias active_user in the declaration "inherits" all conditions specified for user. In addition, the SPARQL compiler will add one more condition to check if the U_E_MAIL is not null because the NULL value is not a valid object and it knows that U_E_MAIL is not declared as NOT NULL.
The last declaration contains an option clause. As usual, this indicates that the basic functionality is good for many tasks but not for all. In this declaration, the ldap:belongs-to property establishes a relation between grantee (subject) and a granted role (object). Both subject and object IRIs are based on account name, DB.DBA.SYS_USERS.U_NAME, so the quad map pattern contains two references to different aliases of DB.DBA.SYS_USERS but no alias for DB.DBA.SYS_ROLE_GRANTS. Hence the declaration could produce a triple for every row of the Cartesian product of the DB.DBA.SYS_USERS. To fix the problem, option (using alias-name) tells the compiler to process the alias-name as if it's used in some quad map value of the pattern.
It is an error to use an alias only in where clause of the quad map pattern but neither in values or in option (using alias-name). To detect more typos, an alias used in quad map values can not appear in option (using alias-name) clause.
Most of IRI classes can be declared by a sprintf format string, but sophisticated cases may require calculations, not only printing the string. create IRI class using function allows the application transform relational values to IRIs by any custom routines.
Let us extend the previous example about users and groups by a new class for grantees. Both users and groups are grantees and we have defined two IRI classes for them. Classes oplsioc:user_iri and oplsioc:group_iri work fine for quad maps of U_ID if and only if the value of U_IS_ROLE is accordingly restricted to FALSE or TRUE, otherwise one may occasionally generate, say, user IRI for a group. To create and parse IRIs that correspond to any U_IDs, two functions should be created:
create function DB.DBA.GRANTEE_URI (in id integer)
returns varchar
{
declare isrole integer;
isrole := coalesce ((SELECT top 1 U_IS_ROLE
FROM DB.DBA.SYS_USERS WHERE U_ID = id ) );
if (isrole is null)
return NULL;
else if (isrole)
return sprintf ('http://%s/sys/group?id=%d', id);
else
return sprintf ('http://%s/sys/user?id=%d', id);
};
create function DB.DBA.GRANTEE_URI_INVERSE (in id_iri varchar)
returns integer
{
declare parts any;
parts := sprintf_inverse (id_iri,
'http://myhost/sys/user?id=%d', 1 );
if (parts is not null)
{
if (exists (SELECT top 1 1 FROM DB.DBA.SYS_USERS
WHERE U_ID = parts[0] and not U_IS_ROLE ) )
return parts[0];
}
parts := sprintf_inverse (id_iri,
'http://myhost/sys/group?id=%d', 1 );
if (parts is not null)
{
if (exists (SELECT top 1 1 FROM DB.DBA.SYS_USERS
WHERE U_ID = parts[0] and U_IS_ROLE ) )
return parts[0];
}
return NULL;
};
These functions may be more useful if the SPARQL web service endpoint is allowed to use them:
grant execute on DB.DBA.GRANTEE_URI to "SPARQL"; grant execute on DB.DBA.GRANTEE_URI_INVERSE to "SPARQL";
The next declaration creates an IRI class based on these two functions:
create iri class oplsioc:grantee_iri using
function DB.DBA.GRANTEE_URI (in id integer)
returns varchar,
function DB.DBA.GRANTEE_URI_INVERSE (in id_iri varchar)
returns integer .
In common case, IRI class declaration contains an N-array function that composes IRIs and N inverse functions that gets an IRI as an argument and extracts the Nth SQL value. IRI composing function should silently return NULL on incorrect arguments instead of error signal. Inverse functions should return NULL if the argument has an incorrect type or value.
It is possible to specify only composing function without any of inverse functions. However option (bijection) can not be used in that case, obviously.
Writing function-based IRI class is overkill when the IRI can in principle be made by a sprintf_iri but the format should contain some context-specific data, such as host name used for the dynamic renaming of local IRIs. Format strings offer a special syntax for that cases. %{varname}U acts as %U but the function sprintf will take the value from client connection variable varname, not from list of arguments. Similarly, sprintf_inverse will not return fragment that match to %{varname}U in the vector of other fragments; instead it will get the value from connection environment and ensure that it matches the fragment of input; mismatch between printed and actual value of variable will means that the whole string do not match the format.
SPARQL optimizer knows about this formatting feature and sometimes it makes more deductions from occurrence of %{varname}U than from occurrence of plain %U, so this notation may be used in option ( returns ...) when appropriate. Of course, the optimizer has no access to the actual value of connection variable because it may vary from run to run or may change between the compilation and the run, but the value is supposed to be persistent during any single query run so %{myvariable}U in one place is equal to %{myvariable}U in other.
Connection variables are set by connection_set and some of them have default values that are used if not overridden by application:
It is inconvenient to write different format strings for different cases. Two most common policies are different host names for default HTTP port of a publicly available service and different non-default ports for one or more host names of an intranet installation; these two approaches are almost never used in a mix. So declaration of IRI classes may use shorthand ^{DynamicLocalFormat}^ in format strings that is expanded either to http://%{WSHost}U or to http://%{WSHostName}U:%{WSHostPort}U/..., depending on absence or presence of port number in the value of DefaultHost parameter of URIQA section of configuration file.
^{DynamicLocalFormat}^ is for IRI class declarations only and is not expanded in any other place, so it is useful sometimes to create an IRI class with empty argument list in order to get "almost constant" IRIs calculated without writing special procedures.
There is one subtle problem with IRI class declarations. To get benefit from a relational index, SPARQL optimizer should compose equality between table column and some known SQL value, not between return value of IRI class and a known composed IRI. In addition, redundant calculations of IRIs takes time. To enable this optimization, an IRI class declaration should end with option (bijection) clause. For some simple format strings the compiler may recognize the bijection automatically but an explicit declaration is always a good idea.
See also: Wikipedia - Bijection. In mathematics, a bijection, or a bijective function is a function f from a set X to a set Y such that, for every y in Y, there is exactly one x in X such that f(x) = y.
Alternatively, f is bijective if it is a one-to-one correspondence between those sets; i.e., both one-to-one (injective) and onto (surjective).
The SPARQL compiler may produce big amounts of SQL code when the query contains equality of two calculated IRIs and these IRIs may come from many different IRI classes. It is possible to provide hints that will let the compiler check if two IRI classes form disjoint sets of possible IRI values. The more disjoint sets are found the less possible combinations remain so the resulting SQL query will contain fewer unions of joins. The SPARQL compiler can prove some properties of sprintf format strings. E.g., it can prove that set of all strings printed by "http://example.com/item%d" and the set of strings printed by "http://example.com/item%d/" are disjoint. It can prove some more complicated statements about unions and intersections of sets of strings. The IRI or literal class declaration may contain option (returns ...) clause that will specify one or more sprintf patterns that cover the set of generated values. Consider a better version of IRI class declaration listed above:
create iri class oplsioc:grantee_iri using
function DB.DBA.GRANTEE_URI (in id integer)
returns varchar,
function DB.DBA.GRANTEE_URI_INVERSE (in id_iri varchar)
returns integer
option ( bijection,
returns "http://myhost/sys/group?id=%d"
union "http://myhost/sys/user?id=%d" ) .
It is very important to keep IRI classes easily distinguishable by the text of IRI string and easy to parse.
In some cases option (returns ...) can be used for IRI classes that are declared using sprintf format, but actual data have more specific format. Consider a literal class declaration that is used to output strings and the application knows that all these strings are ISBN numbers:
create literal class example:isbn_ref "%s" (in isbn varchar not null) option ( bijection, returns "%u-%u-%u-%u" union "%u-%u-%u-X" )
Sometimes interoperability restrictions will force you to violate these rules but please try to follow them as often as possible.
Additional problem appears when the equality is between two IRIs of two different IRI classes. Even if both of them are bijections, the compiler does not know if these IRI classes behave identically on the intersection of their domains. To let the optimizer know this fact, one IRI class can be explicitly declared as a subclass of another:
make oplsioc:user_iri subclass of oplsioc:grantee_iri . make oplsioc:group_iri subclass of oplsioc:grantee_iri .
The SPARQL compiler can not check the validity of a subclass declaration. The developer should carefully test functions to ensure that transformations are really subclasses, as well as to ensure that functions of an IRI class declarations are really inverse to each other.
When declaring that a table's primary key is converted into a IRI according to one IRI class, one usually declares that all foreign keys referring to this class also get converted into an IRI as per this same class, or subclass of same class.
Subclasses can be declared for literal classes as well as for IRI classes, but this case is rare. The reason is that most of literals are made by identity literal classes that are disjoint to each other even if values may be equal in SQL sense, such as "2" of type xsd:integer and "2.0" of type xsd:double.
This section refers to checking and backing up RDF view and storage declarations only. The checks and backup/restore do not affect physical quads, relational schema or tables or data therein. For general backup and restore, see server administration. To detect and fix automatically most popular sorts of RDF metadata corruption use DB.DBA.RDF_AUDIT_METADATA. It is also possible to backup RDF data by DB.DBA.RDF_BACKUP_METADATA and restore the saved state later by using DB.DBA.RDF_RESTORE_METADATA. It is convenient to make a backup before any modification of quad storages, quad map patterns or IRI classes, especially during debugging new RDF Views.
In SQL, adding a new view can not break anything. This is because SQL lacks the ability of querying "everything" so data sources are always specified. This is not true for SPARQL, so please treat any metadata manipulation as potentially destructive operation. If an RDF storage is supposed to be used by more than one application then these applications should be tested together, not one after other, and they should be installed/upgraded on live database in the very same order as they were installed/upgraded on instrumental machine during testing. Always remember that these applications share RDF tables so they may interfere.
RDF View can be created by two or more "sparql alter storage" statements. In each statement can be created one quad map that contains mappings for half or a third of all tables. Quad maps created should have distinct names but may mention same graph. The important fact is that if the RDF View in question is exclusive for a graph then only the last quad map should be exclusive but all previous should not have this option. This is because if a map is exclusive on a graph the rest of maps on that graph will be silently ignored.
The example below shows a sample part of the Virtuoso eCRM Views code, where the RDF view is split in two parts: with quad map virtrdf:ecrmDemo1 and with quad map virtrdf:ecrmDemo2:
SPARQL
prefix ecrm: <http://demo.openlinksw.com/schemas/ecrm#>
prefix oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix cal: <http://www.w3.org/2002/12/cal/ical#>
prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
prefix product: <http://www.swop-project.eu/ontologies/pmo/product.owl#>
prefix owl: <http://www.w3.org/2002/07/owl#>
drop quad map virtrdf:ecrmDemo1 .
;
SPARQL
prefix ecrm: <http://demo.openlinksw.com/schemas/ecrm#>
prefix oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix cal: <http://www.w3.org/2002/12/cal/ical#>
prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
prefix product: <http://www.swop-project.eu/ontologies/pmo/product.owl#>
prefix owl: <http://www.w3.org/2002/07/owl#>
drop quad map virtrdf:ecrmDemo2 .
;
...
SPARQL
prefix ecrm: <http://demo.openlinksw.com/schemas/ecrm#>
prefix oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix cal: <http://www.w3.org/2002/12/cal/ical#>
prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
prefix product: <http://www.swop-project.eu/ontologies/pmo/product.owl#>
prefix owl: <http://www.w3.org/2002/07/owl#>
alter quad storage virtrdf:DefaultQuadStorage
FROM eCRM.DBA.SFA_SALES_QUOTA_VIEW2 as sales_quotas
FROM eCRM.DBA.SFA_COMPANIES_VIEW2 as companies
FROM eCRM.DBA.SFA_COMPANIES as companies_table text literal companies_table.DESCRIPTION of (companies.DESCRIPTION)
FROM eCRM.DBA.SFA_CONTACTS_VIEW2 as contacts
FROM eCRM.DBA.SFA_CONTACTS as contacts_table text literal contacts_table.NAME_FIRST of (contacts.NAME_FIRST)
FROM eCRM.DBA.SFA_EMPLOYMENTS_VIEW2 as employments
FROM eCRM.DBA.SFA_LEADS_VIEW2 as leads
FROM eCRM.DBA.SFA_LEADS as leads_table text literal leads_table.SUBJECT of (leads.SUBJECT)
FROM eCRM.DBA.SFA_OPPORTUNITIES_VIEW2 as opportunities
FROM eCRM.DBA.SFA_OPPORTUNITIES as opportunities_table text literal opportunities_table.OPPORTUNITY_NAME of (opportunities.OPPORTUNITY_NAME)
FROM eCRM.DBA.SFA_ACTIVITIES as activities
FROM eCRM.DBA.SFA_MAIL_MESSAGES as messages
FROM eCRM.DBA.SFA_DOCUMENTS_VIEW2 as documents
FROM eCRM.DBA.SFA_INFLUENCERS_VIEW2 as influencers
FROM eCRM.DBA.SFA_TEAMS_VIEW2 as teams
FROM eCRM.DBA.SFA_NOTES_VIEW2 as notes
FROM eCRM.DBA.SFA_NOTES as notes_table text literal notes_table.DESCRIPTION of (notes.DESCRIPTION)
FROM eCRM.DBA.SFA_COMPETITORS_VIEW2 as competitors
FROM eCRM.DBA.SFA_ISSUES_VIEW2 as issues
FROM eCRM.DBA.SFA_CUSTOM_FIELD_DEFS_VIEW2 as custom_field_defs
FROM eCRM.DBA.SFA_CUSTOM_FIELDS_VIEW2 as custom_fields
FROM eCRM.DBA.SFA_CASES_VIEW2 as cases
FROM eCRM.DBA.SFA_CASES as cases_table text literal cases_table.SUMMARY of (cases.SUMMARY)
FROM eCRM.DBA.SFA_ORDERS_VIEW2 as orders
FROM eCRM.DBA.SFA_ORDERS as orders_table text literal orders_table.EMAIL of (orders.EMAIL)
FROM eCRM.DBA.SFA_ORDER_ITEMS_VIEW2 as order_items
FROM eCRM.DBA.PM_CATEGORIES_VIEW2 as categories
FROM eCRM.DBA.PM_PRODUCT_ATTRIBUTE_DEFS_VIEW2 as product_attribute_defs
FROM eCRM.DBA.PM_PRODUCTS_VIEW2 as products
FROM eCRM.DBA.PM_PRODUCTS as products_table text literal products_table.DESCRIPTION of (products.DESCRIPTION)
FROM eCRM.DBA.PM_PRODUCT_ATTRIBUTES_VIEW2 as product_attributes
FROM eCRM.DBA.PM_CATALOGS_VIEW2 as catalogs
FROM eCRM.DBA.PM_CATALOG_PRODUCTS_VIEW2 as catalog_products
FROM eCRM.DBA.XSYS_MODULES as modules
FROM eCRM.DBA.XSYS_REGISTRY as registries
FROM eCRM.DBA.XSYS_ORGANIZATIONS_DATA as organizations_data
FROM eCRM.DBA.XSYS_MESSAGES as xsysmessages
FROM eCRM.DBA.XSYS_COUNTRIES_VIEW2 as countries
FROM eCRM.DBA.XSYS_PROVINCES_VIEW2 as provinces
FROM eCRM.DBA.XSYS_TIMEZONES as timezones
FROM eCRM.DBA.XSYS_MIME_TYPES as mimetypes
FROM eCRM.DBA.XSYS_MIME_EXTENSIONS as mimeexts
FROM eCRM.DBA.XSYS_CNAMES as cnames
FROM eCRM.DBA.XSYS_QUOTAS as quotas
FROM eCRM.DBA.XSYS_ROLES as roles
FROM eCRM.DBA.XSYS_ACCOUNTS as accounts
FROM eCRM.DBA.XSYS_USERDATA as userdatas
FROM eCRM.DBA.XSYS_GROUPDATA as groupdatas
FROM eCRM.DBA.XSYS_MEMBERS as members
FROM eCRM.DBA.XSYS_SESSIONS_DATA as sessionsdatas
FROM eCRM.DBA.XSYS_SESSION_DATA as sessiondatas
FROM eCRM.DBA.XSYS_LIST_MEMBERS_DEFS as list_members_defs
FROM eCRM.DBA.XSYS_CLASSES as classes
FROM eCRM.DBA.XSYS_ORG_CLASSES as org_classes
FROM eCRM.DBA.XSYS_CLASS_METHODS as class_methods
FROM eCRM.DBA.XSYS_CLASS_VIEWS as class_views
FROM eCRM.DBA.XSYS_ROLE_PRIVILEGES as role_priveleges
FROM eCRM.DBA.XSYS_USER_PRIVILEGES as user_priveleges
FROM eCRM.DBA.XSYS_HISTORY as history
FROM eCRM.DBA.XSYS_USERS as xsys_users
FROM eCRM.DBA.AP_PROCESSES_VIEW2 as ap_processes
FROM eCRM.DBA.AP_RULES_VIEW2 as ap_rules
FROM eCRM.DBA.AP_QUEUE as ap_queues
WHERE (^{companies.}^.COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{contacts.}^.COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{leads.}^.COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{products.}^.COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{orders.}^.SHIP_COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{leads_table.}^.FREETEXT_ID = ^{leads.}^.FREETEXT_ID)
WHERE (^{contacts_table.}^.FREETEXT_ID = ^{contacts.}^.FREETEXT_ID)
WHERE (^{companies_table.}^.FREETEXT_ID = ^{companies.}^.FREETEXT_ID)
WHERE (^{opportunities_table.}^.FREETEXT_ID = ^{opportunities.}^.FREETEXT_ID)
WHERE (^{cases_table.}^.FREETEXT_ID = ^{cases.}^.FREETEXT_ID)
WHERE (^{notes_table.}^.FREETEXT_ID = ^{notes.}^.FREETEXT_ID)
WHERE (^{orders_table.}^.FREETEXT_ID = ^{orders.}^.FREETEXT_ID)
WHERE (^{products_table.}^.FREETEXT_ID = ^{products.}^.FREETEXT_ID)
{
create virtrdf:ecrmDemo1 as graph iri ("http://^{URIQADefaultHost}^/ecrm") option (order 1501)
{
ecrm:Country (countries.COUNTRY_NAME)
a ecrm:Country
as virtrdf:Country-Countrys2 ;
a geo:SpatialThing
as virtrdf:Country-Countrys ;
owl:sameAs ecrm:dbpedia_iri (countries.COUNTRY_NAME) ;
ecrm:countryID countries.COUNTRY_ID
as virtrdf:Country-COUNTRY_ID ;
ecrm:countryID3 countries.COUNTRY_ID3
as virtrdf:Country-COUNTRY_ID3 ;
ecrm:isoCode countries.ISO_CODE
as virtrdf:Country-ISO_CODE ;
ecrm:countryName countries.COUNTRY_NAME
as virtrdf:Country-COUNTRY_NAME .
ecrm:Country (countries.COUNTRY_NAME)
ecrm:has_province
ecrm:Province (provinces.COUNTRY_ID, provinces.PROVINCE_NAME) where
(^{provinces.}^.COUNTRY_ID = ^{countries.}^.COUNTRY_ID) as virtrdf:ecrmCountry-has_province .
...
} .
} .
;
SPARQL
prefix ecrm: <http://demo.openlinksw.com/schemas/ecrm#>
prefix oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
prefix cal: <http://www.w3.org/2002/12/cal/ical#>
prefix product: <http://www.swop-project.eu/ontologies/pmo/product.owl#>
prefix owl: <http://www.w3.org/2002/07/owl#>
alter quad storage virtrdf:DefaultQuadStorage
FROM eCRM.DBA.SFA_SALES_QUOTA_VIEW2 as sales_quotas
FROM eCRM.DBA.SFA_COMPANIES_VIEW2 as companies
FROM eCRM.DBA.SFA_COMPANIES as companies_table text literal companies_table.DESCRIPTION of (companies.DESCRIPTION)
FROM eCRM.DBA.SFA_CONTACTS_VIEW2 as contacts
FROM eCRM.DBA.SFA_CONTACTS as contacts_table text literal contacts_table.NAME_FIRST of (contacts.NAME_FIRST)
FROM eCRM.DBA.SFA_EMPLOYMENTS_VIEW2 as employments
FROM eCRM.DBA.SFA_LEADS_VIEW2 as leads
FROM eCRM.DBA.SFA_LEADS as leads_table text literal leads_table.SUBJECT of (leads.SUBJECT)
FROM eCRM.DBA.SFA_OPPORTUNITIES_VIEW2 as opportunities
FROM eCRM.DBA.SFA_OPPORTUNITIES as opportunities_table text literal opportunities_table.OPPORTUNITY_NAME of (opportunities.OPPORTUNITY_NAME)
FROM eCRM.DBA.SFA_ACTIVITIES as activities
FROM eCRM.DBA.SFA_MAIL_MESSAGES as messages
FROM eCRM.DBA.SFA_DOCUMENTS_VIEW2 as documents
FROM eCRM.DBA.SFA_INFLUENCERS_VIEW2 as influencers
FROM eCRM.DBA.SFA_TEAMS_VIEW2 as teams
FROM eCRM.DBA.SFA_NOTES_VIEW2 as notes
FROM eCRM.DBA.SFA_NOTES as notes_table text literal notes_table.DESCRIPTION of (notes.DESCRIPTION)
FROM eCRM.DBA.SFA_COMPETITORS_VIEW2 as competitors
FROM eCRM.DBA.SFA_ISSUES_VIEW2 as issues
FROM eCRM.DBA.SFA_CUSTOM_FIELD_DEFS_VIEW2 as custom_field_defs
FROM eCRM.DBA.SFA_CUSTOM_FIELDS_VIEW2 as custom_fields
FROM eCRM.DBA.SFA_CASES_VIEW2 as cases
FROM eCRM.DBA.SFA_CASES as cases_table text literal cases_table.SUMMARY of (cases.SUMMARY)
FROM eCRM.DBA.SFA_ORDERS_VIEW2 as orders
FROM eCRM.DBA.SFA_ORDERS as orders_table text literal orders_table.EMAIL of (orders.EMAIL)
FROM eCRM.DBA.SFA_ORDER_ITEMS_VIEW2 as order_items
FROM eCRM.DBA.PM_CATEGORIES_VIEW2 as categories
FROM eCRM.DBA.PM_PRODUCT_ATTRIBUTE_DEFS_VIEW2 as product_attribute_defs
FROM eCRM.DBA.PM_PRODUCTS_VIEW2 as products
FROM eCRM.DBA.PM_PRODUCTS as products_table text literal products_table.DESCRIPTION of (products.DESCRIPTION)
FROM eCRM.DBA.PM_PRODUCT_ATTRIBUTES_VIEW2 as product_attributes
FROM eCRM.DBA.PM_CATALOGS_VIEW2 as catalogs
FROM eCRM.DBA.PM_CATALOG_PRODUCTS_VIEW2 as catalog_products
FROM eCRM.DBA.XSYS_MODULES as modules
FROM eCRM.DBA.XSYS_REGISTRY as registries
FROM eCRM.DBA.XSYS_ORGANIZATIONS_DATA as organizations_data
FROM eCRM.DBA.XSYS_MESSAGES as xsysmessages
FROM eCRM.DBA.XSYS_COUNTRIES_VIEW2 as countries
FROM eCRM.DBA.XSYS_PROVINCES_VIEW2 as provinces
FROM eCRM.DBA.XSYS_TIMEZONES as timezones
FROM eCRM.DBA.XSYS_MIME_TYPES as mimetypes
FROM eCRM.DBA.XSYS_MIME_EXTENSIONS as mimeexts
FROM eCRM.DBA.XSYS_CNAMES as cnames
FROM eCRM.DBA.XSYS_QUOTAS as quotas
FROM eCRM.DBA.XSYS_ROLES as roles
FROM eCRM.DBA.XSYS_ACCOUNTS as accounts
FROM eCRM.DBA.XSYS_USERDATA as userdatas
FROM eCRM.DBA.XSYS_GROUPDATA as groupdatas
FROM eCRM.DBA.XSYS_MEMBERS as members
FROM eCRM.DBA.XSYS_SESSIONS_DATA as sessionsdatas
FROM eCRM.DBA.XSYS_SESSION_DATA as sessiondatas
FROM eCRM.DBA.XSYS_LIST_MEMBERS_DEFS as list_members_defs
FROM eCRM.DBA.XSYS_CLASSES as classes
FROM eCRM.DBA.XSYS_ORG_CLASSES as org_classes
FROM eCRM.DBA.XSYS_CLASS_METHODS as class_methods
FROM eCRM.DBA.XSYS_CLASS_VIEWS as class_views
FROM eCRM.DBA.XSYS_ROLE_PRIVILEGES as role_priveleges
FROM eCRM.DBA.XSYS_USER_PRIVILEGES as user_priveleges
FROM eCRM.DBA.XSYS_HISTORY as history
FROM eCRM.DBA.XSYS_USERS as xsys_users
FROM eCRM.DBA.AP_PROCESSES_VIEW2 as ap_processes
FROM eCRM.DBA.AP_RULES_VIEW2 as ap_rules
FROM eCRM.DBA.AP_QUEUE as ap_queues
WHERE (^{companies.}^.COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{contacts.}^.COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{leads.}^.COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{products.}^.COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{orders.}^.SHIP_COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{leads_table.}^.FREETEXT_ID = ^{leads.}^.FREETEXT_ID)
WHERE (^{contacts_table.}^.FREETEXT_ID = ^{contacts.}^.FREETEXT_ID)
WHERE (^{companies_table.}^.FREETEXT_ID = ^{companies.}^.FREETEXT_ID)
WHERE (^{opportunities_table.}^.FREETEXT_ID = ^{opportunities.}^.FREETEXT_ID)
WHERE (^{cases_table.}^.FREETEXT_ID = ^{cases.}^.FREETEXT_ID)
WHERE (^{notes_table.}^.FREETEXT_ID = ^{notes.}^.FREETEXT_ID)
WHERE (^{orders_table.}^.FREETEXT_ID = ^{orders.}^.FREETEXT_ID)
WHERE (^{products_table.}^.FREETEXT_ID = ^{products.}^.FREETEXT_ID)
{
create virtrdf:ecrmDemo2 as graph iri ("http://^{URIQADefaultHost}^/ecrm") option (exclusive, order 1502)
{
ecrm:Order (orders.ORG_ID, orders.ORDER_ID)
a ecrm:Order
as virtrdf:Order-Orders ;
ecrm:has_ecrm_organization ecrm:OrganizationsData(orders.ORG_ID, organizations_data.DNS_ZONE) where (^{orders.}^.ORG_ID = ^{organizations_data.}^.ORG_ID)
as virtrdf:Order-ORG_ID ;
ecrm:owner ecrm:XSys_User(orders.ORG_ID, xsys_users.ACCOUNT_NAME, orders.OWNER_ID)
where (^{orders.}^.OWNER_ID = ^{xsys_users.}^.ACCOUNT_ID and ^{orders.}^.ORG_ID = ^{xsys_users.}^.ORG_ID)
as virtrdf:Order-OWNER_ID ;
ecrm:FREETEXT_ID orders.FREETEXT_ID
as virtrdf:Order-FREETEXT_ID ;
ecrm:has_company ecrm:Company(orders.COMPANY_NAME, orders.COMPANY_ID, orders.ORG_ID)
as virtrdf:Order-COMPANY_ID ;
ecrm:companyName orders.COMPANY_NAME
as virtrdf:Order-COMPANY_NAME ;
ecrm:has_contact ecrm:Contact(contacts.NAME_FIRST, contacts.NAME_MIDDLE, contacts.NAME_LAST, orders.CONTACT_ID, orders.ORG_ID)
where (^{orders.}^.CONTACT_ID = ^{contacts.}^.CONTACT_ID and ^{orders.}^.ORG_ID = ^{contacts.}^.ORG_ID)
as virtrdf:Order-CONTACT_ID ;
ecrm:contactName orders.CONTACT_NAME
as virtrdf:Order-CONTACT_NAME ;
ecrm:orderNo orders.ORDER_NO
as virtrdf:Order-ORDER_NO ;
ecrm:shipFirstName orders.SHIP_FNAME
as virtrdf:Order-SHIP_FNAME ;
ecrm:shipSecondName orders.SHIP_SNAME
as virtrdf:Order-SHIP_SNAME ;
ecrm:phoneNumber orders.PHONE_NUMBER
as virtrdf:Order-PHONE_NUMBER ;
ecrm:phoneExtension orders.PHONE_EXTENSION
as virtrdf:Order-PHONE_EXTENSION ;
ecrm:email orders.EMAIL
as virtrdf:Order-EMAIL ;
ecrm:shipCountry ecrm:Country(orders.SHIP_COUNTRY_NAME)
as virtrdf:Order-SHIP_COUNTRY_NAME ;
ecrm:shipCountryCode ecrm:Country (countries.COUNTRY_NAME) where (^{countries.}^.COUNTRY_NAME = ^{orders.}^.SHIP_COUNTRY_NAME)
as virtrdf:Order-SHIP_COUNTRY_CODE ;
ecrm:shipProvince orders.SHIP_PROVINCE
as virtrdf:Order-SHIP_PROVINCE ;
ecrm:shipCity orders.SHIP_CITY
as virtrdf:Order-SHIP_CITY ;
ecrm:dbpedia_shipCity ecrm:dbpedia_iri (orders.SHIP_CITY)
as virtrdf:Order-SHIP_dbpedia_CITY ;
ecrm:shipPostalCode orders.SHIP_POSTAL_CODE
as virtrdf:Order-SHIP_POSTAL_CODE ;
ecrm:shipAddress1 orders.SHIP_ADDRESS1
as virtrdf:Order-SHIP_ADDRESS1 ;
ecrm:shipAddress2 orders.SHIP_ADDRESS2
as virtrdf:Order-SHIP_ADDRESS2 ;
ecrm:salesRep orders.SALESREP
as virtrdf:Order-SALESREP ;
ecrm:orderDate orders.ORDER_DATE
as virtrdf:Order-ORDER_DATE ;
ecrm:orderValue orders.ORDER_VALUE
as virtrdf:Order-ORDER_VALUE ;
ecrm:refund orders.REFUND
as virtrdf:Order-REFUND ;
ecrm:year orders.YEAR
as virtrdf:Order-YEAR ;
ecrm:month orders.MONTH
as virtrdf:Order-MONTH ;
ecrm:quarter orders.QUARTER
as virtrdf:Order-QUARTER ;
ecrm:financialYear orders.FINANCIAL_YEAR
as virtrdf:Order-FINANCIAL_YEAR ;
ecrm:CONTACT_REL_ID orders.CONTACT_REL_ID
as virtrdf:Order-CONTACT_REL_ID ;
ecrm:COMPANY_REL_ID orders.COMPANY_REL_ID
as virtrdf:Order-COMPANY_REL_ID .
...
} .
} .
;
Here is sample example of a script to include an additional table alias for a table:
alter quad storage virtrdf:DefaultQuadStorage
:
FROM isports_rdf.prs10_isports_rdf.VRef_Call as Ref_Call_tbl
FROM isports_rdf.prs10_isports_rdf.VRef_Call as Ref_Call_tbl_1
:
{
:
refcall:ref-call_iri (Ref_Call_tbl.Call_Num) a refcall:Ref-Call as
virtrdf:ref-call_pk ;
:
refcall:has_parent refcall:ref-call_iri (Ref_Call_tbl_1.Call_Num)
where ( ^{Ref_Call_tbl.}^.Parent = ^{Ref_Call_tbl_1.}^.Call_Num ) as
virtrdf:Ref-Call_has_parent .
This demonstrates the way to self-join the table VRef_Call with itself. Like in SQL, are needed two different aliases for one table if you want to join it with itself.
use DB;
GRANT SELECT ON TPCH.DBA.PARTSUPP TO "SPARQL";
GRANT SELECT ON TPCH.DBA.SUPPLIER TO "SPARQL";
GRANT SELECT ON TPCH.DBA.CUSTOMER TO "SPARQL";
GRANT SELECT ON TPCH.DBA.HISTORY TO "SPARQL";
GRANT SELECT ON TPCH.DBA.PART TO "SPARQL";
GRANT SELECT ON TPCH.DBA.LINEITEM TO "SPARQL";
GRANT SELECT ON TPCH.DBA.ORDERS TO "SPARQL";
GRANT SELECT ON TPCH.DBA.NATION TO "SPARQL";
GRANT SELECT ON TPCH.DBA.REGION TO "SPARQL";
SPARQL
drop quad map virtrdf:TPCH
;
SPARQL
prefix tpch: <http://www.openlinksw.com/schemas/tpch#>
prefix oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
create iri class tpch:customer "http://^{URIQADefaultHost}^/tpch/customer/%U%d#this" (in custname varchar, in c_custkey integer not null) option (bijection, deref) .
create iri class tpch:lineitem "http://^{URIQADefaultHost}^/tpch/lineitem/%d/%d#this" (in l_orderkey integer not null, in l_linenumber integer not null) option (bijection, deref) .
create iri class tpch:nation "http://^{URIQADefaultHost}^/tpch/nation/%U%d#this" (in name varchar, in l_nationkey integer not null) option (bijection, deref) .
create iri class tpch:order "http://^{URIQADefaultHost}^/tpch/order/%d#this" (in o_orderkey integer not null) option (bijection, deref) .
create iri class tpch:part "http://^{URIQADefaultHost}^/tpch/part/%U%d#this" (in p_partname varchar, in p_partkey integer not null) option (bijection, deref) .
create iri class tpch:partsupp "http://^{URIQADefaultHost}^/tpch/partsupp/%d/%d#this" (in ps_partkey integer not null, in ps_suppkey integer not null) option (bijection, deref) .
create iri class tpch:region "http://^{URIQADefaultHost}^/tpch/region/%U%d#this" (in name varchar, in r_regionkey integer not null) option (bijection, deref) .
create iri class tpch:supplier "http://^{URIQADefaultHost}^/tpch/supplier/%U%d#this" (in name varchar, in s_supplierkey integer not null) option (bijection, deref) .
;
SPARQL
prefix tpch: <http://www.openlinksw.com/schemas/tpch#>
prefix oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix wgs: <http://www.w3.org/2003/01/geo/wgs84_pos#>
alter quad storage virtrdf:DefaultQuadStorage
FROM TPCH.DBA.LINEITEM as lineitems
FROM TPCH.DBA.CUSTOMER as customers
FROM TPCH.DBA.NATION as nations
FROM TPCH.DBA.ORDERS as orders
FROM TPCH.DBA.PART as parts
FROM TPCH.DBA.PARTSUPP as partsupps
FROM TPCH.DBA.REGION as regions
FROM TPCH.DBA.SUPPLIER as suppliers
where (^{suppliers.}^.S_NATIONKEY = ^{nations.}^.N_NATIONKEY)
where (^{customers.}^.C_NATIONKEY = ^{nations.}^.N_NATIONKEY)
{
create virtrdf:TPCH as graph iri ("http://^{URIQADefaultHost}^/tpch") option (exclusive)
{
# Customers
tpch:customer (customers.C_NAME, customers.C_CUSTKEY)
a tpch:customer
as virtrdf:customer-tpch-type ;
a foaf:Organization
as virtrdf:customer-foaf-type ;
tpch:custkey customers.C_CUSTKEY
as virtrdf:customer-c_custkey ;
foaf:name customers.C_NAME
as virtrdf:customer-foaf_name ;
tpch:companyName customers.C_NAME
as virtrdf:customer-c_name ;
tpch:has_nation tpch:nation (nations.N_NAME, customers.C_NATIONKEY)
as virtrdf:customer-c_nationkey ;
tpch:address customers.C_ADDRESS
as virtrdf:customer-c_address ;
foaf:phone customers.C_PHONE
as virtrdf:customer-foaf_phone ;
tpch:phone customers.C_PHONE
as virtrdf:customer-phone ;
tpch:acctbal customers.C_ACCTBAL
as virtrdf:customer-acctbal ;
tpch:mktsegment customers.C_MKTSEGMENT
as virtrdf:customer-c_mktsegment ;
tpch:comment customers.C_COMMENT
as virtrdf:customer-c_comment .
# Nations
tpch:nation (nations.N_NAME, customers.C_NATIONKEY)
tpch:nation_of tpch:customer (customers.C_NAME, customers.C_CUSTKEY)
as virtrdf:customer-nation_of .
tpch:lineitem (lineitems.L_ORDERKEY, lineitems.L_LINENUMBER)
a tpch:lineitem
as virtrdf:lineitem-lineitems ;
tpch:has_order tpch:order (lineitems.L_ORDERKEY)
as virtrdf:lineitem-l_orderkey ;
tpch:has_part tpch:part (parts.P_NAME, lineitems.L_PARTKEY)
where (^{parts.}^.P_PARTKEY = ^{lineitems.}^.L_PARTKEY)
as virtrdf:lineitem-l_partkey ;
tpch:has_supplier tpch:supplier (suppliers.S_NAME, lineitems.L_SUPPKEY)
where (^{suppliers.}^.S_SUPPKEY = ^{lineitems.}^.L_SUPPKEY)
as virtrdf:lineitem-l_suppkey ;
tpch:linenumber lineitems.L_LINENUMBER
as virtrdf:lineitem-l_linenumber ;
tpch:linequantity lineitems.L_QUANTITY
as virtrdf:lineitem-l_linequantity ;
tpch:lineextendedprice lineitems.L_EXTENDEDPRICE
as virtrdf:lineitem-l_lineextendedprice ;
tpch:linediscount lineitems.L_DISCOUNT
as virtrdf:lineitem-l_linediscount ;
tpch:linetax lineitems.L_TAX
as virtrdf:lineitem-l_linetax ;
tpch:returnflag lineitems.L_RETURNFLAG
as virtrdf:lineitem-l_returnflag ;
tpch:linestatus lineitems.L_LINESTATUS
as virtrdf:lineitem-l_linestatus ;
tpch:shipdate lineitems.L_SHIPDATE
as virtrdf:lineitem-l_shipdate ;
tpch:commitdate lineitems.L_COMMITDATE
as virtrdf:lineitem-l_commitdate ;
tpch:receiptdate lineitems.L_RECEIPTDATE
as virtrdf:lineitem-l_receiptdate ;
tpch:shipinstruct lineitems.L_SHIPINSTRUCT
as virtrdf:lineitem-l_shipinstruct ;
tpch:shipmode lineitems.L_SHIPMODE
as virtrdf:lineitem-l_shipmode ;
tpch:comment lineitems.L_COMMENT
as virtrdf:lineitem-l_comment .
tpch:part (parts.P_NAME, lineitems.L_PARTKEY)
tpch:part_of tpch:lineitem (lineitems.L_ORDERKEY, lineitems.L_LINENUMBER)
where (^{parts.}^.P_PARTKEY = ^{lineitems.}^.L_PARTKEY)
as virtrdf:lineitem-part_of .
tpch:order (lineitems.L_ORDERKEY)
tpch:order_of tpch:lineitem (lineitems.L_ORDERKEY, lineitems.L_LINENUMBER) as virtrdf:lineitem-order_of .
tpch:supplier (suppliers.S_NAME, lineitems.L_SUPPKEY)
tpch:supplier_of tpch:lineitem (lineitems.L_ORDERKEY, lineitems.L_LINENUMBER)
where (^{suppliers.}^.S_SUPPKEY = ^{lineitems.}^.L_SUPPKEY)
as virtrdf:lineitem-supplier_of .
# Nation
tpch:nation (nations.N_NAME, nations.N_NATIONKEY)
a tpch:nation
as virtrdf:nation-nations ;
tpch:name nations.N_NAME
as virtrdf:nation-n_name ;
tpch:has_region tpch:region (regions.R_NAME, nations.N_REGIONKEY)
where (^{regions.}^.R_REGIONKEY = ^{nations.}^.N_REGIONKEY)
as virtrdf:nation-n_regionkey ;
tpch:comment nations.N_COMMENT
as virtrdf:nation-n_comment .
tpch:region (regions.R_NAME, nations.N_REGIONKEY)
tpch:region_of tpch:nation (nations.N_NAME, nations.N_NATIONKEY)
where (^{regions.}^.R_REGIONKEY = ^{nations.}^.N_REGIONKEY)
as virtrdf:nation-region_of .
# Order
tpch:order (orders.O_ORDERKEY)
a tpch:order
as virtrdf:order-orders ;
tpch:orderkey orders.O_ORDERKEY
as virtrdf:order-o_orderkey ;
tpch:has_customer tpch:customer (customers.C_NAME, orders.O_CUSTKEY)
where (^{orders.}^.O_CUSTKEY = ^{customers.}^.C_CUSTKEY)
as virtrdf:order-o_custkey ;
tpch:orderstatus orders.O_ORDERSTATUS
as virtrdf:order-o_orderstatus ;
tpch:ordertotalprice orders.O_TOTALPRICE
as virtrdf:order-o_totalprice ;
tpch:orderdate orders.O_ORDERDATE
as virtrdf:order-o_orderdate ;
tpch:orderpriority orders.O_ORDERPRIORITY
as virtrdf:order-o_orderpriority ;
tpch:clerk orders.O_CLERK
as virtrdf:order-o_clerk ;
tpch:shippriority orders.O_SHIPPRIORITY
as virtrdf:order-o_shippriority ;
tpch:comment orders.O_COMMENT
as virtrdf:order-o_comment .
tpch:customer (customers.C_CUSTKEY, orders.O_CUSTKEY)
tpch:customer_of tpch:order (orders.O_ORDERKEY)
where (^{orders.}^.O_CUSTKEY = ^{customers.}^.C_CUSTKEY)
as virtrdf:order-customer_of .
# Part
tpch:part (parts.P_NAME, parts.P_PARTKEY)
a tpch:part
as virtrdf:part-parts ;
tpch:partkey parts.P_PARTKEY
as virtrdf:part-p_partkey ;
tpch:name parts.P_NAME
as virtrdf:part-p_name ;
tpch:mfgr parts.P_MFGR
as virtrdf:part-p_mfgr ;
tpch:brand parts.P_BRAND
as virtrdf:part-p_brand ;
tpch:type parts.P_TYPE
as virtrdf:part-p_type ;
tpch:size parts.P_SIZE
as virtrdf:part-p_size ;
tpch:container parts.P_CONTAINER
as virtrdf:part-p_container ;
tpch:comment parts.P_COMMENT
as virtrdf:part-p_comment .
# Partsupp
tpch:partsupp (partsupps.PS_PARTKEY, partsupps.PS_SUPPKEY)
a tpch:partsupp
as virtrdf:partsupp-partsupps ;
tpch:has_part tpch:part (parts.P_NAME, partsupps.PS_PARTKEY)
where (^{parts.}^.P_PARTKEY = ^{partsupps.}^.PS_PARTKEY)
as virtrdf:partsupp-ps_partkey ;
tpch:has_supplier tpch:supplier (suppliers.S_NAME, partsupps.PS_SUPPKEY)
where (^{suppliers.}^.S_SUPPKEY = ^{partsupps.}^.PS_SUPPKEY)
as virtrdf:partsupp-ps_suppkey ;
tpch:availqty partsupps.PS_AVAILQTY
as virtrdf:partsupp-ps_availqty ;
tpch:supplycost partsupps.PS_SUPPLYCOST
as virtrdf:partsupp-ps_supplycost ;
tpch:comment partsupps.PS_COMMENT
as virtrdf:partsupp-ps_comment .
tpch:part (parts.P_NAME, partsupps.PS_PARTKEY)
tpch:part_of tpch:partsupp (partsupps.PS_PARTKEY, partsupps.PS_SUPPKEY)
where (^{parts.}^.P_PARTKEY = ^{partsupps.}^.PS_PARTKEY)
as virtrdf:partsupp-part_of .
tpch:supplier (suppliers.S_NAME, partsupps.PS_SUPPKEY)
tpch:supplier_of tpch:partsupp (partsupps.PS_PARTKEY, partsupps.PS_SUPPKEY)
where (^{suppliers.}^.S_SUPPKEY = ^{partsupps.}^.PS_SUPPKEY)
as virtrdf:partsupp-supplier_of .
# Region
tpch:region (regions.R_NAME, regions.R_REGIONKEY)
a tpch:region
as virtrdf:region-regions ;
tpch:name regions.R_NAME
as virtrdf:region-r_name ;
tpch:comment regions.R_COMMENT
as virtrdf:region-r_comment .
# Supplier
tpch:supplier (suppliers.S_NAME, suppliers.S_SUPPKEY)
a tpch:supplier
as virtrdf:supplier-suppliers ;
tpch:name suppliers.S_NAME
as virtrdf:supplier-s_name ;
tpch:address suppliers.S_ADDRESS
as virtrdf:supplier-s_address ;
tpch:has_nation tpch:nation (nations.N_NAME, suppliers.S_NATIONKEY)
where (^{nations.}^.N_NATIONKEY = ^{suppliers.}^.S_NATIONKEY)
as virtrdf:supplier-s_nationkey ;
foaf:phone suppliers.S_PHONE
as virtrdf:supplier-foaf_phone ;
tpch:phone suppliers.S_PHONE
as virtrdf:supplier-s_phone ;
tpch:acctbal suppliers.S_ACCTBAL
as virtrdf:supplier-s_acctbal ;
tpch:comment suppliers.S_COMMENT
as virtrdf:supplier-s_comment .
tpch:nation (nations.N_NAME, suppliers.S_NATIONKEY)
tpch:nation_of tpch:supplier (suppliers.S_NAME, suppliers.S_SUPPKEY)
where (^{nations.}^.N_NATIONKEY = ^{suppliers.}^.S_NATIONKEY)
as virtrdf:supplier-nation_of .
} .
} .
;
DELETE FROM db.dba.url_rewrite_rule_list WHERE urrl_list like 'tpch_rule%';
DELETE FROM db.dba.url_rewrite_rule WHERE urr_rule like 'tpch_rule%';
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'tpch_rule2',
1,
'([^#]*)',
vector('path'),
1,
'/sparql?query=CONSTRUCT+{+%%3Chttp%%3A//^{URIQADefaultHost}^%U%%23this%%3E+%%3Fp+%%3Fo+}+FROM+%%3Chttp%%3A//^{URIQADefaultHost}^/tpch%%3E+WHERE+{+%%3Chttp%%3A//^{URIQADefaultHost}^%U%%23this%%3E+%%3Fp+%%3Fo+}&format=%U',
vector('path', 'path', '*accept*'),
null,
'(text/rdf.n3)|(application/rdf.xml)',
0,
null
);
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'tpch_rule1',
1,
'([^#]*)',
vector('path'),
1,
'/about/html/http/^{URIQADefaultHost}^%s%%23this',
vector('path'),
null,
'(text/html)|(\\*/\\*)',
0,
303
);
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'tpch_rule3',
1,
'(/[^#]*)/\x24',
vector('path'),
1,
'%s',
vector('path'),
null,
null,
0,
null
);
create procedure DB.DBA.REMOVE_TPCH_RDF_DET()
{
declare colid int;
colid := DAV_SEARCH_ID('/DAV/home/demo/tpch', 'C');
if (colid < 0)
return;
update WS.WS.SYS_DAV_COL set COL_DET=null where COL_ID = colid;
}
;
DB.DBA.REMOVE_TPCH_RDF_DET();
drop procedure DB.DBA.REMOVE_TPCH_RDF_DET;
create procedure DB.DBA.TPCH_MAKE_RDF_DET()
{
declare uriqa_str varchar;
uriqa_str := cfg_item_value(virtuoso_ini_path(), 'URIQA','DefaultHost');
uriqa_str := 'http://' || uriqa_str || '/tpch';
DB.DBA."RDFData_MAKE_DET_COL" ('/DAV/home/demo/tpch/RDFData/', uriqa_str, NULL);
VHOST_REMOVE (lpath=>'/tpch/data/rdf');
DB.DBA.VHOST_DEFINE (lpath=>'/tpch/data/rdf', ppath=>'/DAV/home/demo/tpch/RDFData/All/', is_dav=>1, vsp_user=>'dba');
}
;
DB.DBA.TPCH_MAKE_RDF_DET();
drop procedure DB.DBA.TPCH_MAKE_RDF_DET;
create procedure DB.DBA.TPCH_DET_REF (in par varchar, in fmt varchar, in val varchar)
{
declare res, iri any;
declare uriqa_str varchar;
uriqa_str := cfg_item_value(virtuoso_ini_path(), 'URIQA','DefaultHost');
uriqa_str := 'http://' || uriqa_str || '/tpch';
iri := uriqa_str || val;
res := sprintf ('iid (%d).rdf', iri_id_num (iri_to_id (iri)));
return sprintf (fmt, res);
}
;
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('tpch_rdf', 1,
'/tpch/(.*)', vector('path'), 1,
'/tpch/data/rdf/%U', vector('path'),
'DB.DBA.TPCH_DET_REF',
'application/rdf.xml',
2,
303);
DB.DBA.URLREWRITE_CREATE_RULELIST (
'tpch_rule_list1',
1,
vector (
'tpch_rule1',
'tpch_rule2',
'tpch_rule3',
'tpch_rdf'
));
DB.DBA.VHOST_REMOVE (lpath=>'/tpch');
DB.DBA.VHOST_DEFINE (lpath=>'/tpch', ppath=>'/DAV/home/demo/tpch/', vsp_user=>'dba', is_dav=>1,
is_brws=>0, opts=>vector ('url_rewrite', 'tpch_rule_list1'));
DB.DBA.VHOST_REMOVE (lpath=>'/tpch/linkeddata');
DB.DBA.VHOST_DEFINE (lpath=>'/tpch/linkeddata', ppath=>'/DAV/home/demo/tpch/', vsp_user=>'dba', is_dav=>1,
is_brws=>1);
Please load ~\binsrc\dav\DET_RDFData.sql before loadding this script (tpc-d has no vad to do this automatic)
use DB;
create procedure DB.DBA.exec_no_error (in expr varchar) {
declare state, message, meta, result any;
exec(expr, state, message, vector(), 0, meta, result);
}
;
DB.DBA.exec_no_error('GRANT \"SPARQL_UPDATE\" TO \"SPARQL\"')
;
GRANT SELECT ON tpcd.DBA.partsupp TO "SPARQL";
GRANT SELECT ON tpcd.DBA.supplier TO "SPARQL";
GRANT SELECT ON tpcd.DBA.customer TO "SPARQL";
GRANT SELECT ON tpcd.DBA.history TO "SPARQL";
GRANT SELECT ON tpcd.DBA.part TO "SPARQL";
GRANT SELECT ON tpcd.DBA.lineitem TO "SPARQL";
GRANT SELECT ON tpcd.DBA.orders TO "SPARQL";
GRANT SELECT ON tpcd.DBA.nation TO "SPARQL";
GRANT SELECT ON tpcd.DBA.region TO "SPARQL";
SPARQL
prefix tpcd: <http://demo.openlinksw.com/schemas/tpcd#>
prefix oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix wgs: <http://www.w3.org/2003/01/geo/wgs84_pos#>
drop quad map graph iri("http://^{URIQADefaultHost}^/tpcd") .
;
SPARQL
prefix tpcd: <http://demo.openlinksw.com/schemas/tpcd#>
prefix oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix wgs: <http://www.w3.org/2003/01/geo/wgs84_pos#>
drop quad map virtrdf:TpcdDemo .
;
SPARQL
prefix tpcd: <http://demo.openlinksw.com/schemas/tpcd#>
prefix oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix wgs: <http://www.w3.org/2003/01/geo/wgs84_pos#>
create iri class tpcd:customer "http://^{URIQADefaultHost}^/tpcd/customer/%d#this" (in c_custkey integer not null) .
create iri class tpcd:lineitem "http://^{URIQADefaultHost}^/tpcd/lineitem/%d/%d#this" (in l_orderkey integer not null, in l_linenumber integer not null) .
create iri class tpcd:nation "http://^{URIQADefaultHost}^/tpcd/nation/%d#this" (in l_nationkey integer not null) .
create iri class tpcd:order "http://^{URIQADefaultHost}^/tpcd/order/%d#this" (in o_orderkey integer not null) .
create iri class tpcd:part "http://^{URIQADefaultHost}^/tpcd/part/%d#this" (in p_partkey integer not null) .
create iri class tpcd:partsupp "http://^{URIQADefaultHost}^/tpcd/partsupp/%d/%d#this" (in ps_partkey integer not null, in ps_suppkey integer not null) .
create iri class tpcd:region "http://^{URIQADefaultHost}^/tpcd/region/%d#this" (in r_regionkey integer not null) .
create iri class tpcd:supplier "http://^{URIQADefaultHost}^/tpcd/supplier/%d#this" (in s_supplierkey integer not null) .
;
SPARQL
prefix tpcd: <http://demo.openlinksw.com/schemas/tpcd#>
prefix oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix wgs: <http://www.w3.org/2003/01/geo/wgs84_pos#>
alter quad storage virtrdf:DefaultQuadStorage
FROM tpcd.DBA.lineitem as lineitems
FROM tpcd.DBA.customer as customers
FROM tpcd.DBA.nation as nations
FROM tpcd.DBA.orders as orders
FROM tpcd.DBA.part as parts
FROM tpcd.DBA.partsupp as partsupps
FROM tpcd.DBA.region as regions
FROM tpcd.DBA.supplier as suppliers
where (^{suppliers.}^.s_nationkey = ^{nations.}^.n_nationkey)
where (^{customers.}^.c_nationkey = ^{nations.}^.n_nationkey)
{
create virtrdf:TpcdDemo as graph iri ("http://^{URIQADefaultHost}^/tpcd") option (exclusive)
{
tpcd:customer (customers.c_custkey)
a tpcd:customer
as virtrdf:tpcdcustomer-c_custkey2 ;
a foaf:Organization
as virtrdf:tpcdcustomer-c_custkey ;
foaf:name customers.c_name
as virtrdf:tpcdcustomer-foaf_name ;
tpcd:companyName customers.c_name
as virtrdf:tpcdcustomer-c_name ;
tpcd:has_nation tpcd:nation (customers.c_nationkey)
as virtrdf:tpcdcustomer-c_nationkey ;
tpcd:address customers.c_address
as virtrdf:tpcdcustomer-c_address ;
foaf:phone customers.c_phone
as virtrdf:tpcdcustomer-foaf_phone ;
tpcd:phone customers.c_phone
as virtrdf:tpcdcustomer-phone ;
tpcd:mktsegment customers.c_mktsegment
as virtrdf:tpcdcustomer-c_mktsegment ;
tpcd:comment customers.c_comment
as virtrdf:tpcdcustomer-c_comment .
tpcd:nation (customers.c_nationkey)
tpcd:nation_of tpcd:customer (customers.c_custkey) as virtrdf:tpcdcustomer-nation_of .
tpcd:lineitem (lineitems.l_orderkey, lineitems.l_linenumber)
a tpcd:lineitem
as virtrdf:tpcdlineitem-lineitems ;
tpcd:has_order tpcd:order (lineitems.l_orderkey)
as virtrdf:tpcdlineitem-l_orderkey ;
tpcd:has_part tpcd:part (lineitems.l_partkey)
as virtrdf:tpcdlineitem-l_partkey ;
tpcd:has_supplier tpcd:supplier (lineitems.l_suppkey)
as virtrdf:tpcdlineitem-l_suppkey ;
tpcd:linenumber lineitems.l_linenumber
as virtrdf:tpcdlineitem-l_linenumber ;
tpcd:returnflag lineitems.l_returnflag
as virtrdf:tpcdlineitem-l_returnflag ;
tpcd:linestatus lineitems.l_linestatus
as virtrdf:tpcdlineitem-l_linestatus ;
tpcd:shipdate lineitems.l_shipdate
as virtrdf:tpcdlineitem-l_shipdate ;
tpcd:commitdate lineitems.l_commitdate
as virtrdf:tpcdlineitem-l_commitdate ;
tpcd:receiptdate lineitems.l_receiptdate
as virtrdf:tpcdlineitem-l_receiptdate ;
tpcd:shipinstruct lineitems.l_shipinstruct
as virtrdf:tpcdlineitem-l_shipinstruct ;
tpcd:shipmode lineitems.l_shipmode
as virtrdf:tpcdlineitem-l_shipmode ;
tpcd:comment lineitems.l_comment
as virtrdf:tpcdlineitem-l_comment .
tpcd:part (lineitems.l_partkey)
tpcd:part_of tpcd:lineitem (lineitems.l_orderkey, lineitems.l_linenumber) as virtrdf:tpcdlineitem-part_of .
tpcd:order (lineitems.l_orderkey)
tpcd:order_of tpcd:lineitem (lineitems.l_orderkey, lineitems.l_linenumber) as virtrdf:tpcdlineitem-order_of .
tpcd:supplier (lineitems.l_suppkey)
tpcd:supplier_of tpcd:lineitem (lineitems.l_orderkey, lineitems.l_linenumber) as virtrdf:tpcdlineitem-supplier_of .
tpcd:nation (nations.n_nationkey)
a tpcd:nation
as virtrdf:tpcdnation-nations ;
tpcd:name nations.n_name
as virtrdf:tpcdnation-n_name ;
tpcd:has_region tpcd:region (nations.n_regionkey)
as virtrdf:tpcdnation-n_regionkey ;
tpcd:comment nations.n_comment
as virtrdf:tpcdnation-n_comment .
tpcd:region (nations.n_regionkey)
tpcd:region_of tpcd:nation (nations.n_nationkey) as virtrdf:tpcdnation-region_of .
tpcd:order (orders.o_orderkey)
a tpcd:order
as virtrdf:tpcdorder-orders ;
tpcd:has_customer tpcd:customer (orders.o_custkey)
as virtrdf:tpcdorder-o_custkey ;
tpcd:orderstatus orders.o_orderstatus
as virtrdf:tpcdorder-o_orderstatus ;
tpcd:orderdate orders.o_orderdate
as virtrdf:tpcdorder-o_orderdate ;
tpcd:orderpriority orders.o_orderpriority
as virtrdf:tpcdorder-o_orderpriority ;
tpcd:clerk orders.o_clerk
as virtrdf:tpcdorder-o_clerk ;
tpcd:shippriority orders.o_shippriority
as virtrdf:tpcdorder-o_shippriority ;
tpcd:comment orders.o_comment
as virtrdf:tpcdorder-o_comment .
tpcd:customer (orders.o_custkey)
tpcd:customer_of tpcd:order (orders.o_orderkey) as virtrdf:tpcdorder-customer_of .
tpcd:part (parts.p_partkey)
a tpcd:part
as virtrdf:tpcdpart-parts ;
tpcd:name parts.p_name
as virtrdf:tpcdpart-p_name ;
tpcd:mfgr parts.p_mfgr
as virtrdf:tpcdpart-p_mfgr ;
tpcd:brand parts.p_brand
as virtrdf:tpcdpart-p_brand ;
tpcd:type parts.p_type
as virtrdf:tpcdpart-p_type ;
tpcd:size parts.p_size
as virtrdf:tpcdpart-p_size ;
tpcd:container parts.p_container
as virtrdf:tpcdpart-p_container ;
tpcd:comment parts.p_comment
as virtrdf:tpcdpart-p_comment .
tpcd:partsupp (partsupps.ps_partkey, partsupps.ps_suppkey)
a tpcd:partsupp
as virtrdf:tpcdpartsupp-partsupps ;
tpcd:has_part tpcd:part (partsupps.ps_partkey)
as virtrdf:tpcdpartsupp-ps_partkey ;
tpcd:has_supplier tpcd:supplier (partsupps.ps_suppkey)
as virtrdf:tpcdpartsupp-ps_suppkey ;
tpcd:availqty partsupps.ps_availqty
as virtrdf:tpcdpartsupp-ps_availqty ;
tpcd:comment partsupps.ps_comment
as virtrdf:tpcdpartsupp-ps_comment .
tpcd:part (partsupps.ps_partkey)
tpcd:part_of tpcd:partsupp (partsupps.ps_partkey, partsupps.ps_suppkey) as virtrdf:tpcdpartsupp-part_of .
tpcd:supplier (partsupps.ps_suppkey)
tpcd:supplier_of tpcd:partsupp (partsupps.ps_partkey, partsupps.ps_suppkey) as virtrdf:tpcdpartsupp-supplier_of .
tpcd:region (regions.r_regionkey)
a tpcd:region
as virtrdf:tpcdregion-regions ;
tpcd:name regions.r_name
as virtrdf:tpcdregion-r_name ;
tpcd:comment regions.r_comment
as virtrdf:tpcdregion-r_comment .
tpcd:supplier (suppliers.s_suppkey)
a tpcd:supplier
as virtrdf:tpcdsupplier-suppliers ;
tpcd:name suppliers.s_name
as virtrdf:tpcdsupplier-s_name ;
tpcd:address suppliers.s_address
as virtrdf:tpcdsupplier-s_address ;
tpcd:has_nation tpcd:nation (suppliers.s_nationkey)
as virtrdf:tpcdsupplier-s_nationkey ;
foaf:phone customers.c_phone
as virtrdf:tpcdsupplier-foaf_phone ;
tpcd:phone suppliers.s_phone
as virtrdf:tpcdsupplier-s_phone ;
tpcd:comment suppliers.s_comment
as virtrdf:tpcdsupplier-s_comment .
tpcd:nation (suppliers.s_nationkey)
tpcd:nation_of tpcd:supplier (suppliers.s_suppkey) as virtrdf:tpcdsupplier-nation_of .
} .
} .
;
create procedure tcpd_rdf_doc (in path varchar)
{
declare r any;
r := regexp_match ('[^/]*\x24', path);
return r||'#this';
};
create procedure tcpd_html_doc (in path varchar)
{
declare r any;
r := regexp_match ('[^/]*#', path);
return subseq (r, 0, length (r)-1);
};
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'tcpd_rule2',
1,
'(/[^#]*)',
vector('path'),
1,
'/sparql?query=CONSTRUCT+{+%%3Chttp%%3A//^{URIQADefaultHost}^%U%%23this%%3E+%%3Fp+%%3Fo+}+FROM+%%3Chttp%%3A//^{URIQADefaultHost}^/tpcd%%3E+WHERE+{+%%3Chttp%%3A//^{URIQADefaultHost}^%U%%23this%%3E+%%3Fp+%%3Fo+}&format=%U',
vector('path', 'path', '*accept*'),
null,
'(text/rdf.n3)|(application/rdf.xml)',
0,
null
);
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'tcpd_rule1',
1,
'(/[^#]*)',
vector('path'),
1,
'/rdfbrowser/index.html?uri=http%%3A//^{URIQADefaultHost}^%U%%23this',
vector('path'),
null,
'(text/html)|(\\*/\\*)',
0,
303
);
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'tcpd_rule3',
1,
'(/[^#]*)/\x24',
vector('path'),
1,
'%s',
vector('path'),
null,
null,
0,
null
);
DB.DBA."RDFData_MAKE_DET_COL" ('/DAV/home/tpcd/RDFData/', 'http://^{URIQADefaultHost}^/tpcd', NULL);
VHOST_REMOVE (lpath=>'/tpcd/data/rdf');
DB.DBA.VHOST_DEFINE (lpath=>'/tpcd/data/rdf', ppath=>'/DAV/home/tpcd/RDFData/All/', is_dav=>1, vsp_user=>'dba');
-- procedure to convert path to DET resource name
create procedure DB.DBA.TPCD_DET_REF (in par varchar, in fmt varchar, in val varchar)
{
declare res, iri any;
iri := 'http://^{URIQADefaultHost}^/tpcd' || val;
res := sprintf ('iid (%d).rdf', iri_id_num (iri_to_id (iri)));
return sprintf (fmt, res);
}
;
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('tpcd_rdf', 1,
'/tpcd/(.*)', vector('path'), 1,
'/tpcd/data/rdf/%U', vector('path'),
'DB.DBA.TPCD_DET_REF',
'application/rdf.xml',
2,
303);
DB.DBA.URLREWRITE_CREATE_RULELIST (
'tpcd_rule_list1',
1,
vector (
'tcpd_rule1',
'tcpd_rule2',
'tcpd_rule3',
'tpcd_rdf'
));
VHOST_REMOVE (lpath=>'/tpcd');
DB.DBA.VHOST_DEFINE (lpath=>'/tpcd', ppath=>'/DAV/home/', vsp_user=>'dba', is_dav=>1, def_page=>'sfront.vspx',
is_brws=>0, opts=>vector ('url_rewrite', 'tpcd_rule_list1'));
DB.DBA.XML_SET_NS_DECL ('tpcd', 'http://demo.openlinksw.com/schemas/tpcd#', 2);
use DB;
create procedure DB.DBA.SPARQL_THALIA_RUN (in txt varchar)
{
declare REPORT, stat, msg, sqltext varchar;
declare metas, rowset any;
result_names (REPORT);
sqltext := string_output_string (sparql_to_sql_text (txt));
stat := '00000';
msg := '';
rowset := null;
exec (sqltext, stat, msg, vector (), 1000, metas, rowset);
--result ('STATE=' || stat || ': ' || msg);
--if (rowset is not null)
-- {
-- foreach (any r in rowset) do
-- result (r[0] || ': ' || r[1]);
-- }
}
;
use thalia;
DB.DBA.exec_no_error('drop View thalia.Demo.asu_v');
create View thalia.Demo.asu_v as select left(Title,3) code,* FROM thalia.Demo.asu;
DB.DBA.exec_no_error('drop View thalia.Demo.gatech_v');
create View thalia.Demo.gatech_v as select *, Room||' '||Building Place FROM thalia.Demo.gatech;
use DB;
DB.DBA.exec_no_error('GRANT \"SPARQL_UPDATE\" TO \"SPARQL\"');
GRANT SELECT ON thalia.Demo.asu TO "SPARQL";
GRANT SELECT ON thalia.Demo.asu_v TO "SPARQL";
GRANT SELECT ON thalia.Demo.brown TO "SPARQL";
GRANT SELECT ON thalia.Demo.cmu TO "SPARQL";
GRANT SELECT ON thalia.Demo.gatech TO "SPARQL";
GRANT SELECT ON thalia.Demo.gatech_v TO "SPARQL";
GRANT SELECT ON thalia.Demo.toronto TO "SPARQL";
GRANT SELECT ON thalia.Demo.ucsd TO "SPARQL";
GRANT SELECT ON thalia.Demo.umd TO "SPARQL";
DB.DBA.SPARQL_THALIA_RUN('drop quad map graph iri("http://^{URIQADefaultHost}^/Thalia") .
')
;
DB.DBA.SPARQL_THALIA_RUN('drop quad map graph iri("http://^{URIQADefaultHost}^/thalia") .
');
DB.DBA.SPARQL_THALIA_RUN('drop quad map virtrdf:ThaliaDemo .
');
DB.DBA.SPARQL_THALIA_RUN('
prefix oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix xsd: <http://www.w3.org/2001/XMLSchema#>
prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
prefix owl: <http://www.w3.org/2002/07/owl#>
prefix dc: <http://purl.org/dc/elements/1.1/>
prefix time: <http://www.w3.org/2006/time#>
prefix event: <http://purl.org/NET/c4dm/event.owl#>
prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix skos: <http://www.w3.org/2004/02/skos/core#>
prefix th: <http://purl.org/ontology/thalia/1.0/>
create iri class th:Asu "http://^{URIQADefaultHost}^/thalia/asu/course/%U#this" (in code varchar not null) .
create iri class th:Brown "http://^{URIQADefaultHost}^/thalia/brown/course/%U#this" (in Code varchar not null) .
create iri class th:BrownInstructor "http://^{URIQADefaultHost}^/thalia/brown/instructor/%U#this" (in Code varchar not null) .
create iri class th:BrownLecture "http://^{URIQADefaultHost}^/thalia/brown/lecture/%U#this" (in Code varchar not null) .
create iri class th:BrownPlace "http://^{URIQADefaultHost}^/thalia/brown/place/%U#this" (in Code varchar not null) .
create iri class th:Cmu "http://^{URIQADefaultHost}^/thalia/cmu/course/%U/%U#this" (in Code varchar not null, in Sec varchar) .
create iri class th:CmuInstructor "http://^{URIQADefaultHost}^/thalia/cmu/instructor/%U/%U#this" (in Code varchar not null, in Sec varchar) .
create iri class th:CmuLecture "http://^{URIQADefaultHost}^/thalia/cmu/lecture/%U/%U#this" (in Code varchar not null, in Sec varchar) .
create iri class th:CmuPlace "http://^{URIQADefaultHost}^/thalia/cmu/place/%U/%U#this" (in Code varchar not null, in Sec varchar) .
create iri class th:CmuEventTime "http://^{URIQADefaultHost}^/thalia/cmu/eventtime/%U/%U#this" (in Code varchar not null, in Sec varchar) .
create iri class th:CmuDatetime "http://^{URIQADefaultHost}^/thalia/cmu/datetime/%U/%U#this" (in Code varchar not null, in Sec varchar) .
create iri class th:Gatech "http://^{URIQADefaultHost}^/thalia/gatech/course/%U/%d/%U#this" (in Department varchar, in Code integer, in Section varchar) .
create iri class th:GatechInstructor "http://^{URIQADefaultHost}^/thalia/gatech/instructor/%U/%d/%U#this" (in Department varchar, in Code integer, in Section varchar) .
create iri class th:GatechLecture "http://^{URIQADefaultHost}^/thalia/gatech/lecture/%U/%d/%U#this" (in Department varchar, in Code integer, in Section varchar) .
create iri class th:GatechEventTime "http://^{URIQADefaultHost}^/thalia/gatech/eventtime/%U/%d/%U#this" (in Department varchar, in Code integer, in Section varchar) .
create iri class th:GatechDatetime "http://^{URIQADefaultHost}^/thalia/gatech/datetime/%U/%d/%U#this" (in Department varchar, in Code integer, in Section varchar) .
create iri class th:GatechPlace "http://^{URIQADefaultHost}^/thalia/gatech/place/%U/%d/%U#this" (in Department varchar, in Code integer, in Section varchar) .
create iri class th:Toronto "http://^{URIQADefaultHost}^/thalia/toronto/course/%U#this" (in No_ varchar) .
create iri class th:TorontoInstructor "http://^{URIQADefaultHost}^/thalia/toronto/instructor/%U#this" (in No_ varchar) .
create iri class th:TorontoLecture "http://^{URIQADefaultHost}^/thalia/toronto/lecture/%U#this" (in No_ varchar) .
create iri class th:TorontoPlace "http://^{URIQADefaultHost}^/thalia/toronto/place/%U#this" (in No_ varchar) .
create iri class th:Ucsd "http://^{URIQADefaultHost}^/thalia/ucsd/course/%U#this" (in Number varchar) .
create iri class th:UcsdInstructor1 "http://^{URIQADefaultHost}^/thalia/ucsd/instructor1/%U#this" (in Number varchar) .
create iri class th:UcsdInstructor2 "http://^{URIQADefaultHost}^/thalia/ucsd/instructor2/%U#this" (in Number varchar) .
create iri class th:UcsdInstructor3 "http://^{URIQADefaultHost}^/thalia/ucsd/instructor3/%U#this" (in Number varchar) .
create iri class th:Umd "http://^{URIQADefaultHost}^/thalia/umd/course/%U#this" (in Code varchar) .
create iri class th:UmdLecture "http://^{URIQADefaultHost}^/thalia/umd/lecture/%U#this" (in Code varchar) .
create iri class th:UmdEventTime "http://^{URIQADefaultHost}^/thalia/umd/eventtime/%U#this" (in Code varchar) .
create iri class th:UmdDatetime "http://^{URIQADefaultHost}^/thalia/umd/datetime/%U#this" (in Code varchar) .
')
;
DB.DBA.RDF_AUDIT_METADATA (1, '*');
DB.DBA.SPARQL_THALIA_RUN('prefix oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix xsd: <http://www.w3.org/2001/XMLSchema#>
prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
prefix owl: <http://www.w3.org/2002/07/owl#>
prefix dc: <http://purl.org/dc/elements/1.1/>
prefix time: <http://www.w3.org/2006/time#>
prefix event: <http://purl.org/NET/c4dm/event.owl#>
prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix skos: <http://www.w3.org/2004/02/skos/core#>
prefix th: <http://purl.org/ontology/thalia/1.0/>
alter quad storage virtrdf:DefaultQuadStorage
FROM thalia.demo.asu_v as asus
FROM thalia.demo.brown as browns
FROM thalia.demo.cmu as cmus
FROM thalia.demo.gatech_v as gatechs
FROM thalia.demo.toronto as torontos
FROM thalia.demo.ucsd as ucsds
FROM thalia.demo.umd as umds
{
create virtrdf:ThaliaDemo as graph iri ("http://^{URIQADefaultHost}^/thalia") option (exclusive)
{
th:Asu (asus.code)
a th:Course
as virtrdf:Asu-Course ;
dc:title asus.Title
as virtrdf:Asu-Title ;
dc:description asus.Description
as virtrdf:Asu-Description ;
rdfs:seeAlso asus.MoreInfoURL
as virtrdf:Asu-MoreInfoURL ;
th:forUniversity "http://purl.org/thalia/university/asu"
as virtrdf:Asu-University ;
skos:subject "http://purl.org/subject/thalia/ComputerScience"
as virtrdf:Asu-Subject
.
th:Brown (browns.Code)
a th:Course
as virtrdf:Brown-Course ;
dc:title browns.Title
as virtrdf:Brown-Title ;
th:hasInstructor th:BrownInstructor (browns.Code)
as virtrdf:Brown-hasInstructor ;
th:hasLecture th:BrownLecture(browns.Code)
as virtrdf:Brown-hasLecture ;
th:forUniversity "http://purl.org/thalia/university/brown"
as virtrdf:Brown-University ;
skos:subject "http://purl.org/subject/thalia/ComputerScience"
as virtrdf:Brown-Subject
.
th:BrownInstructor (browns.Code)
a th:Instructor
as virtrdf:Brown-Instructor ;
dc:homepage browns.Instructor
as virtrdf:Brown-Instructor-Homepage
.
th:BrownLecture (browns.Code)
a event:Event
as virtrdf:Brown-Lecture ;
event:place th:BrownPlace(browns.Code)
as virtrdf:Brown-hasPlace
.
th:BrownPlace (browns.Code)
a geo:Point
as virtrdf:Brown-Place;
dc:title browns.Room
as virtrdf:Brown-Room
.
th:Cmu (cmus.Code, cmus.Sec)
a th:Course
as virtrdf:Cmu-Course ;
dc:title cmus.CourseTitle
as virtrdf:Cmu-CourseTitle ;
th:hasInstructor th:CmuInstructor (cmus.Code, cmus.Sec)
as virtrdf:Cmu-hasInstructor ;
th:hasLecture th:CmuLecture(cmus.Code, cmus.Sec)
as virtrdf:Cmu-hasLecture ;
th:hasUnits cmus.Units
as virtrdf:Cmu-hasUnits ;
th:forUniversity "http://purl.org/thalia/university/cmu"
as virtrdf:Cmu-University ;
skos:subject "http://purl.org/subject/thalia/ComputerScience"
as virtrdf:Cmu-Subject
.
th:CmuInstructor (cmus.Code, cmus.Sec)
a th:Instructor
as virtrdf:Cmu-Instructor ;
foaf:name cmus.Lecturer
as virtrdf:Cmu-Lecturer
.
th:CmuLecture (cmus.Code, cmus.Sec)
a event:Event
as virtrdf:Cmu-Lecture ;
event:time th:CmuEventTime(cmus.Code, cmus.Sec)
as virtrdf:Cmu-hasEventTime ;
event:place th:CmuPlace(cmus.Code, cmus.Sec)
as virtrdf:Cmu-hasPlace
.
th:CmuPlace (cmus.Code, cmus.Sec)
a geo:Point
as virtrdf:Cmu-Place;
dc:title cmus.Room
as virtrdf:Cmu-Room
.
th:CmuEventTime (cmus.Code, cmus.Sec)
a time:Interval
as virtrdf:Cmu-EventTime;
time:inDateTime th:CmuDatetime(cmus.Code, cmus.Sec)
as virtrdf:Cmu-inDateTime
.
th:CmuDatetime (cmus.Code, cmus.Sec)
a time:DateTimeDescription
as virtrdf:Cmu-Datetime;
time:dayOfWeek cmus.Day_
as virtrdf:Cmu-Day ;
time:hour cmus.Time_
as virtrdf:Cmu-Time
.
th:Gatech (gatechs.Department, gatechs.Code, gatechs.Section)
a th:Course
as virtrdf:Gatech-Course ;
dc:title gatechs.Title
as virtrdf:Gatech-Title ;
th:hasInstructor th:GatechInstructor(gatechs.Department, gatechs.Code, gatechs.Section)
as virtrdf:Gatech-hasInstructor ;
dc:description gatechs.Description
as virtrdf:Gatech-Description ;
th:hasLecture th:GatechLecture(gatechs.Department, gatechs.Code, gatechs.Section)
as virtrdf:Gatech-hasLecture ;
th:forUniversity "http://purl.org/thalia/university/gatech"
as virtrdf:Gatech-University ;
skos:subject "http://purl.org/subject/thalia/ComputerScience"
as virtrdf:Gatech-Subject
.
th:GatechInstructor (gatechs.Department, gatechs.Code, gatechs.Section)
a th:Instructor
as virtrdf:Gatech-Instructor ;
foaf:name gatechs.Instructor
as virtrdf:Gatech-InstructorName
.
th:GatechLecture (gatechs.Department, gatechs.Code, gatechs.Section)
a event:Event
as virtrdf:Gatech-Lecture ;
event:time th:GatechEventTime(gatechs.Department, gatechs.Code, gatechs.Section)
as virtrdf:Gatech-hasEventTime ;
event:place th:GatechPlace(gatechs.Department, gatechs.Code, gatechs.Section)
as virtrdf:Gatech-hasPlace
.
th:GatechEventTime (gatechs.Department, gatechs.Code, gatechs.Section)
a time:Interval
as virtrdf:Gatech-EventTime ;
time:inDateTime th:GatechDatetime(gatechs.Department, gatechs.Code, gatechs.Section)
as virtrdf:Gatech-inDateTime
.
th:GatechDatetime (gatechs.Department, gatechs.Code, gatechs.Section)
a time:DateTimeDescription
as virtrdf:Gatech-Datetime ;
time:dayOfWeek gatechs.Days
as virtrdf:Gatech-Days ;
time:hour gatechs.Time_
as virtrdf:Gatech-Time_
.
th:GatechPlace (gatechs.Department, gatechs.Code, gatechs.Section)
a geo:Point
as virtrdf:Gatech-Place ;
dc:title gatechs.Place
as virtrdf:Gatech-RoomBuilding
.
th:Toronto (torontos.No_)
a th:Course
as virtrdf:Toronto-Course ;
dc:title torontos.title
as virtrdf:Toronto-Title ;
dc:description torontos.text_
as virtrdf:Toronto-Description ;
th:hasInstructor th:TorontoInstructor(torontos.No_)
as virtrdf:Toronto-hasInstructor ;
th:hasLecture th:TorontoLecture(torontos.No_)
as virtrdf:Toronto-hasLecture ;
rdfs:seeAlso torontos.coursewebsite
as virtrdf:Toronto-CourseWebSite ;
th:hasPrerequisite torontos.prereq
as virtrdf:Toronto-prereq ;
th:text torontos.text_
as virtrdf:Toronto-text;
th:forUniversity "http://purl.org/thalia/university/toronto"
as virtrdf:Toronto-University ;
skos:subject "http://purl.org/subject/thalia/ComputerScience"
as virtrdf:Toronto-Subject
.
th:TorontoInstructor (torontos.No_)
a th:Instructor
as virtrdf:Toronto-Instructor ;
foaf:name torontos.instructorName
as virtrdf:Toronto-InstructorName ;
foaf:mbox torontos.instructorEmail
as virtrdf:Toronto-InstructorEmail
.
th:TorontoLecture (torontos.No_)
a event:Event
as virtrdf:Toronto-Lecture ;
event:place th:TorontoPlace(torontos.No_)
as virtrdf:Toronto-hasPlace
.
th:TorontoPlace (torontos.No_)
a geo:Point
as virtrdf:Toronto-Place ;
dc:title torontos.location
as virtrdf:Toronto-Location
.
th:Ucsd (ucsds.Number)
a th:Course
as virtrdf:Ucsd-Course ;
dc:title ucsds.Title
as virtrdf:Ucsd-Title ;
th:hasInstructor1 th:UcsdInstructor1 (ucsds.Number)
as virtrdf:Ucsd-hasInstructor1 ;
th:hasInstructor2 th:UcsdInstructor2 (ucsds.Number)
as virtrdf:Ucsd-hasInstructor2 ;
th:hasInstructor3 th:UcsdInstructor3 (ucsds.Number)
as virtrdf:Ucsd-hasInstructor3 ;
th:forUniversity "http://purl.org/thalia/university/ucsd"
as virtrdf:Ucsd-University ;
skos:subject "http://purl.org/subject/thalia/ComputerScience"
as virtrdf:Ucsd-Subject
.
th:UcsdInstructor1 (ucsds.Number)
a th:Instructor
as virtrdf:Ucsd-Instructor1 ;
foaf:name ucsds.Fall2003
as virtrdf:Ucsd-Instructor-Fall2003
.
th:UcsdInstructor2 (ucsds.Number)
a th:Instructor
as virtrdf:Ucsd-Instructor2 ;
foaf:name ucsds.Winter2004
as virtrdf:Ucsd-Instructor-Winter2004
.
th:UcsdInstructor3 (ucsds.Number)
a th:Instructor
as virtrdf:Ucsd-Instructor3 ;
foaf:name ucsds.Spring2004
as virtrdf:Ucsd-Instructor-Spring2004
.
th:Umd (umds.Code)
a th:Course
as virtrdf:Umd-Course ;
dc:title umds.CourseName
as virtrdf:Umd-Title ;
th:hasSection th:SectionTitle
as virtrdf:Umd-hasSection ;
th:hasLecture th:UmdLecture(umds.Code)
as virtrdf:Umd-hasLecture ;
th:forUniversity "http://purl.org/thalia/university/umd"
as virtrdf:Umd-University ;
skos:subject "http://purl.org/subject/thalia/ComputerScience"
as virtrdf:Umd-Subject
.
th:UmdLecture (umds.Code)
a event:Event
as virtrdf:Umd-Lecture ;
event:time th:UmdEventTime(umds.Code)
as virtrdf:Umd-hasEventTime
.
th:UmdEventTime (umds.Code)
a time:Interval
as virtrdf:Umd-EventTime ;
time:inDateTime th:UmdDatetime(umds.Code)
as virtrdf:Umd-inDateTime
.
th:UmdDatetime (umds.Code)
a time:DateTimeDescription
as virtrdf:Umd-Datetime ;
time:hour umds.SectionTime
as virtrdf:Umd-SectionTime
.
}
}
')
;
DB.DBA.RDF_AUDIT_METADATA (1, '*');
create procedure tut_th_rdf_doc (in path varchar)
{
declare r any;
r := regexp_match ('[^/]*\x24', path);
return r||'#this';
};
create procedure tut_th_html_doc (in path varchar)
{
declare r any;
r := regexp_match ('[^/]*#', path);
return subseq (r, 0, length (r)-1);
};
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'tut_th_rule2',
1,
'(/[^#]*)\x24',
vector('path'),
1,
'/sparql?query=CONSTRUCT+{+%%3Chttp%%3A//^{URIQADefaultHost}^%U%%23this%%3E+%%3Fp+%%3Fo+}+FROM+%%3Chttp%%3A//^{URIQADefaultHost}^/Thalia%%3E+WHERE+{+%%3Chttp%%3A//^{URIQADefaultHost}^%U%%23this%%3E+%%3Fp+%%3Fo+}&format=%U',
vector('path', 'path', '*accept*'),
null,
'(text/rdf.n3)|(application/rdf.xml)',
0,
303
);
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'tut_th_rule1',
1,
'(/[^#]*)\x24',
vector('path'),
1,
'/isparql/execute.html?query=SELECT%%20%%3Fp%%20%%3Fo%%20FROM%%20%%3Chttp%%3A//^{URIQADefaultHost}^/Thalia%%3E%%20WHERE%%20{%%20%%3Chttp%%3A//^{URIQADefaultHost}^%U%%23this%%3E%%20%%3Fp%%20%%3Fo%%20}&endpoint=/sparql',
vector('path'),
null,
'(text/html)|(\\*/\\*)',
0,
303
);
DB.DBA.URLREWRITE_CREATE_RULELIST (
'tut_th_rule_list1',
1,
vector (
'tut_th_rule1',
'tut_th_rule2'
));
DB.DBA.VHOST_REMOVE (lpath=>'/thalia');
DB.DBA.VHOST_DEFINE (lpath=>'/thalia', ppath=>'/DAV/Thalia/', is_dav=>1, vsp_user=>'dba', is_brws=>0, opts=>vector ('url_rewrite', 'tut_th_rule_list1'));
DB.DBA.XML_SET_NS_DECL ('th', 'http://purl.org/ontology/thalia/1.0/', 2);
* Demo : Thalia test queries
#service:/sparql
#should-sponge:soft
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX time: <http://www.w3.org/2006/time#>
PREFIX event: <http://purl.org/NET/c4dm/event.owl#>
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX th: <http://purl.org/ontology/thalia/1.0/>
SELECT distinct ?room
FROM <http://demo.openlinksw.com/thalia>
WHERE
{
?course a th:Course;
dc:title ?title;
th:hasLecture ?lecture.
?lecture event:place [dc:title ?room].
FILTER regex(?title, "Software Engineering")
}
#service:/sparql
#should-sponge:soft
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX time: <http://www.w3.org/2006/time#>
PREFIX event: <http://purl.org/NET/c4dm/event.owl#>
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX th: <http://purl.org/ontology/thalia/1.0/>
SELECT ?day, ?hour ?course
FROM <http://demo.openlinksw.com/thalia>
WHERE
{
?course a th:Course;
th:hasLecture [event:time ?time];
dc:title ?title.
?time time:inDateTime [time:dayOfWeek ?day];
time:inDateTime [time:hour ?hour].
FILTER regex(?title, "Computer Networks")
}
#service:/sparql
#should-sponge:soft
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX time: <http://www.w3.org/2006/time#>
PREFIX event: <http://purl.org/NET/c4dm/event.owl#>
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX th: <http://purl.org/ontology/thalia/1.0/>
SELECT distinct ?course
FROM <http://demo.openlinksw.com/thalia>
WHERE
{
?course a th:Course;
dc:title ?Title;
th:hasLecture ?lecture.
?lecture event:time [time:inDateTime ?dateTime].
?dateTime time:hour ?hour.
FILTER regex(?Title, "Database System")
FILTER regex(?hour, "1:30 - 2:50")
}
#service:/sparql
#should-sponge:soft
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX time: <http://www.w3.org/2006/time#>
PREFIX event: <http://purl.org/NET/c4dm/event.owl#>
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX th: <http://purl.org/ontology/thalia/1.0/>
SELECT distinct ?course ?instructor ?name
FROM <http://demo.openlinksw.com/thalia>
WHERE
{
?course a th:Course;
th:hasInstructor ?instructor.
?instructor foaf:name ?name.
}
#service:/sparql
#should-sponge:soft
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX time: <http://www.w3.org/2006/time#>
PREFIX event: <http://purl.org/NET/c4dm/event.owl#>
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX th: <http://purl.org/ontology/thalia/1.0/>
SELECT distinct ?instructor
FROM <http://demo.openlinksw.com/thalia>
WHERE
{
?course a th:Course;
th:hasInstructor ?instructor;
dc:title ?title.
FILTER regex(?title, "Database")
}
#service:/sparql
#should-sponge:soft
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX time: <http://www.w3.org/2006/time#>
PREFIX event: <http://purl.org/NET/c4dm/event.owl#>
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX th: <http://purl.org/ontology/thalia/1.0/>
SELECT distinct ?instructor
FROM <http://demo.openlinksw.com/thalia>
WHERE
{
?course a th:Course;
dc:title ?title;
th:hasInstructor ?instructor.
FILTER regex(?title, "Software")
}
#service:/sparql
#should-sponge:soft
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX time: <http://www.w3.org/2006/time#>
PREFIX event: <http://purl.org/NET/c4dm/event.owl#>
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX th: <http://purl.org/ontology/thalia/1.0/>
SELECT DISTINCT ?course
FROM <http://demo.openlinksw.com/thalia>
WHERE
{
?course a th:Course;
dc:title ?title;
th:forUniversity 'http://purl.org/thalia/university/umd'.
FILTER regex(?title, "Data Structures")
}
#service:/sparql
#should-sponge:soft
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX time: <http://www.w3.org/2006/time#>
PREFIX event: <http://purl.org/NET/c4dm/event.owl#>
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX th: <http://purl.org/ontology/thalia/1.0/>
SELECT distinct ?course
FROM <http://demo.openlinksw.com/thalia>
WHERE
{
?course a th:Course;
dc:title ?Title;
th:hasUnits ?credits.
FILTER (xsd:integer(?credits) > 10)
FILTER regex(?Title, "Database")
}
#service:/sparql
#should-sponge:soft
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX time: <http://www.w3.org/2006/time#>
PREFIX event: <http://purl.org/NET/c4dm/event.owl#>
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX th: <http://purl.org/ontology/thalia/1.0/>
SELECT distinct ?course
FORM <http://demo.openlinksw.com/thalia>
WHERE
{
?course a th:Course;
dc:title ?title;
th:forUniversity 'http://purl.org/thalia/university/umd'.
FILTER regex(?title, "Database")
}
#service:/sparql
#should-sponge:soft
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX time: <http://www.w3.org/2006/time#>
PREFIX event: <http://purl.org/NET/c4dm/event.owl#>
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX th: <http://purl.org/ontology/thalia/1.0/>
SELECT distinct ?course
FROM <http://demo.openlinksw.com/thalia>
WHERE
{
?course a th:Course;
dc:title ?Title;
th:hasUnits ?credits.
FILTER (xsd:integer(?credits) > 10)
FILTER regex(?Title, "Database")
}
#service:/sparql
#should-sponge:soft
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX time: <http://www.w3.org/2006/time#>
PREFIX event: <http://purl.org/NET/c4dm/event.owl#>
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX th: <http://purl.org/ontology/thalia/1.0/>
SELECT distinct ?text_
FROM <http://demo.openlinksw.com/thalia>
WHERE
{
?course a th:Course;
dc:title ?title;
th:text ?text_.
FILTER regex(?title, "Verification")
}
#service:/sparql
#should-sponge:soft
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX time: <http://www.w3.org/2006/time#>
PREFIX event: <http://purl.org/NET/c4dm/event.owl#>
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX : <http://purl.org/ontology/thalia/1.0/>
SELECT distinct ?course
FROM <http://demo.openlinksw.com/thalia>
WHERE
{
?course a th:Course;
dc:description ?description;
th:forUniversity 'http://purl.org/thalia/university/gatech'.
FILTER regex(?description, "JR")
}
The following code creates the Musicbrainz RDF Views Deployment and Demo Scripts:
create text index on ZITGIST.MO.artist ("name") with key id;
create text index on ZITGIST.MO.artistalias ("name") with key id;
create text index on ZITGIST.MO.album ("name") with key id;
create text index on ZITGIST.MO.track ("name") with key id;
vt_batch_update (fix_identifier_case ('ZITGIST.MO.artist'), 'ON', NULL);
vt_batch_update (fix_identifier_case ('ZITGIST.MO.artistalias'), 'ON', NULL);
vt_batch_update (fix_identifier_case ('ZITGIST.MO.album'), 'ON', NULL);
vt_batch_update (fix_identifier_case ('ZITGIST.MO.track'), 'ON', NULL);
VT_INC_INDEX_DB_MO_artist ();
VT_INC_INDEX_DB_MO_artistalias ();
VT_INC_INDEX_DB_MO_album ();
VT_INC_INDEX_DB_MO_track ();
Note: Making sure that the graphs and views are deleting to clean Virtuoso from the old definitions
SPARQL drop quad storage virtrdf:MBZROOT. ; SPARQL prefix mbz: <http://musibrainz.org/schemas/mbz#> drop literal class mbz:duration ; SPARQL prefix mbz: <http://musibrainz.org/schemas/mbz#> drop literal class mbz:created. drop literal class mbz:official_iri. drop literal class mbz:bootleg_iri. drop literal class mbz:promotion_iri. drop literal class mbz:album_iri. drop literal class mbz:single_iri. drop literal class mbz:ep_iri. drop literal class mbz:compilation_iri. drop literal class mbz:soundtrack_iri. drop literal class mbz:spokenword_iri. drop literal class mbz:interview_iri. drop literal class mbz:audiobook_iri. drop literal class mbz:live_iri. drop literal class mbz:remix_iri. ;
The following SPARQL query will fix an issue Virtuoso has with its JSO system. Perform this query for now, the issue should be fixed in a future release
SPARQL define input:storage ""
DELETE FROM GRAPH (iri(bif:JSO_SYS_GRAPH NIL)) { ?s virtrdf:version ?o }
WHERE { graph `iri(bif:JSO_SYS_GRAPH NIL)` {?s virtrdf:version ?o}};
SPARQL_RELOAD_QM_GRAPH();
Creation of IRIs classes.
SPARQL
prefix mbz: <http://musibrainz.org/schemas/mbz#>
prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix dc: <http://purl.org/dc/elements/1.1/>
prefix bio: <http://vocab.org/bio/0.1/#>
prefix rel: <http://vocab.org/relationship/#>
prefix mo: <http://purl.org/ontology/mo/>
prefix timeline: <http://purl.org/NET/c4dm/timeline.owl#>
prefix event: <http://purl.org/NET/c4dm/event.owl#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix sim: <http://purl.org/ontology/sim/>
create iri class mbz:artist_iri "http://zitgist.com/music/artist/%U" (in gid varchar not null) option (bijection) .
create iri class mbz:artist_birth_event_iri "http://zitgist.com/music/artist/birth/%U" (in gid varchar not null) option (bijection) .
create iri class mbz:artist_death_event_iri "http://zitgist.com/music/artist/death/%U" (in gid varchar not null) option (bijection) .
create iri class mbz:sim_link_iri "http://zitgist.com/music/artist/simlink/%U" (in gid varchar not null) option (bijection) .
#create iri class mbz:band_iri "http://zitgist.com/music/band/%U" (in gid varchar not null) option (bijection) .
#create iri class mbz:band_birth_event_iri "http://zitgist.com/music/band/birth/%U" (in gid varchar not null) option (bijection) .
#create iri class mbz:band_death_event_iri "http://zitgist.com/music/band/death/%U" (in gid varchar not null) option (bijection) .
create iri class mbz:record_iri "http://zitgist.com/music/record/%U" (in gid varchar not null) option (bijection) .
create iri class mbz:performance_iri "http://zitgist.com/music/performance/%U" (in gid varchar not null) option (bijection) .
create iri class mbz:composition_iri "http://zitgist.com/music/composition/%U" (in gid varchar not null) option (bijection) .
create iri class mbz:musicalwork_iri "http://zitgist.com/music/musicalwork/%U" (in gid varchar not null) option (bijection) .
create iri class mbz:sound_iri "http://zitgist.com/music/sound/%U" (in gid varchar not null) option (bijection) .
create iri class mbz:recording_iri "http://zitgist.com/music/recording/%U" (in gid varchar not null) option (bijection) .
create iri class mbz:signal_iri "http://zitgist.com/music/signal/%U" (in gid varchar not null) option (bijection) .
create iri class mbz:track_iri "http://zitgist.com/music/track/%U" (in gid varchar not null) option (bijection) .
create iri class mbz:image_iri "http://ec1.images-amazon.com/images/P/%U.01.MZZZZZZZ.jpg" (in image varchar not null) option (bijection) .
create iri class mbz:amazon_asin_iri "http://amazon.com/exec/obidos/ASIN/%U/searchcom07-20" (in gid varchar not null) option (bijection) .
create literal class mbz:created using
function ZITGIST.MO.RECORD_CREATION_DATE (in datestring varchar) returns varchar,
function ZITGIST.MO.RECORD_CREATION_DATE_INVERSE (in datestring varchar) returns varchar .
create iri class mbz:official_iri using
function ZITGIST.MO.RECORD_ATTRIBUTE_OFFICIAL (in attributes varchar) returns varchar
option (returns 'http://purl.org/ontology/mo/official') .
create iri class mbz:promotion_iri using
function ZITGIST.MO.RECORD_ATTRIBUTE_PROMOTION (in attributes varchar) returns varchar
option (returns 'http://purl.org/ontology/mo/promotion') .
create iri class mbz:bootleg_iri using
function ZITGIST.MO.RECORD_ATTRIBUTE_BOOTLEG (in attributes varchar) returns varchar
option (returns 'http://purl.org/ontology/mo/bootleg') .
create iri class mbz:album_iri using
function ZITGIST.MO.RECORD_ATTRIBUTE_ALBUM (in attributes varchar) returns varchar
option (returns 'http://purl.org/ontology/mo/album') .
create iri class mbz:single_iri using
function ZITGIST.MO.RECORD_ATTRIBUTE_SINGLE (in attributes varchar) returns varchar
option (returns 'http://purl.org/ontology/mo/single') .
create iri class mbz:ep_iri using
function ZITGIST.MO.RECORD_ATTRIBUTE_EP (in attributes varchar) returns varchar
option (returns 'http://purl.org/ontology/mo/ep') .
create iri class mbz:compilation_iri using
function ZITGIST.MO.RECORD_ATTRIBUTE_COMPILATION (in attributes varchar) returns varchar
option (returns 'http://purl.org/ontology/mo/compilation') .
create iri class mbz:soundtrack_iri using
function ZITGIST.MO.RECORD_ATTRIBUTE_SOUNDTRACK (in attributes varchar) returns varchar
option (returns 'http://purl.org/ontology/mo/soundtrack') .
create iri class mbz:spokenword_iri using
function ZITGIST.MO.RECORD_ATTRIBUTE_SPOKENWORD (in attributes varchar) returns varchar
option (returns 'http://purl.org/ontology/mo/spokenword') .
create iri class mbz:interview_iri using
function ZITGIST.MO.RECORD_ATTRIBUTE_INTERVIEW (in attributes varchar) returns varchar
option (returns 'http://purl.org/ontology/mo/interview') .
create iri class mbz:audiobook_iri using
function ZITGIST.MO.RECORD_ATTRIBUTE_AUDIOBOOK (in attributes varchar) returns varchar
option (returns 'http://purl.org/ontology/mo/audiobook') .
create iri class mbz:live_iri using
function ZITGIST.MO.RECORD_ATTRIBUTE_LIVE (in attributes varchar) returns varchar
option (returns 'http://purl.org/ontology/mo/live') .
create iri class mbz:remix_iri using
function ZITGIST.MO.RECORD_ATTRIBUTE_REMIX (in attributes varchar) returns varchar
option (returns 'http://purl.org/ontology/mo/remix') .
create iri class mbz:duration_iri "http://zitgist.com/music/track/duration/%U" (in gid varchar not null) .
create literal class mbz:duration using
function ZITGIST.MO.TRACK_DURATION (in duration integer) returns varchar ,
function ZITGIST.MO.TRACK_DURATION_INVERSE (in durationXSD varchar) returns integer .
create iri class mbz:geoname_country_iri "http://www.geonames.org/countries/#%U" (in country varchar not null) .
create iri class mbz:url_iri "%s" (in url varchar not null) .
create iri class mbz:mbz_release_url_iri "http://musicbrainz.org/release/%s.html" (in mbz_gid varchar not null) .
create iri class mbz:mbz_track_url_iri "http://musicbrainz.org/track/%s.html" (in mbz_gid varchar not null) .
create iri class mbz:mbz_artist_url_iri "http://musicbrainz.org/artist/%s.html" (in mbz_gid varchar not null) .
;
List of functions used to compute some IRI classes:
Note:These functions have been developed to handle some weird user cases of the Musicbrainz data model (like the Attribute column of the album table, etc).
create function ZITGIST.MO.TRACK_DURATION_INVERSE(in durationXSD varchar)
{
return null;
};
create function ZITGIST.MO.TRACK_DURATION(in duration integer)
{
declare minutes, seconds, milliseconds integer;
minutes := ((duration / 1000) / 60);
if(minutes >= 1)
{
minutes := cast(minutes as integer);
}
else
{
minutes := 0;
}
seconds := (duration / 1000) - (minutes * 60);
if(seconds >= 1)
{
seconds := cast(seconds as integer);
}
milliseconds := duration - (seconds * 1000) - (minutes * 60000);
return sprintf('PT%dM%dS', minutes, seconds);
}
;
create function ZITGIST.MO.RECORD_CREATION_DATE(in datestring varchar)
{
return sprintf('%sT00:00:00Z', datestring);
};
create function ZITGIST.MO.RECORD_CREATION_DATE_INVERSE(in datestring varchar)
{
declare pos integer;
pos := locate('T00:00:00Z', datestring) - 1;
return substring(datestring, 1, pos);
};
create function ZITGIST.MO.RECORD_ATTRIBUTE(in attribute integer, in attributes varchar)
{
declare attributes_array any;
attributes_array := split_and_decode(ltrim(rtrim(attributes, '}'), '{'), 0, '\0\0,');
foreach(int attr in attributes_array) do
{
attr := cast(attr as integer);
if(attr = attribute)
{
if(attr = 100) return 'http://purl.org/ontology/mo/official';
if(attr = 101) return 'http://purl.org/ontology/mo/promotion';
if(attr = 102) return 'http://purl.org/ontology/mo/bootleg';
if(attr = 1) return 'http://purl.org/ontology/mo/album';
if(attr = 2) return 'http://purl.org/ontology/mo/single';
if(attr = 3) return 'http://purl.org/ontology/mo/ep';
if(attr = 4) return 'http://purl.org/ontology/mo/compilation';
if(attr = 5) return 'http://purl.org/ontology/mo/soundtrack';
if(attr = 6) return 'http://purl.org/ontology/mo/spokenword';
if(attr = 7) return 'http://purl.org/ontology/mo/interview';
if(attr = 8) return 'http://purl.org/ontology/mo/audiobook';
if(attr = 9) return 'http://purl.org/ontology/mo/live';
if(attr = 10) return 'http://purl.org/ontology/mo/remix';
}
}
return null;
}
;
create function ZITGIST.MO.RECORD_ATTRIBUTE_OFFICIAL(in attributes varchar)
{ return ZITGIST.MO.RECORD_ATTRIBUTE(100, attributes); }
;
create function ZITGIST.MO.RECORD_ATTRIBUTE_PROMOTION(in attributes varchar)
{ return ZITGIST.MO.RECORD_ATTRIBUTE(101, attributes);}
;
create function ZITGIST.MO.RECORD_ATTRIBUTE_BOOTLEG(in attributes varchar)
{ return ZITGIST.MO.RECORD_ATTRIBUTE(102, attributes);}
;
create function ZITGIST.MO.RECORD_ATTRIBUTE_ALBUM(in attributes varchar)
{ return ZITGIST.MO.RECORD_ATTRIBUTE(1, attributes);}
;
create function ZITGIST.MO.RECORD_ATTRIBUTE_SINGLE(in attributes varchar)
{ return ZITGIST.MO.RECORD_ATTRIBUTE(2, attributes);}
;
create function ZITGIST.MO.RECORD_ATTRIBUTE_EP(in attributes varchar)
{ return ZITGIST.MO.RECORD_ATTRIBUTE(3, attributes);}
;
create function ZITGIST.MO.RECORD_ATTRIBUTE_COMPILATION(in attributes varchar)
{ return ZITGIST.MO.RECORD_ATTRIBUTE(4, attributes);}
;
create function ZITGIST.MO.RECORD_ATTRIBUTE_SOUNDTRACK(in attributes varchar)
{ return ZITGIST.MO.RECORD_ATTRIBUTE(5, attributes);}
;
create function ZITGIST.MO.RECORD_ATTRIBUTE_SPOKENWORD(in attributes varchar)
{ return ZITGIST.MO.RECORD_ATTRIBUTE(6, attributes);}
;
create function ZITGIST.MO.RECORD_ATTRIBUTE_INTERVIEW(in attributes varchar)
{ return ZITGIST.MO.RECORD_ATTRIBUTE(7, attributes);}
;
create function ZITGIST.MO.RECORD_ATTRIBUTE_AUDIOBOOK(in attributes varchar)
{ return ZITGIST.MO.RECORD_ATTRIBUTE(8, attributes);}
;
create function ZITGIST.MO.RECORD_ATTRIBUTE_LIVE(in attributes varchar)
{ return ZITGIST.MO.RECORD_ATTRIBUTE(9, attributes);}
;
create function ZITGIST.MO.RECORD_ATTRIBUTE_REMIX(in attributes varchar)
{ return ZITGIST.MO.RECORD_ATTRIBUTE(10, attributes);}
;
Definition of the quad map patterns
This what creates the RDF triples from the musicbrainz relational database schema.
SPARQL
prefix mbz: <http://musibrainz.org/schemas/mbz#>
prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix dc: <http://purl.org/dc/elements/1.1/>
prefix dcterms: <http://purl.org/dc/terms/>
prefix bio: <http://vocab.org/bio/0.1/#>
prefix rel: <http://vocab.org/relationship/#>
prefix mo: <http://purl.org/ontology/mo/>
prefix timeline: <http://purl.org/NET/c4dm/timeline.owl#>
prefix event: <http://purl.org/NET/c4dm/event.owl#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix sim: <http://purl.org/ontology/sim/>
create quad storage virtrdf:MBZROOT
#
# Definition of the source tables from the mbz relational database and their joints.
#########
FROM ZITGIST.MO.track as track text literal name
FROM ZITGIST.MO.artist as track_artist
FROM ZITGIST.MO.puid as track_puid
FROM ZITGIST.MO.track as track_track
FROM ZITGIST.MO.url as track_url
FROM ZITGIST.MO.artist as track_artist_creator where (^{track.}^.artist = ^{track_artist_creator.}^.id)
FROM ZITGIST.MO.albumjoin as track_albumjoin where (^{track.}^.id = ^{track_albumjoin.}^.track)
FROM ZITGIST.MO.l_artist_track as l_artist_track2 where (^{track.}^.id = ^{l_artist_track2.}^.link1)
where (^{track_artist.}^.id = ^{l_artist_track2.}^.link0)
FROM ZITGIST.MO.puidjoin as puidjoin where (^{track.}^.id = ^{puidjoin.}^.track)
where (^{puidjoin.}^.puid = ^{track_puid.}^.id)
FROM ZITGIST.MO.l_track_track as l_track_track where (^{track.}^.id = ^{l_track_track.}^.link0)
where (^{track_track.}^.id = ^{l_track_track.}^.link1)
FROM ZITGIST.MO.l_track_url as l_track_url where (^{track.}^.id = ^{l_track_url.}^.link0)
where (^{track_url.}^.id = ^{l_track_url.}^.link1)
FROM ZITGIST.MO.album as album text literal name
FROM ZITGIST.MO.artist as album_artist
FROM ZITGIST.MO.album as album_album
FROM ZITGIST.MO.url as album_url
FROM ZITGIST.MO.country as album_release_country
FROM ZITGIST.MO.track as album_albumjoin_track
FROM ZITGIST.MO.artist as album_artist_creator where (^{album.}^.artist = ^{album_artist_creator.}^.id)
FROM ZITGIST.MO.album_amazon_asin as album_amazon_asin where (^{album.}^.id = ^{album_amazon_asin.}^.album)
FROM ZITGIST.MO.albumjoin as album_albumjoin where (^{album.}^.id = ^{album_albumjoin.}^.album)
where (^{album_albumjoin.}^.track = ^{album_albumjoin_track.}^.id)
FROM ZITGIST.MO.l_album_artist as l_album_artist2 where (^{album.}^.id = ^{l_album_artist2.}^.link0)
where (^{album_artist.}^.id = ^{l_album_artist2.}^.link1)
FROM ZITGIST.MO.l_album_album as l_album_album where (^{album.}^.id = ^{l_album_album.}^.link0)
where (^{album_album.}^.id = ^{l_album_album.}^.link1)
FROM ZITGIST.MO.l_album_url as l_album_url where (^{album.}^.id = ^{l_album_url.}^.link0)
where (^{album_url.}^.id = ^{l_album_url.}^.link1)
FROM ZITGIST.MO.release as album_release where (^{album.}^.id = ^{album_release.}^.album)
where (^{album_release.}^.country = ^{album_release_country.}^.id)
FROM ZITGIST.MO.artist as sim_band
FROM ZITGIST.MO.artist as sim_artist
FROM ZITGIST.MO.url as band_url
FROM ZITGIST.MO.artist as band_member
FROM ZITGIST.MO.album as band_album
FROM ZITGIST.MO.track as band_track
FROM ZITGIST.MO.artist as band text literal name where (^{band.}^.type = 2)
#FROM ZITGIST.MO.artist as artist text literal name where (^{artist.}^.type <> 2)
FROM ZITGIST.MO.artist as artist text literal name where (__or (neq(^{artist.}^.type, 2), isnull (^{artist.}^.type)))
FROM ZITGIST.MO.artist as artist_untyped text literal name where (^{artist_untyped.}^.type <> 2)
where (^{artist.}^.gid = ^{artist_untyped.}^.gid)
FROM ZITGIST.MO.album as band_album_creatorOf where (^{band_album_creatorOf.}^.artist = ^{band.}^.id)
FROM ZITGIST.MO.track as band_track_creatorOf where (^{band_track_creatorOf.}^.artist = ^{band.}^.id)
FROM ZITGIST.MO.artistalias as bandalias text literal name where (^{band.}^.id = ^{bandalias.}^."ref")
FROM ZITGIST.MO.l_artist_artist as band_l_artist_artist where (^{band_member.}^.id = ^{band_l_artist_artist.}^.link0)
where (^{band.}^.id = ^{band_l_artist_artist.}^.link1)
where (^{band_l_artist_artist.}^.link_type = 2)
FROM ZITGIST.MO.artist_relation as band_relation
where (^{artist.}^.id = ^{band_relation.}^.artist)
where (^{band.}^.id = ^{band_relation.}^.artist)
where (^{sim_band.}^.id = ^{band_relation.}^."ref")
FROM ZITGIST.MO.artist_relation as artist_relation
where (^{artist.}^.id = ^{artist_relation.}^.artist)
where (^{band.}^.id = ^{artist_relation.}^.artist)
where (^{sim_artist.}^.id = ^{artist_relation.}^."ref")
FROM ZITGIST.MO.l_artist_url as l_artist_url3 where (^{band.}^.id = ^{l_artist_url3.}^.link0)
where (^{band_url.}^.id = ^{l_artist_url3.}^.link1)
FROM ZITGIST.MO.l_album_artist as l_album_artist3 where (^{band.}^.id = ^{l_album_artist3.}^.link1)
where (^{band_album.}^.id = ^{l_album_artist3.}^.link0)
FROM ZITGIST.MO.l_artist_track as l_artist_track3 where (^{band.}^.id = ^{l_artist_track3.}^.link0)
where (^{band_track.}^.id = ^{l_artist_track3.}^.link1)
FROM ZITGIST.MO.url as artist_url
FROM ZITGIST.MO.artist as artist_artist
FROM ZITGIST.MO.track as artist_track
FROM ZITGIST.MO.album as artist_album
FROM ZITGIST.MO.album as artist_album_creatorOf where (^{artist_album_creatorOf.}^.artist = ^{artist.}^.id)
FROM ZITGIST.MO.track as artist_track_creatorOf where (^{artist_track_creatorOf.}^.artist = ^{artist.}^.id)
FROM ZITGIST.MO.artistalias as artistalias text literal name where (^{artist.}^.id = ^{artistalias.}^."ref")
FROM ZITGIST.MO.l_artist_url as l_artist_url where (^{artist.}^.id = ^{l_artist_url.}^.link0)
where (^{artist_url.}^.id = ^{l_artist_url.}^.link1)
FROM ZITGIST.MO.l_artist_artist as l_artist_artist where (^{artist.}^.id = ^{l_artist_artist.}^.link0)
where (^{artist_artist.}^.id = ^{l_artist_artist.}^.link1)
FROM ZITGIST.MO.l_artist_track as l_artist_track where (^{artist.}^.id = ^{l_artist_track.}^.link0)
where (^{artist_track.}^.id = ^{l_artist_track.}^.link1)
FROM ZITGIST.MO.l_album_artist as l_album_artist where (^{artist.}^.id = ^{l_album_artist.}^.link1)
where (^{artist_album.}^.id = ^{l_album_artist.}^.link0)
{
create virtrdf:MBZ as graph iri ("http://musicbrainz.org/") option (exclusive)
{
# Track Composition Event
mbz:composition_iri (track.gid)
a mo:Composition as mbz:track_is_composition;
dc:title track.name as mbz:title_of_track;
mo:composer mbz:artist_iri (track_artist_creator.gid) as mbz:creator_composer_of_track;
mo:composer mbz:artist_iri (track_artist.gid) where (^{l_artist_track2.}^.link_type = 14) option (using l_artist_track2) as mbz:composer14_of_track;
mo:producesWork mbz:musicalwork_iri (track.gid) as mbz:track_producesWork.
# Track Musical Work
mbz:musicalwork_iri (track.gid)
a mo:MusicalWork as mbz:track_is_mw;
dc:title track.name as mbz:name_of_mw;
mo:productOfComposition mbz:composition_iri(track.gid) as mbz:mw_is_productOfComposition_of;
mo:usedInPerformance mbz:performance_iri(track.gid) as mbz:mw_usedInPerformance.
# Track Performance Event
mbz:performance_iri (track.gid)
a mo:Performance;
dc:title track.name;
mo:performer mbz:artist_iri (track_artist_creator.gid);
mo:performer mbz:artist_iri (track_artist.gid) where (^{l_artist_track2.}^.link_type = 2) option (using l_artist_track2);
mo:conductor mbz:artist_iri (track_artist.gid) where (^{l_artist_track2.}^.link_type = 9) option (using l_artist_track2);
mo:usesWork mbz:musicalwork_iri (track.gid);
mo:producesSound mbz:sound_iri (track.gid);
mo:recordedAs mbz:signal_iri(track.gid).
# Track Sound
mbz:sound_iri (track.gid)
a mo:Sound;
dc:title track.name;
mo:productOfPerformance mbz:performance_iri (track.gid);
mo:usedInRecording mbz:recording_iri (track.gid).
# Track Recording Event
mbz:recording_iri (track.gid)
a mo:Recording;
dc:title track.name;
mo:recordsSound mbz:sound_iri (track.gid);
mo:producesSignal mbz:signal_iri (track.gid).
# Track Signal (Musical Expression)
mbz:signal_iri (track.gid)
a mo:Signal;
dc:title track.name;
mo:remixer mbz:artist_iri (track_artist.gid) where (^{l_artist_track2.}^.link_type = 11) option (using l_artist_track2);
mo:sampler mbz:artist_iri (track_artist.gid) where (^{l_artist_track2.}^.link_type = 12) option (using l_artist_track2);
mo:djmixed mbz:artist_iri (track_artist.gid) where (^{l_artist_track2.}^.link_type = 40) option (using l_artist_track2);
mo:djmix_of mbz:track_iri (track_track.gid) where (^{l_track_track.}^.link_type = 13) option (using l_track_track);
mo:remix_of mbz:track_iri (track_track.gid) where (^{l_track_track.}^.link_type = 6) option (using l_track_track);
mo:remix_of mbz:track_iri (track_track.gid) where (^{l_track_track.}^.link_type = 11) option (using l_track_track);
mo:mashup_of mbz:track_iri (track_track.gid) where (^{l_track_track.}^.link_type = 8) option (using l_track_track);
mo:mashup_of mbz:track_iri (track_track.gid) where (^{l_track_track.}^.link_type = 4) option (using l_track_track);
mo:remaster_of mbz:track_iri (track_track.gid) where (^{l_track_track.}^.link_type = 3) option (using l_track_track);
mo:compilation_of mbz:track_iri (track_track.gid) where (^{l_track_track.}^.link_type = 10) option (using l_track_track);
mo:compilation_of mbz:track_iri (track_track.gid) where (^{l_track_track.}^.link_type = 12) option (using l_track_track);
mo:medley_of mbz:record_iri (track_track.gid) where (^{l_track_track.}^.link_type = 14) option (using l_track_track);
mo:published_as mbz:track_iri (track.gid);
mo:signalTime mbz:duration_iri(track.gid);
mo:puid track_puid.puid option (using puidjoin).
# Track duration
mbz:duration_iri(track.gid)
a timeline:Interval;
timeline:durationXSD mbz:duration(track.length).
mbz:track_iri(track.gid)
a mo:Track;
dc:title track.name;
mo:trackNum track_albumjoin.sequence;
dc:creator mbz:artist_iri (track_artist_creator.gid);
dc:creator mbz:artist_iri (track_artist.gid) where (^{l_artist_track2.}^.link_type = 14) option (using l_artist_track2);
mo:compiler mbz:artist_iri (track_artist.gid) where (^{l_artist_track2.}^.link_type = 39) option (using l_artist_track2);
mo:producer mbz:artist_iri (track_artist.gid) where (^{l_artist_track2.}^.link_type = 18) option (using l_artist_track2);
mo:publisher mbz:artist_iri (track_artist.gid) where (^{l_artist_track2.}^.link_type = 35) option (using l_artist_track2);
mo:engineer mbz:artist_iri (track_artist.gid) where (^{l_artist_track2.}^.link_type = 19) option (using l_artist_track2);
mo:licence mbz:url_iri(track_url.url) where (^{l_track_url.}^.link_type = 21) option (using l_track_url);
mo:paiddownload mbz:url_iri(track_url.url) where (^{l_track_url.}^.link_type = 16) option (using l_track_url);
mo:freedownload mbz:url_iri(track_url.url) where (^{l_track_url.}^.link_type = 17) option (using l_track_url);
mo:olga mbz:url_iri(track_url.url) where (^{l_track_url.}^.link_type = 19) option (using l_track_url);
mo:musicbrainz mbz:mbz_track_url_iri(track.gid);
mo:duration track.length.
# Record Composition Event
mbz:composition_iri (album.gid)
a mo:Composition;
dc:title album.name;
mo:composer mbz:artist_iri (album_artist_creator.gid);
mo:composer mbz:artist_iri (album_artist.gid) where (^{l_album_artist2.}^.link_type = 14) option (using l_album_artist2);
mo:producesWork mbz:musicalwork_iri (album.gid).
# Record Musical Work
mbz:musicalwork_iri (album.gid)
a mo:MusicalWork;
dc:title album.name;
mo:productOfComposition mbz:composition_iri(album.gid);
mo:usedInPerformance mbz:performance_iri(album.gid).
# Record Performance Event
mbz:performance_iri (album.gid)
a mo:Performance;
dc:title album.name;
mo:performer mbz:artist_iri (album_artist_creator.gid);
mo:performer mbz:artist_iri (album_artist.gid) where (^{l_album_artist2.}^.link_type = 2) option (using l_album_artist2);
mo:conductor mbz:artist_iri (album_artist.gid) where (^{l_album_artist2.}^.link_type = 9) option (using l_album_artist2);
mo:usesWork mbz:musicalwork_iri (album.gid);
mo:producesSound mbz:sound_iri (album.gid);
mo:recordedAs mbz:record_iri(album.gid).
# Record Sound
mbz:sound_iri (album.gid)
a mo:Sound;
dc:title album.name;
mo:productOfPerformance mbz:performance_iri (album.gid);
mo:usedInRecording mbz:recording_iri (album.gid).
# Record Recording Event
mbz:recording_iri (album.gid)
a mo:Recording;
dc:title album.name;
mo:recordsSound mbz:sound_iri (album.gid);
mo:producesSignal mbz:signal_iri (album.gid).
# Record Signal (Musical Expression)
mbz:signal_iri (album.gid)
a mo:Signal;
dc:title album.name;
mo:djmix_of mbz:record_iri (album_album.gid) where (^{l_album_album.}^.link_type = 9) option (using l_album_album);
mo:remix_of mbz:record_iri (album_album.gid) where (^{l_album_album.}^.link_type = 7) option (using l_album_album);
mo:remix_of mbz:record_iri (album_album.gid) where (^{l_album_album.}^.link_type = 4) option (using l_album_album);
mo:mashup_of mbz:record_iri (album_album.gid) where (^{l_album_album.}^.link_type = 5) option (using l_album_album);
mo:remaster_of mbz:record_iri (album_album.gid) where (^{l_album_album.}^.link_type = 3) option (using l_album_album);
mo:tribute_to mbz:artist_iri (album_artist.gid) where (^{l_album_artist2.}^.link_type = 44) option (using l_album_artist2);
mo:remixer mbz:artist_iri (album_artist.gid) where (^{l_album_artist2.}^.link_type = 11) option (using l_album_artist2);
mo:djmixed mbz:artist_iri (album_artist.gid) where (^{l_album_artist2.}^.link_type = 38) option (using l_album_artist2);
mo:sampler mbz:artist_iri (album_artist.gid) where (^{l_album_artist2.}^.link_type = 12) option (using l_album_artist2);
mo:published_as mbz:record_iri (album.gid).
# Record (Musical Manifestation)
mbz:record_iri (album.gid)
a mo:Record;
dc:title album.name;
dc:date mbz:created(album_release.releasedate);
mo:image mbz:image_iri(album_amazon_asin.asin);
#Empty for now.
mo:compilation_of mbz:record_iri (album_album.gid) where (^{l_album_album.}^.link_type = 8) option (using l_album_album);
mo:releaseStatus mbz:official_iri(album.attributes);
mo:releaseStatus mbz:promotion_iri(album.attributes);
mo:releaseStatus mbz:bootleg_iri(album.attributes);
mo:releaseType mbz:album_iri(album.attributes);
mo:releaseType mbz:single_iri(album.attributes);
mo:releaseType mbz:ep_iri(album.attributes);
mo:releaseType mbz:compilation_iri(album.attributes);
mo:releaseType mbz:soundtrack_iri(album.attributes);
mo:releaseType mbz:spokenword_iri(album.attributes);
mo:releaseType mbz:interview_iri(album.attributes);
mo:releaseType mbz:audiobook_iri(album.attributes);
mo:releaseType mbz:live_iri(album.attributes);
mo:releaseType mbz:remix_iri(album.attributes);
dc:creator mbz:artist_iri (album_artist_creator.gid);
dc:creator mbz:artist_iri (album_artist.gid) where (^{l_album_artist2.}^.link_type = 14) option (using l_album_artist2);
mo:compiler mbz:artist_iri (album_artist.gid) where (^{l_album_artist2.}^.link_type = 41) option (using l_album_artist2);
mo:producer mbz:artist_iri (album_artist.gid) where (^{l_album_artist2.}^.link_type = 18) option (using l_album_artist2);
mo:publisher mbz:artist_iri (album_artist.gid) where (^{l_album_artist2.}^.link_type = 35) option (using l_album_artist2);
mo:engineer mbz:artist_iri (album_artist.gid) where (^{l_album_artist2.}^.link_type = 19) option (using l_album_artist2);
mo:musicbrainz mbz:mbz_release_url_iri(album.gid);
mo:musicmoz mbz:url_iri(album_url.url) where (^{l_album_url.}^.link_type = 25) option (using l_album_url);
mo:discogs mbz:url_iri(album_url.url) where (^{l_album_url.}^.link_type = 24) option (using l_album_url);
mo:wikipedia mbz:url_iri(album_url.url) where (^{l_album_url.}^.link_type = 23) option (using l_album_url);
mo:discography mbz:url_iri(album_url.url) where (^{l_album_url.}^.link_type = 1) option (using l_album_url);
mo:freedownload mbz:url_iri(album_url.url) where (^{l_album_url.}^.link_type = 21) option (using l_album_url);
mo:discography mbz:url_iri(album_url.url) where (^{l_album_url.}^.link_type = 16) option (using l_album_url);
mo:mailorder mbz:url_iri(album_url.url) where (^{l_album_url.}^.link_type = 19) option (using l_album_url);
mo:imdb mbz:url_iri(album_url.url) where (^{l_album_url.}^.link_type = 27) option (using l_album_url);
mo:paiddownload mbz:url_iri(album_url.url) where (^{l_album_url.}^.link_type = 20) option (using l_album_url);
mo:licence mbz:url_iri(album_url.url) where (^{l_album_url.}^.link_type = 32) option (using l_album_url);
mo:review mbz:url_iri(album_url.url) where (^{l_album_url.}^.link_type = 17) option (using l_album_url);
mo:amazon_asin mbz:amazon_asin_iri(album_amazon_asin.asin);
mo:has_track mbz:track_iri (album_albumjoin_track.gid) option (using album_albumjoin).
# Music Group (Band)
# mbz:band_iri(band.gid)
mbz:artist_iri(band.gid)
a mo:MusicArtist;
a mo:MusicGroup;
a foaf:Group;
foaf:name band.name;
foaf:nick bandalias.name;
# bio:event mbz:band_birth_event_iri(band.gid);
# bio:event mbz:band_death_event_iri(band.gid);
bio:event mbz:artist_birth_event_iri(band.gid);
bio:event mbz:artist_death_event_iri(band.gid);
# mo:similar_to mbz:band_iri(sim_band.gid) option (using band_relation);
mo:similar_to mbz:artist_iri(sim_band.gid) option (using band_relation);
mo:similar_to mbz:artist_iri(sim_artist.gid) option (using artist_relation);
# sim:link mbz:sim_link_iri(sim_band.gid) option (using band_relation);
# sim:link mbz:sim_link_iri(sim_artist.gid) option (using artist_relation);
foaf:member mbz:artist_iri(band_member.gid) option (using band_l_artist_artist);
# l_artist_url
mo:myspace mbz:url_iri(band_url.url) where (^{l_artist_url3.}^.link_type = 19) option (using l_artist_url3);
mo:musicmoz mbz:url_iri(band_url.url) where (^{l_artist_url3.}^.link_type = 12) option (using l_artist_url3);
mo:discogs mbz:url_iri(band_url.url) where (^{l_artist_url3.}^.link_type = 11) option (using l_artist_url3);
mo:wikipedia mbz:url_iri(band_url.url) where (^{l_artist_url3.}^.link_type = 10) option (using l_artist_url3);
mo:discography mbz:url_iri(band_url.url) where (^{l_artist_url3.}^.link_type = 1) option (using l_artist_url3);
mo:freedownload mbz:url_iri(band_url.url) where (^{l_artist_url3.}^.link_type = 8) option (using l_artist_url3);
mo:fanpage mbz:url_iri(band_url.url) where (^{l_artist_url3.}^.link_type = 3) option (using l_artist_url3);
mo:biography mbz:url_iri(band_url.url) where (^{l_artist_url3.}^.link_type = 4) option (using l_artist_url3);
mo:discography mbz:url_iri(band_url.url) where (^{l_artist_url3.}^.link_type = 5) option (using l_artist_url3);
mo:mailorder mbz:url_iri(band_url.url) where (^{l_artist_url3.}^.link_type = 15) option (using l_artist_url3);
mo:imdb mbz:url_iri(band_url.url) where (^{l_artist_url3.}^.link_type = 17) option (using l_artist_url3);
mo:paiddownload mbz:url_iri(band_url.url) where (^{l_artist_url3.}^.link_type = 7) option (using l_artist_url3);
foaf:depiction mbz:url_iri(band_url.url) where (^{l_artist_url3.}^.link_type = 14) option (using l_artist_url3);
foaf:homepage mbz:url_iri(band_url.url) where (^{l_artist_url3.}^.link_type = 2) option (using l_artist_url3);
mo:musicbrainz mbz:mbz_artist_url_iri(band.gid);
# l_album_artist
mo:composed mbz:composition_iri (band_album.gid) where (^{l_album_artist3.}^.link_type = 14) option (using l_album_artist3);
mo:performed mbz:performance_iri (band_album.gid) where (^{l_album_artist3.}^.link_type = 14) option (using l_album_artist3);
mo:performed mbz:performance_iri (band_album.gid) where (^{l_album_artist3.}^.link_type = 2) option (using l_album_artist3);
mo:conducted mbz:performance_iri (band_album.gid) where (^{l_album_artist3.}^.link_type = 9) option (using l_album_artist3);
mo:compiled mbz:record_iri (band_album.gid) where (^{l_album_artist3.}^.link_type = 41) option (using l_album_artist3);
mo:djmixed mbz:record_iri (band_album.gid) where (^{l_album_artist3.}^.link_type = 38) option (using l_album_artist3);
mo:remixed mbz:record_iri (band_album.gid) where (^{l_album_artist3.}^.link_type = 11) option (using l_album_artist3);
mo:sampled mbz:record_iri (band_album.gid) where (^{l_album_artist3.}^.link_type = 12) option (using l_album_artist3);
mo:produced mbz:record_iri (band_album.gid) where (^{l_album_artist3.}^.link_type = 18) option (using l_album_artist3);
mo:published mbz:record_iri (band_album.gid) where (^{l_album_artist3.}^.link_type = 35) option (using l_album_artist3);
mo:engineered mbz:record_iri (band_album.gid) where (^{l_album_artist3.}^.link_type = 19) option (using l_album_artist3);
# # mo:creatorOfRecord mbz:record_iri(band_album_creatorOf.gid);
foaf:made mbz:record_iri(band_album_creatorOf.gid);
# l_artist_track
mo:composed mbz:composition_iri (band_track.gid) where (^{l_artist_track3.}^.link_type = 14) option (using l_artist_track3);
mo:performed mbz:performance_iri (band_track.gid) where (^{l_artist_track3.}^.link_type = 14) option (using l_artist_track3);
mo:performed mbz:performance_iri (band_track.gid) where (^{l_artist_track3.}^.link_type = 2) option (using l_artist_track3);
mo:conducted mbz:performance_iri (band_track.gid) where (^{l_artist_track3.}^.link_type = 9) option (using l_artist_track3);
mo:compiled mbz:record_iri (band_track.gid) where (^{l_artist_track3.}^.link_type = 39) option (using l_artist_track3);
mo:djmixed mbz:track_iri (band_track.gid) where (^{l_artist_track3.}^.link_type = 40) option (using l_artist_track3);
mo:remixed mbz:track_iri (band_track.gid) where (^{l_artist_track3.}^.link_type = 11) option (using l_artist_track3);
mo:sampled mbz:track_iri (band_track.gid) where (^{l_artist_track3.}^.link_type = 12) option (using l_artist_track3);
mo:produced mbz:track_iri (band_track.gid) where (^{l_artist_track3.}^.link_type = 18) option (using l_artist_track3);
mo:published mbz:track_iri (band_track.gid) where (^{l_artist_track3.}^.link_type = 35) option (using l_artist_track3);
mo:engineered mbz:track_iri (band_track.gid) where (^{l_artist_track3.}^.link_type = 19) option (using l_artist_track3).
# # mo:creatorOfTrack mbz:track_iri(band_track_creatorOf.gid).
# Music Group (Band)'s Birth Event
# mbz:band_birth_event_iri(band.gid)
mbz:artist_birth_event_iri(band.gid)
a bio:Birth;
bio:date band.begindate.
# Music Group (Band)'s Death Event
# mbz:band_death_event_iri(band.gid)
mbz:artist_death_event_iri(band.gid)
a bio:Death;
bio:date band.enddate.
# Similarity link
#mbz:sim_link_iri(sim_band.gid)
# sim:relation mo:similar_to;
# sim:level band_relation.weight;
# sim:to sim_band.gid.
# Music Artist
mbz:artist_iri (artist.gid)
# artist
a mo:MusicArtist;
a mo:SoloMusicArtist where (^{artist_untyped.}^.gid is not null) option (using artist_untyped);
a foaf:Person where (^{artist_untyped.}^.gid is not null) option (using artist_untyped);
foaf:name artist.name;
foaf:nick artistalias.name;
bio:event mbz:artist_birth_event_iri(artist.gid);
bio:event mbz:artist_death_event_iri(artist.gid);
mo:member_of mbz:artist_iri(artist_artist.gid) where (^{l_artist_artist.}^.link_type = 2) option (using l_artist_artist);
# l_artist_artist
rel:siblingOf mbz:artist_iri(artist_artist.gid) where (^{l_artist_artist.}^.link_type = 7) option (using l_artist_artist);
rel:friendOf mbz:artist_iri(artist_artist.gid) where (^{l_artist_artist.}^.link_type = 5) option (using l_artist_artist);
rel:parentOf mbz:artist_iri(artist_artist.gid) where (^{l_artist_artist.}^.link_type = 6) option (using l_artist_artist);
rel:collaborated_with mbz:artist_iri(artist_artist.gid) where (^{l_artist_artist.}^.link_type = 11) option (using l_artist_artist);
rel:engagedTo mbz:artist_iri(artist_artist.gid) where (^{l_artist_artist.}^.link_type = 9) option (using l_artist_artist);
rel:spouseOf mbz:artist_iri(artist_artist.gid) where (^{l_artist_artist.}^.link_type = 8) option (using l_artist_artist);
mo:supporting_musician mbz:artist_iri(artist_artist.gid) where (^{l_artist_artist.}^.link_type = 13) option (using l_artist_artist);
mo:supporting_musician mbz:artist_iri(artist_artist.gid) where (^{l_artist_artist.}^.link_type = 14) option (using l_artist_artist);
mo:supporting_musician mbz:artist_iri(artist_artist.gid) where (^{l_artist_artist.}^.link_type = 15) option (using l_artist_artist);
mo:similar_to mbz:artist_iri(sim_artist.gid) option (using artist_relation);
# mo:similar_to mbz:band_iri(sim_band.gid) option (using band_relation);
mo:similar_to mbz:artist_iri(sim_band.gid) option (using band_relation);
# sim:link mbz:sim_link_iri(sim_band.gid) option (using band_relation);
# sim:link mbz:sim_link_iri(sim_artist.gid) option (using artist_relation);
# l_artist_url
mo:myspace mbz:url_iri(artist_url.url) where (^{l_artist_url.}^.link_type = 19) option (using l_artist_url);
mo:musicmoz mbz:url_iri(artist_url.url) where (^{l_artist_url.}^.link_type = 12) option (using l_artist_url);
mo:discogs mbz:url_iri(artist_url.url) where (^{l_artist_url.}^.link_type = 11) option (using l_artist_url);
mo:wikipedia mbz:url_iri(artist_url.url) where (^{l_artist_url.}^.link_type = 10) option (using l_artist_url);
mo:discography mbz:url_iri(artist_url.url) where (^{l_artist_url.}^.link_type = 1) option (using l_artist_url);
mo:freedownload mbz:url_iri(artist_url.url) where (^{l_artist_url.}^.link_type = 8) option (using l_artist_url);
mo:fanpage mbz:url_iri(artist_url.url) where (^{l_artist_url.}^.link_type = 3) option (using l_artist_url);
mo:biography mbz:url_iri(artist_url.url) where (^{l_artist_url.}^.link_type = 4) option (using l_artist_url);
mo:discography mbz:url_iri(artist_url.url) where (^{l_artist_url.}^.link_type = 5) option (using l_artist_url);
mo:mailorder mbz:url_iri(artist_url.url) where (^{l_artist_url.}^.link_type = 15) option (using l_artist_url);
mo:imdb mbz:url_iri(artist_url.url) where (^{l_artist_url.}^.link_type = 17) option (using l_artist_url);
mo:paiddownload mbz:url_iri(artist_url.url) where (^{l_artist_url.}^.link_type = 7) option (using l_artist_url);
foaf:depiction mbz:url_iri(artist_url.url) where (^{l_artist_url.}^.link_type = 14) option (using l_artist_url);
foaf:homepage mbz:url_iri(artist_url.url) where (^{l_artist_url.}^.link_type = 2) option (using l_artist_url);
mo:musicbrainz mbz:mbz_artist_url_iri(artist.gid);
# l_album_artist
mo:composed mbz:composition_iri (artist_album.gid) where (^{l_album_artist.}^.link_type = 14) option (using l_album_artist);
mo:performed mbz:performance_iri (artist_album.gid) where (^{l_album_artist.}^.link_type = 14) option (using l_album_artist);
mo:performed mbz:performance_iri (artist_album.gid) where (^{l_album_artist.}^.link_type = 2) option (using l_album_artist);
mo:conducted mbz:performance_iri (artist_album.gid) where (^{l_album_artist.}^.link_type = 9) option (using l_album_artist);
mo:compiled mbz:record_iri (artist_album.gid) where (^{l_album_artist.}^.link_type = 41) option (using l_album_artist);
mo:djmixed mbz:record_iri (artist_album.gid) where (^{l_album_artist.}^.link_type = 38) option (using l_album_artist);
mo:remixed mbz:record_iri (artist_album.gid) where (^{l_album_artist.}^.link_type = 11) option (using l_album_artist);
mo:sampled mbz:record_iri (artist_album.gid) where (^{l_album_artist.}^.link_type = 12) option (using l_album_artist);
mo:produced mbz:record_iri (artist_album.gid) where (^{l_album_artist.}^.link_type = 18) option (using l_album_artist);
mo:published mbz:record_iri (artist_album.gid) where (^{l_album_artist.}^.link_type = 35) option (using l_album_artist);
mo:engineered mbz:record_iri (artist_album.gid) where (^{l_album_artist.}^.link_type = 19) option (using l_album_artist);
# mo:creatorOfRecord mbz:record_iri(artist_album_creatorOf.gid);
foaf:made mbz:record_iri(artist_album_creatorOf.gid);
# l_artist_track
mo:composed mbz:composition_iri (artist_track.gid) where (^{l_artist_track.}^.link_type = 14) option (using l_artist_track);
mo:performed mbz:performance_iri (artist_track.gid) where (^{l_artist_track.}^.link_type = 14) option (using l_artist_track);
mo:performed mbz:performance_iri (artist_track.gid) where (^{l_artist_track.}^.link_type = 2) option (using l_artist_track);
mo:conducted mbz:performance_iri (artist_track.gid) where (^{l_artist_track.}^.link_type = 9) option (using l_artist_track);
mo:compiled mbz:track_iri (artist_track.gid) where (^{l_artist_track.}^.link_type = 39) option (using l_artist_track);
mo:djmixed mbz:track_iri (artist_track.gid) where (^{l_artist_track.}^.link_type = 40) option (using l_artist_track);
mo:remixed mbz:track_iri (artist_track.gid) where (^{l_artist_track.}^.link_type = 11) option (using l_artist_track);
mo:sampled mbz:track_iri (artist_track.gid) where (^{l_artist_track.}^.link_type = 12) option (using l_artist_track);
mo:produced mbz:track_iri (artist_track.gid) where (^{l_artist_track.}^.link_type = 18) option (using l_artist_track);
mo:published mbz:track_iri (artist_track.gid) where (^{l_artist_track.}^.link_type = 35) option (using l_artist_track);
mo:engineered mbz:track_iri (artist_track.gid) where (^{l_artist_track.}^.link_type = 19) option (using l_artist_track).
# mo:creatorOfTrack mbz:track_iri(artist_track_creatorOf.gid).
# Music Artist''s Birth Event
mbz:artist_birth_event_iri(artist.gid)
a bio:Birth;
bio:date artist.begindate.
# Music Artist''s Death Event
mbz:artist_death_event_iri(artist.gid)
a bio:Death;
bio:date artist.enddate.
# Similarity link
#mbz:sim_link_iri(sim_artist.gid)
# sim:relation mo:similar_to;
# sim:level artist_relation.weight;
# sim:to sim_artist.gid.
}
}
;
grant execute on ZITGIST.MO.RECORD_CREATION_DATE to "SPARQL";
grant execute on ZITGIST.MO.RECORD_CREATION_DATE_INVERSE to "SPARQL";
grant execute on ZITGIST.MO.RECORD_ATTRIBUTE_OFFICIAL to "SPARQL";
grant execute on ZITGIST.MO.RECORD_ATTRIBUTE_PROMOTION to "SPARQL";
grant execute on ZITGIST.MO.RECORD_ATTRIBUTE_BOOTLEG to "SPARQL";
grant execute on ZITGIST.MO.RECORD_ATTRIBUTE_ALBUM to "SPARQL";
grant execute on ZITGIST.MO.RECORD_ATTRIBUTE_SINGLE to "SPARQL";
grant execute on ZITGIST.MO.RECORD_ATTRIBUTE_EP to "SPARQL";
grant execute on ZITGIST.MO.RECORD_ATTRIBUTE_COMPILATION to "SPARQL";
grant execute on ZITGIST.MO.RECORD_ATTRIBUTE_SOUNDTRACK to "SPARQL";
grant execute on ZITGIST.MO.RECORD_ATTRIBUTE_SPOKENWORD to "SPARQL";
grant execute on ZITGIST.MO.RECORD_ATTRIBUTE_INTERVIEW to "SPARQL";
grant execute on ZITGIST.MO.RECORD_ATTRIBUTE_AUDIOBOOK to "SPARQL";
grant execute on ZITGIST.MO.RECORD_ATTRIBUTE_LIVE to "SPARQL";
grant execute on ZITGIST.MO.RECORD_ATTRIBUTE_REMIX to "SPARQL";
grant execute on ZITGIST.MO.TRACK_DURATION to "SPARQL";
grant execute on ZITGIST.MO.TRACK_DURATION_INVERSE to "SPARQL";
grant execute on ZITGIST.MO.album_amazon_asin to "SPARQL";
grant execute on ZITGIST.MO.album_name_WORDS to "SPARQL";
grant execute on ZITGIST.MO.albumjoin to "SPARQL";
grant execute on ZITGIST.MO.albummeta to "SPARQL";
grant execute on ZITGIST.MO.artist to "SPARQL";
grant execute on ZITGIST.MO.artist_name_WORDS to "SPARQL";
grant execute on ZITGIST.MO.artist_relation to "SPARQL";
grant execute on ZITGIST.MO.artistalias to "SPARQL";
grant execute on ZITGIST.MO.artistalias_name_WORDS to "SPARQL";
grant execute on ZITGIST.MO.country to "SPARQL";
grant execute on ZITGIST.MO.l_album_album to "SPARQL";
grant execute on ZITGIST.MO.l_album_artist to "SPARQL";
grant execute on ZITGIST.MO.l_album_url to "SPARQL";
grant execute on ZITGIST.MO.l_artist_artist to "SPARQL";
grant execute on ZITGIST.MO.l_artist_track to "SPARQL";
grant execute on ZITGIST.MO.l_artist_url to "SPARQL";
grant execute on ZITGIST.MO.l_track_track to "SPARQL";
grant execute on ZITGIST.MO.l_track_url to "SPARQL";
grant execute on ZITGIST.MO."language" to "SPARQL";
grant execute on ZITGIST.MO.puid to "SPARQL";
grant execute on ZITGIST.MO.puidjoin to "SPARQL";
grant execute on ZITGIST.MO.release to "SPARQL";
grant execute on ZITGIST.MO.track to "SPARQL";
grant execute on ZITGIST.MO.track_name_WORDS to "SPARQL";
grant execute on ZITGIST.MO.url to "SPARQL";
DB.DBA.XML_SET_NS_DECL ('mbz', 'http://musibrainz.org/schemas/mbz#', 2);
RDF View for ODS (the consolidated Graph)
SPARQL drop quad map virtrdf:ODS_DS . ;
SPARQL prefix ods: <http://www.openlinksw.com/virtuoso/ods/>
create iri class ods:graph "http://^{URIQADefaultHost}^/dataspace/%U" (in uname varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U") .
create iri class ods:user "http://^{URIQADefaultHost}^/dataspace/%U#user" (in uname varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U#user") .
create iri class ods:user_group "http://^{URIQADefaultHost}^/dataspace/%U#group" (in uname varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U#group") .
create iri class ods:person "http://^{URIQADefaultHost}^/dataspace/%U#this" (in uname varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U#this") .
create iri class ods:mbox "mailto:%s" (in email varchar not null)
option (returns "mailto:%s") .
create iri class ods:phone "tel:%s" (in tel varchar not null)
option (returns "tel:%s") .
create iri class ods:geo_point "http://^{URIQADefaultHost}^/dataspace/%U#geo" (in uname varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U#geo") .
create iri class ods:forum "http://^{URIQADefaultHost}^/dataspace/%U/%U/%U"
( in uname varchar not null, in forum_type varchar not null, in forum_name varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/%U/%U") .
create iri class ods:proxy "http://^{URIQADefaultHost}^/proxy/%U" (in url varchar not null)
option (returns "http://^{URIQADefaultHost}^/proxy/%U") .
create iri class ods:site "http://^{URIQADefaultHost}^/dataspace/%U#site" (in uname varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U#site") .
create iri class ods:role "http://^{URIQADefaultHost}^/dataspace/%U/%U/%U#%U"
(in uname varchar not null, in tp varchar not null, in inst varchar not null, in role_name varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/%U/%U#%U" ) .
# Blog
create iri class ods:blog_forum "http://^{URIQADefaultHost}^/dataspace/%U/weblog/%U"
(in uname varchar not null, in forum_name varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/weblog/%U") .
create iri class ods:blog_post "http://^{URIQADefaultHost}^/dataspace/%U/weblog/%U/%U"
(in uname varchar not null, in forum_name varchar not null, in postid varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/weblog/%U/%U" ) .
create iri class ods:blog_comment "http://^{URIQADefaultHost}^/dataspace/%U/weblog/%U/%U/%d"
(in uname varchar not null, in forum_name varchar not null, in postid varchar not null, in comment_id int not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/weblog/%U/%U/%d" ) .
create iri class ods:tag "http://^{URIQADefaultHost}^/dataspace/%U/concept#%U"
(in uname varchar not null, in tag varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/concept#%U") .
create iri class ods:blog_post_text "http://^{URIQADefaultHost}^/dataspace/%U/weblog-text/%U/%U"
(in uname varchar not null, in forum_name varchar not null, in postid varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/weblog-text/%U/%U" ) .
#Feeds
create iri class ods:feed "http://^{URIQADefaultHost}^/dataspace/feed/%d" (in feed_id integer not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/feed/%d" ) .
create iri class ods:feed_item "http://^{URIQADefaultHost}^/dataspace/feed/%d/%d" (in feed_id integer not null, in item_id integer not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/feed/%d/%d" ) .
create iri class ods:feed_item_text "http://^{URIQADefaultHost}^/dataspace/feed/%d/%d/text" (in feed_id integer not null, in item_id integer not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/feed/%d/%d/text" ) .
create iri class ods:feed_mgr "http://^{URIQADefaultHost}^/dataspace/%U/feeds/%U" (in uname varchar not null, in inst_name varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/feeds/%U" ) .
create iri class ods:feed_comment "http://^{URIQADefaultHost}^/dataspace/%U/feeds/%U/%d/%d"
(in uname varchar not null, in inst_name varchar not null, in item_id integer not null, in comment_id integer not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/feeds/%U/%d/%d" ) .
#Bookmark
create iri class ods:bmk_post "http://^{URIQADefaultHost}^/dataspace/%U/bookmark/%U/%d"
(in uname varchar not null, in inst_name varchar not null, in bmk_id integer not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/bookmark/%U/%d") .
create iri class ods:bmk_post_text "http://^{URIQADefaultHost}^/dataspace/%U/bookmark/%U/%d/text"
(in uname varchar not null, in inst_name varchar not null, in bmk_id integer not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/bookmark/%U/%d/text" ) .
create iri class ods:bmk_forum "http://^{URIQADefaultHost}^/dataspace/%U/bookmark/%U"
( in uname varchar not null, in forum_name varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/bookmark/%U") .
#Photo
create iri class ods:photo_forum "http://^{URIQADefaultHost}^/dataspace/%U/photos/%U"
(in uname varchar not null, in inst_name varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/photos/%U") .
create iri class ods:photo_post "http://^{URIQADefaultHost}^%s"
(in path varchar not null) option (returns "http://^{URIQADefaultHost}^/DAV/%s") .
create iri class ods:photo_post_text "http://^{URIQADefaultHost}^%s/text"
(in path varchar not null) option (returns "http://^{URIQADefaultHost}^/DAV/%s/text") .
create iri class ods:photo_comment "http://^{URIQADefaultHost}^%s:comment_%d"
(in path varchar not null, in comment_id int not null)
option (returns "http://^{URIQADefaultHost}^/DAV/%s:comment_%d") .
# Community
create iri class ods:community_forum "http://^{URIQADefaultHost}^/dataspace/%U/community/%U"
(in uname varchar not null, in forum_name varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/community/%U") .
# Briefcase
create iri class ods:odrive_forum "http://^{URIQADefaultHost}^/dataspace/%U/briefcase/%U"
(in uname varchar not null, in inst_name varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/briefcase/%U" ) .
create iri class ods:odrive_post "http://^{URIQADefaultHost}^%s"
(in path varchar not null) option (returns "http://^{URIQADefaultHost}^/DAV/%s") .
create iri class ods:odrive_post_text "http://^{URIQADefaultHost}^%s/text"
(in path varchar not null) option (returns "http://^{URIQADefaultHost}^/DAV/%s/text") .
# Wiki
create iri class ods:wiki_post "http://^{URIQADefaultHost}^/dataspace/%U/wiki/%U/%U"
(in uname varchar not null, in inst_name varchar not null, in topic_id varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/wiki/%U/%U") .
create iri class ods:wiki_post_text "http://^{URIQADefaultHost}^/dataspace/%U/wiki/%U/%U/text"
(in uname varchar not null, in inst_name varchar not null, in topic_id varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/wiki/%U/%U/text" ) .
create iri class ods:wiki_forum "http://^{URIQADefaultHost}^/dataspace/%U/wiki/%U"
( in uname varchar not null, in forum_name varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/wiki/%U" ) .
#Calendar
create iri class ods:calendar_event "http://^{URIQADefaultHost}^/dataspace/%U/calendar/%U/%d"
(in uname varchar not null, in inst_name varchar not null, in calendar_id integer not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/calendar/%U/%d" ) .
create iri class ods:calendar_event_text "http://^{URIQADefaultHost}^/dataspace/%U/calendar/%U/%d/text"
(in uname varchar not null, in inst_name varchar not null, in calendar_id integer not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/calendar/%U/%d/text" ) .
create iri class ods:calendar_forum "http://^{URIQADefaultHost}^/dataspace/%U/calendar/%U"
( in uname varchar not null, in forum_name varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/%U/calendar/%U") .
# NNTPF
create iri class ods:nntp_forum "http://^{URIQADefaultHost}^/dataspace/discussion/%U"
( in forum_name varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/discussion/%U").
create iri class ods:nntp_post "http://^{URIQADefaultHost}^/dataspace/discussion/%U/%U"
( in group_name varchar not null, in message_id varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/discussion/%U/%U" ) .
create iri class ods:nntp_post_text "http://^{URIQADefaultHost}^/dataspace/discussion/%U/%U/text"
( in group_name varchar not null, in message_id varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/discussion/%U/%U/text") .
create iri class ods:nntp_role "http://^{URIQADefaultHost}^/dataspace/discussion/%U#reader"
(in forum_name varchar not null)
option (returns "http://^{URIQADefaultHost}^/dataspace/discussion/%U#reader") .
;
SPARQL
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix sioct: <http://rdfs.org/sioc/types#>
prefix atom: <http://atomowl.org/ontologies/atomrdf#>
prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix dc: <http://purl.org/dc/elements/1.1/>
prefix dct: <http://purl.org/dc/terms/>
prefix skos: <http://www.w3.org/2004/02/skos/core#>
prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
prefix bm: <http://www.w3.org/2002/01/bookmark#>
prefix exif: <http://www.w3.org/2003/12/exif/ns/>
prefix ann: <http://www.w3.org/2000/10/annotation-ns#>
prefix wikiont: <http://sw.deri.org/2005/04/wikipedia/wikiont.owl#>
prefix calendar: <http://www.w3.org/2002/12/cal#>
prefix ods: <http://www.openlinksw.com/virtuoso/ods/>
alter quad storage virtrdf:DefaultQuadStorage
FROM DB.DBA.SIOC_USERS as users
FROM DB.DBA.SIOC_ODS_FORUMS as forums
FROM DB.DBA.SIOC_ROLES as roles
FROM DB.DBA.SIOC_ROLE_GRANTS as grants
FROM DB.DBA.SIOC_KNOWS as knows
FROM DB.DBA.ODS_FOAF_PERSON as person
where (^{person.}^.U_NAME = ^{users.}^.U_NAME)
where (^{forums.}^.U_NAME = ^{users.}^.U_NAME)
where (^{knows.}^.TO_NAME = ^{users.}^.U_NAME)
where (^{knows.}^.FROM_NAME = ^{users.}^.U_NAME)
where (^{grants.}^.U_NAME = ^{users.}^.U_NAME)
where (^{roles.}^.U_NAME = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_BLOG_POSTS as blog_posts
where (^{blog_posts.}^.B_OWNER = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_BLOG_POST_LINKS as blog_links
where (^{blog_links.}^.B_OWNER = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_BLOG_POST_ATTS as blog_atts
where (^{blog_atts.}^.B_OWNER = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_BLOG_POST_TAGS as blog_tags
where (^{blog_tags.}^.U_NAME = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_BLOG_COMMENTS as blog_comms
where (^{blog_comms.}^.U_NAME = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_BMK_POSTS as bmk_posts
where (^{bmk_posts.}^.U_NAME = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_BMK_TAGS as bmk_tags
where (^{bmk_tags.}^.U_NAME = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_ODRIVE_POSTS as odrv_posts
where (^{odrv_posts.}^.U_MEMBER = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_ODRIVE_TAGS as odrv_tags
where (^{odrv_tags.}^.U_OWNER = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_FEED_FEED_DOMAIN as feed_domain
where (^{feed_domain.}^.U_NAME = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_FEED_POSTS as feed_posts
where (^{feed_posts.}^.EFI_FEED_ID = ^{feed_domain.}^.EF_ID)
FROM DB.DBA.ODS_FEED_COMMENTS as feed_comments
where (^{feed_comments.}^.U_NAME = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_FEED_TAGS as feed_tags
where (^{feed_tags.}^.U_NAME = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_FEED_LINKS as feed_links
where (^{feed_links.}^.EFI_FEED_ID = ^{feed_domain.}^.EF_ID)
FROM DB.DBA.ODS_FEED_ATTS as feed_atts
where (^{feed_atts.}^.EFI_FEED_ID = ^{feed_domain.}^.EF_ID)
FROM DB.DBA.ODS_PHOTO_POSTS as photo_posts
where (^{photo_posts.}^.U_MEMBER = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_PHOTO_COMMENTS as photo_comments
where (^{photo_comments.}^.U_MEMBER = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_PHOTO_TAGS as photo_tags
where (^{photo_tags.}^.U_MEMBER = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_WIKI_POSTS as wiki_posts
where (^{wiki_posts.}^.U_NAME = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_COMMUNITIES as community
where (^{community.}^.C_OWNER = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_NNTP_GROUPS as nntp_groups
FROM DB.DBA.ODS_NNTP_POSTS as nntp_posts
FROM DB.DBA.ODS_NNTP_USERS as nntp_users
where (^{nntp_users.}^.U_NAME = ^{users.}^.U_NAME)
FROM DB.DBA.ODS_NNTP_LINKS as nntp_links
{
create virtrdf:ODS_DS as graph ods:graph (users.U_NAME) option (exclusive)
{
ods:user (users.U_NAME) a sioc:User ;
sioc:id users.U_NAME ;
sioc:name users.U_FULL_NAME ;
sioc:email ods:mbox (users.E_MAIL) ;
sioc:email_sha1 users.E_MAIL_SHA1 ;
sioc:account_of ods:person (users.U_NAME) .
ods:person (person.U_NAME) a foaf:Person ;
foaf:nick person.U_NAME ;
foaf:name person.U_FULL_NAME ;
foaf:mbox ods:mbox (person.E_MAIL) ;
foaf:mbox_sha1sum person.E_MAIL_SHA1 ;
foaf:holdsAccount ods:user (person.U_NAME) ;
foaf:firstName person.FIRST_NAME ;
foaf:family_name person.LAST_NAME ;
foaf:gender person.GENDER ;
foaf:icqChatID person.ICQ ;
foaf:msnChatID person.MSN ;
foaf:aimChatID person.AIM ;
foaf:yahooChatID person.YAHOO ;
foaf:birthday person.BIRTHDAY ;
foaf:organization person.ORG ;
foaf:phone ods:phone (person.PHONE) ;
foaf:based_near ods:geo_point (person.U_NAME)
.
ods:geo_point (person.U_NAME) a geo:Point ;
geo:lat person.LAT ;
geo:lng person.LNG .
ods:person (knows.FROM_NAME) foaf:knows ods:person (knows.TO_NAME) .
ods:person (knows.TO_NAME) foaf:knows ods:person (knows.FROM_NAME) .
ods:user_group (grants.G_NAME) a sioc:Usergroup ;
sioc:id grants.G_NAME ;
sioc:has_member ods:user (grants.U_NAME) .
ods:user (grants.U_NAME) sioc:member_of ods:user_group (grants.G_NAME) .
ods:role (roles.U_NAME, roles.APP_TYPE, roles.WAM_INST, roles.WMT_NAME)
sioc:has_scope ods:forum (roles.U_NAME, roles.APP_TYPE, roles.WAM_INST) ;
sioc:function_of ods:user (roles.U_NAME) .
ods:forum (roles.U_NAME, roles.APP_TYPE, roles.WAM_INST)
sioc:scope_of ods:role (roles.U_NAME, roles.APP_TYPE, roles.WAM_INST, roles.WMT_NAME) .
ods:user (roles.U_NAME)
sioc:has_function ods:role (roles.U_NAME, roles.APP_TYPE, roles.WAM_INST, roles.WMT_NAME) .
ods:forum (forums.U_NAME, forums.APP_TYPE, forums.WAM_INST) a sioc:Container ;
sioc:id forums.WAM_INST ;
sioc:type forums.APP_TYPE ;
sioc:description forums.WAI_DESCRIPTION ;
sioc:link ods:proxy (forums.LINK) ;
sioc:has_space ods:site (forums.U_NAME) .
# Weblog
ods:blog_post (blog_posts.B_OWNER, blog_posts.B_INST, blog_posts.B_POST_ID) a sioct:BlogPost ;
sioc:link ods:proxy (blog_posts.B_LINK) ;
sioc:has_creator ods:user (blog_posts.B_CREATOR) ;
foaf:maker ods:person (blog_posts.B_CREATOR) ;
sioc:has_container ods:blog_forum (blog_posts.B_OWNER, blog_posts.B_INST) ;
dc:title blog_posts.B_TITLE ;
dct:created blog_posts.B_CREATED ;
dct:modified blog_posts.B_MODIFIED ;
sioc:content blog_posts.B_CONTENT .
ods:blog_forum (blog_posts.B_OWNER, blog_posts.B_INST)
sioc:container_of
ods:blog_post (blog_posts.B_OWNER, blog_posts.B_INST, blog_posts.B_POST_ID) .
ods:user (blog_posts.B_CREATOR)
sioc:creator_of
ods:blog_post (blog_posts.B_OWNER, blog_posts.B_INST, blog_posts.B_POST_ID) .
ods:blog_post (blog_links.B_OWNER, blog_links.B_INST, blog_links.B_POST_ID)
sioc:links_to
ods:proxy (blog_links.PL_LINK) .
# end Weblog
# Bookmark
ods:bmk_post (bmk_posts.U_NAME, bmk_posts.WAI_NAME, bmk_posts.BD_BOOKMARK_ID)
a bm:Bookmark ;
dc:title bmk_posts.BD_NAME;
dct:created bmk_posts.BD_CREATED ;
dct:modified bmk_posts.BD_LAST_UPDATE ;
dc:date bmk_posts.BD_LAST_UPDATE ;
ann:created bmk_posts.BD_CREATED ;
dc:creator bmk_posts.U_NAME ;
bm:recalls ods:proxy (bmk_posts.B_URI) ;
sioc:link ods:proxy (bmk_posts.B_URI) ;
sioc:content bmk_posts.BD_DESCRIPTION ;
sioc:has_creator ods:user (bmk_posts.U_NAME) ;
foaf:maker ods:person (bmk_posts.U_NAME) ;
sioc:has_container ods:bmk_forum (bmk_posts.U_NAME, bmk_posts.WAI_NAME) .
ods:bmk_forum (bmk_posts.U_NAME, bmk_posts.WAI_NAME)
sioc:container_of
ods:bmk_post (bmk_posts.U_NAME, bmk_posts.WAI_NAME, bmk_posts.BD_BOOKMARK_ID) .
ods:user (bmk_posts.U_NAME)
sioc:creator_of
ods:bmk_post (bmk_posts.U_NAME, bmk_posts.WAI_NAME, bmk_posts.BD_BOOKMARK_ID) .
ods:bmk_post (bmk_tags.U_NAME, bmk_tags.WAM_INST, bmk_tags.ITEM_ID)
sioc:topic
ods:tag (bmk_tags.U_NAME, bmk_tags.BD_TAG) .
ods:tag (bmk_tags.U_NAME, bmk_tags.BD_TAG) a skos:Concept ;
skos:prefLabel bmk_tags.BD_TAG ;
skos:isSubjectOf ods:bmk_post (bmk_tags.U_NAME, bmk_tags.WAM_INST, bmk_tags.ITEM_ID) .
# end Bookmark
# Briefcase
ods:odrive_post (odrv_posts.RES_FULL_PATH) a foaf:Document ;
dc:title odrv_posts.RES_NAME ;
dct:created odrv_posts.RES_CREATED ;
dct:modified odrv_posts.RES_MODIFIED ;
sioc:content odrv_posts.RES_DESCRIPTION ;
sioc:has_creator ods:user (odrv_posts.U_OWNER) ;
foaf:maker ods:person (odrv_posts.U_OWNER) ;
sioc:has_container ods:odrive_forum (odrv_posts.U_MEMBER, odrv_posts.WAI_NAME) .
ods:odrive_forum (odrv_posts.U_MEMBER, odrv_posts.WAI_NAME)
sioc:container_of
ods:odrive_post (odrv_posts.RES_FULL_PATH) .
ods:user (odrv_posts.U_OWNER)
sioc:creator_of
ods:odrive_post (odrv_posts.RES_FULL_PATH) .
ods:odrive_post (odrv_tags.RES_FULL_PATH)
sioc:topic
ods:tag (odrv_tags.U_OWNER, odrv_tags.TAG) .
ods:tag (odrv_tags.U_OWNER, odrv_tags.TAG) a skos:Concept ;
skos:prefLabel odrv_tags.TAG ;
skos:isSubjectOf ods:odrive_post (odrv_tags.RES_FULL_PATH) .
# end Briefcase
# Feeds
ods:feed (feed_domain.EF_ID) a atom:Feed ;
sioc:link ods:proxy (feed_domain.EF_URI) ;
atom:link ods:proxy (feed_domain.EF_URI) ;
atom:title feed_domain.EF_TITLE ;
sioc:has_parent ods:feed_mgr (feed_domain.U_NAME, feed_domain.WAI_NAME) .
ods:feed_mgr (feed_domain.U_NAME, feed_domain.WAI_NAME)
sioc:parent_of ods:feed (feed_domain.EF_ID) .
ods:feed_item (feed_tags.EFI_FEED_ID, feed_tags.EFID_ITEM_ID)
sioc:topic
ods:tag (feed_tags.U_NAME, feed_tags.EFID_TAG) .
ods:tag (feed_tags.U_NAME, feed_tags.EFID_TAG) a skos:Concept ;
skos:prefLabel feed_tags.EFID_TAG ;
skos:isSubjectOf ods:feed_item (feed_tags.EFI_FEED_ID, feed_tags.EFID_ITEM_ID) .
ods:feed_comment (feed_comments.U_NAME, feed_comments.WAI_NAME, feed_comments.EFIC_ITEM_ID, feed_comments.EFIC_ID)
a sioct:Comment ;
dc:title feed_comments.EFIC_TITLE ;
sioc:content feed_comments.EFIC_COMMENT ;
dct:modified feed_comments.LAST_UPDATE ;
dct:created feed_comments.LAST_UPDATE ;
sioc:link ods:proxy (feed_comments.LINK) ;
sioc:has_container ods:feed (feed_comments.EFI_FEED_ID) ;
sioc:reply_of ods:feed_item (feed_comments.EFI_FEED_ID, feed_comments.EFIC_ITEM_ID) ;
foaf:maker ods:proxy (feed_comments.EFIC_U_URL) .
ods:proxy (feed_comments.EFIC_U_URL) a foaf:Person ;
foaf:name feed_comments.EFIC_U_NAME;
foaf:mbox ods:mbox (feed_comments.EFIC_U_MAIL) .
ods:feed (feed_comments.EFI_FEED_ID)
sioc:container_of
ods:feed_comment (feed_comments.U_NAME, feed_comments.WAI_NAME, feed_comments.EFIC_ITEM_ID, feed_comments.EFIC_ID) .
ods:feed_item (feed_comments.EFI_FEED_ID, feed_comments.EFIC_ITEM_ID)
sioc:has_reply
ods:feed_comment (feed_comments.U_NAME, feed_comments.WAI_NAME, feed_comments.EFIC_ITEM_ID, feed_comments.EFIC_ID) .
ods:feed_item (feed_links.EFI_FEED_ID, feed_links.EFI_ID)
sioc:links_to
ods:proxy (feed_links.EFIL_LINK) .
ods:feed_item (feed_atts.EFI_FEED_ID, feed_atts.EFI_ID)
sioc:attachment
ods:proxy (feed_atts.EFIE_URL) .
ods:feed_item (feed_posts.EFI_FEED_ID, feed_posts.EFI_ID) a atom:Entry ;
sioc:has_container ods:feed (feed_posts.EFI_FEED_ID) ;
dc:title feed_posts.EFI_TITLE ;
dct:created feed_posts.PUBLISH_DATE ;
dct:modified feed_posts.PUBLISH_DATE ;
sioc:link ods:proxy (feed_posts.EFI_LINK) ;
sioc:content feed_posts.EFI_DESCRIPTION ;
atom:title feed_posts.EFI_TITLE ;
atom:source ods:feed (feed_posts.EFI_FEED_ID) ;
atom:published feed_posts.PUBLISH_DATE ;
atom:updated feed_posts.PUBLISH_DATE ;
atom:content ods:feed_item_text (feed_posts.EFI_FEED_ID, feed_posts.EFI_ID) .
ods:feed (feed_posts.EFI_FEED_ID) sioc:container_of ods:feed_item (feed_posts.EFI_FEED_ID, feed_posts.EFI_ID) .
ods:feed_item_text (feed_posts.EFI_FEED_ID, feed_posts.EFI_ID) a atom:Content ;
atom:type "text/xhtml" ;
atom:lang "en-US" ;
atom:body feed_posts.EFI_DESCRIPTION .
ods:feed (feed_posts.EFI_FEED_ID)
atom:contains
ods:feed_item (feed_posts.EFI_FEED_ID, feed_posts.EFI_ID) .
# end Feeds
# Photo
ods:photo_post (photo_posts.RES_FULL_PATH) a exif:IFD ;
dc:title photo_posts.RES_NAME ;
dct:created photo_posts.RES_CREATED ;
dct:modified photo_posts.RES_MODIFIED ;
sioc:content photo_posts.RES_DESCRIPTION ;
sioc:has_creator ods:user (photo_posts.U_OWNER) ;
foaf:maker ods:person (photo_posts.U_OWNER) ;
sioc:link ods:proxy (photo_posts.RES_LINK) ;
sioc:has_container ods:photo_forum (photo_posts.U_MEMBER, photo_posts.WAI_NAME) .
ods:photo_forum (photo_posts.U_MEMBER, photo_posts.WAI_NAME)
sioc:container_of
ods:photo_post (photo_posts.RES_FULL_PATH) .
ods:user (photo_posts.U_OWNER)
sioc:creator_of
ods:photo_post (photo_posts.RES_FULL_PATH) .
ods:photo_post (photo_tags.RES_FULL_PATH)
sioc:topic
ods:tag (photo_tags.U_MEMBER, photo_tags.RES_TAG) .
ods:tag (photo_tags.U_MEMBER, photo_tags.RES_TAG) a skos:Concept ;
skos:prefLabel photo_tags.RES_TAG ;
skos:isSubjectOf ods:photo_post (photo_tags.RES_FULL_PATH) .
ods:photo_comment (photo_comments.RES_FULL_PATH, photo_comments.COMMENT_ID) a sioct:Comment ;
sioc:reply_of ods:photo_post (photo_comments.RES_FULL_PATH) ;
sioc:has_container ods:photo_forum (photo_comments.U_MEMBER, photo_comments.WAI_NAME) ;
dc:title photo_comments.RES_NAME ;
dct:created photo_comments.CREATE_DATE ;
dct:modified photo_comments.MODIFY_DATE ;
sioc:content photo_comments.TEXT ;
foaf:maker ods:person (photo_comments.U_MAKER) .
ods:photo_post (photo_comments.RES_FULL_PATH)
sioc:has_reply
ods:photo_comment (photo_comments.RES_FULL_PATH, photo_comments.COMMENT_ID) .
# end Photo
# Polls
# end Polls
# Mail
# end Mail
# Wiki
ods:wiki_post (wiki_posts.U_NAME, wiki_posts.CLUSTERNAME, wiki_posts.LOCALNAME) a wikiont:Article ;
dc:title wiki_posts.LOCALNAME ;
dct:created wiki_posts.RES_CREATED ;
dct:modified wiki_posts.RES_MODIFIED ;
sioc:content wiki_posts.RES_CONTENT ;
sioc:has_creator ods:user (wiki_posts.U_NAME) ;
foaf:maker ods:person (wiki_posts.U_NAME) ;
sioc:has_container ods:wiki_forum (wiki_posts.U_NAME, wiki_posts.CLUSTERNAME) .
ods:wiki_forum (wiki_posts.U_NAME, wiki_posts.CLUSTERNAME)
sioc:container_of
ods:wiki_post (wiki_posts.U_NAME, wiki_posts.CLUSTERNAME, wiki_posts.LOCALNAME) .
ods:user (wiki_posts.U_NAME)
sioc:creator_of
ods:wiki_post (wiki_posts.U_NAME, wiki_posts.CLUSTERNAME, wiki_posts.LOCALNAME) .
# end Wiki
# Community
ods:community_forum (community.C_OWNER, community.CM_COMMUNITY_ID) a sioc:Community ;
sioc:has_part ods:forum (community.A_OWNER, community.A_TYPE, community.CM_MEMBER_APP) .
ods:forum (community.A_OWNER, community.A_TYPE, community.CM_MEMBER_APP)
sioc:part_of
ods:community_forum (community.C_OWNER, community.CM_COMMUNITY_ID) .
# end Community
# NNTP
ods:nntp_forum (nntp_groups.NG_NAME) a sioct:MessageBoard ;
sioc:id nntp_groups.NG_NAME ;
sioc:description nntp_groups.NG_DESC .
ods:nntp_post (nntp_posts.NG_NAME, nntp_posts.NM_ID) a sioct:BoardPost ;
sioc:content nntp_posts.NM_BODY ;
dc:title nntp_posts.FTHR_SUBJ ;
dct:created nntp_posts.REC_DATE ;
dct:modified nntp_posts.REC_DATE ;
foaf:maker ods:proxy (nntp_posts.MAKER) ;
sioc:reply_of ods:nntp_post (nntp_posts.NG_NAME, nntp_posts.FTHR_REFER) ;
sioc:has_container ods:nntp_forum (nntp_posts.NG_NAME) .
ods:nntp_post (nntp_posts.NG_NAME, nntp_posts.FTHR_REFER)
sioc:has_reply
ods:nntp_post (nntp_posts.NG_NAME, nntp_posts.NM_ID) .
ods:nntp_forum (nntp_posts.NG_NAME)
sioc:container_of
ods:nntp_post (nntp_posts.NG_NAME, nntp_posts.NM_ID) .
ods:nntp_role (nntp_groups.NG_NAME)
sioc:has_scope
ods:nntp_forum (nntp_groups.NG_NAME) .
ods:nntp_forum (nntp_groups.NG_NAME)
sioc:scope_of
ods:nntp_role (nntp_groups.NG_NAME) .
ods:user (nntp_users.U_NAME)
sioc:has_function
ods:nntp_role (nntp_users.NG_NAME) .
ods:nntp_role (nntp_users.NG_NAME)
sioc:function_of
ods:user (nntp_users.U_NAME) .
ods:nntp_post (nntp_links.NG_NAME, nntp_links.NML_MSG_ID)
sioc:links_to
ods:proxy (nntp_links.NML_URL) .
# end NNTP
} .
} .
;
URL Rewrite Rules for ODS
create procedure DB.DBA.URL_REW_ODS_ACCEPT ()
{
declare accept, ret any;
accept := http_request_header (http_request_header (), 'Accept');
if (not isstring (accept))
return null;
ret := null;
if (regexp_match ('(application|text)/rdf.(xml|n3|turtle|ttl)', accept) is not null)
{
if (regexp_match ('application/rdf.xml', accept) is not null)
{
ret := 'rdf';
}
else if (regexp_match ('text/rdf.n3', accept) is not null)
{
ret := 'n3';
}
else if (regexp_match ('application/rdf.turtle', accept) is not null or
regexp_match ('application/rdf.ttl', accept) is not null)
{
ret := 'n3';
}
}
return ret;
};
create procedure DB.DBA.URL_REW_ODS_SPQ (in graph varchar, in iri varchar, in acc varchar)
{
declare q, ret any;
iri := replace (iri, '''', '%27');
iri := replace (iri, '<', '%3C');
iri := replace (iri, '>', '%3E');
q := sprintf ('define input:inference <%s> DESCRIBE <%s> FROM <%s>', graph, iri, graph);
ret := sprintf ('/sparql?query=%U&format=%U', q, acc);
return ret;
};
create procedure DB.DBA.URL_REW_ODS_USER (in par varchar, in fmt varchar, in val varchar)
{
-- dbg_obj_print (current_proc_name ());
declare acc, ret any;
declare q, iri, graph any;
acc := DB.DBA.URL_REW_ODS_ACCEPT ();
if (acc is not null)
{
graph := sioc..get_graph ();
iri := sprintf ('%s/%U', graph, val);
if (val like 'person/%')
{
val := substring (val, 8, length (val));
ret := sprintf ('/ods/foaf.vsp?uname=%U&fmt=%U', val, acc);
}
else
ret := DB.DBA.URL_REW_ODS_SPQ (graph, iri, acc);
}
else
{
http_header (http_header_get ()||sprintf ('X-XRDS-Location: %s\r\n',
DB.DBA.wa_link (1, '/dataspace/'||val||'/yadis.xrds')));
if (val like 'person/%')
val := substring (val, 8, length (val));
ret := sprintf ('/ods/uhome.vspx?page=1&ufname=%s', val);
}
return ret;
};
create procedure DB.DBA.URL_REW_ODS_USER_GEM (in par varchar, in fmt varchar, in val varchar)
{
-- dbg_obj_print (current_proc_name ());
declare acc, ret any;
declare q, iri, graph, path, is_person any;
path := http_path ();
if (path like '%.rdf')
acc := 'rdf';
else if (path like '%.n3')
acc := 'n3';
else if (path like '%.ttl')
acc := 'n3';
else if (path like '%/yadis.xrds')
acc := 'yadis';
else
acc := 'rdf';
if (acc <> 'yadis')
{
is_person := matches_like (path, '%/about.%');
graph := sioc..get_graph ();
if (is_person)
{
--iri := sprintf ('%s/person/%U', graph, val);
ret := sprintf ('/ods/foaf.vsp?uname=%U&fmt=%U', val, acc);
}
else
{
iri := sprintf ('%s/%U', graph, val);
ret := DB.DBA.URL_REW_ODS_SPQ (graph, iri, acc);
}
}
else
{
ret := sprintf ('/ods/yadis.vsp?uname=%U', val);
}
return ret;
};
create procedure DB.DBA.URL_REW_ODS_GEM (in par varchar, in fmt varchar, in val varchar)
{
-- dbg_obj_print (current_proc_name ());
declare acc, ret any;
declare q, iri, graph, path, pos any;
path := http_path ();
if (path like '%.rdf')
acc := 'rdf';
else if (path like '%.n3')
acc := 'n3';
else if (path like '%.ttl')
acc := 'n3';
else
acc := 'rdf';
graph := sioc..get_graph ();
pos := strrchr (path, '/');
path := subseq (path, 0, pos);
if (val = 'person')
{
pos := strrchr (path, '/');
val := subseq (path, pos+1, length (path));
ret := sprintf ('/ods/foaf.vsp?uname=%U&fmt=%U', val, acc);
}
else
{
iri := sprintf ('http://%s%s', sioc..get_cname (), path);
ret := DB.DBA.URL_REW_ODS_SPQ (graph, iri, acc);
}
return ret;
};
create procedure DB.DBA.URL_REW_ODS_APP (in par varchar, in fmt varchar, in val varchar)
{
-- dbg_obj_print (current_proc_name (), val);
if (par = 'app')
return sprintf (fmt, wa_app_to_type (val));
return sprintf (fmt, val);
};
create procedure DB.DBA.URL_REW_ODS_BLOG (in par varchar, in fmt varchar, in val varchar)
{
-- dbg_obj_print (par, fmt, val);
-- dbg_obj_print (current_proc_name (), val);
declare acc, ret any;
acc := DB.DBA.URL_REW_ODS_ACCEPT ();
if (acc is not null)
{
if (par = 'inst')
{
declare q, iri, graph any;
graph := sioc..get_graph ();
iri := 'http://' || sioc..get_cname () || http_path ();
ret := DB.DBA.URL_REW_ODS_SPQ (graph, iri, acc);
return ret;
}
else
return '';
}
else if (par = 'inst')
{
declare url any;
val := split_and_decode (val)[0];
url := (SELECT WAM_HOME_PAGE FROM WA_MEMBER WHERE WAM_INST = val AND WAM_MEMBER_TYPE = 1);
if (url is not null)
val := url;
return sprintf (fmt, val);
}
else if (par = 'id' and val <> '')
{
if (atoi (val) = 0 and val <> '0')
fmt := '%s';
else
fmt := '?id=%s';
return sprintf (fmt, val);
}
};
create procedure DB.DBA.URL_REW_ODS_NNTP (in par varchar, in fmt varchar, in val varchar)
{
-- dbg_obj_print (par, fmt, val);
-- dbg_obj_print (current_proc_name (), val);
declare acc, ret any;
acc := DB.DBA.URL_REW_ODS_ACCEPT ();
if (acc is not null)
{
declare q, iri, graph any;
graph := sioc..get_graph ();
iri := 'http://' || sioc..get_cname () || http_path ();
-- dbg_obj_print (iri);
ret := DB.DBA.URL_REW_ODS_SPQ (graph, iri, acc);
return ret;
}
else if (par = 'grp')
{
declare gid int;
val := split_and_decode (val)[0];
gid := (SELECT NG_GROUP FROM DB.DBA.NEWS_GROUPS WHERE NG_NAME = val);
ret := sprintf ('/nntpf/nntpf_nthread_view.vspx?group=%d', gid);
return ret;
}
else if (par = 'post')
{
ret := sprintf ('/nntpf/nntpf_disp_article.vspx?id=%U', encode_base64 (val));
return ret;
}
}
;
create procedure DB.DBA.URL_REW_ODS_XD (in par varchar, in fmt varchar, in val varchar)
{
-- dbg_obj_print (par, fmt, val);
-- dbg_obj_print (current_proc_name (), val);
declare acc, ret any;
acc := DB.DBA.URL_REW_ODS_ACCEPT ();
if (acc is not null)
{
declare q, iri, graph any;
graph := sioc..get_graph ();
iri := 'http://' || sioc..get_cname () || http_path ();
-- dbg_obj_print (iri);
ret := DB.DBA.URL_REW_ODS_SPQ (graph, iri, acc);
return ret;
}
else if (par = 'inst')
{
val := split_and_decode (val)[0];
ret := (SELECT WAM_HOME_PAGE FROM WA_MEMBER WHERE WAM_INST = val and WAM_MEMBER_TYPE = 1);
return ret;
}
}
;
create procedure DB.DBA.URL_REW_ODS_WIKI (in par varchar, in fmt varchar, in val varchar)
{
-- dbg_obj_print (par, fmt, val);
-- dbg_obj_print (current_proc_name (), val);
declare acc, ret any;
acc := DB.DBA.URL_REW_ODS_ACCEPT ();
if (acc is not null)
{
declare q, iri, graph any;
graph := sioc..get_graph ();
iri := 'http://' || sioc..get_cname () || http_path ();
-- dbg_obj_print (iri);
ret := DB.DBA.URL_REW_ODS_SPQ (graph, iri, acc);
return ret;
}
else if (par = 'inst')
{
declare _inst DB.DBA.web_app;
_inst := (SELECT WAI_INST FROM WA_INSTANCE WHERE WAI_NAME = val);
ret := _inst.wa_post_url (null, null, val, val);
-- dbg_obj_print ('ret', ret);
return ret;
}
else if (par = 'post')
{
return '/'||val;
}
}
;
create procedure DB.DBA.URL_REW_ODS_PHOTO (in par varchar, in fmt varchar, in val varchar)
{
-- dbg_obj_print (par, fmt, val);
-- dbg_obj_print (current_proc_name (), val);
declare acc, ret any;
acc := DB.DBA.URL_REW_ODS_ACCEPT ();
if (acc is not null)
{
declare q, iri, graph any;
graph := sioc..get_graph ();
iri := 'http://' || sioc..get_cname () || http_path ();
-- dbg_obj_print (iri);
ret := DB.DBA.URL_REW_ODS_SPQ (graph, iri, acc);
return ret;
}
else if (par = 'inst')
{
val := split_and_decode (val)[0];
ret := (SELECT WAM_HOME_PAGE FROM WA_MEMBER WHERE WAM_INST = val AND WAM_MEMBER_TYPE = 1);
return ret;
}
else if (par = 'post')
{
declare id int;
declare col, nam varchar;
declare exit handler for not found
{
signal ('22023', sprintf ('The resource %d doesn''t exists', id));
};
id := atoi(ltrim(val, '/'));
SELECT RES_FULL_PATH INTO nam FROM WS.WS.SYS_DAV_RES WHERE RES_ID = id;
return nam;
}
}
;
create procedure DB.DBA.URL_REW_ODS_ADDRESSBOOK (in par varchar, in fmt varchar, in val varchar)
{
declare acc, ret any;
acc := DB.DBA.URL_REW_ODS_ACCEPT ();
if (acc is not null)
{
if (par = 'instance')
{
declare q, iri, graph any;
graph := sioc..get_graph ();
iri := 'http://' || sioc..get_cname () || http_path ();
ret := DB.DBA.URL_REW_ODS_SPQ (graph, iri, acc);
return ret;
}
else
return '';
}
else if (par = 'instance')
{
declare id, url any;
val := split_and_decode (val)[0];
id := AB.WA.domain_id (val);
if (id is not null) {
url := AB.WA.ab_url (id);
if (url is not null)
val := url;
}
return sprintf (fmt, val);
}
else if (par = 'params')
{
if (atoi (val) = 0 and val <> '0')
fmt := '%s';
else
fmt := '?id=%s';
return sprintf (fmt, val);
}
}
;
create procedure DB.DBA.URL_REW_ODS_BOOKMARK (in par varchar, in fmt varchar, in val varchar)
{
declare acc, ret any;
acc := DB.DBA.URL_REW_ODS_ACCEPT ();
if (acc is not null)
{
if (par = 'instance')
{
declare q, iri, graph any;
graph := sioc..get_graph ();
iri := 'http://' || sioc..get_cname () || http_path ();
ret := DB.DBA.URL_REW_ODS_SPQ (graph, iri, acc);
return ret;
}
else
return '';
}
else if (par = 'instance')
{
declare id, url any;
val := split_and_decode (val)[0];
id := BMK.WA.domain_id (val);
if (id is not null) {
url := BMK.WA.bookmark_url (id);
if (url is not null)
val := url;
}
return sprintf (fmt, val);
}
else if (par = 'params')
{
if (atoi (val) = 0 and val <> '0')
fmt := '%s';
else
fmt := '?id=%s';
return sprintf (fmt, val);
}
}
;
create procedure DB.DBA.URL_REW_ODS_BRIEFCASE (in par varchar, in fmt varchar, in val varchar)
{
declare acc, ret any;
acc := DB.DBA.URL_REW_ODS_ACCEPT ();
if (acc is not null)
{
if (par = 'instance')
{
declare q, iri, graph any;
graph := sioc..get_graph ();
iri := 'http://' || sioc..get_cname () || http_path ();
ret := DB.DBA.URL_REW_ODS_SPQ (graph, iri, acc);
return ret;
}
else
return '';
}
else if (par = 'instance')
{
declare id, url any;
val := split_and_decode (val)[0];
id := ODRIVE.WA.domain_id (val);
if (id is not null) {
url := ODRIVE.WA.odrive_url (id);
if (url is not null)
val := url;
}
return sprintf (fmt, val);
}
else if (par = 'params')
{
if (atoi (val) = 0 and val <> '0')
fmt := '%s';
else
fmt := '?id=%s';
return sprintf (fmt, val);
}
}
;
create procedure DB.DBA.URL_REW_ODS_CALENDAR (in par varchar, in fmt varchar, in val varchar)
{
declare acc, ret any;
acc := DB.DBA.URL_REW_ODS_ACCEPT ();
if (acc is not null)
{
if (par = 'instance')
{
declare q, iri, graph any;
graph := sioc..get_graph ();
iri := 'http://' || sioc..get_cname () || http_path ();
ret := DB.DBA.URL_REW_ODS_SPQ (graph, iri, acc);
return ret;
}
else
return '';
}
else if (par = 'instance')
{
declare id, url any;
val := split_and_decode (val)[0];
id := CAL.WA.domain_id (val);
if (id is not null) {
url := CAL.WA.calendar_url (id);
if (url is not null)
val := url;
}
return sprintf (fmt, val);
}
else if (par = 'params')
{
if (atoi (val) = 0 and val <> '0')
fmt := '%s';
else
fmt := '?id=%s';
return sprintf (fmt, val);
}
}
;
create procedure DB.DBA.URL_REW_ODS_FEEDS (in par varchar, in fmt varchar, in val varchar)
{
declare acc, ret any;
acc := DB.DBA.URL_REW_ODS_ACCEPT ();
if (acc is not null)
{
if (par = 'instance')
{
declare q, iri, graph any;
graph := sioc..get_graph ();
iri := 'http://' || sioc..get_cname () || http_path ();
ret := DB.DBA.URL_REW_ODS_SPQ (graph, iri, acc);
return ret;
}
else
return '';
}
else if (par = 'instance')
{
declare id, url any;
val := split_and_decode (val)[0];
id := ENEWS.WA.domain_id (val);
if (id is not null) {
url := ENEWS.WA.enews_url (id) || 'news.vspx';
if (url is not null)
val := url;
}
return sprintf (fmt, val);
}
else if (par = 'params')
{
if (atoi (val) = 0 and val <> '0')
fmt := '%s';
else
fmt := '?id=%s';
return sprintf (fmt, val);
}
}
;
create procedure DB.DBA.URL_REW_ODS_POLLS (in par varchar, in fmt varchar, in val varchar)
{
declare acc, ret any;
acc := DB.DBA.URL_REW_ODS_ACCEPT ();
if (acc is not null)
{
if (par = 'instance')
{
declare q, iri, graph any;
graph := sioc..get_graph ();
iri := 'http://' || sioc..get_cname () || http_path ();
ret := DB.DBA.URL_REW_ODS_SPQ (graph, iri, acc);
return ret;
}
else
return '';
}
else if (par = 'instance')
{
declare id, url any;
val := split_and_decode (val)[0];
id := POLLS.WA.domain_id (val);
if (id is not null) {
url := POLLS.WA.polls_url (id);
if (url is not null)
val := url;
}
return sprintf (fmt, val);
}
else if (par = 'params')
{
if (atoi (val) = 0 and val <> '0')
fmt := '%s';
else
fmt := '?id=%s';
return sprintf (fmt, val);
}
}
;
create procedure DB.DBA.URL_REW_ODS_FOAF_EXT (in par varchar, in fmt varchar, in val varchar)
{
if (par = '*accept*')
{
declare ext any;
ext := 'rdf';
if (val = 'text/rdf+n3')
ext := 'n3';
return sprintf (fmt, ext);
}
else
return sprintf (fmt, val);
}
;
create procedure ur_ods_rdf_doc (in path varchar)
{
declare r any;
r := regexp_match ('[^/]*\x24', path);
return r||'#this';
};
create procedure ur_ods_html_doc (in path varchar)
{
declare pos, r any;
if (path like '%/foaf.%')
{
pos := strrchr (path, '/');
}
else if (path like '%#%')
{
pos := strrchr (path, '#');
}
if (pos > 0)
r := subseq (path, 0, pos);
else
r := '/';
return r;
};
-- ODS Rules
-- http://cname/dataspace/uname
-- http://cname/dataspace/person/uname
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_rule1', 1,
'/dataspace/((person/)?[^/#]*)', vector('ufname'), 1,
'%s', vector('ufname'),
'DB.DBA.URL_REW_ODS_USER');
-- http://cname/dataspace/uname with Accept will do 303 to the /sparql
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_rule2', 1,
'/dataspace/([^/]*)', vector('ufname'), 1,
'/sparql?query=define+input%%3Ainference+%%3Chttp%%3A//^{URIQADefaultHost}^/dataspace%%3E+DESCRIBE+%%3Chttp%%3A//^{URIQADefaultHost}^/dataspace/%U%%3E+FROM+%%3Chttp%%3A//^{URIQADefaultHost}^/dataspace%%3E&format=%U', vector('ufname', '*accept*'),
null,
'(application|text)/rdf.(xml|n3|turtle|ttl)',
0,
303);
-- http://cname/dataspace/uname/app_type
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_rule3', 1,
'/dataspace/((?!person)[^/]*)/([^\\./]*)', vector('ufname', 'app'), 2,
'/ods/app_inst.vspx?app=%s&ufname=%s&l=1', vector('app', 'ufname'),
'DB.DBA.URL_REW_ODS_APP');
-- http://cname/dataspace/uname/file.ext
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_rule4', 1,
'/dataspace/([^/]*)/(sioc|about|yadis)\\.(rdf|n3|ttl|xrds)', vector('ufname', 'file', 'fmt'), 3,
'%s', vector('ufname'),
'DB.DBA.URL_REW_ODS_USER_GEM');
-- Rules for FOAF profile
-- http://cname/dataspace/person/uname with Accept, do 303 to http://cname/dataspace/person/uname/foaf.ext
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_rule5', 1,
'/dataspace/person/([^/#]*)/?', vector('ufname'), 1,
'/dataspace/person/%U/foaf.%s', vector('ufname', '*accept*'),
'DB.DBA.URL_REW_ODS_FOAF_EXT',
'(application|text)/rdf.(xml|n3|turtle|ttl)',
2,
303);
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_rule6', 1,
'/dataspace/person/([^/]*)/page/([^/]*)/?', vector('ufname', 'page'), 1,
'/dataspace/person/%U/foaf.%s?page=%s', vector('ufname', '*accept*', 'page'),
'DB.DBA.URL_REW_ODS_FOAF_EXT',
'(application|text)/rdf.(xml|n3|turtle|ttl)',
2,
303);
-- http://cname/dataspace/person/uname/foaf.ext
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_rule7', 1,
'/dataspace/person/([^/]*)/foaf.(rdf|n3|ttl)', vector('ufname', 'fmt'), 1,
'/ods/foaf.vsp?uname=%U&fmt=%U', vector('ufname', 'fmt'),
null,
null,
2,
null);
-- App Instance Gem
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_post_gem_rule', 1,
'/dataspace/([^/]*)/([^/]*)/([^/]*/)?([^/]*/)?(sioc|about)\\.(rdf|n3|ttl)', vector('ufname', 'app', 'inst'), 4,
'%s', vector('ufname'),
'DB.DBA.URL_REW_ODS_GEM');
-- Weblog Rules
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_blog_rule1', 1,
'/dataspace/([^/]*)/weblog/([^/]*)', vector('ufname', 'inst'), 2,
'%s', vector('inst'),
'DB.DBA.URL_REW_ODS_BLOG');
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_blog_rule2', 1,
'/dataspace/([^/]*)/weblog/([^/]*)/([^/]*)', vector('ufname', 'inst', 'id'), 3,
'%s%s', vector('inst', 'id'),
'DB.DBA.URL_REW_ODS_BLOG');
-- Discussion rules
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_nntp_rule1', 1,
'/dataspace/discussion/([^/]*)', vector('grp'), 1,
'%s', vector('grp'),
'DB.DBA.URL_REW_ODS_NNTP');
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_nntp_rule2', 1,
'/dataspace/discussion/([^/]*)/((?!sioc)(?!about)[^/]*)', vector('grp', 'post'), 2,
'%s', vector('post'),
'DB.DBA.URL_REW_ODS_NNTP');
-- Community
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_xd_rule1', 1,
'/dataspace/([^/]*)/community/([^/]*)', vector('ufname', 'inst'), 2,
'%s', vector('inst'),
'DB.DBA.URL_REW_ODS_XD');
-- Wiki
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_wiki_rule1', 1,
'/dataspace/([^/]*)/wiki/([^/]*)', vector('ufname', 'inst'), 2,
'%s', vector('inst'),
'DB.DBA.URL_REW_ODS_WIKI');
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_wiki_rule2', 1,
'/dataspace/([^/]*)/wiki/([^/]*)/([^/]*)', vector('ufname', 'inst', 'post'), 2,
'%s%s', vector('inst', 'post'),
'DB.DBA.URL_REW_ODS_WIKI');
-- Gallery
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_photo_rule1', 1,
'/dataspace/([^/]*)/photos/([^/]*)', vector('ufname', 'inst'), 2,
'%s', vector('inst'),
'DB.DBA.URL_REW_ODS_PHOTO');
DB.DBA.URLREWRITE_CREATE_REGEX_RULE ('ods_photo_rule2', 1,
'/dataspace/([^/]*)/photos/([^/]*)/([^/]*)', vector('ufname', 'inst', 'post'), 2,
'%s', vector('post'),
'DB.DBA.URL_REW_ODS_PHOTO');
-- AddressBook
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'ods_addressbook_rule1',
1,
'/dataspace/([^/]*)/addressbook/([^/]*)',
vector('uname', 'instance'),
2,
'%s', vector('instance'),
'DB.DBA.URL_REW_ODS_ADDRESSBOOK');
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'ods_addressbook_rule2',
1,
'/dataspace/([^/]*)/addressbook/([^/]*)/(.*)',
vector('uname', 'instance', 'params'),
3,
'%s%s',
vector('instance', 'params'),
'DB.DBA.URL_REW_ODS_ADDRESSBOOK');
-- Bookmark
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'ods_bookmark_rule1',
1,
'/dataspace/([^/]*)/bookmark/([^/]*)',
vector('uname', 'instance'),
2,
'%s', vector('instance'),
'DB.DBA.URL_REW_ODS_BOOKMARK');
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'ods_bookmark_rule2',
1,
'/dataspace/([^/]*)/bookmark/([^/]*)/(.*)',
vector('uname', 'instance', 'params'),
3,
'%s%s',
vector('instance', 'params'),
'DB.DBA.URL_REW_ODS_BOOKMARK');
-- Briefcase
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'ods_briefcase_rule1',
1,
'/dataspace/([^/]*)/briefcase/([^/]*)',
vector('uname', 'instance'),
2,
'%s', vector('instance'),
'DB.DBA.URL_REW_ODS_BRIEFCASE');
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'ods_briefcase_rule2',
1,
'/dataspace/([^/]*)/briefcase/([^/]*)/(.*)',
vector('uname', 'instance', 'params'),
3,
'%s%s',
vector('instance', 'params'),
'DB.DBA.URL_REW_ODS_BRIEFCASE');
-- Calendar
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'ods_calendar_rule1',
1,
'/dataspace/([^/]*)/calendar/([^/]*)',
vector('uname', 'instance'),
2,
'%s', vector('instance'),
'DB.DBA.URL_REW_ODS_CALENDAR');
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'ods_calendar_rule2',
1,
'/dataspace/([^/]*)/calendar/([^/]*)/(.*)',
vector('uname', 'instance', 'params'),
3,
'%s%s',
vector('instance', 'params'),
'DB.DBA.URL_REW_ODS_CALENDAR');
-- Feeds
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'ods_feeds_rule1',
1,
'/dataspace/([^/]*)/feeds/([^/]*)',
vector('uname', 'instance'),
2,
'%s', vector('instance'),
'DB.DBA.URL_REW_ODS_FEEDS');
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'ods_feeds_rule2',
1,
'/dataspace/([^/]*)/feeds/([^/]*)/(.*)',
vector('uname', 'instance', 'params'),
3,
'%s%s',
vector('instance', 'params'),
'DB.DBA.URL_REW_ODS_FEEDS');
-- Polls
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'ods_polls_rule1',
1,
'/dataspace/([^/]*)/polls/([^/]*)',
vector('uname', 'instance'),
2,
'%s', vector('instance'),
'DB.DBA.URL_REW_ODS_POLLS');
DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
'ods_polls_rule2',
1,
'/dataspace/([^/]*)/polls/([^/]*)/(.*)',
vector('uname', 'instance', 'params'),
3,
'%s%s',
vector('instance', 'params'),
'DB.DBA.URL_REW_ODS_POLLS');
-- ODS Base rules
DB.DBA.URLREWRITE_CREATE_RULELIST ('ods_base_rule_list1', 1,
vector(
'ods_rule1', 'ods_rule2', 'ods_rule3', 'ods_rule4'
));
DB.DBA.URLREWRITE_CREATE_RULELIST ('ods_foaf_rule_list1', 1,
vector(
'ods_rule5', 'ods_rule6', 'ods_rule7'
));
DB.DBA.URLREWRITE_CREATE_RULELIST ('ods_gems_rule_list1', 1,
vector(
'ods_post_gem_rule'
));
-- ODS Blog rules
DB.DBA.URLREWRITE_CREATE_RULELIST ('ods_blog_rule_list1', 1,
vector(
'ods_blog_rule1', 'ods_blog_rule2'
));
-- ODS Discussion rules
DB.DBA.URLREWRITE_CREATE_RULELIST ('ods_nntp_rule_list1', 1,
vector(
'ods_nntp_rule1', 'ods_nntp_rule2'
));
-- ODS Community rules
DB.DBA.URLREWRITE_CREATE_RULELIST ('ods_xd_rule_list1', 1,
vector(
'ods_xd_rule1'
));
-- ODS Wiki rules
DB.DBA.URLREWRITE_CREATE_RULELIST ('ods_wiki_rule_list1', 1,
vector(
'ods_wiki_rule1', 'ods_wiki_rule2'
));
-- ODS Gallery rules
DB.DBA.URLREWRITE_CREATE_RULELIST ('ods_photo_rule_list1', 1,
vector(
'ods_photo_rule1', 'ods_photo_rule2'
));
-- ODS AddressBook rules
DB.DBA.URLREWRITE_CREATE_RULELIST (
'ods_addressbook_rule_list1',
1,
vector (
'ods_addressbook_rule1',
'ods_addressbook_rule2'
));
-- ODS Bookmark rules
DB.DBA.URLREWRITE_CREATE_RULELIST (
'ods_bookmark_rule_list1',
1,
vector (
'ods_bookmark_rule1',
'ods_bookmark_rule2'
));
-- ODS Briefcase rules
DB.DBA.URLREWRITE_CREATE_RULELIST (
'ods_briefcase_rule_list1',
1,
vector (
'ods_briefcase_rule1',
'ods_briefcase_rule2'
));
-- ODS Calendar rules
DB.DBA.URLREWRITE_CREATE_RULELIST (
'ods_calendar_rule_list1',
1,
vector (
'ods_calendar_rule1',
'ods_calendar_rule2'
));
-- ODS Feeds rules
DB.DBA.URLREWRITE_CREATE_RULELIST (
'ods_feeds_rule_list1',
1,
vector (
'ods_feeds_rule1',
'ods_feeds_rule2'
));
-- ODS Polls rules
DB.DBA.URLREWRITE_CREATE_RULELIST (
'ods_polls_rule_list1',
1,
vector (
'ods_polls_rule1',
'ods_polls_rule2'
));
-- All ODS Rules
DB.DBA.URLREWRITE_CREATE_RULELIST ('ods_rule_list1', 1,
vector(
'ods_base_rule_list1',
'ods_foaf_rule_list1',
'ods_blog_rule_list1',
'ods_nntp_rule_list1',
'ods_xd_rule_list1',
'ods_wiki_rule_list1',
'ods_photo_rule_list1',
'ods_addressbook_rule_list1',
'ods_bookmark_rule_list1',
'ods_briefcase_rule_list1',
'ods_calendar_rule_list1',
'ods_feeds_rule_list1',
'ods_polls_rule_list1',
'ods_gems_rule_list1'
));
DB.DBA.XML_SET_NS_DECL ('ods', 'http://www.openlinksw.com/virtuoso/ods/', 2);
-- Setup script for RDF view of OpenLink Product Portfolio version 2
--
/*
* This view is currently designed to SPECIFICALLY load on data.openlinksw.com. We recommend you
* search for `openlinksw.com' (as `http://' is sometimes encoded) and replace with either the default
* URIQA macro or your own hostname as appropriate.
*/
-- views to get the codes out instead of the id's
---- in hindsight these should ALL have been handled with
---- proper IRI functions like the rest, oh well, no harm AFAIK
---- will know for next time or might even replace this later.
DROP VIEW oplweb2.oplweb.product_formats_categories;
CREATE VIEW oplweb2.oplweb.product_formats_categories (
product_cat_code, product_cat_id, product_format_code, product_format_id, product_family_id
) as
SELECT distinct
pc.product_cat_code,
pc.product_cat_id,
pf.product_format_code,
pf.product_format_id,
pc.product_family_id
FROM
oplweb2.oplweb.product_category pc,
oplweb2.oplweb.product_format pf,
oplweb2.oplweb.product p
WHERE
p.product_cat_id = pc.product_cat_id and
p.product_format_id = pf.product_format_id
AND EXISTS (
SELECT 1 FROM oplweb2.oplweb.component_archive ca
WHERE ca.product_id = p.product_id
)
;
DROP VIEW oplweb2.oplweb.product_with_code;
CREATE VIEW oplweb2.oplweb.product_with_code as
SELECT p.*, pc.product_cat_code, pf.product_format_code
FROM
oplweb2.oplweb.product_category pc,
oplweb2.oplweb.product_format pf,
oplweb2.oplweb.product p
WHERE
p.product_cat_id = pc.product_cat_id and
p.product_format_id = pf.product_format_id ;
DROP VIEW oplweb2.oplweb.product_category_with_code;
CREATE VIEW oplweb2.oplweb.product_category_with_code as
SELECT pc.*, pf.product_family_code
from
oplweb2.oplweb.product_category pc,
oplweb2.oplweb.product_family pf
where
pc.product_family_id = pf.product_family_id ;
DROP VIEW oplweb2.oplweb.product_family_features_with_code;
CREATE VIEW oplweb2.oplweb.product_family_features_with_code as
SELECT a.*, b.product_family_code
from
oplweb2.oplweb.product_family_features a,
oplweb2.oplweb.product_family b
where
a.product_family_id = b.product_family_id
;
DROP VIEW oplweb2.oplweb.product_format_features_with_code;
CREATE VIEW oplweb2.oplweb.product_format_features_with_code as
SELECT a.*, b.product_format_code
from
oplweb2.oplweb.product_format_features a,
oplweb2.oplweb.product_format b
where
a.product_format_id = b.product_f