http://docs.openlinksw.com/virtuoso/rdfandsparql.html OpenLink Virtuoso Universal Server: Documentation virtuoso.docs@openlinksw.com Mon, 28 Jul 2008 11:35:38 GMT OpenLink Software Documentation Team webmaster@openlinksw.com http://docs.openlinksw.com/virtuoso/../images/misc/logo.jpg http://docs.openlinksw.com/virtuoso/rdfandsparql.html OpenLink Virtuoso Universal Server: Documentation http://docs.openlinksw.com/virtuoso/rdfoverview.html virtuoso.docs@openlinksw.com Overview http://docs.openlinksw.com/virtuoso/rdfoverview.html Mon, 28 Jul 2008 11:35:38 GMT The support of SPARQL in Virtuoso consists of two "layers" -- RDF support in SQL engine and SPARQL front-end compiler that translates SPARQL queries to SQL. In its core, Virtuoso extends relational storage and SQL language with datatypes and language constructs required for handling RDF data like any "traditional" SQL datatype. There exist datatypes for RDF references (IRIs and blank nodes) and for RDF literals with types and languages. These datatypes are widely supported by built-in functions and there exists convenient conversions between RDF literals and other SQL datatypes. SQL query language is extended with constructs that are convenient for "very heterogeneous" data of RDF graph. In traditional SQL queries, data types of retrieved values are mostly known in advance from the database schema via explicitly declared column types and types of return values of functions. An RDF graph contains mix of literals of all types, so cast errors are very frequent and a typical query should not terminate after some occasional type error in a huge data set. http://docs.openlinksw.com/virtuoso/rdfdatarepresentation.html virtuoso.docs@openlinksw.com Data Representation http://docs.openlinksw.com/virtuoso/rdfdatarepresentation.html Mon, 28 Jul 2008 11:35:38 GMT This section covers how Virtuoso stores RDF triples. The IRI_ID built-in data type is introduced, along with the default table structures used for triple persistency. These details are mostly hidden from users of RDF, thus this section is not necessary reading for typical use of Virtuoso with RDF. http://docs.openlinksw.com/virtuoso/rdfapiandsql.html virtuoso.docs@openlinksw.com RDF and SPARQL API and SQL http://docs.openlinksw.com/virtuoso/rdfapiandsql.html Mon, 28 Jul 2008 11:35:38 GMT SPARQL can be used inline wherever SQL can be used. The only API functions that one needs to know are the ones for loading RDF data into the store. Dynamic SQL client applications can issue SPARQL queries against Virtuoso through the regular SQL client API, ODBC, JDBC or other, simply by prefixing the SPARQL query with the SPARQL keyword. Parameters work just as with dynamic SQL. Stored procedures can have SPARQL expressions inline and can declare cursors over SPARQL result sets. Value conversions between SQL and SPARQL are most often automatic and invisible. In some cases one needs to be aware of the different SPARQL value representations (valmodes). SPARQL offers declarations for determining if graphs to be returned are to be represented as XML or Turtle text serialization or whether these will be hash tables of triples. See dict_new() and related functions for a description of the hash table SQL data type. The use of dict's is convenient for further programmatic processing of graphs. RDF-related procedures use Virtuoso/PL vectors and dictionaries to represent RDF triples and sets of triples. http://docs.openlinksw.com/virtuoso/rdfsparul.html virtuoso.docs@openlinksw.com SPARUL -- an Update Language For RDF Graphs http://docs.openlinksw.com/virtuoso/rdfsparul.html Mon, 28 Jul 2008 11:35:38 GMT http://docs.openlinksw.com/virtuoso/rdfinsertmethods.html virtuoso.docs@openlinksw.com RDF Insert Methods in Virtuoso http://docs.openlinksw.com/virtuoso/rdfinsertmethods.html Mon, 28 Jul 2008 11:35:38 GMT http://docs.openlinksw.com/virtuoso/virtuososponger.html virtuoso.docs@openlinksw.com Virtuoso Sponger http://docs.openlinksw.com/virtuoso/virtuososponger.html Mon, 28 Jul 2008 11:35:38 GMT The Virtuoso Sponger is a middleware component of Virtuoso that generates RDF Linked Data from a variety of data sources. The sponger is transparently integrated into the Virtuoso SPARQL Query Processor, where it serves as part of the URI/IRI dereferencing functionality. It is also optionally used by the Virtuoso Content Crawler. A majority of the worlds data naturally resides in non RDF form at the current time. The Sponger delivers middleware that accelerates the bootstrap of the Semantic Data Web by generating RDF from non RDF data sources, unobtrusively. When an RDF aware client requests data from a network accessible resource via the Sponger the following events occur: http://docs.openlinksw.com/virtuoso/rdfiridereferencing.html virtuoso.docs@openlinksw.com Dereferencable IRIs and RDF Linked Data http://docs.openlinksw.com/virtuoso/rdfiridereferencing.html Mon, 28 Jul 2008 11:35:38 GMT There are many cases when RDF data should be retrieved from remote sources only when really needed. E.g., a scheduling application may read personal calendars from personal sites of its users. Calendar data expire quickly, so there's no reason to frequently re-load them in hope that they are queried before expired. Virtuoso extends SPARQL so it is possible to download RDF resource from a given IRI, parse them and store the resulting triples in a graph, all three operations will be performed during the SPARQL query execution. The IRI of graph to store triples is usually equal to the IRI where the resource is download from, so the feature is named "IRI dereferencing" There are two different use cases for this feature. In simple case, a SPARQL query contains from clauses that enumerate graphs to process, but there are no triples in DB.DBA.RDF_QUAD that correspond to some of these graphs. The query execution starts with dereferencing of these graphs and the rest runs as usual. In more sophisticated case, the query is executed many times in a loop. Every execution produces a partial result. SPARQL processor checks for IRIs in the result such that resources with that IRIs may contain relevant data but not yet loaded into the DB.DBA.RDF_QUAD. After some iteration, the partial result is identical to the result of the previous iteration, because there's no more data to retrieve. As the last step, SPARQL processor builds the final result set. http://docs.openlinksw.com/virtuoso/rdfviews.html virtuoso.docs@openlinksw.com RDF Views -- Mapping Relational Data to RDF http://docs.openlinksw.com/virtuoso/rdfviews.html Mon, 28 Jul 2008 11:35:38 GMT RDF Views map relational data into RDF and allow customizing RDF representation of locally stored RDF data. To let SPARQL clients access relational data as well as physical RDF graphs in a single query, we introduce a declarative Meta Schema Language for mapping SQL Data to RDF Ontologies. As a result, all types of clients can efficiently access all data stored on the server. The mapping functionality dynamically generates RDF Data Sets for popular ontologies such as SIOC, SKOS, FOAF, and ATOM/OWL without disruption to the existing database infrastructure of Web 1.0 or Web 2.0 solutions. RDF views are also suitable for declaring custom representation for RDF triples, e.g. property tables, where one row holds many single-valued properties. http://docs.openlinksw.com/virtuoso/rdfsparqlrule.html virtuoso.docs@openlinksw.com RDF Inference in Virtuoso http://docs.openlinksw.com/virtuoso/rdfsparqlrule.html Mon, 28 Jul 2008 11:35:38 GMT http://docs.openlinksw.com/virtuoso/rdfsparqlrulefulltext.html virtuoso.docs@openlinksw.com Using Full Text Search in SPARQL http://docs.openlinksw.com/virtuoso/rdfsparqlrulefulltext.html Mon, 28 Jul 2008 11:35:38 GMT Virtuoso's triple store supports optional full text indexing of RDF object values since version 5.0. It is possible to declare that objects of triples with a given predicate or graph get indexed. The graphs and triples may be enumerated or a wildcard may be used. The triples for which a full text index entry exists can be found using the bif:contains or related filters and predicates. For example, the query: http://docs.openlinksw.com/virtuoso/rdfsparqlprotocolendpoint.html virtuoso.docs@openlinksw.com Virtuoso SPARQL Query Service http://docs.openlinksw.com/virtuoso/rdfsparqlprotocolendpoint.html Mon, 28 Jul 2008 11:35:38 GMT http://docs.openlinksw.com/virtuoso/sparqlbi.html virtuoso.docs@openlinksw.com Business Intelligence Extensions for SPARQL http://docs.openlinksw.com/virtuoso/sparqlbi.html Mon, 28 Jul 2008 11:35:38 GMT Virtuoso extends SPARQL with expressions in results, subqueries, aggregates and grouping. These extensions allow a straightforward translation of arbitrary SQL queries to SPARQL. This extension is called "SPARQL BI", because the primary objective is to match needs of Business Intelligence. The extended features apply equally to querying physical quads or relational tables mapped through RDF views. http://docs.openlinksw.com/virtuoso/sparqldebug.html virtuoso.docs@openlinksw.com Debugging SPARQL queries http://docs.openlinksw.com/virtuoso/sparqldebug.html Mon, 28 Jul 2008 11:35:38 GMT A short SPARQL query can be compiled into a long SQL statement, especially if data comes from many quad map petterns. Middle-size application with 50 tables and 10 columns per table may create thousand of quad map patterns for subjects of hundred different types. An attempt to "select everything" from RDF view of that complexity may easily create 5000 lines of SQL code. Thus it is known in advance that some queries will be rejected even if same queries work fine on default storage of "physical quads". In addition, an SQL compiler signals an error if table or column name is unknown, efficiently catching typos. SPARQL uses IRIs that are long and sometimes unreadable, but there is no "closed world" schema of the data so a typo in an IRI is not an error, it is simply some other IRI. So typo in a IRI or in a namespace prefix cause missing bindings of some triple patterns of the query and incomplete result, but no errors are usually reported. Typo in graph or predicate IRI may cause the SPARQL compiler to generate code that access deafult storage instead of relational source or to generate empty code that accesses nothing. The SQL compiler does not signal casting errors when it runs the statement generated from SPARQL, because the generated SQL code contains option (QUIETCAST). This means that mismatches between expected and actual datatypes of values stay invisible and may cause rounding errors (say, integer division instead of floating-point) and even empty joins (due to join conditions that silently return NULL instead of return comparison error). http://docs.openlinksw.com/virtuoso/rdfperformancetuning.html virtuoso.docs@openlinksw.com Virtuoso RDF Performance Tuning http://docs.openlinksw.com/virtuoso/rdfperformancetuning.html Mon, 28 Jul 2008 11:35:38 GMT For RDF query performance, we have the following possible questions: http://docs.openlinksw.com/virtuoso/rdfstorebenchmarks.html virtuoso.docs@openlinksw.com RDF Store Benchmarks http://docs.openlinksw.com/virtuoso/rdfstorebenchmarks.html Mon, 28 Jul 2008 11:35:38 GMT http://docs.openlinksw.com/virtuoso/rdfsparqlimplementationextent.html virtuoso.docs@openlinksw.com SPARQL Implementation Details http://docs.openlinksw.com/virtuoso/rdfsparqlimplementationextent.html Mon, 28 Jul 2008 11:35:38 GMT Virtuoso's RDF support includes in-built support for the SPARQL query language. It also includes a number of powerful extensions that cover path traversal and business intelligence features. In addition, there is in-built security based on Virtuoso's support for Row Level policy based security, custom authentication, and Named Graphs. The current implementation does not support some SPARQL features. On the other hand, Virtuoso implements some extensions to SPARQL: http://docs.openlinksw.com/virtuoso/rdfnativestorageproviders.html virtuoso.docs@openlinksw.com Native RDF Storage Providers http://docs.openlinksw.com/virtuoso/rdfnativestorageproviders.html Mon, 28 Jul 2008 11:35:38 GMT