16.2.1.SPARQL Implementation Details

Virtuoso's RDF support includes in-built support for the SPARQL query language. It also includes a number of powerful extensions that cover path traversal and business intelligence features. In addition, there is in-built security based on Virtuoso's support for row level policy-based security, custom authentication, and named graphs.

The current implementation does not support some SPARQL features:

  • Unicode characters in names are not supported.

  • Comments inside SPARQL queries are not supported when the query is inlined in SQL code.

On the other hand, Virtuoso implements some extensions to SPARQL:

  • SPARUL statements, such as

    insert , modify , load etc, are supported.

  • The SPARQL compiler can be configured using

    define ...

    clauses, e.g.

    define output:valmode "LONG"

    .

  • Expressions are allowed in triple patterns, both in a

    where

    clause and in constructor patterns. Such expressions are delimited by backquotes.

  • Expressions are allowed in select statement result lists.

  • Parameters can be passed to the query from outside, using

    ?:variablename

    syntax.

  • Aggregate functions are supported.

  • Subqueries may appear where group patterns are allowed.

  • A set of operators has been added to configure the mapping of relational data to RDF (aka Linked Data Views).

The following listing shows the SPARQL grammar expressed in BNF, including all Virtuoso extensions but excluding rules for the syntax of each lexical element. Rule numbers in square brackets are from W3C normative SPARQL grammar. An asterisk indicates that the rule differs from the W3C grammar due to Virtuoso extensions - [Virt] means that the rule is Virtuoso-specific, [DML] indicates a data manipulation language extension from SPARUL.

[1]*    Query            ::=  Prolog ( QueryBody | SparulAction* | ( QmStmt ('.' QmStmt)* '.'? ) )
[1]     QueryBody        ::=  SelectQuery | ConstructQuery | DescribeQuery | AskQuery
[2]*    Prolog           ::=  Define* BaseDecl? PrefixDecl*
[Virt]  Define           ::=  'DEFINE' QNAME (QNAME | Q_IRI_REF | String )
[3]     BaseDecl         ::=  'BASE' Q_IRI_REF
[4]     PrefixDecl       ::=  'PREFIX' QNAME_NS Q_IRI_REF
[5]*    SelectQuery      ::=  'SELECT' 'DISTINCT'? ( ( Retcol ( ','? Retcol )* ) | '*' )
                        DatasetClause* WhereClause SolutionModifier
[6]     ConstructQuery   ::=  'CONSTRUCT' ConstructTemplate DatasetClause* WhereClause SolutionModifier
                        DatasetClause* WhereClause? SolutionModifier
[8]     AskQuery         ::=  'ASK' DatasetClause* WhereClause
[9]     DatasetClause    ::=  'FROM' ( DefaultGraphClause | NamedGraphClause )
[10]*   DefaultGraphClause       ::=  SourceSelector SpongeOptionList?
[11]*   NamedGraphClause         ::=  'NAMED' SourceSelector SpongeOptionList?
[Virt]  SpongeOptionList         ::=  'OPTION' '(' ( SpongeOption ( ',' SpongeOption )* )? ')'
[Virt]  SpongeOption     ::=  QNAME PrecodeExpn
[Virt]  PrecodeExpn      ::=  Expn      (* Only global variables can occur in Expn, local cannot *)
[13]    WhereClause      ::=  'WHERE'? GroupGraphPattern
[14]    SolutionModifier         ::=  OrderClause?
                        ((LimitClause OffsetClause?) | (OffsetClause LimitClause?))?
[15]    OrderClause      ::=  'ORDER' 'BY' OrderCondition+
[16]*   OrderCondition   ::=  ( 'ASC' | 'DESC' )?
                        ( FunctionCall | Var | ( '(' Expn ')' ) | ( '[' Expn ']' ) )
[17]    LimitClause      ::=  'LIMIT' INTEGER
[17]    LimitClause      ::=  'LIMIT' INTEGER
[18]    OffsetClause     ::=  'OFFSET' INTEGER
[18]    OffsetClause     ::=  'OFFSET' INTEGER
[19]*   GroupGraphPattern        ::=  '{' ( GraphPattern | SelectQuery ) '}'
[20]    GraphPattern     ::=  Triples? ( GraphPatternNotTriples '.'? GraphPattern )?
[21]*   GraphPatternNotTriples   ::=
                        QuadMapGraphPattern
                        | OptionalGraphPattern
                        | GroupOrUnionGraphPattern
                        | GraphGraphPattern
                        | Constraint
[22]    OptionalGraphPattern     ::=  'OPTIONAL' GroupGraphPattern
[Virt]  QuadMapGraphPattern      ::=  'QUAD' 'MAP' ( IRIref | '*' ) GroupGraphPattern
[23]    GraphGraphPattern        ::=  'GRAPH' VarOrBlankNodeOrIRIref GroupGraphPattern
[24]    GroupOrUnionGraphPattern         ::=  GroupGraphPattern ( 'UNION' GroupGraphPattern )*
[25]*   Constraint       ::=  'FILTER' ( ( '(' Expn ')' ) | BuiltInCall | FunctionCall )
[26]*   ConstructTemplate        ::=  '{' ConstructTriples '}'
[27]    ConstructTriples         ::=  ( Triples1 ( '.' ConstructTriples )? )?
[28]    Triples          ::=  Triples1 ( '.' Triples? )?
[29]    Triples1         ::=  VarOrTerm PropertyListNotEmpty | TriplesNode PropertyList
[30]    PropertyList     ::=  PropertyListNotEmpty?
[31]    PropertyListNotEmpty     ::=  Verb ObjectList ( ';' PropertyList )?
[32]*   ObjectList       ::=  ObjGraphNode ( ',' ObjectList )?
[Virt]  ObjGraphNode     ::=  GraphNode TripleOptions?
[Virt]  TripleOptions    ::=  'OPTION' '(' TripleOption ( ',' TripleOption )? ')'
[Virt]  TripleOption     ::=  'INFERENCE' ( QNAME | Q_IRI_REF | SPARQL_STRING )
[33]    Verb             ::=  VarOrBlankNodeOrIRIref | 'a'
[34]    TriplesNode      ::=  Collection | BlankNodePropertyList
[35]    BlankNodePropertyList    ::=  '[' PropertyListNotEmpty ']'
[36]    Collection       ::=  '(' GraphNode* ')'
[37]    GraphNode        ::=  VarOrTerm | TriplesNode
[38]    VarOrTerm        ::=  Var | GraphTerm
[39]*   VarOrIRIrefOrBackquoted  ::=  Var | IRIref | Backquoted
[40]*   VarOrBlankNodeOrIRIrefOrBackquoted       ::=  Var | BlankNode | IRIref | Backquoted
[Virt]  Retcol   ::=  ( Var | ( '(' Expn ')' ) | RetAggCall ) ( 'AS' ( VAR1 | VAR2 ) )?
[Virt]  RetAggCall       ::=  AggName '(', ( '*' | ( 'DISTINCT'? Var ) ) ')'
[Virt]  AggName  ::=  'COUNT' | 'AVG' | 'MIN' | 'MAX' | 'SUM'
[41]*   Var      ::=  VAR1 | VAR2 | GlobalVar | ( Var ( '+>' | '*>' ) IRIref )
[Virt]  GlobalVar        ::=  QUEST_COLON_PARAMNAME | DOLLAR_COLON_PARAMNAME
                        | QUEST_COLON_PARAMNUM | DOLLAR_COLON_PARAMNUM
[42]*   GraphTerm        ::=  IRIref | RDFLiteral | ( '-' | '+' )? NumericLiteral
                        | BooleanLiteral | BlankNode | NIL | Backquoted
[Virt]  Backquoted       ::=  '`' Expn '`'
[43]    Expn             ::=  ConditionalOrExpn
[44]    ConditionalOrExpn        ::=  ConditionalAndExpn ( '||' ConditionalAndExpn )*
[45]    ConditionalAndExpn       ::=  ValueLogical ( '&&' ValueLogical )*
[46]    ValueLogical     ::=  RelationalExpn
[47]*   RelationalExpn   ::=  NumericExpn
                        ( ( ('='|'!='|'<'|'>'|'<='|'>='|'LIKE') NumericExpn )
                        | ( 'IN' '(' Expns ')' ) )?
[49]    AdditiveExpn     ::=  MultiplicativeExpn ( ('+'|'-') MultiplicativeExpn )*
[50]    MultiplicativeExpn       ::=  UnaryExpn ( ('*'|'/') UnaryExpn )*
[51]    UnaryExpn        ::=   ('!'|'+'|'-')? PrimaryExpn
[58]    PrimaryExpn      ::=
                        BracketedExpn | BuiltInCall | IRIrefOrFunction
                        | RDFLiteral | NumericLiteral | BooleanLiteral | BlankNode | Var
[55]    IRIrefOrFunction         ::=  IRIref ArgList?
[52]*   BuiltInCall      ::=
                        ( 'STR' '(' Expn ')' )
                        | ( 'IRI' '(' Expn ')' )
                        | ( 'LANG' '(' Expn ')' )
                        | ( 'LANGMATCHES' '(' Expn ',' Expn ')' )
                        | ( 'DATATYPE' '(' Expn ')' )
                        | ( 'BOUND' '(' Var ')' )
                        | ( 'sameTERM' '(' Expn ',' Expn ')' )
                        | ( 'isIRI' '(' Expn ')' )
                        | ( 'isURI' '(' Expn ')' )
                        | ( 'isBLANK' '(' Expn ')' )
                        | ( 'isLITERAL' '(' Expn ')' )
                        | RegexExpn
[53]    RegexExpn        ::=  'REGEX' '(' Expn ',' Expn ( ',' Expn )? ')'
[54]    FunctionCall     ::=  IRIref ArgList
[56]*   ArgList  ::=  ( NIL | '(' Expns ')' )
[Virt]  Expns    ::=  Expn ( ',' Expn )*
[59]    NumericLiteral   ::=  INTEGER | DECIMAL | DOUBLE
[60]    RDFLiteral       ::=  String ( LANGTAG | ( '^^' IRIref ) )?
[61]    BooleanLiteral   ::=  'true' | 'false'
[63]    IRIref           ::=  Q_IRI_REF | QName
[64]    QName            ::=  QNAME | QNAME_NS
[65]*   BlankNode        ::=  BLANK_NODE_LABEL | ( '[' ']' )
[DML]   SparulAction     ::=
                        CreateAction | DropAction | LoadAction
                        | InsertAction | InsertDataAction | DeleteAction | DeleteDataAction
                        | ModifyAction | ClearAction
[DML]*  InsertAction     ::=
                        'INSERT' ( ( 'IN' | 'INTO ) 'GRAPH' ( 'IDENTIFIED' 'BY' )? )? PrecodeExpn
                        ConstructTemplate ( DatasetClause* WhereClause SolutionModifier )?
[DML]*  InsertDataAction         ::=
                        'INSERT' 'DATA' ( ( 'IN' | 'INTO ) 'GRAPH' ( 'IDENTIFIED' 'BY' )? )?
                        PrecodeExpn ConstructTemplate
[DML]*  DeleteAction     ::=
                        'DELETE' ( 'FROM' 'GRAPH' ( 'IDENTIFIED' 'BY' )? )? PrecodeExpn
                        ConstructTemplate ( DatasetClause* WhereClause SolutionModifier )?
[DML]*  DeleteDataAction         ::=
                        'DELETE' 'DATA' ( 'FROM' 'GRAPH' ( 'IDENTIFIED' 'BY' )? )?
                        PrecodeExpn ConstructTemplate
[DML]*  ModifyAction     ::=
                        'MODIFY' ( 'GRAPH' ( 'IDENTIFIED' 'BY' )? PrecodeExpn?
                        'DELETE' ConstructTemplate 'INSERT' ConstructTemplate
                        ( DatasetClause* WhereClause SolutionModifier )?
[DML]*  ClearAction      ::=  'CLEAR' ( 'GRAPH' ( 'IDENTIFIED' 'BY' )? PrecodeExpn )?
[DML]*  LoadAction       ::=  'LOAD' PrecodeExpn
                        ( ( 'IN' | 'INTO' ) 'GRAPH' ( 'IDENTIFIED' 'BY' )? PrecodeExpn )?
[DML]*  CreateAction     ::=  'CREATE' 'SILENT'? 'GRAPH' ( 'IDENTIFIED' 'BY' )? PrecodeExpn
[DML]*  DropAction       ::=  'DROP' 'SILENT'? 'GRAPH' ( 'IDENTIFIED' 'BY' )? PrecodeExpn
[Virt]  QmStmt           ::=  QmSimpleStmt | QmCreateStorage | QmAlterStorage
[Virt]  QmSimpleStmt     ::=
                        QmCreateIRIClass | QmCreateLiteralClass | QmDropIRIClass | QmDropLiteralClass
                        | QmCreateIRISubclass | QmDropQuadStorage | QmDropQuadMap
[Virt]  QmCreateIRIClass         ::=  'CREATE' 'IRI' 'CLASS' QmIRIrefConst
                        ( ( String QmSqlfuncArglist )
                        | ( 'USING' QmSqlfuncHeader ',' QmSqlfuncHeader ) )
[Virt]  QmCreateLiteralClass     ::=  'CREATE' 'LITERAL' 'CLASS' QmIRIrefConst
                        'USING' QmSqlfuncHeader ',' QmSqlfuncHeader QmLiteralClassOptions?
[Virt]  QmDropIRIClass   ::=  'DROP' 'IRI' 'CLASS' QmIRIrefConst
[Virt]  QmDropLiteralClass       ::=  'DROP' 'LITERAL' 'CLASS' QmIRIrefConst
[Virt]  QmCreateIRISubclass      ::=  'IRI' 'CLASS' QmIRIrefConst 'SUBCLASS' 'OF' QmIRIrefConst
[Virt]  QmIRIClassOptions        ::=  'OPTION' '(' QmIRIClassOption (',' QmIRIClassOption)* ')'
[Virt]  QmIRIClassOption         ::=
                        'BIJECTION'
                        | 'DEREF'
                        | 'RETURNS' STRING ('UNION' STRING)*
[Virt]  QmLiteralClassOptions    ::=  'OPTION' '(' QmLiteralClassOption (',' QmLiteralClassOption)* ')'
[Virt]  QmLiteralClassOption     ::=
                        ( 'DATATYPE' QmIRIrefConst )
                        | ( 'LANG' STRING )
                        | ( 'LANG' STRING )
                        | 'BIJECTION'
                        | 'DEREF'
                        | 'RETURNS' STRING ('UNION' STRING)*
[Virt]  QmCreateStorage  ::=  'CREATE' 'QUAD' 'STORAGE' QmIRIrefConst QmSourceDecl* QmMapTopGroup
[Virt]  QmAlterStorage   ::=  'ALTER' 'QUAD' 'STORAGE' QmIRIrefConst QmSourceDecl* QmMapTopGroup
[Virt]  QmDropStorage    ::=  'DROP' 'QUAD' 'STORAGE' QmIRIrefConst
[Virt]  QmDropQuadMap    ::=  'DROP' 'QUAD' 'MAP' 'GRAPH'? QmIRIrefConst
[Virt]  QmDrop   ::=  'DROP' 'GRAPH'? QmIRIrefConst
[Virt]  QmSourceDecl     ::=
                        ( 'FROM' QTABLE 'AS' PLAIN_ID QmTextLiteral* )
                        | ( 'FROM' PLAIN_ID 'AS' PLAIN_ID QmTextLiteral* )
                        | QmCondition
[Virt]  QmTextLiteral    ::=  'TEXT' 'XML'? 'LITERAL' QmSqlCol ( 'OF' QmSqlCol )? QmTextLiteralOptions?
[Virt]  QmTextLiteralOptions     ::=  'OPTION' '(' QmTextLiteralOption ( ',' QmTextLiteralOption )* ')'
[Virt]  QmMapTopGroup    ::=  '{' QmMapTopOp ( '.' QmMapTopOp )* '.'? '}'
[Virt]  QmMapTopOp       ::=  QmMapOp | QmDropQuadMap | QmDrop
[Virt]  QmMapGroup       ::=  '{' QmMapOp ( '.' QmMapOp )* '.'? '}'
[Virt]  QmMapOp          ::=
                        ( 'CREATE' QmIRIrefConst 'AS' QmMapIdDef )
                        | ( 'CREATE' 'GRAPH'? QmIRIrefConst 'USING' 'STORAGE' QmIRIrefConst QmOptions? )
                        | ( QmNamedField+ QmOptions? QmMapGroup )
                        | QmTriples1
[Virt]  QmMapIdDef       ::=  QmMapTriple | ( QmNamedField+ QmOptions? QmMapGroup )
[Virt]  QmMapTriple      ::=  QmFieldOrBlank QmVerb QmObjField
[Virt]  QmTriples1       ::=  QmFieldOrBlank QmProps
[Virt]  QmNamedField     ::=  ('GRAPH'|'SUBJECT'|'PREDICATE'|'OBJECT') QmField
[Virt]  QmProps          ::=  QmProp ( ';' QmProp )?
[Virt]  QmProp           ::=  QmVerb QmObjField ( ',' QmObjField )*
[Virt]  QmObjField       ::=  QmFieldOrBlank QmCondition* QmOptions?
[Virt]  QmIdSuffix       ::=  'AS' QmIRIrefConst
[Virt]  QmVerb           ::=  QmField | ( '[' ']' ) | 'a'
[Virt]  QmFieldOrBlank   ::=  QmField | ( '[' ']' )
[Virt]  QmField          ::=
                        NumericLiteral
                        | RdfLiteral
                        | ( QmIRIrefConst ( '(' ( QmSqlCol ( ',' QmSqlCol )* )? ')' )? )
                        | QmSqlCol
[Virt]  QmCondition      ::=  'WHERE' ( ( '(' SQLTEXT ')' ) | String )
[Virt]  QmOptions        ::=  'OPTION' '(' QmOption ( ',' QmOption )* ')'
[Virt]  QmOption         ::=  ( 'SOFT'? 'EXCLUSIVE' ) | ( 'ORDER' INTEGER ) | ( 'USING' PLAIN_ID )
[Virt]  QmSqlfuncHeader  ::=  'FUNCTION' SQL_QTABLECOLNAME QmSqlfuncArglist 'RETURNS' QmSqltype
[Virt]  QmSqlfuncArglist         ::=  '(' ( QmSqlfuncArg ( ',' QmSqlfuncArg )* )? ')'
[Virt]  QmSqlfuncArg     ::=  ('IN' | QmSqlId) QmSqlId QmSqltype
[Virt]  QmSqltype        ::=  QmSqlId ( 'NOT' 'NULL' )?
[Virt]  QmSqlCol         ::=  QmSqlId | spar_qm_sql_id
[Virt]  QmSqlId          ::=  PLAIN_ID | 'TEXT' | 'XML'
[Virt]  QmIRIrefConst    ::=  IRIref | ( 'IRI' '(' String ')' )

Example: Using OFFSET and LIMIT

Virtuoso uses a zero-based index for OFFSET. Thus, in the example below, the query returns 1000 rows starting from, and including, record 9001 of the result set. Note that the default value of the MaxSortedTopRows parameter in the [Parameters] section of the virtuoso.ini configuration file defaults to 10000, so in this example its value will need to have been increased beforehand.

SQL>SELECT ?name
ORDER BY ?name
OFFSET 9000
LIMIT 1000

LIMIT applies to the solution resulting from the graph patterns specified in the WHERE CLAUSE. This implies that SELECT and CONSTRUCT/DESCRIBE queries will behave a little differently. In the case of a SELECT, there is a straight translation i.e. LIMIT 4 implies 4 records in the result set. In the case of CONSTRUCTs where the solution is a graph (implying that the existence of duplicates and/or unbound variables is common) LIMIT is basically a maximum triples threshold of: [Solution Triples] x [LIMIT].

Example query:

SQL>SPARQL
prefix dct:<http://purl.org/dc/terms/>
prefix rdfs:<http://www.w3.org/2000/01/rdf-schema#>

CONSTRUCT { ?resource dct:title ?title ;
                      a ?type }

FROM <http://msone.computas.no/graphs/inferred/classification>
FROM <http://msone.computas.no/graphs>
FROM <http://msone.computas.no/graphs/instance/nfi>
FROM <http://msone.computas.no/graphs/instance/mo>
FROM <http://msone.computas.no/graphs/ontology/mediasone>
FROM <http://msone.computas.no/graphs/vocab/mediasone>
FROM <http://msone.computas.no/graphs/inferred/nfi/realisation1>
FROM <http://msone.computas.no/graphs/inferred/mo/realisation1>
FROM <http://msone.computas.no/graphs/inferred/nfi/realisation2>
FROM <http://msone.computas.no/graphs/inferred/mo/realisation2>
FROM <http://msone.computas.no/graphs/inferred/agent-classification>
FROM <http://msone.computas.no/graphs/ontology/mediasone/agent>

WHERE {
  {
?resource a ?type .
  FILTER (?type = <http://www.w3.org/2002/07/owl#Class> ) .
  ?resource rdfs:label ?title .
  } UNION {
?resource a ?type .
  FILTER (?type in (
          <http://musicbrainz.org/mm/mm-2.1#Track> ,
          <http://www.csd.abdn.ac.uk/~ggrimnes/dev/imdb/IMDB#Movie> ,
          <http://xmlns.com/foaf/0.1/Image> ,
          <http://www.computas.com/mediasone#Text> ) ) .
  ?resource dct:title ?title .
  }
  FILTER regex(?title, "turi", "i")
}
ORDER BY ?title LIMIT 4 OFFSET 0

Example: Prevent Limits of Sorted LIMIT/OFFSET query

The DBpedia SPARQL endpoint is configured with the following INI setting:

MaxSortedTopRows = 40000

The setting above sets a threshold for sorted rows. Thus, when using basic SPARQL queries that include OFFSET and LIMIT the following query will still exist the hard limit set in the INI:

DEFINE sql:big-data-const 0
SELECT DISTINCT  ?p ?s
FROM <http://dbpedia.org>
WHERE
  {
    ?s ?p <http://dbpedia.org/resource/Germany>
  }
ORDER BY ASC(?p)
OFFSET  40000
LIMIT   1000

returns the following error on execution:

HttpException: 500 SPARQL Request Failed

Virtuoso 22023 Error SR353: Sorted TOP clause specifies more then 41000 rows to sort.
Only 40000 are allowed.
Either decrease the offset and/or row count or use a scrollable cursor

To prevent the problem outlined above you can leverage the use of subqueries which make better use of temporary storage associated with this kind of quest. An example would take the form:

SELECT ?p ?s
WHERE
  {
    {
      SELECT DISTINCT ?p ?s
      FROM <http://dbpedia.org>
      WHERE
        {
          ?s ?p <http://dbpedia.org/resource/Germany>
        } ORDER BY ASC(?p)
    }
  }
OFFSET 50000
LIMIT 1000

SPARQL and XQuery Core Function Library

In the current implementation, the XQuery Core Function Library is not available from SPARQL.

As a temporary workaround, string parsing functions are made available, because they are widely used in W3C DAWG examples and the like. They are:

xsd:boolean (in strg any) returns integer
xsd:dateTime (in strg any) returns datetime
xsd:double (in strg varchar) returns double precision
xsd:float (in strg varchar) returns float
xsd:integer (in strg varchar) returns integer

(assuming that the query contains the declaration: 'PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>')