www.openlinksw.com
docs.openlinksw.com

Book Home

Contents
Preface

RDF Data Access and Data Management

Data Representation
SPARQL
Extensions
RDF Graphs Security
Linked Data Views over RDBMS Data Source
Automated Generation of Linked Data Views over Relational Data Sources
Virtuoso R2RML Support
Examples of Linked Data Views
RDF Insert Methods in Virtuoso
Using API functions SPARQL endpoint REST API HTTP PUT using Content-Type: application/rdf+xml SPARQL Insert using LOAD SPARQL Insert via /sparql endpoint SPARQL Insert via SPARQL endpoint REST API and ODS wiki Using WebDAV Using Virtuoso Crawler Using SPARQL Query and Sponger (i.e. we Fetch the Network Resources in the FROM Clause or values for the graph-uri parameter in SPARQL protocol URLs) Using Virtuoso PL APIs Using SIMILE RDF Bank API Using RDF NET Using the RDF Proxy (Sponger) Service
RDFizer Middleware (Sponger)
Virtuoso Faceted Browser Installation and configuration
Virtuoso Faceted Web Service
Linked Data
Inference Rules & Reasoning
RDF and Geometry
RDF Performance Tuning
RDF Data Access Providers (Drivers)
RDF Graph Replication

16.9. RDF Insert Methods in Virtuoso

16.9.1. Using API functions

See more details for loading Performance Tuning specifics.


16.9.2. SPARQL endpoint REST API

With POST can be accomplished SPARQL Insert/Update etc.

The result is in the rdf_quad.

With GET Methods you can get the triples which are saved.

Examples:

Example 1:

Create a DAV collection xx for user demo with password demo.

Execute the following command:

curl -i -d "INSERT {<http://demo.openlinksw.com/DAV/home/demo_about.rdf>
<http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://rdfs.org/sioc/ns#User>}" -u "demo:demo"
-H "Content-Type: application/sparql-query" http://localhost:8890/DAV/xx/yy

The response should be:

HTTP/1.1 201 Created
Server: Virtuoso/05.00.3023 (Win32) i686-generic-win-32  VDB
Connection: Keep-Alive
Content-Type: text/html; charset=ISO-8859-1
Date: Fri, 28 Dec 2007 12:50:12 GMT
Accept-Ranges: bytes
MS-Author-Via: SPARQL
Content-Length: 0

The result in the DAV/xx location will be a new WebDAV resource with name "yy" containing the inserted RDF:

<?xml version="1.0" encoding="utf-8" ?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
<rdf:Description
rdf:about="http://demo.openlinksw.com/DAV/home/demo_about.rdf">
<ns0pred:type xmlns:ns0pred="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
rdf:resource="http://rdfs.org/sioc/ns#User"/>
</rdf:Description>
</rdf:RDF>

Example 2:

Create a DAV collection, for ex. with name "test" for user ( for ex. demo).

Execute the following command:

curl -i -d "INSERT IN GRAPH <http://mygraph.com>
{ <http://www.openlinksw.com/dataspace/kidehen@openlinksw.com#this>
  <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
  <http://rdfs.org/sioc/ns#User> .
  <http://www.openlinksw.com/dataspace/kidehen@openlinksw.com#this>
  <http://www.w3.org/2000/01/rdf-schema#label>
  <Kingsley Uyi Idehen> .
  <http://www.openlinksw.com/dataspace/kidehen@openlinksw.com#this>
  <http://rdfs.org/sioc/ns#creator_of>

<http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1300>
  } " -u "demo:demo" -H "Content-Type: application/sparql-query" http://localhost:8890/DAV/home/demo/test/myrq

As result the response will be:

HTTP/1.1 201 Created
Server: Virtuoso/05.00.3023 (Win32) i686-generic-win-32  VDB
Connection: Keep-Alive
Content-Type: text/html; charset=ISO-8859-1
Date: Thu, 20 Dec 2007 16:25:25 GMT
Accept-Ranges: bytes
MS-Author-Via: SPARQL
Content-Length: 0

Now let's check the inserted triples. Go to the sparql endpoint, i.e. http://localhost:8890/sparql and:

  1. Enter for Default Graph URI:
    http://mygraph.com
    
  2. Enter in the Query area:
    SELECT * WHERE {?s ?p ?o}
    
  3. Click the button "Run Query"
  4. As result will be shown the inserted triples:
    s  	                                                          p                                                 o
    http://www.openlinksw.com/dataspace/kidehen@openlinksw.com#this   http://www.w3.org/1999/02/22-rdf-syntax-ns#type   http://rdfs.org/sioc/ns#User
    http://www.openlinksw.com/dataspace/kidehen@openlinksw.com#this   http://www.w3.org/2000/01/rdf-schema#label 	    Kingsley
    http://www.openlinksw.com/dataspace/kidehen@openlinksw.com#this   http://rdfs.org/sioc/ns#creator_of                http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1300
    

16.9.3. HTTP PUT using Content-Type: application/rdf+xml

The URI in a PUT request identifies the entity enclosed with the request. Therefore using HTTP PUT is a more useful and meaningful command than using POST (which is more about submitting data to a script).

Example:

Suppose there is myfoaf.rdf file with the following content:

<rdf:RDF xmlns="http://www.example/jose/foaf.rdf#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:log="http://www.w3.org/2000/10/swap/log#"
    xmlns:myfoaf="http://www.example/jose/foaf.rdf#"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">

    <foaf:Person rdf:about="http://www.example/jose/foaf.rdf#jose">
        <foaf:homepage rdf:resource="http://www.example/jose/"/>
        <foaf:knows rdf:resource="http://www.example/jose/foaf.rdf#juan"/>
        <foaf:name>Jose Jimen~ez</foaf:name>
        <foaf:nick>Jo</foaf:nick>
        <foaf:workplaceHomepage rdf:resource="http://www.corp.example/"/>
    </foaf:Person>

    <foaf:Person rdf:about="http://www.example/jose/foaf.rdf#juan">
        <foaf:mbox rdf:resource="mailto:juan@mail.example"/>
    </foaf:Person>

    <foaf:Person rdf:about="http://www.example/jose/foaf.rdf#julia">
        <foaf:mbox rdf:resource="mailto:julia@mail.example"/>
    </foaf:Person>

    <rdf:Description rdf:about="http://www.example/jose/foaf.rdf#kendall">
        <foaf:knows rdf:resource="http://www.example/jose/foaf.rdf#edd"/>
    </rdf:Description>
</rdf:RDF>

Now let's upload the myfoaf.rdf file to destination server demo.openlinksw.com for user demo:

curl -T myfoaf.rdf http://demo.openlinksw.com/DAV/home/demo/rdf_sink/myfoaf.rdf -u demo:demo

As result the response should be:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<HTML>
<HEAD>
  <TITLE>201 Created</TITLE>
</HEAD>
<BODY>
  <H1>Created</H1>
  Resource /DAV/home/demo/rdf_sink/ myfoaf.rdf has been created.
</BODY>
</HTML>

Then you can execute:

curl -F "query=SELECT DISTINCT ?p FROM <http://demo.openlinksw.com/DAV/home/demo/rdf_sink/> WHERE {?s ?p ?o}" http://demo.openlinksw.com/sparql

The result should be:

<?xml version="1.0" ?>
<sparql xmlns="http://www.w3.org/2005/sparql-results#" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.w3.org/2001/sw/DataAccess/rf1/result2.xsd">
 <head>
  <variable name="p"/>
 </head>
 <results distinct="false" ordered="true">
  <result>
   <binding name="p"><uri>http://www.w3.org/1999/02/22-rdf-syntax-ns#type</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/nick</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/name</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/homepage</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/knows</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/workplaceHomepage</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/mbox</uri></binding>
  </result>
 </results>
</sparql>

Other examples with curl:

curl -F "query=SELECT distinct ?Concept FROM <http://dbpedia.org> WHERE {?s a ?Concept} limit 10" http://dbpedia.org/sparql
curl -F "query=SELECT distinct ?Concept FROM <http://myopenlink.net/dataspace/person/kidehen> WHERE {?s a ?Concept} limit 10" http://demo.openlinksw.com/sparql
curl -F "query=SELECT distinct ?Concept FROM <http://data.openlinksw.com/oplweb/product_family/virtuoso> WHERE {?s a ?Concept} limit 10" http://demo.openlinksw.com/sparql
curl -F "query=SELECT distinct ?Concept FROM <http://openlinksw.com/dataspace/organization/openlink> WHERE {?s a ?Concept} limit 10" http://demo.openlinksw.com/sparql

16.9.4. SPARQL Insert using LOAD

SPARQL INSERT operation can be done using the LOAD features:


SPARQL INSERT INTO <..> { .... } [[FROM ...] { ... }]

SPARQL LOAD <x> [INTO <y>]

-- <ResourceURL> will be the Graph IRI of the loaded data:
SPARQL LOAD <ResourceURL>

Examples:

  1. Load from ISQL:
    SPARQL insert in graph <http://mygraph.com>
    {
      <http://myopenlink.net/dataspace/Kingsley#this>
      <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
      <http://rdfs.org/sioc/ns#User> .
    
      <http://myopenlink.net/dataspace/Kingsley#this>
      <http://rdfs.org/sioc/ns#id>
      <Kingsley> .
    
      <http://myopenlink.net/dataspace/Caroline#this>
      <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
      <http://rdfs.org/sioc/ns#User> .
    
     <http://myopenlink.net/dataspace/Caroline#this>
      <http://rdfs.org/sioc/ns#id>
      <Caroline> .
    
       <http://myopenlink.net/dataspace/Matt#this>
      <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
      <http://rdfs.org/sioc/ns#User> .
    
      <http://myopenlink.net/dataspace/Matt#this>
      <http://rdfs.org/sioc/ns#id>
      <Matt> .
    
       <http://myopenlink.net/dataspace/demo#this>
      <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
      <http://rdfs.org/sioc/ns#User> .
    
      <http://myopenlink.net/dataspace/demo#this>
      <http://rdfs.org/sioc/ns#id>
      <demo> .};
    
  2. Load from .rq file:
    1. Create DAV collection which is visible to public, for ex: http://localhost:8890/DAV/tmp
    2. Upload to the DAV collection the following file for ex. with name listall.rq and with the following content:
      SPARQL
      PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
      PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
      PREFIX sioc: <http://rdfs.org/sioc/ns#>
      SELECT ?x ?p ?o
      FROM <http://mygraph.com>
      WHERE
        {
          ?x rdf:type sioc:User .
          ?x ?p ?o.
          ?x sioc:id ?id .
          FILTER REGEX(str(?id), "^King")
        }
      ORDER BY ?x
      
    3. Execute from ISQL the following command:
      SQL>SPARQL
      load bif:concat ("http://", bif:registry_get("URIQADefaultHost"), "/DAV/tmp/listall.rq") into graph <http://myNewGraph.com>;
      

      As result should be shown:

      callret-0
      VARCHAR
      _______________________________________________________________________________
      
      Load <http://localhost:8890/DAV/tmp/listall.rq> into graph <http://myNewGraph.com> -- done
      
      1 Rows. -- 321 msec.
      
  3. Load from Resource URL:
    SQL> SPARQL LOAD <http://www.w3.org/People/Berners-Lee/card#i>;
    callret-0
    VARCHAR
    _______________________________________________________________________________
    
    Load <http://www.w3.org/People/Berners-Lee/card#i> into graph <http://www.w3.org/People/Berners-Lee/card#i> -- done
    
    1 Rows. -- 703 msec.
    SQL>
    
    

16.9.5. SPARQL Insert via /sparql endpoint

SPARQL INSERT operation can be sent to a web service endpoint as a single statement and executed in sequence.

Example:

Using the Virtuoso ISQL tool or using the /sparql UI at http://host:port/sparql, execute the following:

  1. Insert into graph http://BookStore.com 3 triples:
    SQL>SPARQL insert in graph <http://BookStore.com>
    { <http://www.dajobe.org/foaf.rdf#i> <http://purl.org/dc/elements/1.1/date> <1999-04-01T00:00:00> .
      <http://www.w3.org/People/Berners-Lee/card#i> <http://purl.org/dc/elements/1.1/date> <1998-05-03T00:00:00> .
      <http://www.w3.org/People/Connolly/#me> <http://purl.org/dc/elements/1.1/date> <2001-02-08T00:00:00> };
    
  2. As result will be shown the message:
    SQL>Insert into <http://BookStore.com>
    3 triples -- done
    
  3. Next we will select all triples from the graph http://BookStore.com:
    SQL>SPARQL SELECT * FROM <http://BookStore.com> WHERE {?s ?p ?o};
    
  4. As result will be shown:
    s                                              p                                       o
    VARCHAR                                        VARCHAR                                 VARCHAR
    _______________________________________________________________________________
    
    http://www.w3.org/People/Berners-Lee/card#i    http://purl.org/dc/elements/1.1/date    1998-05-03T00:00:00
    http://www.w3.org/People/Connolly/#me          http://purl.org/dc/elements/1.1/date    2001-02-08T00:00:00
    http://www.dajobe.org/foaf.rdf#i               http://purl.org/dc/elements/1.1/date    1999-04-01T00:00:00
    
    3 Rows. -- 0 msec.
    
    
  5. Now let's insert into graph another http://NewBookStore.com graph's values:
    SQL>SPARQL
    PREFIX dc:  <http://purl.org/dc/elements/1.1/>
    PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
    INSERT INTO GRAPH <http://NewBookStore.com> { ?book ?p ?v }
    WHERE
      { GRAPH  <http://BookStore.com>
       { ?book dc:date ?date
         FILTER ( xsd:dateTime(?date) < xsd:dateTime("2000-01-01T00:00:00")).
         ?book ?p ?v.
       }
      };
    
  6. As result will be shown:
    callret-0
    VARCHAR
    _______________________________________________________________________________
    
    Insert into <http://NewBookStore.com>, 2 triples -- done
    
  7. Finally we will check the triples from the graph NewBookStore.com:
    SQL> SPARQL
    SELECT *
    FROM <http://NewBookStore.com>
    WHERE {?s ?p ?o};
    
  8. As result will be shown:
    s                                             p                                      o
    VARCHAR                                       VARCHAR                                VARCHAR
    _______________________________________________________________________________
    
    http://www.w3.org/People/Berners-Lee/card#i   http://purl.org/dc/elements/1.1/date   1998-05-03T00:00:00
    http://www.dajobe.org/foaf.rdf#i              http://purl.org/dc/elements/1.1/date   1999-04-01T00:00:00
    
    2 Rows. -- 10 msec.
    

16.9.6. SPARQL Insert via SPARQL endpoint REST API and ODS wiki

With HTTP Post and ODS wiki can be written an rdf document and respectively to be performed over it INSERT/UPDATE action.

You can write to a file using SIOC terms for ODS-Wiki

You can check with sparql the inserted / updated triples in the Quad Store.

Example:

Suppose there is ODS user test3 with ODS password 1, which has testWiki wiki instance.

Execute the following:

curl -i -d "INSERT {<http://localhost:8890/dataspace/test3/wiki/testWiki> <http://atomowl.org/ontologies/atomrdf#contains> <http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest> . <http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest> <http://rdfs.org/sioc/ns#has_container> <http://localhost:8890/dataspace/test3/wiki/testWiki> . <http://localhost:8890/dataspace/test3/wiki/testWiki> <http://atomowl.org/ontologies/atomrdf#entry> <http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest> . <http://localhost:8890/dataspace/test3/wiki/testWiki> <http://rdfs.org/sioc/ns#container_of> <http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest> . <http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest> <http://rdfs.org/sioc/ns#topic>  <http://localhost:8890/dataspace/test3/wiki/testWiki> . <http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest> <http://atomowl.org/ontologies/atomrdf#source> <http://localhost:8890/dataspace/test3/wiki/testWiki> . <http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://rdfs.org/sioc/types#Comment> . <http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://atomowl.org/ontologies/atomrdf#Entry> . <http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest> <http://www.w3.org/2000/01/rdf-schema#label> 'MyTest' . <http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://atomowl.org/ontologies/atomrdf#Link> . <http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest> <http://rdfs.org/sioc/ns#content> <test>}" -u "test3:1" -H "Content-Type: application/sparql-query" http://localhost:8890/DAV/home/test3/wiki/testWiki/MyTest

As result we should have 2 files created:

Now let's check what data was inserted in the Quad Store:

  1. Go to the sparql endpoint, i.e. for ex. to http://localhost:8890/sparql
  2. Enter for Default Graph URI:
    http://localhost:8890/DAV/home/test3/wiki/testWiki/MyTest
    
  3. Enter for Query text:
    SELECT * WHERE {?s ?p ?o}
    
  4. Click the "Run Query" button.
  5. As result will be shown the inserted triples:
    s  	                                                    p  	                                             o
    http://localhost:8890/dataspace/test3/wiki/testWiki 	    http://rdfs.org/sioc/ns#container_of 	     http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest
    http://localhost:8890/dataspace/test3/wiki/testWiki 	    http://atomowl.org/ontologies/atomrdf#entry      http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest
    http://localhost:8890/dataspace/test3/wiki/testWiki 	    http://atomowl.org/ontologies/atomrdf#contains   http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest
    http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest  http://www.w3.org/1999/02/22-rdf-syntax-ns#type  http://rdfs.org/sioc/types#Comment
    http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest  http://www.w3.org/1999/02/22-rdf-syntax-ns#type  http://atomowl.org/ontologies/atomrdf#Entry
    http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest  http://www.w3.org/1999/02/22-rdf-syntax-ns#type  http://atomowl.org/ontologies/atomrdf#Link
    http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest  http://www.w3.org/2000/01/rdf-schema#label 	     MyTest
    http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest  http://rdfs.org/sioc/ns#has_container 	     http://localhost:8890/dataspace/test3/wiki/testWiki
    http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest  http://rdfs.org/sioc/ns#content 	             test
    http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest  http://rdfs.org/sioc/ns#topic 	             http://localhost:8890/dataspace/test3/wiki/testWiki
    http://localhost:8890/dataspace/test3/wiki/testWiki/MyTest  http://atomowl.org/ontologies/atomrdf#source     http://localhost:8890/dataspace/test3/wiki/testWiki
    

16.9.7. Using WebDAV

Example using WebDAV (mount folder to DAV and dump; if this is the rdf_sink the Quad Store is updated automatically, or you can load from DAV manually to quad store)

Example:

Example 1: Using ODS Briefcase

  1. Go to your ods location, for ex. http://localhost:8890/ods
  2. Register user, for ex. user test1
  3. Login if not already in ods
  4. Go to ODS -> Briefcase
  5. Go to ODS -> Briefcase
  6. Click the "New folder" icon from the Main Briefcase horizontal navigation
  7. Enter for name for ex. "mytest" and click the "Create" button.
    Using Briefcase UI

    Figure: 16.9.7.1.1. Using Briefcase UI
  8. Go to folder "mytest" and click the click the "Upload" icon from the Main Briefcase horizontal navigation
  9. Enter for name for ex. "mytest" and click the "Create" button.
    Using Briefcase UI

    Figure: 16.9.7.1.2. Using Briefcase UI
  10. In the shown form set:
    • Destination: RDF Store
    • RDF graph name for ex. with the value: http://localhost:8890/DAV/home/test2/mytest/
    • Select URL or File. For ex. you can select the following file with name jose.rdf:
      <rdf:RDF xmlns="http://www.example/jose/foaf.rdf#"
          xmlns:foaf="http://xmlns.com/foaf/0.1/"
          xmlns:log="http://www.w3.org/2000/10/swap/log#"
          xmlns:myfoaf="http://www.example/jose/foaf.rdf#"
          xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
      
          <foaf:Person rdf:about="http://www.example/jose/foaf.rdf#jose">
              <foaf:homepage rdf:resource="http://www.example/jose/"/>
              <foaf:knows rdf:resource="http://www.example/jose/foaf.rdf#juan"/>
              <foaf:name>Jose Jimen~ez</foaf:name>
              <foaf:nick>Jo</foaf:nick>
              <foaf:workplaceHomepage rdf:resource="http://www.corp.example/"/>
          </foaf:Person>
      
          <foaf:Person rdf:about="http://www.example/jose/foaf.rdf#juan">
              <foaf:mbox rdf:resource="mailto:juan@mail.example"/>
          </foaf:Person>
      
          <foaf:Person rdf:about="http://www.example/jose/foaf.rdf#julia">
              <foaf:mbox rdf:resource="mailto:julia@mail.example"/>
          </foaf:Person>
      
          <rdf:Description rdf:about="http://www.example/jose/foaf.rdf#kendall">
      
              <foaf:knows rdf:resource="http://www.example/jose/foaf.rdf#edd"/>
          </rdf:Description>
      </rdf:RDF>
      
  11. You can also perform the steps from above by uploading the file in the rdf_sink folder i.e. in Briefcase it will be with this path: DAV/home/test2/rdf_sink and respectively the "RDF graph name" will have this value: http://host:port/DAV/home/username/rdf_sink/

Execute from ISQL or from the SPARQL endpoint the following query:

SELECT * FROM <http://localhost:8890/DAV/home/test2/mytest/>
WHERE {?s ?p ?o}

As result should be shown:

s  	                                  p  	                                            o
http://www.example/jose/foaf.rdf#jose 	  http://www.w3.org/1999/02/22-rdf-syntax-ns#type   http://xmlns.com/foaf/0.1/Person
http://www.example/jose/foaf.rdf#jose 	  http://xmlns.com/foaf/0.1/nick 	            Jo
http://www.example/jose/foaf.rdf#jose 	  http://xmlns.com/foaf/0.1/name 	            Jose Jimen~ez
http://www.example/jose/foaf.rdf#jose 	  http://xmlns.com/foaf/0.1/knows 	            http://www.example/jose/foaf.rdf#juan
http://www.example/jose/foaf.rdf#jose 	  http://xmlns.com/foaf/0.1/homepage 	            http://www.example/jose/
http://www.example/jose/foaf.rdf#jose 	  http://xmlns.com/foaf/0.1/workplaceHomepage 	    http://www.corp.example/
http://www.example/jose/foaf.rdf#kendall  http://xmlns.com/foaf/0.1/knows 	            http://www.example/jose/foaf.rdf#edd
http://www.example/jose/foaf.rdf#julia 	  http://www.w3.org/1999/02/22-rdf-syntax-ns#type   http://xmlns.com/foaf/0.1/Person
http://www.example/jose/foaf.rdf#julia 	  http://xmlns.com/foaf/0.1/mbox 	            mailto:julia@mail.example
http://www.example/jose/foaf.rdf#juan 	  http://www.w3.org/1999/02/22-rdf-syntax-ns#type   http://xmlns.com/foaf/0.1/Person
http://www.example/jose/foaf.rdf#juan 	  http://xmlns.com/foaf/0.1/mbox 	            mailto:juan@mail.example

Example 2: Using Conductor UI

  1. Go to Conductor UI, for ex. at http://localhost:8890/conductor
  2. Login as dba user
  3. Go to Linked Data -> Quad Store Upload
    Quad Store Upload

    Figure: 16.9.7.2.1. Quad Store Upload
  4. In the shown form click the "Browse" button in order to select a file, for ex. the file jose.rdf and set the "RDF IRI*"
    Quad Store Upload

    Figure: 16.9.7.2.2. Quad Store Upload
  5. Click the "Upload" button.
    Quad Store Upload

    Figure: 16.9.7.2.3. Quad Store Upload

16.9.8. Using Virtuoso Crawler

Using Virtuoso Crawler (which includes the Sponger options so you crawl non-RDF but get RDF and this can go to the Quad Store).

Example:

  1. Go to Conductor UI. For ex. at http://localhost:8890/conductor :
    Using Virtuoso Crawler

    Figure: 16.9.8.1.1. Using Virtuoso Crawler
  2. Enter admin user credentials:
    Using Virtuoso Crawler

    Figure: 16.9.8.1.2. Using Virtuoso Crawler
  3. Go to tab Web Application Server:
    Using Virtuoso Crawler

    Figure: 16.9.8.1.3. Using Virtuoso Crawler
  4. Go to tab Content Imports:
    Using Virtuoso Crawler

    Figure: 16.9.8.1.4. Using Virtuoso Crawler
  5. Click the "New Target" button:
    Using Virtuoso Crawler

    Figure: 16.9.8.1.5. Using Virtuoso Crawler
  6. In the shown form set respectively:
    1. "Crawl Job Name": Tim Berners-Lee's electronic Business Card ;
    2. "Data Source Address (URL)": http://www.w3.org/People/Berners-Lee/ ;
    3. "Local WebDAV Identifier" for ex.: /DAV/home/demo/my-crawling/ ;
    4. Choose from the list "Local resources owner": demo ;
    5. Leave checked by default the check-box "Store documents locally". -- Note: if "Store document locally" is not checked, then in this case no documents will be save as DAV resource and the specified DAV folder from above will not be used ;
    6. Check the check-box with label "Store metadata" ;
    7. Specify which cartridges to be involved by hatching their check-box ;
    8. Note: when selected "Convert Link", then all HREFs in the local stored content will be relative.
    Using Virtuoso Crawler

    Figure: 16.9.8.1.6. Using Virtuoso Crawler
    Using Virtuoso Crawler

    Figure: 16.9.8.1.6. Using Virtuoso Crawler
    Using Virtuoso Crawler

    Figure: 16.9.8.1.6. Using Virtuoso Crawler
  7. Click the button "Create":
    Using Virtuoso Crawler

    Figure: 16.9.8.1.9. Using Virtuoso Crawler
  8. Click the button "Import Queues":
    Using Virtuoso Crawler

    Figure: 16.9.8.1.10. Using Virtuoso Crawler
  9. For "Robot target" with label "Tim Berners-Lee's electronic Business Card" click "Run".
  10. As result should be shown the number of the pages retrieved.
    Using Virtuoso Crawler

    Figure: 16.9.8.1.11. Using Virtuoso Crawler
  11. To view the crawled data, go for instance to public SPARQL Endpoint ex. http:/host:port/sparql:
    Using Virtuoso Crawler

    Figure: 16.9.8.1.12. Using Virtuoso Crawler
  12. Set respectively a Default Graph URI: <http://www.w3.org/People/Berners-Lee/> and execute the following query:
    SELECT *
    FROM <http://www.w3.org/People/Berners-Lee/>
    WHERE
      {
        ?s ?p ?o
      }
    LIMIT 10
    
    Using Virtuoso Crawler

    Figure: 16.9.8.1.13. Using Virtuoso Crawler
    Using Virtuoso Crawler

    Figure: 16.9.8.1.13. Using Virtuoso Crawler

Example: Use of schedular to interface Virtuoso Quad Store with PTSW using the following program:



create procedure PTSW_CRAWL ()
{
  declare xt, xp any;
  declare content, headers any;

  content := http_get ('http://pingthesemanticweb.com/export/', headers);
  xt := xtree_doc (content);
  xp := xpath_eval ('//rdfdocument/@url', xt, 0);
  foreach (any x in xp) do
    {
      x := cast (x as varchar);
      dbg_obj_print (x);
      {
	declare exit handler for sqlstate '*' {
	  log_message (sprintf ('PTSW crawler can not load : %s', x));
	};
        sparql load ?:x into graph ?:x;
	update DB.DBA.SYS_HTTP_SPONGE set HS_LOCAL_IRI = x, HS_EXPIRATION = null WHERE HS_LOCAL_IRI = 'destMD5=' || md5 (x) || '&graphMD5=' || md5 (x);
	commit work;
      }
    }
}
;

insert soft SYS_SCHEDULED_EVENT (SE_SQL, SE_START, SE_INTERVAL, SE_NAME)
	values ('DB.DBA.PTSW_CRAWL ()', cast (stringtime ('0:0') as DATETIME), 60, 'PTSW Crawling');
See Also:

Other Methods to Set Up the Content Crawler for RDF gathering.


16.9.9. Using SPARQL Query and Sponger (i.e. we Fetch the Network Resources in the FROM Clause or values for the graph-uri parameter in SPARQL protocol URLs)

Example:

Execute the following query:

SQL>SPARQL
SELECT ?id
FROM NAMED <http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/sioc.ttl>
 OPTION (get:soft "soft", get:method "GET")
WHERE { GRAPH ?g { ?id a ?o } }
limit 10;

As result will be shown the retrieved triples:

id
VARCHAR
_______________________________________________________________________________

http://www.openlinksw.com/dataspace/kidehen@openlinksw.com#this
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D
http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/612
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/612
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/610
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/610
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/856
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/856

10 Rows. -- 20 msec.

16.9.10. Using Virtuoso PL APIs

16.9.10.1. Basic Sponger Cartridge Example

In the example script we implement a basic mapper which maps a text/plain mime type to an imaginary ontology, which extends the class Document from FOAF with properties 'txt:UniqueWords' and 'txt:Chars', where the prefix 'txt:' we specify as 'urn:txt:v0.0:'.

use DB;

create procedure DB.DBA.RDF_LOAD_TXT_META
	(
	 in graph_iri varchar,
	 in new_origin_uri varchar,
	 in dest varchar,
         inout ret_body any,
	 inout aq any,
	 inout ps any,
	 inout ser_key any
	 )
{
  declare words, chars int;
  declare vtb, arr, subj, ses, str any;
  declare ses any;
  -- if any error we just say nothing can be done
  declare exit handler for sqlstate '*'
    {
      return 0;
    };
  subj := coalesce (dest, new_origin_uri);
  vtb := vt_batch ();
  chars := length (ret_body);
  -- using the text index procedures we get a list of words
  vt_batch_feed (vtb, ret_body, 1);
  arr := vt_batch_strings_array (vtb);
  -- the list has 'word' and positions array , so we must divide by 2
  words := length (arr) / 2;
  ses := string_output ();
  -- we compose a N3 literal
  http (sprintf ('<%s> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Document> .\n', subj), ses);
  http (sprintf ('<%s> <urn:txt:v0.0:UniqueWords> "%d" .\n', subj, words), ses);
  http (sprintf ('<%s> <urn:txt:v0.0:Chars> "%d" .\n', subj, chars), ses);
  str := string_output_string (ses);
  -- we push the N3 text into the local store
  DB.DBA.TTLP (str, new_origin_uri, subj);
  return 1;
}
;

--
DELETE FROM DB.DBA.SYS_RDF_MAPPERS WHERE RM_HOOK = 'DB.DBA.RDF_LOAD_TXT_META';

INSERT SOFT DB.DBA.SYS_RDF_MAPPERS (RM_PATTERN, RM_TYPE, RM_HOOK, RM_KEY, RM_DESCRIPTION)
    VALUES ('(text/plain)', 'MIME', 'DB.DBA.RDF_LOAD_TXT_META', null, 'Text Files (demo)');

-- here we set order to some large number so don't break existing mappers
update DB.DBA.SYS_RDF_MAPPERS set RM_ID = 2000 WHERE RM_HOOK = 'DB.DBA.RDF_LOAD_TXT_META';
  1. Paste the whole of this code into Conductor's iSQL interface and execute it to define and register the cartridge.
  2. Create a simple text document with a .txt extension. For ex. with name: summary.txt
  3. The .txt file must now be made Web accessible. A simple way to do this is to expose it as a WebDAV resource using Virtuoso's built-in WebDAV support:
    1. Log in to Virtuoso's ODS Briefcase application;
    2. Navigate to your Public folder;
    3. Upload your text document, ensuring that the file extension is .txt, the MIME type is set to text/plain and the file permissions are rw-r--r--.
    4. As result the file would be Web accessible via the URL http://cname/DAV/home/username/Public/summary.txt .
    5. Note: you can also check our live demo.
  4. To test the mapper we just use /sparql endpoint with option 'Retrieve remote RDF data for all missing source graphs' to execute (for ex.):
    SELECT *
    FROM <http://cname/DAV/home/username/Public/summary.txt>
    WHERE {?s ?p ?o}
    
  5. Click the "Run Query" button.
  6. As result should be shown the found triples, for ex.:
    s  	                                               p  	                                            o
    http://cname/DAV/home/username/Public/summary.txt  http://www.w3.org/1999/02/22-rdf-syntax-ns#type  http://xmlns.com/foaf/0.1/Document
    http://cname/DAV/home/username/Public/summary.txt  urn:txt:v0.0:UniqueWords 	                      47
    http://cname/DAV/home/username/Public/summary.txt  urn:txt:v0.0:Chars 	                            625
    

Important: Setting Sponger Permissions

In order to allow the Sponger to update the local RDF quad store with triples constituting the fetched Network Resource structured data, the role "SPARQL_SPONGE" must be granted to the account "SPARQL", i.e., to the owner account of /sparql web service endpoint. This should normally be the case. If not, you must manually grant this permission. As with most Virtuoso DBA tasks, the Conductor provides the simplest means of doing this.

See Also:


16.9.11. Using SIMILE RDF Bank API

Virtuoso implements the HTTP-based Semantic Bank API that enables client applications to post to its RDF Triple Store. This method offers an alternative to using Virtuoso/PL functions or WebDAV uploads as the triples-insertion mechanism.

Example:

From your machine go to Firefox->Tools->PiggyBank->My Semantic Bank Accounts

Add in the shown form:

Go to http://demo.openlinksw.com/ods

Log in as user demo, password: demo

Go to the Weblog tab from the main ODS Navigation

Click on weblog instance name, for ex. "demo's Weblog".

When the weblog home page is loaded, click Alt + P.

As result is shown the "My PiggyBank" page with all the collected information presented in items.

For several of the items add Tags from the form "Tag" shown for each of them.

As result should be shown the message "Last updated: [here goes the date value].

You can also click "Save" and "Publish" for these items.

Go to http://demo.openlinksw.com/sparql

Enter for the "Default Graph URI" field: http://simile.org/piggybank/demo

Enter for the "Query text" text-area:

prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix sioc: <http://rdfs.org/sioc/ns#>
SELECT *
FROM <http://simile.org/piggybank/demo>
WHERE {?s ?p  ?o}

Click "Run Query".

As results are shown the found results.


16.9.12. Using RDF NET

Example:

Execute the following query:

SQL> SELECT DB.DBA.HTTP_RDF_NET ('sparql load
"http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com"
into graph <http://www.openlinksw.com/>');

As result should be shown:

callret
VARCHAR
_______________________________________________________

<?xml version="1.0" ?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:vcard="http://www.w3.org/2001/vcard-rdf/3.0#"
xmlns="http://example.org/book/" xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:ns="http://example.org/ns#">
<rdf:Description>
<callret-0>Load <http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com>
into graph <http://www.openlinksw.com/> -- done</callret-0>
</rdf:Description>
</rdf:RDF>

1 Rows. -- 1982 msec.

16.9.13. Using the RDF Proxy (Sponger) Service

Triples can be inserted also using the Sponger Proxy URI Service. For more information and examples see here.