Name

SERV_QUEUE_TOP — Retrieve target website and store within Virtuoso

Synopsis

WS.WS. SERV_QUEUE_TOP ( in target varchar ,
  in WebDAV_collection varchar ,
  in update integer ,
  in debug integer ,
  in function_hook varchar ,
  in data any );
 

Description

Web Robot site retrieval can be performed with the WS.WS.SERV_QUEUE_TOP PL function integrated in to the Virtuoso server.

To run multiple walking robots all you simply need to do is kick them off from separate ODBC/SQL connections and all robots will walk together without overlapping.

From a VSP interface, after calling the retrieval function you may call http_flush to keep running tasks in the server and allowing the user agent to continue with other tasks.

Parameters

target

URI to target site.

WebDAV_collection

Local WebDAV collection to copy the content to.

update

Flag to set updatable, can be 1 or 0 for on or off respectably.

debug

Debug flag, must be set to 0

function_hook.

Fully qualified PL function hook name. If not supplied or NULL then the default function will be used.

data

application dependent data, usually an array, is passed to the PL function hook to perform next queue entry extraction. In our example we use an array with names of non-desired sites.

Examples

Example 24.370. Retrieve External Sites

WS.WS.SERV_QUEUE_TOP (
  'www.foo.com', 'sites/www_foo_com', 0, 0, 'DB.DBA.my_hook',
    vector ('www.skip.me','www.bar.com')
);