Virtuoso supports most SQL 92 data types.

  • CHARACTER

  • VARCHAR

  • VARCHAR '(' INTNUM ')'

  • VARCHAR

  • NVARCHAR '(' INTNUM ')'

  • CHARACTER '(' INTNUM ')'

The CHAR, CHARACTER and VARCHAR datatypes are implemented as a single string type with dynamic length. The precision that may be specified controls how the column is described by SQLColumns() , SQLDescribeCol() and so on. If a precision is not specified for a VARCHAR then the default precision will be 0, which means do not check. If a precision is not specified for a CHARACTER then Virtuoso sets the precision to 1. An explicit precision of 0 can be specified to turn off length checking for values stored in the column. If a value other than string or NULL is assigned to the column it is cast to a varchar (using CAST internally) and then stored into the column. If the value is not castable to a varchar then Virtuoso returns an error. Additionally if the column precision is greater than 0 and the value string length is greater than the column precision Virtuoso will also return an error.

The length is stored separately. Space required is 2+length for

A varchar column may contain binary 0 bytes.

A string literal is delimited by single quotes.

  • ANY

The ANY datatype is implemented as a single binary string type with dynamic length. It is reported as a VARCHAR in SQLColumns() , SQLDescribeCol() and so on. The precision returned by these columns is 24 but has no effect. This type can contain arbitrary binary data, including zeros.

The length is stored separately. The space required is 2+length

  • NUMERIC

  • NUMERIC '(' INTNUM ')'

  • NUMERIC '(' INTNUM ',' INTNUM ')'

  • DECIMAL

  • DECIMAL '(' INTNUM ')'

  • DECIMAL '(' INTNUM ',' INTNUM ')'

The various forms of NUMERIC and DECIMAL refer to one variable-precision floating point decimal data type that provides accurate arithmetic and decimal rounding. The default maximum precision and scale are 40 and 20. The precision is the number of decimal digits used in the computation. The scale is the maximum number of decimal digits to the right of the decimal point. Internal calculations are always precise but numbers are truncated to the column's precision and scale when stored. If a value being stored has more digits to the left of the decimal point than allowed in the column, Virtuoso signals an error. If a number being stored has more digits to the right of the decimal point than allowed in a column the decimal part is rounded to the precision of the column.

The space consumption of a number is

3 + precision / 2

bytes. The precision and scale of a column of this type are returned by functions such as SQLColumns() and SQLDescribeCol() .

A DECIMAL or NUMERIC with precision <= 9 and scale = 0 is transformed to INTEGER.

Literal numbers outside of the 32 bit signed integer range are of type decimal. Any numeric literals with a decimal point are of type decimal. Literals with an exponent are of type double precision.

  • INT

  • INTEGER

  • SMALLINT

These types are represented as a 32-bit signed binary integer, described as having a precision of 9 and a scale of 0, although the range is +/- 2**31. Storage space is 2 bytes for SMALLINT and 4 bytes otherwise.

A column declared SMALLINT is described as SQL_SMALLINT. A column declared INTEGER or INT is described as SQL_INTEGER.

Literals composed of of an optional sign and digits are of the integer type if they fit in the 32 bit range.

  • FLOAT

  • FLOAT '(' INTNUM ')'

  • DOUBLE PRECISION

These types refer to the 64-bit IEEE floating-point number, the C double type. This is a fixed-precision binary floating point number is described as having a precision of 15 and a scale of 0. This type is preferable to NUMERIC if decimal rounding is not required since it is precise enough for most uses and more efficient than NUMERIC. The storage requirement is 8 bytes.

Any number literal with an exponent has the double type, e.g. 2e9.

  • REAL

This type is the 32-bit IEEE floating point number corresponding to the C float type. The storage requirement is 5 bytes.

  • LONG VARCHAR

  • LONG VARBINARY

These types implement a binary large object (BLOB) type. The length can be up to 2**31 bytes (2GB). If manipulated with the SQLGetData() and SQLPutData() ODBC functions a BLOB need not fit in the DBMS's or the client's memory. The LONG VARCHAR and LONG VARBINARY types are distinct only because certain ODBC applications gain from being able to distinguish long text from long binary. The types are described as SQL_LONGVARCHAR and SQL_LONGVARBINARY respectively, with a precision of 2GB.

Several long columns may exist on a single row. A long column may not be a key part in an index or primary key.

Data in long columns is stored as a linked list of database pages. Thus, a long column that does not fit in-line on the containing row will require an integer number of 8K database pages. If a long column's value is short enough to fit within the row containing it, the BLOB will be stored on the row and will not take more space than a VARCHAR of the same length. A long column fits on a row if the sum of the lengths of columns, including the long column, is under 4070 bytes.

ORDER BY, GROUP BY and DISTINCT may not reference long data types. Comparison of long data is not allowed unless first converted to the corresponding short type (varchar, nvarchar or varbinary). This conversion is only possible if the value is under 10MB in size. String functions accept long varchars and long nvarchars and convert them to varchar and nvarchar automatically. There is no long literal type per se, the corresponding character or binary type is assignable to a long type.

  • VARBINARY

This type is internally like VARCHAR but is distinct for compatibility with ODBC applications. A VARBINARY column is described as SQL_BINARY to ODBC clients. The storage requirement is the same as for a corresponding VARCHAR column. VARBINARY and VARCHAR data are never equal even if the content is the same, but they can be cast to each other. VARBINARY data sorts in the unsigned order of the bytes comprising the data.

A varbinary literal is introduced by 0x followed by a hexadecimal representation of the bytes, 2 characters per byte, e.g. 0x0123456789abcdef.

  • TIMESTAMP

  • DATETIME

  • TIME

  • DATE

All the time- and date-related types are internally represented as a single 'datetime' type consisting of a Julian day, hour, minute, second, 6-digit fraction and timezone. The range of the year is from 0 to over 9999. This type can accommodate all values of any SQL92 time-related type.

Although the internal representation is the same, a column of a time-related type is described as being of the appropriate ODBC type, i.e. SQL_TIMESTAMP for TIMESTAMP and DATETIME and SQL_DATE for DATE and SQL_TIME for TIME.

A DATETIME is described as precision 19, a DATE as precision 10 and a TIME as precision 8.

A column declared a TIMESTAMP is automatically set to the timestamp of the transaction that inserts or updates any column of the table containing it. The timestamp of a transaction is guaranteed to be distinct from that of any other transaction. For compatibility reasons a TIMESTAMP column is described to ODBC clients as a binary of 10 bytes. It is possible to use any date-related functions on TIMESTAMPs and to bind a TIMESTAMP column to a DATE or DATETIME variable (SQL_C_TIMESTAMP type in ODBC). Binding to a binary will also work but the data will then be opaque.

SQL92 provides for types with a timezone. Although the ODBC API does not expose the timezone, it is stored with these types and can be retrieved with the timezone() function. The timezone has a precision of minutes from UTC.

The storage requirement for these types is 10 bytes.

There is no date literal per se, but the ODBC shorthand for datetime literals can be used. The datetime/timestamp literal is of the form {dt 'YYYY-MM-DD HH:MM.SS'}. The date literal is of the form {d 'YYYY-MM-DD'}. Dates and datetimes may be compared between themselves but not with other types without explicit casting.

Some traditional relational databases keep all values of DATETIME type as combination of time and timezone data. Other keep time in some specific timezone without paying any attention to timezone at all. In RDF, the incoming triples may contain literals of types like xsd:dateTime with arbitrary values matching ISO 8601, and this standard permits the use of time values with optional timezone. Thus there should be a way of handling both "timezoned" and "timezoneless" datetimes inside one database. Virtuoso server supports this starting from version 07.20.3214.

Important note: The use of timezoneless datetimes may result in subtle errors in data processing. Applications that worked fine with timezoned datetimes may work incorrectly if timezoneless datetimes are used. The related application errors may stay unnoticed during local testing and reveal after worldwide use. To stay on safe side, the use of timezoneless datetimes with pre-07.20.3214 databases remains blocked even after the server executable is upgraded, so old applications will continue to work as before. When developing new applications, please pay attention to the check-list at the end of this section.

Different applications may require different behavior when input data contain timezoneless values. In some cases it is better to "cast" all of them to timezoned than to upgrade existing code. Virtuoso offers 5 different modes of support. The mode is selected by TimezonelessDatetimes parameter in [Parameters] section of virtuoso.ini . This should be set before creating the database and the set value is stored in the database. After database is created, an attempt to change the mode by patching virtuoso.ini will have no effect and virtuoso.log will contain a warning about mismatch between virtuoso.ini and the database file.

The possible variants are:

  • Never use timezoneless, as it was in old databases. Always set local timezone on parsing strings if no timezone specified. An attempt to set timezoneless by calling function forget_timezone() will signal error. Timezoneless values still may come from outside as dezerializations of timezoneless DATETIME values, serialized by other database instances, but not in any other way:

    TimezonelessDatetimes=0
    
  • When parsing strings, set timezoneless if ISO format tells so:

    TimezonelessDatetimes=1
    
  • Set timezoneless always, exception is when the parsed string contains explicit timezone or when RFC requires the use of GMT or when timezone is set by function adjust_timezone() . This is default for new databases if TimezonelessDatetimes parameter is missing in virtuoso.ini

    TimezonelessDatetimes=2
    
  • Never use timezoneless. Always set local timezone on parsing strings if not timezone specified. An attempt to set timezoneless by calling function forget_timezone() will signal error. Timezoneless values still may come from outside as deserializations of timezoneless DATETIME values, serialized by other database instances, but not in any other way. The difference with TimezonelessDatetimes=0 is that timezones are always printed on cast datetimes to strings etc. so timezoneless-aware clients will get unambiguous data.

    TimezonelessDatetimes=3
    
  • On parsing string, set timezone to GMT if no timezone specified. However, timezoneless can be set by calling function forget_timezone() . This mode can be convenient for global web services when real "local" timezones of specific users are not known.

    TimezonelessDatetimes=4
    

For new applications, consider the use of TimezonelessDatetimes=2 as primary variant, TimezonelessDatetimes=1 as the second best.

Traditional SQL strings are of format "YYYY-MM-DD hh:mm:ss " with optional decimal fraction at the end and optional timezone data. Depending on software, the timezone can be specified as "timezone offset", i.e. the difference with GMT in minutes or as "timezone label", i.e. an identifier of timezone in special system dictionary that contains not only an offset in minutes but also information about daylight saving changes of the offset. Virtuoso does not support timezone labels, only numerical timezone offsets. Depending on system, the notation without the timezone data at the end means timezoneless value or, more probably, the value in some "default" timezone, such as local timezone of the server or GMT.

ISO 8601 introduced format "YYYY-MM-DDThh:mm:ss ", with "T " character between "date " and "time " parts. It also prescribed an unambiguous difference between timezoneless and timezoned values: absent timezone means timezoneless value.

The timezone offset is written as "+hh:mm " or "-hh:mm ", the "+00:00 " is usually shortened to "Z ". Oracle Java may use 1 to 4 digits without delimiting ":", in that case 1 or 2 digits mean whole hours whereas 3 or 4 digits mean an 1 or 2 digits of hour and two digits of minutes. For historical reasons, "-00:00 " notation differs from "+00:00 " and mean timezoneless, not GMT datetime.

ISO 8601 explicitly warns that comparison of timezoned and timezoneless datetime is not always possible. Valid timezones vary from -14:00 to +14:00, the fact that the span can exceed 24 hours may be not obvious. Nevertheless, storing rows in a database table require some unambiguous order; any order is OK as soon as it does not break the rules and common sense, but it should be well-defined. Virtuoso's order for mix of timezoned and timezoneless datetimes is very simple.

  1. All timezoned datetimes are sorted in natural chronological order, like if they are converted to GMT first. The value of timezone offset does not matter.

  2. All timezoneless datetimes are sorted in natural chronological order, like they are in GMT already.

  3. For each GMT calendar day, all timezoned datetimes are placed before all timezoneless datetimes.

  • is_timezoneless() -- The function returns 1 for timezoneless arguments, zero for timezoned.

    integer is_timezoneless (in dt datetime)
    
  • timezone() -- The function returns timezone offset of its first argument, as an integer value in minutes. If the first argument is timezoneless and second argument is missing or zero then the returned value is NULL. If the first argument is timezoneless and second argument is nonzero then the returned value is 0.

    integer timezone (in dt datetime [, in ignore_tzl integer])
    
  • adjust_timezone() -- The function returns its first argument with unchanged GMT value but new timezone offset, as it is specified by the second argument. If the first argument is timezoneless and third argument is missing or zero then error 22023 is signaled. If the first argument is timezoneless and third argument is nonzero then no error is signaled and the argument is handled like it is a GMT value.

    datetime adjust_timezone (in dt datetime, in tz_offset integer [, in ignore_tzl integer])
    
  • dt_set_tz() -- The function returns its first argument with unchanged GMT value but new timezone offset. Unlike adjust_timezone() , if the argument is timezoneless then no error is signaled.

    datetime dt_set_tz (in dt datetime, in tz_offset integer)
    
  • forget_timezone() -- The function returns its first argument as a timezoned value. If the first argument is timezoneless then it is returned unchanged. If the first argument is timezoned and second argument is missing or zero then the result is timezoneless value that "looks like" local time notation. If the first argument is timezoned and second argument is nonzero then the value is first made GMT and then it becomes timezoneless.

    datetime forget_timezone (in dt datetime [, in ignore_timezone integer])
    
  • now() -- returns the current transaction timestamp:

    datetime now ()
    
  • rdf_now_impl() -- returns the timestamp associated with current transaction as a DATETIME . Alias of now() :

    datetime        rdf_now_impl ()
    
  • getdate() -- returns the current transaction timestamp, alias of now() :

    datetime getdate ();
    
  • get_timestamp() -- returns the timestamp of the current transaction:

    datetime get_timestamp ()
    
  • current_timestamp() -- All these names refer to one function that returns the timestamp of current transaction. It is the datetime of the beginning of current transaction with the fractional part of seconds replaced with serial number of a transaction within the second. If TimezonelessDatetimes=0 then the time has local timezone offset (as it was set at the time of last server start); otherwise it is timezoneless.

    datetime current_timestamp ()
    
  • curdatetime() -- The function returns current datetime, like now() , but fractional part of seconds can be adjusted by providing the number of "microseconds" as the argument.

    datetime curdatetime ([in fraction_microseconds integer])
    
  • curdatetimeoffset() -- The function is like curdatetime() but the returned datetime is in GMT timezone.

    datetime curdatetimeoffset ([in fraction_microseconds integer])
    
  • curutcdatetime() -- Refers to function that is similar to curdatetime() but the returned datetime is in GMT timezone.

    datetime curutcdatetime ([in fraction_microseconds integer])
    
  • sysutcdatetime() -- Refers to function that is similar to curdatetime() but the returned datetime is in GMT timezone.

    datetime sysutcdatetime ([in fraction_microseconds integer])
    

Virtuoso allows 30-bit Unicode data to be stored and retrieved from database fields. The data are stored internally as UTF-8 encoded strings for storage space optimization. Unicode fields are easily intermixable with other character data as all SQL functions support wide-string case and convert to the most wide character representation on demand. The native width of the wide character type may differ between platforms. Windows has a 16 bit wide character, whereas some Unixes have a 32 bit wide character type. The native width applies to the Virtuoso NVARCHAR data type when used as SQL data.

There are 3 additional data types to enable storing of Unicode data:

  • NCHAR

  • NVARCHAR

  • LONG NVARCHAR

All the Unicode types are equivalent to their corresponding "narrow" type - CHAR, VARCHAR and LONG VARCHAR - except that instead of storing data as one byte they allow Unicode characters. Their lengths are defined and returned in characters instead of bytes. They collate according to the active wide character collation, if any. By default this is the order of the Unicode serialization values. These types can be used anywhere the narrow character types can be used, except in LIKE conditions.

Unicode literals are introduced by n' and closed with ' (single quote). See Internationalization section on the interpretation of wide literals. This may be either UTF-8 according to some character set.

When there is a need to convert a wide string to a narrow one or vice versa, a character set is used. A character set returns a wide string code for a wide char. For example there can be a definition of the ISO-8859-5 "narrow" character set which describes mapping of non-ASCII character codes to their Unicode equivalents. Virtuoso relies on the fact that the ASCII character codes are represented in Unicode by type-casting and in UTF8 as one-byte tokens with the same value as in ASCII.

When conversion is done on the server-side using cast or some of the SQL built-in functions, the wide characters are converted to narrow using a system-independent server-side character set. In the absence of such a character set, Virtuoso uses the Latin1 character set to project narrow character codes into the Unicode space as equally valued wide-character codes.

When conversion is done client-side - for example, when binding a VARCHAR to a wide buffer - the default client's system character set is used.

Wide-character literals have ANSI SQL92 syntax: N'xxx' (prefixing normal literals with the letter N). These strings process escapes with a values large enough to represent all the Unicode characters.

Virtuoso supports user-definable data types that can be based on any hosted language or classes such as C#. New types can be further derived producing sub-types. User-defined types can include methods and constructors to create any potentially complicated system to house data as exactly required.

User defined types can be used to defined database table columns.

[Tip] See Also:

The User Defined Types section.

All the built-in SQL functions that take character attributes and have a character input calculate their output type such that if any attribute is a wide string or a wide BLOB, then the result is a wide string; otherwise, the output character type is narrow.

Functions like make_string() that have character result types but that do not have character parameters produce narrow strings. Virtuoso provides equivalent functions for wide output, such as make_wstring() .

Virtuoso' ODBC client implements the SQL...W functions (like SQLConnectW() ) that take Unicode arguments. This enables faster wide-character processing and allows binding of the SQL_C_WCHAR output type. Since Virtuoso's SQL parser does not allow Unicode data in SQL commands, they should be bound as parameters or should be represented as escapes.

Attached tables use the default collation of the data source for narrow strings. Virtuoso maps Wide-string columns in remote tables to the appropriate local wide-character type. The data are then passed intact in case of wide-to-wide mapping. When data are converted client-side in the VDB the Server's system character set is used (where available).

The built-in data types denoting sequences of characters, wide or narrow, long or short, are:

Varchar : a string of 8-bit characters, including 0's, up to 16MB long. These are contiguously stored, so long contents, such as in the megabytes, will be inefficient.
NVARCHAR : A string of wide characters, of 2 or 4 bytes each, depending on the platform. Because of the 16MB limit, the longest strings will be of 4M or 8M characters, depending on the platform. Again long strings are not recommended due to inefficiencies.
Binary : A string of 8-bit bytes, up to 16 MB long, like a varchar but not usable for character functions. There is a distinct binary type only for compatibility with the SQL92 standard and ODBC, where the binary type is treated differently in parameter binding.
Long varchar, long nvarchar : These are long data types, stored persistently as a series of linked pages and accessible to clients in fragments using the SQLGetData() and SQLPutData() calls. The length limit is 2GB. The wide variant, LONG NVARCHAR, is internally stored as UTF8.
String_output : This is not a database column type but a run-time object that can be used in stored procedures for accumulating a long sequence of 8-bit bytes, including 0's. This type is not contiguously stored, hence it stays efficient for large output and has no built-in size limit; however, it is not automatically paged to disk, so it will consume virtual memory for all its length. This type is useful for buffering output for a next processing step.
Long varbinary : This is a binary BLOB, identical to long varchar but distinct for reasons of compatibility with SQL92 and ODBC, where this can behave differently from long varchar for parameter binding.
XML Entity : This type is a pointer to an element of an XML tree. The XML tree itself may be either memory- or disk-based. In both cases there is a reference-counted set of XML entities for each tree that Virtuoso uses to reference individual elements of the tree. These are used for navigating an XML tree in XPath or XSLT; hence, one entity gives access to it parents, siblings, and so on. This is not properly a string type, but it can be converted to one, producing the XML string value.

All these types have the common trait of representing sequences of characters and hence some common operations and conversions are possible between them.

The descriptions below apply to insert and update operations for these types:

  • Long varchar = x, where x is:

    varchar - The text is stored as is.
    Long varchar - the text is stored as is.
    string output - the contents in the string output are stored as the value, unchanged. The state of the string output is not changed.
    XML entity - The XML tree rooted at the entity is stored as persistent XML (disk-based) if the entity references a persistent XML tree. Note that this may either extract a subtree or copy a tree, depending on whether the entity references the root. If the entity references a memory-based tree, the text of the tree with the element as the topmost (document) element is produced and set as the value of the column.
    Nvarchar - The text is stored as wide, thus the value is internally a long nvarchar although the declared column type is long varchar.
    Long nvarchar - The value is stored as a long nvarchar, as with an nvarchar.
  • Long nvarchar = x

    The cases are identical to long varchar. Thus a wide value stays wide and a narrow value stays narrow. Specifically, a string output and XML entity result in a narrow value, although the character combination in the XML entity may be interpreted as wide.

  • Long varbinary = x

    Identical to long varchar. The binary type is only distinct in column metadata for ODBC clients, where its type conversions may be different.

  • Varchar = x, where x is:

    long varchar, string output, XML entity - as with long varchar.
    Nvarchar, Long nvarchar - the text is stored as wide; no information is lost.
  • Nvarchar = x, where x is:

    Long varchar, varchar - the string is converted to wide according to the character set effective in the connection.
    Long nvarchar, Nvarchar - The text is stored as is.

'String output' and 'XML entity' are not valid types for a column. These types are only created by evaluating SQL expressions and are converted as specified above if stored as a column value.

A BLOB column (long varchar, long nvarchar, long varbinary) may return either a long varchar or a long nvarchar BLOB handle. If the actual value is short enough to be inlined, a varchar or nvarchar value can be returned as the column value instead. These are indistinguishable for assignment and as arguments to SQL functions or for returning to a client application. Only specific SQL functions (isblob() , isstring() , etc.) allow you to determine the difference at run time. One exception is persistent XML entities, which come back as persistent XML entities and are not compatible with string functions but are assignable to various character columns.

An nvarchar column is always nvarchar.

A varchar value is either varchar or nvarchar. If the value stored was a memory-based XML tree entity it comes back as a long varchar. If it was a persistent XML tree, it comes back as an XML entity.

PL variables are typed at run time.

A string (varchar, nvarchar, or varbinary) can be freely assigned and passed as parameter. This makes a copy, except for reference (inout) parameters.

A BLOB (long varchar, long nvarchar, long varbinary) is a reference to a disk based structure, unless stored inline. Therefore, passing these as parameters does not take significant time. If these are inline, these are strings of under 4K bytes; hence assigning them is still efficient, although it involves copying.

A string output cannot be assigned between two variables, though it can be passed as a reference (inout) parameter in a PL procedure call. Copying streams has problematic semantics and can be very resource-consuming.

An XML entity can be assigned and passed as parameter without restrictions.

All SQL92 string functions will accept varchar, long varchar, nvarchar or long nvarchar arguments. If the argument is long and its actual length is above the maximum length of a varchar, the conversion fails and Virtuoso signals an error. You can interchange long and varchar types as long as the length remains under the varchar maximum of 16MB.

[Note] Note:

Varchars or nvarchars stored in columns have a much lower limit due to the 4K row length limit. Intermediate results or values converted from long columns are not affected by this limit.

If Virtuoso converts a value from long varchar to varchar or from long nvarchar to nvarchar when passing the value as an argument to a string function, the value changes in place. This has the effect of replacing the handle with the string. Users normally do not see this, but may detect it with type test functions such as isblob() .

You can declare string values that might be long and that do not have to be key parts in indices as long varchar. These will automatically be inlined if the row with the data inlined will fit within the 4K limit. Otherwise the long values will be stored as separate LOBs. The difference between varchar and long varchar is distinguishable only with special test functions if the length is under the varchar limit.

A varchar column is sometimes substantially faster on update than a long varchar column, even if the value ends up inlined. If the value is inlined there is no difference in retrieval speed.

LOBs of up to 2GB can be handled as streams without demand on memory from ODBC clients using SQLGetData() and SQLPutData() . All other ways of processing long data will need to make either a contiguous or non-contiguous copy in memory.

To transfer long data between PL procedures and files one can use the string_to_file() function, which will accept a handle and will not need to copy the content to memory in order to write it.

To read a large object from a file to a table, you can use the file_to_string_output() function to get contents that may be longer than the varchar limit into a string output. This can then be assigned to a BLOB column.

For long file-resident XML data you can use the xml_persistent() function with the file:// protocol.

[Tip] See Also:

The XML Support chapter.