Rasqal RDF Query Library - Release Notes

Rasqal 0.9.16 Changes

WARNING: ABI AND API CHANGED IN THIS RELEASE.

This means: functions were removed, modified and added.

Consequently the shared library major soname version has changed from 0 to 1.

The header files moved to be installed in PREFIX/include/rasqal such as /usr/include/rasqal for usual installs.

Major Changes (break ABI and API)

Removed rasqal_init() and rasqal_finish() functions.

Added a new rasqal_world object to manage library allocations and classes with new library constructor/initializing function rasqal_world *rasqal_new_world(void) and destructor/cleanup function: void rasqal_free_world(rasqal_world*)

The following functions now take a librdf_world* world argument as the first argument: rasqal_language_name_check(), rasqal_languages_enumerate(), rasqal_new_query(), rasqal_query_results_formats_check (), rasqal_new_query_results_formatter(), rasqal_new_query_results_formatter_by_mime_type(), rasqal_new_integer_literal(), rasqal_new_typed_literal(), rasqal_new_double_literal(), rasqal_new_float_literal(), rasqal_new_uri_literal(), rasqal_new_pattern_literal(), rasqal_new_string_literal(), rasqal_new_simple_literal(), rasqal_new_boolean_literal(), rasqal_new_variable_literal(), rasqal_new_decimal_literal(), rasqal_new_decimal_literal_from_decimal()

Removed deprecated rasqal_new_floating_literal(), rasqal_graph_pattern_get_flags(), rasqal_expression_foreach() functions, RASQAL_LITERAL_FLOATING macro and rasqal_expression_foreach_fn typedef.

Modified rasqal_triples_match struct to add a rasqal_world* world field.

There is no more static data such as error pointers, arrays of query languages, URI objects; it is all attached to the rasqal_world object.

Misc changes

Fixed memory leaks in lang() expression evaluation. (Lauri Aalto)

Made several low-memory resiliance fixes in query construction and execution. (Lauri Aalto)

Library requires raptor 1.4.17 or newer to get the newly public raptor SAX2 API for reading SPARQL XML Results format.

Updated mpfr configure check to look for modern functions rather than functions that are also in the older gmp-mpfr library too.

Only the first error is reported for syntax errors in lex/parse of SPARQL and RDQL since the lack of error recovery means that the consequent messages are of little use.

Fatal errors in SPARQL and RDQL lexer/parser no longer does an abort, but exits via setjmp/longjmp.

Query Engine Changes

Added internal rowsource class to get rows from an execution engine or a syntax (query result formatter).

Graph Pattern Class Changes

Modified rasqal_graph_pattern_add_sub_graph_pattern() to have an int return value for catching allocation failures.

Query Class Changes

Modified rasqal_query_add_variable() and rasqal_query_add_prefix() to have int return values for catching allocation failures.

Query Results Class Changes

Updated the internals to allow construction of a query result outside the query engine, such as from a syntax.

Added rasqal_query_results_read() to read a query results from an iostream in some format.

Modified rasqal_query_results_formats_enumerate() to add a flags argument with RASQAL_QUERY_RESULTS_FORMAT_FLAG_READER and RASQAL_QUERY_RESULTS_FORMAT_FLAG_WRITER choices to choose formats for read or write enumeration.

Removed function rasqal_query_results_formats_enumerate_full now handled by rasqal_query_results_formats_enumerate().

Added rasqal_query_results_formatter_read() to read query results from a syntax via an iostream.

Variable Class Changes

Added rasqal_new_variable_from_variable() copy constructor.

Rasqal 0.9.15 Changes

General Changes

Many updates for better SPARQL support. Rasqal 0.9.15 passes over 92% of the DAWG SPARQL Test Suite as of 2007-11-13.

Rasqal still does not support either UNION or GROUP and OPTIONAL remains buggy. There remain a few other minor areas not yet completely supported such as XSD date and XSD dateTime equality and comparisons.

WARNING: ABI AND API CHANGE in Rasqal 0.9.16. The next release of Rasqal will be incompatible with this one.

It will have both ABI and API changes that will require source updates.

The shared library major soname will change from 0 to 1.

The header files will move to be installed in PREFIX/include/rasqal such as /usr/include/rasqal for usual installs.

The full details of the API change have not been decided but the driving change is that using the library will be done via a new Rasqal world object,k will now be used to working like a Redland librdf_world. All constructors will take this world object as an argument and all static data will be indirected from the object. Handlers such as the uri_handler/data and error handlers will also move to be set/get via the world object.

There are almost certainly other API changes and cleanups that will be done, and some possibly some of the data structures may be made internal.

Fixed issues:

SPARQL Query Language Changes

Many updates to support the SPARQL Candidate Recommendation and Proposed Recommendation versions of the language including:

Query Engine Changes

Many internal changes were made including refactoring the engine core and adding of additional internal documentation for the engine functions.

Substantial improvements were made to datatypes support:

Improved the way that literal equality and comparisons work especially with respect to datatypes. Triple pattern matching (equality) works with RDF terms. SPARQL FILTER literal equality and comparisons follow SPARQL rules with possible numeric type promotion, then using RDF Term if the types are different. ORDER comparison or DISTINCT equality checks use RDF Term. RDQL execution uses it's own existing "stringy" rules.

Set the query base URI.

Make queries that bind the origin (SPARQL GRAPH) match correctly against the default graph.

Memory reduction in stack usage when evaluation expressions (such as SPARQL FILTERs).

Public API changes

Added rasqal_home_url_string static constant.

Added rasqal_xsd_decimal class and API calls described below.

Added a decimal part to the rasqal_literal union value for RASQAL_LITERAL_DECIMAL types.

Added a parent_type part to the rasqal_literal when the type can be promoted.

Added RASQAL_EXPR_SAMETERM expression

Added RASQAL_COMPARE_RDF and RASQAL_COMPARE_URI comparison types for RDF Term comparsion rules and allowing URI comparison (without STR()) respectively.

Added rasqal_new_typed_literal() to make an RDF typed literal from a type and string (no language).

Added rasqal_new_float_literal() for wrapping a C float as XSD float.

Added new decimal literal constructor rasqal_literal* rasqal_new_decimal_literal_from_decimal(const unsigned char *string, rasqal_ xsd_decimal* decimal)

Added rasqal_literal_datatype method to get the datatype URI of a literal.

Added rasqal_literal_value method to get the literal value of a literal, looking up any variables to their value. FIXME should this not involve query results?

Added addition rasqal_triple_parts enums: RASQAL_TRIPLE_GRAPH for just the named graph part RASQAL_TRIPLE_SPO for subject, predicate and objects and RASQAL_TRIPLE_SPOG for all parts including named grpah.

Decimal Class

A new rasqal_xsd_decimal class was added to handle XSD decimals with accurate decimal arithmetic.

Constructor:

rasqal_xsd_decimal* rasqal_new_xsd_decimal(void);

Destructor:

void rasqal_free_xsd_decimal(rasqal_xsd_decimal* dec);

Methods:

double rasqal_xsd_decimal_get_double(rasqal_xsd_decimal* dec);
int rasqal_xsd_decimal_set_double(rasqal_xsd_decimal* dec, double d);
iint rasqal_xsd_decimal_set_string(rasqal_xsd_decimal* dec, const char* string);
int rasqal_xsd_decimal_set_long(rasqal_xsd_decimal* dec, long l);
int rasqal_xsd_decimal_print(rasqal_xsd_decimal* dec, FILE* stream);
char* rasqal_xsd_decimal_as_string(rasqal_xsd_decimal* dec);
char* rasqal_xsd_decimal_as_counted_string(rasqal_xsd_decimal* dec, size_t* len_p);

Numeric methods (writing to the result argument):

int rasqal_xsd_decimal_add(rasqal_xsd_decimal* result,
                           rasqal_xsd_decimal* a, rasqal_xsd_decimal* b);
int rasqal_xsd_decimal_subtract(rasqal_xsd_decimal* result,
                                rasqal_xsd_decimal* a, rasqal_xsd_decimal* b);
int rasqal_xsd_decimal_multiply(rasqal_xsd_decimal* result,
                                rasqal_xsd_decimal* a, rasqal_xsd_decimal* b);
int rasqal_xsd_decimal_divide(rasqal_xsd_decimal* result,
                              rasqal_xsd_decimal* a, rasqal_xsd_decimal* b);
int rasqal_xsd_decimal_negate(rasqal_xsd_decimal* result,
                              rasqal_xsd_decimal* a);

Comparison methods:

int rasqal_xsd_decimal_compare(rasqal_xsd_decimal* a, rasqal_xsd_decimal* b);
int rasqal_xsd_decimal_equals(rasqal_xsd_decimal* a, rasqal_xsd_decimal* b);
int rasqal_xsd_decimal_is_zero(rasqal_xsd_decimal* d);

Query Results Formatter Class Changes

Updated to the 4th version of the SPARQL XML Results format by removing the ordered and distict attributes. Removed support for all the earlier versions 1 to 3.

roqet Utility Changes

Added -D / --data options for specifying data URis for the default graph and -G / --named for URIs of named graphs (the existing -s is a also a synonym for the latter).

Other Changes

Many internal fixes and cleanups after failures so that memory is not lost and/or rasqal does not crash. These many changes were primarily made by Lauri Aalto.

Several portability fixes for helping with low-memory, embedded systems and using a compiler that is not gcc.

The SPARQL test runner check-sparql was improved to read the SPARQL XML Results format (but not via XML::Simple because it is insanely complex and counterintuitive), handle DAWG test manifests and generate EARL reports. It also has new help and usage messages.

Rasqal 0.9.14 Changes

General Changes

Added syntax and API support (but no query engine support) for experimental extensions to SPARQL called LAQRS: LAQRS Adds to Querying RDF in SPARQL.

Updated the query results formatter to allow better introspection of supported formats.

Fixed issues:

LAQRS Adds to Querying RDF in SPARQL (LAQRS)

LAQRS is experimental and Rasqal provides only syntax and API support (but no query engine support) for:

Repeating THERE IS NO QUERY ENGINE SUPPORT FOR ANY OF THE ABOVE

LAQRS is not enabled by default in Rasqal (unless using maintainer mode or building from Subversion). To enable it, configure Rasqal like this:

$ ./configure '--enable-query-languages=sparql rdql laqrs'
...

Rasqal build summary:
  RDF query languages available : rdql sparql
  RDF query languages enabled   : sparql rdql laqrs
  Triples source                : raptor 1.4.15
$

The API changes for LAQRS are as follows:

Added new query verbs RASQAL_QUERY_VERB_DELETE and RASQAL_QUERY_VERB_INSERT

Added a constructor to create a constant or 0-operand expression (for COUNT):

rasqal_expression* rasqal_new_0op_expression(rasqal_op op);

Added a query method to get the explain flag:

int rasqal_query_get_explain(rasqal_query* query);

Added a query method to get the 'group by' condition expressions:

raptor_sequence*
rasqal_query_get_group_conditions_sequence(rasqal_query* query);

rasqal_expression*
rasqal_query_get_group_condition(rasqal_query* query, int idx);

Added rasqal_expression expression types:

rasqal_variable now has a rasqal_expression* field expression.

Configuration and Build Changes

The pkgconfig rasqal.pc file and the rasqal-config script now separates directly linked shared libraries and indirectly linked ones. This prevents unnecessary references to libraries. The regular libraries includes -L plus -lrasqal and the dynamic and static libraries are available with:

So for example to statically link with rasqal, use either:

$ cc .. -o foo foo.c `pkgconfig rasqal --static --libs`

$ cc .. -o foo foo.c `rasqal-config --libs --private-libs`

Query Results Formatter Changes

Made it easier to discover and create of query results in a syntax by using a MIME Type.

Added query results formatter constructor by required mime type:

rasqal_query_results_formatter*
rasqal_new_query_results_formatter_by_mime_type(const char *mime_type);

Added a method to return the mime type of a selected formatter:

const char*
rasqal_query_results_formatter_get_mime_type(
  rasqal_query_results_formatter *formatter);

Added another query results formatter static method to enumerate possible results formats, their names, labels and (new) mime type.

int
rasqal_query_results_formats_enumerate_full(const unsigned int counter,
  const char **name, const char **label,
  const unsigned char **uri_string, const char **mime_type);

Added a static method to check for support for a query results format.

int rasqal_query_results_formats_check(const char *name,
  raptor_uri* uri, const char *mime_type);

SPARQL Query Language Changes

Updated for the SPARQL Query Language for RDF, W3C Working Draft 26 March 2007

Query Engine Changes

Added the possibility that a query's native result is a syntax (rather than natively bindings, boolean or triples) and added a method to check for that:

int rasqal_query_results_is_syntax(rasqal_query_results* query_results);

Many query engine internal changes, but these remain unsupported: OPTIONAL graph pattern (partially working), GROUP graph pattern (no support), UNION graph pattern (no support) and new SELECT REDUCED (no support).

roqet Utility Changes

roqet can use -r to generate a chosen result syntax for bindings results or triples (using raptor).

roqet now generates namespace declarations for triples when serializing output. The result is that they are used when the serializer handles them such as generating RDF/XML (Abbreviated) or Turtle.

Other Changes

rasqal_expression lost the unused rasqal_variable* field variable

Added exported variables to return the license string in rasqal_license_string and home page in rasqal_home_url_string

Rasqal 0.9.13 Changes

General Changes

Added a new rasqal_query_results_formatter class.

Many internal query engine changes, mostly to separate query and query_results internals so that eventually the run-time execution context is entirely separated from the static query structure.

Added query features system with one initial feature RASQAL_FEATURE_NO_NET.

Configuration and Build Changes

Allow shell variable LEX to be set to things that aren't exactly flex Recommend flex 2.5.33 and update configure to point at the main flex site whichq finally gets 2.5.33 after 9 years.

PKGCONFIG vars not substituted correctly in configure.ac. Remove @LDFLAGS from rasqal.pc.in. Fixes Issue#0000098.

Win32 build fixes.Correct win32 static linking of rasqal. (Fixes Issue#0000100) and include float.h and #define isnan on win32. (Fixes Issue#0000111).

In maintainer mode, add all the supported compiler -W warning flags to the CFLAGS.

Declare several rasqal functions with GCC printf-formatting attributes when using a new enough GCC.

Remove broken debug messages. #0000131.

SPARQL Query Language Changes

Added labels to grammar tokens using bison to give better error messages.

Fix looking for EOF in an unsigned variable since it may be negative on some systems.

The query engine now implements SPARQL GRAPH matching.

RDQL Query Language Changes

Added labels to grammar tokens using bison to give better error messages.

Fix looking for EOF in an unsigned variable since it may be negative on some systems.

Query Engine Changes

The query engine now implements SPARQL GRAPH matching.

Merge only all adjacent sequences of basic graph pattern triples, not anything with triples such as optional BGP triples. Fixes Issue#0000080.

When an exact triple match is present, do not execute a query forever. Fixes Issue#0000094.

Fail if a variable argument to LANG() has an undefined value rather than try to execute a NULL pointer. Fixes Issue#0000113.

Add query / query_results link early so reference counting is done right. Fixes Issue#0000095.

Report failure as well as query finished when an error occurs during making results. Fixes Issue#0000118.

Added a hack to check an outer constraint expression. Fixes Issue#0000129.

Free origins (GRAPH nodes) in triples to prevent memory leak. Fixes Issue#0000133.

Query API Changes

Added a new rasqal_query_results_formatter class to format librdf_query_results into a syntax. Also added a rasqal_query_results_formats_enumerate() function to enumerate all the supported formats. The query results method rasqal_query_results_write() was changed to use this class internally.

Added a new feature system to allow setting parameters on the query parser/engine. Initially have one feature RASQAL_FEATURE_NO_NET that prevents network requests during a query execution. Added functions: rasqal_features_enumerate, rasqal_get_feature_count, rasqal_feature_from_uri, rasqal_feature_value_type, rasqal_query_set_feature, rasqal_query_set_feature_string, rasqal_query_get_feature, rasqal_query_get_feature_string.

roqet gained a -f / --feature option to set query features.

Rasqal 0.9.12 Changes

General Changes

Added JSON as a query results format based on the Serializing SPARQL Query Results in JSON draft.

Added query writing to a raptor_iostream which allows reading in one query language and writing in another. The only useful consequence of this now is that it enables roqet to become an RDQL to SPARQL translator.

Rasqal now requires raptor 1.4.9 or newer.

Configuration and Build Changes

Now using subversion.

roqet utility Changes

Altered the -d (--dump-query) option to now take an argument of the query format to dump. The values it can take are debug, structure or sparql.

To convert a query from RDQL to SPARQL, use:

roqet -q -i rdql -d sparql -n query.rdql > query.sparql

Added -r json for JSON query results output based on the draft defined in the Serializing SPARQL Query Results in JSON draft

SPARQL Query Language Changes

Updated to match SPARQL Query Language for RDF, W3C Candidate Recommendation 6 April 2006. All DAWG syntax tests now pass. The update includes near alignment with SPARQL grammar term names except where the flex/bison approach diverged.

Updated SPARQL tests from DAWG CVS

Finally removed long deprecated ASC[] and DESC[] syntax.

RDQL Query Language Changes

Handle empty regex // Fixes Issue#0000071

Query Engine Changes

Some updates to handle latest changes to SPARQL grammar.

Added RASQAL_EXPR_REGEX 3-arg operation to match the SPARQL regex() 2/3-arg function.

Many internal changes to move more of the query execution state from inside rasqal_query to inside rasqal_query_results and related structures. This is not yet complete but when done, you will be able to execute the same query more than once at the same time.

Query API Changes

Added rasqal_query_write to write a query structure into a syntax onto a raptor_iostream. Implements writing SPARQL format with format URI NULL, http://www.w3.org/TR/rdf-sparql-query/ or http://www.w3.org/TR/2006/CR-rdf-sparql-query-20060406/

Added a JSON SPARQL query results format with format URI identifier http://www.w3.org/2001/sw/DataAccess/json-sparql/ based on the draft defined in http://www.w3.org/2001/sw/DataAccess/json-sparql/ Implemented document with version alt-ser-sparql.rst,v 1.3 2006/02/16 16:03:08 k Exp.

rasqal_query_get_all_variable_sequence was corrected to match the definition - return all variables, not just the bound (SELECTed) ones. Fixes Issue#00000079.

Added rasqal_query_get_anonymous_variable_sequence to return only anonymous variables after change above.

Added rasqal_graph_pattern_get_index to get the absolute index of a graph pattern in a graph.

IOstream class Changes

Added rasqal_query_iostream_write_escaped_counted_string and rasqal_query_escape_counted_string to format an escaped string suitable for using inside a query.

Documentation Changes

Updated roqet.1 for --exec option and new usages.

Rasqal 0.9.11 Changes

General Changes

Duplicated SELECT variables are now detected, give a warning and are removed.

Rasqal now requires raptor 1.4.8 or newer to get the guess parser that uses URI/name and mime type returned from protocol request (if available) to choose the correct parser.

Added an internal rasqal_map key : value map for use with result duplicate detection and ordering.

A major source code regoranisation moved all library source code to the src/ dir, roqet to utils/ and other documentation to docs/. This should only affect packagers, not any functionality.

Many more function documentation were added to the sources for use by gtk-doc.

SPARQL Query Language Changes

Updated to the SPARQL Query Language for RDF W3C Working Draft 23 November 2005:

Added a post 2005-11-23 SPARQL WD syntax change: added langMatches operator and initial query engine support.

Invalid \uHHHH and \UHHHHHHHH hex escapes and floating point number formats are detected and warned.

Updates to the query engine make DISTINCT and ORDER BY now work, including with LIMIT.

The lexer now counts line numbers correctly over multi-line comments.

Updated changes to the valid characters allowed in SPARQL names for variable, prefix and QNames.

RDQL Query Language Changes

The lexer now counts line numbers correctly over multi-line comments.

Query Engine Changes

When query result sorting is used with limit (SPARQL LIMIT) and offset, they now work correctly, performing the limit and offset after ordering the result. This matches the SPARQL query result semantics.

The query engine now implements distinct results (SPARQL DISTINCT) including with ordering (SPARQL ORDER BY).

More cases with optional graph patterns (SPARQL OPTIONALs) are now executed correctly. Note: this does not mean all cases work, there are known problems with executing optional clauses.

Added types for xsd:float (C float) and xsd:double (C double); xsd:decimal (handled as a string) and xsd:datetime (as a string).

Fixed (a compare b) between integers and doubles to be the right sign.

Do not promote literals early on in comparisons to strings "perl style" but do the best type-based comparsion as late as possible.

Query API Changes

Added rasqal_new_double_literal.

Added rasqal_graph_pattern_visit and type for visitor function rasqal_graph_pattern_visit_fn.

Added rasqal_new_expression_from_expression.

Added rasqal_expression_visit and type for visitor function rasqal_expression_visit_fn.

Deprecated function rasqal_new_floating_literal replaced by new rasqal_new_double_literal.

Deprecated function rasqal_expression_foreach, replaced by new rasqal_expression_visit.

Function rasqal_expression_evaluate gains a flag argument of new enum rasqal_compare_flags.

Added enum rasqal_compare_flags flags for rasqal_expression_evaluate or rasqal_literal_compare. Type RASQAL_COMPARE_XQUERY can be used with to ask for use of XQuery comparison and type promotions.

Added rasqal_literal types: RASQAL_LITERAL_DECIMAL, RASQAL_LITERAL_DATETIME, RASQAL_LITERAL_DOUBLE (replacing deprecated RASQAL_LITERAL_FLOATING) and RASQAL_LITERAL_FLOAT. Reordered the rasqal_literal types in the enum.

The enum types in rasqal_literal_type were reordered to make them match sparql type promotion.

roqet utility program changes

roqet gains -e/--exec option to execute a query string directly rather than via a file or URI containing the query.

Now writes out XML results format in format with version URI http://www.w3.org/2005/06/sparqlResults and short name "xml". The previous version is available with short name "xml-v2". The latest version now writes ordered and distinct boolean-valued attributes on the results element.

Configuration Changes

The autogen.sh script for building from CVS was revamped to be more modular.

configure now takes an --enable-gtk-doc option to enable building of the documentation using the gtk-doc utility. It is by default enabled only if the utility is available.

Documentation Changes

The GNOME gtk-doc program is now used to automatically extract documentation from source comments into reference documentation. This is then merged with templates and additional documentation to provide a reference manual for rasqal as XML document which is turned into HTML along with GNOME devhelp support.

This new documentation intended to replace the librasqal manual page/web page as easier to read document with scope for better expanding with more detail of rasqal including examples and tutorial information. The librasqal manual page will continue to contain the summary information for the present.

Rasqal 0.9.10 Changes

General Changes

The query engine now supports ordering query results, in SPARQL this is the ORDER BY clause.

SPARQL Query Language Changes

The SPARQL parser now passes (accepts) all the 70+ SyntaxFull tests in the RDF Data Access Working Group (DAWG) test suite for SPARQL. This mostly means accepting optional punctuation in some places.

Sorting results with ORDER BY now partially works however comparison is not done with the SPARQL literal comparison rules (after XQuery) but using the existing RDQL-style "string-y" comparisons.

Allow <> as a URI.

The SPARQL parser now accepts some changes made by the DAWG after the last published working draft: SPARQL Query Language for RDF, W3C Working Draft, 19 April 2005. This includes:

Omitting WHERE gives a warning. Don't do that!

Query Engine Changes

The query engine now allows sorting of results (ORDER BY in SPARQL). It performs comparisons using the existing RDQL style and does not yet support the SPARQL literal comparison rules. Sorting can be done by a sequence of expressions, ascending or descending. If the final result is equality, the object pointers are used.

Note: removal of duplicates is not supported yet.

Some test code for datatypes support was added, not enabled by default and not yet useful.

Query API Changes

Removed functions deprecated in 0.9.9.

Added rasqal_graph_pattern_operator enumeration with values for the main SPARQL graph patterns, graph pattern class method rasqal_graph_pattern_get_operator and graph pattern utility function rasqal_graph_pattern_operator_as_string

Modified the type of the final argument of rasqal_new_graph_pattern_from_sequence and rasqal_graph_pattern_add_triples from an integer to a rasqal_graph_pattern_operator enumeration.

Deprecated function rasqal_graph_pattern_get_flags now always returning 0 replaced by the graph pattern operator functions listed above .

roqet utility program changes

Removed deprecated flags -f and -o

Rasqal 0.9.9 Changes

General Changes

SPARQL is now the default query language

Added a signing memory debugging system to aid checking when rasqal-allocated memory is freed in another library or vice-versa enabled by --with-memory-signing configure option (defaults to on in maintainer mode).

Fixed a few internal malloc/frees to use RASQAL_MALLOC / RASQAL_FREE so that the above signed memory system worked.

SPARQL Query Language Changes

Grammar updated to follow the SPARQL Query Language for RDF, W3C Working Draft, 19 April 2005. This changes many parts of the syntax:

(Note: the query engine does not support sorting amongst other recently added SPARQL features)

SPARQL keywords are now case independent rather than just upper or lowercase.

Added """long literals""" after N3 (and soon, Turtle) and '''long literals''' for SPARQL only.

Added many SPARQL tests from DAWG most not yet approved by the working group. Not all pass either because not implemented or known bugs.

Added the revised SPARQL Query Results XML Format (the latest version not yet published) support to rasqal_query_results_write when called with format URI http://www.w3.org/2001/sw/DataAccess/rf1/result2.

Query Engine Changes

Uses the query results limit and offset information to skip initial results (offset) and provide a maximum number of results (limit).

Moved some engine-specific operations from RDQL and SPARQL parser.

Added internal rasqal_formula class for the Turtle part of the SPARQL parser

Deprecated the rasqal_triples_source factory method new_triples_match. It is replaced by a new method init_triples_match and will be removed from the next release.

Query API Changes

Added bnode identifier generation functions: rasqal_query_set_default_generate_bnodeid_parameters, rasqal_query_set_generate_bnodeid_handler based on the raptor API for generating blank node identifiers in the Turtle/N3 triple style.

Added order_conditions_sequence to the query structure to store query ordering conditions and added access methods rasqal_query_get_order_conditions_sequence and rasqal_query_get_order_condition to read the resulting expressions.

Added expressions for SPARQL ordering conditions: RASQAL_EXPR_ORDER_COND_ASC, RASQAL_EXPR_ORDER_COND_DESC and RASQAL_EXPR_ORDER_COND_NONE.

Added types for variables in two version - normal and anonymous with an new enum rasqal_variable_type and rasqal_new_variable_typed to allow creating them.

Completed the distinct/limit and offset get/set methods by adding: rasqal_query_set_distinct, rasqal_query_set_limit, rasqal_query_get_offset and rasqal_query_set_offset

Deprecated the rasqal_triple flags field - the value RASQAL_TRIPLE_FLAGS_EXACT was only used in internals of the execution engine. The value RASQAL_TRIPLE_FLAGS_OPTIONAL was never used. Optionals are now noted on graph patterns. Deprecated rasqal_triple_set_flags and rasqal_triple_get_flags.

Added enum rasqal_query_verb for the main query verbs: RASQAL_QUERY_VERB_SELECT RASQAL_QUERY_VERB_CONSTRUCT RASQAL_QUERY_VERB_DESCRIBE and RASQAL_QUERY_VERB_ASK. Added method rasqal_query_get_verb to access the verb.

Added rasqal_query_get_query_graph_pattern to get the top graph pattern.

Added rasqal_query_get_wildcard to get the query verb wildcard flag such as with SPARQL SELECT *

Added rasqal_query_get_construct_triples_sequence and rasqal_query_get_construct_triple to get the triples made from a SPARQL CONSTRUCT query.

roqet utility program changes

Updated roqet to include better query structure walking with -w / --walk including listing query verb, distinct, limit, offset flags, construct triples and easier to read groupings of graph patterns and triples.

Deprecated roqet arguments -f / --format and -o / --output in place of -r / --results as both choose the query results format.

SPARQL is now the default query language for roqet.

Rasqal 0.9.8 Changes

SPARQL Query Language Changes

Grammar updated to follow the SPARQL Query Language for RDF, W3C Working Draft, 17 February 2005.

Added support for casting in constraint expresions for datatyped literals with the syntax qname(literal) such as xsd:decimal("10").

Added named graphs parsing with LOAD and note that UNION cannot be handled yet, SOURCE and [...] syntax for optionals are deprecated.

Query Engine Changes

Engine now operates over a hierarchy of rasqal_graph_pattern objects, where at the root the query has a single query graph rather than a sequence of graph patterns. The query graph contains the former sequence of graph patterns.

Constraints can be applied to any graph pattern, not just at the top level of a query.

Added casting expressions type RASQAL_EXPR_CAST and constructor rasqal_new_cast_expression.

Fixed setting of a triple origin (SPARQL GRAPH) which was not working in 0.9.7

xsd:boolean RDF literals are now promoted into internal RASQAL_LITERAL_BOOLEAN typed rasqal_literal like for XSD integer and double.

The query engine uses the named graph information when evaluating queries, passing it to the rasqal_triples_match factory methods either implemented with raptor or redland as a rasqal_triples_source_factory and returning them as results of variable bindings.

Query API Changes

Added no-arg constructor rasqal_new_graph_pattern

Added method rasqal_graph_pattern_add_triples to add triples to an existing graph pattern.

Added methods rasqal_graph_pattern_add_constraint, rasqal_graph_pattern_get_constraint_sequence and rasqal_graph_pattern_get_constraint to add constraints to a graph pattern.

Added method rasqal_graph_pattern_add_sub_graph_pattern to add a sub-graph pattern to a graph pattern.

Added a Data Graph class with constructor rasqal_new_data_graph, destructor rasqal_free_data_graph and debug method rasqal_data_graph_print.

Added query methods for data graphs: rasqal_query_add_data_graph, rasqal_query_get_data_graph_sequence, rasqal_query_get_data_graph.

Deprecated query methods rasqal_query_add_constraint, rasqal_query_get_constraint_sequence rasqal_query_get_constraint, rasqal_query_add_source, rasqal_query_get_source_sequence and rasqal_query_get_source. These will be removed in the next release.

Removed functions deprecated in previous version: rasqal_query_get_variable_sequence and rasqal_query_add_triple.

Rasqal 0.9.7 Changes

General Changes

Fix a resource clean up when query results are either not read at all or not fully exhausted. This caused a problem in Redland when it held on to MySQL database handles despite a query having ended.

Added option -n/--dryrun to roqet to stop after preparing the query. Added option -w/--walk-query to walk and print the query using the graph API.

New build configuration and portability fixes for win32 (John Barstow)

Portability fixes for win32 - define RASQAL_INLINE and use it as for raptor. (Dave Viner)

Make rasqal tests work when building against raptor in a sibling source tree.

Query Engine Changes

Added a graph pattern API with rasqal_graph_pattern typedef and rasqal_pattern_flags enum with value RASQAL_PATTERN_FLAGS_OPTIONAL to allow complete walking of the query structure.

Two new methods to the query class were added to support this: rasqal_query_get_graph_pattern_sequence to get the raptor_sequence of graph patterns in a query and rasqal_query_get_graph_pattern to get individual an graph pattern by index.

The new graph pattern methods added are:

Deprecated rasqal_query_add_triple as it doesn't construct the graph patterns and is of little use as it does not build the correct query structures.

Added query API methods to get the bound and all variables in a query: rasqal_query_get_bound_variable_sequence and rasqal_query_get_all_variable_sequence respectively. Deprecated rasqal_query_get_variable_sequence as too ambiguous.

With the aid of gcc -Wswitch-enum, found and fixed cases where switch on enumerated values were missing cases. The remaining cases return failure or an error message rather than abort(). This fixed a crash when comparing for equality a variable ?x with a blank node value and an interger.

Deleted RASQAL_EXPR_PATTERN as it was never used.

Added rasqal_query_get_distinct to get a query-specified distinct results flag.

Added rasqal_query_get_limit to get a query-specified limit on the number of results.

SPARQL Query Language Changes

Updated some of the SPARQL syntax to match the changes in the SPARQL Query Language for RDF W3C Working Draft of 17 February 2005. Added GRAPH replacing SOURCE and WITH replacing FROM. Added warnings about using the older forms and about using [...] instead of OPTIONAL.

Added recording of DISTINCT and LIMIT in the query structures but not yet used by the query engine.

RDQL Query Language Changes

None.

Rasqal 0.9.6 Changes

General Changes

Fix a bug in writing a query result to XML when blank nodes are used. Now also correctly uses the base URI when pass in to the writing method.

Added support for three types of query result - bindings (the only one in previous versions), an RDF graph and a boolean. Added rasqal_query_results methods to test the type of the result: rasqal_query_results_is_bindings, rasqal_query_results_is_boolean and rasqal_query_results_is_graph.

Added rasqal_query_results_get_triple to get current triple as a raptor_statement and rasqal_query_results_next_triple to move to the next triple.

Added rasqal_query_results_get_boolean to get a boolean query result value.

Added rasqal_triple copy constructor rasqal_new_triple_from_triple.

Query Engine Changes

Constructs an RDF graph result as a sequence of triples. This enables the SPARQL CONSTRUCT form to work.

Can return a boolean result indicating that there were variable binding results, but not returning them. This enables the SPARQL ASK form to work.

Added a new function expression - rasqal_op enum value RASQAL_EXPR_FUNCTION, added new name and args fields to the rasqal_expression structure and added rasqal_new_function_expression to construct this expression.

Added new 1-argument expression RASQAL_EXPR_STR returning a string version of the expression.

Added new 1-argument expressions operating on RDF literals: RASQAL_EXPR_LANG returning the language string of an RDF literal, RASQAL_EXPR_DATATYPE returning the datatype URI of an RDF literal.

Added new 1-argument testing expressions returning booleans: RASQAL_EXPR_BOUND true if the variable argument is bound to a value, RASQAL_EXPR_ISURI true if the argument is a URI, RASQAL_EXPR_ISBLANK true if the argument is a blank node and RASQAL_EXPR_ISLITERAL true if the argument is an RDF literal.

Fixed a problem in optional results returning extra results with no new bindings.

SPARQL Query Language Changes

The SPARQL Parser now allows UTF-8 and \u, \U in prefixes and QNames.

Now supports CONSTRUCT and ASK queries using the updated query engine features.

OPTIONAL can now be used wherever [ ... ] could be used (parser bug fix).

Now supports the SPARQL built-in functions STR() returning a string version of the expression, LANG() returning the language string of an RDF literal, DATATYPE() returning the datatype URI of an RDF literal, BOUND() testing if a variable is bound, ISURI() testing if an expression is a URI, ISBLANK() testing if an expression is a blank node and ISLITERAL() testing if an expression is an RDF literal.

Now supports SPARQL BASE directive for setting the base URI of a query. This is experimental and may not be in the next version or may not be in the final SPARQL language.

The parser now expands QNames correctly when used in literals like "blah"^^xsd:string.

RDQL Query Language Changes

The RDQL Parser now allows UTF-8 and \u, \U in prefixes and QNames. This makes RDQL testsuite tests S-03, S-04 and S-05 work.

Rasqal 0.9.5 Changes

A release with major changes internal to the query engine to operate over graph patterns instead of sequences of triple patterns and to add execution of optional graph patterns. Additionally the query results can now be written in a recognised XML format.

This version of Rasqal requires Raptor 1.4.4.

Core Changes

Added a rasqal_graph_pattern class for handling matching a graph pattern (set of triple patterns) in an rasqal_query. RDQL only has one of these wheras SPARQL may have many, and they may be nested (this may change, as SPARQL is under development).

Added rasqal_query_results_write method to write variable binding query results to a raptor_iostream in the XML format defined by SPARQL Variable Binding Results XML Format, W3C Working Draft, 21 December 2004.

Query Engine Changes

Updated the query engine to operate over graph patterns rather than a single sequence of triple patterns. The graph patterns may be optional, which is used to implement SPARQL optionals. Nested graph patterns are constructed but cannot yet be executed.

Added a unary minus expression type.

Improved the type promotion for expression evaluation. Try to evaluate strings as doubles or integers if the operation contains one double or integer respectively.

Fixed a memory leak with string matches/non-matches.

SPARQL Query Language Changes

Optional graph patterns OPTIONAL and [ ... ] now work.

Added the { ... } group syntax and removed the ( ... ) form.

$ was added as an allowed variable prefixes.

A warning was added to show SOURCE * is ignored.

Added the UNION keyword, unsupported.

Rasqal now passes the DAWG SPARQL test cases: Optional triples (dawg-opt-query-001, dawg-opt-query-002, dawg-opt-query-003, dawg-opt-query-004) and examples from query WD (sparql-query-example-2.1a, sparql-query-example-2.2a, sparql-query-example-2.3a, sparql-query-example-2.4a, sparql-query-example-3)

Allow UTF-8 qname prefixes and check name against XML name constraints.

Recover lexer state from errors in regexes.

Terminate the lexer on a syntax error, rather than produce multiple errors.

RDQL Query Language Changes

The lexer now handles UTF-8 qname prefixes, enabling RDQL testsuite test S-02 to pass.

Allow UTF-8 qname prefixes and check name against XML name constraints.

Recover lexer state from errors in regexes.

Terminate the lexer on a syntax error, rather than produce multiple errors.

Rasqal 0.9.4 Changes

Pass on raptor parser errors to rasqal's error routines and upwards.

Clean up several memory allocations when a query fails to prepare or execute fully.

Fixed lexer jams when a query contained an illegal variable name after a '?'.

Fixed lexer crashes when a query ended in a regex pattern like /abc/.

Tidied up the testing framework so 'make distcheck' works.

Rasqal 0.9.3 Changes

Rasqal's License was changed from LGPL 2.1/MPL 1.1 to LGPL 2.1/Apache 2

Added support for the SPARQL Query Language for RDF, W3C Working Draft, 12 October 2004 from the W3C RDF Data Access Working Group (DAWG). The support is at the basic syntax level - token lexer and grammar parser which passes the first simple tests from the proposed DAWG testcases. No additional query engine support has been added for SPARQL features that are not supported by RDQL. The Rasqal to do list contains full details of the implementation state.

Queries can now be prepared and then executed multiple times.

Added configure --enable-query-languages option with rdql and sparql as the choices.

Thanks to Chris Pointon for several patches to make Rasqal easier to build under Win32. Applied with some modifications. Renamed the RDQL and SPARQL syntax tokens ERROR to ERROR_TOKEN to help.

Added experimental support for querying multiple sources; the same query is executed over each source in turn. Also added some support for returning the source URI which is not yet enabled as this feature is in flux in the DAWG work. This may change or be removed in future versions.

roqet can specify a data source URI on the command line with -s / --source) URI for queries executing against an implicit model.

Added the DAWG SPARQL test cases driven from the manifest.n3 format. Created tests from the examples in the working draft.

Added an experimental XML result format for bindings, based on DAWG work. Enabled in roqet with -o xml. This is very likely to change or replaced in future versions.

Added roqet argument -d / --dump-query to dump the processed query, rather than do it by default (0.9.2 and earlier).

roqet can now read queries from URIs, using the raptor_www class to do the retrieval.

Fixed the configure checks for a posix regex function regcomp to correctly discover it is present.

Internal changes

Added support for declaring prefixes for XML-style QNames both before and after their use.

The struct rasqal_prefix gained a declared field.

The struct rasqal_triple gained an origin field; not used at present but intended to support work on tracking triple provenance such as provided by Redland Contexts.

Added methods rasqal_triple_set_origin and rasqal_triple_get_origin to support the above.

struct rasqal_triple_meta now takes a 4-array of bindings, the fourth being the origin.

Exported function rasqal_set_triples_source_factory publically as originally intended.

Rasqal 0.9.2 Changes

This release made minor portability fixes for building with C++ and Win32 (untested).

Several functions changed their parameters or return values from char* to unsigned char* or const unsigned char* to reflect the actual use.

Changed to return a const unsigned char*:
rasqal_literal_as_string

Changed to take const unsigned char* (or add const):
rasqal_new_floating_literal
rasqal_new_pattern_literal
rasqal_new_prefix
rasqal_new_simple_literal
rasqal_new_string_literal
rasqal_new_variable
rasqal_query_has_variable
rasqal_query_results_get_binding_name
rasqal_query_results_get_binding_value_by_name
rasqal_query_results_get_bindings
rasqal_query_set_variable

Rasqal 0.9.1 Changes

This release changed the way that results were returned. A new rasqal_query_results class was created. This allows for a clean separation of the two concepts allowing multiple form of the results to be retrieved as methods of the query_results class, rather than the query class. The only form of results currently available is variable bindings, but RDF graphs will be returned in future, as a set of triples in a serialised form.

The execution method rasqal_query_execute was changed to return a new object of that class rather than an int success or failure.

The following methods of the rasqal_query class were renamed and moved to the new rasqal_query_results class:

0.9.0 name 0.9.1+ name
rasqal_query_get_result_count rasqal_query_results_get_count
rasqal_query_next_result rasqal_query_results_next
rasqal_query_results_finished rasqal_query_results_finished
rasqal_query_get_result_bindings rasqal_query_results_get_bindings
rasqal_query_get_result_binding_value rasqal_query_results_get_binding_value
rasqal_query_get_result_binding_name rasqal_query_results_get_binding_name
rasqal_query_get_result_binding_by_name rasqal_query_results_get_binding_value_by_name
rasqal_query_get_bindings_count rasqal_query_results_get_bindings_count

A new rasqal_free_query_results destructor was created to delete the query results. (There is no public constructor).

Other minor bugs were fixed such as multiple constraints now being ANDed and not ORed, and adding defensive code for when queries fail and results methods are invoked.

Rasqal 0.9.0 Changes

It is all new.

Copyright (C) 2004-2008 Dave Beckett
Copyright (C) 2003-2005 University of Bristol