The main changes to this release are:
Added two new JSON serializers: resource-centric 'json' (Talis RDF/JSON) and triple-centric 'json-triples'.
Made I/O Stream class raptor_iostream support
reading as well as writing with new constructors and
new methods.
Added a new public SAX2 API class raptor_sax2
exposing the existing internal API which has been around since the
first release of Raptor 8 years ago and runs on top of either expat
or libxml2.
Added new public error handlers structure
raptor_error_handlers containing a set of
(function, data pointers) pairs called raptor_handler_closure
for each error log level.
Added raptor_log_level enum for the error log level.
Added an initialization function for the structure,
raptor_error_handlers_init().
Several other API changes, fixes and improvements were made.
Fixed Issues:
Made I/O Stream class raptor_iostream support
reading data in addition to writing. (Dave B):
raptor_iostream_handler structure for new
raptor_iostream_handler2 structure which contains the
new factory functions for reading.raptor_iostream_read_bytes_func and
raptor_iostream_read_eof_func.
raptor_new_iostream_from_handler2() I/O
stream constructor to allow building of read and write iostreams
deprecating raptor_new_iostream_from_handler().
raptor_iostream_tell() deprecating
raptor_iostream_get_bytes_written_count.
raptor_new_iostream_from_sink(),
raptor_new_iostream_from_filename()
raptor_new_iostream_from_file_handle() and
raptor_new_iostream_from_string()
raptor_iostream_read_bytes() and
and raptor_iostream_read_eof().
Added new write I/O Stream method
raptor_iostream_write_string_python() to write an
encoded string to an I/O stream using python / JSON / Turtle /
N-Triples / SPARQL escaping rules. (Dave B)
Added two new JSON serializers (Dave B):
json based on
Talis RDF/JSON Specificationjson-triples based on the SPARQL results in JSON format.Added new serializer features for the JSON serializers (DaveB):
RAPTOR_FEATURE_JSON_CALLBACK (name 'jsonCallback') to
set the top-level callback function name wrapper above the outer object.RAPTOR_FEATURE_JSON_EXTRA_DATA (name 'jsonExtraData') to
add extra top-level JSON object data.Example of using the resource-centric serializer while defining a callback:
$ rapper -q -o json -f jsonCallback=foo http://librdf.org/raptor/raptor.rdf
foo(
{
"http://librdf.org/raptor/#raptor" : {
"http://usefulinc.com/ns/doap#description" : [ {
...
Added raptor_statement_compare() to provide an
ordering between raptor_statement objects. (Dave B)
Added new parser features to control HTTP headers in
web requests (Dave B, based on a patch in the bug):
Also never Pragma: header with libcurl ever.
Fixes Issue#0000252
RAPTOR_FEATURE_WWW_HTTP_CACHE_CONTROL to control
sending Cache-Control (default: none)RAPTOR_FEATURE_WWW_HTTP_USER_AGENT to control
sending User-Agent (default: none)Turtle parser:
' and
'''-quoted strings and to forbid () in
triple predicate position following the updated Turtle spec. (Dave B)Added new methods raptor_qname_get_local_name(),
raptor_qname_get_value() and
raptor_qname_get_counted_value(). (Dave B)
Added new public SAX2 API class raptor_sax2 exposind
th existing internal one which has been around since the first
release of Raptor 8 years ago and runs on top of either expat or
libxml2. (Dave B)
raptor_new_sax2()raptor_free_sax2()raptor_sax2_set_start_element_handler(),
raptor_sax2_set_end_element_handler(),
raptor_sax2_set_characters_handler(),
raptor_sax2_set_cdata_handler(),
raptor_sax2_set_comment_handler(),
raptor_sax2_set_unparsed_entity_decl_handler() and
raptor_sax2_set_external_entity_ref_handler().
raptor_sax2_start_element_handler,
raptor_sax2_end_element_handler,
raptor_sax2_characters_handler,
raptor_sax2_cdata_handler,
raptor_sax2_comment_handler,
raptor_sax2_unparsed_entity_decl_handler and
raptor_sax2_external_entity_ref_handler.
raptor_sax2_set_namespace_handler()
raptor_sax2_parse_start() and
raptor_sax2_parse_chunk()
raptor_sax2_inscope_xml_language() and
raptor_sax2_inscope_base_uri()
Abbreviated serializers (RDF/XML-Abbrev and Turtle):
Turtle serializer:
RAPTOR_FEATURE_WRITE_BASE_URI
added to control writing @base directive to Turtle. (Dave R)Update URI resolving for RFC3986 changes (Dave B)
Added new method raptor_www_set_http_cache_control()
to set the HTTP Cache-Control: header in requests.
(Dave B, based on a patch in the bug)
Fixes Issue#0000252
Added new method raptor_xml_element_get_language()
to get the language associated with an element. (Dave B)
Pass on error failures in parser and serializer factory construction. (Lauri)
Abbreviated serializers (RDF/XML-abbrev and Turtle): low memory and allocation failure fixes. (Lauri)
Altered API function signatures of
raptor_uri_set_handler(),
raptor_uri_get_handler(),
raptor_new_namespaces(),
raptor_namespaces_init() and
raptor_new_xml_writer() to add appropriate
consts. (Lauri)
Portability fixes for RAPTOR_API and other macros. (Lauri)
Removal of many sets of writable static data in N-Triples parser, URI class, Unicode NFC code, libxml support, Turtle writer and XML writer. (Lauri)
Portability fixes for round() and
trunc() that are not always available in libc but might
be in libm. (Dave B)
Turtle/N3 parsers and serializers, RDF/XML_Abbrev serializer: many low memory fixes and better out of memory errors. (Lauri)
Rewrote internal error log functions to use new error handlers structures and simplify the calls. (Dave B)
Expanded internal raptor_avltree datatype support to
add a cursor, allowing it to be used for creating large ordered
sequences that need to be walked. (Dave B)
Updated rdfdiff utility to handle duplicate triples
in inputs. (Dave B)
raptor_sequence_shift() and
raptor_sequence_unshift() are now as efficient as the
sequence push and pop operations: O(1). (Lauri)
autogen.sh was updated.
rapper utility can now accept multiple
-f / --feature options; previously
only one parser and one serializer feature was possible.
The main changes to this release are:
Provide 100% support for the GRDDL W3C Recommendation of 2007-09-11.
The
Turtle
parser and serializer were updated to support
@base for specifying a base URI, following
Turtle of 2007-09-11.
The Turtle and RDF/XML serializers had performance improvements for large graphs.
Added a TRiG Parser based on Turtle with named graph support.
Several other API changes, fixes and improvements were made.
Fixed Issues:
raptor_namespaces_init() now returns an integer
status.
Added raptor_graph_handler typedef and
raptor_set_graph_handler() to return named graph
identifiers during parsing, initially for the TRiG parser.
These were added the GRDDL parser:
RAPTOR_FEATURE_MICROFORMATS (microformats) to enable
hCard and hCal microformatsRAPTOR_FEATURE_HTML_TAG_SOUP (htmlTagSoup) to use
the HTML tag soup parser if the XML parsing failsRAPTOR_FEATURE_HTML_LINK (htmlLink) to enable html
<link>RAPTOR_FEATURE_WWW_TIMEOUT (wwwTimeout) for setting
URI retrieval timeouts during processingAdded
raptor_new_xml_element_from_namespace_local_name()
constructor to make an XML element from a local name relative
to a raptor_namespace.
Defined a new raptor_unichar typedef for a Unicode
codepoint defined as unsigned long which was the
previous type used. Altered the Unicode function to take it as a
parameter. raptor_unicode_char_to_utf8(),
raptor_utf8_to_unicode_char(),
raptor_unicode_is_xml11_namestartchar(),
raptor_unicode_is_xml10_namestartchar(),
raptor_unicode_is_xml11_namechar() and
raptor_unicode_is_xml10_namechar().
Added raptor_uri_compare() and
raptor_uri_compare_func function pointer for
implementing it in the raptor_uri_handler. The handler
now has a version field initialised to trigger the new
factory method for uri compare when the version is 2 or more.
Added raptor_www_set_connection_timeout()
to set the WWW retrieval connection timeout in seconds.
Added raptor_www_final_uri_handler typedef and
raptor_www_set_final_uri_handler() to return
the final URI seen during WWW retrieval such as after
redirects.
Added raptor_www_get_final_uri() to return the final URI
after a WWW retrieval which might include redirects.
The GRDDL parser/processor was substantially updated and now supports 100% of the Gleaning Resource Descriptions from Dialects of Languages (GRDDL) syntax, W3C Recommendation of 2007-09-11:
it also:
RAPTOR_FEATURE_MICROFORMATS is enabled (default
enabled).RAPTOR_FEATURE_HTML_LINK is
enabled (default enabled).RAPTOR_FEATURE_HTML_TAG_SOUP is enabled
(default enabled).RAPTOR_FEATURE_NO_NET feature to prevent
all networking.raptor_parser_set_uri_filter().RDF/XML parser recognising was updated to just the start of the document for guessing if it should handle content and to try to avoid html URLs.
RSS Tag soup parser recognising was updated to accept with the string 'feed' in the identifier.
TRiG Parser was added based on the Turtle parser, adding named
graphs. It returns name graph URis via a callback set with new API
call raptor_set_graph_handler()
Turtle parser added @base support, fixed turtle
escapes to URIs. Recognising was updated to look for
@prefix early in the document.
Turtle serialiser changes:
@base when an output base URI is given.RDF/XML serialiser was changed to emit a legal empty RDF/XML document when no triples are serialised and to skip emitting statements with bad predicate uris rather than returning an error.
RDF/XML Abbrev serialiser was changed to use an AVL tree rather than sequence for significant performance improvement for large serialisations.
rapper Utility ChangesAdded an --show-graphs option to print named graph
URIs as seen (such as with TRiG).
Added -I / --input-uri and
-O / --output-uri options to set the
input / parsing and output / serializing base URIs
separately. Defaults remain the same - the serializer base URI
defaults to the input base URI, however it was set.
Fixes for when building from Subversion on cygwin (EOL issues, Makefiles).
Remove unused semicolons for prevention of compiler warnings.
Fix some uninitialized variables that some compilers complain about.
Allow RAPTOR_ASSERT_DIE to be externally defined.
Allow RAPTOR_WWW_BUFFER_SIZE to be externally defined.
autogen.sh was updated to handle program versions
better using an inline perl helper.
Start to add resiliance to memory allocation failures and errors inside the library.
Added AVL Tree code to make much faster key:value lookups. This is used for RDF/XML parser XML ID checks and in the 'abbrev' serializers - Turtle and RDF/XML-Abbrev for looking up nodes.
Better libxml error messages are now returned, mentioning some of the names and values that caused the error.
GRDDL parser now passes the (unapproved) test suite for the GRDDL W3C Working Draft 2 March 2007 except for two tests that have been reported as having errors.
When using libcurl as the WWW retrieval library, errors in resolving a URI such as not found (404) are now reported as proper errors and cause parsing to fail rather than just return no triples.
Some improvments where made to guessing for a parser to match some content. Firstly, any mime type with Q <10 is added to the score, don't lose the influence of the mime type entirely. The consequence of this is that Turtle can pretend to be a partial N3 parser. Secondly, the XHTML mime type is now correctly recognised by the GRDDL parser rather than the RSS Tag Soup parser.
Fixed Issues:
xml:base attribute, but does truncate rdf:about and rdf:resource valuesAdded better error reporting for XML errors using the libxml structured error reporing api. From
$ rapper -i grddl http://librdf.org/LICENSE.txt rapper: Parsing URI http://librdf.org/LICENSE.txt rapper: Error - URI http://librdf.org/LICENSE.txt - XML error - http://librdf.org/LICENSE.txt:2: rapper: Error - URI http://librdf.org/LICENSE.txt - XML error - parser rapper: Error - URI http://librdf.org/LICENSE.txt - XML error - error : rapper: Error - URI http://librdf.org/LICENSE.txt - XML error - Document is empty rapper: Error - URI http://librdf.org/LICENSE.txt - XML error - Redland RDF Application Framework - License rapper: Error - URI http://librdf.org/LICENSE.txt - XML error - ^ rapper: Failed to parse URI http://librdf.org/LICENSE.txt grddl content rapper: Parsing returned 0 triples
To this:
$ rapper -i grddl http://librdf.org/LICENSE.txt rapper: Parsing URI http://librdf.org/LICENSE.txt rapper: Error - URI http://librdf.org/LICENSE.txt:1 - XML parser error: Document is empty rapper: Error - URI http://librdf.org/LICENSE.txt:1 - XInclude processing failed for GRDDL document rapper: Failed to parse URI http://librdf.org/LICENSE.txt grddl content rapper: Parsing returned 0 triples
GRDDL parser updated to support the GRDDL W3C Working Draft 2 March 2007:
Turtle parser was changed to accept the N3 mime
type text/rdf+n3 at low Q(quality)
so it might work for N3 that is the RDF subset - quite common.
Changed the RSS Tag Soup parser and RSS 1.0 serializer to stop sharing use of the declared namespaces so that when using both at the same time, there is no double-free of the same objects.
Correct the content: namespace URI in the RSS parser and serializers.
rapper gains a -t/--trace option to show
URIs traversed. Handy for GRDDL.
raptor_uri_resolve_uri_reference() no longer goes
past the end of buffer when the relative URI is ,/
Added an internal API for capturing parsed data as it is seen. Use by GRDDL parser but with no public API.
Added an internal API for structured error reporting. Updates made throughout the library but with no public API.
Internal API raptor_new_sax2() signature changed to
just have an error_handlers pointer argument rather than
multiple function / user_data pairs.
Added a Turtle Terse RDF Triple Language serialiser by Dave Robillard based on the existing RDF/XML-Abbrev serialiser.
Added a GraphViz DOT format serialiser by Evan Nemerson.
The GRDDL parser now does namespace and profile URI recursion and has other improvements and fixes.
Fixed Issues:
rdf:type predicate with a literal object.raptor-config gains a --private-libs
for the internal libraries used in building raptor, with the public
ones only emitted with --libs.
raptor.pc now uses Libs.private for
internal dynamically linked libraries.
The libxml minimum version is now 2.6.8 since 2.6.7 crashes on PPC64 Linux. 2.6.8 was released March 2004 so this should be no burden.
Do not use PATH_MAX so raptor can build on Hurd.
RDF/XML parser now looks for the RDF/XML root element and
namespace declaration in the initial bytes of content when guessing.
This allows content that is in other mime types such as
application/xml to be more likely guessed as RDF/XML.
When guessing a parser to use, if an an exact match is found for the mime type (q=10), then that parser is used.
The GRDDL parser has several changes:
Added a new Turtle Terse RDF Triple Language serializer and two new internal APIs based on the existing RDF/XML-Abbrev serialiser, written by Dave Robillard:
turtle_writer for serializing triples to Turtle This
may be moved to the public API in a future release.raptor_abbrev for the common 'abbreviated
serializer' core that is shared between the RDF/XML-Abbrev and Turtle
serializer.Added a new GraphViz DOT format serialiser writen by Evan Nemerson.
Note that testing the turtle serializing (make test)
requires the rdfdiff -u and a few of the tests take some
time to run.
Added raptor_home_url_string
and raptor_license_string exported strings.
Added raptor_parser_generate_id() as a public function
to generate an identifier for a parser.
rdfdiff gains the -u/--base-uri option to specify the
from file base URI so that if the from file is a local file or
relative URI, it can be given an absolute base.
Failures to retrieve content from a URI using
the raptor_www class implementations now return a
failure as well as setting the HTTP status code to 403 or 404 as
appropriate. Previously success may have been returned with no
bytes.
Prevent losing memory for a raptor_xml_writer when a
serializer is reused several times.
Fixed issues reported on the Redland Issue Tracker:
In maintainer mode, add all the supported compiler -W
warning flags to the CFLAGS.
Allow LEX to be set to things that aren't exactly
'flex'.
Added single triple serializing example to the tutorial to
demonstrate serializing without parsing and building
a raptor_statement.
Declare several raptor functions with GCC printf-formatting attributes when using a new enough GCC.
RDF/XML parser now creates literals with
raptor_stringbuffer so that it does a lot less copying
when constructing longer literals.
Added single raptor_statement serializing example
to demonstrate serializing alone without parsing.
Restored the order of serialized syntaxes back to the same as in Raptor 1.4.10 which Redland was relying on - asking to serialize to mime type 'application/rdf+xml' without specifying a parser name in Redland with Raptor 1.4.11 wrote it in XMP instead of RDF/XML as it used to. This happened more often with language bindings. That problem will be fixed in a future release of Redland but for now, this stops wierd things like that happening.
Added raptor_get_feature_count() to return
the count of features, in preference to using the
macro value RAPTOR_FEATURE_LAST.
Added raptor_www_set_uri_filter() method of the
WWW class (raptor_www) objects to have an
optional URI filter function that checks if the URL given is
allowed to be retrieved, or denied entirely.
Fixed issues reported on the Redland Issue Tracker:
The Raptor Reference Manual now includes descriptions of all the parsers and serializers and the tutorial has a new section describing how to filter URIs and deny network requests.
Added functionality to prevent network requests either via setting
a new feature RAPTOR_FEATURE_NO_NET that denies network
requests during a parser operation or with a URI filter function
raptor_parser_set_uri_filter(). This function uses
raptor_www_set_uri_filter() internally.
Added raptor_get_need_base_uri() to tell if a parser
requires a base URI argument. Presently the N-Triples parser
is the only parser that does not require a base URI.
raptor_start_parse() will now throw an error if no base
URI is given and it is needed.
The GRDDL parser was changed to
handle a list of URIs in the profile so it now
can support dataview:transformation in XML taking a list
of transformations as defined in
The GRDDL profile for XHTML part of the GRDDL specification.
It now also recognises
Embedded RDF and
HCalendar
using well known XPaths and transforms them to RDF triples using well
known XSLT sheet URIs.
The Guess parser now resets after each parse and does a fresh guess on the syntax based on the incoming information. Fixes Issue#0000091
The Turtle parser (and experimental N3 parser) were changed to now require base URIs as they always should have. The error messages when reporting problems with grammar tokens now return better responses. Added better memory cleanup during parser error recovery.
The RSS 1.0 Serializer now works again.
Updated the RDF/XML Abbreviated serializer to do proper reference counting on the blank/resource nodes used as subjects and objects to prevent dual-triple generation. Fixes the reported Issue#0000014
The internal SAX2 API can also prevent network fetches with the
feature RAPTOR_FEATURE_NO_NET.
Fixed a SAX2 problem that caused parsers that use it to leak memory for 1 URI, affected RDF/XML and RSS Tag Soup.
rapper help and verbose message formats were tidied.
No parser will now generate a triple with an identifier type
RAPTOR_IDENTIFIER_TYPE_ORDINAL. Only identifier type
resource, anonymous (blank node) and literal will be generated.
All serializers will convert any
RAPTOR_IDENTIFIER_TYPE_ORDINAL type on input to type
resource.
No longer adds LDFLAGS to pkgconfig file raptor.pc
and raptor-config fixing
Issue#0000097.
All parsers no longer generate a triple with an identifier type
RAPTOR_IDENTIFIER_TYPE_ORDINAL, as deprecated in 1.4.8.
The replacement type generated is RAPTOR_IDENTIFIER_TYPE_RESOURCE.
The RSS Tag Soup (rss-tag-soup) parser now makes the
triples appear before parser destruction. This caused odd symptoms
like parsing in python returning no triples and the parser then
crashing during object destruction.
The RDF/XML (rdfxml) parser no longer crashes if a
comment is seen outside an element, such as before or after the root
element.
The RDF/XML (rdfxml) serializer no longer crashes if
the serializer is used more than once.
Now using Subversion for version control and the Raptor installation instructions explain how to get Raptor from Subversion.
configure now allows --enable-parsers=node and
--enable-serializers=none. Using both is possible!
No longer require libxml2 for the RSS Tag Soup parser
Various Win32 fixes and VC build files updates from John Barstow.
A new Raptor Tutorial was written covering using all parsing and serializing functions along with example code.
The Raptor Reference Manual now covers 100% of all functions, structs and defines with gtkdoc generated documentation.
rapper utility Changesrapper now uses namespaces found in parsing to give
hints to the serializer as to how to format the output. The result
of this is that rapper can be used as an RDF
pretty-printer and is especially good at such things as turning flat
N-Triples to RDF/XML or RDF/XML-Abbrev. such as:
rapper -q -i ntriples -o rdfxml-abbrev example.nt
All parsers no longer generate
RAPTOR_IDENTIFIER_TYPE_PREDICATE as the statement
predicate type, as deprecated in 1.4.8. The replacement
type generated is RAPTOR_IDENTIFIER_TYPE_RESOURCE.
The Turtle parser now has true and false
boolean literals, which were accidently omiited from the parser in
the 1.4.8 update.
Parsers can register capabilities for handling multiple mime types
with Q values. These are then used in WWW requests for content
in the Accept: header for HTTP. Added
raptor_parser_factory_add_mime_type for registering,
raptor_parser_get_accept_header to get the accept header
values for the types supported by one parser.
From the previous change, the RSS parser now accepts several
unregistered RSS mime types as well as the registered Atom one; the
RDF/XML parser accepts unregistered mime type text/rdf
seen occasionally; the Turtle parser accepts several experimental
mime types. All unregistered or experimental types are accepted with
lower Q than any registered type.
The RSS Tag Soup parser for RSS* and Atom no longer requires libxml2 (for it's XML Reader API). Internal changes mean that it will fully work on top of expat.
The RSS/Atom serializer now uses input namespace declarations to choose namespaces on output.
Added raptor_serialize_set_namespace_from_namespace
to set a namespace for serializing from an existing
raptor_namespace.
Serializing to RDF/XML (or RDF/XML Abbrev) now does not double-free URI strings. Fixes Issue#0000065
RSS serializer no longer writes the XML header twice.
Added raptor_iostream_write_uri to directly write a URI
to an iostream without the need to go via a string.
Fixed bug in raptor_iostream_write_xml_any_escaped_string
failing to write ';' after escaping U+0009 and U+000A
Added raptor_namespaces_qname_from_uri to do URI
splitting into qname prefering to use the current in-scope namespaces
before having to search.
raptor_namespaces_format now NULL-terminates the namespace string.
Fixes Issue#0000062
Added raptor_namespace_get_counted_prefix to return a
namespace prefix and it's length.
Added raptor_qname_get_namespace to get the namespace
associated with a QName.
raptor_stringbuffer_append_counted_string and
raptor_stringbuffer_append_string now Do nothing on
appending a NULL string or a string of length 0.
Fixes Issue#0000073
raptor_utf8_to_unicode_char now also checks for
overlong UTF-8 sequences, illegal code positions or out of range
codes.
Deprecated raptor_uri_is_file_uri which takes a
URI string argument for new function raptor_uri_string_is_file_uri
which more clearly says that.
Changed all URI string calloc/mallocs to add enough room for a full pointer at the string end to stop valgrind moaning on 64bit systems when looking for the end of string NUL.
raptor_uri_set_handler and
raptor_new_iostream_from_handler now take const
handler arguments.
Get the curl success status into a long, not an int which causes failure on 64 bit. Fixes Issue#0000075
WWW requests for content to parse now always send an appropriate
Accept: header with Q values for the parser, or for the
guess parser, all supported mime types.
Added XML element methods
raptor_xml_element_get_attributes and
raptor_xml_element_get_attributes_count,
raptor_xml_element_is_empty
to the SAX2 API.
Many internal changes were made to the SAX2 API to finally separate XML and RDF/XML parts. The SAX2 API is now fully usable on either libxml2 or expat. That last sentence implies a lot of work, by the way.
A large source re-arrangement was performed. All C sources and headers that build the library are now in the src dir, general documentation in the doc dir and utilities in the utils dir. This both tidied up the mixture of files at the top level and also enabled better use with gtk-doc.
Future API change: From the next release of
Raptor, raptor_statement predicates will return
identifiers of type RAPTOR_IDENTIFIER_TYPE_RESOURCE
instead of RAPTOR_IDENTIFIER_TYPE_PREDICATE.
Identifiers of type RAPTOR_IDENTIFIER_TYPE_ORDINAL may
no longer be returned in any statement position (to be confirmed).
Version Control change: Raptor will be switching to use Subversion for version control after the 1.4.8 release. Please check the Redland Subversion site for the latest status or the online Raptor installation notes for the raptor specific subversion installation information.
The autogen.sh script for building from CVS was revamped to be more modular.
configure now takes an --enable-gtk-doc
option to enable building of the documentation using the
gtk-doc utility. It is by default enabled only if the
utility is available.
Added a new configure option --enable-serializers (in
1.4.7) to allow the selection of the required RDF serializers from
any of those supported.
raptor-config now has a --options
argument to list the configured or discovered options of the library
such as parsers, serializers and other choices.
The GNOME gtk-doc program is now used to
automatically extract documentation from source comments into
reference documentation. This is then merged with templates and
additional documentation to provide a reference manual for raptor
as XML document which is turned into HTML along with GNOME devhelp
support.
This new documentation intended to replace the libraptor manual page/web page as easier to read document with scope for better expanding with more detail of raptor including examples and tutorial information. The manual page will continue to contain the summary information for the present.
Fixed a long-standing URI resolution bug on win32 - only remove leading / if there is one present (patch from John C. Barstow)
rapper utility ChangesAltered the -g argument to invoke the guessing parser
rather than guess on file/URI name alone. This is now equivalent to
choosing an input syntax of guess with -i guess.
Added a --show-namespaces long option (no short version)
to show namespaces that are declared in the parsed content.
A new guessing parser was added, picking the actual parser to use at run-time based on protocol or other information.
Allow a content type returned by a protocol (such as HTTP) to
enable choosing of parser at run-time. Added a new optional
parser factory method content_type_handler to return this.
Allow parsers to handle several syntaxes rather than only 1 or 2.
Parsers can now return namespace prefix/URI declarations as
they are given in the content by the means of a new handler type
raptor_namespace_handler and parser method
raptor_set_namespace_handler. Duplicate namespace
prefix/URIs can be returned.
Bug fix when the entire content is in one chunk (René Puls).
A new parser that guesses the actual parser to use at run-time based on a combination of MIME Content-Type, file or URI name and in future, iniital bytes of the content. If the Content-Type is an exact match to a known parser, it is always chosen before trying heuristics.
When emitting literals, handle a datatyped empty literal. This is a post-REC errata for the revised RDF/XML recommendation. See archived example for further information.
Added atom 1.0 support including use of the new namespace. Atom 0.3 namespace terms are turned into new properties. Replace atom copies of Dublin Core or RSS properties with the original terms:
| Atom 1.0 term | Original term |
|---|---|
atom:content | rss:description |
atom:id | rss:link |
atom:published | dc:date |
atom:rights | dc:rights |
atom:title | rss:title |
Apply the in-scope base URI (such as from xml:base)
to atom 1.0 fields that take URI values:
atom:id, atom:icon and atom:logo.
Added optional date parsing code to turn XML RSS date fields into ISO format ones, suitable for Atom and XML schema datatypes format. Will use library parsedate code from curl or INN if available.
XML RSS field pubDate is now turned into Dublin Core
dc:date field in the ISO format.
XML RSS field content is turned into
content:encoded in RDF triples on output with
escaping.
Updated to support Turtle version 2006-01-02 (announcement).
Switch qname, blank node and prefix definitions to SPARQL ones.
Check for illegal not-hexadecimal \u and \U escape values.
Fix greedy matching of long literals ("""....""") that ended on the last """ found rather than the first.
Added double and decimal constants.
Added optional +- sign to all numeric constants.
Allow \" escape inside long strings.
Take care to reset the generated raptor_statement
language and datatype fields when not used.
Added a new Atom 1.0 serializer (name atom) by parameterising the RSS 1.0 serializer.
Added a new Adobe XMP compatible serializer (name rdfxml-xmp) by parameterising the RDF/XML Abbreviated serializer. Patch provided by Sid Steward.
All serializers can be chosen at configure time from
those available using configure option --enable-serializers.
The RSS parser and serializer can now be independently enabled or disabled. The RSS serializer no longer requires an XML parser.
A new XML Writer feature
RAPTOR_FEATURE_WRITER_XML_VERSION was added to allow
chosing XML 1.0 (value 10) or XML 1.1 output (value 11). This
feature is also accepted by serializers as an option and used by the
RDF/XML and RDF/XML-Abbrev serializers.
A new XML Writer feature
RAPTOR_FEATURE_WRITER_XML_DECLARATION was added to allow
omitting the XML declaration (default true).
Added functions raptor_xml_any_escape_string()
and raptor_iostream_write_xml_any_escaped_string()
which take an XML version. The XML 1.0 functions give errors
when attempting to write #x1-#x1f (excluding #x9, #xA, #xD) or #x7F.
Added a new serializer using the Atom 1.0 format and namespace. This reads RDF triples in the RSS 1.0 model, along with any additional atom 1.0 properties and serializes an Atom 1.0 feed file.
Added a new serializer writing RDF/XML in the profile used by
Adobe XMP. Note that this does require RDF triples to be used in a
certain style; for example all triple subjects are the "current
documment" giving rdf:about="".
Fix a bug when adding a default path of / to a URI
in functions raptor_new_uri_for_xmlbase()
and raptor_new_uri_for_retrieval().
(Bug #0000045)
raptor_uri_equals was altered to accept NULL pointers, which do not compare equal to a non-NULL URI. NULL does equal NULL.
The internal SAX2 class was extensively changed so that remaining interdependencies with the RDF/XML parser were removed and it can now be re-used for other syntaxes cleanly. Several functions were modified or added.
Removed old and hardly-tested internal support for XML entity resolution (libxml only).
Various fixes for GCC 4 warnings.
Fix a couple of crashes in the RSS tag soup parser / serializer (Dave Beckett, Suzan Foster).
configure now looks for the
libxslt/xslt.h header as well as the
libxslt library and disables XSLT and GRDDL support it
if is missing. This catches systems with the libraries without
headers as has happened on some OSX versions.
In serializers rdfxml and rdfxml-abbrev, report failure to serialize to RDF/XML if the predicate URI is not absolute.
Added --with-xslt-config configure option
Added a new parser for
Gleaning Resource Descriptions from Dialects of Languages (GRDDL)
which allows reading XHTML and XML as RDF triples by using profiles
in the document that declare XSLT transforms from the XHTML/XML
content into RDF/XML which is the RDF content. It does not
support all the GRDDL styles, for example
dataview:namespaceTransformation,
or perform recursive transformations.
The turtle parser now accepts """long literals"""
XML writer feature support were added in 1.4.5 and not documented.
The new functions are:
raptor_xml_writer_features_enumerate,
raptor_xml_writer_set_feature,
raptor_xml_writer_set_feature_string,
raptor_xml_writer_get_feature and
raptor_xml_writer_get_feature_string.
The three XML writer features added are
\fBRAPTOR_FEATURE_WRITER_AUTO_INDENT\fR with boolean value (default true)
to auto-indent the XML,
\fBRAPTOR_FEATURE_WRITER_AUTO_EMPTY\fR with boolean value (default true)
to automatically generate empty elements if a start/end element sequence
has no content and
\fBRAPTOR_FEATURE_WRITER_INDENT_WIDTH\fR with an integer value (default 2)
to set the indenting level for the XML.
New build configuration and portability fixes for win32 (John Barstow)
Portability fixes for win32 - added
SIZEOF_UNSIGNED_SHORT (Dave Viner, others)
Added a signing memory debugging system to aid checking when
raptor-allocated memory is freed in another library or vice-versa
enabled by --with-memory-signing configure option
(defaults to on in maintainer mode).
Fixed a few internal malloc/frees to use RAPTOR_MALLOC / RAPTOR_FREE so that the above signed memory system worked.
RDF/XML serializer:
Use the maximal name when splitting a predicate.
Turn datatyped literals that are rdf:XMLLiteral into inline XML with
rdf:parseType="Literal" rather than XML-escaped.
RDF/XML abbreviated serializer:
Fix a crash when there is a NULL base URI.
Use the maximal name when splitting a predicate.
Turn datatyped literals that are rdf:XMLLiteral into inline XML with
rdf:parseType="Literal" rather than XML-escaped.
RSS tag soup parser:
Fix crash with unexpected use of alternate attribute.
Update from Suzan Foster to reflect the latest status of the
enclosure vocabulary and allow multiple common items and fields.
RSS 1.0 serializer: Added RSS enclosures serializing.
grapper example GTK program now stores the window
width and height using gconf2.
Added a new RDF/XML with abbreviations serializer
rdfxml-abbrev written by Steve Shepard which handles
several of the abbreviations specified by the
RDF/XML Syntax Specification (Revised)
W3C Recommendation. It is suitable for writing small documents as
there are known scaling issues.
The RSS tag soup parser was updated to work better when there is no base URI given. It also now supports reading the RSS 1.1 format and turning it into RSS 1.0 model triples.
Deprecated raptor_ntriples_string_as_utf8_string as
rather too internal to be useful, since it only works with a parser.
More fixes to work around the broken libxml2 on Apple OSX 10.3.x with inconsistent shared libraries / headers.
Experimental and incomplete Notation 3 parser - updated to match changes to Turtle. CVS changes only, not enabled in standard builds.
Make the RSS tag soup parser handle RSS 0.9 namespace elements by turning them into RSS 1.0.
Fix a couple of crashes in the RSS 1.0 serialiser when no base URI is used.
Make raptor_uri_to_relative_counted_uri_string
work when the base or reference URI have no paths such as like
http://example.org
Added portability fixes for Win32 to get Raptor 1.4.3 building with MS Visual Studio using expat and libcurl. The RAPTOR_INTERNAL define was moved to the build configuration and defines added for integral type sizes. Patch from Dave Viner (dviner at apache dot org).
A release with the major new feature of an XML writer API. This is now used along with a new supporting XML element class to improve the existing RDF/XML serializer and to provide a new RSS 1.0 serializer.
This API it is also used by the next release of Rasqal to provide serializing of query results to XML.
The new raptor_xml_writer class functions added are:
raptor_new_xml_writer (constructor),
raptor_free_xml_writer (destructor),
raptor_xml_writer_empty_element,
raptor_xml_writer_start_element,
raptor_xml_writer_end_element,
raptor_xml_writer_cdata,
raptor_xml_writer_cdata_counted,
raptor_xml_writer_raw,
raptor_xml_writer_raw_counted,
raptor_xml_writer_comment and
raptor_xml_writer_comment_counted.
The new raptor_xml_element class functions added are:
raptor_new_xml_element (constructor),
raptor_free_xml_element (destructor),
raptor_xml_element_get_name,
raptor_xml_element_set_attributes,
raptor_xml_element_declare_namespace and
raptor_iostream_write_xml_element.
RSS tag soup parser now works with older libxml2s (2.5.10+), including the one shipped with some Apple OSX versions that has an inconsistent header file and library.
RSS tag soup parser recognises/scores more common XML RSS file names.
RSS tag soup parser turns XML RSS
<guid isPermaLink="true">val</guid>
into RDF/XML form <guid rdf:resource="val"/>, leaving
the non isPermaLink form to be a literal value.
A bug was found in libxml2 that causes double expanding of XML entities in RDF/XML. This has been reported but cannot be worked around from raptor. The expat XML parser can be used as an alternative, as it does not have this problem. A test was added for this bug but it will not cause the test suite ('make check') to fail.
Added additional Turtle parser tests that cover Notation 3 syntax that is not part of the Turtle language.
Added
raptor_parser_set_feature_string and
raptor_parser_get_feature_string
methods to set/get string feature values.
Added feature relative_uris for serializers. This
is used by the RDF/XML serializer and enabled by default.
Added feature start_uri for serializers with a string
value to set the start URI for serializing. Not used at present.
Added new methods raptor_serializer_features_enumerate
to list serializer features and functions to set/get serializer
feature integer or strings values:
raptor_serializer_set_feature,
raptor_serializer_get_feature,
raptor_serializer_set_feature_string and
raptor_serializer_get_feature_string.
Added raptor_serialize_set_namespace to allow user
declaration of prefix/URI namespaces pairs as serializing hints.
the RDF/XML serializer was improved using the new XML Writer class so it now uses any user-declared namespace hints in it's output and emits relative URIs whenever possible. The latter was provided by a patch from René Puls.
A new RSS 1.0 serializer was added, using the new XML Writer class and using the same structures, classes and properties as the RSS tag soup parser.
Added relative URI generating code from a patch written by René Puls
and provide this with two new methods
raptor_uri_to_relative_uri_string and
raptor_uri_to_relative_counted_uri_string.
Added raptor_uri_print to print a URI to a file handle.
Added methods raptor_uri_to_string and
raptor_uri_to_counted_string to return a URI as newly
allocated strings.
Many classes gained methods to write to iostreams, supporting the
new XML Writer class functionality. The added methods are:
raptor_iostream_write_namespace,
raptor_iostream_write_ntriples_string,
raptor_iostream_write_qname,
raptor_iostream_write_statement_ntriples,
raptor_iostream_write_stringbuffer,
raptor_iostream_write_xml_element and
raptor_iostream_write_xml_escaped_string.
Added raptor_namespace_copy copy
constructor and raptor_new_namespace_from_uri
constructor to build a namespace from a raptor_uri object.
Added utility function raptor_new_namespace_parts_from_string
to decode syntax of the form
xmlns:prefix="uri"
into prefix and uri string pairs.
Added raptor_namespaces_find_namespace_by_uri method
for namespace stack to find a declared namespace by URI. This
complements raptor_namespaces_find_namespace which
already provides searching by prefix.
Added several methods for checking characters forming
parts of XML 1.0 or XML 1.1 names:
raptor_unicode_is_xml10_namestartchar,
raptor_unicode_is_xml11_namestartchar,
raptor_unicode_is_xml10_namechar and
raptor_unicode_is_xml11_namechar.
Added a function raptor_utf8_check to check that a
string is legal UTF-8 and all the encoded Unicode characters are in
the range U+0 <= character <= U+10FFFF
Added a function raptor_xml_name_check to check that
a string is a legal XML name (1.0 or 1.1) as well as legal UTF-8.
Feature support: Added raptor_feature_value_type to
determine value of a feature - either integer (most) or string.
XML QName class: Added raptor_qname_copy copy
constructor.
Sequence class: Added raptor_sequence_join
to join two sequences of items, leaving one empty.
Statement class: Added raptor_statement_copy copy
constructor and raptor_free_statement destructor.
Previously these were internal to raptor.
The rapper utility was modified to add a feature form:
-f xmlns:PREFIX="URI"
allowing the setting of output serializer namespaces.
The namespace URI string constants exported by raptor are now of type unsigned char*.
Make raptor_xml_escape_string fail correctly when
given bad UTF-8 to escape.
Fixed a buffer overrun in decoding a URI scheme in
raptor_uri constructors such as
raptor_new_uri.
Fixed a crash in RSS enclosures crash when the url
attribute seen on a non-<enclosure> element
raptor_xml_escape_string return value has changed to
be an int, returning <0 on failure. This allows the empty string
encoding an empty string case to work and be distinguished from an
error.
A release with the major new feature of providing serializing of RDF triples to syntaxes. It also added a new support class for I/O streams and had other minor fixes.
Added a Raptor Serializer class (raptor_serializer)
with similar style to Parser (raptor_parser). Two
serializers are provided, for RDF/XML and N-Triples. The serializing
can be done to files, C FILE* or to strings. The
raptor_iostream class that provides this also allows writing
to any other form by creating a custom iostream.
The new raptor_serializer class functions added are:
raptor_serializers_enumerate,
raptor_serializer_syntax_name_check,
raptor_new_serializer,
raptor_free_serializer,
raptor_serialize_start,
raptor_serialize_start_to_filename,
raptor_serialize_start_to_string,
raptor_serialize_start_to_file_handle,
raptor_serialize_statement,
raptor_serialize_end,
raptor_serializer_get_iostream,
raptor_serializer_set_error_handler,
raptor_serializer_set_warning_handler and
raptor_serializer_get_locator
Added a Raptor I/O stream abstraction in
raptor_iostream class to support serializing of RDF to
multiple output streams such as to filenames, to C standard I/O
FILE* handles and to strings especially for
cross-language use. A raptor_iostream_handler can
be used to construct a user-defined iostream.
The new raptor_iostream class functions added are:
raptor_new_iostream_from_handler,
raptor_new_iostream_to_sink,
raptor_new_iostream_to_filename,
raptor_new_iostream_to_file_handle,
raptor_new_iostream_to_string,
raptor_free_iostream,
raptor_iostream_write_bytes,
raptor_iostream_write_byte,
raptor_iostream_write_end,
raptor_iostream_write_string,
raptor_iostream_write_counted_string,
raptor_iostream_get_bytes_written_count,
raptor_iostream_write_decimal and
raptor_iostream_format_hexadecimal.
The rapper utility was modified to use serializer
class so that the output formats supported are now N-Triples
(-o ntriples) - the default, and RDF/XML (-o
rdfxml).
Raptor now exports more static namespace URI strings for
general application use:
raptor_xml_namespace_uri,
raptor_rdf_namespace_uri,
raptor_rdf_schema_namespace_uri,
raptor_xmlschema_datatypes_namespace_uri,
raptor_owl_namespace_uri,
and the length
raptor_rdf_namespace_uri_len.
The raptor_stringbuffer class gained a new method
raptor_stringbuffer_copy_to_string
which allows efficient copy-out of a constructed string.
The raptor_www class gained a new method
raptor_www_fetch_to_string to allow retrieving of
web content as a single string.
RSS tag soup parser gained support for generating triples for enclosures, after a patch from Suzan Foster. Changes made include correcting the enclosures namespace and tidying some memory leaks.
A release with major improvements along with several minor fixes.
Raptor's License was changed from LGPL 2.1/MPL 1.1 to LGPL 2.1/Apache 2
Thanks to Chris Pointon for several patches to make Raptor easier to build under Win32 which were applied, with some slight modifications.
Increased WWW content retrieval buffer size from 256 bytes to 4K since this was causing problems for even moderate size documents.
After testing raptor on a very large RDF/XML file with many
rdf:ID values, the check for duplicate values was found
to be inefficient in memory and slow. The implementation was
improved to be more memory efficient and a new parser feature
check_rdf_id was added to disable checking (default is
enabled).
Added a new Unicode NFC checker to replace the functionality
formally available by calling the GNOME glib function
g_utf8_normalize. This new checker is done via several tables and
adds approximately 50K to the object size of the library when
compiled on x86. This code and tables can be disabled with configure
option --disable-nfc-check causing all checks to
succeed.
Fix the exporting of
raptor_xml_literal_datatype_uri_string and
raptor_xml_literal_datatype_uri_string_len as constants
for use by applications. Previously raptor.h wasn't doing this
correctly.
Added raptor_calloc_memory for allocating zeroed
memory inside raptor, for use by applications passing memory in/out
of raptor.
Added a new configure option --enable-parsers to
allow the selection of the required RDF parsers from any of those
supported (RDF/XML, Turtle, N-Triples, RSS tag soup).
Reorganised the sources to split parsing support from RDF/XML to support compiling without this parser.
Updated the RSS Tag Soup parser to start to handle the Atom 0.3 currently being standardised by the IETF Atom Publishing Format and Protocol working group.
Altered the Turtle parser to work with large source documents that exceeded bison limits. Thanks to Geoff Chappell for providing a fix for this.
Rewrote the URI parsing to create an internal structure and improved the relative URI resolving in preparation for future work such as potentially supporting URI canonicalisation such as proposed to be used by Atom.
A release with some minor fixes.
Added a new configure option
--with-expat-source=DIR
to allow the use of external expat source trees in either the old or
newer directory structure style. (Patch from Mark Smith).
Added raptor_alloc_memory for handlers that need to
allocate memory in the same heap as raptor uses for
raptor_free_memory. This is mostly useful for allocating
memory that is freed by raptor in error, ID and statement handlers
on win32 which has separate heaps for different DLLs.
A bug was fixed where errors which happened when fetching WWW content were always printed to stderr. They are now passed to the main error routines which allows applications to retrieve them.
Accessor functions were added for parts of the public
raptor_locator structure which makes it possible to get
structured error information from language bindings via Redland
(Patch from Edd Dumbill). The new functions are:
int raptor_locator_line(raptor_locator *locator);int raptor_locator_column(raptor_locator *locator);int raptor_locator_byte(raptor_locator *locator);const char * raptor_locator_file(raptor_locator *locator);const char * raptor_locator_uri(raptor_locator *locator);The Unicode Normal Form C (NFC) checking via the GNOME glib
library function g_utf8_normalize is broken, comparing
the data it says is failed against other NFC checkers. It is also
slower than need be since it is doing full normalizing rather than
just checking for NFC, and adds a rather large dependency for just
one function. A new portable checker will be added in a later
release.
A release primarily to fix some win32 and portability issues.
raptor.h now includes stdarg.h
Corrected the raptor_print_statement declaration in
raptor.h for the argument statement to have one less 'const' which
matches the actual code.
Made several portability fixes for compiling natively on win32 which doesn't quite do POSIX or C99.
Changed the support for file: URIs and converting to and from
filenames. It now %-escapes spaces and % characters on conversion
to and from filenames with
raptor_uri_uri_string_to_filename,
raptor_uri_uri_string_to_filename_fragment
and
raptor_uri_filename_to_uri_string.
For Win32, more tests were added and the
format of URIs supported corrected to use the file:///c:
form rather than file://c|/
URIs that resolve to directories now return an error when lstat is available to check.
The Turtle parser was updated to only allow language with non-datatyped literals, allow a '_' immediately after a ':' in qnames and to make a bare ':' qname work correctly.
The Turtle parser was fixed to re-initialise correctly when performing multiple parsings. The other parsers already did this correctly.
Added a warning to the RDF/XML parser for unknown
rdf:parseType values, when parsing in lax mode - which
is the default. It now tells the user when the parsing is working as
'Literal' mode by finding an unknown value. This is controlled by a
new parser feature warn_other_parsetypes which is default set true
in lax mode. Parser modes are controlled by the
raptor_set_parser_strict method.
A release primarily to provide support for the new Rasqal RDF query library but with some new features and fixes.
Added a new constructor
raptor_new_parser_for_content to guess the parser to use
from hints in URIs or content, using a new utility function
raptor_guess_parser_name.
Additional checks were added to the RDF/XML parser for RDF-namespaced names in element and attributes and if they are forbidden giving an error otherwise if unknown, giving a warning.
The
Turtle
parser was updated to correct the collections syntax, allow '-' in
names and QNames and to add integer literals. This parser now correctly
uses raptor_generate_id when a blank identifier name is
needed.
Completed parser feature support by adding raptor_get_feature,
raptor_feature_from_uri, and
raptor_features_enumerate to get values and enable
discovery of supported features at run time.
raptor_set_feature was changed to give return a success
value
Added a new method raptor_get_mime_type to get the
MIME type of the syntax for a parser
raptor_parse_uri_with_connection (which is called by
raptor_parse_uri) now sets the HTTP Accept:
header to the MIME type of the parser in WWW requests using the new
raptor_www_set_http_accept().
rapper changesAdded options -f/--feature for setting
features and -g/--guess for guessing syntax
from some content or identifiers. See rapper(1)
for all rapper options.
Added raptor_syntax_name_check to check for valid
syntax language names.
Added raptor_free_memory to free memory returned by
raptor functions.
Added Unicode utility functions raptor_unicode_char_to_utf8 and
raptor_utf8_to_unicode_char.
Exported URI string raptor_xml_literal_datatype_uri_string.
Deprecated raptor_print_statement_detailed always
intended to be internal.
Added support to set the HTTP Accept: header for curl
and libxml2 when retrieving HTTP content by the new
raptor_www_set_http_accept method.
Added a utility class raptor_sequence providing
simple sequences that can handle stacks and queues
Added a utility class raptor_stringbuffer
for constructing strings from substrings appended or prepended.
Release notes for 1.2.0 and earlier are in the NEWS page or ChangeLog
Copyright (C) 2003-2008 Dave Beckett
Copyright (C) 2003-2005 University of Bristol