Outlook 2004: XQuery's Final Countdown Underway

The W3C has completed its latest draft proposals for specs governing XQuery data models. The Last Call drafts for XQuery 1.0 and XPath 2.0 are available for review and open to comment until February. Except for tabling a plan for DTD-to-XML inter-communications, the W3C specifies how to create, store, recall and publish data and documents for XQuery access, and a test suite is under review. Get prepped with access to the spec, links to tools, dev (not vendor) briefings and forums.

Tags: XQuery, XML, Proposals, Developers, XML Query, Support, Specs,

Despite disharmony among database vendors over web services workflow (or choreography), they do agree on XQuery, the proposed standard for querying XML data and documents.

The W3C has completed its latest draft proposals for specs governing XQuery data models.
The Last Call drafts for XML Query (XQuery) Data Model 1.0 and XPath 2.0 are available for review and open to comment until February.

Also available for review are the W3C's set of early XQuery Use Cases, which take developers through using various datatypes of the proposed spec, explaining how to apply rules for descriptions and DTD (Document Type Definition), as well as outlining sample data, queries and results.

The W3C's overall posiiton on XQuery Data Model 1.0 is as follows: "The XML Query Data Model relies on information provided by XML Processors and Schema Processors, and it 'must' ensure that it does not require information that is not made available by such processors." For specific technical info, developers should note that the XQuery Data Model will be based on the W3C XML Infoset, and will include support for the long-standing XML Namespaces specs, first ratified in 1999.

Despite the W3C's strong commitment to ensure XQuery Data Model 1.0 (and XPath 2.0) continuity with other XML-related specs, the W3C found it necessary to back away from some technical aspects of the original vision for the XQuery Data Model -- and how developers and data would interact with XQuery-able assets.

In specific, W3C decided to table two technically demanding XQuery proposed requirements and include them in a later version of the XQuery and/or XPath specs. They are both in section 3.4 (Operations on Schemas) of the XQuery proposal.

  • The W3C voted to delay a proposed requirement that "queries should provide access to the XML schema or DTD for a document, if any." In its decision, the W3C said support for the requirement "was felt to be too complex for XQuery 1.0," adding that "an XML schema document can be queried independently by a user."

  • The W3C voted not to support a proposed spec that would have mandated that "if the schema is represented as a DTD, a mapping to an appropriate XML schema representation may [also] be required." In foregoing such a requirement, the W3C cited its above action, tabling the requirement that queries provide access to both XML and DTD for a document.

Inside XQuery's Bridging SQL-to-XML
XQuery is a proposed method for working with unstructured, nonrelational data and a likely follow-on to relational data that uses SQL. The standard also will lay out core specifications for bridging the two data worlds.

The test suite aims to provide W3C with a mechanism to ensure that different vendors' XQuery implementations are sufficiently similar to allow developers to work with XQuery across multiple database platforms. Once the test suite is submitted, the W3C will put it through internal testing for compliance with the proposed XQuery specs, and then submit the test suite to public comment before incorporating it in the final proposed standards package.

In addition, the W3C is conducting a final review of a test suite for XQuery, evaluating a proposal for such a suite based on work submitted by Microsoft and IBM. That work is being supported by other leading database providers, including Oracle and Informix, as well as data-mining vendor Business Intelligence.

In parallel with the W3C's XQuery test work, other XQuery test suites/harnesses developed outside the W3C process are being revved to comply with W3C's final specs. One XQuery test was developed by the federal government's National Institute of Standards and Technology (NIST). Another was developed by BumbleBee, a joint venture of Clarkware Consulting, Inc. and Hunter Digital Ventures, LLC.

More Dev Resources of XQuery Tools
But before the release of the final XQuery Data Model 1.0 spec, a number of developer tools that anticipate most of the W3C specs will become available.

The W3C proposals impose the following requirements on XQuery tools:
  • Must be declarative
  • Must be namespace aware
  • Must coordinate with XML Schema
  • Must work whether or not a schema is available
  • Must support simple and complex datatypes
  • Must support universal and existential quantifiers
  • Must combine information from multiple documents
  • Must support aggregation
  • Must be able to transform and to create XML structures
  • Must be able to traverse ID references

[This list comes from a comprehensive XQuery overview recently prepared and presented by Mark Lehmann, of the University of Klagenfurt (Germany).]
A flurry of XQuery tools is already available that comport to comply with W3C's requirements. Among those getting some traction among devs are:
  • IBM'sXML for Tables is designed to provide applications and end users an XML view of relational tables and a query of those views as if they were XML documents. Written in Java, XML for Tables allows querying of SQL data (stored in the relational tables) in XQuery, and provides functions for creating XML views of relational tables so that the SQL data is treated like virtual XML documents and can be queried in XQuery.
  • Microsoft's www.xqueryservices.com provides a wide variety of XQuery support and tools, including downloadable libraries available for the .NET framework, as well as an online XML Query Language Demo where users can run queries against a defined set of documents. Microsoft has also committed to support XQuery in a future version of SQL Server. In the meantime, Microsoft is already taking steps to ease current SQL Server 2000 developers' task of integrating their data with web pages and XML. For instance, SQL Server 2000's XML query features and XSL stylesheets let web servers running Internet Information Server (IIS) 5.0 stream formatted query result sets to browsers in vanilla HTML.

  • XML software and storage firm Software AG has also developed a prototype of XQuery. The firm's Quip allows users to experiment with XQuery syntax and run queries against any document. Among its functions are: a graphical user interface for writing queries and viewing results; online help including syntax diagrams for XQuery; demonstration of W3C use case queries; and support for queries that can be made either on XML files stored in a file system or on XML stored in the Software AG Tamino XML Server.
There are also a growing number of Open Source support projects for XQuery. Among them:

  • SourceForge's Kweelt. Kweelt is a framework to query XML data. Built as an evaluation engine for the Quilt XML query language (proposed by Chamberlin, Florescu and Robbie), Kweelt's developers now say a developer can run all the use cases published by W3C for the XML query requirements. [Kweelt's website notes: Kweelt offers multiple XML back-ends. The query evaluator does not impose any specific storage for XML, but relies on a set of interfaces (Node and NodeList) implemented by a NodeFactory. Kweelt comes with a couple of built-in node factories (DOM, DOM+, SAX, Wizdom), but the user can easily provide his/her own.]

  • SourceForge's XQuench is a JJTree parser project under development. The project aims to provide devs with a cross-platform query engine and an API that implements the W3C specs. Also, while it doesn't process the query, it will evaluate your syntax. To learn more or get on their mailing list, go to http://xquench.sourceforge.net/.

For a more extensive list of implementations and corresponding links, visit Elliotte Rusty Harold's Café con Leche, which also has a great list of XQuery and XPath tools resources. Devs can also find an ongoing chat/forum on XQuery tools (for devs) at OASIS' XML.org.