XQuery's Niche

December 29, 2004

Welcome to the last XML-Deviant of 2004. Hauling myself up out of last week's excess, I find there have been several interesting developments in XML over the last fortnight. The Christmas season tends to yield more than average productivity, as programmers hide in darkened rooms to avoid boisterous family.

Before we get into the main reports, there are a couple of things I'd like to remind you of. First, as chair of the XTech 2005 conference, I want to point out to potential speakers that there's only just over a week left before the call for participation closes. We're looking for speakers on topics covering XML, the web, open standards, and open data.

Second, don't forget that I intend to give out another selection of "Anti-Awards" for the most ludicrous happenings in XML over the last year. Nominations should be sent to me

XQuery, What Is It Good For?

At the beginning of December, Roger Costello posed a question that has given rise to a thread that has lasted all month.

1. The capabilities provided by XQuery is a subset of the capabilities provided by XSLT/XPath 2.0

2. The XQuery syntax is a hybrid--it has some XML characteristics but is not XML.

Norm Walsh notes: "It's a non-queryable, non-transformable subset of XSLT 2.0".

Given this, I am wondering what niche XQuery is expecting to fill?

It was a timely moment to ask such a thing, as over the course of this year lines of division have been forming over the question of XSLT 2 vs XQuery. As key implementations emerge, such as Microsoft's, XQuery has been welcomed by those who are mentally incompatible with XSLT's declarative nature. At the same time, XSLT devotees just can't see what the fuss is about, perceiving XQuery as a custom syntax for what XSLT can already do.

Jonathan Robie, who given his long experience with XQuery should be well-placed to answer, gave a couple of answers to Costello:

The first niche is programming with XML. XQuery makes programming with XML easier because it directly supports the things that matter most for XML--construction, path expressions, and restructuring with FLWOR.

The second niche is data integration, where the ability to translate XQuery into other systems is very important.

FLWOR figures prominently in the thread and is worth an explanation. It is an acronym for the sequence for, let, where, order by, return. Elliotte Rusty Harold has a pretty illustration of this. In essence, Robie's FLWOR point is a microcosm of the declarative versus procedural contrast in mindset between XSLT and XQuery.

In a later post, Robie highlights the difference between XQuery and XSLT as a matter of design intent:

XSLT was designed for the transformations used in formatting applications, which is why we speak of XSLT stylesheets. XQuery was designed as a query language but also to handle many of the more 'XML programming' oriented scenarios found in our use cases.

But is it true that XSLT can't do FLWOR, the feature of XQuery that allows restructuring of the queried XML? Burak Emir thought so, but Michael Kay begged to differ:

Actually, all the components of a FLWOR expression are there in XSLT, and in Saxon I had no trouble compiling a FLWOR expression to the internal representation of XSLT instructions....

You can argue that the equivalent syntax in XSLT is very cumbersome, but that only becomes a problem if you're doing heavy joins, and I very rarely see applications that need that: in XML 90% of the relationships tend to be within the hierarchy, so you use the XPath axes rather than value-based joins.

That being said, perhaps XQuery's peculiar advantage is where the XML sources are more disparate than typically we find with XSLT. Mike Champion suggested this, and Liam Quin agreed:

Doing a join between data in multiple databases, especially where the databases are of very different types, used to require proprietary middleware.

I wrote [using XQuery] a keyword search for my Web site that queries over a relational database, a bunch of RDF, a (filebased) geographical database and a bunch of XML documents....

Strictly speaking it could also have been done in XSLT, but there's a danger that the XSLT implementation would have had to examine my entire database (e.g. if I used apply-templates) and a lower chance that it would have made use of an index... in other words, even if the languages are very similar in coverage, the focus of implementations is likely to be very different.

Whenever a conversation like the current one starts, it's tempting to let it degenerate into a polarized argument, and an extended discussion about the merits or otherwise of an XML syntax ensued. This seems like a bit of an irrelevance to me, and it is highly plausible that XSLT and XQuery will coexist for the foreseeable future. Why can't we get along, asks Jonathan Robie:

I'm surprised by the continuing length of this thread. Isn't it obvious that both XSLT and XQuery have advantages, a reason to exist, and a market?

Not everyone is as convinced as Robie by the massive vendor investment in XQuery. Just because something's big and expensive, that doesn't mean it will succeed. Tim Bray:

No, for XQuery that's not obvious at all. It seems plausible, and it will certainly be interesting to find out, but obvious?

To conclude, let's go back to Microsoft. It seems to me at least that Microsoft's implementation of XQuery is one of the major factors in its adoption. So what does Dare Obasanjo make of it?

At Microsoft we felt it [XQuery] targetted the needs of people who were tired of the limitations of XPath and wanted the power of XSLT but couldn't deal with the verbose syntax or the need for recursive thinking [with regard to] templates.

Pretty unequivocal, then. But what of Obasanjo's personal experience?

Of course, XQuery got so complex that now I personally would rather stick to XSLT. For the average developer I'd suggest using something like X#/Xen/C-Omega or E4X before I'd suggest XQuery.

Obasanjo's mention of C-Omega--a variant of C# with syntactic support for XML--and E4X--a similar extension for ECMAScript-- returns us to what I think is the most interesting question about programming with XML or relational databases: Whether integrating the data models into the language will be persuasively superior to calling out into the little languages of XQuery or SQL.

Births, Deaths, and Marriages

The latest announcements from the XML-DEV mailing list.

Amara XML Toolkit 0.9.0

This offering from Uche Ogbuji seems to offer much promise as a Python XML toolkit. He describes Amara as "Not just tools that happen to be written in Python, but tools built from the ground up to use Python idioms and take advantage of the many advantages of Python over other programming languages."

Fighting talk. So what's in the box? "The components of Amara are:

Bindery: data binding tool (fancy way of saying: a very Pythonic XML API)
Scimitar: implementation of the ISO Schematron schema language for XML; converts Schematron files to Python scripts
domtools: set of tools to augment Python DOMs
saxtools: set of tools to make SAX easier to use in Python."

Definitely sounds worth a look for Python programmers: open source and under the same license as Ogbuji's 4Suite.

Free XSL-FO Debugger from Altsoft

A much-needed aid for those writing XSL-FO stylesheets. As Altsoft's blurb says, "Why are these nested blocks not present on where they should be? What happened to that line? Where on earth is the header? Is it my error or I should send a letter to the producer of my formatter?" Free, but not open source. Sadly, Windows .NET platform only. Maybe we can get a Mono port?

draft-rsalz-qname-urn

A new Internet Draft from Rich Salz and David Orchard, which defines a Uniform Resource Name namespace for XML namespace-qualified names, QNames. "As long as the URN is encoded in the same character set as the document containing the original QName, the QName URN provides enough information to maintain the semantics, and optionally the exact syntax, of the original name."

W3C Issues XInclude 1.0 as a W3C Recommendation

It's been four years, but now XML has #include.

Architecture of the World Wide Web, Volume One

The first installment of the W3C Technical Architecture Group's description of the web as it really ought to be. Volume Two due shortly after the seventh Harry Potter book.

W3C Charters Patents and Standards Interest Group

A "forum for W3C Members and Invited Experts to discuss policy issues regarding the implementation of the W3C Patent Policy as well as new patent-related questions that arise which require action or attention from the W3C Membership."

Scrapings

Never underestimate the reach of John Cowan ... cart, meet horse: attribute use ill-advised because WXS can't handle them well ... when the XML gravy train runs out of town, there's always a career in retail ... c'mon Roger, haven't you heard of OASIS CIQ TC UPU CEFACT CCTS UBL TC BSCM NIST HL7 RIM CAM SCM XSD RELAX XSLT CC BIE ABIE ACC JCAM BPSS UN/CEFACT ARTEMIS?