TAG: Managing the Complex Web

January 23, 2002

The Web Needs a Technical Architecture Group

The Web's evolution as a technical and social, altogether human institution has meant a growth in its complexity. While many of its fundamental technical principles remain simple -- though some, perhaps, only superficially so, for example, the URI -- a certain measure of complexity is necessary in order to make the Web rich and comprehensively useful.

The W3C, for all its warts and stumbles, has played a vital role toward the end of managing the Web's maturation and necessary increase in complexity. One way, then, to think about the W3C's recommendations is as field markers, as a means of setting out the bounds within which the Web may here complexify or there simplify itself.

However, as the number of W3C recommendations grew, covering increasingly difficult concepts, the number of interactions between them, and between them and other standards and conventions (e.g. IETF RFCs, informal best practices, even social and legal constraints and values), created a situation which seemed unmanageable. The W3C Director's job is to take the long, overarching view, but the magnitude of the Web's complexity threatened to overwhelm Tim Berners-Lee, as it would any individual.

In response to these general pressures, and likely to its own internal pressures to which we aren't privy, the W3C recently chartered the Technical Architecture Group (TAG), which is intended to have une vue d'ensemble, an overview of the whole messy thing.

The TAG, according to its charter, has been asked "to document and build consensus around principles of Web architecture and to interpret and clarify these principles when necessary... [to] resolve issues involving general Web architectural [sic] brought to the TAG, and help coordinate cross-technology architecture [sic] developments inside and outside the W3C". Members of the group include the W3C's Director, pro forma, and Tim Bray (Antarcti.ca), Dan Connolly (W3C), Paul Cotton (Microsoft), Roy Fielding (Ebuilt), Chris Lilley (W3C), David Orchard (BEA Systems), Norm Walsh (Sun), and Stuart Williams (Hewlett-Packard).

Media Types, Namespaces, and XML Content

The TAG's charter offers little information as to the actual working process it will employ. The minutes of a recent TAG meeting indicate that it will adopt the following process, about which more below:

The TAG decides whether it will add an issue to its long-term agenda, whether it will resolve the issue "on the spot", or whether it will refuse to consider the issue.
If the TAG accepts the issue for its long-term agenda, the issue receives a unique identifier and is logged on the TAG's public (archived) mailing list.
One or more TAG participants will prepare for each issue and will present to the rest of the group.
When an issue is resolved, the TAG should explain how the resolution will be manifested (e.g., a document update, an Architecture Recommendation, etc.)

Not long after its first meeting, held on 7 January 2002, the TAG was approached by Mark Baker, for the XML Protocol Working Group, who asked some interesting questions about the adoption and management of MIME (the relevant standards of which are RFCs 2045, 2046, 2047, 2048, and 2049) media types in W3C Working Groups:

What are the general guidelines or policies (if any) for W3C working groups in defining their own media types? Should they be defining them at all?
If custom media types are required, should there be any commonality between them?
What is, or what should be, the relationship between a media type and an XML namespace?

Baker is, perhaps, the inevitable person to ask these questions, given his participation in RFC 3023 "XML Media Types" (which established five media types: text/xml, application/xml, text/xml-external-parsed-entity, application/xml-external-parsed-entity, and application/xml-dtd) and registration of the application/xhtml+xml media type. RFC 3023 also established the +xml suffix convention, which is to be used when creating media types that represent XML media types other than the five that RFC 3023 explicitly established. Baker subsequently clarified his second question; it was meant, he said, to determine whether, in a WG-created media type, RFC 3023's +xml convention and the obligatory parts of RFC 3023's Section 7.1 "Referencing" were to be observed.

The justification RFC 3023 offers for the +xml suffix convention is that it "will allow applications that can process XML generically to detect that the MIME entity is supposed to be an XML document, verify this assumption by invoking some XML processor, and then process the XML document accordingly".

Perhaps it is Baker's third question, the one about media types and namespaces, that is the most intriguing, especially given the now well-rehearsed technical debates about what a namespace is, its relation to a URI, and the like. Again, upon subsequent clarification, it became clear that Baker was asking whether a specific XML media type could be declared by joining a namespace declaration with a general XML media type. In other words, whether it would be useful to declare that some entity is XHTML, not by using application/xhtml+xml, but, rather, in disregard of RFC 3023's +xml suffix convention, by saying (something like) application/xml; xmlns="http://www.w3.org/1999/xhtml.

The issues Baker raises are not merely matters of XML semantics, addressing the way XML content is transported across the Internet, the degree to which that content is or should be self-describing, how or whether MIME can continue to be used to describe some or all of that content, where XML content should describe itself, and so on.

David Orchard quickly suggested that RFC 3023's +xml convention is fundamentally broken.

The way I see it, media types are broken for multiple namespace'd xml documents, especially documents that are targeted to be frameworks like soap. The "+" syntax for media-types simply doesn't scale to these kinds of documents...

Norm Walsh suggested that the XML document itself should be used as the basis of subsequent dispatch, which is one of the virtues of XML's self-describing property.

It seems to me that I might as well say text/xml and let the receiver figure it out from the namespace URIs (XML is self-describing for just this reason, no?). About the only useful distinction I can see is a flag to indicate that the document only uses a single namespace (so that I know from the root element what namespace its in).

Clearly there are problems with this approach, but I'm not sure that having specific MIME types really solves any of them in the general case of mixed namespace documents.

There was some disagreement over whether the root namespace should be given some measure of precedence in determining what "type" the document is and, thus, how it should be dispatched to a handling mechanism or processor. Mark Nottingham took a contrarian position, saying that

Namespaces provide namespaces, nothing more. They do *not* identify the "type" of document. In some cases, they can be used like this, but the conditions under which this is true have to be explicitly specified.

Using the top-level namespace to identify a document's application is tempting, but doesn't always prove useful. Media types and namespaces are used for different purposes, in different manners, and should not be thought of as a one-to-one mapping.

Tim Berners-Lee came down on the side of those who think that the root element's namespace has precedence: "the sort of document you have is for an XML document defined fundamentally by the namespace of the outermost element". For example, Berners-Lee said, "the meaning of a document whose outermost element is RDF is the semantics of the RDF statements defined by the RDF spec and the specs of the properties used in the outermost level of the RDF document".

He further suggested that for "widely adopted" W3C recommendations which define a namespace that will be used on the root element of XML instances that "it is wise but not essential to make a special MIME type" and that RFC 3023's conventions and requirements should be observed thereby.

Paul Prescod, however, noted that some extant W3C standards do not comply with this general rule, most notably the XSLT specification, from which he offered the example of an XSLT stylesheet with an XSLT namespace and an XHTML Strict default namespace on its root element. This throws a kink into the general rule that the root namespace should always be given precedence; as Prescod says, "the question is, should the MIME type be inferred from the root namespace, some list of namespaces, or completely independent? ... people want to sniff the namespace (either from the MIME header or the XML itself) and then *choose* an XML processor based on the namespace. XSLT stylesheets with an HTML root-element are one example where this would not work".

Tim Bray offered a few practical issues to take account of.

[Y]ou're creeping in the direction of deprecating media type MIME headers in the case where the resource body is XML...I'm a bit nervous -- speaking as a person who's written a *lot* of in-the-web-server code, I know that in the case where something is RDF or XHTML or SVG, I'd rather be able to dispatch to the appropriate handler right there in the server on the basis of the headers than have to wake up the general-purpose XML handler to figure out the dispatching. XML parsers are reasonably cheap to instantiate, but "reasonably cheap" can become very expensive in the context of a high-transaction-volume server, relative to the almost invisible cost of reading/parsing a few headers.

Also bear in mind that it's kind of technically tricky to get an XML processor to read just the start-tag of the root element but not go on to process the whole document -- and since your friendly local SVG/RDF processor probably includes its own XML parser, you're going to end up piping the whole resource body through two XML processors. Aren't media type headers a better solution?

Simon St. Laurent, who published an XML.com piece last year about these issues, said that he "suspect[s] that it may be time to move past two-part MIME Content-Type identifiers entirely, and start moving toward identification approaches which are capable of supporting the diversity XML makes possible". St. Laurent, not one to avoid putting his money where his mouth is, rather quickly published an Internet Draft, " Registration of xmlns Media Feature Tag", which uses the Content-features header, per RFC 2506 and RFC 2912, to specify a finely-grained description of content. So, to use one of St. Laurent's examples, to describe an XHTML document which contains SVG, MathML, SMIL, and XLink content, you might say

Content-Type: application/xhtml+xml Content-features: (xmlns="http//www.w3.org/1999/xhtml") Content-features: (xmlns="http//www.w3.org/2000/svg") Content-features: (xmlns="http//www.w3.org/1998/Math/MathML") Content-features: (xmlns="http//www.w3.org/2001/SMIL20/") Content-features: (xmlns="http//www.w3.org/1999/xlink")

Whether TAG or IETF adopts this approach remains to be seen. Credit is due to St. Laurent, however, for making concrete suggestions and moving the discussion forward.

Bray offered an early look at TAG consensus-in-formation about as to each of Baker's original questions.

Yes, in general you SHOULD create a media type for a new XML language, unless you are prepared to argue the case why you're not.
Yes, in general you SHOULD follow the recommendations of RFC3023, unless you are prepared to argue the case why you're not.
We don't have enough consensus in place yet to make an architectural finding about the proper relationship between media types and namespaces, except to note that it is highly appropriate for software to dispatch control based on namespaces, and that documents with mixed namespaces present problems, some still unsolved, of the proper way to dispatch control and integrate results.

Also in XML-Deviant

The More Things Change

As part of Bray's responsibility as the TAG "owner" of the architectural issue raised by Baker's third question -- the relation of media types and XML namespaces -- Bray offered a few additional bits of TAG's initial consensus and lack thereof.

P1. Media types are an important part of the web architecture; dispatching on them, when possible, is efficient and robust and well-understood.

P2. When processing XML resources, dispatching to software modules on the basis of namespaces is desirable and correct behavior.

P3. Clearly in many cases context matters: you can't in the general case reach into the middle of a resource and safely process some element based only on its namespace.

P4. The namespace of the root element of an XML resource has a special status, if only because it provides the outermost level of context.

...

C2. Namespace processing obviously becomes more relevant in the case where the resource is served as text/xml or application/xml. There is currently no consensus as to whether or when it's desirable to serve resources with either of these media types.

C3. The issue has been raised of whether MIME headers or media types are useful in signaling the makeup of XML resources which contain markup from multiple namespaces. There's no consensus on this issue.

TAG and the Future of Web Evolution

The issues raised by Mark Baker and the XML Protocol Working Group are interesting and non-trivial in their own right. What's equally interesting, however, in addition to the substantive technical content, is the way TAG is working through issues in this relatively early period. The TAG mailing list, for example, has so far mostly been a place for non-TAG members to express their opinions, ask their questions, and make their arguments about TAG issues -- none of whom, as near as I can tell, have yet been formally invited to serve as expert advisers to TAG itself, but, rather, participate as an expression of their interest, both commercial and personal, in helping to resolve Web architectural issues.

While the initial three issues that TAG has agreed to address might seem XML-specific, they touch on HTTP and other transport mechanisms, as well as on MIME, which is not a Web or HTTP-specific technology. The early signs of Working Group readiness to consult TAG are very welcome. The Web is far too complicated for any one person to hold it all in their head at once. We may also be seeing early indications that even a group as technically capable as the TAG will need to cultivate a wide range of technical input in order for it to reach sound decisions about the way in which the Web's overall architecture will continue to evolve.