The Courtship of Atom

May 19, 2004

Kendall Grant Clark

We need more process. We've been open, and need to have a decision-making process to make the hard decisions. -- Sam Ruby, 18 May 2004

We've followed Atom -- the community-driven XML vocabulary and protocol API for web resource syndication -- on XML.com since it was first announced. In fact, I suspect that we've published as much about Atom, including columns by Mark Pilgrim and Joe Gregorio, as any other web publication. In this week's XML-Deviant column I talk about the issues surrounding the news that the W3C is considering chartering a Working Group to standardize Atom. As with every great romance, and most notable marriages, it's not entirely clear here who's courting whom.

Atom is an interesting project; it stands in the same relation to XML vocabularies as SAX stands to XML parsing models -- both are successful, well-engineered projects designed by an informal, community-driven processes. And therein lies the problem....

On the Vices and Virtues of Community

First, a word about the virtues of community. SAX and Atom are both insanely successful, especially if judged by the proportion of inputs to outputs. Which is to say that both SAX and Atom are very widely implemented, cost very little to develop, and those costs were paid by a wide range of people and institutions, most if not all of whom freely chose to participate. Plus, and we can never underestimate this aspect, they both have oodles of technical merit. While some people don't care for event-centric APIs, there's no question that SAX is a fundamental element of XML's success. Likewise, while many people prefer RSS 1.0 or RSS 2.0 to Atom, there is little doubt that Atom-the-syntax is a fine piece of engineering.

In sum, community processes can sometimes create good technology cheaply. Hooray for the do-it-ourselves ethic!

And, now, a few words about vices.... The "community-driven" process that has created Atom thus far is really no process at all. It lacks the most crucial feature a standardization process must have in order to be sustainable, namely, a procedurally fair and substantively neutral means of resolving conflict. That means may be formal and rule-driven, or it may be informal and organic; but in the absence of any such means, what you're left with is either viciously ineffective or simply devolves into straight power play. Or both.

And, what's worse, it's extremely hard to retrofit a conflict resolution mechanism because, or so I once suggested to Mark Pilgrim, as soon as the community includes parties with conflicting interests, every process decision has to be made in a contested context. If I want Atom to be, for example, a generic resource monitoring mechanism, and you really want it to be all about syndicating news stories, then we are unlikely to agree to a conflict resolution mechanism once the fur starts flying. I'm not going to be as interested in finding a procedurally fair, substantively neutral mechanism as I am in pushing my agenda and killing yours. You can't turn a multiparty conflictual process into a cooperative process by asking everyone to play fair when you have no non-disputed, shared understanding of what playing fair means.

So the parties not only disagree about the details of the syntax or protocol, for example, but they also disagree about how to resolve that conflict. And whereas before they might only have had first-order reasons for disagreeing -- that is, in this context, mostly technical reasons -- now they also have second-order reasons for disagreeing, that is, they want to further their own interests, retard the interests of their now-competitors -- preferably both.

In sum, community processes can sometimes strangle good technology in its infancy for lack of a way to resolve conflicts. Boo for the do-it-ourselves reality!

Atom Disputes, W3C Processes

The only sane, reasonable, and realistic way to avoid this mess is to agree to the process before it begins, before the social space is structured by mutual conflict and competing interests. Yes, of course, there will be competing interests before the process begins; but what will structure the interactions between parties is an interest in designing a procedurally fair, substantively neutral means of resolving conflicts. In advance, before you know the actual details, who will line up to support whom, what the vectors and lines of loyalty and disloyalty will be, the rational thing to do -- the thing that is most likely to advance rather than harm one's interests -- is to design a process that, first, focuses on technical merit and, second, gives everyone a fair chance at influencing outcomes. And by "everyone", I really mean everyone, all the relevant, interested stakeholders -- from Microsoft to me, from Google to George over there, from Werner Vogels to Dave Winer.

I like to think of this as the technical standards process analogue of John Rawls's Original Position. I won't go into a long explanation of Rawls's views here; I will say, however, that if you want to think seriously about standards-making in terms of politics and fairness, you really must understand what Rawls was up to.

And that's precisely what the W3C offers. Okay, it isn't exactly the same thing as Rawls's Original Position, but it's about as close as we have. The W3C has in place a process for adjudicating and resolving conflict between parties that may have, as a matter of contingent fact, competing interests. It put that process into place, with all the appropriate safeguards and escape hatches, well before any of the present Atom stakeholders realized that their interests didn't always align perfectly. And it did so with the goal of making the Web a better thing. Now, of course, the reality on the ground is a bit different. There have been cases where the W3C was essentially steamrolled by its largest constituents -- can you say "XML Schema" and "RELAX NG"? I knew you could. So W3C process is flawed, not flawless. Every social process is flawed, not flawless. But it's almost incalculably better than no process at all.

Let me mention the features of the process that generates W3C Recommendation that would likely be helpful to the Atom community: consensus-driven decision making; an institutional mandate, albeit a pretty weak one, to improve the Web; incremental, working draft publication schedules; consultation and liaison with IETF and other W3C Working Groups, as well as with the Technical Architecture Group; mandatory acknowledgment of and response to Last Call comments; flexible, open membership, including to public-interest groups; invited, non-member experts on WGs; preservation and review of minority and dissenting technical positions; participation constraints (Microsoft gets no more WG members than anyone else); boundary and scope-setting WG charters.

W3C Expertise and Resources

So, clearly, I think that the W3C's process is the Big Win for Atom. But there are other reasons why taking Atom to the W3C would be a good move, including the W3C's expertise and its resources. The W3C has as much expertise as any standards body doing XML vocabularies, including the hard bits like internationalization, interoperability with other standards and technologies, especially XHTML, and being "good for the Web", which means here not making the Web crash hard.

In addition to process and expertise, the W3C also has a set of resources that fit perfectly with the needs of Atom, at least as I understand them. My guess -- based on observing the W3C carefully for six years -- is that there just is no better setting within which to do distributed standards development than the W3C. From the mailing lists, the teleconferences, the face-to-face meeting habits, to the publication structures, procedures, and reviews, there is very little that goes wanting. I can't imagine the Atom community not finding a productive home within the W3C, at least with regard to resources.

Atom Courting W3C or W3C Courting Atom?

I have assumed so far that the W3C is the lover, Atom the beloved; that the W3C is actively courting Atom more than Atom is courting the W3C. Syndication is important enough -- and technically challenging enough, especially if Werner Vogels is right about aggregation and syndication scalability -- to the future of the Web that the W3C has to try to influence the outcome. Atom has enough brand cachet already that it can't hurt the W3C to play in this space; to say nothing of the fact that some of Atom's biggest corporate backers want a W3C-blessed technology to promote. It seems to be a mutually beneficial match.

Atom is, however, a bit of an odd duck for the W3C, which I have criticized in the past for squatting in specific technical spaces, trying to shut out alternatives. I don't like that aspect of W3C culture very much, but I don't think it will be very harmful in this case. For example, one thing about W3C culture which irks me is the habit of choosing the most generic possible name for its Recommendations. Though it probably hasn't come up yet, I'd be surprised if the Atom community would sit still for a renaming and rebranding. It was too hard finding a name to begin with. In addition, the various flavors of RSS and their assorted partisans will see to it that the W3C does not promote Atom as the one and only solution in this space. In some ways it would make more sense for Atom to have gone to OASIS because the W3C should have already adopted RSS 1.0, if not an earlier version. But that's the history of a possible world different from our own.

Some Technical Details

Process, expertise, resources. Sounds a good match. But what about technical details? I haven't kept up with Atom-the-syntax very closely, but a few things seem obvious. First, Atom-the-protocol seems likely to find a very happy home inside the W3C, given its RESTfulness. I don't care whether it gains a SOAP binding personally, but it won't bother me if it does. Seems overkill, but some people and corporations love SOAP -- <shrug/>.

The RDF Question

Second, I think Atom-the-syntax would gain a lot from a closer relationship with RDF. But this issue has a lot of facets. It's good hear that the W3C would send an Atom spec to Recommendation status without it becoming an RDF application. In my view, syndication is all about saying things about web resources, that is, things with URIs. That's RDF's business and so Atom should be an RDF vocabulary, in my opinion. That's one reason I favor RSS 1.0. That RSS 1.0, thanks to it being an RDF vocabulary, is extensible and decentralized, is the other reason I favor RSS 1.0 over RSS 2.0. Atom faces the same extensibility pressures faced by both RSS and by FOAF. RDF is the ideal response to those pressures -- and one infinitely preferable to RSS 2.0's solution, anyway.

All that having been said, I think, as between any lover and its beloved, wooing is better than pressuring. If, as a result of moving into the W3C, Atom's developers find benefits to adopting the RDF model, than that's all to the good. If not, that's fine, too. I will point out that many people's main objection to RDF, that the canonical XML serialization is painful, is less and less trenchant these days. (See, for example, Mark Pilgrim's "Should Atom Use RDF?".)

Why? Because, first, there are at least six good alternatives to RDF-XML for serializing RDF models: Notation 3 (aka, N3), NTriples, Turtle, TriG (which is, roughly, Turtle++), TriX, and RXR ("Regular XML RDF"); and, second, because at least two of them are "ordinary" XML vocabularies (TriX and RXR) -- amenable to XSLT transformations, for example. That means that you can do lots of RDF, all the RDF you might ever need or want to do, and never produce or consume the canonical RDF-XML serialization. Yay for alternatives!

I suspect that, if there was to be an Atom WG, and were it to adopt the RDF data model and any of these alternative XML syntaxes, that would not only be a good thing for the W3C, but it would benefit RDF and Atom, too. But I know that if an Atom WG were to take a hard look at RDF and its data model, and then to pass politely, it will still be a win for all concerned for Atom to have found a home in the W3C.

The Semantic Web Gamble

Lastly, let me say a word about the Semantic Web and, more crucially, the Semantic Web Activity within the W3C. If there is to be an Atom WG, it will almost certainly be located within the Semantic Web Activity. That should make exactly no one nervous whatsoever. Why not? Because the Semantic Web, finally, is about machine-readable web resources. All RSS feeds, all syndication formats, including Atom, are part of the Semantic Web, at least in my definition.

	Also in XML-Deviant The More Things Change Agile XML Composition Apple Watch Life After Ajax?

Yes, some people want to do Knowledge Representation and logic programming as part of the Semantic Web; and, yes, some of what they want to do is experimental or very hard or maybe impossible. But so what? Some people want to do advanced stuff with FOAF, and so they do it, and my mother can still have her weblog, with her auto-generated FOAF resource, and who really cares that she'll never give a damn about KR? People often say that FOAF is to the Semantic Web what home pages were to the Web. That's wrong; or, rather, it's at best only half right.

FOAF plus Atom (or FOAF plus your favorite RSS flavor) is to the Semantic Web what home pages were to the Web. A machine-readable description of a person, plus a machine-readable version of that person's web space, is enough Semantic Web for us to do really great things, whether or not the hard KR stuff ever amounts to anything at all. Atom has a shot to be FOAF's best friend, the two foundational vocabularies for a new Web. Letting Atom come to life within the W3C gives it a very good chance indeed. But even if I'm all wrong about this Semantic Web stuff, there is a world of good reasons why Atom and the W3C belong together.