XML.com's Year in Review

December 22, 1999

Edd Dumbill

Xtraordinary Times

There can be no doubt that we are living in extraordinary times. This year has seen the increasing pervasiveness of web technology, the seemingly unlimited financial support of dot-com startups, and the appearance of "globalization" in the national consciousness. Mixed in with all this is the incredible success of XML as a technology—its rapid acceptance by software vendors who credit it with near-talisman qualities.

The energy associated with XML has breathed life into hitherto niche areas of interest, spawning an unusually collaborative developer community and leading vendors to unprecedented levels of cooperation. Readers of XML.com will not need any reminder of the attention garnered by XML this year. The recent XML'99 show saw a sharp rise in the number of attendees, and a significant shift from document-centric to e-business-centric products and vendors.

In this article I will attempt to document some of the more important events of the last year, and preview areas of activity for the coming year.

Plumbing

1999 has seen a lot of the necessary "plumbing" being put in place by the W3C. Namespaces, XPath, and XSLT—all of which form key components of the XML architecture—have reached recommendation stage. While each has enjoyed controversies along the way, their establishment as base XML technologies is enabling the rapid development of further specifications.

In a sense, XML is also a victim of its own success. Although the above-mentioned specifications have been recommended, we have also seen disappointing delays on the XML Linking and XML Schema specifications this year. An XML query language seems even more distant. The W3C has plainly been stretched as it attempts to build the framework for XML. It is to their credit that they have preferred getting things right to delivering them early.

However, XML schemas and an XML query language must be delivered as early as possible during year 2000. The most asked-for tools from the business programming community, these will enable better use of structured data deep inside organizations than ever before.

Community

"Openness of information" characterizes the community in which XML specifications and applications evolve. The XML-DEV mailing list, started in 1997, has continued to play a vital role this year in the development of XML. Input from list members does not go unnoticed by the W3C. Freer from the political constraints inherent in a consortium such as the W3C, members of the XML-DEV list have created new XML initiatives. They've also developed a useful acceptance test for new XML specifications.

The XML-DEV list is best known for its collaborative development of SAX. As the year ends, work is underway on the development of SAX2, which will provide a standard interface for parsers and support applications using additional layers of XML (Namespaces, DOM2, etc.).

Many other online XML communities have built up this year, generally centered on particular languages or platform technologies. Although XML-DEV typically gets the spotlight, the work conducted on the XSL-List this year should not be overlooked. At least three XSLT implementations were being developed at the same time as the XSLT specification; these provided useful feedback in the specification process.

In recent months the W3C has moved to formalize such implementation work, introducing a "Candidate Recommendation" phase in which implementation feedback is specifically requested. DOM2 is currently at this stage, and the XML Schema work is expected to reach the same stage soon. This move echoes the IETF's policy of requiring two independent implementations before allowing a draft to become an RFC, and should hopefully ensure that fewer specifications get lost along the way.

Tool Support

XML-DEV is by no means the beginning and end to XML communities. In fact, at 1500 members, it is a relatively small community. Add to this the fact that there seem to be at most 30 to 50 people on XML-DEV who really know XML well, and you get a more accurate picture of where XML stands with programmers.

The reality is that most programmers, constrained by deadlines and pragmatism, will be using the APIs and tools created by the XML experts. The ability to either harness or trivialize the power of XML lies with the tool vendors, either open source or commercial. This is one reason why the participation of Sun and IBM in the Apache XML Project is very welcome. In effect, they have made their core XML technologies accountable to a large, independent group. The pressure on vendors to conform to open standards is increasing, and to try and capture market based on proprietary XML base technology is at best counterproductive.

Yet, in these early days, programmers need to beware the XML hype. As a supporter of XML, I'm very glad it is getting all the attention it is. As an advocate for developers, however, I'm disappointed in some of the marketing hype for XML products. Frequently "XML-enabled" means at best the presence of an XML import/export facility. Some vendors are simply adding XML to their bullet list of features and re-releasing software under the banner of XML tool support.

But to be truly XML-enabled means much more than an XML import/export facility. It means a deep, embedded knowledge of the structure and idiom of XML. It means the concepts of documents, document fragments, and links can be used in the application. It means that the powerful notions of interoperability, portability, and transformation are not lost as soon as the XML document gets inside the product.

At this point you will observe that not all of the technologies needed to make XML-enabled tools are W3C recommendations yet. The current level of XML support in tools is only a start. Next year will see a maturing of both specifications and products. If you're buying XML tools now, be aware of their current limitations.

It would be wrong to paint the situation as gloomy, however. There are some genuinely useful and innovative XML tools available. Just be aware that these are the early days, and things will change a lot over the next few years. Year 2000 will be an experimental period for XML implementations. As the analyst panel at this year's XML'99 said, it will be messy, you will fail, but you will learn a lot through it. Don't expect your first XML rollout to be perfect. And don't expect perfection from vendors either.

Developer Education

One key to extracting the maximum value from XML is developer education. The aim would not be to introduce newcomers to arcane syntaxes and pull them into religious XML arguments. Rather, it would be to make the best use of the frameworks provided by SAX, DOM and the other XML technologies.

What is XML in syntax may not be XML in spirit. Simply wrapping < > around field names doesn't bring all the joys of structured data. The beauty of XML is that those who only go this far still get something. But many, attracted by the hype, have come to XML to see only angle brackets. Their disappointment is understandable.

As with any technology, developers need to understand the XML idiom in order to utilize it best. What makes a good DTD? Does it matter whether I use elements or attributes? What about character encodings? All these questions must be answered satisfactorily. The open source movement may have provided some excellent tools for XML, but it has not provided satisfactory instruction. I hope that over the next year the quality and availability of XML training will increase.

Again, it is worth pointing out the power held by those who decide the APIs for XML processing. It is via this route that most developers will come upon XML (i.e., through their use of Oracle, Informix, MS SQL, Perl etc.). The creators of these APIs have a responsibility to the XML community at large to present and utilize standard XML in order to preserve interoperability. At the basic XML processing level, this is not much of a problem. Elements and attributes are pretty much atomic, and it's hard to break their cross-implementation compatibility. But consider the application level: a library to process, say, purchase orders must resolve links within a document. Does it use W3C XML linking, or does it use a proprietary method? Who cares? Not the developer using the Acme Purchase Order API. However, functionality that exists as a standard may have been removed from the document definition and placed in a proprietary API, immediately limiting the portability of the document. Users beware! Not everything XML is open.

Pervasive XML

Throughout this year, XML libraries for various platforms have been in development. The major database vendors are adding XML capability. Libraries exist for most programming languages. Web browsers are adding XML awareness. Yet mainly these XML facilities exist as an add-on to existing products. The true XML revolution will happen when XML handling facilities are an integral part of language and platform distributions.

When XML is that ubiquitous, the real excitement can start. Most developers are in the business of enabling productivity. Built-in XML capabilities enable a major increase in interoperability and communication between previously isolated applications. The area where most vendors are currently recognizing this is in so-called e-business. XML heralds the dawn of low-cost electronic commerce between businesses, enabling many smaller enterprises to participate where costs were previously prohibitive.

XML will also power the revolution of increasing integration between web, desktop, and consumer electronic devices. As the variety of devices used to interact increases, so will end the purely proprietary data formats.

XHTML, the reformulation of HTML 4.0 in XML, looks as though it will become a recommendation in the near future. With it can start the clean-up of the Web, and one hopes, the ultimate resolution of browser incompatibilities. That goal may have to wait several years, however! What will be enabled ultimately is a "cleaner," modular way of creating web documents, with standard ways of including fragments from other XML vocabularies and of indicating which browser features a document requires.

Long before the Web cleans up its act, we will have been overtaken by a plethora of access methods that don't involve a personal computer at all. Access via TV, PDAs and mobile phones is already on the increase. For content developers looking to target every medium, XML will prove indispensable. Every development platform bearing an "XML-inside" badge will be of great value in the new multi-device environment.

The Semantic Marketplace

Of course, pervasive XML means that there will be a certain level of agreement on basic XML technology. That is the low-level stuff defined by the W3C, IETF, and initiatives like SAX. The new arena for interoperability will be the XML applications, one layer up from the W3C work. The W3C does get involved in some XML applications particular to the web, such as SVG and XHTML, but in doing so restricts their scope.

Peter Murray-Rust, founder of XML-DEV and inventor of CML, the Chemical Markup Language.

It is at this application layer that organizations such as OASIS and the GCA are playing a larger part. In his keynote speech at XML'99, Peter Murray-Rust warned of the fragmentation that could occur if commercial interests could not agree on standard XML vocabularies in application areas. He saw independent organizations like OASIS as vital to regulating that "semantic marketplace."

It would be surprising if the same level of openness we have experienced with the XML core technologies was maintained in all these application areas. Why? Because there is one level of added value here, a greater advantage to being the first to bring a product into a vertical market. Take as an example Vignette's close involvement with ICE. For a long time their server product was the only viable ICE implementation—a clear benefit. Although the W3C is trying to get closer to developers, I expect that other consortia will act more behind closed doors, favoring their members before releasing their specifications. Some of this is a plain economic reality, and does not necessarily represent any conspiracy. The very fact that such diverse players are working together is a testament to the significance of XML in electronic business. Preferably, however, such developments would be conducted under the auspices of trusted independent bodies, such as OASIS.

Taking a Harder Look at Data

How will XML affect the job of the developer in the coming year? Is it as easy as "plug and play"? In terms of understanding data, XML demands more from organizations than they may recognize at first. In order to speak a common language, you must first understand your own. Looking deeply into the structure (or otherwise) of her company's data, the developer may well find it difficult to describe.

It is not uncommon for companies to have multiple notations, standards, and ways of storing the same things. Whether via acquisition, evolution, or poor organization, such replication is frequent. The good news about XML is that, in addition to being useful in inter-company scenarios, it offers great promise as a platform from which to standardize internal documents and data formats. The bad news is that this is a tough problem.

Next year will belong to the information architects, the librarians, and the data-miners. Those who can determine the structure of their organization's information and work to translate it to interoperable formats. The obvious business motivation is to enable e-commerce with other companies; but if the same spirit is turned inward, an organization will find the ability to extract more value from its data.

Taken seriously, the deployment of XML inside an organization will be revolutionary. The investment will be large, and the rewards long term. Yet the conversion of data to a standardized, structured form will yield dividends. In the same way that HTML/CGI interfaces enabled new value to be obtained from legacy terminal-based applications, XML application gateways will enable a business to leverage legacy data and applications, bringing them into an e-business framework.

The problems are hard, but you have to start somewhere. Bite off as much as you can chew, turn it into XML, explore the new possibilities given you, and learn from the experience. But be aware that a serious analytical approach is required in order to get the most out of XML within a company. No tool from any vendor can teach you about your own data.

Conclusions

In summary, 1999 has been a great year for XML, and 2000 should be even greater. The marketing success of XML has been incredible; it seems all you need now to reach IPO is a cool DTD. Next year we'll hit the reality barrier as more real-world XML applications appear. The theory in place, practical considerations will come to the fore and be resolved. Those who have accepted XML will need to look seriously at its impact on their entire organization.

What's on my personal wish-list for the new year? Well, as I indicated earlier, I'd like to see XML support become core to more development platforms. In tandem with that, I'd like to see the same for Unicode, along with an increased awareness of internationalization issues. My other wish would be for mature tool support and better developer education, as these would drive the real-world use of XML for many developers.

In XML.com over the coming year, we'll be looking to support those aims, particularly in providing real-world applications for XML and more practical XML tutorials.