XML Inter-Application Protocols

October 13, 1999

Edd Dumbill

In his recent speech given in Tokyo (and reproduced as "Where the Web Leads Us" on XML.com), Tim O'Reilly made the point that stand alone computer software is no longer where all the action is. Instead, web sites are becoming the focus for exciting new applications of computing technology.

Upon the foundation of established, largely open-source, software that drives the Internet, innovative companies (EBay, Amazon, E*Trade and so on) are adding layers of new applications.

While each of these applications stands by itself, the next generation of online applications will need to interoperate, as users are going to want to bolt different applications together.

In fact, the survival of online applications depends on their ability to interoperate. If they don't do it, then somebody else will, stealing their business. The success of PC office application suites demonstrates the attractiveness of an "under-one-umbrella" solution.

An evolution of the Web

The great thing about the Web is that it is hypertext—you can link from one place to another simply by clicking on a link. At present, the extent of that functionality means that you are able to move from viewing one document to viewing another document.

As users do more on the Web—interacting with applications rather than simply reading documents—that connectivity needs to become more than just a simple link.

Take, as an example, an online scheduling web application. If I'm going to use that seriously, it needs to become my daily "portal," just like my diary is at the moment—every note, to-do, reminder, and thing-of-interest should be in there.

As it stands, the only things that get into my web diary application are those I put there. I'm signed up with various email alerts, and read several web sites daily to watch for news. Currently, I have to process that information separately, making notes of URLs to visit later, software to download, things to do.

What is needed is integration. I want an entry automatically put on my to-do list when a new version of some software I use is released. When an article I'm interested in is published, I'd like to know about that too—all in one place, my diary portal.

This kind of integration goes beyond the ability of one vendor—sure, you can get email, diarying, and file storage into one web site, but site publishers need to start cooperating with other site publishers in order to minimize these limitations.

XML as the means of communication

It probably won't come as much of a surprise when I say that XML is in a great position to form these links between online application providers.

The features of XML are just right for this kind of development:

• Extensible: We can start with simple data interchange protocols and evolve in a (backwardly) compatible manner.

• Human readable: Interchange protocols can be made very easy for others to understand and implement.

• Internationalized: XML is designed with international communication in mind. Some internalization issues come pre-solved by the use of Unicode and XML.

The above factors enable a key aspect of development: speed. In the Internet market, one's ability to get out a product quickly is very important. And simple ideas and protocols catch on more quickly than complex ones, despite their possible technical inferiority.

Required foundations

Assuming online application providers agree to interact using XML, there's yet more to define. A common "carrier" needs to be agreed upon, in the same way the Web has agreed on HTTP.

Granted, protocols for interoperation in different areas exist: IMAP, POP, NNTP, WebDAV, etc. However, these protocols are disparate, and have relatively high implementation costs.

A carrier protocol will provide much of the plumbing needed for the construction of application-specific protocols. Examples of such carriers include XML-RPC and SOAP. These protocols don't specify how to represent, say, interaction with a diary. Instead, they define enough to allow a standard way of exchanging data and activating remote functions.

Essentially, they enable the developer to worry about his application's API and data formats, rather than his transport protocols.

Once the carrier protocol is settled, then the excitement can start. Providers of applications can expose interfaces for third parties to use via a carrier protocol.

Standards control

Who then is going to control the standard for interacting with online diaries, grocery stores, travel booking services and so on? There's no way it's going to be just the W3C (World Wide Web Consortium).

Standards are generally set in one of the following ways:

• IETF: The IETF is a more open organization than the W3C. Anybody can submit an Internet Draft (a proposal for a standard) and have it become an RFC, providing certain conditions are met (including the existence of two independent implementations of the proposal).
• W3C: The World Wide Web Consortium focuses mainly on the lower level standards—the "plumbing" of the Web—with the widest possible applicability. While they may accept "Notes" on emergent inter-application standards, they are unlikely to do any work on them.
• Consortia: When the issue at hand is big (or specialized) enough, companies will get together to ensure interoperability. Examples of consortia-produced standards so far include ICE and eCo.
• Companies: Market-leaders (or those first-to-market) set standards. Examples of this include Netscape's RSS format and Microsoft's CDF format.
• Communities: Less often, developer communities will act collaboratively to solve a common problem. An example of this is the SAX effort on the XML developers mailing list.

Moving down this list, it is possible to see that there are a number of ways standards can be set, and they vary according to the number of interested parties and how fundamental the standard is. What threatens the continued interoperability of the Web most is the ability of single companies to set standards alone and then abuse their power.

The area most prone to the setting of closed standards by individual companies is the consumer market. In other areas, businesses more often work together out of necessity.

Yet it can be argued that refusing to open up standards and communication interfaces will run counter to the interests of even consumer online applications. There's no way one company could produce all the online applications that a consumer would want to interact with each other—interoperability must come into play sooner or later.

Open up!

The adoption of HTML spread so quickly because the language was completely transparent. The same can be seen to be happening in the world of XML and application provider interaction.

For example, take Netscape's RSS format. This is a format which allows anyone to syndicate headlines from Netscape's site into the My Netscape portal. The key to RSS's success has been its simple format, easy to use and easy to understand. We are now at the stage where most online publications either provide, or are looking to provide, an RSS file of their content.

Online application providers need to open up in the same way that Netscape opened up their portal to interaction with third parties.

Once interfaces are available for interaction, the development community will come into play. Smaller vendors and hackers will find a way of sticking CDNow, E*Trade and Desktop.com together. As a result, each of those applications will receive added value as they become more integrated into the lives of their users.

One example of openness in action with XML can be found in Userland Software. The people at Userland were heavily involved in XML-RPC and have since been making their servers open for developers to use to construct services. The result has been a growing community of people who help test Userland's ideas and software, contribute to it, and provide complementary functionality. Userland has thus earned a high profile despite its small size.

What Userland opened was not the source to its servers or content management system, Frontier, but its ideas, protocols, and bandwidth.

Open source software has provided us with a sound foundation, and given the little guy a chance in the Internet market. Keeping protocols and standards open is vital for continuing innovation in online applications.

Communication is the key

In this new environment, open standards count for more than open source, although open source will still play an essential role. Capturing the imagination of the users, and the developers, is vital. Giving away implementations and examples can only help foster support.

Simplicity, example code, and aids to understanding, play an important part in adoption of new protocols—it's not for nothing that the major software companies now devote a lot of time and effort to free education sites and tools for developers.

A new market is being created: it's the race to get support in the developer community for a protocol or API. For an online applications provider, that will mean their online application is going to be connected up to as large a number of users as possible. Developers watch out! You're going to become ever more targeted.

Computing is becoming more about communication and less about software. The imminent generation of "killer-apps" are all about connecting users with things they want to do in real life. XML and the Web enable that communication.

Summary

In this article, I've demonstrated the ever-increasing importance of inter-application protocols for web applications. The acceptance and openness of these protocols is important for the development of communicating applications—and ultimately for the satisfaction a user derives from web applications.

XML is the new hyperlink: a conduit for integrating applications with each other. It provides a great starting point for open standards. XML-based protocols are not fundamentally superior to existing ones (e.g. CORBA, EDI)—but the human-readability, low barriers to entry, and sheer enthusiasm surrounding XML, give them a huge advantage.

Now is an exciting time to be a developer of online applications, and a fun time to be a user. Let's hope that through collaboration and openness, Internet development will remain innovative and rewarding.