Web Service Sublimation

March 20, 2002

Kendall Clark's "All That is Solid Melts into Air" column from earlier this month discusses some recent debates over XML and HTTP and their role in building Web services. Most working definitions of "Web Service" (including ours) mention these and other technologies specifically. If they are not appropriate for building Web Services, we have to wonder what "Web Service" really means. We want to take this, our six-month anniversary column, to address the issue.

XML does not a Web Service make

The XML technology suite (the Infoset, XSD, XML 1.0 + Namespaces, XPath, DOM, SAX, XSLT, XML Query, and so on) is useful because it allows us to model, represent, and manipulate data in a platform, language, and vendor neutral way. If you build systems that represent information in XML, you'll be able to work with your data anywhere. Most developers who understand XML agree that this is a very good thing.

In the broadest possible sense, Web Services are an attempt to use XML to build distributed information processing systems that work across the Internet without necessarily requiring a browser as the client. Many present Web Services as a silver bullet that makes building this sort of system easy, but this view is naive.

Serialized XML messages are easy to parse because the syntactical rules of XML 1.0 + Namespaces are well understood. Once parsed, XML messages are easy to manipulate using a range of technologies. However, while the ability to parse and interpret messages is necessary, it is not sufficient to build a distributed system. There are a lot of other issues that must be resolved. How should messages flow between different parts of the system? Should the messages by typed or untyped? If they are typed, what type system should be used? And should parts of the system be strongly or loosely coupled? Answering these questions is key to deciding what Web Services really are. But deciding on answers is extremely difficult, as recent debates in the XML world have shown.

Going with the flow...

Let's consider four questions, starting with how messages should flow between different parts of a distributed system. It is certainly the case that the original version of SOAP was designed to facilitate remote procedure calls in an Internet friendly way, i.e., by encoding call stacks in XML and sending messages over HTTP. The language of the original SOAP specification made this clear. But the language of the W3C's current SOAP working draft makes it just as clear that SOAP is a messaging protocol based on "fundamentally one-way transmissions from a SOAP sender to a SOAP receiver", which can be used to build systems with any number of different message flows. The specification facilitates the definition of new message flows using an abstraction called a Message Exchange Pattern (MEP). MEPs codify specific message flows. The new SOAP specification defines one MEP for request-response exchanges, which is what is needed to model RPC invocations.

The move away from a specifically RPC-centric model is no accident. Many people believe that a message-oriented model makes more sense for widely distributed systems where connections are intermittent at best. Unfortunately, this leads to a sticky problem, raised, to lively debate, by Don Box at DevWeek in London in late February. While the SOAP specification allows the definition of arbitrary message flows, some of them are quite difficult to implement on top of HTTP. And though the SOAP specification doesn't mandate the use of HTTP, it is crucially important as a transport protocol because the entire Internet infrastructure is designed to support it. But there are some things HTTP is not good at: for example, delivering event notifications to clients or supporting long-lived message exchanges.

There are at least two solutions to this problem. One would be to move away from HTTP altogether. However, this leads to the inevitable question: will the Internet infrastructure support it. Put bluntly, if messages don't go through port 80, is it going to work? Another alternative is to build a virtual protocol on top of HTTP which implements the desired semantics using short-lived client-server HTTP exchanges. Yet another approach would be to retool HTTP to provide the desired behavior. Which ever approach is taken, issues of client addressability would also have to be dealt with. If clients access the Internet via NAT translation, connecting to them from the outside is problematic.

To type or not to type, that is the question?

Once you've worked out how messages are going to flow in a distributed system, you've got to decide what the messages themselves will contain. One way to do that is to define messages in terms of types. The advantage of this approach is that you can build generic plumbing that validates messages as they enter and leave parts of your system to ensure that they are of the right type. This makes your application more robust. If a message containing the wrong information is received, the mistake can be caught automatically. This can also make your system more efficient because a mistake in a message can be caught before the message is sent.

While many people believe in the value of defining strongly typed messages, it is worth noting that this is another topic of raging debate. There are people who like to implement systems using typeless technologies, arguing that this simplifies development. This is a topic of endless discussion in the XML community.

There can be only one

Assuming a distributed system's messages are to be strongly typed, what type system should be used? If the messages are XML, it makes sense to use XML Schema as the type system. XSD allows the definition of problem-domain specific types (as opposed to the generic types defined by the XML Infoset, e.g., element, attribute, text, comment, etc.) and is increasingly well integrated with the other XML technologies. While the use of XSD seems an obvious choice to most people, there is disagreement here, too. The W3C's xml-dist-app mailing list has seen a lot of traffic recently focused on the role of XSD in SOAP.

The original SOAP specification was written before XML Schema was finished, so the authors were forced to define an ad hoc data model for describing what went into SOAP messages. The SOAP data model is defined in terms of generic constructs, specifically: structure, array, and references. Like the Infoset constructs (element, attribute, text, comment, and so on), the SOAP data model constructs do not allow you to define strongly typed messages in a meaningful way. While the SOAP data model defines generic types, it doesn't define a type system. Unfortunately, the SOAP data model is incompatible with the XSD type system, which does not directly support either typed references or arrays.

While many view the SOAP data model as a stop-gap measure that should be discarded now that XML Schema is finished, not everyone agrees with that. If the SOAP data model remains prevalent, it opens the question of how to describe typed XML messages that are based on it. If they described with another technology, like Java, the promise of interoperability based on XML technologies will be lost.

Concerns about coupling

Even if everyone agreed that we should use XSD to describe strongly typed messages, there is still one more problem to address. Should those strongly typed messages define tightly or loosely coupled contracts? This point is both subtle and extremely relevant.

It is often said that XML is advantageous because it is self-describing. More accurately, the syntax of XML is self-describing. That makes it easy to write generic plumbing to parse, query, and manipulate information represented as an XML Infoset. However, the semantics of any particular XML dialect are not self-describing. In many cases, applications are implemented to process specific XML dialects. In some cases, generic processing is possible (e.g., a browser that will display XML on the screen).

While it is syntactically legal to extend a message by adding all sorts of additional, previously unknown content to it, is it reasonable to expect that a software component receiving such a message will process it correctly? The answer? It depends. If the content being added to the message is a whitespace-only text node, a comment, or a processing instruction, it is reasonable to expect that well-written XML processing code will handle the changes gracefully. But what if new, unknown elements or attributes are added to the message?

More from Rich Salz

The answer, in this case, depends on the degree of coupling you want. In a tightly-coupled system, the contract between two parties is very specific -- message formats are strictly limited as to their content. In a loosely-coupled system, the contract between two parties is more flexibly -- message formats are more flexible and extensible. Even with XML, if you want an application to be loosely-coupled, you have to go out your way to implement it that way.

Coupling should not be confused with typing. XSD can help clarify how rigid an XML messaging contract is. If a component in a distributed system describes the messages it is willing to accept using XSD, it can define types that are as open or as closed as desired.

Coupling has been a recent topic on the XML-DEV list, in spirit if not in name. The discussion earlier this month about whether or not XML was really extensible is in fact a discussion of how forgiving applications that rely on XML ought to be about the structure of a message or document. In other words, the debate was over what constitutes an acceptable degree of coupling. Unfortunately, many XML-based applications are actually loosely typed but tightly coupled, which is the worst combination. This is one of the reasons adoption of XSD is a good idea.

Where are we

Where does that leave us? Collectively, while people envision Internet-based distributed information processing applications sending messages based on a wide array of patterns, including but not limited to request-response, it isn't clear how that will be done relative to HTTP as it is defined today. Independent of message flows, people have not yet agreed on whether messages should be typed, which type system to use, and what degree of coupling is acceptable. (For the record, messages should be typed, described in XSD, and as loosely coupled as possible -- as specified in their XSD definitions.)

So what is a Web Service? It is an Internet-friendly distributed application that uses XML. That's about all anyone can say for now.