TAXI to the Future
March 14, 2001
Network Mapping and XML History
Since mid-1999, I've been working on network mapping and visualization; as a side-effect, I had to name it ("Visual Net") and found a company -- there's a public showcase for this stuff at map.net.
The Visual Net technology is cool and 3-D and cyberspacey; but from the viewpoint of a web developer the interesting part is the architecture, which is unlike that of traditional web applications. Architectures need names so you can talk about them: I've been calling this one TAXI, and I've spoken about it at several conferences, but this is the first time I've written about it.
There's not much new about TAXI. I'll claim that if you polled the original group of a dozen or so people, led by Jon Bosak, that defined XML 1.0, you'd find out that something like TAXI was what most of us had in mind. As we all know, XML has mostly been used in backend and middleware data interchange, not in front of the user the way its designers intended and the way TAXI does it. It's long past time for the TAXI model to catch on.
A Brief History of Application Architectures
Gen 1
Many years ago, when I was getting into this profession, dinosaurs stomped the earth; they were called mainframes, and they were at the center of so-called serious computing applications. The architecture of these Generation 1 applications is illustrated below.
In Gen 1, all the code and data lived on the mainframe (often in plain files, before the RDBMS took over the world). People who used this kind of system sat in front of a box, a block-mode terminal, of the IBM 3270 family. Today there are still lots of mainframes talking to lots of what they think are 3270's but are actually PCs running emulators -- I see these applications every time I go to my bank.
A block-mode terminal worked something like this: the mainframe sent it a batch of data, and it displayed a screenful to the user; in most cases, this would be a form with fields you could fill in and tab back and forth between. When you were ready, you'd hit the Enter key, and the terminal would transmit what you'd entered back to the mainframe, which would do some computing and then send you another screenful.
Gen 1 had a lot of advantages. Most important, all the expensive code and data were in one place, in one back room with a locked door, managed by professionals. Of course, it also had some big disadvantages. Most important: all the power and control were in one place, i one back room with a locked door, controlled by professionals -- the kind of professional that appears regularly in Dilbert as "Mordac the Preventer of IT".
There were some secondary disadvantages like no graphics or mouse; niceties like color and upper-lower case text were often seen as optional extras.
You might ask how the computer and terminals sent data back and forth, but you wouldn't get an answer; that was managed in a black box by extremely expensive "communications controllers".
Gen 2
Gen 1 ended with the more-or-less simultaneous arrival of the PC and the RDBMS. Generation 2 was the classic client-server application, illustrated below.
People quickly became addicted to having a their own personal computer; they could print their own documents and do their own budgets without having to go through Mordac the Preventer. After a brief rearguard action, the profession rallied around and built systems where the majority of the business code ran on the PC, and the mainframe-database tandem served as a large, fast, reliable data store.
Gen 2 had the obvious advantages of bringing color, graphics, and local interactivity to end users. It had some pretty severe costs, though: the software, instead of having to run on just one tightly-controlled box, had to be shipped all over the company and was at the mercy of whatever hare-brained (or not) user might have misconfigured his or her personal computer. The total cost of ownership (remember, this was the before the days of sub-$1,000 computers) was pretty high, and the system had a lot of failure points. But there was no going back.
Gen 3
Another problem was that distributing all the business logic to the desktop didn't work very well. Some parts of it -- the collaborative part, where the work of different people had to be coordinated -- really needed to be on the server. The result was the rise of multi-tier application architectures, illustrated below.
This was a lot like pure client-server but with some of the application code, as well as the database, running on the server. Of course, multi-tier systems suffered from the same cost-of-ownership problems that had plagued pure client-server.
Both client-server and multi-tier applications were like old Gen 1 apps in that nobody ever really concerned themselves with what the protocol was between the client and the server; that was private application business, not something that ordinary customers needed to bother their pretty little heads about.
Gen X
Multi-tier (often called n-tier) application architectures were about the state of the art in the early 1990s when the Web came along and rewrote the rules. Let's refer to the web generation as "X", since it was the early 90's. The general architecture is illustrated below.
The crucial new thing is that the whole system is defined by what is sent back and forth between client and server, rather than what the client and server actually are. Specifically, client and server exchange messages using the HTTP protocol; from client to server, the content is a URL, and from server to client, the content is a lot of HTML-encoded text, mixed up with some multimedia in PNG or JPEG or WAV or whatever; HTTP took care of labeling the formats so the client knew what it was getting.
The difference was magic: as long as the client was prepared to deal with a HTML and a bit of multimedia, the server could become truly a black box; nobody cared how much code or how many layers were running on it. Even more important, as long as your desktop box was running a reasonably modern browser, nobody cared whether it was a PC or a Macintosh, and which version.
There was some initial resistance to the web revolution from Mordac and his gang, largely because web interfaces tended to be simple, have very few controls, and all look about the same, removing their ability to express their personalities by writing VB applications with dozens of funny-shaped buttons and bizarre screen layouts, that took weeks-long training courses to learn how to use. But the users voted with their feet for the browser and we got to about where we are now.
But the Web Isn't Perfect
In some important ways, this architecture is remarkably like the Gen 1 mainframe setup. All the code and logic (except for HTML and bitmap rendering) is living on the server in a locked-up back room. To interact with an application typically requires that the user fill out some form fields, send it off to the server, and wait for the server to send another screenful back.
This shares some of the advantages (and problems) of those older architectures but also highlights an important new problem: performance. People have simply become accustomed to the near-instant performance of their word processor and spreadsheet, running at their fingertips on a processor quite a bit faster than those Gen 1 mainframes. The Web, which requires a round-trip to the server and back across the network for any interaction with the system, is slow in comparison.
Let me make a strong statement: not only is the architecture slow, it probably just can't be made fast. Fatter pipes won't do it. Gigabit routers won't do it. Just like the freeways feeding a city, the traffic on a network expands to fill the available capacity, and in the big picture, there is no way you'll ever make a remote application hiding behind a web server feel as fast as if the work were happening on your desktop. I've been working actively with the Web since 1994, and to this day I've never seen a web application that I could describe as fast.
This is serious; whenever you have people sitting around in front of a machine waiting for it to react, you're practicing bad management, because computers are cheap and people are expensive.
An Example: Visual Net
At this time, if you have a browser handy, you might want to visit my company's showcase site; here is a 2-D map that we can use as a starting point for our examples. It's a map of the top-level Arts category. View the source; what you see will depend on whether you're using Internet Explorer or Netscape, but it will be 2-D vector graphics encoded in XML: VML on IE, SVG on Netscape.
Then, if you're using an XML-capable browser, go to this version. Finally, if you don't mind a bit of waiting and plug-in downloading, dive deep into the map and click anywhere on it to get a 3-D view.
Here's how it works.
This is a lot like the traditional Web architecture; the server is a black box (with in this case a lot of map-database code in it). As with the Web, the defining principle is the data formats, only we're exchanging mostly XML instead of HTML -- the 2-D and 3-D graphics rendering is done by the client. Unlike the traditional Web, there's application code (specifically, the map renderers) running right on the client.
This is an example of the TAXI architecture at work.
TAXI: Transform, Aggregate, send XML, Interact
My claim is that TAXI delivers many of the benefits, and hardly any of the problems, of the previous generations of application architecture discussed above. Let's walk through it.
Transform
A lot of business logic boils down to one kind of data transformation or another: applying transactions, generating reports, updating master files. The right place to do most of this work is on the server, where you can assume a rich, high-powered computing environment, with infrastructure including:
- The ability to deal with large data objects
- Support for safe concurrent access
- Indexing facilities for finding and fetching data
- Transaction semantics for safe updates
- A multi-layer environment with rich middleware tools
Concretely, you expect to be able to run your application code, issue SQL commands, run XSLT transformations, and use templates to generate output as with PHP or ASP -- and that's just getting started on the acronyms.
Aggregate
The next architectural principle is the aggregation of enough data from around the server to support some interaction with the user. An example would be a list of airplane flights that could be sorted and filtered or (as with Map.net) a description of a scene that can be rendered.
This is where a lot of judgement at application design time has to come into play; you want to send enough information to support some real interaction but not so much as to overwhelm the browser or the desktop machine that it's running on.
Send XML
Once you've gathered an appropriate amount of data together on the server side, you
encode
it in XML and send it off to the client over HTTP. There's no need to get fancy; we
generate
XML using printf statements in C code.
If you're fortunate, there'll be a well-established XML vocabulary available that someone else invented for use in your application. But probably not, and you'll have to invent your own. We did. Now here's a key point: once you've invented your vocabulary, document it! Because once you've got an XML-based application interface that runs over HTTP, and you've documented the XML vocabulary, you've invented an API. Yes, you could dress it up with additional layers like XML-RPC or SOAP, and that might be a good idea, but there's really not that much need; an HTTP-XML interface is one of the easiest things in the world to do application integration with.
|
|
How do you document an XML vocabulary? A DTD is a good start; working your way through one will help clarify your thinking. And if you're one of the special people who understand the W3C XML Schema system, you might want to use that too.
More important, you need to create lots of examples of your sample XML output and make them available.
Most important, you need to create some well-written human-readable documentation explaining what the tags and attributes mean and what goes inside them. Once you've done this, you've provided an interface that any reasonably-competent programmer in the world can deal with.
One key thing not to forget: invent a namespace for your XML vocabulary and declare it at the top of each chunk you send; this will allow other people to extend your interface without breaking your software.
Interact
Once the XML has arrived in the client, probably a Web browser, you'll need to parse it. Your browser probably has this built-in; it may be more convenient to compile in Expat or Xerces or one of the other excellent processors out there.
Based on our experience in building Visual Net, these are the things you're going to need to build a compelling TAXI-architecture client:
- You'll want to have the parsed XML loaded into memory, ideally with a W3C DOM API to get at it.
- You'll want to be able to get access to the browser's HTML and graphics rendering machinery, so your application appears to be a natural part of the interface.
- You'll need access to user input events.
- You'll probably need to call out from your application code back to the server to pick up more data on demand.
All of these things can be done now in today's browsers at an immense cost from building different versions for each revision of each browser. If they all supported the W3C DOM, stylesheets, and HTML processing in the same way, life would be immensely simpler; something that I and my colleagues at the Web Standards Project have been saying for years.
Why TAXI is a Good Idea
First, it comes at the user through the browser, something that they've proved they want. Second, the application can run faster and scale bigger than traditional web applications in which the server does all the work. Third, the system is defined from the interfaces out, so nobody can lock it up, and you can switch your black-box clients and servers around with little difficulty or breakage.
TAXI will win for the same reason that the multi-tier architecture swept away its predecessors; people, once they've had a taste of big systems with the responsiveness of a personal computer, just won't go back.
I can speak only for myself. I've been building Web applications since 1994, and having gone the TAXI route once, I'm certainly not going back.