Far from Patchy Progress
December 5, 2001
During a particularly quiet spell on the XML-DEV mailing list, the XML-Deviant takes some time out to visit the Apache XML project and report back on current developments.
Past Successes
The last time the Deviant reported on current events at the Apache XML project was in July 2000, during the turbulent beginnings of the Xerces 2 parser that saw the open source community coming to terms with with working alongside commercial development teams at Sun and IBM. After this initially rocky start, the community rallied around the task of building a next generation parser with the result that its third beta release was announced just over a month ago.
During the same period the Apache XML umbrella project has itself significantly expanded, adding a range of new projects. These were all formed from outside donations, arising from a mixture of commercial and non-commercial sources. In the latter cases the project has provided a forum in which competing projects can combine forces.
These new projects include Batik, the SVG toolkit, Apache SOAP, and the recently contributed XML Security implementation. As Kimbro Staken noted in his XML.com article last week, this list will soon include a native XML database implementation, Xindice, that will evolve from the donated code of the dbXML project. The Apache XML Project Management Committee (PMC) is just now going through the final logistics of getting this project off the ground.
It seems as if its integration skills are about to be put to the test once more with the impending donation of a further commercial project, the WASP Lite SOAP toolkit.
Stung Into Action
Anne Thomas Manes of Systinet recently proposed a poll on the Apache XML general discussion list to submit the Wasp Lite toolkit, a entry level Web services framework, as an Apache sub-project:
The product is currently distributed under a free commercial binary license. It has a large user base, many of whom have requested that we make the source available. A number of companies and individuals have expressed interest in contributing to the development of this code.
...We would like to submit this proposal to the members of the Apache XML project for consideration of accepting this donation as a sub-project.
Immediate questions were raised about viability, particularly the expected level of commitment from Systinet and the likelihood of a community forming around the framework. Manes responded by noting that, as the entry level product in their portfolio, Systinet would continue to donate development time to moving the project forward, and she expressed the desire to see some convergence between the current SOAP projects running at Apache.
This is an interesting approach as it means that the community can potentially benefit from a free open source tool backed by ongoing commercial support, while Systinet can also reap the benefits of a free workforce that will contribute to the ongoing development of it's product line. While not a new strategy -- the IBM and Sun donations follow a similar approach -- for a growing company it has some obvious benefits.
While the potential donation was warmly received, Stefano Mazzocchi asked a couple of hard questions:
...we already have two projects which have a pretty high overlap (SOAP and Axis). Why do we need another one? Can't this "WASP lite" be donated as directly to Axis instead of creating yet another project on webservices?
Mazzocchi makes reference here to several existing projects. Apache SOAP formed from the IBM donated SOAP4J. Axis is a ground-up implementation of a SOAP framework aimed at superseding Apache SOAP. Sam Ruby summarized partial histories of these projects during the discussion, including the fact that WASP actually grew out of a second, separate effort to create a replacement for Apache SOAP.
The addition of WASP Lite would mean a third SOAP implementation to the Apache framework. Seriously concerned about the prospect of divergent efforts, and referencing the previous problems with the competing Crimson/Xerces development, Mazzocchi noted that donators should:
...please keep in mind that it's up to the apache communities to decide what to do with donated code.
If you decide to donate it in order to communities to benefit, it should not be your concern where the code ends up living, but should be the community's itself.
Fortunately the various parties involved were able to conclude that there was a common desire to move forward the Axis project by taking the best ideas of both the existing and the freshly donated WASP Lite source. In fact WASP Lite may have already realized the modular architecture which Axis is working toward. During the discussion, Sanjiva Weerawarana suggested that an appropriate home for the project would be under a new web services effort, i.e. webservices.apache.org. However the political overheads in creating such an infrastructure were viewed as prohibitive, and the final conclusion seems to be to reorganize the Apache SOAP, Axis, and WASP Lite projects under a single web services sub-project. Subsequent promotion of this sub-project to a separate full-scale project, complete with it's own Project Management Committee, may still take place in the future once the web services field stabilizes.
The addition of WASP isn't the only impending donation that may cause some restructuring at the Apache XML project. Norman Walsh has been talking to Stefano Mazzocchi about donating an entity resolver implementation. This might precipitate the formation of a separate xml-commons project that could have a similar charter to the Jakarta commons project. While making a natural home for these kind of utility classes, it would also help to further factor out aspects of the some of the existing projects (and potentially reduce the installation footprint for users who have several Apache tools installed).
Recurring Patterns
|
|
|
Also in XML-Deviant |
|
The continuing evolution of Apache SOAP toolkits highlights a recurring pattern of code development, one that follows an old adage first documented by Fred Brooks in The Mythical Man Month: "Plan to throw the first version away. You will anyway."
Each of the major projects has seen active development, then been forked to produce a next generation deliverable. The most recent example is the release of Cocoon 2, a complete rewrite of the Cocoon framework. There may be a hint of a secondary pattern here: do the first implementation using DOM, then re-write using SAX for performance. Both Cocoon 2 and Axis has followed this secondary approach. The recent XML-DEV discussion on the differing accessibilities of the two APIs (summarized in a recent XML-Deviant column) might partially explain this pattern. Or perhaps it's just easy to prototype using a document object model because it's The Simplest Thing That Can Possibly Work.
An unfortunate side effect of these ongoing project refactorings is that the number of different, overlapping projects can become dizzying. Providing a useful roadmap, including historical background, to each project would be a useful addition to the Apache XML documentation, as Ben Coman observed during the WASP Lite discussion. Stefano Mazzocchi supported this request suggesting that the documentation on the site be beefed up to help guide users directly to the tools they need. This would be a much more worthy addition than a site new look-and-feel the community is currently prototyping.
In sum, the Apache XML community has a proven track record of organizing widescale development based on donated codebases, and it has obviously had to learn a few lessons about coordinating a diverse range of development teams that often have competing goals and priorities. The community appears to be going from strength to strength with the addition of new tools making Apache XML nearly a one-stop-shop for XML developers.