Menu

Extended XQuery for SOA

September 14, 2007

Dino Fancellu and Edmund Gimzewski

In a services-oriented architecture (SOA), a business process is implemented as a web service that programs (orchestrates in SOA terminology) other web services. An orchestrator web service is usually coded in a language outside the XML domain (e.g., Java), and in this context XQuery is used only to query and transform data—not to orchestrate other web services. However, here we show how a few extensions to XQuery give it the additional role of web service orchestrator, allowing this XML-domain-specific language to implement all the steps in a complex SOA processes. This article explains the choice of extensions, outlines their implementation for a specific XQuery processor, and shows how extended XQuery was used to create web services to process complex financial data. While web services created in this way are usable within any SOA, they can also act as the highest-level orchestrators in what some authors refer to as SOA lite. The extensions applied here for XQuery work equally well for XSLT 2.0.

Why orchestrate with XQuery?

XQuery (See References 1 and 2) has the potential to become the most applicable language for extracting and transforming data from any source that can be represented as a real or virtual XML document. Its SQL-like syntax is relatively easy to learn and it already has a role in SOA for extracting and transforming data. In its unextended state, XQuery has some capability to orchestrate web services via the XPath 2.0 function doc(uri), which can call the uri using HTTP GET. But XQuery was designed to be extensible by custom functions, which can call code modules external to the processor. Most XQuery processors offer their own API for extension functions and but there is also a pending standardized Java API for XQuery (XQJ) (See reference 3). As we shall show, with a few relatively simple extensions, XQuery can take on the additional role of web service orchestrator, allowing a single XQuery script to implement all the steps in a complex SOA processes. Although other languages can do all this, it is simpler and more efficient to use a XML-domain-specific language like XQuery.

The Relevance for SOA Lite

The label SOA lite has appeared in articles (References 4 and 5), which discuss the levels of sophistication which are possible in a SOA. Lite here refers to a SOA which has all services wrapped in web service interfaces and in which some web services are specially created for orchestration. Therefore, SOA lite need not implement the many so-called governance services, such as for security, discovery, and testing; neither would it involve using an engine to automatically generate the orchestrator web service from a BPEL document. However, SOA lite has the core features of a full SOA, and extended XQuery is ideally suited to create the orchestrator web services for it.

Extending XQuery

What components are required to create XQuery-based orchestrator web services for use in a SOA? And what functions are needed to extend XQuery for this purpose?

Components

The components are simple enough:

  • A servlet engine
  • An XQuery processor that supports custom extension functions (not to be confused with user-defined functions in unextended XQuery).
  • A servlet that calls the XQuery processor (referred to later as the XQuery servlet).
  • An XQuery file, with an extension of, e.g., .xqx.

With these components, the sequence of events is as follows:

  • A client application posts an XML message to an .xqx url at the servlet engine.
  • The servlet engine invokes the XQuery servlet, passing it the url of the XQuery file and the posted XML.
  • The XQuery servlet adds any headers and parameters to the posted XML and invokes the XQuery processor, setting the augmented XML as the context document.
  • The output from the XQuery processor is streamed back to the client application.

Extension Use Cases

The following use cases for extending XQuery are derived from our experience in processing complex data such as structured finance deals and company financial statements.

  • Call another web service. This is the essential use case and, as mentioned, is available using HTTP GET in unextended XQuery. But the facility for HTTP POST, and to set HTTP headers, allows XML messages to be sent to any web service, and this raises XQuery to a new level of usefulness.
  • Access HTTP headers from the received XML. For example, if the XQuery is wrapped as a SOAP web service, any SOAPAction header should be extractable. Or the received XML may contain a cookie if it originates from a browser.
  • XSchema validation of XML received or sent. Validating XML, received or sent by the orchestrator, using an XSchema is generally a very sensitive test of data structure, data types and referential integrity. This is not equivalent to schema-aware XQuery (SA-XQuery), which assumes the document is validated and uses the in-memory validated instance to offer safer and better-optimized processing. The option to use SA-XQuery is independent of the use of XSchema validation, but if SA-XQuery is required, it makes sense to check first that the received XML will pass validation. It should be noted that XSchema is not suitable for rule-based validation; for example: "for a deal whose type is variable subordination the tranche section must include a time series of points whose detachment value must exceed the attachment value." However, unextended XQuery is ideally suited for this.

Specific Implementation

Our components are Tomcat 5.5 as servlet engine and Saxonica SA 8.9 as XQuery processor. This XQuery processor supports custom extension functions and was easily integrated into a servlet; it also supports XSchema validation, as well as SA-XQuery; and its own library of extension functions includes one for indexing a sequence, a feature equivalent to <xsl:key> in XSLT, which can improve performance massively.

Extensions

Based on the above use cases, the main extensions allow for HTTP POST, getting and setting headers, and XSchema validation. These are best illustrated by some code. Code Sample 1 is the WSDL for a simplified SOAP web service whose input message is a deal structure and whose output defines the status of validating and saving that deal. The message structures are defined in an imported schema. Code Sample 2 shows the important parts of an extended XQuery, which implements the service as defined in the WSDL. The tasks performed are: XSchema validation (extended XQuery); collecting lookup data from another SOAP web service (extended XQuery); rule-based validation and creation of a database-compatible XML structure by user-defined functions (standard XQuery); posting the database-compatible XML to a SOAP service, which updates tables (extended XQuery). These calls are commented in Code Sample 2.

Real-World Example

XML is ideally suited to represent the complex hierarchical data in a structured finance deal. These deals come in many varieties, which are distinguished not just by their structure but by many rules. We were tasked to create web services to manage deals in our database and the client applications were to be a browser, an Excel-add-in or other server. All the web services were created using extended XQuery, and the most complex one performed these tasks

  • Calling an authentication web service
  • Validating the posted data with XSchema
  • Calling several database web services to get the data required for the rule-based validation
  • Applying approximately 30 rules
  • Transforming the final deal into an XML structure to post to a web service for saving the deal to the database
  • Transforming the final deal into an XML structure to post to a SOAP web service for pricing the deal

We had previously implemented a simpler version of the above validation process using a more traditional approach. But using extended XQuery there was less code, it was easier to debug, and it performed better.

Conclusion

We have extended XQuery to give it a new role as web service orchestrator, so that complex web services, involving validation and orchestration, can be implemented entirely in this XML-domain-specific language. Extended XQuery, as applied to the processing of structured finance deals, has greatly simplified the code engineering and given very good performance. We see extended XQuery as suitable for SOA lite or as part of any SOA. These extensions work identically for XSLT 2.0.

References

  1. W3C XML Query (XQuery) at http://www.w3.org/XML/Query/
  2. XQuery by Priscilla Walmsley,O'Relly, March 2007.
  3. XQuery API for JavaTM (XQJ) at http://jcp.org/en/home/index
  4. Denis Sosnoski at http://www.parleys.com/display/PARLEYS/SOA lite
  5. Aberdeen Group at http://www.aberdeen.com/summary/report/benchmark/RA_IT_ESB_PK_3170.asp

Acknowledgment

The authors thank Michael Kay of Saxonica for his very prompt and helpful responses to queries on the Saxonica extension API.