Using XPath with SOAP

September 16, 2003

Massimiliano Bigatti

XPath is a language for addressing parts of an XML document, used most commonly by XSLT. There are various APIs for processing XPath. For the purposes of this article I will use the open source Jaxen API. Jaxen is a Java XPath engine that supports many XML parsing APIs, such as SAX, DOM4J, and DOM. It also supports namespaces, variables, and functions.

XPath is useful when you need to extract some information from an XML document, such as a SOAP message, without building a complete parser using JAXM (Java API for XML Messaging) or JAX-RPC (Java API for XML-Based RPC). Moreover, the loosely-coupled nature of web services suggests that the use of dynamic data extraction is sometimes better than using static proxies like the ones produced using JAX-RPC.

In the article I'll show a JAXM Web Service for calculating statistics and a generic JAXM client that uses the service, demonstrating the use of XPath for generic data extraction.

Introducing Jaxen

The Jaxen library implements the XPath specification on the Java Platform. Jaxen supports different XML object models, including DOM4J, JDOM, W3C DOM, and Mind Electric's EXML. It supports so many object models by abstracting the XML document using the XML Infoset specification, which provides a representation of XML documents using abstract "information items".

The Jaxen library includes several packages:

• org.jaxen -- Core API;
• org.jaxen.expr.* -- Support for XPath expressions;
• org.jaxen.function.* -- Support for XPath functions;
• org.jaxen.pattern -- Support for XSLT Pattern objects, and
• org.jaxen.saxpath.* -- Event based on parsing and handling for XPath expressions
• org.jaxen.util - Utility objects;

In addition, the Jaxen library features adapters for the different XML object models:

• org.jaxen.dom - W3C DOM object model;
• org.jaxen.dom4j - DOM4J object model, and
• org.jaxen.jdom - JDom object model.

For example, when an XPath expression is evaluated against an initial context, which is typically a Document node, Jaxen uses the correct Document class related to the object model in use (that is, org.dom4j.Document, org.jdom.Document, org.w3c.dom.Document).

Resolving Expressions

Resolving an XPath expression against a XML document requires the use of the main interface in Jaxen, org.jaxen.XPath. This interface defines several methods, the most useful of which are

• Object evaluate (Object context)
• List selectNodes (Object context)
• Object selectSingleNode (Object context)

The return values of these methods are generic types; at compile time, Jaxen doesn't know what kind of XML object model will be used. Further, the evaluate() method can return multiple results in the form of java.util.List objects based on the XPath expression and the contents of the current context. Note that Jaxen doesn't make copies of the returned nodes but merely references them. The contexts supported by these methods are documents, elements, or a set of elements.

To use an XPath expression in Jaxen you must first create a concrete XPath object by passing the expression to the constructor. Then it's possible to run the expression against the context you want. For each XML object model supported, a concrete XPath class is provided:

• org.jaxen.dom4j.Dom4jXPath
• org.jaxen.jdom.JDOMXPath
• org.jaxen.dom.DOMXPath

For example, the following code runs an expression against a DOM Document:

import org.jaxen.*;
import org.jaxen.dom.*;

org.w3c.dom.Document document = createDocument();

String query = "//journal/article[2]";

XPath xpath = new DOMXPath(query);

List results = (List)xpath.selectNodes(document);

The procedure is simple. First you create a DOMXPath object, then you evaluate the expression using one of the methods described above.

Using Namespaces

Namespace support in Jaxen is implemented by the NamespaceContext interface and the SimpleNamespaceContext class. The latter provides a container for a set of namespace definitions, formed by the prefix and the URI. The interface defines one single method used to resolve a specific prefix to the related URI:

public String translateNamespacePrefixToUri(String prefix)

To assign a SimpleNamespaceContext to an XPath object, use the setNamespaceContext() method. For example,

XPath xpath = new DOMXPath( query );

SimpleNamespaceContext ns = new SimpleNamespaceContext();
ns.addNamespaces("ns1", "http://namespaces.bigatti.it");

xpath.setNamespaceContext(ns);

It is important, when dealing with namespaced XML documents, to define those namespaces correctly, in order to be able to obtain all the nodes from the document.

Creating a Test Service

To be able to test the generic JAXM client I'm going to create, I developed a simple test JAXM Web Service that computes some statistics on an input sequence of float numbers. It calculates the mean, standard deviation, coefficient of variation, minimum, maximum; it also produces in output the recognized (correctly parsed) numbers in the input string. An example of a response is showed in the following figure.

The presence of several parts of information in the response will allow us to test different XPath expressions in the client, pointing at different places in the returned XML.

The MathServlet servlet ( MathServlet.java) is a simple JAXM servlet (it extends JAXMServlet) used to implement the service. It also implements ReqRespListener because this is a request-response service:

package it.bigatti.soap;

//imports...

import javax.xml.messaging.*;
import javax.xml.soap.*;

public class MathServlet extends JAXMServlet
                    implements ReqRespListener {
    //...
}

Each SOAP request is serviced by the onMessage() method, which implements a simple traversal of the content searching for the values parameter embedded in the //Envelope/Body/Calculate node. For simplicity, the service does not strictly check the conformance of the request.

public SOAPMessage onMessage(SOAPMessage message) {
    SOAPMessage response = null;
    String values = null;

    try {
        message.writeTo(System.out);

        SOAPPart sp = message.getSOAPPart();
        SOAPEnvelope env = sp.getEnvelope();
        SOAPHeader hdr = env.getHeader();
        SOAPBody bdy = env.getBody();

        Iterator ii = bdy.getChildElements();
        while (ii.hasNext()) {

            SOAPElement e = (SOAPElement)ii.next();
            Iterator kk = e.getChildElements();

            while (kk.hasNext()) {

                SOAPElement ee = (SOAPElement)kk.next();
                String name = ee.getElementName().getLocalName();

                if( name != null && name.equals("values") ) {
                    values = ee.getValue();

                    System.out.println("values = " + values);
                    break;
                }
            }
        }

        if (values != null) {
            response = createResponse(new MathSupport(values));
            response.writeTo(System.out);
        }

    } catch(Exception e) {
        e.printStackTrace(); 
    }

    return response;
}

If input values are found, the servlet returns a SOAPMessage provided by the createResponse() method. Here is an excerpt of that method, showing the JAXM code required to create a SOAPMessage and the SOAPElements needed to contain the response.

protected SOAPMessage createResponse(MathSupport ms)
        throws SOAPException {
    MessageFactory mf = MessageFactory.newInstance();
    SOAPMessage msg = mf.createMessage();
    SOAPPart sp = msg.getSOAPPart();
    SOAPEnvelope env = sp.getEnvelope();
    SOAPHeader hdr = env.getHeader();
    SOAPBody bdy = env.getBody();

    String xsi = "http://www.w3.org/2001/XMLSchema-instance";
    env.addNamespaceDeclaration("xsi", xsi); 
    env.addNamespaceDeclaration("xsd",
        "http://www.w3.org/2001/XMLSchema");
    env.addNamespaceDeclaration("soapenc",
        "http://schemas.xmlsoap.org/soap/encoding/");
    env.setEncodingStyle("http://schemas.xmlsoap.org/soap/encoding/");

    Name xsiTypeString = env.createName("type", "xsi", xsi);
    
    SOAPBodyElement gltp = bdy.addBodyElement(
        env.createName("CalculateResponse",
            "ns1", "http://namespaces.bigatti.it")
    );

    SOAPElement e1 = gltp.addChildElement(
        env.createName("Summary")
    );

    SOAPElement e2 = e1.addChildElement(
            env.createName("Mean")
        ).addTextNode("" + ms.getMean() );
    e2.addAttribute( xsiTypeString, "xsd:float" );

    e2 = e1.addChildElement(
            env.createName("StandardDeviation")
        ).addTextNode("" + ms.getStandardDeviation() );
    e2.addAttribute( xsiTypeString, "xsd:float" );

    //... other code

    return msg;
}

The createResponse() method uses a MathSupport object that performs the statistical calculations. Complete source code is provided at the bottom of this article. You'll also find an Ant script for building the service along with a list of libraries required.

Coding the Client

While the web service uses the Sun reference implementation of the JAXM API, which is included in the JWSDP 1.1 package, the client uses the Apache Axis implementation. This is due to the fact that JAXM embeds DOM4J, which in turn embeds Jaxen. The two versions of the Jaxen library had some incompatibilities; therefore, to be able to experiment with the version consistent with the documentation available on the Jaxen web site, I chose to use the JAXM implementation offered by the Axis project.

The SOAPClient class ( SOAPClient.java) implements the client. The invoke() method performs the SOAP call and stores the answer both in SOAPMessage form and as a parsed XML tree.

public Object invoke() throws SOAPException, 
        SAXException, IOException {
    SOAPElement element;
    document = null;
    buffer = null;

    MessageFactory mf = MessageFactory.newInstance();
    SOAPMessage msg = mf.createMessage(); 
    SOAPPart sp = msg.getSOAPPart(); 
    SOAPEnvelope env = sp.getEnvelope(); 
    SOAPHeader hdr = env.getHeader(); 
    SOAPBody bdy = env.getBody();

    env.setEncodingStyle("http://schemas.xmlsoap.org/soap/encoding/");

    SOAPBodyElement gltp = bdy.addBodyElement( 
        env.createName(operation, "m", operationURI)); 

    Iterator param = parameters.keySet().iterator();
    while(param.hasNext()) {
        String name = (String)param.next();
        String value = (String)parameters.get(name);

        element = gltp.addChildElement( 
                env.createName(name)
            ).addTextNode(value);
    }

    if (soapAction != null) {
        MimeHeaders mh = msg.getMimeHeaders();
        mh.setHeader("SOAPAction",
            "\"" + soapAction + "\"");
        msg.saveChanges();
    }

    URLEndpoint endpoint = new URLEndpoint(serviceURI); 

    SOAPConnectionFactory scf = SOAPConnectionFactory.newInstance(); 
    SOAPConnection conn = scf.createConnection();

    response = conn.call (msg, endpoint);

    if (response != null) {
        buffer = messageToString(response);
        document = builder.parse(new ByteArrayInputStream(buffer.getBytes()));
    }

    return buffer;
}

When the client needs to extract a node or a list of nodes using an XPath expression, an XPath object is created, using the createXPath() method shown below. This method alsos extracts the namespaces defined in the SOAP Envelope element and in the first body child, which contains the SOAP operation being called. These namespaces are then associated with the XPath object. This operation is required to be able to address nodes from a particular namespace, such as the SOAP Envelope and Body elements.

XPath createXPath(String query) throws SOAPException, JaxenException {
    //Uses DOM to XPath mapping
    XPath xpath = new DOMXPath(query);

    //Define a namespaces used in response
    SimpleNamespaceContext nsContext = new SimpleNamespaceContext();

    SOAPPart sp = response.getSOAPPart();
    SOAPEnvelope env = sp.getEnvelope();
    SOAPBody bdy = env.getBody();
    
    //Add namespaces from SOAP envelope
    addNamespaces(nsContext, env);

    //Add namespaces of top body element
    Iterator bodyElements = bdy.getChildElements();
    while(bodyElements.hasNext()) {
        SOAPElement element = (SOAPElement)bodyElements.next();
        addNamespaces(nsContext, element);
    }

    xpath.setNamespaceContext( nsContext );
    return xpath;
}

The createXPath() method relies on addNamespaces(), which performs the actual namespace addition on the SimpleNamespaceContext, and is shown below. Note that uncommenting the println trace will show the prefixes found in the particular SOAP response.

void addNamespaces(SimpleNamespaceContext context,
        SOAPElement element) {
    Iterator namespaces = element.getNamespacePrefixes();

    while(namespaces.hasNext()) {
        String prefix = (String)namespaces.next();
        String uri = element.getNamespaceURI(prefix);

        context.addNamespace( prefix, uri );
        //System.out.println( "prefix " + prefix + " " + uri );
    }
}

Once the XPath object is in place, with all namespaces defined, you can evaluate the expression against the SOAP response. The SOAPClient class implements several methods to get response element value:

• String getValue(String query)
• Map getValuesAsMap(String query)
• List getValuesAsList(String query)

The first method is useful when extracting a single node value, but when dealing with muliple return values, a set is needed. The SOAPClient class provides two methods of this kind, the first returns a Map, where the key is the element name, and the value is the element value; the second one returns a List, containing only the element values. The getValuesAsMap is implemented as shown:

public Map getValuesAsMap( String query ) throws SOAPException,
        JaxenException, IllegalArgumentException,
        SAXException, IOException {

    XPath xpath = createXPath( query );
    List results = (List)xpath.selectNodes( document );

    Map result = new HashMap();
    Iterator iter = results.iterator();

    while(iter.hasNext()) {
        Object element = iter.next();

        if (element instanceof org.w3c.dom.Node) {
            org.w3c.dom.Node node = (org.w3c.dom.Node)element;
            result.put(node.getNodeName(), nodeContent(node));
        }
    }

    return result;
}

Running the Client

The SOAPClient class includes a main() method that contains a test call to the service, located on the same machine. The steps required to perform the invocation are

SOAPClient client = new SOAPClient( serviceURI );

client.setOperationData("calculate",
    "http://namespaces.bigatti.it");
client.addParameter("values", "1 3 5 7 9 11 13 17");
String result = (String)client.invoke();

After the call, the response is in memory and you can perform your XPath queries. The following table shows a list of XPath expressions and related result values used as a test.

Downloading the Source Code

The full source code is available here. Notice that the full libraries required (JAXM, JAX-RPC, Axis and Jaxen) are not provided. They can be downloaded from the web sites mention in the Resources section below. The example uses JWSDP 1.1 JAXM and SAAJ APIs and reference implementations. The generic client uses Axis (which is JAXM complaint) and the Jaxen library.

Resources

• Homepage of the Jaxen project;
• Official SUN JAXM page;
• Apache Axis project;
• W3C XML Infoset specifications;
• W3C XPath specifications;