Web Services Security, Part 2

April 1, 2003

Bilal Siddiqui

In my previous article I discussed the security requirements of web services in B2B integration applications. I also introduced some XML-based security standards from W3C and OASIS.

In this article, I will discuss three XML-based security standards -- XML Signatures, XML Encryption and Web Services Security -- which offer user authentication, message integrity and confidentiality features in SOAP communications. You can safely bet that these three standards fill the SOAP security hole I described previously. In what follows I explain how that hole is filled by demonstrating the creation, exchange, and processing of XML messages inside XML firewalls.

Basic Cryptographic Concepts

The discussion of message integrity, user authentication, and confidentiality employs some core concepts: keys, cryptography, signatures, and certificates. I will briefly discuss cryptographic basics. If you're in further details may refer to the Resources section, which contains a link to a freely downloadable handbook on applied cryptography.

Asymmetric cryptography

A popular cryptographic technique is to use a pair of keys consisting of a public and a private key. First, you use a suitable cryptographic algorithm to generate your public-private key pair. Your public key will be open for use by anyone who wishes to securely communicate with you. You keep your private key confidential and do not give it to anybody. The public key is used to encrypt messages, while the matching private key is used to decrypt them.

In order to send you a confidential message, a person may ask for your public key. He encrypts the message using your public key and sends the encrypted message to you. You use your private key to decrypt the message. No one else will be able to decrypt the message, provided you have kept your private key confidential. This is known as asymmetric encryption. Public-private key pairs are also sometimes known as asymmetric keys.

Symmetric cryptography

There is another encryption method known as symmetric encryption. In symmetric encryption, you use the same key for encryption and decryption. In this case, the key has to be a shared secret between communication parties. The shared secret is referred to as a symmetric key. Symmetric encryption is computationally less expensive than compared to asymmetric encryption. Which is why asymmetric encryption is ordinarily only used to exchange the shared secret. Once both parties know the shared secret, they can use symmetric encryption.

Message digests

Message digests are another concept used in secure communications over the Internet. Digest algorithms are like hashing functions: they consume (digest) data to calculate a hash value, called a message digest. The message digest depends upon the data as well as the digest algorithm. The digest value can be used to verify the integrity of a message; that is, to ensure that the data has not been altered while on its way from the sender to the receiver. The sender sends the message digest value with the message. On receipt of the message, the recipient repeats the digest calculation. If the message has been altered, the digest value will not match and the alteration will be detected.

But what if both the message and its digest value are altered? That kind of change may not be detectable at the recipient end. So a message digest algorithm alone is not enough to ensure message integrity. That's where we need digital signatures.

Digital signatures

Keys are also used to produce and verify digital signatures. You can use a digest algorithm to calculate the digest value of your message and then use your private key to produce a digital signature over the digest value. The recipient of the message first checks the integrity of the hash value by repeating the digest calculation. The recepient then uses your public key to verify the signature. If the digest value has been altered, the signature will not verify at the recipient end. If both the digest value and signature verification steps succeed, you can conclude the following two things:

• The message has not been altered after digest calculation (message integrity); and
• the message is really coming from the owner of the public key (user authentication).

Certificates

In its most basic form a digital certificate is a data structure that holds two bits of information:

• The identification (e.g. name, contact address, etc.) of the certificate owner (a person or an organization); and
• the public key of the certificate owner.

A certificate issuing authority issues certificates to people or organizations. The certificate includes the two essential bits of information, the owner's identity and public key. The certificate issuing authority will also sign the certificate using its own private key; anyone interested party can verify the integrity of the certificate by verifying the signature.

Message Integrity and User Authentication with XML Signatures

The XML Signature specification, XML Digital Signature, (XMLDS) has been jointly developed by W3C and IETF. It has been released as a recommendation by W3C. XML Signature defines the processing rules and syntax to wrap message integrity, message authentication, and user authentication data inside an XML format.

Recall from my previous article the interaction between a vacation tour operator and her partner hotels. Let's assume that the tour operator wants to invoke the GetSpecialDiscountedBookingForPartners method of a partner hotel's web service. This method provides online hotel booking service at special discounted rates. These special discounted rates are only available for trusted business partners and are not meant for the general public.

The tour operator includes message integrity and user authentication information within the GetSpecialDiscountedBookingForPartners SOAP method invocation. The hotel's XML firewall, on receipt of the invocation, will need to look into the SOAP message to verify that:

• The message has not been altered while on its way to the hotel's web service (message integrity); and
• the requester is really a trusted business partner (user authentication).

The XML firewall will only let the request pass onto the SOAP server if both these conditions are met. Figure 1 illustrates the process of user authentication in which the following sequence of events occurs:

1. The tour operator sends a method invocation SOAP request to the hotel's web service. The request includes all the relevant security information (message integrity and user authentication).
2. The hotel's web service is protected by an XML firewall, which receives all requests destined to the SOAP server. The XML firewall checks that the message as received is identical to the one that the requester intended to send.
3. If the message integrity is found to be in order, the XML firewall reads the requester's identification information from the SOAP request and makes sure that the user is really a trusted business partner.
4. If the requester is found to be a trusted business partner, the XML firewall allows the request pass onto the SOAP server.

Listing 1 is a simple SOAP request that carries the GetSpecialDiscountedBookingForPartners method call to the hotel's web service. The SOAP request of Listing 1 does not contain any message integrity or user authentication data. Listing 1 is the starting point to demonstrate XMLDS.

I'm using SOAP as an example XML format to demonstrate XMLDS, which isn't SOAP-specific. XMLDS can be used to insert signatures and message digests into any XML instance, SOAP or otherwise.

The following example will insert XMLDS tags inside the SOAP header. XMLDS is flexible and allows the insertion of XMLDS tags anywhere in an XML file. In fact there are three types of XML signatures: enveloping, enveloped and detached. When an XML signature wraps the data being signed, it is said to be an enveloping signature. If the XML data being signed wraps the signature (i.e. the XML signature becomes an element of the XML data being signed), it is said to be an enveloped signature. If the signature and the data being signed are kept separate -- the element being signed and the signature element are siblings -- it is said to be a detached signature. The XMLDS authoring example presented in this article uses detached signatures.

Four Steps to XML Digital Signature Authoring

The first step is to create a Signature element. The Signature element will eventually wrap all the other XMLDS elements. Have a look at Listing 2, which has exactly the same body as that of Listing 1. The only difference between Listings 1 and 2 is that Listing 2 contains the XMLDS namespace declaration (http://www.w3.org/2000/09/xmldsig#) and a SOAP header. The SOAP header wraps a Signature element.

The Signature element in Listing 2 contains three child elements: SignedInfo, SignatureValue, and KeyInfo.

Listing 2 shows that the Signature element is only a wrapper for other XMLDS tags. In steps 2, 3, and 4, we'll create the child nodes of the three Signature children (SignedInfo, SignatureValue, and KeyInfo).

The second step is to create the child nodes of the SignedInfo element. Listing 3 is the result of inserting the SignedInfo child nodes into Listing 2. The complete SignedInfo structure tells the details of the process that leads to an XML signature. You can notice from Listing 3 that there are several children of the SignedInfo element and each of its children contains some bit of information as explained below.

The CanonicalizationMethod is a required element that identifies the canonicalization algorithm applied to the SignedInfo element before producing the signature.

Canonicalization algorithms are important in XML signature applications because message digest algorithms treat XML data as octet streams. Two different octet streams can represent the same XML resource. For example, if you change the sequence of attributes occurring in an XML element, the resulting XML file will be a logically equivalent version of the original XML file. However the two logically equivalent XML files will contain two different octet streams and will produce different digest values.

Canonicalization algorithms are meant to produce identical octet streams for logically equivalent XML data. In order to make sure that logically equivalent XML documents produce the same digest value (and the same signature), we need to canonicalize our XML resources before digesting their octet streams.

The CanonicalizationMethod element in Listing 3 has an attribute named Algorithm, which has a URI string as value (http://www.w3.org/2001/10/xml-exc-c14n#). This URI string identifies Exclusive XML Canonicalization, an algorithm by the W3C. The details of XML canonicalization are beyond the scope of this article. Please refer to the resources section for a series of articles that discusses XML canonicalization in detail.

At this stage, we have just created the CanonicalizationMethod element. We have not yet applied the canonicalization algorithm to anything. We will apply the canonicalization algorithm to the SignedInfo element after authoring all its children.

The next child of the SignedInfo element in Listing 3 is a SignatureMethod element, whose Algorithm attribute identifies the algorithm that will be used to produce the cryptographic signature.

The third child of the SignedInfo element is a Reference element. There should be at least one Reference element inside a SignedInfo element. The Reference element is used to hold various bits of information as explained below.

1. A reference to the data that is being signed. This is the job of the URI attribute of the Reference element. You may include the data to be signed within the XML document or you may keep it external. If your data and the signature reside within the same XML document, you will refer to it using a fragment identifier as a value of the URI attribute of the Reference element. This is what we have done in Listing 3. The value of the URI attribute points to the GetSpecialDiscountedBookingForPartners element. If, on the other hand, your data is external to the XMLDS file, you will refer to it using a URI as the URL attribute value of the Reference element.

   XMLDS allows you to perform some operations on your data before digesting and signing it. For example, you can canonicalize your data before signing it Or you may want to apply some XSL transformations on your data before digesting it. For instance, you may have some pricing data in a simple tabular form of model numbers and prices and you may want to transform the tabular form into a formal invoice before signing it. In this case, you may use an XSL transform as a template representing your invoice. This would mean that you intend to sign the complete formal invoice and not just the raw data included in XMLDS file.

   The Transforms element holds the information regarding what operations you performed on your data before signing it. Look at the Transforms element in Listing 3, which contains one Transform child element. There can be any number of Transform elements.

   Each Transform element identifies a transformation algorithm. When you apply a transformation to your data before signing it, you will include a reference to what you did by adding a Transform element. This will tell the recipient application of your signed file to do the same transformation before attempting to verify the signature. In our case, we have applied just one operation, which is the canonicalization algorithm specified by the Algorithm attribute of the Transform element in Listing 3.

   If there is more than one Transform element, their order is important. Transformations are applied in the same order that they appear in a Transforms element. All the transformations are performed before digesting the data. Hence, the output of the last Transform element is the input to the message digest algorithm.

2. What algorithm did you use to produce the digest value? The XMLDS specification suggests the use of SHA-1 digest algorithm. The DigestMethod child of the Reference element holds this information in its Algorithm attribute value (http://www.w3.org/2000/09/xmldsig#sha1).

3. The digest value itself. The DigestValue element in Listing 3 contains the actual digest value produced by digesting the canonicalized form of the GetSpecialDiscountedBookingForPartners element. Note that binary data in raw form (such as the sequence of octets produced by message digest, signature, and encryption algorithms) cannot be wrapped inside XML markup as such; it may produce problems while XML parsing. Such data is base-64 encoded before wrapping inside XML markup. The result of base-64 encoding is that the encrypted data does not contain any byte that conflicts with XML processing rules.

Once the SignedInfo and its child elements have been authored, you will canonicalize the complete SignedInfo element with the algorithm identified by the CanonicalizationMethod element. You will then produce the signature value and wrap the signature value inside a SignatureValue element as shown in Listing 4. While signing, you will use the canonical form of the complete SignedInfo element as data to be signed. This includes all the child elements of the SignedInfo element.

Notice that the SignedInfo structure contains a reference to the data being signed (the URI attribute of the Reference element), the digest value, and the name of the signature method as well as other bits of information. Therefore, signing the SignedInfo structure effectively means that you are signing the digest value of your data along with a reference to the data itself.

The Signature element in Listing 2 contains another child named KeyInfo. The fourth step is to create its child elements. In Listing 5, the KeyInfo element contains a KeyName child element. The KeyName element is an identifier for the key that will be used for signature verification. KeyName is just a placeholder for key identifiers. XMLDS does not specify the mechanism which will relate the identifier with the actual key pair used for signing. It is up to XMLDS applications to design their own mechanism for key identification. For example, the key identifier in Listing 5 (MyKeyIdentifier) may identify a shared secret (a symmetric key) previously exchanged between the tour operator and the hotel.

Moreover, the KeyInfo element is optional: you may or may not include a KeyInfo element in a signature. The KeyInfo element is optional because a signature application may not want to include key information inside the XMLDS file. The KeyInfo element may also be used in XML Encryption applications that we will demonstrate in the next section.

These four steps are a very simple demonstration of XMLDS. Listing 5 is a complete SOAP message that carries message integrity and user authentication data in its header.

Now it's time to demonstrate the processing of the XMLDS-based header of Listing 5 at the hotel's web service end.

XML Digital Signature Validation

The validation procedure is simple and can be logically deduced from XMLDS authoring steps discussed earlier. It involves three main tasks.

First, canonicalize the SignedInfo element. Recall that the CanonicalizationMethod element specifies the canonicalization algorithm. Use this canonical form of the SignedInfo element for the rest of the validation process.

Second, check the integrity of the message by verifying the digest value contained in the Reference element that we authored in step 2 above. For digest verification, you need to know three things:

1. The data that needs to be digested. You dereference the URI attribute of the Reference element in order to get the data that needs to be digested.

2. Any transformations that may have been applied to the data before applying the digest algorithm. The Transforms element contains this information. You will apply the same transformations to the data before digesting it.

3. The digest algorithm. This information is contained in the Algorithm attribute value of the DigestMethod element. You will apply the message digest and verify that the digest value is the same as that contained in the DigestValue element.

If the digest verification fails, the validation process fails and we're done.

If the digest value is found to be in order, the third task is to verify the signature. For the signature verification, you need the signer's key (the public key or the shared secret). You obtain the key information from the KeyInfo element if it is present (or your application may already know the keying information from some other means). Once you know the key to be used in signature validation, read the signature method used to produce the signature. The Algorithm attribute of the SignatureMethod element contains this information. Then use the canonical form of the SignedInfo element and the key to confirm the signature value.

An XMLDS implementation can create SOAP headers to produce signed SOAP messages. The XML firewall sitting at the recipient's end will process the SOAP header to verify the signatures before forwarding the request to the SOAP server. This process is graphically illustrated in Figure 1. We can achieve the following two security objectives through this procedure:

• We can verify that the SOAP message that we received was really sent by the sender we think it came from.
• We can verify that the data we received has not been changed while on its way and is the same that the sender intended to send.

So now we are sure that the request for special discounted booking is really coming from a trusted partner hotel and that no one has altered the data on its way. But hackers can still see the data while traveling across the Internet. So let's see how the XML encryption specification solves this problem.

XML Encryption

The XML Encryption specification satisfies confidentiality requirements in XML messages. XML encryption offers several features.

• You can encrypt a complete XML file.
• You can encrypt any single element of an XML file.
• You can encrypt only the contents of an XML element.
• You can encrypt non-XML data (e.g. a JPG image).
• You can encrypt an already encrypted element (i.e., "super-encryption").

Encrypting a Complete XML File

We'll start with encrypting a complete XML file. Have a look at Listing 6, where we have shown an XML encrypted file. We have not shown the XML document that we encrypted to arrive at Listing 6 because it doesn't matter. Encrypting any XML file will produce the same XML structure, except the encrypted value wrapped inside the CipherValue element in Listing 6.

The root EncryptedData element in Listing 6 holds the encrypted data along with relevant information such as the algorithm used for encryption. The EncryptedData element contains the XML Encryption namespace declaration (http://www.w3.org/2001/04/xmlenc#) and has an attribute named MimeType with value text/xml. This attribute advises the recipient of the XML encrypted file that we encrypted an XML file to produce the EncryptedData structure.

The first child of the root EncryptedData element is the EncryptionMethod element. The EncryptionMethod element has an attribute named Algorithm, which specifies the algorithm we used for encryption. The value that we have used for the Algorithm attribute is http://www.w3.org/2001/04/xmlenc#3des-cbc, which specifies the triple DES algorithm for encryption.

The ds:KeyInfo element in Listing 6 is the same as the one used in XMLDS. Note that the ds:KeyInfo element has been borrowed form the XMLDS namespace.

The EncryptedData element has another child element named CipherData, which in turn has a child element named CipherValue. The CipherValue element holds the encrypted content (the encrypted version of the XML document that we wanted to encrypt). Therefore, encrypting the XML file produces the contents of the CipherValue element.

Encrypting a Single Element

We have seen that the EncryptedData structure holds the encrypted data along with relevant information. While encrypting a single element of an XML file, we will use the same concept. Look at Listing 7, in which we have we have encrypted the complete GetSpecialDiscountedBookingForPartners element of Listing 1 by simply replacing it with the EncryptedData element.

You can compare the EncryptedData element of Listing 6 with the EncryptedData element of Listing 7. You will notice that there is one difference. Instead of the MimeType attribute of Listing 6, we now have a Type attribute in Listing 7. The value of this attribute is http:///www.w3.org/2001/04/xmlenc#Element, which means we encrypted an XML element.

Whenever you are encrypting an element of an XML file, you will use the identifier http:///www.w3.org/2001/04/xmlenc#Element as the Type attribute value. This tells the recipient of the XML encrypted file that the encrypted data should be treated as an XML element in decrypted plain text form.

Encrypting the Contents of an Element

Look at Listing 8, in which we have encrypted only the contents of the GetSpecialDiscountedBookingForPartners element by replacing the contents with the EncryptedData structure. This is similar to what we did while encrypting an element (Listing 7. There is a difference; this time, the value of the Type attribute of the EncryptedData tag says http://www.w3.org/2001/04/xmlenc#Content. The value tells that the encrypted data should be treated as element content.

XML Encryption Processing

How will our XML firewall work with these encryption concepts? It will receive Listing 7 or 8 (SOAP messages with encrypted elements or content) and translate the contents to a decrypted form before forwarding the decrypted SOAP message request to the SOAP server.

The recipient of an XML encrypted file (e.g. the hotel's XML firewall in our case) will decrypt the XML encrypted file in the following sequence:

1. Extract the encrypted content of the CypherValue element.
2. Read the algorithm attribute value of the EncryptionMethod element.
3. Read the Type attribute values of the EncryptedData element.
4. Obtain the keying information form the ds:KeyInfo element.
5. Use the information gathered in steps 1, 2, 3, and 4 to construct the plain text (decrypted) file.

An Introduction to Web Service Security

How will our XML firewall use XML signatures and encryption to protect SOAP servers? We have given many examples of using the two technologies individually, but the question of how to apply these two technologies in an XML firewall application to protect a SOAP server still needs to be addressed, especially since neither XMLDS nor XML Encryption are SOAP-specific. So why have we put all the signature related information in the SOAP header? Why not wrap it inside the SOAP body?

The Web Services Security (WSS) specification from OASIS defines the details of how to apply XML signature and XML encryption concepts in SOAP messaging. WSS relies on XMLDS and XML encryption for low level details and defines a higher-level syntax to wrap security information inside SOAP messages.

WSS describes a mechanism for securely exchanging SOAP messages. It provides the following three main security features:

1. Message Integrity
2. User Authentication
3. Confidentiality

Have a look at Listing 9. It is a SOAP message that carries security information according to the WSS syntax. Listing 9 is the same GetSpecialDiscountedBookingForPartners SOAP request that we have seen many times in this article. This time the request's header is carrying digital signature information according to the WSS syntax.

Following are the simple points about Listing 9 that will help you understand WSS syntax:

1. The SOAP:Envelope element in Listing 9 contains namespace declarations for SOAP, WSS, and XMLDS.

2. The SOAP:Header element contains just one child element (wsse:Security), which is the wrapper for all the security information in Listing 9. The wsse:Security element in Listing 9 has two child elements, namely a wsse:BinarySecurityToken element and a ds:Signature element.

3. The wsse:BinarySecurityToken element contains a security token. A security token is like a security pass or an identity card that you are required to show if you want to enter a restricted access area. There are several types of electronic security tokens.

   The most popular and widely used security token is a login-password pair, like the one you use while checking your e-mail.

   A login-password pair is a human readable security token. There are some security tokens that are in binary form (and therefore not necessarily human readable). Such tokens are referred to as binary security tokens. For example an X509 certificate (a widely popular format for digital certificates developed by ITU-T) is a binary security token.

   The ValueType attribute of the wsse:BinarySecurityToken element in Listing 9 tells what type of binary security token is wrapped inside this BinarySecurityToken element. In Listing 9, the ValueType attribute contains wsse:X509v3 as its value, which identifies X509 certificates.

   The EncodingType attribute of the wsse:BinarySecurityToken element tells the encoding of the binary security token. As already explained, it is not possible to wrap binary data inside XML format as such. Therefore, we have to encode binary data (usually as a sequence of base-64 encoded values) before wrapping inside XML. The X509 certificate is wrapped inside the wsse:BinarySecurityToken element as the element content.

4. The ds:Signature element is the same as the one we discussed in the section on XML signatures. Note two important things:

   • Look at the URI attribute of the Reference element. Its value (#myDiscountRequestBody) is a fragment identifier that points toward the SOAP:Body element. This means that the SOAP:Body element is the one that we have signed and wrapped the signature in XMLDS tags.

   • Secondly, also look at what the ds:KeyInfo element contains. It is a wsse:SecurityTokenReference element. The wsse:SecurityTokenReference element contains references to security tokens. In our case, it has a child element named wsse:Reference, whose URI attribute points toward the wsse:BinarySecurityToken element discussed in point 3 above. This means that the public key inside the X509 certificate (which the wsse:BinarySecurityToken element wraps) will be used to verify the signature.

This is a very simple example to introduce signed WSS messages. The third and fourth parts of this series will explore further details of XML-based security, the different types of security tokens that we can use with WSS, and the use of XML encryption in WSS messages.

In the next article, we will discuss Security Assertions Markup Language (SAML), which provides a way for web service applications to share user authentication information. This sharing of authentication data is commonly referred to as single sign-on. SAML can be used as a security token in WSS applications. The next article will elaborate why, when, and how.