Internet Scripting: Zope and XML-RPC

January 12, 2000

Amos Latteier

XML-RPC is a simple protocol for sending remote procedure calls over the Internet using XML and HTTP. Zope is an Open Source application server that publishes objects on the Internet. Together they form a simple system of remotely scriptable web objects. Zope provides the web objects and XML-RPC provides a language-neutral communication protocol. This article will demonstrate how to script web objects using XML-RPC with Zope.

Scripting Web Objects

Zope puts objects on the Web and XML-RPC lets you script those objects.

All Zope objects have a URL, and all Zope objects have methods that can be called over the Web. Normally you call methods on Zope objects through your web browser. XML-RPC gives you the power to control your web site programmatically. By using XML-RPC with Zope, your web site literally becomes an application that can service client programs as well as web browsers.

Since XML-RPC is built into Zope, all your Zope objects are XML-RPC-enabled. You don't have to do any extra work to set up your web site. You can use XML-RPC to script your Zope objects using Zope's standard API.

The ability to script remote objects can be very powerful. Since you have access to the full Zope API through XML-RPC, you've got full control. Any action you can perform with the Zope management interface can be programmed with XML-RPC. You can create and edit documents via XML-RPC. You can query and manipulate object properties. You can search Zope. You can call your own custom Zope objects and methods. You can even create new users and control security policies. Plus, XML-RPC is language-independent so you don't have to learn a new language to use it.

An Example

Let's jump right in. Here's how to retrieve the title of the Zope.org web site via XML-RPC in Perl:

use Frontier::Client;

$server = Frontier::Client->new(url => "http://www.zope.org/");

print $server->call('title_or_id'); # prints "Welcome to Zope.org"

Note: to run this example you'll need Ken MacLeod's XML-RPC Perl package.

How does this program work? The program opens an HTTP connection to http://www.zope.org. Then it sends a message encoded in XML that requests the title_or_id method to be called. Zope locates the target object and calls its title_or_id method. Then Zope encodes the response and returns it to the program. Finally, the program decodes the response and prints it.

XML-RPC Protocol

XML-RPC is an extremely simple protocol. It uses XML to encode function calls and responses and sends these messages over HTTP.

In the example above, the Perl program sent this XML message to Zope as the body of an HTTP request.

<?xml version='1.0'?>

<methodCall>

<methodName>title_or_id</methodName>

<params>

</params>

</methodCall>

This request tells Zope to call the title_or_id method with no arguments. Zope called the method and then returned this response as the body of an HTTP response.

<?xml version='1.0'?>

<methodResponse>

<params>

<param>

<value><string>Welcome to Zope.org</string></value>

</param>

</params>

</methodResponse>

The response indicates that the method call was successful and that the result was the string, "Welcome to Zope.org".

XML-RPC leverages existing standards to create a basic remote procedure call protocol. This makes XML-RPC simple and appealing.

For a fuller discussion of the XML-RPC protocol, see Backends Sharing Data and the official XML-RPC web site.

XML-RPC Security

XML-RPC provides no security provisions. This may sound like a shortcoming, but in many respects it is an advantage. Since XML-RPC does not mandate any security protocol, Zope's normal security policies work just fine with XML-RPC.

One of Zope's greatest features is its simple and powerful security model. Every public method of a Zope object is protected by a security permission. Zope allows you to flexibly assign permissions to users. Zope authenticates users over HTTP with basic authentication or cookies.

To access protected Zope methods, your XML-RPC client must know how to perform HTTP basic authentication. Not all XML-RPC clients currently support basic authentication. Here's an example of how to extend an XML-RPC client to support basic authentication.

XML-RPC Versus SOAP

Recently a group of vendors (including Microsoft) announced the creation of the SOAP specification, a distributed objects protocol similar to XML-RPC. Digital Creations, the original developers of Zope, announced that a future version of Zope will support SOAP.

The main differences between SOAP and XML-RPC are that SOAP provides a more complete model for calling methods on remote objects, SOAP is on track to become standardized by the IETF, and SOAP is likely to have greater support in the world of COM and Visual Basic.

We're happy to have Zope speak as many powerful, open protocols as possible. Zope's SOAP support is likely to operate in a manner similar to its XML-RPC support.

XML-RPC Limitations

There are limits to what you can do with XML-RPC and Zope. XML-RPC's marshalling is limited in the kinds of objects it can pass to and from methods. This makes it difficult to call Zope methods that require Zope objects as arguments. XML-RPC's notion of method calling does not allow named method parameters. In addition, most Zope APIs were designed to be called from a web browser or a Zope template. So accomplishing some tasks with XML-RPC may be more awkward than you would like. Fortunately, these problems have solutions in sight.

SOAP should address most of the technical shortcomings of XML-RPC. Additionally, an effort currently underway to overhaul the Zope API to provide a less HTML-centric interface should improve the ease of remote scripting.

To illustrate how to communicate with Zope via XML-RPC let's build a simple Zope client. The program will allow you to connect to Zope and edit documents. This example uses Python but you could use Perl or TCL or Java or any of a number languages which support XML-RPC. To run the following interactive examples you'll need my extended version Pythonware's Python XML-RPC module which includes support for basic authentication. Note: the completed client does not need this extended module.

Reading a Document

The first thing we'll want to do is to be able to retrieve the contents of a Zope document.

Connect to the Zope with your web browser and identify a document that you would like to work with. For this example I'll use a document located at http://localhost:8080/Test.

Now let's connect to this object via XML-RPC.

Fire up the Python interpreter and import the extended XML-RPC module.

D:\>python

Python 1.5.2 (#0, Apr 13 1999, 10:51:12) [MSC 32 bit (Intel)] on win32

Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam

>>> import xmlrpclib

>>>

If you get an import error at this point you haven't correctly installed the extended Python XML-RPC module.

Next create a Server object which points at your Zope document. You'll need the URL of your document and a manager username and password for Zope. Recall that your document's URL is shown at the top of its Zope management screen.

>>> d=xmlrpclib.Server('http://localhost:8080/Test',

... xmlrpclib.BasicAuthTransport('myusername','mypassword'))

Note: you should substitute your Zope manager username and password in the above example.

Now you have a local Python object which is a proxy for your remote Zope document object. You can call methods on your proxy object and XML-RPC will take care of marshalling the arguments, sending them to your Zope object and returning the results. It's like having the remote Zope object locally available.

To get the contents of a Zope document you can call the document_src method.

>>> print d.document_src()

<dtml-var standard_html_header>

<h2><dtml-var title_or_id></h2>

<p>

This is my sample test document.

</p>

<dtml-var standard_html_footer>

Notice how you just call the method on our proxy object as though it were the actual Zope document object. This is the magic of XML-RPC--simple distributed objects.

Uploading a Document

Now let's make some changes to the contents of the document.

>>> content=d.document_src()

>>> import string

>>> content=string.replace(doc, 'test', 'CHANGED')

>>> print content

<dtml-var standard_html_header>

<h2><dtml-var title_or_id></h2>

<p>

This is my sample CHANGED document.

</p>

<dtml-var standard_html_footer>

Now that you've changed your document let's upload it to Zope. You can do this by calling the manage_upload method on your Zope document.

>>> d.manage_upload(content)

Zope will return a confirmation HTML message indicating that the upload was successful. You can now verify that the upload worked by going to the Zope management interface with your web browser and checking the source of the document.

Sure enough, your changes have been made.

Congratulations! You've just successfully managed Zope with XML-RPC.

Exploring Zope Folders

In addition to editing documents a Zope client should be able to locate Zope documents.

You can find out about the contents of a Zope folder by calling its objectIds method. This method tells you the names of contained objects. Here's how to query the Zope folder that contains the test document.

>>> f=xmlrpclib.Server('http://localhost:8080',

... xmlrpclib.BasicAuthTransport('myusername','mypassword'))

>>> print f.objectIds()

['Control_Panel', 'standard_html_header', 'standard_html_footer',

'standard_error_message', 'Test']

This shows you all the objects contained in your top-level Zope folder. If you just want to see certain kinds of Zope objects you can pass the type(s) of the objects you are interested in to the objectIds method.

>>> print f.objectIds('DTML Method')

['standard_html_header', 'standard_html_footer',

'standard_error_message']

>>> print f.objectIds('DTML Document')

['Test']

>>> print f.objectIds('Folder')

[]

Using the objectIds method a Zope client can easily explore a Zope folder and its sub-folders.

The Finished Client

Now that you've seen how to locate and edit Zope documents using XML-RPC, it's relatively straightforward to build a graphical user interfacethat allows you to browse Zope folders and edit documents.

The example client includes source code so you can see how it works. The client connects to a Zope web site and allows you to edit documents. The left panel displays a tree control that lets you browse through folders. The client calls the objectIds method on Zope folders to get the information needed for this tree. When you click on a document in the left panel the right panel displays its contents. The client retrieves the document contents by calling the document_src method. You can then edit the contents and save your changes to Zope. The client commits the changes using the manage_upload method.

Here's a screen shot of the finished example client.

To try it out and see how it works download the Zope client here.

The example client should run on both Linux and Windows. To install the Zope client on you local machine, first untar and ungzip the ZopeClient.tgz file. If you are using Windows, WinZip should be able to do this. Next you will need Python, wxPython, and xmlrpclib installed on you local machine. Now you're ready to run the example Zope client. See the included README.txt file for more information.

In addition to being an XML-RPC server Zope can act as an XML-RPC client. While Zope doesn't come with built-in XML-RPC client objects it does support Python scripting and includes the standard Python XML-RPC client library. Pretty much anything you can do in Python you can do in Zope.

Here's a step by step recipe for using Zope as an XML-RPC client.

Creating an External Method

To use Zope as an XML-RPC client, first create an External Method. An External Method is a Zope object which uses Python scripting. For more information about External Methods see the Zope Content Manager's Guide.

Here's the code for an External Method that calls a method running on Frontier using XML-RPC.

import xmlrpclib



def stateName(self, number):

    """

    Returns a state name given an integer between 1 and 50.

    """

    server=xmlrpclib.Server('http://betty.userland.com')

    return server.examples.getStateName(number)

Save this code to a file named 'xmlrpc_test.py' in the Zope top-level 'Extensions' directory. Choose External Method from the product add list. Specify 'stateName' as the Id, 'stateName' as the function name, 'xmlrpc_example' as the Python module file, and click 'Add'.

Calling the External Method

Now that you've created an External Method that uses XML-RPC you need to call it from Zope.

In general you will call your External Methods from other Zope objects such as DTML Documents. Here's an example of how to call the 'stateName' External Method. Create a DTML Document by selecting 'DTML Document' from the product add list. Give the document an Id of 'stateForm' and click 'Add and Edit'. Now enter this for the document content:

<dtml-var standard_html_header>



<p>

<form>

State Number (1-50): <input type="text" name="number:int">

<br>

<input type="submit" value="Find State Name">

</form>

</p>



<dtml-if number>

<h2>Results</h2>

<p>

<dtml-var number> : <dtml-var "stateName(number)">

</p>

</dtml-if>

 

<dtml-var standard_html_footer>

Now view the document to try it out. Type a number in the form and submit it.

The document displays a form that collects the state number and displays the results. The first paragraph displays the form. Notice that the form has no action, so this document will call itself. When you submit the form, the number variable will be set. Then the document's second paragraph will be displayed. The second paragraph calls the stateName External Method with the number variable as an argument.

Conclusion

You've seen first hand how to use Zope as an XML-RPC server and client. It's not hard to build a simple web application with Zope and XML-RPC.

Zope and XML-RPC provide a easy-to-use, scriptable distributed object system. Simplicity, reliance on Internet standards, and language independence make XML-RPC an appealing protocol. Zope's web object architecture make it an obvious choice for XML-RPC work.

Zope and XML-RPC provide a compelling example of how XML is allowing a shift from a world of web sites to a world of web applications.