Making Web Services Work at Amazon
by Edd DumbillDecember 09, 2003
Barr set the scene by outlining the various groups that Amazon's customers fall into: buyers, sellers (merchants who sell on Amazon's platform), web site owners (associates), and developers (people who use Amazon's web services.) Amazon's associates scheme has been very successful: founded in 1996, it now has over a million registered associates. This success augured well for the uptake of Amazon web services.
As Amazon's systems developed, they developed in the direction of interoperating feature components inside the firewall; e.g., the catalog, shopping cart, and personalization engine. Through their web services platform, Amazon is beginning to open these features up to public use, and Barr said they have ambitious plans to expose much more functionality.
So how did Amazon arrive at the decision to provide web services? One of the main drivers was that its partners needed better data access -- some major ones had XML data feeds, others simply scraped Amazon's web pages -- so the process of collaboration was both expensive and brittle. A move to defined and reusable web services was thus a logical solution.
|
Related Reading 
Amazon Hacks |
Amazon's aims in providing web services were, according to Barr, to support industry standards, provide remote access to data and functionality, decouple their data from its presentation, create a software development platform, unlock creativity from collaborators, and realize more benefit from their internal development investment. Particularly key for Barr, previously employed in Microsoft's developer tools division, is the aspect of creating a development platform, in much the same way as traditional software vendors do.
Considerations for Providing Web Services
Barr then outlined the various issues Amazon considered in the web service deployment, as well as their resolution. The first of these was revenue -- should Amazon pursue web services as an experiment or with the intention of generating revenue? Building on the success of their associates and sellers, Amazon decided to pursue revenue, building on the relationships with these two groups in the first instance. It seems that the choice to do this was key, as it provided a real-world customer base driving their design decisions.
The second issue to consider was that of licensing. In order to create conditions for success, the terms needed to protect the rights of both the developers and of Amazon itself. While providing a degree of openness in order to foster creativity and adoption, the license needed also to sustain Amazon's business model. Practical issues include ensuring data freshness and preventing excessive server load. These needs were met with licensing constraints including one API call per second, a ban on reselling data, storing non-price relevant data for 24 hours maximum and pricing data for 1 hour maximum, and a mandatory link to amazon.com.
The next issue was that of protocols. Should they support SOAP or XML over HTTP (that is, REST)? In the end Amazon provided both and let developers make the choice. Despite it being the "standard", only about 15% of Amazon web services calls are made with SOAP, the remainder with REST.
The fourth issue was to consider how to create a software platform for developers. This was addressed by borrowing best practices from the regular software world: document the APIs, commit to API stability, and provide backward compatibility across API revisions.
The final consideration for Amazon in deploying its web services was developer support -- how to help developers to succeed with the API. Amazon use a combination of a discussion board, weekly developer chats, a regular newsletter, frequent releases of the software development kit, and an online FAQ. Being responsive and open to their developer community has worked well for Amazon.
Barr's talk provided many good pointers for large businesses considering opening themselves to greater programmatic interaction with developers. Amazon's decisions certainly seem to have set them on a course for success. Perhaps the best mark of this success and future promise is that Amazon is increasing the size of its internal team by five times for 2004.
Did you attend this XML 2003 session? Post your comments on the talk in our forum.
(* You must be a member of XML.com to use this feature.)
Comment on this Article
| Titles Only | Titles Only | Newest First |
- SOAP Bloat and poor uptime
2003-12-11 07:48:17 Rich Z [Reply]
I've mainly used the Amazon merchant platform and I can honestly say that it's the biggest piece of junk I've ever seen. I will admit that the thought and design that went into the merchant side is really amazing, but it just doesn't work.
The merchant platform allows me as a merchant to post my products/prices/inventory/images etc., to the amazon site on a daily basis via XML/SOAP. It also enables me to get orders on their system that are placed for my product. But the system is set up to facillitate both very large and very small businesses, which means the system doesn't do a very good job at servicing either. For instance, they don't want or care about UPC info for products, they only want to know the SKU. Well, that's great unless you're talking about something with both sizes and widths (like shoes, pants, etc.) because then the SKU might not be a unique identifier. The UPC always is unique for a product. So, then what happens if another merchant selling completely different product has the same SKU as you? You guessed it, both of your products wind up being garbage on the site.
And since about black friday on this year, their uptime for receiving SOAP requests has been about 50%. Not very stable. SOAP works great except when one side is down. I recognize that all shopping sites experience load at the holidays but how many years have they been doing this? Did they just forget that traffic would be high now?
It drives me nuts that they spend so much time and effort talking about how great their associates program is working, meanwhile all of us high profile merchants that have signed on with them are just blowing in the wind. The whole web services concept seems to be just a toy to them and not a legitimate business process, because they certainly don't treat it as such.
In case you're wondering, I work for a major department store chain that's been featured on the amazon site for about a year now.
- SOAP Bloat and poor uptime
2005-10-06 22:20:29 Kalbo [Reply]
Hi,
We are new to Amazon merchant Integration and we are using PHP and/or Perl. Unfortunately, we cannot see any resource that would help us in our integration.
As of now, and after some weeks already, we are still trying to find a way to connect to the SOAP server (merchant) of Amazon.
We tried using NuSOAP and PEAR::Soap in PHP, and tried as well SOAP::Lite and HTTP::POST in Perl to no avail. There seems to be a deadlock somewhere that we couldn't establish a connection. Btw, we have no problem connecting with unsecure connection (not HTTPS).
Will anyone there could be of help?
Thank you.
Ed Aspra
- SOAP Bloat and poor uptime
2004-04-10 21:20:15 Mike Dierken [Reply]
I'm sorry that you encountered problems using the Amazon merchants platform.
"For instance, they don't want or care about UPC info for products, they only want to know the SKU".
Actually, Amazon would love to require UPC for as many product types as possible, it is a huge help in distinguishing products apart. It is true that SKU is required, but that's pretty much a given (a SKU is the unique identifier of the product submission by the merchant).
"Well, that's great unless you're talking about something with both sizes and widths (like shoes, pants, etc.) because then the SKU might not be a unique identifier."
In this case, there should be a separate SKU for each variant.
"So, then what happens if another merchant selling completely different product has the same SKU as you? You guessed it, both of your products wind up being garbage on the site."
The SKU should be additionally qualified by which merchant submitted it - collisions should be avoided. If you have a specific situation that you can refer me to, I would appreciate hearing about it.
"In case you're wondering, I work for a major department store chain that's been featured on the amazon site for about a year now."
Tell me which one privately & I'll see what I can do.
"In my opinion, creating/parsing the rather complicated XML messages was the hardest part of the whole implementation."
This is good feedback - have you provided that to Amazon?
- SOAP Bloat and poor uptime
2004-01-18 23:05:40 Yani Brinski [Reply]
Rich, could we solicit a bit of advice?
We are looking at Amazon as well and have been advised to use a SOAP interface called WASH.
Do you advise writing a SOAP interface to Amazon or using 3rd party tools such as WASH?
- To WASH or not?
2004-02-02 10:14:54 Rich Z [Reply]
It really depends on your level of awareness with SOAP tools. Amazon provides a few sample toolkits, we used their sample Perl module and hacked it to fit our needs.
WASH is kind of a middleware package that is going to send/receive/track the soap messages in a basic way. It is NOT going to create the XML for you to send nor will it translate the incoming order XML into something meaningful like EDI.
In my opinion, creating/parsing the rather complicated XML messages was the hardest part of the whole implementation. For us it didn't make sense to purchase WASH since we'd still have to all the other work ourselves anyway.
- To WASH or not?
- SOAP Bloat and poor uptime
2003-12-16 07:13:04 Trent Minneman [Reply]
Has the merchant services program with Amazon been worth the effort and headaches? We have just recently been invited to 'the dance' and are in the process of trying to determine what type of ROI we can expect.
- SOAP Bloat and poor uptime
2003-12-23 06:30:50 Rich Z [Reply]
Well, luckily for us, we had most of the work done already for other sites that we deal with... but sales have been very disappointing...
- SOAP Bloat and poor uptime
2004-02-03 06:06:48 Juan Carlos Gonzalez [Reply]
Hi guys, our company has just started the merchant integration process with Amazon, we are a microsoft shop using .net framework. It was brought to our attention the Amazon won't support the use of this platform to work with their web services. Do you know of any SOAP product that runs on a MIcrosoft 2000 server that we can use to interact with them?
We are pretty new in the .net enviroment/SOAP functionality.
Thanks
- SOAP Bloat and poor uptime
2004-02-11 21:43:04 David Reynolds [Reply]
>>Do you know of any SOAP product that runs on a MIcrosoft 2000 server that we can use to interact with them?
WASH from wrinklebrain.com (as Rich Z mentioned above) looks like it might be right for you, providing you can handle the XML generation. I had thought that might be the hard part, but then after looking closer I suspect the real challenge is in tracking submissions and responses, and not dropping _any_ of them... wrinklebrain has both win32 and a linux solution, but if you call them, it sounds like the future of the linux offering isn't that good.
Myself, I'm going to doggedly try to roll my own, but it looks like this will be an unpleasant two months. BTW, I understand that the initial provided soap wrapper perl mod was quite buggy, and I'm underwhelmed that the current one is still dated from the begining of the service rollout (July 2002).
- SOAP Bloat and poor uptime
2004-03-13 10:22:04 Sean Doyle [Reply]
Building a highly reliable and trustworthy SOAP framework for Amazon is somewhat tricky, and you are correct, tracking the entire process workflow is crucial to success.
The decision to build or buy should factor in, features, time to market, complexity, testing/certification and maintenance going forward. The Amazon platform is evolving, albeit at a measured pace, so if you do choose to build over buy you will need to update it occasionaly to comply with new features and SLAs. Make sure that you factor this in before making your decision.
<saleson>For more information on our WASH product and a demo, feel free to give us a call at WrinkleBrain. </saleson>
- SOAP Bloat and poor uptime
- SOAP Bloat and poor uptime
2004-02-06 08:09:51 Matt Eggertson [Reply]
Hey, I am in the same boat as Juan Carlos Gonzalez, my company has just started the merchant integration process as well in the .net framework with little luck. I admit I have very limited experience with soap and web services and have spent the last week beating my head against the wall trying to make heads or tails of half the information on the Amazon integration website.
Any help or guidance would be great
-Thanks
- SOAP Bloat and poor uptime
2004-02-19 08:32:21 Juan Carlos Gonzalez [Reply]
Hey Matt, I have found an easy way to create the xml feed to Amazon, I generated classes based on the xsd files, and then use the serialize method.
My problem now is in communicating with Amazon, their support is worthless, I'm getting an unauthorize message. They said I have to force .net to preauthenticate, I have tried so many different ways with no results. Are you there yet?
- SOAP Bloat and poor uptime
2004-03-13 10:24:35 Sean Doyle [Reply]
.NET preauthentication to comply with Amazon's SOAP framework is indeed tricky. Don't forget that you must also override the DIME encoding methods with SWA compliant methods.
- SOAP Bloat and poor uptime
2005-07-14 12:29:32 Schenz [Reply]
I see now that Amazon.com has a DIME version of it's SOAP platform available for Microsoft.NET users. Does anyone have any better examples that the one on the Seller Central help pages?
- SOAP Bloat and poor uptime
2005-10-07 16:18:44 Kalbo [Reply]
Hi,
We are new to Amazon merchant Integration and we are using PHP and/or Perl. Unfortunately, we cannot see any resource that would help us in our integration.
As of now, and after some weeks already, we are still trying to find a way to connect to the SOAP server (merchant) of Amazon.
We tried using NuSOAP and PEAR::Soap in PHP, and tried as well SOAP::Lite and HTTP::POST in Perl to no avail. There seems to be a deadlock somewhere that we couldn't establish a connection. Btw, we have no problem connecting with unsecure connection (not HTTPS).
Will anyone there could be of help?
Thank you.
- SOAP Bloat and poor uptime
2006-06-19 14:02:16 Lily06 [Reply]
Hi,
I am in similar situation as Kalbo. Has anyone figure out a way to do it in perl after Oct, 2005?
Any help is greatly appreciated.
- SOAP Bloat and poor uptime
- SOAP Bloat and poor uptime
- SOAP Bloat and poor uptime
- SOAP Bloat and poor uptime
- SOAP Bloat and poor uptime
- SOAP Bloat and poor uptime
- SOAP Bloat and poor uptime
- SOAP Bloat and poor uptime
- SOAP Bloat and poor uptime
