Parasoft SOAPtest 4.0
Cost: US$3,995 (NZ$5,575) for a single licence
Platforms: Windows 2000/XP, Linux, Solaris
Bottom line: Parasoft’s SOAPtest 4.0 is a web service testing tool that will be useful to developers and QA engineers alike. Its strength is the speed with which robust test suites can be built from humble unit-test beginnings. New security tests for web services are a fine addition.
Whereas most of us surf the "visible" HTTP exchanges between browser and web server, web services are transporting an increasing load of otherwise invisible traffic. With the help of SOAP, web service clients and servers carry on unseen conversations, similar to messages traveling over a subfrequency.
They may be invisible, but these conversations are important: more and more frequently, web services are being used to conduct vital business transactions.
Parasoft's SOAPtest is one of a growing number of tools that test these increasingly critical applications. Although its name implies its only job is identifying and diagnosing problems in SOAP messages, SOAPtest delivers comprehensive regression, unit and security testing for web services and web service client applications.
Version 4.0 adds features that make SOAPtest easier to use than its earlier editions. It also adds new test categories -- for example, SOAPtest's new security tests ascertain how well your web service is protected against a malevolent user's attacks.
SOAPtest has two primary uses, but it functions as an automatic test generator in both cases.
First, it creates static WSDL verification tests for the content of a web service's WSDL document. The tests that SOAPtest builds check not only for proper WSDL syntax but also for correct WSDL semantics — for example, it verifies the document's internal consistency.
Second, SOAPtest reads the WSDL document's contents and creates unit tests for the web service's methods. SOAPtest can also create its tests from the web service's UDDI information, as well as from an HTTP traffic log file involving web services.
This is SOAPtest's real power. Beginning with a small, simple set of static and dynamic tests automatically generated by a wizard (one of the new features of SOAPtest 4.0), you can extend, duplicate and modify those tests to build — Lego-style — an increasingly powerful suite. Tests are executed in sequence; by properly ordering tests, you can simulate a lengthy session between web client and web service.
I ran SOAPtest on a pair of sample web services I had been using to explore various ASP.Net features. The tool's best feature is its seemingly instantaneous creation of tests, and the equally speedy way I could iteratively extend and modify those tests. SOAPtest allowed me to assemble my tests in such a way that what began as a collection of unit tests blossomed into an array of functional tests. And throughout this process, I never needed to stoop into the source code.
With SOAPtest, subsequent tests can use data from preceding tests as their inputs, and that allows you to build a "mock" workflow of a user carrying out a complex series of transactions with the particular web service. By incrementally layering tests on a suite's foundation, you produce a comprehensive project that — taken as a whole — ends up being greater than the sum of its parts.
Also new in SOAPtest 4.0 is its capability of launching attacks against a web service to assay its security vulnerabilities. The set of new security tests currently generated by SOAPtest includes — but is not limited to — parameter fuzzing, which passes irregular parameters that might cause the web service to throw an exception or reveal information it shouldn't; XML bombs, detonated if an XML entity is defined recursively in a DTD, causing an XML parser to expand the enclosing document to an unmanageable size; SQL injection, which injects unwanted strings into the text of a SQL query, causing the application to throw an exception or execute improper SQL code; and XML external entity attacks that exploit a DTD's capability of making an external reference.
As if that's not enough, version 4.0 also extends SOAPtest's performance and load testing. Now you can simulate — and view the effects of — multiple clients posting requests to your web service and graphically tune the simulation's evolution. For example, you can see how your web service performs if client requests increase with a gradual linearity, peak suddenly, or rise and fall in a bell curve.
Finally, SOAPtest 4.0 is not limited to testing web services; it also exercises a web service client. By configuring SOAPtest to act as the web service itself, you cause it to mimic the server; as such, it can verify that clients post proper requests. It's a valuable feature because it allows you to test the "whole" web service application — client and service/server — instead of just the service.
SOAPtest 4.0's strength lies in the ease and agility with which you can create and modify tests. In seconds, you can build a test, run it and examine the response. If the test looks good, add it to your suite; if not, modify or extend it — or throw it away and create a new one.
SOAPtest 4.0 effectively walks that narrow line between being entirely code-free (allowing the wizard to build the tests) and customisable (coding an involved script by hand). This balance makes it useful to QA engineers and developers. Because you can run it from the command line, it is easily incorporated into automated builds. If you're building serious web services, you need SOAPtest.