Downloads: WS-I Issues First Interoperability Tests

The WS-I has released the first set of interoperability testing tools aimed at helping vendors and enterprise developers ensure compliance with its Basic Profile 1.0, the key to making sure their implemented web services will work across Java, C#/.NET and legacy environments. Download the first prerelease tests and read what they will do -- and won't -- for "out-of-the-box" web services interoperability testing.

Tags: Web Service, Profile, Testing Tools, WS-I, Basic Profile, Analyzer, Monitor,

The WS-I has released its first set of interoperability testing tools aimed at helping vendors and enterprise developers ensure compliance with its Basic Profile 1.0, the key to ensuring their implemented web services will work across Java, C#/.NET and legacy environments.

The tools are The Web Service Communication Monitor ("Monitor") and The Web Service Profile Analyzer ("Analyzer"). Developed by WS-I's Test Tools Working Group, they come in implementations for both C# and Java, and can be used on any web services platform.

The Monitor captures messages exchanged between web services and the software that invokes them, storing the messages for later analysis. Today's prerelease version captures HTTP-based SOAP messages. In specific, the Monitor is both a message capture and logging tool. The interceptor captures messages, and the logger reformats them and stores them for later analysis in the message log. The monitor is implemented using a man-in-the-middle approach to intercept and record messages.

For its part, the Analyzer is an analysis tool that verifies the conformance of web services artifacts to the Basic Profile. For example, it analyzes the messages sent to and from a web service, after these have been stored in the message log by the Monitor. In specific, the Analyzer evaluates messages captured by the Monitor, and also validates the description and registration artifacts of the web service. This includes the WSDL document(s) that describes the web service, and the XML schema files that describe the data types used in the WSDL service definition and the UDDI registration entries.

The Analyzer produces a report that indicates whether or not a web service meets the interoperability guidelines of the WS-I Basic Profile 1.0, detailing specific deviations and failures, so that users know which requirements of the WS-I Basic Profile were not met.

The Analyzer assesses files and/or data artifacts that aren't part of the test framework, but are dependent on the web service to be tested: These include:
  1. Web Service artifacts -- These inputs to the Analyzer are target material for testing, and will be reported on;
  2. Message Log -- Contains the monitoring trace of messages captured at transport level;
  3. WSDL definitions -- Contains the definitions related to the web service; and
  4. UDDI entries -- Contains references to Web Service definitions, as well as bindings.
In a statement, Jacques Durand, chair of the WS-I's Test Tools Working Group, which built the tools and created the documentation, described the tools and their scope: "The tools have been designed in such a way to allow for expansion and extension, so they can accommodate the Basic Profile as well as future profiles. Testing results will help developers ensure that their web services meet the WS-I interoperability guidelines." Durand, who is also the director of Industry Relations at Fujitsu Software Corp., added, "They can be configured to specifically address whichever profile definition they need to verify. "

Inside the WS-I Testing Tools

Inside the Monitor
The Monitor has two distinct sets of functionality:
  • It is responsible for sending messages on to some other endpoint that is capable of accepting the traffic while preserving the integrity of communication between the two endpoints.; and
  • It is responsible for recording the messages that flow through it to a log file.

  • The WS-I document offers the following description of its use:
    One can think of these two pieces as an interceptor and a logger. For this first version of the Monitor, the interceptor and logger functionality will exist in the same application. The working group recognizes that we may later desire to separate the interceptor and the logger into two, stand-alone entities. This design discusses how one would go about structuring an application today that should be able to be broken into separate pieces in future versions.
    In the current tool, the interceptor captures messages by situating itself between the sender and receiver of the message. The interception technique varies depending on the transport. For version 1.0, WS-I addressed only HTTP riding on top of TCP/IP. For this, the interceptor is responsible for grabbing the message and any related data. It then takes that data and must forward it to its ultimate destination, and also sends the data on to the message logger.

    To send the data to the message logger, the interceptor should format the data into an XML stream or an object capable of being easily transformed into XML. The data captured is dependent on the transport protocol in use. With HTTP, the interceptor must capture the HTTP headers as well as the type of message (HTTP request or response). The interceptor sits between the message sender and receiver. When a message goes through the interceptor, the message is recorded in the logger and then passed on, unchanged, to the receiver.

    Inside the Analyzer
    The Analyzer's purpose is to validate the messages that were sent to and from a web service. The tool is also responsible for verifying the web service's description. This includes the WSDL document that describes the web service and the XML schema files that describe the data types used in the WSDL service definition.

    The Analyzer tool has a defined set of input files, all of which are used to verify conformance to a profile definition. These include:
    • Analyzer configuration file
    • Test assertion definition file
    • Message log file
    • WSDL for the web service
    As noted above, the Analyzer produces a conformance report that indicates whether input to the Analyzer conforms to a specific WS-I profile. All deviations from the profile specification must be listed in the report.

    The FAQ -- WS-I Tools' "Wills and Won'ts"
    In an accompanying Q&A, the WS-I goes in-depth about what the tools can -- and cannot -- provide.

    For instance, in response to the question, "Can the testing tools certify that a web service is conforming to the Profile?", WS-I provided a candid reply, which in part reads:
    The tools can only verify the conformance of Web Service artifacts that are produced during a testing session. Some artifacts belong to the definition of the Web Service (WSDL); some others result from the observable behavior of the Web Service at run-time. It is rather difficult to test all possible behaviors that a Web Service can exhibit, mostly because exercising these behaviors is application-dependent and requires an application-level understanding of the Web Service."
    The WS-I explains that, for the above reason, the Testing WS-I working group has not attempted to provide certification criteria. Further explaining this decision, the WS-I goes on to say:
    Indeed, using certification criteria that are too general or incomplete will not guarantee interoperability for every use case, and therefore a certification stamp would have little meaning. Instead, the tools are intended to observe and verify the messages produced during an interaction, possibly in a real deployment environment (because the tools are non-intrusive).
    The tools can also be used at development time, to verify that web service definitions are profile-conforming. The testing tools are, then, an indicator of a web service's conformance to the Basic Profile, based on the artifacts produced. In turn, this conformance indicates interoperability with other business partners who also have tested in conformance with the Basic Profile.

    Here are some other candid replies from the WS-I's FAQ document that accompanies the downloadable tests for C# and Java versions. [NOTE: The following responses have been summarized by IDN editors, where appropriate.]

    Q: Can the testing tools verify all the requirements of the Basic Profile?

    WS-I: No. However, by addressing requirements that concern the run-time interaction between a Web Service and another party, the tools provide a powerful indicator of the ability of this Web Service to interoperate with any external party known to also comply with the Basic Profile.

    Q: How can we be sure that all the operations of a Web Service have been covered in the testing?

    WS-I: Because the test framework -- in the current version -- does not include a Test Driver, a complete coverage of all the operations will rely on the client program involved in the testing of the Web Service, which is either ad-hoc, or is a real application in deployment over which the test operator does not have much control.

    Q: What are some practical situations where the testing tools show value to Web Services users or vendors?

    WS-I: An industry may define industry-specific Web Services (e.g., purchase order submission, request for product information) and specific usage scenarios. In order to achieve this, this industry will likely define... an industry-specific test harness and certification criterion for interoperability, based on the Basic Profile. If such a Web Service passes the tests, a vendor in this industry can claim that it is interoperable with any user application, provided that the user also complies with the Basic Profile.

    Another scenario shows value for interoperability troubleshooting. Because the testing tools can monitor messages from both interacting parties, the tools can be used to diagnose a failure to interoperate, and to identify the cause: either the client application or the Web Service may exhibit non-conforming behavior during this particular interaction.

    Q: Is it acceptable for a Web Service to have some operations conforming to the Profile, and some other not conforming?

    WS-I: Yes. The Profile requirements are not defined at Service level, but at a lower level, typically at WSDL port level. Some requirements are about operations, or bindings. A Web Service may have some of its ports using the profile-conforming SOAP binding, some other ports using a non-conforming SOAP binding, and some ports using a non-SOAP binding. Yet, some business users may only be interested in interoperating with those ports that are Profile-conforming. Therefore the testing tools will be able to assess the conformance of a Web Service at port level. This will simply require exercising this port only, during a monitoring session. (As well as targeting this port only when testing the WSDL file.)

    Q: Will the WS-I Test Framework also support functional testing of the Web Service?

    WS-I: This is outside of the scope of conformance testing to the Basic Profile. Such testing would involve knowledge of the application semantics that is specific to each Web Service. The Monitor developed by the Test working group could, however, be reused -- for example by the Sample Application working group -- to provide the message capture necessary to such testing.

    Q: Are there some restrictions in using the testing tools?

    WS-I: This version of the Monitor will not handle secure connections, SSL in particular. (This does not preclude one from using SSL with the Basic Profile.)

    Download the WS-I Testing Tools

    The WS-I download available here includes:

    • Configuration Files -- These are XML files used to control the execution of the Testing Tools.
    • Monitor Configuration File -- Controls the execution of the Monitor;
    • Analyzer Configuration File -- Controls the execution of the Analyzer; and
    • Test Assertion Document -- Defines the test assertions that will be processed by the Analyzer.
    The available download also includes several examples of input and output from the tests. Key examples include: a configuration file used by the Monitor and Analyzer tools, a sample output report from the Monitor and Analyzer, an Analyzer configuration file with UDDI reference, an Analyzer configuration file with type binding and service location specified, and a sample WSDL file for Retailer WS-I sample application.

    WS-I is also requesting public comment to ensure quality and broad applicability of the testing tools. Feedback should be directed to Final versions of the testing tools are expected this summer, closely following the final release of the WS-I Basic Profile 1.0.