Forbidden Words of Web Services

Abstract

Did you ever have to send raw data through a web service? High level, string-oriented abstractions sometimes are resistant to binary data. SOAP web services are particularly sensitive in this case. This battle has its pitfalls, but with a little help it can be won flawlessly and elegantly at the same time.

Sending Binary Data through SOAP

Working with hardware devices from the software standpoint sometimes forces us to get our hands dirty with low-level data. Device firmware is usually written in programming languages that in the end don’t care about charset and symbols: they treat the data as plain bytes. But taking that data to high level interfaces flawlessly sometimes poses a problem: databases and interpreted languages tend to try to convert that data into a known charset, and we end up losing data in that conversion, or even worse: entirely breaking our program, rendering our software unstable.

XML Formatting and Limited Charset

SOAP Web Services pose a particular problem: their message protocol prohibits certain characters from the start. If at some point inside a request or a response we put non-sanitized data, it will probably break the execution, either at the server or at the client side.

Alternatives

• Just filter out the special characters of the data.
• Use CDATA (Character Data) blocks.
• Use a string type, and programmatically encode and decode the data at both ends.

The first alternative is the cheapest and dirtiest one. Setting aside the fact that it is not correct to do that (because you are removing data), you can probably pull it out with this approach. But not without collateral damage: not showing the full data can have unexpected consequences. If you are using that data later to match against the original source, it will not match anymore.

The second alternative relies on the CDATA specification of the XML standard. It will do well for reserved characters of the XML spec, but not for control characters outside the scope of the charset, which is independent of the XML spec and depends mostly on the deployment setup.

The third one will solve the problems, but forces you to modify existing code (in the case of already implemented solutions) and you are forcing the client-side implementation to do the extra job of decoding something that in the WSDL you are declaring to be a plain string, which may generate inconsistencies later.

Our approach

It is not well known, but SOAP standard has predefined data types for this kind of scenario: base64Binary. And it is even less known that the SOAP libraries of PHP can handle that for us fully automatically.

Working example

Prerequisites: you must use a WSDL file and use the SoapServer and SoapClient objects in WSDL mode. Non-WSDL mode (also called XML-RPC mode) is not supported because you don’t have a way to specify data types for the attributes.

In order to run this example you must place all the files on your web server root, and have a web server that supports PHP installed locally. Preferably it should be Apache.

First let’s create a WSDL file to define our interface:

example.wsdl

<wsdl:definitions xmlns:tns=”Intraway/Base64Encode” targetNamespace=”Intraway/Base64Encode” xmlns:soap=”http://schemas.xmlsoap.org/wsdl/soap/” xmlns:s=”http://www.w3.org/2001/XMLSchema” xmlns:wsdl=”http://schemas.xmlsoap.org/wsdl/” xmlns:soapenc=”http://schemas.xmlsoap.org/soap/encoding/”>
<wsdl:types>
<s:schema targetNamespace=”Intraway/Base64Encode”>
</s:schema>
</wsdl:types>
<wsdl:message name=”base64ParameterSoapIn”>
<wsdl:part name=”string” type=”s:base64Binary” />
</wsdl:message>
<wsdl:message name=”base64ParameterSoapOut”>
<wsdl:part name=”return” type=”s:string” />
</wsdl:message>
<wsdl:message name=”base64ResponseSoapIn”>
<wsdl:part name=”string” type=”s:string” />
</wsdl:message>
<wsdl:message name=”base64ResponseSoapOut”>
<wsdl:part name=”return” type=”s:base64Binary” />
</wsdl:message>
<wsdl:portType name=”Base64EncodeExampleSoap”>
<wsdl:operation name=”base64Parameter”>
<wsdl:documentation>
base64Binary used as parameter
</wsdl:documentation>
<wsdl:input message=”tns:base64ParameterSoapIn” />
<wsdl:output message=”tns:base64ParameterSoapOut” />
</wsdl:operation>
<wsdl:operation name=”base64Response”>
<wsdl:documentation>
base64Binary used in response
</wsdl:documentation>
<wsdl:input message=”tns:base64ResponseSoapIn” />
<wsdl:output message=”tns:base64ResponseSoapOut” />
</wsdl:operation>
</wsdl:portType>
<wsdl:binding name=”Base64EncodeExampleSoap” type=”tns:Base64EncodeExampleSoap”>
<soap:binding transport=”http://schemas.xmlsoap.org/soap/http” style=”rpc” />
<wsdl:operation name=”base64Parameter”>
<soap:operation soapAction=”Intraway/Base64Encodebase64Parameter” />
<wsdl:input>
<soap:body use=”encoded” encodingStyle=”http://schemas.xmlsoap.org/soap/encoding/” namespace=”Intraway/Base64Encode” parts=”string” />
</wsdl:input>
<wsdl:output>
<soap:body use=”encoded” encodingStyle=”http://schemas.xmlsoap.org/soap/encoding/” namespace=”Intraway/Base64Encode” parts=”return” />
</wsdl:output>
</wsdl:operation>
<wsdl:operation name=”base64Response”>
<soap:operation soapAction=”Intraway/Base64Encodebase64Response” />
<wsdl:input>
<soap:body use=”encoded” encodingStyle=”http://schemas.xmlsoap.org/soap/encoding/” namespace=”Intraway/Base64Encode” parts=”string” />
</wsdl:input>
<wsdl:output>
<soap:body use=”encoded” encodingStyle=”http://schemas.xmlsoap.org/soap/encoding/” namespace=”Intraway/Base64Encode” parts=”return” />
</wsdl:output>
</wsdl:operation>
</wsdl:binding>
<wsdl:service name=”Base64EncodeExample”>
<wsdl:documentation>
Example to show the base64Encode capabilities of the php’s native SoapServer class.
</wsdl:documentation>
<wsdl:port name=”Base64EncodeExampleSoap” binding=”tns:Base64EncodeExampleSoap”>
<soap:address location=”http://localhost/example_server.php” />
</wsdl:port>
</wsdl:service>
</wsdl:definitions>

Note the data types used. This definition suits the most commonly used cases: binary data as a parameter, and as a response.

Client side code:
example_client.php

<?php

$location = ‘http://’ . $_SERVER[‘HTTP_HOST’];

$SoapClient = new SoapClient($location . ‘/example.wsdl’, array(‘trace’ => 1));

$base64Parameter = ‘this text will be automatically converted to base64 before sending the request’;

$base64ParameterResult = $SoapClient->base64Parameter($base64Parameter);

$DomRequest = new DOMDocument();
$DomRequest->preserveWhiteSpace = false;
$DomRequest->formatOutput = true;
$DomRequest->loadXML($SoapClient->__getLastRequest());

$DomResponse = new DOMDocument();
$DomResponse->preserveWhiteSpace = false;
$DomResponse->formatOutput = true;
$DomResponse->loadXML($SoapClient->__getLastResponse());

echo ‘<pre>’;

echo “nnParameter as seen by PHP : ‘$base64Parameter’nn”;

echo “base64Parameter Request:nn “;

echo htmlentities($DomRequest->saveXML());

echo “nnbase64Parameter Response:nn”;

echo htmlentities($DomResponse->saveXML());

echo ‘</pre>’;

$stringParameter = ‘this text will be returned by the server in base64, and decoded automatically by the SoapClient class’;

$base64ResponseResult = $SoapClient->base64Response($stringParameter);

$DomRequest = new DOMDocument();
$DomRequest->preserveWhiteSpace = false;
$DomRequest->formatOutput = true;
$DomRequest->loadXML($SoapClient->__getLastRequest());

$DomResponse = new DOMDocument();
$DomResponse->preserveWhiteSpace = false;
$DomResponse->formatOutput = true;
$DomResponse->loadXML($SoapClient->__getLastResponse());

echo ‘<pre>’;

echo “base64Response Request:nn”;

echo htmlentities($DomRequest->saveXML());

echo “nnbase64Response Response:nn”;

echo htmlentities($DomResponse->saveXML());

echo “nnResponse as seen by PHP : ‘$base64ResponseResult’nn”;

echo ‘</pre>’;

Server side code:
example_server.php

<?php

$location = ‘http://’ . $_SERVER[‘HTTP_HOST’] . $_SERVER[‘REQUEST_URI’];

/**
* This class implements the Web Service public methods
*/
class WebServiceHandler
{

/**
* base64Binary used as parameter
*
* @param string $string
*
* @return string
*/
public function base64Parameter($string)
{
return $string;
}

/**
* base64Binary used in response
*
* @param string $string
*
* @return string
*/
public function base64Response($string)
{
return $string;
}

}

/**
* Options for the SoapServer
* soap_version must match the WSDL definition
*/
$options = array(‘soap_version’ => SOAP_1_1,
‘encoding’ => ‘UTF-8’,
‘uri’ => $location,
‘location’ => $location,
‘trace’ => 1
);

$SoapServer = new SoapServer(‘http://’ . $_SERVER[‘HTTP_HOST’] . ‘/example.wsdl’, $options);

$Object = new WebServiceHandler();
$SoapServer->setObject($Object);
$SoapServer->handle();

Once you have put those files on your web server root, simply enter http://localhost/example_client.php

Here you will find each request and response from the web service calls, with their corresponding encoding and explanation. What happens is described in the following flowchart:

All the encoding and decoding are done “behind the scenes”, inside the native PHP objects, allowing you to work with the data as simple strings on both sides, and warranting that your message format will honor the original data.

Use Cases

We use this approach with data that comes from SNMP requests, and has to be preserved intact through the entire application flow. For example: device’s model, vendor, and hardware version.

Small binary files such as images and bitmaps are good candidates for this technique. You should keep in mind though that SOAP requests have limits on payload size, usually around 2 Megabytes. Responses are less restrictive about maximum size. Anyway, it is good to keep an eye on size to avoid issues with network transports, such as a response timing out.

Conclusion

With this approach to raw data you can accomplish a clear and elegant solution to a common problem in the telecommunications industry, by using standard techniques.

When using an existing spec we have more possibilities of interoperation with other programming languages and OSS/BSS tools, avoiding time-consuming issues on systems integration.

Do you want to know more powerful secrets of PHP? Come to work with us! http://www.linkedin.com/company/intraway-corp