Three Tools for Testing Software During Development

March 12, 2021

Several factors go into choosing the right tools for testing software. Here we discuss three popular ones that you can use today.

Today’s development teams should be familiar with the meaning of being agile. But, what does it mean? An agile team is based on iterative and incremental development with only a few stages. Every time we iterate, we evolve the product through iteration.

One of these stages is development. This stage focuses on definitions that our stakeholders made before. These definitions are behaviors or actions that the software should do. In developing software, we often create new features, fix bugs, and refactor code. In any case, it is an excellent practice to tests our code. This way, we will deliver a good quality of software and be even faster in fixing bugs.

Several factors go into choosing the right testing tool for the application that needs to be tested. Many questions will need to be answered to decide which testing tool is the best suit to test an application’s changes, releases, and upgrades.

Background

Several automated tools for testing software products are available in the marketplace. This white paper will discuss the three highly-regarded tools that help us in our development stage: SoapUI, Postman, and MockServer.

Understanding some of the key differences of each tool for testing software will help determine which one is best suited for your needs.

Key Factors to Consider

Development Speed
Ease of Use
Investment
Reports
Browsers and Operating Systems
General Considerations

If you’re a developer who’s always working to stay up to date and be aware of the latest technology, we’d like to talk with you. Are you looking for a challenging opportunity to further your career? Check out our open positions.

Three Tools for Testing Software

SoapUI

SoapUI is an open-source tool and offers support for SOAP and REST web services. We use this tool to test our internal systems ( SOAP web services). Today, it’s tough to find a public soap web service, so here we list three for you to investigate later:

I’ll take the CountryInfo1 web service to show you how you could test using this tool. SoapUI is very powerful with SOAP protocol because you could make several requests in a specific order and wait until a request is processed. SmartBear designed soapUI to make story tests and functional tests.

Remember: many functional tests will conform to a story test, and a story could have many scenarios.

In this example, I will explain which features are helpful for me. One feature that I did not find in Postman is that SoapUI lets you load all methods from a WSDL3 URL4 and make a template for sample requests, mock servers, and a set of a dummy test suite.

The example soapUI.xml has only One long TestCase with ten Test Steps. Each step has a different feature that is frequently used. You could download this example and try to import it in this way:

Click the menu “File >> Import Project”. This will open a pop-up window so you could select an XML file to import.

Next, when we select the file, we could see something like this: A new Test Case contains one or many Test Suites (the green block), composed of Ten Test steps (the purple block).

[SoapUI] Describing the ten steps

Step 1: I called “Get A random number”, a groovy script that lets you extend soapUI, adding more flexibility. The scripts use features from groovy. It’s not only a client for web services; also, you could program a more complex logic here.
Step 2: “GetCountryIsoCodeByCountryName” is a Soap Request which takes one parameter from the scope of TestCase.
Step 3: “UsingCountryCodeIso” is a Property Transfer. You could read some data from any Soap Response within the XPath5 path and save it like a case or project property with this element. Why did we do this? Because later in another test which depends on the first one we’ll use it.
Step 4: After You execute a property transfer, you want to use it with a Soap Request.
Step 5: Sleep/Delay let you wait for a time set by default 1000ms and continue with the next step
Step 6: It is a Groovy script used to get a random number and choose a country from the list.
Step 7: It is a Soap Request, which uses a property value from the Test Case scope, and the groovy script in step 6 fills this property.
Step 8: “IF_Condition” is an element that lets you iterate over the steps whether a condition is true. The next step will be the target defined in this element. For example, in this element, the target element is step 5, so the functional test will be iterating until we will get in the response of step 7 a country code iso = AR (Argentine)
Step 9: It is a groovy script to add some comments or final behaviors.

So in the steps below, we describe some feature we used very often:

A friendly interface to load WSDL by URL, a soap client that lets you use properties in any scope: TestCase or Project to request a remote address.
Groovy script: you could extend the functionality from soapUI, using groovy scripts to set variables, make iterations, test validations, test assertions.
Property Transfer: you could take data from one response and use it in another request; this shows you that soap request has some dependencies and even a sort of order of execution.
IF_Condition: in any case, you don’t want to add a groovy script; you could use this to iterate over steps until the condition defined in the element “IF_Condition” it’s met. This element usually is in the company with the delay element to wait N milliseconds to request the same IP address or domain.

Postman

Postman was made with REST services to load a WSDL, but we can import it from swagger.io.

Postman has many features that I love. You could log in to your account and keep synchronized with all your collections of Requests, so if you have to move between notebooks only with login into your account, you could retrieve the collection you saved before. In this way, you will not lose this data again.

Postman lets you generate documentation from your API, share it with a coworker and keep information organized with workspaces. Also, you could easily save your environment data. You could have many servers with specific environment variables and switch quickly to a new environment configuration every time you need it (green block in the image below).

Postman has a mock server that lets you try it with a few constraints. In a Postman free account, you can only make 1000 calls to your new mock server. This app will generate a new URL: https://6a61f89f-f542-4766-8edf-740316658aab.mock.pstmn.io, ready to use with your development environment. Mock servers are public by default. Public mock servers are accessible to anyone without a Postman API key.

This mock could be edited only in the Postman web app (Workspace>> Mock Server) when you are logged into your account. You could edit the response and have the log for every Call made it to the MockServer in other features.

The only things to take into account are the policies about data and privacy. Postman collects some data about your Collection Requests, usages. If you want to turn off this feature disabling all tracking, you could do it switch off in the postman configuration “Sending anonymous data” (File >> Settings), which you could see in the image below:

SecurityTrails currently offers an API (https://api.securitytrails.com/v1/) usually is used by SIEM systems and security automation products and lets you use it for free. You would make 50 query requests looking in his 12 years of DNS records for any given domain. The API uses JSON as the data format for any data input outside of headers so that any response will be in JSON format. You could set headers, make a query string in the URL, for example:

In the image above, I set the API key in the HTTP header (3) and add in the URL an environment variable(2), setting an environment with vars(1) and all this was saved in a collection(4) inside of a workspace (0): for more details, you can download from here: Postman Security Trails7.

About assertions and tests, this tool lets you do some tests to an API. I did some examples with Postman to show you how you could use it and test many endpoints with assertions.

The image above is an individual request to set the access token environment(1) through the Javascript code. In the tab “Tests” (4), I set up a test that evaluates when the HTTP status response is equal to 200; also, you could write some javascript code to extend the rest-client application and make chained requests, debug with javascript command console.log (2), and see in the postman console(5) all details about the current execution.

The big picture about a test suite or Collection Runner after his execution is in the image below; the first step is to click on the button Run (1) in the collection view, a second window appears “Collection Runner” (2) and it will indicate How many tests will be running(4) and which one was failed, each request is a step, and in the “Postman Console,” you could see the details in each case, even use javascript code to debug. Also, you could set delayed and iterations, i.e., how many times you want to execute this collection (5). In this example, I set this variable to one time.

About authentication, you could use many methods:

Sending apikey by header (Postman Security Trails7)

Has basic authentication with HTTP

In this, you could fill the password field with the string in plain text, but Postman insists on their users keep the API key, passwords, auth token in an environment variable; thus, we can share the collections with no risks in sharing our identity or ID. You could see an example here: Basic HTTP Auth Censys API

Oauth with token

Today many sites are using OAuth v2, so it was not difficult to find a public API, the first one I try it, was ok, so here is the postman collection with an example for Test Assertions, Environment variables, this collection will be running like a Test Cases with three steps. The third step depends on the second and this one depends on the first one, and so on.

OAuth Postman Collection Petfinder API

MockServer

In some cases, we don’t need to have all elements accessible through the network, or we don’t have a third-party server up to dispatch the request that our rest-client will do. In this case, the Mock Server is used to define all request and response when is matched “MockServer” response with the configuration and request defined before.

This tool could be used as a proxy; MockServer has good documentation here: mock-server.com. In this article, I’ll explain how to use it for Mocking. A MockServer ready to use could be downloaded from hub.docker.com:

There are many ways to use the MockServer. For more information, you could consult here: mock-server.com/getting_started.html. With the command below, we are executing a MockServer without defining requests, matchings, and responses:

After execute you will see executing docker ps

By default, MockServer does not offer an interactive bash, but you could configure it to execute in that way. MockServer can be run using docker-compose by adding the container as a service. MockServer can be controlled in the following ways: REST API, Java client, Javascript Client. In this example, we’ll use Rest API (swagger format, here you could get the curl format to make the request).

Get MockServer Status

Creating An expectation: I created a docker-compose.yml with only one service, you could download from here: docker-compose.yml and try MockServer in the way that was explained here:

After creating it, I could request the postman app to this path; also, you could use curl http://localhost:1080/helloIntraway and get the response defined before.

Each request created through API (create expectation) could persist only if you share the volume where the files are saved in the docker-compose.yml. By default, in MockServer, expectations are only held in memory and the file path used to persist expectations can be configured using the persistedExpectationsPath configuration property.

The environment variable MOCKSERVER_PERSISTED_EXPECTATIONS_PATH and MOCKSERVER_INITIALIZATION_JSON_PATH should have the same value. For more information, you could download the example from the repository in GitLab: Blogspot’s Repo

There is no one perfect tool for everyone. Every organization has different requirements; the key is to read through the descriptions carefully and decide which best fits your team’s needs. They’re just a small sampling of the open-source API testing tools out there—a high-level overview of some tools that we used.

Conclusion of Three Tools for Testing Software

In the end, the choice is yours. While all three tools for testing software will enable your company to create your test cases, each has its distinct advantages and disadvantages. Regardless of your choice, remember to keep your projects agile.

References:

1 CountryInfoService Web Service

http://webservices.oorsprong.org/websamples.countryinfo/CountryInfoService.wso?WSDL

2 Functional tests: “Functional testing consists of a sequence of tests that define entry values for an operation and observe if the result is what was expected.” Raul Sidnei Wazlawick, in Object-Oriented Analysis and Design for Information Systems, 2014.

3 WSDL: The Web Services Description Language

4 URL: A Uniform Resource Locator

5 Xpath: “XPath stands for XML Path Language. It uses a non-XML syntax to provide a flexible way of addressing (pointing to) different parts of an XML document” https://developer.mozilla.org/en-US/docs/Web/XPath

6 Story Test: “Stories are written by the whole team and used as both requirement and executable test cases. It’s a way to perform TDD with clarity..” página 42, Test-Driven Java Development https://books.google.com.ar/books?id=tRl1CgAAQBAJ

7 Security Trails API docs: https://docs.securitytrails.com/docs/examples