Testing Reactive Microservices With Spring-Webflux

January 16, 2018

Reactive Microservices is a hot term these days. Every developer is trying to implement software around this new architecture pattern. But if you have already deployed microservices into production and manage a decent amount of load, you might have noticed that transactions that involve chained RPC calls may be a risk.

What is a simple perceptron? Read here to understand a basic Code Java Example of a Simple Perceptron.

There is a pattern call circuit-breaker (https://martinfowler.com/bliki/CircuitBreaker.html) that handles the latter problem. But, at the expense of memory, its usage and thread blocking.

When an RPC call is waiting for another service response, a thread is blocked, and memory is being allocated. What happens when you have to manage hundreds of incoming request per seconds?

A few years ago concepts like Non-Blocking with Event Loop callbacks were born. They run in a single thread and achieve impressive concurrency metrics. NodeJS has proven us that we can do concurrency can without multithreading implementation.

Reactive Programming With Spring Framework

“In computing, reactive programming is an asynchronous programming paradigm concerned with data streams and the propagation of change”. (from Wikipedia)

The concept of reacting programming is not new. But Spring Framework started to address it in the latest version (release notes). There is a new module called spring-webflux that contains support for reactive HTTP and WebSocket clients. As well as for reactive server web applications including REST, HTML browser, and WebSocket style interactions.

Here a few links If you want to go deeper into the subject:

There are a lot of use cases that Reactive Paradigm can solve. In this post, I will go deeper into one concern that affects transactions with chained RPC rest calls.

Typical Use Case

Why not implement this use case?

End users call a Restful operation that as part of its transaction calls another Restful service. The main issue behind this pattern is how to handle the request if the invoked service, in this case, Service B, doesn’t respond properly.

Netflix, for instance, has developed a library that now is part of Spring-Cloud Framework and a standard for the community called Hystrix. Circuit Breaker pattern will ensure that Service A will continue to respond even if Service B starts failing.

But what happened to thread and memory usage? Indeed when an RPC call takes place, the invoking thread is blocked until its receives an answer or timeout. In high I/O systems, this is a waste of resources that could have a better use.

For this particular problem, I have implemented two identical services. One with a traditional approach and another with Reactive programming.

RestTemplate implementation & Tomcat

We all know how to implement this call.

Reactive Web Client alternative & netty

Here is one way to do it with spring-webflux. I also added reactive MongoDB support

Build.gradle:

Important -> spring-webflux is available since spring boot 2.0.0!!!

Testing our apps

Time for testing. I deployed in server 1 both apps and one MongoDB. In server 2 I deployed the second service (ServiceB).

I wrote my stress tests with Gatling (https://gatling.io/). The scenario I prepared was very simple. I ramped up users from 0 to 1000 during 15 minutes.

Spring 4 or Spring MVC results

Highlights:

12% of the requests respond under 800ms
Response time crashes when the load surpass 350 requests/sec
Throughput is around 221 rps.
43% of the request KO
Until 300 rps, mean response time was 300ms
Mean response time is 20 seconds

Spring 5 with webflux Results (Reactive)

Highlights

35% of the requests respond under 800ms
Response time crashes when the load surpass 520 requests/sec
Throughput is around 362 rps.
18% of the request KO
Until 500 rps, mean response time was 20ms
Mean response time is 5 seconds.

Spring-webflux looks very promising. Even with this simple use case, the improvements are impressive. Throughput and average response time improve significantly. As throughput rises, errors go down from 48% to only 18%.

The combination of Servlet 3.1 with non-blocking IO support and async runtime such as Netty overcomes over the traditional package.

On the other hand, code gets much more complex and follow it or debugging it may be not so simple. I tested other simple use cases such as CRUD restful, and I didn’t see any improvements. What I rapidly understand as everything in life, a solution wouldn’t deal with every problem.

In my humble opinion Spring boot, 2.0 M6 have a long way before I can even think of migrating my services. At this moment, there is no compatibility with Spring-Cloud or full Actuator. There is almost no documentation about webflux, so a simple error could make you suffer a headache. I am impatient to know how Spring 5 evolves.

If you need to scale a lot in high blocking IO Use case, this is the solution you must pursue. Netflix, one of the sites with more traffic on the web, has started this journey to migrate their ApiGateway Zuul to a non-blocking system. There is an interesting article about it in this link.

Another performance tests found on the web:

https://tech.willhaben.at/reactive-java-performance-comparison-c4d248c8d21f

http://blog.ippon.tech/spring-5-webflux-performance-tests/