Analyzing the blog’s topic, what are these tests and what do they ensure?
They are a set of tests that try to emulate the real environment transaction load level that the solution should handle, but these events are usually executed on laboratory machines. Often, running the tests in similar environments such as production it is not possible due to hardware resources availability , but you can avoid this issue by comparing the available resources against the production environment and making calculations to extrapolate and get the result.
An important specific topic to clarify is that these tests are not functional, which meticulously test the result of each command executed and usually they are not performed with simulators, but by people.
You can also execute the particular scenario that stresses test transactions that are badly formed. This test is intended to analyze the impact on the module to be tested, in order to validate that it remains healthy after and during an attack.
For the stress test, automated transactions generators are commonly used as data entry and response simulators to close the e2e stage.
The main test scenarios are:
- Snowball Effect: This is a test that duplicates the load for a pre-defined period of time. It is used to test the behavior upon transactions avalanches.
- Constant load: Constant transactions load for a period of time.
- Variable load: Variable transactions load for a period of time.
- Limit discover: This test is oriented to get the upper limit of transactions before the product performance (response times) is degraded. This test will give us important information to know the solution’s limits.
- Mix: Set of test that comes together “Snowball Effect”, “Constant load”, “Variable load” but without reaching the result obtained in Limit discover test.
Now, once we have listed scenarios and we are running stress tests, what is the next step? which are the important KPIs to watch?
For all scenarios shown above, KPIs should be defined before executing any test, those will serve to evaluate the module’s behavior under stress situations. Usually, those are:
- Resources KPIs
- Operating System global consumption (memory, CPU cores, iowait the case of access to file systems)
- Resource consumption per module (v leak , CPU, iowait for the case of access filesystems)
- Application
- Response time (minimum, maximum and average)
- Input Orders per second
- Orders successfully answered
- Orders answered with errors
- Output Orders per second
With this type of test, based on the guidelines shown above, we can ensure that our application can meet both functional and stress expectations.
Hope that this technique is useful for everyone!
Please, remember to leave your opinion in the section below. If you would like to know more about this topic, feel free to contact me at santiago.gonzalez@intraway.com.
