One of the questions we at NuWave are often asked about our products is "What is the performance like?". While our own internal benchmarking shows that LightWave Server™ and LightWave Client™ can process thousands of transactions per second with minimal impact on CPU utilization, the answer to that question is generally a little more complicated.
Any product's performance in any given NonStop environment will be based on many factors, including:
Capabilities of the platform it is running on (CPU, disk etc)
Priority of the process relative to the other processes on the system
Type of work being done by the product (i.e is it I/O bound or CPU bound?)
In the case of products like LightWave Server™ and LIghtWave Client™, you can add to this list the following:
Size of messages being parsed
Complexity of messages being parsed
Network capabilities
Any security implications, e.g TLS sessions being established or signatures being generated as part of the measured period
And there likely will be other specific issues to consider in your environment. Having said all that, LightWave products include a range of features to help you determine how your implementation is performing in your own environment. When combined with standard tools available on the NonStop and elsewhere, it's not too difficult to get an accurate idea of how well your entire solution is performing, and even improve performance, often by addressing some low-hanging fruit.
The following will hopefully give you some guidance on how you might setup a performance benchmarking test that takes into account all of your specific application and environmental requirements.
Standard NonStop tools
Tne most important tool in your NonStop performance meaasurement arsenal is MEASURE. MEASURE is an extremely powerful, and reasonably complex tool, however you can start using it pretty easily and often identify areas of concern without spending months or years learning how to use it. In particular the PROCESSH entity is worth knowing. This entity will show you which parts of your code are being most heavily utilized when running a performance test. It is beyond the scope of this article to get into MEASURE in more detail, but the HPE documentation on this product (here) is extremely helpful.
If your application uses Pathway, you might want to spend some time tuning your Pathway environment. The TS/MP System Management Manual located here has the following useful sections:
Managing PATHMON Process Performance pg 104
Tuning Your System by Using Statistics pg 120
Client-side Tools
There are a number of client tools that can help with performance testing, including:
Apache JMeter - https://jmeter.apache.org/
The Apache JMeter™ application is open source software, a 100% pure Java application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions.
SoapUI - https://www.soapui.org/
Whether open source or commercial, SoapUI testing tools make it easy to create, manage, and execute end-to-end tests on REST, SOAP, & GraphQL APIs, JMS, JDBC, and other web services so you can deliver software faster than ever.
LightWave Performance Tools
Both LightWave Server and LightWave Client have extensive MEASURE support, including support for custom MEASURE counters. Custom counters can gather data such as:
Time spent processing SSL/TLS handshake for a connection
Amount of time spent deserializing the request payload (JSON) into the server IPM request
Total number of requests processed
Total number of bytes sent/received over the network
Amount of time spent signing an API request, where required
Amount of time spent serializing the API request (converting from IPM to JSON format)
Total amount of time processing the request and the response
And many others. The complete list of counters is summarized in the LightWave product documentation available here:
LightWave Server™ - https://docs.nuwavetech.com/display/LWSERVER/Using+MEASURE+Counters
LightWave Client™ - https://docs.nuwavetech.com/display/LWCLIENT/Using+MEASURE+Counters
Other considerations to keep in mind as you are building your performance benchmarking environment include:
If the application uses TLS, test with TLS.
The TLS handshake is expensive. Applications should use persistent connections to minimize connections and TLS handshake. See https://en.wikipedia.org/wiki/HTTP_persistent_connection or https://tools.ietf.org/html/rfc7230#section-6.3
Use messages that are representative of the actual application.
It's not just the size of a message, it's the complexity of the message. A small message with lots of elements may take longer to process than a large message with few elements.
Run load tests over a network representative of the actual application. Then, if possible, run load test with as little network distance as possible (run on localhost if possible). Use the difference to factor out network performance.
Conclusion
Performance benchmark testing is often a complex process, and in some cases might even be considered a dark art. It is important to get performance figures that apply to your own specific application and environment. There are tools that can assist you in this process, and NuWave is very happy to help you better understand the LightWave products in your own organization. We are currently working on a performance demo that we will make available to customers - please get in touch with us if you'd like to know more about this, or any aspect of LightWave performance.
Comments