K6: An Open-Source Load Testing Tool for Engineering

During the year, I had the pleasure of working on the creation of a new Product for one of my clients. The Product has had a very positive impact on their business and, interestingly, will soon replace something I helped build many, many years ago. The original version served its purpose very well mind you, but times have moved on. The new Product was built with modern app standards, consisting of a stateless API backend (Kubernetes, not Serverless) and a React UI.

My role on the project was to validate that the performance of the API backend would perform well under various load conditions. The client already had some JMeter tests being used onsite, but they had little skills in-house to maintain them in the future. I wanted to establish a testing approach that would grow well alongside the future enhancements and deliveries of the Product, so instead of JMeter, I turned to K6. K6 is an “Open-source load testing tool for engineering teams”. While K6 has a Go based runtime, it presents to the test script developer as JavaScript.

Advantages Over JMeter:

• WYSIWYG. What I develop and see on my IDE as JavaScript is exactly what I check into source control and see for history and audit. JMeter on the other hand looks like one thing in the IDE but is checked in quite differently as XML.
• Because it’s JavaScript, I instantly have more people on the team that would be comfortable contributing to and maintaining the suite. I am a huge advocate of testing early and often, and the easier I can shift-left this activity, the better. Developers who can easily adapt to JavaScript now have no reason not to deliver both code and test in parallel.
• The underlying Go runtime is significantly more efficient than Java. I can run large tests with many API calls, with far less resources than I would require with JMeter.

The rest of this writeup isn’t a beat-up on JMeter, though Reselman’s article, ‘Why JMeter Sucks for Developers’, is a good read. On the other hand, K6 is tagged as “The best developer experience for load testing”. I started off with K6 by reading through most of the documentation before commencing any test development. I wanted to get well-versed with its overall capabilities first and form the framework I wanted in my mind. When I did begin, things happened very quickly.

My test solution supported Authentication, multiple environments, multiple API calls with versioning, orchestration and various Workload Models, several datasets and of course, comments throughout. The bulk of the suite consists of 1 easy to read JavaScript file and 1 equally easy to read JSON file. Both of which check into source control and remained the same.

The client was also using Grafana for some dashboarding. Out of the box, K6 sends telemetry to Grafana via InfluxDB. Perfect. I obtained a starter dashboard off the internet and added support for both dynamic environment & dynamic test suite selection as I wanted to reuse the single view for other projects. K6 easily supports both using ‘tags’. The icing on the cake came when Grafana Labs acquired K6 during this period. This was great validation with the client that we were on the right track. It was also around this time that I updated one of the architects on K6 and my progress. I’ll never forget his response when he said, ‘you had me at hello’. Priceless!

A summary report generated at the end of a K6 test includes various metrics such as transfer volumes, times, and percentiles. While it’s possible to add more telemetry to the report, the important part is establishing checks or thresholds in the test script. For example, 95th percentile < 500ms, errors < 0.1% etc. This way, if they breach, the test fails. With a resulting binary pass/fail output available, it is then possible to include the execution of a K6 test into a CI/CD pipeline. At that point, I only need to refer to the Grafana views for triage.

The client was using GitLab for CI/CD, and I was able to leverage their Kubernetes environment as a Runner. I was also able to automatically pull the latest K6 image from Docker Hub at the start of execution. The cherry on top came when GitLab endorsed K6 for Load Testing in pipelines and added visibility for previous vs current checks & metrics for decision making use, during the merge processes. More great validation with the client.

While K6 has fewer options, plugins, and protocols than JMeter, it should have what you need to load test a modern application. What it lacks, positively, is the 20+ years of bloat and often hard slog to get going. It does support UI testing, but I have not used that yet. And K6 does offer Cloud services with Runners and Dashboards, but these are completely optional. I have been very happy with the Open-source offering.

To learn more about K6, visit http://k6.io, and feel free to contact us if you have any questions.