AI, AI, AI… What about Testing, Testing, Testing?

What do a university, a health agency, and a major meat processor and exporter have in common? Quite a lot, it turns out. They all share a keen interest in enhancing quality engineering practices, particularly in the application of AI in testing.

Our October edition of the Dunedin Software Testing Professionals meetup featured two presentations on the impact and use of AI in testing, along with insights into the testing tools landscape at Silver Fern Farms. This served as a segue into our next event, which will focus on testing toolchains and reporting. It was great to witness the passion for quality engineering in the room, highlighted by engaging discussions post-presentation.

The meetup began with Mei Reyes-Tsai, General Manager of Technology and Delivery, who shared TTC's research on AI in testing. She provided an engaging overview of the profession's evolution and the current AI landscape. Mei presented a variety of potential use cases for integrating AI into testing, emphasizing the growing need for automation in this area. One key takeaway was the value of efficiently running multiple scenarios to assess outcomes, with at least ten use cases discussed.

You can view her presentation here

Following Mei, Matthew James from the University of Otago detailed their real-life implementation of AI, and the testing processes used to ensure both functionality and compliance. Though the two presentations were not planned to dovetail, the similarity in their messages was impressive. In developing and testing the AI-based Study Match function (go ahead—give it a try! 😊), many of the concepts Mei presented were applied in practice. These included using guardrails in AI prompts to limit response variation, focusing the AI engine on a specific content set, and leveraging automation to rapidly execute multiple test scenarios for quick and easy assessment of response variances. The tool has proven to be intuitive and effective, sparking my own interest in studying at Otago next year!

Matthew’s presentation can be viewed here

Recognizing that excessive AI use can sometimes have drawbacks, we concluded the meetup with a well-articulated presentation from Matt Tulloch of Silver Fern Farms
on their testing tool landscape, which also set the stage for our next Dunedin Test Professionals meeting in February 2025. Matt’s overview of Silver Fern Farms' quality engineering approach across 'Run', 'Grow', and 'Transform,'
along with their tooling landscape, was both informative and thought-provoking, paving the way for our next meetup.

You can view his presentation here

We encourage everyone to explore Otago University’s adoption of AI and consider their innovative testing approach. If you’d like to learn more about the ten use cases we identified for AI in testing and how they currently rate, feel free to reach out.

At our next gathering, we will delve deeper into testing tool landscapes, focusing on metrics and their utility. We look forward to seeing you at our next meetup in February 2025!