STARWEST Conference Recap

It was a great chance to attend some tutorials and talks about professional development, to hear about and see the latest tools and to meet some new prospective clients and hear what questions they are asking.

One of the highlights of STARWEST for me was the talk: "Industrial Strength Test Automation - When You Should and How You Can" by Mike Duskis of CyberGRX.

I spoke with Mike and he graciously agreed to share the link to his slide deck, which does a really solid job of laying out his definition of industrial strength test automation, when it makes sense to build it, when it makes sense not to build it, and shows off one specific technique he has developed to help make your testing infrastructure more of an industrial strength design. I strongly recommend you check it out.

After some interesting exchanges of views on LinkedIn, Mike invited me to "attend and heckle" (his words) his talk. I found myself strongly agreeing with almost everything in it. So my heckling was probably substandard. But, I would offer four quick thoughts:

1. I would offer this simple heuristic for when to decide to craft some industrial strength test automation; when you find yourself wondering if you need tests for your tests or tests for your test infrastructure / test framework - congrats you need industrial strength test automation.
   
2. Mike does a great job of highlighting emergent behavior as crucial cause of issues in software projects and the detection of them as a key priority for software testers. If this was a new idea for you, I'd strongly recommend some time spent on the broader topic. It was a great job of highlighting the importance of paying attention to topics outside the specific focused area of expertise and integrating them into your process.
   
3. Mike introduced a tool and approach that he uses to help make it possible to more deeply investigate the type of long tail bugs that are often dismissed as test flakiness. Industrial strength test automation isn't something you build once and get, its also a standard you hold yourself to as you use the automation to check the product. Building industrial strength systems may reduce maintenance but diving deep instead of just looking for green will take more time during your testing cycles. Make sure you can allocate the resources to using, interpreting and investigating these long tail bugs before you spend a bunch of time building the tools that enable that kind of investigation.
   
4. Mike correctly points out that industrial strength test automation requires treating the systems you are building as a software project of their own and that this requires the same things as all other good development projects: a product owner with a clear vision, architects, developers, people with ops experience and testers of the project. If you don't have the time / skills / resources to actually do all of this then I'd strongly suggest not trying to reinvent that wheel. Either use a open source tools that have this or purchase software / services that can provide the robust tools and systems that are required for these kinds of systems.
   
   I've been guilty myself of building too many parts of solutions instead of spending my limited resources on the most important things to my testing project. I have had a lot more success in my own career building on top of advanced open source frameworks. Too often I mistook being the person at my company who knew the most about how to do something as being the person in the world who knew the most about how to do something.