Playwright (& Selenium) Desktop Application Automation Testing

This quick blog post is to share something I found rather cool and interesting (IMHO).

I’ve successfully been using Playwright test automation for browser-based applications but now need to automate some Windows Desktop applications. If you google Playwright Desktop Application, you’ll find it can’t be done. But it can! (with some help).

While looking into options for the desktop apps, I see the traditional Microsoft one, WinAppDriver, seems to have fallen by the wayside. Plenty of nudges by the tester community, yet radio silence by Microsoft e.g. Future of WinAppDriver? https://github.com/microsoft/WinAppDriver/issues/1704

I then found FlaUI, and it's looking extremely good as a fully capable, maintained and popular option for UIA2 & 3 generations of Windows Desktop Applications. Regardless of slim pickings in this space, FlaUI stands out.

However, FlaUI only comes in the C# flavour. While I could use Playwright .NET to get a common skillset across the two, they are still separate frameworks. Plus, I was really hoping to use Javascript. Regardless, direct use of FlaUI doesn’t solve my Playwright to Desktop Application motivation.

Interestingly (and this is the secret sauce here), FlaUI also has a WebDriver offering: github.com/FlaUI/FlaUI.WebDriver.

Big ups, props, and full credit to the FlaUI community, because the rest of this blog and solution would simply not exist without it!

Running as a cmd .exe server, FlaUI.WebDriver fronts, or proxies the Desktop Application and presents it as a Browser-like application accessible via the HTTP WebDriver protocol.

Using this handy WebDriver collection, I can do things like:

• GET findElement (aka FindFirstDecendant in FlaUI terms)

or

• POST element/<id>/text (aka AsTextBox().Text in FlaUI Terms)

I’m now able to Automate a standard WPF (Windows Presentation Foundation) Windows Desktop Application via HTTP! That’s pretty cool.

You may be thinking, ‘but Playwright uses WebSockets to talk to Browsers, not WebDriver’. That’s OK; it’s fairly trivial to create a lightweight layer to take care of that e.g.

A custom findElement function calling a standard W3C API (WebDriver)

Where ‘using’ is one of the FlaUI.WebDriver supported locator types below and ‘value’ is one of the identifiers visible in a Windows Inspection tool for your required element. One inspection tool example: github.com/FlaUI/FlaUInspect.

Available selector types and their required values

To be more complete, you would also want some other functions like text, element click, keypress etc. I’ve only implemented a small number for what I needed initially.

Selenium uses WebDriver natively, so the solution there is even lighter with that framework as you would not need the WebSocket to WebDriver conversion layer.

Ultimately this is what you get:

Three ways of using FlaUI to automate Desktop Applications

FlaUI.WebDriver technically only works on UIA3 applications currently and is showing as EXPERIMENTAL.

But it works, and right out of the box, it allowed me to run Automation across Windows Desktop Applications, from Playwright using Javascript. Which I think is pretty cool!

Again, full credit to the FlaUI community for providing not only FlaUI, but this awesome bridging layer.