Should you use AI to help you migrate test automation code? And what should you actually migrate, the tests coverage hasn't changed. In this episode we discus show abstractions and AI can be used to migrate... and discuss when you shouldn't.
Welcome to The Evil Tester Show! In this episode, host Alan Richardson dives into the complex world of test automation migrations. Have you ever wondered what it really takes to move your automated test execution code from one tool or language to another—like switching from WebDriver to Playwright, or migrating from Java to TypeScript? Alan breaks down the pitfalls, challenges, and best practices you need to consider before taking the leap. He explains why migrating isn’t just about copying test cases, how abstraction layers can save you time and headaches, and why using AI and solid design principles can streamline your transition. Whether you’re facing unsupported tools, evolving frameworks, or strategic changes in your testing approach, this episode offers practical advice to plan and execute a seamless migration—without burying new problems beneath old ones.
00:00 Migration Challenges
02:43 Tool Evaluation
04:05 Migrating to Playwright: Considerations
06:00 Migration Process
06:25 Migrate: Easy First, Hardest Next
09:37 Effective Migration Strategies for Tests
10:23 Focusing Abstractions
14:39 Optimize Test Code Migration
15:44 Focus on Abstraction, Not Auto-Healing
**1. Why Migrate—And When You Really Shouldn’t** Before any big move, Alan urges teams to get their “why” straight. Is your current tool unsupported? Is your framework truly incompatible, or are you missing some hidden potential? Migrate for the right reasons and make sure your decision isn’t just papering over problems that could follow you to the next tool.
**2. Don’t Confuse Migration with a Rewrite** Too many teams treat migration like a rewrite—often with disastrous results. Alan emphasizes the importance of planning ahead, solving existing flakiness and coverage issues _before_ you move, and carefully evaluating all options (not just the shiny new tool you think you want).
**3. The Secret Weapon: Abstraction Layers** The podcast’s biggest takeaway: Don’t migrate “test cases”—migrate _abstractions_. If your tests are full of direct calls like `webdriver.openPage()`, you’ve got work to do. Build out robust abstraction layers (think page objects or logical user flows) and keep your tests clean. When it comes time to migrate, you’ll only need to move those underlying layers, not thousands of individual test case scripts.
**4. Taming Flakiness and the Risks of Retries** Migration is not the time to rely on self-healing tests or retries. Any test flakiness _must_ be rooted out and fixed before porting code. Bringing instability into a new stack only multiplies headaches later.
**5. Harnessing AI—But Stay in Control** AI-assisted migration really shines at mapping old code to new, but Alan warns against “agentic” (hands-off) approaches. Use AI as a powerful tool, not as the driver—you need understanding and control to ensure things work reliably in CI/CD pipelines.
**6. Learn Fast: Tackle the Hardest Stuff Early** Pro tip: Once you’re ready, start your migration with the simplest test, just to get going—then dive into the hardest, flakiest, most complex workflows. You’ll uncover potential blockers early and kick-start team learning.
“We’re not migrating test cases when we change a tool. We’re migrating the physical interaction layer with our application... ”
South Africa