Evolving My Automation Journey: Building a Modern Playwright CI/CD Pipeline

For years, I’ve worked with automation frameworks, mainly Selenium, where Java, TestNG, and Page Object Models, felt like second nature.

But as the testing landscape evolved, I realized that modern test frameworks like Playwright were redefining speed, simplicity, and integration with DevOps pipelines.

That curiosity, to modernize my skills and adapt to the new wave of automation tools, led me to a focused project I named “User Registration Practice”.

At first glance, the name might sound simple, but behind it was a deeper goal: to rebuild automation from the ground up, this time using JavaScript, Playwright, and CI/CD integration.

Why This Project?

I didn’t want to just run a few test scripts locally. My goal was to:

• Understand how Playwright differs architecturally from Selenium.
• Learn asynchronous programming and modern JS concepts hands-on.
• Design a lightweight test structure that was CI/CD ready.
• Automate the entire testing cycle, from execution to reporting and email alerts.

I chose a simple user workflow: Login and Registration — to focus on the framework and pipeline rather than application complexity.

From Selenium Mindset to Playwright Thinking

Transitioning from Selenium to Playwright wasn’t just about syntax, it was about rethinking automation.

Playwright’s built-in features like auto-waiting, parallel execution, and native trace support were refreshing. The async/await syntax felt unusual at first, but once I internalized it, it gave me much finer control over browser interactions.

I structured the framework with a modular approach:

• Page classes handled reusable methods like navigation and actions.
• Locator management was centralized through a selectors helper.
• Auth setup tests generated a persistent storage state for authenticated sessions.

It was a familiar design, but much cleaner and faster, thanks to Playwright’s simplicity.

Building a Modular Test Framework

One of the most satisfying parts of this journey was structuring the Playwright framework from the ground up. I followed a modular and scalable design, integrating the Page Object Model (POM) pattern for better maintainability and reusability.

Each key component (such as base.page.js, login.page.js, and dashboard.page.js) was built to serve a distinct purpose, separating test logic from page interactions. This not only simplified debugging but also made the framework easier to extend as new modules and test cases were added.

The structure ensures that any future enhancements, like API integration, data-driven testing, or extended reporting, can be seamlessly introduced without disrupting the core setup. The design also lays the groundwork for integration with API tests or performance validation, extending its real-world applicability beyond UI automation.

Folder Structure showing page object implementation

Bringing CI/CD Into the Mix

With the framework ready, I wanted to take it a step further — integrate it into GitHub Actions. The aim was to build a full smoke test pipeline that could:

• Run headless Playwright tests automatically on every commit.
• Upload reports and test artifacts.
• Notify results through email.

The YAML workflow started simple: checkout → install → test. But as every automation engineer knows, that’s where the real debugging begins.

The Early CI Failures and Roadblocks

The first GitHub Action run didn’t go as planned. Playwright couldn’t navigate to URLs, browsers weren’t installed, and dependencies failed silently.

The first CI/CD run showing Playwright setup failure due to missing dependencies

Initially, my GitHub Actions pipeline failed repeatedly, sometimes due to Playwright browsers not installing correctly, and other times because parallel executions caused state conflicts. Each error forced me to better understand how GitHub Actions handles caching, job dependencies, and environment setups.

Each failure was a reminder of how different local vs CI environments can be. I learned that Playwright’s installation commands behave differently in containerized environments. Eventually, I split the setup into a dedicated ‘Setup Playwright Environment’ job that installed dependencies once and reused the environment across jobs — eliminating redundant browser installs.

Fixing it meant explicitly adding browser dependencies using:

• name: Install Playwright Browsers

run: npx playwright install — with-deps

Setup Playwright Environment job in Playwright

That one step fixed the setup, and the browser finally launched on CI. It wasn’t a code issue; it was an environment orchestration issue. That distinction is something you only appreciate once you debug CI/CD pipelines yourself.

The Authentication Challenge

Once Playwright was stable, my next hurdle was session state management. The authentication setup test (auth.setup.js) worked perfectly on local runs but failed in CI.

The logs showed:

“Cannot navigate to invalid URL: undefined/login.”

Showing error encountered

It turned out to be a simple misconfiguration — the base URL wasn’t being passed properly through CI secrets. Once corrected, I made sure the test saved the storage state reliably before the dependent tests executed.

These weren’t just fixes — they were validations of how tightly Playwright integrates with environment variables and CI secrets.

The Evolving Pipeline: From Smoke Tests to Reports

After stabilizing authentication, I introduced a dedicated smoke test suite. This pipeline:

• Filtered tests using tags (@smoke).
• Uploaded HTML reports after every run.
• Retained screenshots and videos for any failure.
• Used continue-on-error to ensure artifacts were always available, even when tests failed.

The successful Playwright CI/CD workflow with green check marks across all jobs

For me, this wasn’t just test execution — it was observability in testing. Each step of the pipeline became a feedback loop, helping me understand where time, resources, and errors occurred.

Playwright Test Report: After the successful pipeline execution, Playwright automatically generated a detailed HTML report capturing each test step, execution time, and result status. This visibility helped validate that the login workflow and smoke suite were stable and performing as expected.

Playwright report displaying step-by-step execution for the login smoke test

The Final Touch: Automated Email Notifications

One of the highlights of this project was setting up email notifications. After the test run, a summary email was sent automatically with key details: repository, workflow, commit ID, and report link.

The final automated email received from the Playwright CI bot summarizing the smoke test run

It may sound small, but that first success email marked a milestone, proof that the pipeline was fully autonomous from code to communication.

Lessons I Took Away

This wasn’t just about Playwright or JavaScript, it was about evolution in automation thinking.

Here’s what I learned along the way:

• Moving to Playwright requires adapting to async-first automation; but the payoff is significant in speed and reliability.
• CI/CD debugging is a skill in itself; it tests your patience and reasoning far more than your syntax.
• The key to building robust automation is not more tests, but smarter integration.
• Small, consistent wins, like passing a single smoke test on CI, build strong confidence in system reliability.

As the project evolved, I realized how much this exercise refined not just my automation skills, but also my problem-solving approach. Tools like Cursor played a quiet yet meaningful role in the process — especially during debugging and workflow optimization. It acted as a helpful assistant for resolving configuration issues and reviewing code behavior, allowing me to focus on analysis and learning rather than being stuck in loops of trial and error.

By the time everything came together, from code structure to CI execution, I realized this project had grown beyond a simple experiment. It had turned into a complete automation ecosystem where quality validation happened automatically, and feedback loops were instant.

What’s Next?

While the current setup focuses on login and smoke scenarios, the registration module is the next major milestone. It’s intentionally being built in phases so that every addition, whether it’s a new test module or a new environment configuration, fits seamlessly into the same CI/CD ecosystem.

This project isn’t finished, it’s continuously evolving, just like the tools and practices that inspired it. This project is still evolving, with new modules, deeper validations, and lessons I continue to uncover with each iteration. Every new feature I add refines the pipeline, making it more scalable, modular, and closer to real-world test orchestration.

Closing Thoughts

This project started as a practice ground but became a complete learning loop — combining my Selenium experience, new-age tools like Playwright, and the engineering mindset behind CI/CD pipelines.

It reminded me that automation today is not just about scripts, it’s about creating systems that continuously validate quality with minimal human touch.

I didn’t build it for perfection. I built it to learn, adapt, and evolve. And that, I believe, defines what modern quality engineering truly stands for.