From Plain English to Playwright Tests: Automating Web Testing with LLM(MiniMax) + Playwright CLI

Transform Gherkin Scenarios into Executable Test Scripts Using AI

TL;DR

This article demonstrates how to use LLM(MiniMax) + Playwright CLI to convert plain English (Gherkin) into fully functional Playwright test scripts.

Inspired by Microsoft opensource project AutoGenesis, this approach enables engineers to generate, debug, and maintain tests in minutes instead of hours.

Why This Matters

Test automation has always been a bottleneck:

• Writing test scripts is time-consuming
• Debugging UI tests is frustrating
• Maintenance cost grows with every UI change
• Non-engineers cannot easily contribute

So the question is:

What if AI could write and maintain test scripts for us?

Goal

Build a workflow where:

Plain English → AI → Playwright Test Script

Specifically, we will:

• Write test cases in Gherkin (BDD style)
• Use an LLM via MCP to interpret and execute steps
• Generate production-ready Playwright TypeScript tests

Architecture

The LLM interacts with the browser through Playwright cli, observes the UI, and generates structured test code.

Key Idea

The LLM doesn’t just generate code, it:

• Interacts with a live browser
• Observes the DOM
• Decides selectors dynamically
• Writes structured test code

Run web test with playwright test script

Tech Stack

• OpenCode (https://opencode.ai/docs/)
• MiniMax (https://platform.minimax.io/docs/token-plan/opencode)
• Playwright (https://playwright.dev/docs/intro)
• Playwright Cli (https://github.com/microsoft/playwright-cli)

Test Data Design

Separate test data from test logic to keep tests reusable and improve maintainability.

{
  "test_site": "https://www.saucedemo.com/",
  "username": "standard_user",
  "password": "secret_sauce"
}

Writing Tests in Plain English (Gherkin)

Feature: Shopping Cart and Checkout
  As a registered customer
  I want to add products to my cart
  So that I can purchase them

  Background:
    Given I am logged in as a customer
    And I am on the product page

  Scenario: Successfully add item and checkout
    When I add a product to the cart
    And I click on the cart icon
    Then the cart should contain 1 item
    And the total price should be correct
    When I proceed to checkout
    And I fill in shipping details
    And I submit the order
    Then I should see an order confirmation page

This is the only input needed.

Prompt Engineering

A structured prompt is used to instruct the LLM:

• Interpret Gherkin steps
• Use Playwright CLI for browser interaction
• Generate TypeScript test scripts
• Apply Page Object Model (POM) design
• Prompt

[embed]llm_playwrightcli/prompts/gherkin-to-playwright-prompt.md at main · liangway/llm_playwrightcli Contribute to liangway/llm_playwrightcli development by creating an account on GitHub.github.com

Running the Workflow

opencode run "$(cat prompts/gherkin-to-playwright-prompt.md); \
echo 'input gherkin file = tests/features/shopping-cart-checkout.feature \
output test file = tests/specs/shopping-cart-checkout.spec.ts'"

LLM + Playwright Cli in Action

From the logs, we can observe how the LLM:

1. Opens the application

$ playwright-cli open https://www.saucedemo.com/

2. Captures page structure

$ playwright-cli snapshot

3. Interacts with UI elements

$ playwright-cli fill e11 "standard_user" --submit

4. Automatic Page Object Model Generation

← Write tests/pages/LoginPage.ts
Wrote file successfully.

← Write tests/pages/ProductsPage.ts
Wrote file successfully.

The LLM generates structured POM classes:

import { type Page, expect } from '@playwright/test';

export class LoginPage {
  readonly page: Page;

  constructor(page: Page) {
    this.page = page;
  }

  get usernameInput() {
    return this.page.locator('[data-test="username"]');
  }

  get passwordInput() {
    return this.page.locator('[data-test="password"]');
  }

  get loginButton() {
    return this.page.locator('[data-test="login-button"]');
  }

  async login(username: string, password: string) {
    await this.usernameInput.fill(username);
    await this.passwordInput.fill(password);
    await this.loginButton.click();
  }
}

This demonstrates that the LLM not only generates tests, but also applies best practices in test design.

Test failure -> Self-healing

Initial failure

    1) [chromium] › tests/specs/shopping-cart-checkout.spec.ts:29:7 › Shopping Cart and Checkout › Successfully add item and checkout

    Error: expect(locator).toHaveCount(expected) failed
...

    > 37 |     await expect(cartPage.cartItem).toHaveCount(1);

...
        at /Users/liang/Projects/llm_playwrightcli/tests/specs/shopping-cart-checkout.spec.ts:37:37

The LLM analyzed the failure and corrected the locator:

- [data-test="cart-item"]
+ [data-test="inventory-item"]

It also improved related selectors:

- .cart_quantity
+ [data-test="item-quantity"]

Final Result

After refinement:

npx playwright test tests/specs/shopping-cart-checkout.spec.ts

Output:

✓ Successfully add item and checkout

Generated Artifacts

Test Script

tests/specs/shopping-cart-checkout.spec.ts

Page Objects

LoginPage
ProductsPage
CartPage
CheckoutPage
CheckoutOverviewPage
CheckoutCompletePage

Screenshot Evidence

Conclusion

This approach lowers the barrier to test automation:

• Non-engineers can define tests in Gherkin
• Engineers can focus on architecture instead of scripting
• Maintenance overhead is reduced through AI-assisted updates

While not a complete replacement for human validation, this workflow demonstrates a strong step toward AI-driven QA automation at scale.

REFERENCE

[embed]AutoGenesis：基于 AI + MCP 的跨平台自动化测试实践 自动化测试之所以难以真正铺开，很多时候并不是因为团队不重视，而是因为门槛太高: 业务人员不会写代码，测试脚本又难维护。Microsoft Edge QA 团队开源的 AutoGenesis，想解决的正是这个问题…www.infoq.cn

[embed]GitHub - microsoft/AutoGenesis: AutoGenesis is an AI-powered automated testing framework based on… AutoGenesis is an AI-powered automated testing framework based on Model Context Protocol (MCP), supporting multiple…github.com

[embed]GitHub - liangway/llm_playwrightcli Contribute to liangway/llm_playwrightcli development by creating an account on GitHub.github.com