Visual Testing That Doesn’t Suck: Playwright + GitHub Actions = Chef’s Kiss 👌

Modern teams rely on visual regression testing to keep UI changes intentional. In this guide, I’ll show how to run Playwright visual tests on every pull request, publish rich HTML reports to AWS S3, comment the results back to the PR, and provide a manual workflow to safely update snapshots when changes are expected.

CI Flow of playwright.yml

What you will get from this article.

• Visual diffs: Compare screenshots to detect regressions.
• Rich reporting: Upload the HTML report and logs to S3, link them in the PR.
• Snapshot management: Manually trigger a second workflow to update snapshots and commit them back to the same PR.
• Traceability: GitHub App token for safe commits; S3 path keyed by branch and short SHA.

Prerequisites

• A working *Playwright* test suite, including visual snapshot tests.
• AWS S3 bucket access (programmatic credentials for CI).
• GitHub App credentials (App ID, private key) for making commits from CI.
• A self‑hosted runner(optional but strongly recommended) or GitHub‑hosted runner with Chromium dependencies. The examples below run on self-hosted.
• Project secrets configured in your repo settings:

CI_GITHUB_APP_ID
CI_GITHUB_APP_PRIVATE_KEY
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_REGION
AWS_S3_BUCKET

• Any app‑specific runtime envs (example: APP_PLUGIN_ID, APP_SECRET_KEY, APP_ROLE).

https://github.com/{company}/{repository}/settings/secrets/actions

Workflow 1: Run Playwright on every PR and publish reports 📷

Create a file like .github/workflows/playwright.yml to run on pull_request to develop. Key responsibilities:

• Install dependencies and Playwright browsers
• Start your web app under test
• Execute Playwright
• Upload HTML report and logs to S3 (scoped by branch/commit for preventing cache and rerun issues)
• Comment on the PR with a friendly summary and deep links

Here’s the essential structure:

name: Playwright Tests
on:
  pull_request:
    branches:
      - develop
jobs:
  test:
    timeout-minutes: 60
    runs-on: self-hosted #change it with your runner name.

And you need to add your Playwright job here, step by step.

Add your GitHub App token generator:

    steps:
      - name: Create GitHub App token
        id: github-app-token
        uses: actions/create-github-app-token@v2
        with:
          app-id: ${{ secrets.GITHUB_APP_ID }}
          private-key: ${{ secrets.GITHUB_APP_PRIVATE_KEY }}

      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: lts/*

Install dependencies & Playwright Browsers:

      - name: Install dependencies
        run: npm ci --legacy-peer-deps

      - name: Install Playwright Browsers
        run: npx playwright install --with-deps chromium # We just need chromium

Start your web app with nohup: Nohup keeps your project open in the background, and you can continue to use the terminal.

      - name: Start web server
        run: |
          nohup npm start > web.log 2>&1 &
          sleep 10
        env:
          APP_PLUGIN_ID: ${{ secrets.APP_PLUGIN_ID }}
          APP_SECRET_KEY: ${{ secrets.APP_SECRET_KEY }}
          APP_ROLE: ${{ secrets.APP_ROLE }}

Start running Playwright Tests:

Use --reporter=dot,html to generate both concise CI output and a browsable HTML report.

      - name: Run Playwright tests
        id: playwright-tests
        run: |
          npx playwright test --reporter=dot,html
          echo "exit_code=$?" >> $GITHUB_OUTPUT #we need this later
        env:
          APP_PLUGIN_ID: ${{ secrets.APP_PLUGIN_ID }}
          APP_SECRET_KEY: ${{ secrets.APP_SECRET_KEY }}
          APP_ROLE: ${{ secrets.APP_ROLE }}

Configure AWS Credentials:

      - name: Configure AWS Credential for S3
        if: always()
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ secrets.AWS_REGION_EU_WEST }}

Upload to AWS S3: Here, it creates a unique path for your test results like playwright/TASK-ID/SHA

      - name: Upload test report to S3
        if: always()
        run: |
          # Create short sha for unique folder names
          COMMIT_SHA=${GITHUB_SHA:0:7}
          BRANCH_NAME=$(echo "${{ github.head_ref }}" | cut -d'/' -f2)
          PROJECT_NAME=playwright

          # Determine test status
          if [ -f "playwright-report/index.html" ]; then
            TEST_STATUS="completed"
          else
            TEST_STATUS="failed"
          fi

          # Upload HTML report
          if [ -d "playwright-report" ]; then
            aws s3 sync playwright-report/ s3://${{ secrets.AWS_S3_BUCKET }}/$PROJECT_NAME/$BRANCH_NAME/$COMMIT_SHA/ --delete
            echo "Report uploaded to: s3://${{ secrets.AWS_S3_BUCKET }}/$PROJECT_NAME/$BRANCH_NAME/$COMMIT_SHA/"
          fi

          # Upload server logs if they exist
          if [ -f "web.log" ]; then
            aws s3 cp web.log s3://${{ secrets.AWS_S3_BUCKET }}/$PROJECT_NAME/$BRANCH_NAME/$COMMIT_SHA/web.log
          fi

          if [ -f "test.log" ]; then
            aws s3 cp test.log s3://${{ secrets.AWS_S3_BUCKET }}/$PROJECT_NAME/$BRANCH_NAME/$COMMIT_SHA/test.log
          fi

          # Create a summary file with metadata
          cat > report-metadata.json << EOF
          {
            "repository": "$GITHUB_REPOSITORY",
            "branch": "$BRANCH_NAME",
            "commit": "$GITHUB_SHA",
            "workflow_run_id": "$GITHUB_RUN_ID",
            "test_status": "$TEST_STATUS",
            "report_url": "s3://${{ secrets.AWS_S3_BUCKET }}/$PROJECT_NAME/$BRANCH_NAME/$COMMIT_SHA/"
          }
          EOF

          aws s3 cp report-metadata.json s3://${{ secrets.AWS_S3_BUCKET }}/$PROJECT_NAME/$BRANCH_NAME/$COMMIT_SHA/metadata.json

          # Output the S3 URL for easy access
          echo "::notice::Playwright report uploaded to S3: s3://${{ secrets.AWS_S3_BUCKET }}/$PROJECT_NAME/$BRANCH_NAME/$COMMIT_SHA/"

          # Save S3 URL to a file for the comment step
          echo "https://${{ secrets.AWS_S3_BUCKET }}.s3.eu-west-1.amazonaws.com/$PROJECT_NAME/$BRANCH_NAME/$COMMIT_SHA" > s3_report_url.txt

Prepare a comment message for sending to the Pull Request

Since links in Github Comment’s not opening on new tab, i added a 💡 Tip message for developers

      - name: Comment PR with S3 URL
        if: always() && github.event_name == 'pull_request'
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const s3Url = fs.readFileSync('s3_report_url.txt', 'utf8').trim();
            const testStatus = '${{ steps.upload-test-report.outputs.test_status }}' || 'unknown';
            const statusIcon = testStatus === 'completed' ? '✅' : '❌';

            const comment = `## Playwright Test Results ${statusIcon}

            **Status:** ${testStatus === 'completed' ? 'All tests passed' : 'Some tests failed'}

            **📊 Test Report:** [View Report](${s3Url}/index.html)

            **🔍 Logs:**
            - [Web Server Log](${s3Url}/web.log) (if available)
            - [Test Server Log](${s3Url}/test.log) (if available)
            - [Metadata](${s3Url}/metadata.json)

            ---
            *💡 Tip: Right-click links and select "Open in new tab" or use Cmd+Click*
            *Report generated at: ${new Date().toLocaleString('tr-TR', { 
              day: '2-digit', 
              month: '2-digit', 
              year: 'numeric', 
              hour: '2-digit', 
              minute: '2-digit', 
              second: '2-digit',
              hour12: false 
            }).replace(/\//g, '.')}*
            *Workflow Run: [#${{ github.run_number }}](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})*`;

            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: comment
            });```

Add some scenario if it fails

      - name: Fail if tests failed
        if: steps.playwright-tests.outputs.exit_code != '0'
        run: exit 1

⚠️ Before running this on your branch, you have to push to develop. Otherwise it will not trigger.

Now, you have a CI for playwright testing. Now, you can open a pull request to develop branch, and you will be able to see the action under your PR.

It will send a message when it is done.

Example for failing cases.

Workflow 2: Manually update visual snapshots 📸

When a change is intentional (new UI, redesign), you can update snapshots safely via a separate, manual workflow. This keeps the validation workflow strict and only updates snapshots on demand.

CI Flow of playwright-update-snapshots.yml

Create a file to .github/workflows/playwright-update-snapshots.yml like this

It triggers manually because of workflow_dispatch

name: Update Playwright Snapshots
on:
  workflow_dispatch:

jobs:
  update-snapshots:
    timeout-minutes: 60
    runs-on: self-hosted #change it with your runner name

Create a GitHub token and checkout:

    steps:
      - name: Create GitHub App token
        id: github-app-token
        uses: actions/create-github-app-token@v2
        with:
          app-id: ${{ secrets.GITHUB_APP_ID }}
          private-key: ${{ secrets.GITHUB_APP_PRIVATE_KEY }}

      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: lts/*

Install Dependencies & Start Web Server:

      - name: Install dependencies
        run: npm ci --legacy-peer-deps

      - name: Install Playwright Browsers
        run: npx playwright install --with-deps chromium # We just need chromium

      - name: Start web server
        run: |
          nohup npm start > web.log 2>&1 &
          sleep 10
        env:
          APP_PLUGIN_ID: ${{ secrets.APP_PLUGIN_ID }}
          APP_SECRET_KEY: ${{ secrets.APP_SECRET_KEY }}
          APP_ROLE: ${{ secrets.APP_ROLE }}

Let the fight begin: Run the update command for Playwright:

- name: Update Playwright snapshots
        id: update-snapshots
        run: |
          npx playwright test --update-snapshots --reporter html
          echo "exit_code=$?" >> $GITHUB_OUTPUT

(Optional) You can save results as a GitHub artifact for debugging:

      - name: Upload Playwright Report as Artifact
        uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 10

Now, Screenshots updated 🎉 Let’s commit them to the PR.

      - name: Commit updated snapshots
        id: commit-snapshots
        run: |
          git config --local user.email "github-actions[bot]@users.noreply.github.com"
          git config --local user.name "Visual Snapshot Updater [bot]"
          git add __tests__/visual #give your test path here
          git diff --staged --quiet || git commit -m "Update Playwright snapshots [skip ci]"
          git push
          echo "snapshots_updated=true" >> $GITHUB_OUTPUT

Also developer should know the result of this runner. So, let's send a message to PR.

Also, I added the username of who triggered the job.

      - name: Comment on PR with results
        if: always()
        uses: actions/github-script@v7
        with:
          script: |
            const updateExitCode = '${{ steps.update-snapshots.outputs.exit_code }}';
            const snapshotsUpdated = '${{ steps.commit-snapshots.outputs.snapshots_updated }}';

            let statusIcon, statusMessage, detailsMessage;
            if (updateExitCode === '0') {
              if (snapshotsUpdated === 'true') {
                statusIcon = '✅';
                statusMessage = 'Visual tests passed and snapshots updated successfully!';
                detailsMessage = 'The visual regression tests have been completed and any necessary snapshot updates have been committed to your branch.';
              } else {
                statusIcon = '✅';
                statusMessage = 'Visual tests passed - no snapshot updates needed';
                detailsMessage = 'All visual tests passed and your existing snapshots are up to date. No changes were committed.';
              }
            } else {
              statusIcon = '❌';
              statusMessage = 'Visual tests failed - check the logs for details';
              detailsMessage = 'The visual regression tests encountered failures. Please review the test output and logs to identify the issues.';
            }

            const comment = `## 🎨 Visual Test Results ${statusIcon}\n\n` +
              `**Status:** ${statusMessage}\n\n` +
              `**Details:** ${detailsMessage}\n\n` +
              `---\n` +
              `*This workflow was manually triggered to update Playwright visual test snapshots.*`;

            const { data: pulls } = await github.rest.pulls.list({
              owner: context.repo.owner,
              repo: context.repo.repo,
              head: context.repo.owner + ':' + context.ref.replace('refs/heads/', ''),
              state: 'open'
            });

            if (pulls.length > 0) {
              await github.rest.issues.createComment({
                issue_number: pulls[0].number,
                owner: context.repo.owner,
                repo: context.repo.repo,
                body: comment
              });
            } else {
              console.log('No open PR found for this branch, skipping comment');
            }

If Fails unexpectedly, let us upload the results to the artifact for debugging:

      - name: Upload logs (if failed)
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: server-logs
          path: |
            web.log
            test.log

⚠️ Before running this on your branch, you have to push to develop. Otherwise it will not trigger.

Now we can test 🎉

• You can reach this action in this path REPO_URL/actions/workflows/playwright-update-snapshots.yml
• Click to Run Workflow , choose your branch and Run.
• You will see it succeed.

Success page of playwright-update-snapshots.yml

Also success message has already been sent to the PR

Commit and Success Message for playwright-update-snapshots.yml

Why a separate workflow ❓

• Keeps the PR check deterministic: it must pass against current snapshots.
• Snapshot updates are an intentional, auditable action.
• Teams can review diffs before accepting new baselines.

Improvement Suggestions ⚡

• You can add a CODEOWNER check for the second runner. It provides updates to screenshots by inexperienced developers.
• Do not forget to configure your TTL(time to live) for S3 buckets and GitHub Artifacts to prevent extra charges.
• Create your custom runner for better performance.

Conclusion 🗞️

With two small workflows, you get robust PR‑level visual validation and a safe, auditable path for updating snapshots. Developers get fast feedback and rich diagnostics; reviewers get confidence that UI changes are intentional.

Drop these YAMLs into .github/workflows, wire up the secrets, and you’ll have a production‑grade visual testing pipeline that scales with your team.

If you are interested in more performance tricks, you can check my other article.