I Had a Bad Day With Test Failures. Then I Built an MCP Agent So It Never Happens Again

Sometimes the most useful thing a bad day can do is make you angry enough to fix something properly.

It started with a failing test I couldn’t explain.

The dashboard data had changed overnight. Nothing was broken in the application — the numbers were right, the reports were accurate, everything was working exactly as it should. But my tests didn’t know that. They were asserting hardcoded values against dynamic data, and they were screaming failure.

So I did what you do. I downloaded the Excel report from the dashboard. Opened it. Cross-referenced the actual values manually. Updated the assertions. Re-ran the tests. Watched them pass.

Forty minutes. For a test suite that was never actually broken.

I sat there staring at the screen thinking: I’m a QA engineer. My job is to automate the things that shouldn’t need a human. And I just spent forty minutes doing something a script should do.

That was the day I started building an MCP agent.

What I actually wanted

Not a fancier test suite. Not better assertions. I wanted the whole loop — run tests, check failures, understand context, decide what’s real — to happen without me babysitting it.

Specifically for this dashboard problem, I wanted something that could:

• Run the Playwright tests
• When assertions fail on dashboard data, fetch the actual values from the downloaded Excel report
• Compare what the tests expected vs what the report actually shows
• Tell me: is this a real failure or just dynamic data that changed?

That’s not a test runner. That’s an agent. Something that perceives, reasons, and acts — not just executes.

MCP is what makes that possible.

The setup — what you need before you start

• Windows machine with Python 3.10+
• Playwright + pytest already running
• Claude Desktop installed
• The mcp Python package and openpyxl for reading Excel files
• Your dashboard report saved locally as .xlsx

:: Activate your venv first
venv\Scripts\activate

pip install mcp openpyxl pytest-json-report

Step 1 — Make your test output readable

The agent needs structured output to reason from. Add JSON reporting to pytest.ini:

[pytest]
addopts = --json-report --json-report-file=test-results.json

Run your tests once:

pytest tests\ -v

Confirm test-results.json appears in your project root.

Step 2 — Build the MCP server

This is the bridge between Claude and your local data. Two tools: one reads test failures, one reads the Excel report.

Create mcp_servers\dashboard_agent_server.py:

import json
import openpyxl
from pathlib import Path
from mcp.server import Server
from mcp.server.stdio import stdio_server
from mcp.types import Tool, TextContent

app = Server("dashboard-agent")

# Update these paths to match your project
RESULTS_FILE = Path(r"C:\your-project\test-results.json")
REPORT_FILE = Path(r"C:\your-project\reports\dashboard_report.xlsx")

@app.list_tools()
async def list_tools() -> list[Tool]:
    return [
        Tool(
            name="get_failed_tests",
            description="Get failed Playwright tests with error details",
            inputSchema={"type": "object", "properties": {}}
        ),
        Tool(
            name="read_dashboard_report",
            description="Read the latest downloaded dashboard Excel report",
            inputSchema={
                "type": "object",
                "properties": {
                    "sheet_name": {
                        "type": "string",
                        "description": "Sheet name to read (default: first sheet)"
                    }
                }
            }
        )
    ]

@app.call_tool()
async def call_tool(name: str, arguments: dict) -> list[TextContent]:

    if name == "get_failed_tests":
        if not RESULTS_FILE.exists():
            return [TextContent(type="text", text="No test results file found. Run pytest first.")]

        results = json.loads(RESULTS_FILE.read_text())
        failed = [t for t in results.get("tests", []) if t.get("outcome") == "failed"]

        if not failed:
            return [TextContent(type="text", text="All tests passed.")]

        lines = []
        for t in failed:
            node_id = t.get("nodeid", "unknown")
            error = t.get("call", {}).get("longrepr", "No error details")[:300]
            lines.append(f"Test: {node_id}\nError: {error}\n")

        return [TextContent(type="text", text="\n".join(lines))]

    if name == "read_dashboard_report":
        if not REPORT_FILE.exists():
            return [TextContent(type="text", text="Dashboard report not found. Please download it first.")]

        wb = openpyxl.load_workbook(REPORT_FILE)
        sheet_name = arguments.get("sheet_name") or wb.sheetnames[0]
        ws = wb[sheet_name]

        rows = []
        for row in ws.iter_rows(values_only=True):
            if any(cell is not None for cell in row):
                rows.append(" | ".join(str(cell) if cell is not None else "" for cell in row))

        content = f"Sheet: {sheet_name}\n\n" + "\n".join(rows[:50])  # First 50 rows
        return [TextContent(type="text", text=content)]

    return [TextContent(type="text", text=f"Unknown tool: {name}")]

async def main():
    async with stdio_server() as (read_stream, write_stream):
        await app.run(read_stream, write_stream, app.create_initialization_options())

if __name__ == "__main__":
    import asyncio
    asyncio.run(main())

Step 3 — Connect it to Claude Desktop

Open %APPDATA%\Claude\claude_desktop_config.json and add:

{
  "mcpServers": {
    "dashboard-agent": {
      "command": "C:\\your-project\\venv\\Scripts\\python.exe",
      "args": ["C:\\your-project\\mcp_servers\\dashboard_agent_server.py"]
    }
  }
}

Use the full path to python.exe inside your venv — not the system Python.

Fully quit Claude Desktop from the system tray and reopen it. Look for the tools icon near the message box — you should see dashboard-agent listed.

Step 4 — The prompt that changed my mornings

This is the part that made me feel like I’d actually solved something.

After a test run, instead of opening the CI report and manually cross-referencing the Excel file, I type this into Claude:

Check my failed Playwright tests. For each failure, read the dashboard 
Excel report and tell me whether the test failure looks like a genuine 
bug or whether the test assertion is just out of date with the current 
dashboard data. Recommend whether I should fix the test or raise a bug.

Claude calls get_failed_tests, reads the failures. Then calls read_dashboard_report, reads the actual current data. Compares the two. Comes back with:

Test: test_should_display_correct_monthly_revenue
Failure: AssertionError — expected £124,500, got £156,230
Dashboard report shows: Monthly Revenue = £156,230

Verdict: Test assertion is outdated. The dashboard data is correct.
Recommendation: Update the test assertion to read dynamically from 
the report rather than asserting a hardcoded value.

---

Test: test_should_show_active_users_count
Failure: Element .active-users not found on page
Dashboard report shows: Active Users = 1,842

Verdict: Element is missing from the page entirely — this looks like 
a genuine UI bug. The data exists in the report but isn't rendering.
Recommendation: Raise a bug ticket.

Two failures. Two completely different causes. Five seconds of Claude reasoning vs forty minutes of mine.

That’s it. That’s the thing I wanted.

What the agent gets right

Context it has that a plain test runner doesn’t:

• The actual current values from your report
• The difference between “assertion is wrong” and “element is missing”
• The ability to reason about why something failed, not just that it failed

Time it saves:

• No more downloading reports and manually cross-referencing
• No more false alarms from dynamic data changes
• Failures that need attention get flagged immediately

What the agent still can’t do

Be honest with yourself about the limits:

• It can’t download the report for you — you still need to download the Excel file from your dashboard manually before running the check. Automating that download depends on your specific application.
• It can’t judge business impact — if a revenue figure is wrong by £100 vs £100,000, the agent doesn’t know which matters more for tomorrow’s board meeting. You do.
• It reasons from what it sees — if your Excel report has ambiguous column headers or merged cells, the output gets messy. Clean reports give clean reasoning.
• It’s advisory, not autonomous — it recommends. You decide. Keep it that way until you fully trust the output.

The real lesson from that bad day

The forty minutes I spent manually cross-referencing that Excel report wasn’t a one-off. It was happening every time the dashboard data changed — which was often. I’d just stopped noticing how much time it was taking because I’d normalised it.

That’s the thing about tedious manual work in QA. You adapt to it. You build it into your morning. You stop questioning whether it should exist.

The bad day was useful because it made me angry enough to question it.

The agent didn’t make me a better QA engineer. It gave back time I was wasting on work that shouldn’t have needed a human — so I could spend it on work that does.

That’s the only reason to build one.