The Death of the Dashboard: Using Evidence.dev and OpenLineage to Build ‘Audit-First’ Analytics

I spent years perfecting dashboards that executives looked at once a month and then ignored. Pixel-perfect bar charts. Colour-coded KPI tiles. Slicers that nobody touched. The work was real; the engagement was polite at best.

But when the EU AI Act’s high-risk system provisions came into force in August 2025, the question in the room changed completely. It stopped being “what does the chart say?” and became “exactly where did this number come from, and who touched it on the way?”

That question exposed something I’d been building around for years without naming it: the traditional dashboard is a trust gap. It shows you a number. It doesn’t show you the chain of decisions, transformations, and source systems that produced that number. For an executive asking whether to act on a KPI, that’s a UX problem. For a regulator asking whether an automated system’s output was properly governed, it’s a compliance problem.

The fix isn’t a better dashboard. It’s a different kind of artefact entirely — one where the analytical logic, the data lineage, and the audit trail are first-class outputs, not afterthoughts.

The Dashboard Trap: How Visual-First BI Hides Lineage

The drag-and-drop BI paradigm — Power BI, Tableau, Qlik — was designed to make analytics accessible. A business analyst with no engineering background could connect to a database, drag fields into a chart, and publish a report by afternoon. That democratisation was genuinely valuable.

The cost, which wasn’t visible until regulatory pressure made it visible, is that the analytical logic lives in the tool’s internal representation — a proprietary format that’s opaque to external auditors, difficult to version-control, and essentially impossible to trace back to source data without manual reconstruction.

Consider what happens when a regulator asks: “How was this credit risk score calculated, and what data went into it?” In a traditional BI setup, the answer requires: identifying which dashboard showed the score, finding the measure definition in the semantic model, tracing the measure to the underlying table, tracing the table to the ETL job that populated it, finding the ETL job’s source system configuration, and documenting all of this manually — a process that typically takes days and produces documentation that’s already partially outdated.

In a code-first BI setup with OpenLineage instrumentation, the answer is a query against a lineage graph. The entire chain from source system to final number is a traversable data structure, not a reconstructed narrative.

The trust gap isn’t just a regulatory concern. It surfaces in practice every time a dashboard shows a number that contradicts someone’s intuition and nobody can explain why. “The dashboard says revenue is down 12%” — is that because revenue actually fell, or because the ETL job failed to pick up three days of transactions, or because someone changed the date filter logic in the report? In a drag-and-drop BI environment, answering that question requires knowing where to look. In an audit-first environment, the answer is in the lineage record.

Introducing Evidence.dev: Analytics as Software

Evidence.dev is a code-first BI framework. You write Markdown files with embedded SQL queries, and Evidence renders them as interactive web reports. The report logic lives in a git repository. Every change is a commit. Every deployment is a build. The analytical output is reproducible from the source files — a property that traditional BI tools fundamentally cannot provide.

The workflow looks like this:

project/
├── pages/
│   ├── index.md                    # Home dashboard
│   ├── claims-summary.md           # Claims overview report
│   └── denial-analysis.md          # Denial reason deep-dive
├── sources/
│   └── claims_duckdb/
│       └── connection.yaml         # DuckDB source configuration
└── package.json

A report page is a Markdown file where SQL queries are first-class citizens:

---
title: Claims Denial Analysis
description: Weekly denial patterns by reason code and payer
---

```sql denial_summary
SELECT
    denial_reason_code,
    payer_name,
    COUNT(*)               AS claim_count,
    SUM(billed_amount)     AS total_billed,
    AVG(billed_amount)     AS avg_billed,
    SUM(billed_amount - allowed_amount) AS total_denied
FROM claims_analytics
WHERE claim_date >= DATE_TRUNC('month', CURRENT_DATE) - INTERVAL 3 MONTHS
GROUP BY denial_reason_code, payer_name
ORDER BY total_denied DESC

Denial Trends

The top denial reason this quarter is {denial_summary[0].denial_reason_code}, accounting for {fmt(denial_summary[0].total_denied, 'usd')} in denied claims. <BarChart data={denial_summary} x=denial_reason_code y=total_denied series=payer_name title="Denied Amount by Reason Code" />

Every number in that report is traceable to the SQL query that produced it, which is traceable to the source table, which is traceable to the ETL job. The traceability is structural, not documented — it exists because the report is code, and code has a history.

This matters for the EU AI Act not as a compliance trick but as a design consequence: when your analytical output is a deterministic function of your SQL queries and your source data, explainability is a property of the build system, not an additional layer you add after the fact.

# Integrating OpenLineage: Automatic Lineage from Source to KPI

OpenLineage is a specification for recording data lineage events — the who-touched-what-when record that regulators and auditors need. It’s not a tool you install; it’s a protocol that your data tools emit events to, and a backend (Marquez, DataHub, or OpenMetadata) that stores and exposes those events.

The integration point with Evidence.dev is the SQL query execution layer. When Evidence runs a query against DuckDB, that’s a lineage event: input dataset (the DuckDB table or view), output dataset (the rendered report component), transformation (the SQL text), timestamp, and execution metadata. Capturing this requires a thin wrapper around Evidence’s query execution:

evidence_lineage_hook.py

Place in your Evidence project's custom scripts directory

from openlineage.client import OpenLineageClient from openlineage.client.run import RunEvent, RunState, Run, Job from openlineage.client.facet import ( DatasetFacet, SqlJobFacet, SchemaDatasetFacet, SchemaField ) import hashlib import uuid from datetime import datetime, timezone client = OpenLineageClient.from_environment() def emit_query_lineage( report_name: str, query_name: str, sql_text: str, source_tables: list[str], row_count: int ): """ Emit an OpenLineage event for an Evidence.dev query execution. Call this from Evidence's query lifecycle hooks. """ run_id = str(uuid.uuid4()) sql_hash = hashlib.sha256(sql_text.encode()).hexdigest()[:12] namespace = "evidence-analytics" client.emit(RunEvent( eventType=RunState.COMPLETE, eventTime=datetime.now(timezone.utc).isoformat(), run=Run(runId=run_id), job=Job( namespace=namespace, name=f"{report_name}.{query_name}", facets={"sql": SqlJobFacet(query=sql_text)} ), inputs=[ DatasetFacet(name=table, namespace="duckdb") for table in source_tables ], outputs=[ DatasetFacet( name=f"evidence://{report_name}#{query_name}", namespace=namespace ) ] )) return run_id

For the DuckDB source layer — the materialised views and transformed tables that Evidence queries — OpenLineage events are emitted by the dbt runs that build those tables. dbt’s OpenLineage integration (via the `dbt-ol` package) automatically emits lineage events for every model execution, capturing the source → transformed → gold chain without manual instrumentation.

profiles.yml addition for OpenLineage emission from dbt

config: send_anonymous_usage_stats: false

openlineage: transport: type: http url: http://marquez:5000 endpoint: api/v1/lineage

The result is a complete lineage graph: raw source tables → dbt models → DuckDB views → Evidence queries → rendered report components. Every node is timestamped. Every edge records the SQL transformation. Every report component is linked to the source data it was computed from, through every transformation step in between.

# The Audit-First Architecture: DuckDB + Evidence + Git

The full architecture has four layers, each with a specific governance property:

**Layer 1: Source ingestion with version-controlled contracts**

sources/claims_duckdb/connection.yaml

name: claims_duckdb type: duckdb database: /data/claims_analytics.duckdb

DataContract reference - schema is governed, not assumed

contract_ref: datacontracts/claims_analytics_v2.yaml

The DataContract reference means that when the source schema changes, the contract validation fails before Evidence renders the report. Schema drift surfaces at build time, not when an analyst notices a suspicious number.

**Layer 2: dbt transformation layer**

-- models/silver/silver_claims.sql -- Emits OpenLineage events automatically via dbt-ol

{{ config( materialized='incremental', unique_key='claim_id', incremental_strategy='merge' ) }} SELECT claim_id, SHA256(CAST(member_id AS VARCHAR)) AS member_id_hash, claim_date, denial_reason_code, payer_name, billed_amount, allowed_amount, billed_amount - allowed_amount AS denied_amount, CURRENT_TIMESTAMP AS transformed_at, '{{ invocation_id }}' AS dbt_run_id FROM {{ source('raw', 'claims_feed') }} {% if is_incremental() %} WHERE claim_date > (SELECT MAX(claim_date) FROM {{ this }}) {% endif %}

The `{{ invocation_id }}` macro stamps every row with the dbt run that produced it. Combined with the OpenLineage event that records which dbt run corresponds to which source snapshot, you can reconstruct exactly what data was available at the time any specific row was produced.

**Layer 3: Evidence.dev report layer**

Evidence reports are committed to git. Every change to a query or a report layout is a commit with an author, a timestamp, and a diff. The git history is the audit trail for the analytical logic — not a separate documentation system, but the version control system you’re already using.

Git history for a claims report

git log --oneline pages/denial-analysis.md

a3f7c21 Fix date range to exclude current incomplete month 8b2d441 Add payer breakdown to top denial reasons chart f19c3a8 Switch from billed_amount to denied_amount as primary metric 2c1a809 Initial denial analysis report

That four-line git log tells a compliance auditor: who changed the metric definition, when, and exactly what changed. The `f19c3a8` commit records the decision to use `denied_amount` instead of `billed_amount` as the primary KPI — a decision that has regulatory implications in RCM contexts, and one that would be invisible in a drag-and-drop BI environment.

**Layer 4: CI/CD pipeline with lineage validation**

.github/workflows/evidence-build.yml

name: Evidence Build and Lineage Validation

on: [push, pull_request] jobs: build: runs-on: ubuntu-latest steps:

• uses: actions/checkout@v4
• name: Validate DataContracts run: datacontract test datacontracts/claims_analytics_v2.yaml
• name: Run dbt models run: | dbt deps dbt run --select silver+ dbt test --select silver+
• name: Build Evidence reports run: npm run build
• name: Validate OpenLineage graph completeness run: python scripts/validate_lineage_graph.py env: MARQUEZ_URL: ${{ secrets.MARQUEZ_URL }}
• name: Deploy to staging if: github.ref == 'refs/heads/main' run: npm run deploy

The validate_lineage_graph.py step checks that every Evidence query in the report layer has a corresponding lineage path to a known source system. If a report queries a table that isn't in the lineage graph — because someone added a direct DuckDB query without going through dbt — the build fails. You cannot deploy an Evidence report that doesn't have a documented provenance chain.

Solving for the EU AI Act: Explainability and Bias Audits in Code

The EU AI Act’s requirements for high-risk AI systems include two that are directly addressed by this architecture.

Article 13 — Transparency. High-risk AI systems must be designed and developed in such a way that their operation is sufficiently transparent to allow deployers to understand what the system does and to interpret its output. For an Evidence.dev report, transparency is structural: the SQL query that produced each number is in the same file as the number. An auditor reading the report source can see exactly what computation produced each KPI.

Article 12 — Record-keeping. High-risk AI systems must be capable of automatically recording logs throughout their lifetime. The OpenLineage graph satisfies this requirement for data lineage; the git history satisfies it for analytical logic changes; the dbt run history satisfies it for transformation execution. Three independent audit systems, each maintained automatically by the tooling rather than manually by analysts.

The bias audit requirement is more interesting to implement. The EU AI Act requires that high-risk systems be tested for discriminatory bias before deployment and monitored for it in production. For analytical outputs that influence decisions about individuals — credit risk scores, insurance pricing, claims processing priority — this means the analytical pipeline must be testable at the query level.

Evidence.dev’s SQL-first structure makes this natural. A bias audit test is a SQL query:

-- tests/bias_audit_claim_priority.sql
-- Fails build if claim prioritisation shows demographic disparity

WITH priority_by_demographics AS (
    SELECT
        r.region_type,        -- urban/rural, not individual demographics
        r.median_income_tier,
        AVG(p.priority_score) AS avg_priority,
        COUNT(*)              AS claim_count
    FROM claim_priority_scores p
    JOIN member_regions r ON p.region_id = r.region_id
    WHERE p.scored_at >= CURRENT_DATE - INTERVAL 30 DAYS
    GROUP BY r.region_type, r.median_income_tier
),
disparity_check AS (
    SELECT
        MAX(avg_priority) - MIN(avg_priority) AS priority_disparity
    FROM priority_by_demographics
)
SELECT priority_disparity
FROM disparity_check
WHERE priority_disparity > 0.15   -- configurable threshold; >0.15 fails the build

This test runs in the CI/CD pipeline on every deployment. If claim prioritisation shows a disparity above the threshold across demographic-adjacent regional factors, the build fails and the deployment is blocked. The threshold, the test logic, and the failure history are all in version control — auditable, traceable, and explainable to a regulator who asks whether bias was tested before the system was deployed.

Performance Trade-offs: Code-First vs. Cached Dashboards

The honest performance comparison is not straightforwardly in Evidence’s favour. Traditional cached dashboards (Power BI Import mode, Tableau extracts) pre-compute aggregations at refresh time and serve results from memory — query response times measured in milliseconds for pre-cached views.

Evidence.dev with DuckDB runs SQL at query time. For a report page with five queries, the render time is the sum of those five query execution times. For well-optimised DuckDB queries against local files, this is typically 200–800ms per query, or 1–4 seconds total page render. For Power BI Import mode, the same page renders in under 200ms from cache.

The performance gap is real, and the right framing for it is: you’re trading query-time freshness for render-time speed. Evidence reports are always current — they query live data at render time. Cached dashboards are as current as the last refresh, which for Power BI Pro is 8 times per day maximum.

For regulated industries where data freshness is a compliance requirement (intraday risk positions, same-day claims status), the cache model is structurally inadequate regardless of render speed. For monthly executive reporting where data freshness beyond a daily refresh is unnecessary, the cache model’s speed advantage is real.

The mitigation for Evidence’s render-time cost is DuckDB’s columnar performance and materialised view pre-computation. Moving the slow aggregations into DuckDB materialised views (pre-computed incrementally as new data arrives) and having Evidence query those views rather than the raw tables brings render times below 500ms for most analytical workloads — comparable to cached dashboards, with the freshness advantage retained.

The Analyst’s New Workflow: From Report Builder to Analytics Engineer

The shift this architecture requires from analysts is real and worth naming honestly. Analysts who built their careers around drag-and-drop BI tools — whose skill was knowing where to click to produce the chart that answered the business question — find themselves in a different kind of work.

Writing SQL is not new for most analysts. Writing Markdown is low-friction. What’s new is the engineering discipline: version control, code review, CI/CD pipelines, test-driven data validation. These are practices that software engineers take for granted and that most BI practitioners have had limited exposure to.

The organisations that navigate this transition successfully do three things. They pair analytics engineers with BI analysts on the initial Evidence.dev builds, so the engineering patterns transfer through practice rather than documentation. They treat the git history as a first-class artefact of the analyst’s work — a portfolio of analytical decisions that is searchable and auditable — rather than as overhead. And they start with the reports where lineage and audit trails matter most (regulatory submissions, executive KPIs that drive hiring or investment decisions) rather than trying to migrate every dashboard simultaneously.

The traditional dashboard isn’t dead in the sense that it stops being used. It’s dead in the sense that it stops being sufficient. In a regulatory environment where “where did this number come from?” is a compliance question rather than a curiosity, the answer needs to be structural. Code-first BI, combined with OpenLineage instrumentation and DataContract governance, makes it structural.

The analyst who builds that infrastructure is doing more valuable work than the one who builds the chart. That’s a reorientation worth making deliberately.

Aasir writes about analytics engineering, code-first BI architecture, and governance in regulated data environments. Moving from drag-and-drop BI to Evidence.dev and hitting friction points? The comment section is the right place.