Medster: A Token-Efficient Approach to Medical Document Analysis with AI

Moving beyond the context window — how code-generation sandboxing creates more reliable, auditable clinical AI

There’s a fundamental problem with how most AI systems analyze medical documents today. You upload a PDF, it gets stuffed into a context window, and the model “looks” at it. Sometimes it catches everything. Sometimes it misses the one lab value that matters. You never quite know which you’re getting.

I found this approach deeply unsatisfying. Medicine demands reliability. When you’re reviewing a patient’s chart, you don’t skim. You have a process. You check specific things in a specific order.

So I built something different.

The Problem with Traditional LLM Document Analysis

When you pass a medical document directly to an LLM’s context window, several things happen:

Token inefficiency. A single progress note can consume thousands of tokens. Multiply that across a patient’s complete record, and you’re burning through context space — and money — at an alarming rate.

Inconsistent extraction. LLMs don’t “read” documents the way clinicians do. They process tokens probabilistically. Ask the same model to find all HbA1c values in a document twice, and you might get slightly different results. That’s fine for creative writing. It’s unacceptable for clinical data extraction.

Black box analysis. When a model tells you “the patient has diabetes based on the uploaded record,” you have no insight into how it reached that conclusion. Did it find the ICD-10 code? The problem list? A medication like metformin? You’re trusting the output without understanding the methodology.

Attention limitations. Even with expanding context windows, LLMs have attention constraints. Information at the beginning and end of documents gets more weight than the middle. In a 50-page medical record, that middle section might contain the most important clinical data.

A Different Architecture: Code-Generation Sandboxing

Medster takes a fundamentally different approach. Instead of having the LLM analyze documents directly in its context window, it writes code to programmatically extract data, then synthesizes the structured results.

Here’s the flow:

User Query: "Analyze this uploaded medical record"
                    │
                    ▼
┌─────────────────────────────────────────────────────────┐
│  Claude Sonnet 4.5 (Planning/Action)                    │
│  - Sees uploaded file markers in query                  │
│  - Decides to use generate_and_run_analysis tool        │
│  - WRITES Python code using available primitives        │
└─────────────────────────────────────────────────────────┘
                    │
                    ▼
┌─────────────────────────────────────────────────────────┐
│  Sandbox Execution (code_generator.py)                  │
│  - Receives generated code + uploaded_content           │
│  - Executes analyze() function                          │
│  - Primitives available:                                │
│    • uploaded_content (the file text)                   │
│    • search_uploaded_content(pattern)                   │
│    • extract_sections(start, end)                       │
│  - Returns structured dict result                       │
└─────────────────────────────────────────────────────────┘
                    │
                    ▼
┌─────────────────────────────────────────────────────────┐
│  Claude Sonnet 4.5 (Synthesis)                          │
│  - Receives sandbox execution results                   │
│  - Generates human-readable clinical summary            │
└─────────────────────────────────────────────────────────┘

The key insight: the model doesn’t analyze the document directly. It writes code to programmatically extract data, then synthesizes the structured results.

Why This Matters

This architecture delivers three critical benefits:

1. Token Efficiency

The generated code is compact — typically under 100 tokens. The results are structured dictionaries, not verbose prose. A 10,000-token medical document becomes a 500-token structured extraction. That’s a 20x efficiency gain

def analyze():
    results = {
        'medications': search_uploaded_content(r'(?i)(metformin|lisinopril|atorvastatin)\s*\d+\s*mg'),
        'lab_values': search_uploaded_content(r'HbA1c[:\s]+(\d+\.?\d*)'),
        'diagnoses': search_uploaded_content(r'(?i)(diabetes|hypertension|hyperlipidemia)')
    }
    return results

2. Reliability Through Determinism

Regex patterns find all matches, not just what the LLM happens to notice. Run the same extraction twice, get the same results. This is the difference between “probably found most of the medications” and “definitively extracted every medication mention matching this pattern.”

In clinical contexts, that determinism matters. A lot.

3. Auditability

You can see exactly what code ran. If the system missed something, you can examine the regex pattern and understand why. If it found something unexpected, you can trace the extraction logic. This creates an audit trail that’s impossible with direct context-window analysis.

[DEBUG] Executed code:
    search_uploaded_content(r'(?i)A1c[:\s]+(\d+\.?\d*)')

[DEBUG] Found matches: ['7.2', '6.8', '7.1']

[DEBUG] Synthesis: "Patient's HbA1c values show improving 
        glycemic control: 7.2% → 7.1% → 6.8% over 6 months"

The Broader Medster Architecture

This code-generation approach is one component of a larger autonomous clinical analysis system. Medster adapts the proven multi-agent architecture from Dexter (a financial research agent) for the medical domain:

              MEDSTER CLI / WEB UI

                        │
          ┌─────────────┼─────────────┐
          ▼             ▼             ▼
    ┌──────────┐  ┌──────────┐  ┌──────────┐
    │ Coherent │  │   MCP    │  │ Claude   │
    │ Data Set │  │  Server  │  │ Sonnet   │
    │          │  │          │  │   4.5    │
    │ FHIR     │  │ Analyze  │  │ Planning │
    │ DICOM    │  │ Complex  │  │ Reasoning│
    │ ECG/Notes│  │ Notes    │  │ Synthesis│
    └──────────┘  └──────────┘  └──────────┘

Planning Module — Decomposes clinical queries into discrete tasks. “Analyze this patient’s cardiac risk” becomes: retrieve labs → get medication list → check drug interactions → calculate CHA₂DS₂-VASc score → synthesize findings.

Action Module — Selects and executes appropriate tools. Labs come from the FHIR data source. Uploaded documents go through the code-generation sandbox. Drug interactions hit the safety checking module.

Validation Module — Verifies task completion. Did we actually get the troponin values? Did the regex find any medications? If not, iterate with different approaches.

Synthesis Module — Generates the final clinical summary from structured data, not from raw document text.

Working with Real Data

Medster integrates with the Coherent Data Set — a 9GB synthetic medical dataset that includes linked FHIR resources, DICOM images, genomic data, and ECG waveforms. This allows comprehensive multimodal analysis:

FHIR: Patient records, labs, vitals, medications, clinical notes

DICOM: Medical imaging (298 brain MRIs in the dataset)

Genomics: 889 genomic data files

Physiological: ECG waveforms and continuous monitoring data

All data types are linked via FHIR references, enabling queries like “show me the brain MRI for any patient with an Alzheimer’s diagnosis.”

HIPAA-Ready Infrastructure

An important note on compliance: Anthropic offers HIPAA-ready infrastructure for Claude, meaning organizations can establish Business Associate Agreements (BAAs) to process Protected Health Information (PHI) in compliant workflows. This was formalized with the January 2026 launch of Claude for Healthcare.

This doesn’t mean Claude is FDA-approved as a medical device — it isn’t, and Medster should be treated as a research and educational tool, not a clinical decision-making system. But it does mean the underlying infrastructure supports legitimate healthcare use cases when proper compliance frameworks are in place.

As with any AI tool in healthcare: verify findings with appropriate clinical resources, and use clinical judgment for all patient care decisions.

Example: Differential Diagnosis Workup

Here’s Medster handling a complex clinical query:

medster>> Patient presents with fatigue, weight loss, and night 
         sweats. Generate a prioritized differential and optimal 
         workup sequence.

[Planning] Breaking down clinical query...
  Task 1: Generate differential diagnosis list
  Task 2: Prioritize by probability and severity  
  Task 3: Design optimal diagnostic sequence
  Task 4: Identify critical "can't miss" diagnoses

[Action] Generating differential...
  - Malignancy (lymphoma, leukemia, solid tumor)
  - Infection (TB, HIV, endocarditis)
  - Autoimmune (SLE, vasculitis)
  - Endocrine (hyperthyroidism, adrenal insufficiency)
  - Hematologic (anemia workup)

[Action] Calculating optimal workup sequence...
  Initial: CBC w/diff, CMP, LDH, ESR/CRP, TSH
  If abnormal: CT chest/abd/pelvis, HIV, ANA
  Consider: Bone marrow biopsy if cytopenias

[Synthesis] The constellation of B-symptoms (fatigue, weight 
loss, night sweats) raises concern for lymphoproliferative 
disorder as the leading diagnosis...

The system thinks, plans, and executes — with full transparency into its reasoning at each step.

Safety Mechanisms

Autonomous agents in healthcare require robust safety controls. Medster implements:

Global step limits: Maximum 20 steps per query to prevent infinite loops

Per-task limits: 5 steps per subtask

Loop detection: Identifies and breaks repetitive action patterns

Critical value flagging: Automatic alerts for dangerous lab values

Drug interaction checking: Real-time safety screening

Try It Yourself

Medster is open source under MIT license:

git clone https://github.com/sbayer2/Medster_dev.git
cd Medster_dev
cp env.example .env
# Add your ANTHROPIC_API_KEY to .env
uv sync
uv run medster-agent

The web interface runs on localhost:3000, with a Python backend on port 8000. Cloud deployment to Google Cloud Run is documented in the README.

I’m continuing to develop Medster’s capabilities:

Expanded multimodal analysis: Better DICOM interpretation, ECG pattern recognition

MCP server integration: Connecting to specialized medical analysis services

Clinical decision support: More sophisticated scoring calculators and risk stratification

Real-time monitoring: Integration with streaming vital sign data

The code-generation sandbox pattern has applications beyond medicine. Any domain requiring reliable, auditable document extraction — legal discovery, financial analysis, research literature review — could benefit from this architecture.

The Bigger Picture

We’re at an inflection point in clinical AI. The naive approach — dump documents into a context window and hope for the best — is giving way to more sophisticated architectures that respect the reliability requirements of healthcare.

Medster represents one path forward: use AI for what it’s good at (planning, reasoning, synthesis) while delegating deterministic extraction to code. The result is a system that’s more reliable, more efficient, and more auditable than pure LLM approaches.

Medicine has always been about systematic methodology. It’s time our AI tools reflected that.

Steven Bayer is a physician working in urgent care, with experience in clinical AI development. Medster is available at github.com/sbayer2/Medster_dev.

AI #Healthcare #ClinicalAI #MachineLearning #MedicalInformatics #Claude #Python #OpenSource