What Karpathy Just Released Changes the Job Description of Every Scientist

Your AI Is Running 100 Experiments While You Sleep. Are You Designing the Right Arena?

Andrej Karpathy’s autoresearch isn't just a cool ML trick — it's a blueprint for how humans will direct AI progress across every industry.

March 2026 · 14 min read

Friend Link — click here to read in case you are blocked by a paywall

tl;dr

Karpathy released a 630-line script that lets an AI agent run ~100 ML experiments overnight on a single GPU — proposing code changes, testing them, keeping winners, discarding losers — completely unattended.

The human’s only job: write a Markdown file (program.md) defining the goal, the metric, and the constraints.

The pattern generalizes to any domain with a measurable metric — drug discovery, marketing A/B tests, quant trading, clinical trial design, software optimization, curriculum design, energy grids.

The role shift: you stop running experiments. You design the arena.

Here’s a question nobody is asking loudly enough: What is the actual job of a researcher in 2026?

Not the credential. Not the title. The function. What does a researcher do that a well-designed AI loop cannot?

Last week, Andrej Karpathy — co-founder of OpenAI, former Tesla AI director, and the person who coined “vibe coding” — dropped a 630-line Python script on GitHub that quietly answered that question. He called it autoresearch. The AI community lost its mind. And once you understand what it actually does, you'll see why.

The script ran 126 experiments in a single overnight session. It discovered improvements that Karpathy himself, with twenty years of hands-on model training experience, had missed. It committed the winning changes to Git. It kept working.

He was asleep.

“The goal is to engineer your agents to make the fastest research progress indefinitely and without any of your own involvement.” — Andrej Karpathy, X/Twitter, March 7, 2026

But here’s what the headlines missed: this isn’t only for ML researchers. The loop Karpathy built is a universal pattern. It works wherever you have a thing to optimize, a metric to measure, and a budget of time. Which means it works almost everywhere.

Let’s break down how it works — and then go through seven practical use cases that have nothing to do with training language models.

The Idea Is Almost Embarrassingly Simple

At its core, autoresearch is a loop with four steps:

1. Hypothesize — The AI reads the current code/config and proposes one change
2. Experiment — It implements the change and runs a fixed-time trial (exactly 5 minutes)
3. Evaluate — It measures whether the metric improved
4. Commit or Revert — Winning changes stay; losing ones get discarded

That’s it. No PhD required. No elaborate infrastructure. One GPU, one file, one metric.

The genius is in the constraints. Every experiment gets exactly the same time budget — 5 minutes, regardless of what the AI changes. This makes results directly comparable. You can change model depth, learning rate, optimizer, and batch size all in the same overnight session, and the comparison is fair because the clock doesn’t lie.

The result: roughly 12 experiments per hour, ~100 per overnight run. In two days of unattended running, the system executed ~700 changes, found ~20 genuine improvements, and cut a key benchmark from 2.02 hours to 1.80 hours — an 11% efficiency gain on a project already considered well-tuned.

The most uncomfortable part of Karpathy’s writeup: the agent caught oversights in attention scaling and regularization that he had missed manually over two decades. Not because it’s smarter. Because it never gets tired, never gets attached to prior decisions, and never stops running experiments.

The human’s only job is to write program.md — a plain-text Markdown file that describes the research goal, the constraints, and how to evaluate success. The AI does everything else.

“The human writes a Markdown file. The AI runs 100 experiments overnight. The bottleneck isn’t compute. It’s your program.md.” — Garry’s List analysis, March 2026

The Shift Nobody’s Named Properly Yet

Let’s call it what it is: the researcher’s role is collapsing upward.

Not disappearing — collapsing upward. The boring, repetitive parts of the job (run experiment, check metric, tweak parameter, repeat) are being handed to agents. What’s left is the part that actually requires human judgment: designing the arena.

What goal are we chasing? What’s the metric that actually matters? What constraints must the agent never violate? What counts as a valid experiment? What’s the time budget?

These questions require domain knowledge, ethical judgment, and strategic clarity. They cannot be automated. They are now the entire job.

This is the same pattern that emerged in software with “vibe coding” — the programmer’s role collapsed upward from writing syntax to describing systems. What Karpathy has done with autoresearch is extend that collapse to the scientific method itself.

The implications are not limited to AI labs. Let’s walk through where this pattern lands in the real world.

7 Practical Arenas for the Autoresearch Loop

1. Drug Discovery and Molecular Optimization

The pharmaceutical industry runs on exactly this loop — it’s just brutally slow when humans are in it.

A medicinal chemist proposes a molecular modification, synthesizes it, tests binding affinity, measures toxicity, iterates. A single drug candidate might take years and hundreds of experiments before the right combination of modifications is found.

Autoresearch — applied to computational chemistry — collapses that cycle dramatically.

The arena: a molecular simulation environment (tools like Schrödinger, AutoDock Vina, or AlphaFold 3). The metric: binding affinity score, ADMET properties (absorption, distribution, metabolism, excretion, toxicity), or selectivity ratio. The program.md equivalent: the target protein, the chemical scaffold, the constraints (no certain functional groups, molecular weight limits, oral bioavailability requirements).

The AI proposes modifications, runs the simulation (the fixed time budget), evaluates the score, commits improvements.

Stanford’s Biomni system, which uses Claude as its reasoning engine, already demonstrated this pattern — analyzing 450 wearable data files in 35 minutes, a task estimated at three human weeks. The autoresearch loop simply makes the architecture explicit and open-source.

What you design: the molecular search space, the scoring function, the hard constraints. What the agent does: run thousands of simulations while you’re unavailable.

2. Marketing and Growth Experiment Loops

This one hit the mainstream fastest. Eric Siu, founder of ad agency Single Grain, put it plainly on X: most marketing teams run 20–30 experiments per year. “The next generation will run 36,500+.”

The arena: a landing page, email sequence, or ad creative. The metric: conversion rate, reply rate, cost per acquisition. The program.md equivalent: the product, the target audience, the brand constraints, and the list of variables the agent is allowed to modify (headline, CTA, hero image, subject line) versus those it must leave alone (legal disclaimers, pricing).

The agent generates a variant, deploys it (or queues it for deployment), waits for the fixed time window of traffic, measures the metric, keeps or reverts.

The practical unlock here is the fixed time budget. Current A/B testing is slow because we wait for statistical significance, which can take weeks for low-traffic pages. Autoresearch’s answer is to run so many experiments in parallel that aggregate patterns emerge rapidly, even if individual runs are underpowered. The signal isn’t in any one experiment — it’s in the pattern across hundreds.

What you design: the brand guardrails, the success metric, the list of levers. What the agent does: generate and test variants at a pace no human team can match.

3. Financial Model and Trading Strategy Optimization

Quantitative finance already runs this loop — it’s called backtesting. The difference is that human quants backtest strategies they thought of. Autoresearch backtests strategies the agent thought of, drawn from a hypothesis space the human defines.

The arena: historical market data (equity, FX, crypto, options). The metric: Sharpe ratio, maximum drawdown, or Calmar ratio over a defined lookback window. The program.md equivalent: the instrument universe, the allowed signal types (momentum, mean-reversion, factor-based), the position sizing rules, and the transaction cost model.

The agent proposes a parameter change — adjust the lookback window, modify the entry threshold, add a volatility filter — runs the backtest for the fixed time budget, evaluates the metric, commits or reverts.

The critical constraint to build into your program.md: prevent overfitting. The agent must be instructed to validate on out-of-sample data, not optimize on the entire history. This is the arena design problem that requires human judgment — the metric that looks great in-sample can be worthless live.

What you design: the signal space, the risk constraints, the out-of-sample validation protocol. What the agent does: systematically explore the parameter space faster than any human analyst can.

4. Clinical Trial Protocol Design

This is where the pattern gets genuinely high-stakes — and genuinely transformative.

Clinical trial design is currently a slow, committee-driven process. Protocol amendments — changes to inclusion/exclusion criteria, dosing schedules, endpoint definitions — require regulatory submissions and can take months. But before a trial goes live, the computational phase of protocol optimization is exactly the autoresearch loop.

The arena: a patient population simulation (synthetic cohort models or historical trial data from registries like ClinicalTrials.gov). The metric: statistical power, enrollment velocity, or expected hazard ratio at the planned N. The program.md equivalent: the target indication, the regulatory constraints (FDA/EMA guidelines for the specific endpoint type), the ethical guardrails (minimum follow-up periods, safety monitoring rules).

The agent proposes protocol modifications — adjust dosing intervals, widen age range, change from co-primary to hierarchical endpoints — runs the simulation, evaluates power, commits improvements.

Anthropic has already connected Claude directly to Medidata’s clinical trial enrollment data and ClinicalTrials.gov, providing exactly the data infrastructure this loop needs. The missing piece is closing the loop with a simulation environment — which is now a tractable engineering problem.

What you design: the regulatory boundaries, the clinical hypotheses, the simulation model. What the agent does: find the protocol that maximizes power within constraints, overnight.

5. Software Performance Tuning and Compiler Optimization

This is perhaps the most immediately accessible use case for engineers.

Every production codebase has hot paths — functions that run millions of times and where a 10% speed improvement compounds into significant infrastructure cost savings. Traditionally, a performance engineer profiles the code, hypothesizes an optimization, implements it, benchmarks, and iterates. This is slow, context-dependent work.

The autoresearch loop applied here: the arena is your production code (or a representative microbenchmark). The metric is wall-clock time, memory allocation, or database query time. The program.md equivalent: the invariants the agent cannot break (interface contracts, test coverage must remain green), the subsystems it is allowed to modify, and the hardware profile of your production environment.

The agent proposes a change — switch from list to deque, add an LRU cache, vectorize a loop, change a database index — runs the benchmark for exactly N seconds, evaluates the metric, commits if improved.

Karpathy’s own Claude Code auto-accept mode is a step toward this: agents that run code, check tests, and iterate without waiting for approval. Autoresearch just adds the explicit outer loop and the fixed time budget.

What you design: the test suite that defines “still correct,” the performance metric, the allowed modification scope. What the agent does: explore the optimization space exhaustively.

6. Educational Content and Curriculum Optimization

This one’s underrated.

Online learning platforms have engagement data that most curricula never use. Which lesson sequences produce the best quiz scores? Which explanation styles reduce drop-off? Which practice problem orderings build skills fastest?

The arena: a learning management system with A/B testing capability (or a simulation model of learner behavior). The metric: quiz score improvement, lesson completion rate, or long-term retention (measured via spaced repetition performance). The program.md equivalent: the learning objectives, the prerequisite graph (some concepts must precede others), the pedagogical constraints (don't introduce more than N new concepts per lesson).

The agent proposes a curriculum modification — swap two modules, adjust the difficulty ramp, add an analogy before an abstract concept — runs it against a cohort, evaluates the metric, commits improvements.

Khan Academy, Duolingo, and Coursera already run controlled experiments on their curricula. What autoresearch provides is the closed loop: the agent proposes and evaluates variants without a human in the loop for each experiment. The human defines the optimization problem; the agent searches it.

What you design: the prerequisite graph, the learning objectives, the success metric. What the agent does: find the curriculum that maximizes learning outcomes across thousands of micro-experiments.

7. Energy Systems and Grid Optimization

Power grids, HVAC systems, and renewable energy dispatch problems are all metric-driven optimization loops — which makes them natural targets for the autoresearch pattern.

The arena: a grid simulation model or a digital twin of a building’s HVAC system. The metric: energy cost per unit output, peak demand reduction, or carbon intensity. The program.md equivalent: the physical constraints (equipment limits, safety margins), the regulatory boundaries (grid interconnection rules), and the optimization objective (minimize cost vs. minimize carbon vs. balance both).

The agent proposes a dispatch strategy modification — adjust the battery charge/discharge threshold, change the solar curtailment policy, shift a flexible load by 30 minutes — runs the simulation for the fixed time budget, evaluates the metric, commits if improved.

National Grid and utilities like E.ON are already piloting AI-assisted grid optimization. What autoresearch adds is the explicit self-improving loop: the system doesn’t just optimize once, it keeps running experiments as conditions change (new generation assets come online, load patterns shift, carbon prices move).

What you design: the physical constraints, the optimization objective, the scenario model. What the agent does: continuously improve the dispatch strategy without human intervention between sessions.

Bonus: Protein Engineering with AlphaFold 3 / OpenFold

This is where autoresearch becomes genuinely transformative for biology — not incremental, but paradigm-shifting.

Protein engineering is the art of modifying amino acid sequences to produce proteins with desired properties: higher binding affinity, greater thermal stability, improved catalytic efficiency, and reduced immunogenicity. Traditional directed evolution does this in a wet lab — mutate, express, test, repeat. It’s slow, expensive, and limited by the throughput of physical assays.

Computational protein engineering, powered by structure prediction, collapses the cycle to silicon. And autoresearch closes the loop.

The arena: AlphaFold 3 (DeepMind, 2024) or OpenFold (open-source PyTorch reimplementation, suitable for GPU-local inference) as the structure prediction engine, coupled with a scoring function for your property of interest. The metric: predicted binding affinity (ΔΔG), predicted thermostability (melting temperature Tm), or interface psolv score — all computable from the predicted structure without wet lab synthesis.

The program.md equivalent: the wildtype sequence, the functional constraints (active site residues must not be mutated, signal peptide must be preserved), the allowed mutation space (single point mutations, combinatorial within a defined region, insertions/deletions), and the target property.

The agent proposes a sequence variant — a point mutation, a loop swap, a charge modification — runs AlphaFold 3 structure prediction for the fixed time budget, evaluates the score, and commits if improved.

The key unlock is AlphaFold 3’s multimer support. Earlier versions predicted single-chain structures. AlphaFold 3 predicts protein-ligand, protein-protein, and protein-nucleic acid complexes — which means your metric can directly predict binding affinity to a target receptor, not a proxy structural score. This makes the optimization signal biologically meaningful in a way that wasn’t possible before 2024.

For teams without access to AlphaFold 3’s full API, ESMFold (Meta AI) provides fast single-sequence structure prediction in milliseconds per query — fast enough that your fixed 5-minute budget could evaluate dozens of variants per round rather than one. Combine ESMFold’s speed with ProteinMPNN for inverse folding (designing sequences that fold into a target structure), and you have a two-stage autoresearch pipeline: ProteinMPNN generates candidate sequences, ESMFold evaluates their structures, and your scoring function ranks them.

A concrete example: you’re engineering a nanobody (single-domain antibody) against a viral spike protein. Your program.md defines the CDR3 loop as the editable region, the framework residues as frozen, predicted binding affinity to the spike receptor-binding domain as the metric, and a solubility constraint (predicted instability index must remain below 40). The agent mutates CDR3, runs ESMFold + docking score, and commits improvements. Overnight: hundreds of CDR3 variants explored, all without touching a pipette.

What you design: the mutational search space, the fixed scaffold, the scoring function, and the wet lab validation gate.

What the agent does: exhaustively explore sequence space at a speed no directed evolution campaign can match.

The wet lab doesn’t disappear — it becomes the validation step for the top candidates that survive computational filtering. You’re no longer asking the lab to search; you’re asking it to confirm.

Let me know in the comments section if you want a detailed step-by-step tutorial on this use case. I am thinking maybe a simpler approach to improve the GLP1 peptides used in all weight loss medications like Ozempic, etc.

The Architecture That Makes This Possible

Three structural choices in Karpathy’s design are what make the pattern generalizable.

Fixed time budgets, not fixed experiment sizes. By capping each run at exactly 5 minutes rather than at a fixed number of training steps, the system ensures experiments are comparable regardless of what the agent changes. This is the key to making the metric meaningful. In your domain, the equivalent might be: 10 seconds of simulation time, one week of backtest data, or 1,000 user sessions.

A frozen fixture + an editable parameter space. The prepare.py file is locked — it defines the data pipeline and evaluation function. Only train.py is editable. This prevents the agent from "cheating" by changing how the metric is computed. In every domain, you must separately define what is frozen (the ground truth, the evaluation protocol) from what is fair game.

Git as the experiment log. Every successful change is a commit. Failed experiments are reverted. The result is a complete, auditable history of what the agent tried, what worked, and in what order. This is crucial for understanding why improvements stack — and for catching cases where the agent has found a metric hack rather than a genuine improvement.

The Risks You Need to Build Around

Autoresearch is not a set-and-forget oracle. There are failure modes that your program.md must explicitly address.

Metric hacking. The agent optimizes what you measure, not what you care about. If your metric is validation loss on a held-out set, the agent can overfit to idiosyncrasies in that specific evaluation set. If your metric is A/B conversion rate, the agent might find a dark pattern that temporarily lifts conversions at the cost of trust. Fix: measure multiple metrics simultaneously, with explicit constraints on secondary metrics (e.g., “improve conversion rate, but user satisfaction score cannot fall below X”).

Local optima traps. A greedy loop that only commits improvements will get stuck in local optima. Karpathy’s system runs continuously, which mitigates this somewhat, but a production autoresearch setup should include periodic random restarts or explicit exploration phases. Fix: periodically reset from a random initialization or implement simulated annealing in the commit logic.

Brittleness of improvements. An optimization found on one hardware platform or one data split may not transfer. Karpathy explicitly validated that his improvements transferred from a depth-12 to a depth-24 model before stacking them. Fix: your program.md must specify a transfer validation protocol — improvements must be verified on at least one out-of-distribution evaluation before committing.

Scope creep. Without tight constraints on what the agent is allowed to modify, it will find clever ways to technically satisfy the metric while violating the spirit of the task. Fix: explicitly enumerate both what is allowed and what is forbidden in your program.md. "You may modify any parameter in the optimizer block. You may not change the loss function. You may not change the evaluation dataset."

Your program.md Template — Start Here

The only thing standing between you and a working autoresearch loop for your domain is a well-designed program.md. Here's a template to adapt:

# program.md — [Your Domain] Autoresearch

## Goal
[One sentence: what are we optimizing?]
Example: "Minimize validation loss on the held-out set for a GPT-style language model."
Example: "Maximize Sharpe ratio on out-of-sample equity data, 2020–2022."
Example: "Maximize quiz score improvement for learners in Unit 3 of the curriculum."

## Metric
Primary: [single number we commit on]
Secondary constraints: [metrics that cannot get worse]

## Fixed (do not modify)
- [File/component 1] — reason
- [File/component 2] — reason
- [Evaluation protocol] — immutable

## Editable (fair game)
- [File/component 1] — allowed changes: [specific scope]
- [File/component 2] — allowed changes: [specific scope]

## Experiment Protocol
- Each run lasts exactly [N minutes / N seconds / N samples]
- A run is valid only if: [validity criteria]
- Transfer validation required before committing: [protocol]

## Hard constraints (never violate)
- [Safety/ethical constraint 1]
- [Domain constraint 2]
- [Regulatory/legal constraint 3]

## Hypothesis generation guidance
When proposing changes, prioritize:
1. [Domain-specific area known to have high variance]
2. [Another area with known improvement potential]
Avoid: [known dead ends or forbidden approaches]

The Next Step: SETI@home for Science

Karpathy’s current setup runs a single synchronous agent on one GPU. He’s already pointed to where this goes next.

In a follow-up post, he described the vision as massively asynchronous and collaborative — like SETI@home, but for AI-driven research. Dozens of agents working in parallel, sharing findings, deduplicating failed experiments, and building on each other’s committed improvements.

The shift: from emulating a single PhD student working late nights to emulating an entire distributed research community — except the “community” runs 24 hours a day, never gets exhausted, and never stops generating hypotheses.

For the domains above, that vision becomes: thousands of drug candidates explored simultaneously across a protein target family; hundreds of marketing strategy variants tested across different customer segments in parallel; dozens of curriculum variants running in cohort experiments at the same time.

The bottleneck, as Karpathy keeps emphasizing, is not compute. It is the quality of your arena design.

The human who writes a brilliant program.md — who defines the right metric, the right constraints, the right hypothesis space — will unlock orders of magnitude more research progress per dollar than the human who writes a mediocre one.

That is the new research job.

Your Action Items — Start Here

Immediate (< 5 minutes)

• [ ] Star the karpathy/autoresearch GitHub repo — the README contains the cleanest explanation of the loop
• [ ] Read Karpathy’s original X thread for the framing in his own words

This week

• [ ] Clone the repo and run a weekend experiment on the nanochat baseline — even if you don’t care about LLMs, running the loop teaches you more than reading about it
• [ ] Draft a program.md for one domain you work in (use the template above) — just define the metric and the constraints, even if you don't implement the loop yet
• [ ] Identify one process in your work that runs as a hypothesis-experiment-evaluate cycle — that’s your candidate domain

Ongoing

• [ ] Track Karpathy’s next release — he’s explicitly signaled a multi-agent parallelism version is coming
• [ ] Follow the NanoClaw community forks for adaptations to your domain (smaller GPUs, non-ML metrics, domain-specific fixtures)
• [ ] Revisit your program.md every month: as the agent finds improvements, the frontier moves, and the areas with highest remaining variance change

Key Resources

Primary Sources

• 🛠️ karpathy/autoresearch — GitHub
• 📄 Karpathy’s X thread announcing autoresearch — March 7, 2026
• 📄 Follow-up: 700 changes, 11% improvement — Karpathy on X

Analysis

• 📄 Karpathy Just Turned One GPU Into a Research Lab — Garry’s List
• 📄 Andrej Karpathy’s new open source autoresearch — VentureBeat
• 📄 Exploring Autoresearch — Ken Huang, Substack

Context

• 📄 How scientists are using Claude to accelerate research — Anthropic
• 📄 Anthropic multi-agent research system engineering blog
• 📄 Claude for Healthcare and Life Sciences — Anthropic

Before You Go

The shift Karpathy is describing — from researcher as experimenter to researcher as experimental designer — is one of the most significant role changes in knowledge work since the spreadsheet replaced the accountant’s ledger.

The question I can’t stop thinking about is whether most domains are ready to define a single honest metric that captures what they actually care about because that’s harder than it sounds.

A few questions I’d love your take on:

• What domain would you most want to apply the autoresearch loop to — and what’s the program.md equivalent in your field?
• What’s the metric in your work that you wish you could optimize, but haven’t had time to explore systematically?
• What breaks first when you try to design the arena for a real-world problem — the metric definition, the constraint specification, or the evaluation protocol?

If this was useful:

👏 Clap — it directly helps this reach practitioners who need it (up to 50 claps, yes it matters) 💬 Comment with your domain’s version of program.md — the best ones I'll feature in a follow-up 🔔 Follow for the next piece — I'm writing about multi-agent autoresearch and the SETI@home vision for distributed AI science

AI-assisted research is moving faster than most practitioners can track. The teams that learn to design good arenas early will compound their advantages. The ones waiting for the loop to be more “finished” will be running 30 experiments a year while their competitors run 36,500.

The program.md template in this article is available as a GitHub Gist. The autoresearch repo itself is MIT licensed — fork freely.

March 2026 — Best practices in this space are evolving rapidly. The core pattern (fixed budget · one metric · agent loop) is stable; the tooling around it is not.