← Back to list

Analysis of Anthropic’s 400,000 Recent Claude Code Sessions: What Matters More Than AI Writing Code

AI Engineering · 2026-06-16 23:50 · 0 claps · 4.3 min read paywalled
#ai #claude-code #anthropics #coding-agents #software-development
Open on Medium ↗
Wiki topics: LLM · Large Language Models AGT · AI Agents AI · AI · General 💻 · Programming

Analysis of Anthropic’s 400,000 Recent Claude Code Sessions: What Matters More Than AI Writing Code

Anthropic recently released an economic research report analyzing real-world usage of Claude Code. The study drew on nearly 400,000 user sessions between October 2025 and April 2026, conducted with full privacy protections, and its conclusions upend a lot of common assumptions about coding AI agents.

First, let’s break down how work is split between humans and AI: in real-world usage, the boundary between human and AI responsibility is surprisingly clear-cut. The research split decision-making into two categories: planning decisions, which answer the question of what to build, and execution decisions, which answer how to build it. The data shows that on average, users own 70% of all planning decisions, and only handle 20% of execution work. For Claude, that dynamic flips: the AI handles 80% of all hands-on execution work, and only participates in 30% of planning.

What that means is the dominant collaborative agentic coding model right now has humans setting direction and making the final call, while AI handles the actual work of writing code, fixing bugs and running commands. Even with high-capability agents, the user still remains the one calling the shots.

Next, looking at the success rate that users care about most, the study uncovered two fascinating findings.

First finding: Formal coding expertise has very little impact on task success rates.

The research inferred users’ professional backgrounds from their session content, and importantly, didn’t automatically classify anyone who writes code as a software professional. For example, a lawyer writing a script to batch-process legal contracts was still categorized as a legal professional. The data shows that for every major professional category included in the study, the verified success rate for coding tasks was within 7 percentage points of the success rate for professional software engineers. Even management roles actually posted a slightly higher verified success rate than software practitioners.

Software-related professions posted a 34% verified success rate for coding sessions, while all other professions averaged 29% — a gap of just 5 percentage points. When counting partial successes, the two groups posted success rates of 89% and 88% respectively: almost identical.

Second finding: Domain expertise is the core determinant of success, and most gains come from basic proficiency, not deep mastery.

Expertise here doesn’t refer to degrees or job titles — it refers to specific understanding of the task at hand: can you clearly describe what you need? Can you point out where the AI got it wrong? Can you clearly tell the AI what it needs to validate? A senior accountant who doesn’t know Python is still an expert for a task if they can clearly lay out the rules for end-of-month reconciliation and catch edge cases the AI missed. Conversely, a senior engineer writing Rust for the first time is a novice for that specific task.

The data paints a very clear picture: novices posted a 15% verified success rate, which jumped to 28% for intermediate users, and only rose to 33% for expert users. Almost all of the performance gain comes from progressing from novice to intermediate; the bump from intermediate to expert is negligible.

Another statistic drives this point home even more clearly: for every prompt a domain expert writes, Claude completes 12 consecutive actions and outputs 3,200 words. For the same prompt from a novice, Claude only completes 5 actions and outputs 600 words. Experts ask clear, precise questions, so the AI can get more work done in one go without constant back-and-forth. When tasks hit a snag, novices abandon the work 19% of the time, compared to just 5 to 7% for experts. People who know their domain know far better how to get the AI back on track.

Beyond these core conclusions, the study tracked usage trends over seven months and uncovered clear patterns.

First, the share of sessions focused solely on debugging dropped from 33% to 19%, almost a 50% cut. Correspondingly, the share of end-to-end work like deploying and running software, data analysis, and documentation writing doubled from 10% to 20%. Users are increasingly comfortable letting AI handle full, complex projects, rather than only asking it to fix bugs.

Second, the average value of every type of task increased over the period. By comparing session work to quoted rates for similar work on freelance platforms, the study estimates that the average value per session rose 27% over seven months, with value gains of more than 30% for new build, deployment, and debugging tasks. Users are increasingly using Claude Code to work on higher-value projects.

Looking at the current user base, software-related professions are still the largest user group, but the fastest-growing non-software professions are management, sales, and law. More and more professionals across every industry are starting to use coding AI agents to solve their own problems.

The core takeaway from this study is surprisingly straightforward: Coding agents are replacing coding execution skills, but they can’t replace domain judgment. You don’t need to know how to write code to get technical work done today, but you do need to understand the problem you’re trying to solve. As long as you have basic proficiency in your field, you can get the job done with a coding agent, and the gap between you and a professional coder is already negligible.

This study is an aggregated analysis conducted with full privacy protections: researchers never access individual user session content, and all statistics are compiled from anonymous aggregated data. You can read the full report on the Anthropic official website.


메타데이터
post_id
e47f78585e80
slug
analysis-of-anthropics-400-000-recent-claude-code-sessions-what-matters-more-than-ai-writing-code-e47f78585e80
url
https://medium.com/@ai-engineering-trend/analysis-of-anthropics-400-000-recent-claude-code-sessions-what-matters-more-than-ai-writing-code-e47f78585e80
canonical_url
https://medium.com/@ai-engineering-trend/analysis-of-anthropics-400-000-recent-claude-code-sessions-what-matters-more-than-ai-writing-code-e47f78585e80
author_url
https://medium.com/@ai-engineering-trend
status
ok
fetched_at
2026-06-17 12:55:42