Stop Backslashing ChatGPT 5: You’re Looking at the Wrong Problem

What the data actually shows about GPT-5’s strengths

Mastering Software Architecture for the AI Era — Bonus Post

GPT-5 understands legacy systems, then makes them modular

I do remember, and maybe it's just because I'm getting older, how familiar this kind of backlash feels. Back in the late 1990s, when Windows 98 was released, many people complained that Windows 95 was better. Tech magazines were filled with side-by-side comparisons claiming the older version was faster, more intuitive, and more reliable. Features like the new “Active Desktop” were called useless bloat, and although Windows 98 added real improvements like multi-monitor support, Internet Connection Sharing, and the first version of Windows Update, the overall feeling at the time was disappointment. People weren’t ready to see the familiar system they had just adapted to replaced so quickly.

In the early 2000s, the same thing happened with MySpace. Part of its popularity came from how users could customize their profiles with backgrounds, music, and widgets. When redesigns started removing some of that customization, people revolted. Petitions circulated, blogs filled with angry posts, and some users abandoned the platform. For many, the feeling was that something fun and personal had been taken away without their approval.

The pattern repeats itself. When Facebook introduced the News Feed in 2006, millions of users formed protest groups demanding its removal. At that time, it seemed like an invasion of privacy, as people’s activities were suddenly broadcast on a central page. Then, in 2011, when Facebook launched the Timeline design, the backlash was just as strong, with users complaining that it was confusing, cluttered, and unnecessary.

Apple experienced a similar reaction in 2013 when iOS 7 was launched. The glossy skeuomorphic icons and textures that once defined the iPhone experience were replaced. In their stead was a flat, minimalist design featuring heavy use of parallax and zoom animations. Some users even reported motion sickness, and critics argued that the new design was more difficult to navigate. Despite this, Apple moved forward, claiming it was the biggest change to iOS since the iPhone’s debut.

The lesson is straightforward. Humans, in general, tend to resist change. For many, change seems threatening or uncomfortable, and at first, it rarely appears logical.

Part of the reason is how our brains are wired. Change causes a stress response because the brain’s amygdala sees uncertainty as a threat. This releases stress hormones like cortisol, making people feel anxious and defensive. Meanwhile, the brain’s reward system favors routine and predictability because familiar patterns take less mental effort. In other words, sticking with what we know feels safe, while change seems risky and costly, even if it might lead to positive results in the end.

I try not to take those reactions for granted. Instead, I want to understand what the other side sees, and whether there is a deeper story hidden beneath the noise. That is why this past weekend I set aside some time to read the research on GPT-5, run experiments, and look closely at what the data actually shows.

The Wrong Question

Most of the criticism around ChatGPT-5 focuses on the wrong metric. The usual benchmarks you see online or in think pieces are about whether it can produce clean, working code on the first try. That is like judging a master carpenter only by how quickly they can hammer a nail. Sure, it is measurable, but it misses the point of where the true skill actually lies.

What the research community has begun to notice is that GPT-5's greatest improvements over earlier models are not in raw code generation but in long-context reasoning, dependency tracking, and maintaining functional consistency across large codebases. In other words, GPT-5 excels in understanding and restructuring existing systems rather than creating a completely new file from scratch.

This is why many quick hot-takes miss the mark. They describe GPT-5 as a “junior developer that writes buggy code,” but the real story is that it acts more like a code archaeologist capable of tracing logic across thousands of lines, identifying hidden dependencies, and preserving behavior while restructuring.

When you combine that with contracts, the picture becomes even clearer. Research on Software Contracts as First-Class Abstractions (ACM SIGPLAN, 2019: https://dl.acm.org/doi/10.1145/3314221.3314610) shows how explicit contracts improve modularity and testability. GPT-5 can use those contracts as anchors, guiding safe refactoring. And when contracts are missing, its long-context abilities enable it to infer implicit behavior, effectively creating “virtual contracts” from the code itself.

This matters because real-world engineering isn't about writing isolated functions all day. It's about managing complex systems that have become messy over time. Modernization projects, API migrations, or breaking monoliths into microservices are core tasks in serious software work. Benchmarks that ignore this reality are asking the wrong questions, and in doing so, they underestimate GPT-5’s most valuable strength.

Extracting Capabilities from a Monolith

To see where GPT-5 truly excels, consider a common challenge: a large monolithic codebase where all business logic is intertwined. Imagine being asked to separate the user authentication feature into its own service.

Here is a simplified version of what the monolith might look like in Python:

# legacy_app.py
def handle_request(request):
    user = get_user(request["user_id"])
    if not check_password(user, request["password"]):
        return {"error": "Invalid credentials"}

    if not has_permission(user, request["resource"]):
        return {"error": "Access denied"}

    # business logic continues...
    return {"data": "some important result"}
def get_user(user_id):
    # fetch from database
    pass
def check_password(user, password):
    # compare with hash
    pass
def has_permission(user, resource):
    # permission logic
    pass

In a monolith like this, authentication, authorization, and core business logic are all mixed together.

When prompted with “Extract the authentication capability into a standalone service. Show inputs, outputs, and dependencies. GPT-5 produces something like this:

# auth_service.py
class AuthService:
    def __init__(self, user_repo):
        self.user_repo = user_repo
    def authenticate(self, user_id, password):
        user = self.user_repo.get_user(user_id)
        if not self.check_password(user, password):
            return {"error": "Invalid credentials"}
        return {"status": "ok", "user": user}
    def check_password(self, user, password):
        # compare with hash
        pass

And it identifies the contract of this new service:

Contract: AuthService.authenticate(user_id: str, password: str) -> {status: "ok" | "error", user?: User}

Notice what happened here. GPT-5 didn't just rearrange code. It unraveled dependencies, integrated the authentication logic into a clear boundary, and identified the contract that defines the interaction. This is exactly the kind of task that is tedious and error-prone for humans, especially at scale, and precisely what GPT-5’s long-context reasoning is designed for.

This diagram shows GPT-5 separating the authentication capability from a monolithic request handler. Authentication becomes a standalone service with a clear contract, authenticate(user_id, password), which makes testing, maintenance, and independent deployment easier.

Refactoring in Practice

When engineers hear “AI writes code,” their immediate thought is a machine generating snippets. But real software work isn't just about creating new code; it's about reshaping existing code. Refactoring, not greenfield development, accounts for much of the actual effort and risk.

Recent benchmarks confirm that GPT-5 is strong in this domain.

[embed]

You can see the official OpenAI announcement with benchmarks here: https://openai.com/index/introducing-gpt-5-for-developers Media analysis has confirmed the same numbers: PC Gamer coverage: https://www.pcgamer.com/software/ai/openais-performance-charts-in-the-gpt-5-launch-video-are-such-a-mess-you-have-to-think-gpt-5-itself-probably-made-them-and-the-companys-attempted-fixes-raise-even-more-questions AWS Plain English breakdown: https://aws.plainenglish.io/gpt-4-vs-gpt-5-how-a-74-9-swe-bench-score-rewires-coding-fa2b08907067 Vellum benchmark summary: https://www.vellum.ai/blog/gpt-5-benchmarks WIRED developer reactions: https://www.wired.com/story/gpt-5-coding-review-software-engineering

On SWE-bench Verified, which assesses how well a model can analyze actual GitHub repositories and generate bug fixes, GPT-5 scored 74.9 percent, compared to 54.6 percent for GPT-4.1. In the Aider Polyglot Code Editing benchmark, which tests refactoring tasks across multiple languages, GPT-5 achieved an 88 percent pass rate with reasoning enabled, significantly surpassing GPT-4.1’s 52 percent.

These results demonstrate that GPT-5 is not only better at generating new code but also significantly more capable of analyzing, understanding, and restructuring large codebases. This ability directly benefits core tasks in software projects such as maintenance, modernization, and safe refactoring.

Consider this simplified Java example:

public class LegacyUserManager {
    public void handleUser(String username, String password) {
        if (!authenticate(username, password)) {
            throw new RuntimeException("Auth failed");
        }
        logAction(username);
        sendWelcomeEmail(username);
        updateUserRecord(username);
    }
}

Prompt GPT-5 to “refactor into focused classes while preserving behavior,” and it might produce:

public class AuthService { … }
public class AuditService { … }
public class NotificationService { … }
public class UserService {
    // Refactored with clear dependencies, consistent behavior
}

This isn’t flashy. It’s essential. Most teams spend endless hours disentangling legacy code. GPT-5 can accelerate that work, giving engineers the headspace to validate, test, and innovate.

Contracts as Anchors

One of the biggest misconceptions about GPT-5 is that it performs best when creating new code from scratch. In reality, its effectiveness is heightened when code is organized around contracts that clearly specify what a component expects and guarantees.

Contracts serve as the “anchors” that enable GPT-5 to analyze large, complex systems. Instead of viewing code as an unstructured collection of functions, it can utilize explicit contracts as reference points for safe navigation and modification.

This isn’t a new idea. Research on software contracts shows that they increase modularity, make systems more testable, and reduce the risks of change. A good reference is the ACM paper Software Contracts as First-Class Abstractions, which you can find here: https://dl.acm.org/doi/10.1145/3314221.3314610

In practice, GPT-5 can use these contracts in two ways:

1. When contracts are present: it uses them as strong guardrails to preserve behavior while refactoring.
2. When contracts are missing: it can often infer implicit behavior, generating what are effectively “virtual contracts” from patterns in the code.

Imagine a legacy service that integrates authentication, logging, and notification logic into a single method. A contract-driven version might define:

// Contract: authenticate(user, password) -> boolean
// Contract: logAction(user, action) -> void
// Contract: sendWelcomeEmail(user) -> void

With these contracts, GPT-5 not only splits the code but also ensures that every new service meets its original promises. That’s what enables large-scale refactoring.

This is crucial in real-world engineering. Modernization projects, API migrations, and breaking monoliths into microservices depend on how well implicit assumptions are identified. GPT-5, combined with contract-driven development, makes those assumptions explicit and testable.

Diagram: Contracts act like landmarks in a messy legacy codebase. GPT-5 analyzes the mixed handler, infers or uses explicit contracts, then guides a clean split into focused services that preserve original guarantees.

Understanding Large Codebases

Most legacy work begins with a simple question that’s tough to answer quickly: where does this thing actually reside? GPT-5 helps by holding and reasoning over very large inputs at once, which is exactly what’s needed when mapping a big repo. OpenAI’s public materials list a very large context window for GPT-5 in the API and product docs, enabling the feeding of file trees, key source files, logs, and specs together without losing context. See: https://openai.com/gpt-5 and https://platform.openai.com/docs/models/gpt-5-chat-latest and https://platform.openai.com/docs/models/gpt-5-mini.

A simple workflow you can copy

1 Give it the map first Provide a compact file tree and the paths of the most important files. Ask for a feature map and cross-module entry points.

You are analyzing a legacy repository. Here is the file tree (trimmed) and a set of key files.
Goal: identify all entry points for "user auth" and the modules they touch. Return a dependency map and a list of side effects to watch.

2 Ask for a dependency graph plus hotspots Request a call graph for the feature, plus a list of risky couplings, global state, and I/O boundaries.

3 Validate with a quick grep pass Have the model generate ripgrep commands you can run locally to confirm the hotspots it identified.

# example commands the model can suggest
rg -n "authenticate\\(|login\\(|verifyToken\\(" -g "!dist" -g "!node_modules"
rg -n "UserService|AuthService|Permission" -S src/

4 Have it propose contracts If the code lacks an explicit interface, ask GPT-5 to infer a minimal contract for the capability you plan to extract. Use that as an anchor in your refactor plan. For background on why this helps, see the ACM paper “Software Contracts as First-Class Abstractions”: https://dl.acm.org/doi/10.1145/3314221.3314610.

Why this works in practice

Benchmarks that challenge real repositories, like SWE-bench Verified, demonstrate GPT-5's ability to navigate and edit code in context, which is the same skill needed for mapping a large codebase before a refactor. OpenAI’s public posts summarize these results, which you can reference here: https://openai.com/index/introducing-gpt-5 and https://openai.com/index/introducing-gpt-5-for-developers.

Diagram: GPT-5 ingests a trimmed file tree and key files, identifies the real entry points and side effects, then proposes minimal contracts for the capability you plan to extract. Those contracts become anchors for a safe refactor.

“Show me” example, tiny repo walk-through

Here is a copy-pasteable prompt and a tiny repo you can use to demonstrate how GPT-5 maps a codebase, finds entry points, infers contracts, and proposes a safe extraction. Keep it small so readers can follow the flow.

Step 1, a tiny repo

repo/
  package.json
  src/
    api/
      login.ts
      users.ts
    services/
      auth.ts
      permissions.ts
      email.ts
      user.ts
    db/
      index.ts

Key files, trimmed for clarity

// src/api/login.ts
import { AuthService } from "../services/auth";
import { Permissions } from "../services/permissions";
import { Email } from "../services/email";
import { UserRepo } from "../services/user";

export async function login(req, res) {
  const { username, password } = req.body;
  const user = await UserRepo.findByUsername(username);
  const ok = await AuthService.authenticate(user, password);
  if (!ok) return res.status(401).json({ error: "Invalid credentials" });
  const allowed = await Permissions.canAccess(user, "portal");
  if (!allowed) return res.status(403).json({ error: "Access denied" });
  await Email.sendWelcome(user.email);
  return res.json({ status: "ok", id: user.id });
}

// src/services/auth.ts
import { hashCompare } from "../db";
export const AuthService = {
  async authenticate(user, password) {
    return hashCompare(password, user.passwordHash);
  },
};

// src/services/permissions.ts
export const Permissions = {
  async canAccess(user, resource) {
    if (user.role === "admin") return true;
    return resource === "portal";
  },
};

// src/services/email.ts
export const Email = {
  async sendWelcome(to: string) {
    // calls external SMTP
  },
};

// src/services/user.ts
export const UserRepo = {
  async findByUsername(u: string) {
    // DB call, returns user { id, email, role, passwordHash }
  },
};

Step 2: Copy-paste the prompt to GPT-5

You are a senior engineer helping map a legacy codebase before a refactor.
Goals:
1) List all entry points for user authentication and the modules they touch.
2) Produce a dependency map for login, including side effects.
3) Infer minimal contracts for capabilities we might extract.
4) Propose a safe extraction plan that preserves behavior and reduces coupling.
5) Generate 3–5 shell commands I can run locally to validate your findings.
Inputs:
- File tree (trimmed)
- Key source files (pasted below)
Output format:
- Entry points
- Dependency map
- Side effects
- Inferred contracts
- Extraction plan
- Validation commands

Paste the file tree and the key files after the prompt.

Step 3: What a solid GPT-5 answer should look like

Entry points

• src/api/login.ts::login(req, res)

Dependency map

• login uses UserRepo.findByUsername, AuthService.authenticate, Permissions.canAccess, Email.sendWelcome
• AuthService.authenticate depends on db.hashCompare
• Permissions.canAccess reads user.role
• Email.sendWelcome calls external SMTP

Side effects

• Database read on user lookup
• External SMTP send on welcome email
• HTTP response codes 401 and 403 for failure cases

Inferred contracts

authenticate(user, password) -> boolean
canAccess(user, resource) -> boolean
sendWelcome(email) -> void
findByUsername(username) -> User

Extraction plan

1. Create an AuthCapability with two interfaces: `- IAuth.authenticate(userId or username, password) -> { ok: boolean, user?: User }

• IAuthorize.canAccess(userId or user, resource) -> boolean`

2. Move AuthService and Permissions into an auth module, expose only the interfaces above.
3. Decouple email by injecting an INotifier.sendWelcome(email) -> void dependency, do not call SMTP directly from the API layer.
4. Keep UserRepo as the data boundary, pass user ids instead of whole objects across module lines.
5. Add contract tests that assert status codes for invalid credentials and access denied, and use golden responses from the current behavior.

Validation commands

rg -n "login\\(" src/api
rg -n "authenticate\\(" src
rg -n "canAccess\\(" src
rg -n "sendWelcome\\(" src
rg -n "passwordHash|hashCompare" src

Step 4: Optional glue code that the model can generate

Interfaces to anchor the refactor

// contracts/auth.ts
export interface IAuth {
  authenticate(username: string, password: string): Promise<{ ok: boolean; user?: User }>;
}
export interface IAuthorize {
  canAccess(userId: string, resource: string): Promise<boolean>;
}
export interface INotifier {
  sendWelcome(email: string): Promise<void>;
}

Refactored API using contracts

// src/api/login.ts
export function makeLogin(auth: IAuth, authz: IAuthorize, notify: INotifier, users: UserRepo) {
  return async function login(req, res) {
    const { username, password } = req.body;
    const { ok, user } = await auth.authenticate(username, password);
    if (!ok || !user) return res.status(401).json({ error: "Invalid credentials" });

    const allowed = await authz.canAccess(user.id, "portal");
    if (!allowed) return res.status(403).json({ error: "Access denied" });

    await notify.sendWelcome(user.email);
    return res.json({ status: "ok", id: user.id });
  };
}

Why this matters for the real industry

Modernization isn't a side project; it's the core focus for most software organizations. Multiple studies demonstrate that tech debt and legacy complexity take up a significant portion of time and resources, where GPT-5’s abilities in understanding, refactoring, and contract alignment offer substantial benefits.

Budgets and tech-debt drag CIOs report that 10 to 20 percent of the technology budget designated for new products is diverted to managing technical debt, and many estimate that tech debt accounts for 20 to 40 percent of the total value of their entire technology estate.

• Source, McKinsey overview: https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/tech-debt-reclaiming-tech-equity McKinsey & Company
• Deeper discussion of the same finding, including the budget diversion and operational impact: https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/breaking-technical-debts-vicious-cycle-to-modernize-your-business McKinsey & Company
• A related McKinsey piece quantifies the estate impact and budget diversion in more detail: https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/demystifying-digital-dark-matter-a-new-standard-to-tame-technical-debt McKinsey & Company

Developer time lost to maintenance and bad code Stripe’s Developer Coefficient study found that developers spend over 17 hours each week on maintenance tasks like debugging and refactoring, with about 4 hours per week on bad code, which Stripe estimates results in tens of billions of dollars in lost opportunity cost annually.

• Full report PDF: https://stripe.com/files/reports/the-developer-coefficient.pdf Stripe

Modernization is a board-level market, not a niche Forrester’s 2025 report highlights application modernization as a core market, with vendors evaluated based on modernization and migration capabilities, which are increasingly enhanced by generative AI.

• Overview blog: https://www.forrester.com/blogs/introducing-the-forrester-wave-application-modernization-and-multicloud-managed-services-q1-2025/ Forrester
• Prior coverage pointing to rising demand for modernization services: https://www.forrester.com/blogs/introducing-the-forrester-wave-applications-modernization-and-migration-services-q1-2024/ Forrester

Benchmarks that reflect real engineering work SWE-bench tests models on real GitHub issues, where the task is to understand a repository and create a patch. This closely matches workflows involving refactoring and code understanding, not just starting from scratch.

• Leaderboards and docs: https://www.swebench.com/ and https://www.swebench.com/SWE-bench/ SWE-bench+1
• OpenAI’s description of SWE-bench Verified explains the human-validated subset used in many public evaluations: https://openai.com/index/introducing-swe-bench-verified/ OpenAI

So what should teams do differently

1. Point GPT-5 at the debt, not the blank file. Start with understanding the code, tracing dependencies, and safe extractions instead of adding new features.
2. Use contracts as scaffolding. Where explicit contracts exist, feed them to GPT-5 and enforce them in tests. Where they do not, have the model infer minimal contracts and validate them. Background on contract value: https://dl.acm.org/doi/10.1145/3314221.3314610 McKinsey & Company
3. Measure with realistic benchmarks. Track success on repo-level tasks, such as patch correctness and regression outcomes, rather than only on snippet-level codegen. SWE-bench offers a solid template for this type of evaluation.
4. Make change observable. Pair refactors with contract tests and regression suites to ensure behavior stays consistent while the structure gets better.

Hands-on, contracts-first refactor

Below is a small, copyable example demonstrating how contracts make GPT-5 safer and faster at refactoring. It begins with a mixed handler, adds explicit contracts, then uses a short prompt to facilitate automated extraction. Everything is intentionally kept tiny so readers can easily follow or replicate it in their own repo.

1) Before, mixed responsibilities

// src/api/login.ts
import { db } from "../db";
import { hashCompare } from "../crypto";
import { send } from "../smtp";

export async function login(req, res) {
  const { username, password } = req.body;
  const user = await db.users.findByUsername(username);
  if (!user) return res.status(404).json({ error: "User not found" });
  const ok = await hashCompare(password, user.passwordHash);
  if (!ok) return res.status(401).json({ error: "Invalid credentials" });
  const allowed = user.role === "admin" || req.query.feature === "portal";
  if (!allowed) return res.status(403).json({ error: "Access denied" });
  await send(user.email, "welcome", { name: user.name });
  return res.json({ status: "ok", id: user.id });
}

Problems, authentication logic, authorization rules, and notification side effects live inside the HTTP layer. Unit tests and reuse are hard.

2) Add explicit contracts

// contracts/auth.ts
export interface IAuth {
  authenticate(username: string, password: string): Promise<{ ok: boolean; user?: User }>;
}
export interface IAuthorize {
  canAccess(userId: string, resource: string): Promise<boolean>;
}
export interface INotify {
  sendWelcome(email: string): Promise<void>;
}
export interface IUsers {
  findByUsername(username: string): Promise<User | undefined>;
}
export type User = { id: string; name: string; email: string; role: "admin" | "user"; passwordHash: string };

Optional OpenAPI shape for the endpoint that must still behave the same:

# contracts/login.yaml
openapi: 3.0.0
paths:
  /api/login:
    post:
      requestBody:
        required: true
        content:
          application/json:
            schema: { $ref: "#/components/schemas/LoginRequest" }
      responses:
        "200": { $ref: "#/components/responses/Ok" }
        "401": { $ref: "#/components/responses/AuthFailed" }
        "403": { $ref: "#/components/responses/Denied" }

3) Copy, paste, and run this prompt with GPT-5

You are refactoring a legacy handler into contract-driven modules.

Goals:
1) Keep HTTP behavior identical for 200, 401, 403, and 404.
2) Move authentication, authorization, and notification behind the contracts.
3) Inject dependencies, do not import db or smtp in the HTTP layer.
4) Generate a quick contract test that locks current behavior.
Inputs:
- contracts/*.ts and optional OpenAPI snippet
- src/api/login.ts
Deliverables:
- refactored login factory that takes IAuth, IAuthorize, INotify, IUsers
- minimal implementations that call existing helpers for now
- a Jest test file that asserts the four outcomes
- a short plan for extracting real implementations later

4) After, extracted and contract-driven

// src/api/login.refactored.ts
import { IAuth, IAuthorize, INotify, IUsers } from "../../contracts/auth";

export function makeLogin(auth: IAuth, authz: IAuthorize, notify: INotify, users: IUsers) {
  return async function login(req, res) {
    const { username, password } = req.body;
    const { ok, user } = await auth.authenticate(username, password);
    if (!ok || !user) return res.status(401).json({ error: "Invalid credentials" });
    const allowed = await authz.canAccess(user.id, "portal");
    if (!allowed) return res.status(403).json({ error: "Access denied" });
    await notify.sendWelcome(user.email);
    return res.json({ status: "ok", id: user.id });
  };
}

// adapters/auth.impl.ts
import { IAuth, IAuthorize, IUsers, User } from "../../contracts/auth";
import { db } from "../db";
import { hashCompare } from "../crypto";

export const usersImpl: IUsers = {
  async findByUsername(username) {
    return db.users.findByUsername(username) as Promise<User | undefined>;
  },
};
export const authImpl: IAuth = {
  async authenticate(username, password) {
    const user = await usersImpl.findByUsername(username);
    if (!user) return { ok: false };
    const ok = await hashCompare(password, user.passwordHash);
    return ok ? { ok: true, user } : { ok: false };
  },
};
export const authzImpl: IAuthorize = {
  async canAccess(userId, resource) {
    const user = await db.users.findById(userId);
    if (!user) return false;
    return user.role === "admin" || resource === "portal";
  },
};

// adapters/notify.impl.ts
import { INotify } from "../../contracts/auth";
import { send } from "../smtp";
export const notifyImpl: INotify = { async sendWelcome(email) { await send(email, "welcome", {}); } };

// tests/login.contract.test.ts
import { makeLogin } from "../src/api/login.refactored";
import { authImpl, authzImpl, usersImpl } from "../src/adapters/auth.impl";
import { notifyImpl } from "../src/adapters/notify.impl";

function mockRes() {
  const res: any = {};
  res.status = (c: number) => ((res.code = c), res);
  res.json = (b: any) => ((res.body = b), res);
  return res;
}
test("404 when user not found", async () => {
  const login = makeLogin(authImpl, authzImpl, notifyImpl, { ...usersImpl, findByUsername: async () => undefined });
  const res = mockRes();
  await login({ body: { username: "x", password: "p" } } as any, res);
  expect(res.code).toBe(404);
});
test("401 invalid credentials", async () => {
  const login = makeLogin({ ...authImpl, authenticate: async () => ({ ok: false }) }, authzImpl, notifyImpl, usersImpl);
  const res = mockRes();
  await login({ body: { username: "x", password: "p" } } as any, res);
  expect(res.code).toBe(401);
});
test("403 access denied", async () => {
  const authz = { ...authzImpl, canAccess: async () => false };
  const res = mockRes();
  await makeLogin(authImpl, authz, notifyImpl, usersImpl)({ body: { username: "a", password: "b" } } as any, res);
  expect(res.code).toBe(403);
});
test("200 ok", async () => {
  const res = mockRes();
  await makeLogin(authImpl, authzImpl, notifyImpl, usersImpl)({ body: { username: "a", password: "b" } } as any, res);
  expect(res.body.status).toBe("ok");
});

5) Quick verification commands

# find all call sites before and after
rg -n "login\\(" src
rg -n "sendWelcome\\(" src
# prove the HTTP layer no longer imports db or smtp
rg -n "from \"../db\"|from \"../smtp\"" src/api/login.refactored.ts

Why this works The contracts establish clear boundaries, allowing GPT-5 to rewire dependencies without altering behavior. Tests ensure the four outcomes are consistent. Adapters maintain legacy helpers, making it safe to ship refactors incrementally.

Small contracts, big leverage.

Takeaways

• Most hot takes criticize GPT-5 based on new code. The real value appears in refactoring and extracting capabilities from messy systems.
• Contracts, whether explicit or implied, turn GPT-5 into a safe accelerator. They provide the model with anchors and give teams testable guarantees.
• Repo level benchmarks and practical examples are more important than snippet scores. Focus on measuring patch correctness, regression stability, and behavior parity.
• Focus GPT-5 on discovery, mapping, and scaffolding. Keep humans involved for validation, integration, and rollout planning.

Limitations to keep teams honest

• Refactors still require tests. Contract tests and a small regression suite are essential before merging.
• Models can overlook rare edge cases. Maintain a quick rollback option and review logs before and after.
• Inferred contracts are a starting point, not an absolute truth. Verify with owners and real traffic whenever possible.
• Tooling is important. Make it simple to supply file trees, key files, and specs. Without good inputs, the outputs will be affected.

References and links

• OpenAI benchmarks and GPT-5 overview https://openai.com/index/introducing-gpt-5-for-developers
• SWE-bench and SWE-bench Verified background https://www.swebench.com/ https://openai.com/index/introducing-swe-bench-verified/
• Contract research, ACM paper https://dl.acm.org/doi/10.1145/3314221.3314610
• Independent coverage and analysis PC Gamer https://www.pcgamer.com/software/ai/openais-performance-charts-in-the-gpt-5-launch-video-are-such-a-mess-you-have-to-think-gpt-5-itself-probably-made-them-and-the-companys-attempted-fixes-raise-even-more-questions
• Vellum benchmark summary https://www.vellum.ai/blog/gpt-5-benchmarks
• AWS Plain English SWE-bench context https://aws.plainenglish.io/gpt-4-vs-gpt-5-how-a-74-9-swe-bench-score-rewires-coding-fa2b08907067

Copy-paste prompt pack

Repo mapping

You are analyzing a legacy repo. Return: entry points for "auth", a dependency map, side effects, and a minimal contract for the capability to extract. Inputs: trimmed file tree, key files.

Contract inference

Infer minimal contracts for auth, authz, notify from these files. Show inputs, outputs, invariants, and failure modes. Output as TypeScript interfaces and optional OpenAPI stubs.

Safe extraction

Refactor the handler to depend on IAuth, IAuthorize, INotify, IUsers. Keep HTTP behavior identical for 200, 401, 403, and 404. Generate a Jest test locking current behavior.

Validation commands

Generate 5 ripgrep commands to confirm call sites and prove the HTTP layer no longer imports db or smtp.

🧠 Found this valuable? If this post sparked new thinking or gave you something useful to take back to your team, consider hitting the 👏 button. It also helps others discover the series.

🚀 Following along? This article is part of the ongoing series: Mastering Software Architecture for the AI Era: Designing & Building with AI-Driven Architecture Principles

[embed]Mastering Software Architecture for the AI Era This series explores how AI is transforming software architecture, offering practical insights into modular design…medium.com

Each post examines how AI is transforming the way we design, scale, and reason about software systems.