The Hidden Vulnerabilities in AI-Generated Code: A Security Engineer’s Field Guide

What years of research into Copilot, Cursor, and their peers tells us about the code your team is shipping today — and what to actually do about it

Somewhere on your team right now, a developer is accepting a suggestion from Copilot, Cursor, or Claude without reading it past the first two lines. The code compiles. The tests pass, if there are tests. It ships. Nobody asks whether the AI just handed them a SQL injection, a hardcoded credential, or an import for a package that doesn’t exist.

This isn’t a hypothetical. It’s the default state of software development in 2026, and the data on what’s actually in that generated code is no longer thin. We have multiple independent studies, a growing list of real supply-chain incidents, and a well-documented psychological trap that makes the problem worse than the raw numbers suggest. This is a field guide to what’s actually wrong, with the CWE classifications, code examples, and a defense framework you can take into your next architecture review.

What the research actually says

The first serious look at this came out of NYU in 2022, when researchers prompted GitHub Copilot to complete 89 coding scenarios spanning MITRE’s CWE Top 25 list. Across 1,689 generated programs, about 40 percent contained an exploitable bug or design flaw — and the rate was worse in C, around 50 percent, than in Python, around 39 percent. That was the first hard number anyone had, and it set the tone for everything that followed.

Since then, the picture has gotten more nuanced but not noticeably better. A 2023 study published in ACM Transactions on Software Engineering and Methodology pulled real Copilot, CodeWhisperer, and Codeium suggestions out of live GitHub projects — not lab scenarios, actual production code — and found 29.5 percent of Python snippets and 24.2 percent of JavaScript snippets carried identifiable security weaknesses, spread across 42 distinct CWE categories. A separate replication study tracking Copilot’s improvement over time found the vulnerable-suggestion rate dropping from roughly 35 percent to 25 percent as the model improved — real progress, but still a one-in-four chance that the next accepted suggestion has a flaw in it.

The most recent large-scale analysis, published in 2025, scanned over 7,700 files explicitly attributed to ChatGPT, Copilot, CodeWhisperer, and Tabnine using CodeQL, and found roughly 12 percent contained a mapped CWE — with Python again the weakest language (16 to 18.5 percent) compared to JavaScript (under 9 percent) and TypeScript (as low as 2.5 percent). The trend across five years of research is consistent even as the exact number moves around: AI assistants generate vulnerable code at a rate that should worry anyone signing off on a release.

The part that should worry you more than the percentage

Here’s the finding that changes how you should think about all of this. In 2023, a Stanford team ran a controlled user study: some developers got AI assistance on security-sensitive tasks, others didn’t. The group with AI assistance wrote measurably less secure code — and, in the same study, rated their own code as more secure than the group working without it did.

That combination is the actual danger. It’s not just that the model is sometimes wrong. It’s that the tool’s fluency makes developers less likely to apply the scrutiny they’d normally bring to unfamiliar code. A junior engineer’s pull request gets read carefully because everyone knows to expect mistakes. A Copilot suggestion that looks like idiomatic, well-formatted code from a senior engineer gets a glance and an accept. The interface doesn’t communicate uncertainty, so the human stops supplying it.

Why the model gets it wrong in the first place

None of this is a flaw that a better model fully fixes, because the failure mode is structural, not incidental.

An AI coding assistant completes the most statistically likely continuation of your prompt and the surrounding code. It has no model of your threat landscape, no awareness of which fields in your database hold PII, and no idea that the endpoint it just scaffolded sits behind an internet-facing load balancer. It also learned its patterns from a training corpus that includes a great deal of code with exactly the weaknesses described above — tutorials, Stack Overflow answers, and abandoned side projects that were never written with production security in mind. Ask it to “parse this file the user uploaded” and you’ll often get something that works perfectly on the happy path and trusts every input completely, because that’s what most of the example code it learned from does.

It also has no way to check the world. It cannot query a package registry to confirm a library exists, query your IAM policy to know what permissions are appropriate, or run a penetration test against the endpoint it just wrote. Every one of its outputs is a plausible guess rendered with total confidence — which is exactly why the false-sense-of-security effect is so consistent across studies.

A field guide to the five patterns you’ll actually see

These are the categories that show up over and over in the research above, with realistic examples of what an assistant might hand you and how to fix it.

CWE-89 — SQL injection

AI assistants frequently default to string formatting when building queries, because it’s the simplest pattern and the one most heavily represented in tutorial code.

# What the assistant suggests
@app.route("/user")
def get_user():
    user_id = request.args.get("id")
    query = f"SELECT * FROM users WHERE id = {user_id}"
    cursor.execute(query)
    return cursor.fetchone()

# What it should look like
@app.route("/user")
def get_user():
    user_id = request.args.get("id")
    cursor.execute("SELECT * FROM users WHERE id = %s", (user_id,))
    return cursor.fetchone()

The fix is one line, but the assistant won’t reach for it unless the prompt or your custom instructions explicitly demand parameterized queries.

CWE-798 — Use of hard-coded credentials

Ask for a quick script to talk to a cloud SDK and it’s common to get literal-looking key material dropped straight into the example, because that’s how a huge share of public tutorial code is written.

# What the assistant suggests
import boto3

s3 = boto3.client(
    "s3",
    aws_access_key_id="AKIAIOSFODNN7EXAMPLE",
    aws_secret_access_key="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
)

# What it should look like
import boto3

s3 = boto3.client("s3")  # resolves credentials from an IAM role, env vars, or a secrets manager

This pattern is exactly how secrets end up committed to a repository and picked up by automated scanners within minutes of a public push.

CWE-79 — Cross-site scripting

Front-end suggestions frequently render user-controlled content directly into the DOM, because innerHTML is the shortest path to the visible result the prompt asked for.

// What the assistant suggests
function renderComment(comment) {
  document.getElementById("comments").innerHTML += `<div>${comment.text}</div>`;
}

// What it should look like
function renderComment(comment) {
  const div = document.createElement("div");
  div.textContent = comment.text;
  document.getElementById("comments").appendChild(div);
}

CWE-22 — Path traversal

File-handling suggestions tend to trust the filename the user supplied, since that’s what makes the demo work on the first try.

# What the assistant suggests
@app.route("/download")
def download_file():
    filename = request.args.get("file")
    path = os.path.join("uploads", filename)
    return send_file(path)

# What it should look like
@app.route("/download")
def download_file():
    filename = secure_filename(request.args.get("file", ""))
    path = os.path.join(UPLOAD_DIR, filename)
    if not os.path.realpath(path).startswith(os.path.realpath(UPLOAD_DIR)):
        abort(403)
    return send_file(path)

Without the realpath check, a request for ../../etc/passwd walks straight out of the uploads directory.

CWE-862 — Missing authorization

This is the one that won’t show up in any static analyzer, because the code is functionally correct — it’s just missing a business rule the model had no way to know about.

// What the assistant suggests
app.get("/api/orders/:id", async (req, res) => {
  const order = await Order.findById(req.params.id);
  res.json(order);
});

// What it should look like
app.get("/api/orders/:id", requireAuth, async (req, res) => {
  const order = await Order.findById(req.params.id);
  if (order.userId !== req.user.id) {
    return res.status(403).json({ error: "Forbidden" });
  }
  res.json(order);
});

Ask the assistant to “add an endpoint to fetch an order by ID” and it will do exactly that — correctly, and with zero idea that any customer can currently fetch any other customer’s order by incrementing a number in the URL.

The newer threat: when the AI invents a dependency

Everything above is about flaws inside the code the assistant writes. The fastest-growing risk in 2026 is different — it’s about code the assistant references but didn’t write at all.

Large language models occasionally hallucinate package names: plausible-sounding libraries that don’t exist in any registry. A 2025 USENIX Security paper that generated 2.23 million code samples across sixteen popular code-generating models found that 19.7 percent contained at least one reference to a package that was never real. These aren’t random strings — researchers found the same hallucinated names recur consistently across runs, which means an attacker who simply asks the same models the same questions can predict, with real accuracy, what name to squat on.

That’s the entire attack. Register the hallucinated name on PyPI or npm before anyone else does, load it with a credential-stealing post-install script, and wait. This is now called slopsquatting, and it’s no longer theoretical: a researcher registered the name huggingface-cli as an empty placeholder after watching it get recommended by an AI assistant, and it picked up roughly 30,000 downloads in three months purely from people copy-pasting AI-suggested install commands. In January 2026, a hallucinated npm package called react-codeshift spread through 237 repositories on its own, propagated through AI-generated agent skill files rather than any single developer's mistake.

The part that should change how your team handles dependencies: a simple “does this package exist” check isn’t sufficient defense, because the attacker’s entire business model depends on making sure it does exist by the time your build runs.

Building a defense that assumes the model will be wrong

None of this is an argument against using AI coding assistants — the productivity gains are real and your competitors are using them regardless. It’s an argument for treating AI-generated code the way you’d treat code from a contractor you’ve never worked with before: useful, often correct, and unverified until proven otherwise.

A few of these are worth calling out specifically:

Treat your CI/CD gate as the real control, not code review. Reviewers get fatigued and AI output is dense; a human catching CWE-862-style missing authorization checks by reading a diff is a hope, not a control. Static analysis tools like CodeQL or Semgrep wired into a blocking CI step catch the injection and traversal classes reliably and don’t get tired at 6pm on a Friday.

Verify packages before they’re trusted, not after. Pin dependencies, check registries before install rather than relying on the install step to fail safely, and treat any newly published package with a short history and a name that closely matches something well-known as a signal worth investigating — that’s exactly the profile of a slopsquatted package.

Write security requirements into the prompt, not just the policy doc. Custom instructions or system prompts that explicitly require parameterized queries, output encoding, and centralized auth middleware measurably change what these models produce. It’s a weak control compared to a CI gate, but it’s free and it shifts the starting point.

Disclose AI-authored code in review. A one-line PR tag — “drafted with AI assistance” — costs nothing and gives reviewers permission to apply exactly the scrutiny the Stanford study showed they were otherwise skipping.

Don’t let agentic tools install dependencies unsupervised. As coding agents move from suggesting code to executing build steps autonomously, the slopsquatting window closes faster than a human would ever click through it. Any agent with install permissions needs the same registry-verification gate a human would be expected to use.

The honest takeaway

The vulnerability rate in AI-generated code isn’t going to zero, and it’s not really a model-quality problem you can wait out. It’s a structural consequence of a tool that completes patterns without context, trained on a body of code that was never uniformly secure to begin with. The studies above span four years and several generations of these models, and the rate has moved from roughly 40 percent down to roughly 12 to 25 percent depending on methodology — real improvement, but nowhere near zero, and the supply-chain risk from hallucinated packages is actively getting worse as agentic tools take on more autonomous responsibility.

The teams that will avoid the next slopsquatting headline or the next leaked credential aren’t the ones that ban AI assistants. They’re the ones that stopped treating AI output as trusted code and started treating it as what it actually is: a fast, confident, occasionally wrong contributor that needs the same gates as anyone else on the team.

If you found this useful, I write about application security, secure SDLC, and the practical side of defending modern software — follow for more.

References

• Pearce, H. et al., “Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions,” USENIX Security 2022
• Fu, Y. et al., “Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Study,” ACM Transactions on Software Engineering and Methodology, 2023
• Negri-Ribalta et al., replication study on Copilot security weaknesses across CWE Top 25 prompts, 2023
• Schreiber, M. and Tippe, P., “Security Vulnerabilities in AI-Generated Code: A Large-Scale Analysis of Public GitHub Repositories,” 2025
• Perry, N. et al., “Do Users Write More Insecure Code with AI Assistants?,” Stanford University, 2023
• “We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs,” USENIX Security 2025
• Socket.dev, Snyk, Trend Micro, and Cloud Security Alliance research on slopsquatting incidents, 2025–2026