Why Your Local AI Code Reviewer Hallucinates Line Numbers (And How to Fix It)

The frustrating gap between “running a local LLM” and “getting a code review you can actually trust” — and what closes it.

Photo by Joshua Wilkinson on Unsplash

A reader of my previous article on building a local AI code review assistant left a comment that I’ve been thinking about for weeks. I’ll paraphrase, but the gist was this:

“The possibilities of having a local-only model for code review would be great. But I’m finding that the model tends to hallucinate, or not understand the code as well as I’d hoped. Whether I use qwen2.5-coder:7b or the 32b, line numbers do not line up. I thought it was the comments at the top of the file, so I removed them — no improvement. The 7b told me to use parameterized queries while literally referencing the parameterized queries already in the code. This system would be almost usable if line numbers of reported problems actually aligned. So I have to ask: what am I doing wrong?”

That question deserves a real answer, because I’ve watched a lot of people hit exactly this wall and conclude that local code review just isn’t viable. It is viable. But the version most tutorials walk you through — including, honestly, parts of my own previous article — leaves out the engineering that turns a working-but-unreliable prototype into something you’d actually trust on a Monday morning.

This article is that missing middle. The reader isn’t doing anything wrong. The problems they’re describing are inherent to how LLMs handle code, and almost all of them are fixable with techniques that nobody writing introductory tutorials seems to mention.

Let’s get into it.

The Single Most Important Thing to Understand: LLMs Don’t Count

If you take only one idea away from this article, let it be this one:

Large language models do not count lines. They estimate positions from token patterns.

This is the root cause of the line-number problem, and most other “the model is so close but not quite right” frustrations. To understand why, you have to look at what the model actually sees when you feed it code.

A model doesn’t read your file as 247 lines of PHP. It reads it as a sequence of tokens — chunks of text that are sometimes whole words, sometimes punctuation, sometimes fragments. The string function getUserById($id) { might be split into eight or nine tokens. A blank line is a token. A docblock paragraph is many tokens. A long string literal is a token-dense region. A multi-line array initializer is a token-light region.

When you ask the model “what line is this issue on?” you are asking it a question it has no native machinery to answer. It has no counter. It has no line-by-line state. What it does instead is estimate: “based on where this issue appeared in the token stream, and based on roughly how many tokens have been newline characters so far, this is probably around line… 47? 52? Sure, let’s say 49.”

It is guessing. Sometimes well. Usually badly. Always inconsistently.

This is not a bug. This is the architecture. The same model that can flawlessly explain a complex algorithm cannot reliably tell you what line that algorithm appears on, because line numbers are a property of the physical layout of text, and the model has no representation of physical layout — only of meaning.

And once you internalize that, every fix in this article follows naturally.

Fix #1: Make the Model Read Line Numbers Instead of Generating Them

The fundamental fix for line-number drift is breathtakingly simple: don’t ask the model to generate line numbers. Give them to it as part of the input.

Before you send the file to the model, preprocess it so each line is prefixed with its line number. Something like:

1 | <?php
  2 |
  3 | namespace App\Services;
  4 |
  5 | class UserRepository
  6 | {
  7 |     public function findByEmail(string $email): ?User
  8 |     {
  9 |         $stmt = $this->pdo->prepare(
 10 |             'SELECT * FROM users WHERE email = :email'
 11 |         );
 12 |         $stmt->execute(['email' => $email]);
 13 |         return $stmt->fetch() ?: null;
 14 |     }
 15 | }

Now when the model wants to refer to a specific line, it doesn’t have to count anything. It quotes the number that’s already there. The cognitive task shifts from “estimate position from token patterns” to “copy this number that’s right next to the line I’m talking about.” That’s something LLMs are extremely good at, because copying tokens from input to output is the most basic operation they perform.

The implementation in PHP is five lines:

function prefixLineNumbers(string $code): string
{
    $lines = explode("\n", $code);
    $width = strlen((string) count($lines));
    return implode("\n", array_map(
        fn ($i, $line) => sprintf("%{$width}d | %s", $i + 1, $line),
        array_keys($lines),
        $lines
    ));
}

Pad-width matters more than you’d think. If you don’t pad, then line 9 | is followed by line 10 |, and the change in character width can confuse smaller models. Padding all line numbers to the same width (using the line count to compute it) keeps the visual structure consistent, which keeps the model's attention pattern consistent.

In my testing, this single change takes line-number accuracy from roughly 40–60% (depending on file length) to 95%+ for files that fit comfortably in the model’s context window. It’s not a panacea — we’ll cover the failure modes — but it’s the single highest-impact change you can make.

Update your prompt to tell the model about the format:

You are reviewing PHP code. Each line in the source below is prefixed
with its line number followed by ' | '. When you report an issue, quote
the line number exactly as it appears in the prefix. Do not estimate or
calculate line numbers — use only the numbers shown in the input.

That last sentence is important. Without it, some models will still try to “be helpful” by recalculating, especially smaller models that have been trained to show their reasoning.

Fix #2: The Small-Model Hallucination Problem

The reader’s second complaint is even more revealing than the first:

“The 7b told me I needed to do SQL using parameterized queries, literally referencing the parameterized query.”

This is a different beast entirely, and it has a name. In the literature, it’s sometimes called “training-data echo” or “pattern-matched advice.” In practice, it’s what happens when a small model encounters a recognized context (PHP code, SQL, the word “review”) and emits the statistically most likely advice for that context — without actually checking whether the advice applies.

The 7B-parameter version of qwen2.5-coder, like most small coder models, has been trained on enormous amounts of PHP code review content. A meaningful percentage of that content says “use parameterized queries to prevent SQL injection.” When you ask it to review PHP code that touches a database, the model’s most-probable next-token sequence often includes that advice, regardless of whether the code already uses parameterized queries.

The model isn’t lying. It’s not even confused, in any meaningful sense. It’s pattern-completing. The pattern says: PHP code that contains SQL gets a “use parameterized queries” comment from the reviewer. So it produces one.

There are three reliable ways to fight this.

Tighten the prompt with negative instructions

Most prompt engineering advice tells you what you want the model to do. For small models doing code review, the more important half is telling it what not to do, and being unusually specific:

RULES:
1. Only report issues that are actually present in the code shown.
2. If the code already uses prepared statements, parameterized queries,
   or PDO with bound parameters, DO NOT flag it for SQL injection.
3. If you mention an issue, quote the exact line of source code
   that demonstrates the issue.
4. If you cannot quote a line that demonstrates the issue, do not
   report it.
5. Generic best-practice advice is not a finding. Only report concrete
   issues visible in this file.

Rule 3 is the lynchpin. Forcing the model to quote the line demonstrating the issue means the model can’t pattern-complete a generic “use parameterized queries” finding when the line it would have to quote is itself a parameterized query. The hallucination becomes self-defeating, because executing it requires producing evidence against it.

This won’t eliminate all false positives, but it will eliminate the most egregious category — the ones where the model recommends a fix that’s already in place.

Add a verification pass

For higher-stakes reviews, a second LLM call dramatically improves accuracy. The first pass generates candidate findings. The second pass receives the findings along with the original code and is asked, for each finding: “Is this issue actually present in this code? Quote the offending line verbatim. If you cannot quote it, return ‘not present.’”

The verification pass acts as a critic. Most hallucinated findings don’t survive it, because the model can’t fabricate a quote from the source that doesn’t exist in the source. You’ll filter out a significant fraction of false positives at the cost of one extra inference call per file.

$verified = [];
foreach ($candidateFindings as $finding) {
    $verifyPrompt = sprintf(
        "Given the source code below, is this finding actually present?\n" .
        "Finding: %s\n\n" .
        "Source:\n%s\n\n" .
        "Reply with 'YES: <exact line from source>' or 'NO: not present'.",
        $finding['message'],
        $sourceWithLineNumbers
    );
    $response = $ollama->generate($verifyPrompt);
    if (str_starts_with(trim($response), 'YES:')) {
        $verified[] = $finding;
    }
}

It’s slower. It’s also dramatically more useful, because the findings you do receive have been adversarially filtered. False positive rates drop from 30–40% (typical for raw 7B output) to under 10%.

Use a bigger or better model

There’s no clever prompt that turns a 7B model into a 70B model. There are limits. If you’re consistently frustrated with qwen2.5-coder:7b, the honest answer is that the model is too small for serious code review on real-world files, and no amount of prompt engineering will fix it.

A few options worth trying, in rough order of effort vs. impact:

• qwen2.5-coder:32b — same family, much better reasoning, much less prone to pattern-matched hallucination. Needs roughly 20GB of VRAM in 4-bit quantization. If you have a 24GB GPU, this is a meaningful step up.
• deepseek-coder-v2:16b — fewer parameters than qwen2.5-coder:32b but a different training mix; in my testing, less prone to “generic advice” hallucination. Worth trying if qwen still echoes too much.
• Newer coder-tuned releases — the space moves quickly. By the time you read this, there may be a Qwen3-Coder or a DeepSeek-Coder-V3 release that surpasses everything mentioned here. Check the Ollama registry and pick the most recent coder-tuned model your hardware can run.

The pattern holds across all of them: a 7B model is a curiosity. A 30B+ model is a tool. The gap between them is not subtle.

Fix #3: Context Window Discipline

The reader didn’t mention this one, but I’d bet money it’s part of the problem. Local models have context windows that range from 8K tokens (older models) to 128K+ tokens (newer ones), but effective context — the size at which the model still pays attention to all of it — is usually much smaller than the advertised maximum.

For most coder-tuned models in the 7B-30B range, attention starts degrading noticeably past 4,000–8,000 tokens of code. This means:

• A 200-line PHP file: probably fine.
• A 1,000-line PHP file: the model is processing it, but its “attention” is unevenly distributed. Issues near the top get more weight than issues in the middle.
• A 3,000-line legacy controller: most of the file may as well not exist for the model. It will hallucinate findings about parts it didn’t actually attend to.

You can verify this empirically by passing a large file to the model with a specific bug at line 50 vs the same bug at line 2500. Smaller models will catch the line-50 bug consistently and miss the line-2500 bug consistently, even when both are nominally inside the context window.

The fix is chunked review: break the file into chunks that fit comfortably inside the model’s effective context, review each chunk independently, and aggregate findings.

A reasonable chunking strategy for PHP:

1. Split by structural boundaries, not by line count. Don’t cut a function in half. Use a lightweight parser (or even a regex tuned for PHP) to find class/function boundaries and chunk at those.
2. Include enough context for each chunk to be understandable. If you’re reviewing a single method, include the class declaration and the constructor, so the model can see what $this->repository actually is.
3. Aim for chunks of 300–500 lines maximum for 7–13B models, 800–1200 lines for 30B+ models. These are conservative numbers; tune to your specific model.
4. Deduplicate findings across chunks when they overlap. The same $user variable might appear in three chunks; you don't want three copies of the same finding.

This is more code than most local-AI-review tutorials show, and it’s not glamorous. But it’s the difference between a tool that reviews a single 200-line file well and a tool that can be pointed at a real codebase.

Fix #4: Output Format Discipline

Most local LLMs will, by default, return code review feedback as freeform prose. This is a disaster for any system you want to build on top of the review, because freeform prose is impossible to reliably parse, and it’s impossible to deduplicate findings, integrate with CI, or build any kind of dashboard around.

Force structured output. Almost every modern coder-tuned model will reliably produce JSON if you ask precisely:

Respond ONLY with a JSON array of findings. Each finding is an object
with these exact keys:
  - line: integer (the line number from the source prefix)
  - severity: one of "error", "warning", "info"
  - category: one of "security", "performance", "correctness", "style", "maintainability"
  - message: string (one sentence, under 200 characters)
  - quote: string (the exact line of source code from the input)
Return [] if there are no findings.
Do not include any text outside the JSON array.
Do not wrap the JSON in markdown code fences.

Two practical notes:

Models lie about “no findings.” A model that has been trained on code review will want to find something, because that’s what the training data shows reviewers doing. Empty arrays are unnatural. Push back against this by allowing — even encouraging — empty arrays in your prompt: “If the code is well-written, return []. Producing low-quality findings to fill the array is worse than returning none.”

Always validate the output. Even with explicit instructions, smaller models will occasionally include preamble (“Here is the JSON review:”) or wrap the response in markdown fences. Strip both before parsing:

$response = preg_replace('/^```(?:json)?\s*|\s*```$/m', '', trim($response));
$findings = json_decode($response, true);
if (!is_array($findings)) {
    // log and skip; this file's review failed
    return [];
}

Structured output also pairs beautifully with the verification pass from Fix #2. The verification step can iterate over the JSON array and filter rejected findings, leaving a clean validated list.

Fix #5: Temperature and Sampling Are Not Optional

Most tutorials run local LLMs with default sampling settings. For chat, this is fine. For code review, it’s an active source of inconsistency.

The default temperature for most Ollama models is around 0.7–0.8, which is appropriate for creative tasks (write me a poem) but actively harmful for analytical tasks (find the bugs in this code). High temperature produces output diversity, which means running the same code review twice can give you meaningfully different findings.

For code review, temperature should be low. I use 0.1–0.2:

$response = $ollama->generate([
    'model' => 'qwen2.5-coder:32b',
    'prompt' => $prompt,
    'options' => [
        'temperature' => 0.1,
        'top_p' => 0.9,
        'num_ctx' => 16384,  // explicitly set the context size you need
        'seed' => 42,        // reproducible runs
    ],
]);

Two things worth pointing out:

num_ctx defaults to 2048 on most Ollama setups. Two thousand tokens. That's nothing for a real PHP file. If you don't set this explicitly, you may be silently truncating most of your input. This is one of the most common "why isn't this working" causes I've seen, and it's invisible until you measure it. Always set num_ctx to the size you actually need.

A fixed seed makes runs reproducible, which is essential for debugging your prompt. Without it, you'll change something, run the review, get different results, and not know whether the change helped or whether you're just looking at sampling noise. With a fixed seed, the same input always produces the same output, so you can isolate the effect of each prompt tweak.

Putting It All Together

A local AI code reviewer that actually works combines all five fixes. The end-to-end pipeline:

1. Read the file.

2. Prefix every line with its line number, padded to consistent width.

3. Chunk the file by structural boundaries, with overlap and surrounding context.

4. For each chunk, call the model with:

• Low temperature (0.1)
• Fixed seed
• Explicit num_ctx matching the chunk size
• A system prompt with explicit negative instructions and an output schema

5. Parse the JSON response. Discard chunks that fail to produce valid JSON.

6. Run a verification pass on each finding. The verifier sees only the finding and the original chunk, and must quote the offending line verbatim or reject the finding.

7. Aggregate findings across chunks. Deduplicate by (line, message-similarity).

8. Sort by severity and emit.

Each of these steps is a few dozen lines of PHP. Together, they take you from “the model hallucinates line numbers and recommends fixes that are already in place” to “the model produces findings that match what a human reviewer would find, with line numbers that point at the right code.” That’s the threshold between a demo and a tool.

What You Cannot Fix

There are a few things no amount of engineering will rescue, and it’s worth naming them honestly.

Architectural review. A local model can spot security issues, point out obvious correctness bugs, suggest refactors. It cannot tell you that your authentication flow has a subtle privilege escalation, because spotting that requires holding the entire system’s data flow in mind simultaneously. Local models — even 30B+ models — don’t have the context window or the reasoning depth for whole-system reasoning. They review files. They don’t review architectures.

Novel bug detection. Models are pattern matchers. They find bugs that look like bugs they’ve seen in training data. Novel categories of bugs — the ones unique to your specific business domain — slip through. This is true of human reviewers too, but humans get better with exposure to your specific codebase; local LLMs don’t.

The 30% the model just doesn’t see. Even with everything tuned, there will be issues in your code that the model misses, and you will have no way to know which issues those are without an independent reviewer. This is not a problem you can engineer your way out of. The honest framing of local AI code review is: it’s a second pair of eyes, not a replacement reviewer. Treat it as an additional safety net, not as the safety net.

A Note on Model Choice

The reader was using qwen2.5-coder, which is a reasonable starting point. Some general advice for picking models in 2026:

• For prototyping and learning: any current coder-tuned 7B model. You’ll hit the limits described in this article, but you’ll learn the pipeline.
• For serious solo work: a 30B+ coder-tuned model. The current generation (qwen2.5-coder:32b, deepseek-coder-v2:16b, and successors) handles the patterns in this article well. Expect to need 20–40GB of VRAM, or fall back to CPU inference with patience.
• For team or org use: probably a hosted model still wins on accuracy per dollar of effort, if you can send the code. If you can’t (NDA, regulated industry, IP concerns — see my previous article), commit to a 30B+ local model and the engineering described here.

And critically: the field moves fast. The specific model names in this article may be obsolete by the time you read it. The pipeline — line-number prefixing, structured output, verification pass, low temperature, explicit context — outlasts any specific model. Build the pipeline well; swap in better models as they appear.

Back to the Original Question

To the reader who asked “what am I doing wrong?” — you were doing nothing wrong. You were following a tutorial (mine, partly) that showed you the happy path and skipped the engineering that makes the happy path actually happy. Local AI code review can work. It just requires more scaffolding than most introductions admit.

The line number problem is solved by prefixing line numbers to the input. The “recommends fixes that already exist” problem is solved by negative prompting and a verification pass. The “model loses focus on long files” problem is solved by chunking. The “results are inconsistent between runs” problem is solved by temperature and seed control. The “output is unparseable prose” problem is solved by enforcing JSON.

Stack all five, and the system stops feeling like a curiosity and starts feeling like a tool.

If you try this and hit new walls — and you will, because there are always more walls — leave a comment. The next article gets shaped by what people actually run into. That’s how this one happened.

If you found this useful, the previous article on building a local AI code reviewer is the setup this one assumes. And if you’ve built something on top of these patterns, I’d genuinely like to hear about it — especially the things that broke in ways I didn’t anticipate.