Q-GPT: Teaching AI to Know When It Doesn’t Know

How Quantum Computing Helps Detect LLM Hallucinations

Can we make language models honest about their uncertainty? I built a quantum neural network to find out.

The Confidence Problem

You’ve probably noticed that ChatGPT, Claude, and other AI assistants have a peculiar trait: they always sound confident. Ask them about quantum physics or anything else, and they’ll respond with the same assured tone.

This is a problem.

Large Language Models (LLMs) are trained to produce fluent, coherent text — not to express uncertainty. They’ll confidently tell you that the first person to walk on Mars was Neil Armstrong (he wasn’t — he walked on the Moon, and no one has walked on Mars yet). They’ll invent scientific papers that don’t exist. They’ll hallucinate.

What if we could teach AI to say “I’m not sure about this”?

Enter Q-GPT

Q-GPT is my attempt to solve this problem using an unlikely tool: quantum computing.

The idea is simple: attach a small quantum neural network to any language model. This “quantum head” analyzes the model’s internal state and estimates how confident the model actually is — not how confident it sounds.

How It Works

When an LLM generates text, it doesn’t just output words. Internally, it processes high-dimensional vectors called “hidden states” — think of them as the model’s thoughts before they become words.

Q-GPT intercepts these hidden states and runs them through a variational quantum circuit:

LLM's "Thoughts"  →  [Compress]  →  [Quantum Circuit]  →  Confidence Score

(2880 dims)         (4 dims)       (4 qubits)           (0-100%)

Step 1: Compression

The hidden states are massive — 2880 dimensions for GPT-OSS. A small neural network compresses this to just 4 numbers, normalized between -1 and +1.

Step 2: Quantum Encoding

These 4 numbers become rotation angles for 4 qubits. Each qubit starts in state |0⟩ and gets rotated based on what the model was “thinking.”

|0⟩  →  RY(θ₁)  →  RZ(θ₁)  →  |ψ₁⟩

Step 3: Entanglement

Here’s where quantum gets interesting. CNOT gates create entanglement between qubits — a phenomenon where measuring one qubit instantly affects others, no matter how far apart.

This entanglement lets the circuit capture complex correlations in the data that classical neural networks struggle with.

Step 4: Measurement

We measure the expectation value ⟨Z⟩ of each qubit. This gives us 4 numbers that encode information about the model’s uncertainty in a fundamentally different way than classical approaches.

Step 5: Output

A final layer converts these quantum measurements into:

• Confidence (0–100%): How sure is the model?
• Uncertainty: A quantum-derived uncertainty measure
• Should Refuse: A flag suggesting whether the model should decline to answer

Why Quantum?

You might be wondering: why not just use a regular neural network?

Three reasons:

1. Natural Uncertainty Representation

Quantum mechanics is inherently probabilistic. Unlike classical bits (0 or 1), qubits exist in superposition — they’re both 0 AND 1 simultaneously. This makes them naturally suited for representing uncertainty.

2. Entanglement Captures Correlations

When qubits are entangled, they can represent correlations that would require exponentially more classical parameters. Four entangled qubits can capture patterns that 16 classical neurons couldn’t.

3. Compact but Powerful

The quantum circuit has only ~2,000 trainable parameters (the rotation angles), yet it can represent incredibly complex transformations. This makes it efficient to train and deploy.

Real-World Example

from q_gpt import load_qgpt

model, tokenizer = load_qgpt("squ11z1/gpt-oss-nano")

# Ask something factual

outputs = model.generate_with_confidence(

"What is the capital of France?"

)

# Confidence: 94% ("very high")

# Should refuse: False

# Ask something obscure

outputs = model.generate_with_confidence(

"What was the third word spoken on the Moon?"

)

# Confidence: 31% ("low")

# Should refuse: True

When Q-GPT detects low confidence, it signals that the model should either refuse to answer or clearly state its uncertainty.

The Bigger Picture

Q-GPT is part of a broader vision: AI systems that know their own limitations.

Imagine a medical AI that says “I’m 95% confident this is a benign mole, but you should see a dermatologist” versus one that just says “This is probably fine.”

Imagine a coding assistant that says “I’m not sure about this edge case — please verify” instead of confidently introducing bugs.

Honest AI is safer AI.

Try It Yourself

Q-GPT is open source and available on HuggingFace:

• Model: huggingface.co/squ11z1/Q-GPT
• Base LLM: huggingface.co/squ11z1/gpt-oss-nano

Install with:

pip install pennylane torch transformers

The code includes a classical fallback, so you can experiment even without a quantum simulator.

What’s Next?

This is just the beginning. Future directions include:

• Running on real quantum hardware (currently it’s simulated)
• Training on larger datasets of model correctness
• Extending to multimodal models (images, code, etc.)
• Fine-grained uncertainty — not just “confident or not” but “confident about facts, uncertain about opinions”

Conclusion

Q-GPT demonstrates that quantum computing isn’t just for breaking encryption or simulating molecules — it can also help make AI more honest and trustworthy.

The next time an AI gives you information, wouldn’t you want to know: “How confident are you, really?”

If you found this interesting, check out the code on HuggingFace and let me know what you build with it!

Follow me for more experiments at the intersection of quantum computing and AI.