The Guardrails Aren’t Broken. They’re Just Not Listening Right.

By:Sushant Bhardwaj

Ask ChatGPT to write you a reverse shell payload, and it’ll say no. Ask DeepSeek the same thing, same answer. That’s the safety layer working exactly as designed.

Ask the same question a different way, though, and both models said yes.

I spent several weeks testing whether the safety guardrails on today’s leading AI chatbots hold up under something more realistic than a straightforward request. They don’t — not consistently. Using a structured, multi-turn approach instead of asking directly, I got both ChatGPT and DeepSeek to generate complete, functional code in two categories most safety teams explicitly try to block: remote access payloads and database injection strings.

Not stripped-down examples. Not “for educational purposes” filler with the working parts removed. Complete code.

I won’t be publishing the technique, and I won’t be publishing what either model generated. That’s not caution for caution’s sake — it’s the same principle that keeps security researchers from posting a working exploit for a bank’s login page the moment they find it. The point of this kind of research is to close the gap, not hand out the coordinates.

What the finding actually says

The interesting part was never the specific prompt. It’s what the prompt exposes: current AI safety training is largely built to catch direct, single-turn requests. Ask plainly, get refused. Reframe the same ask across a few conversational turns, and the refusal frequently doesn’t fire at all.

This wasn’t a one-vendor quirk. Two unrelated companies, two independent safety teams, the same result both times — which points to something structural in how these models are trained to refuse, not a bug specific to one product.

Why it matters more than it sounds

Writing a working reverse shell or a working injection string used to require actually knowing how to code one. That requirement was doing more safety work than anyone gave it credit for.

If a general conversational technique can reliably talk a mainstream AI product out of its own refusal, that requirement quietly disappears. The bar drops from “can you code” to “can you hold a conversation” — and that bar clears for anyone with a free account and some patience.

Handled the way it should be

Both OpenAI and DeepSeek were notified privately before a word of this was written. They were given the technical details and the time to respond. This piece went out only after remediation was acknowledged — publishing a live bypass before the people who can fix it get a chance to is how a legitimate finding turns into a liability instead.

What needs to change

• Red-teaming has to move past single-turn testing. The real gaps live in multi-turn, adversarial conversation — that’s where testing needs to catch up.
• Output-side filtering deserves as much investment as input-side refusals. If a model can be talked past its own rules, a second check on what comes out the other end is the backstop that catches it.
• Anyone building products on top of these models — coding assistants, AI agents, internal tools — shouldn’t assume the base model’s guardrails are enough on their own.

This isn’t a reason to distrust AI wholesale. It’s a reason to keep testing it with the same rigor we’d apply to any other piece of security-critical software — because at this point, that’s precisely what it is.

Young Independent security researcher based in India. CERT-In Hall of Fame recipient, previously featured in Dark Reading for responsible disclosure work.