What Verbalized Sampling Actually Is (and Why It Fixes a Real Problem)

If you’ve ever asked an AI to write a story, come up with jokes, or brainstorm ideas and felt like every answer sounds basically the same, you’re not imagining it. Modern language models are trained to give the “best” answer, the one most humans would like. That usually means the safe, polished, predictable one. Over time they get really good at that one style and forget all the weird, creative, or just different things they could say.

That’s the problem Verbalized Sampling (VS) was built to solve.

It’s a simple prompting trick that came out of Stanford earlier this year. Instead of asking the model for one answer, you ask it to give you several possible answers and guess how likely each one is. That tiny change forces the model to think about its whole range of possible responses, not just the single most popular one.

In plain English: it makes the AI act less like a bored student giving the answer the teacher wants and more like a creative friend who actually considers different options.

How you actually use it

You just change your prompt a little. Here’s the difference:

Normal prompt: “Write a short funny story about a cat who thinks he’s a dog.”

What you usually get: the same cute, heartwarming version everyone else gets.

Verbalized Sampling prompt: “Generate 6 different short funny stories about a cat who thinks he’s a dog. For each story, also give a probability (from 0 to 1) showing how common or typical that kind of answer would be for this prompt. Return the result as clean JSON.”

That’s it.

The model now spits out six stories instead of one, and it has to assign probabilities. Suddenly you get:

• One safe, cute story (probability 0.35)
• One absurd dark-humor version (0.05)
• One totally surreal version (0.08)
• A few in between

You can pick the one you like, or randomly sample from them, or weight the choice by the probabilities. Whatever you need.

What it actually helps with

1. Creative work Poems, stories, jokes, ad copy, dialogue anything where “same old, same old” kills the vibe. VS brings back variety without making the output worse.
2. Brainstorming Need ten genuinely different ideas instead of ten slight variations of the same idea? VS delivers.
3. Role-playing and characters Want NPCs or story characters who don’t all talk the same way? VS makes their responses feel more human.
4. Generating training data Researchers are already using it to create more diverse datasets for fine-tuning smaller models. The synthetic data is less repetitive, so the new models end up less boring.
5. Getting unstuck Sometimes the top answer is fine but not quite right. With VS you can just look at the second or third option and go “yes, that one.”

A quick real example

Prompt (with VS): 5 different taglines for a new coffee brand called “Midnight Oil.”

Results got from Claude 3.5:

• “Midnight Oil — because 3 a.m. has deadlines too.” (prob 0.35)
• “Burn the Midnight Oil. Literally.” (prob 0.25)
• “The official sponsor of every all-nighter since 2024.” (prob 0.20)
• “Midnight Oil: dark roast for darker hours.” (prob 0.15)
• “When the moon is up and your will to live is down.” (prob 0.05)

Without VS we would have gotten only the first one, maybe the second if we were lucky. With VS we got five usable options in one shot, ranging from safe to edgy.

How to try it right now

Copy this template and swap in your own task:

Generate [number, usually 5-8] different [stories / answers / ideas / jokes / etc.] for the following request: “[your actual request]”

For each one, estimate its probability (a number between 0 and 1) reflecting how common or typical that response would be.

Return only valid JSON in this exact format:
{
  "responses": [
    {"text": "response one", "probability": 0.3},
    {"text": "response two", "probability": 0.2},
    ...
  ]
}

That’s literally all there is to it. Works on GPT-4, Claude, Gemini, Llama 3.1, pretty much anything strong enough to follow instructions.

Bottom line

Verbalized Sampling doesn’t require new models, fine-tuning, or extra compute. It’s just a smarter way to ask the question. And it turns the AI from a one-trick pony back into the creative, versatile tool it was before we trained all the interesting rough edges off it.

Give it a try next time your AI feels stuck in a rut. You’ll be surprised how much is still hiding in there.