I Tried Direct Prompting, Zero-Shot CoT and Few-Shot CoT — Here’s What I Learned

When I first started experimenting with LLMs, I assumed prompt engineering was mostly about finding the “right words” to put into a prompt.

If the model gave me a wrong answer, I would make the prompt longer.

If that didn’t work, I’d add an example.

If that still didn’t work, I’d tell the model to think step by step.

But after experimenting with different prompting strategies, I realized something more interesting:

A more complicated prompt is not automatically a better prompt.

Sometimes a simple question is enough.

Sometimes adding a reasoning instruction helps significantly.

And sometimes adding examples increases token usage without improving the result.

In this article, I’ll walk through what I learned while experimenting with Direct Prompting, Zero-Shot Chain-of-Thought, Few-Shot Prompting, Few-Shot CoT, and Self-Consistency.

I’ll keep the examples simple because the goal isn’t to understand a complicated problem.

The goal is to understand what changes when we change the prompt.

1. First, A Simple Mental Model of LLMs

Before talking about prompting techniques, there’s one thing I had to understand.

An LLM isn’t a traditional function where:

input → fixed output

Instead, it generates tokens based on probabilities.

A simplified view looks like:

Prompt
   ↓
Probability distribution
   ↓
Next token
   ↓
Next token
   ↓
Next token
   ↓
Final response

This is why the same question can sometimes produce different responses.

The model is essentially choosing among possible continuations based on probabilities.

So when I think about prompt engineering now, I don’t think:

“How do I force the model to think?”

I think:

“How can I structure the prompt so that the probability of getting the answer I want increases?”

That’s the bigger idea behind the techniques we’re about to discuss.

2. Direct Prompting — Just Ask

Let’s start with the simplest possible approach.

Suppose I ask:

What is the capital of France?

The model can simply answer:

Paris.

That’s it.

No examples.

No special instructions.

No reasoning.

This is essentially Direct Prompting.

The structure is:

Question
   ↓
Answer

For simple questions, this is usually exactly what I want.

The advantages are obvious:

• Simple
• Fast
• Cheap
• Easy to maintain
• Low token consumption

But problems can appear when the question requires multiple steps of reasoning.

For example:

A movie starts at 7:30 PM and lasts 2 hours.
What time does it finish?

The model needs to understand:

7:30 PM
+ 2 hours
= 9:30 PM

For simple reasoning like this, direct prompting may still work perfectly.

But as the reasoning becomes more complicated, I found it useful to explicitly encourage the model to reason.

That’s where Chain-of-Thought comes in.

3. Zero-Shot Chain-of-Thought

The simplest version of Chain-of-Thought is Zero-Shot CoT.

“Zero-shot” means:

I don’t give the model any examples.

Instead, I simply give it a reasoning instruction.

For example:

A movie starts at 7:30 PM and lasts 2 hours.
What time does it finish?

Let's think step by step.

The important addition is:

Let's think step by step.

We’re encouraging the model to work through intermediate steps before arriving at the answer.

Conceptually:

Direct Prompting

Question
   ↓
Answer

Whereas:

Zero-Shot CoT

Question
   ↓
Reasoning
   ↓
Answer

That’s the core idea.

4. Why Would Reasoning Help?

Let’s take a slightly different example.

Imagine I ask:

A shop has 20 chocolates.
It sells 7 in the morning and 5 in the evening.
How many chocolates are left?

A direct response might simply be:

8

But the reasoning is:

Start with 20.

20 - 7 = 13

13 - 5 = 8

Answer: 8

The reasoning itself isn’t complicated.

But for a much more complex problem, having intermediate steps can help the model avoid jumping directly to an incorrect answer.

That’s why Chain-of-Thought became an important prompting technique for reasoning tasks.

5. Few-Shot Prompting — Show the Model Some Examples

Now let’s change the approach.

Instead of simply asking the model to perform a task, we can show it examples of how the task should be performed.

Suppose we’re building a system that classifies messages as either Urgent or Normal.

We could write:

Example 1:
Message:
"My account has been hacked."
Category:
Urgent

Example 2:
Message:
"Can you tell me your weekend opening hours?"
Category:
Normal

Now classify:
Message:
"I can't access my account."
Category:

Here, we’re giving the model examples before asking it to solve the new problem.

This is Few-Shot Prompting.

The model sees:

Example
   ↓
Example
   ↓
Example
   ↓
New Question

The examples help the model understand the pattern we want.

6. Few-Shot Prompting Is Basically “Show, Don’t Just Tell”

This is one of the easiest ways to understand few-shot prompting.

Imagine you’re teaching someone how to format dates.

You could tell them:

“Always use DD/MM/YYYY.”

Or you could show them:

10 January 2026 → 10/01/2026

25 March 2026 → 25/03/2026

Now convert:

4 August 2026 → ?

The examples demonstrate the behavior.

That’s essentially what we’re doing with an LLM.

And there’s an important detail here:

The examples need to be good.

If I provide confusing or inconsistent examples, the model may learn the wrong pattern.

So few-shot prompting isn’t simply about adding more examples.

It’s about providing useful examples that represent the task clearly.

7. Few-Shot Chain-of-Thought

Now we can combine the two ideas.

We can give the model examples and include the reasoning in those examples.

For example:

Example:
Question:
A box contains 10 balls.
You remove 3.
How many are left?

Reasoning:
Start with 10.
Remove 3.
10 - 3 = 7.
Answer:
7

Then we give it:

Question:
A box contains 15 balls.
You remove 6.
How many are left?

Reasoning:

Now the model has seen not just the answer, but an example of the reasoning structure.

That’s Few-Shot Chain-of-Thought.

The flow becomes:

Example
   ↓
Reasoning
   ↓
Answer

  ↓

New Question
   ↓
Reasoning
   ↓
Answer

Compared with Zero-Shot CoT:

Zero-Shot CoT

Question
+
"Let's think step by step."

Few-Shot CoT

Solved examples
+
Reasoning examples
+
New question

Few-Shot CoT gives us more control, but that additional control comes at a cost.

8. Diminishing Returns From More Examples

This is where I started thinking more carefully about few-shot prompting.

Adding examples can help, but the benefit doesn’t necessarily increase forever.

Imagine this:

Examples Added    Performance
1                 Low
2                 Better
3                 High
4                 High
5                 High
10                High

The first few examples may significantly help the model understand the task.

But once the model understands the pattern, additional examples may contribute very little.

This is called diminishing returns.

So I wouldn’t ask:

“How many examples can I fit into my prompt?”

I’d ask:

“What is the smallest set of examples that gives me the result I need?”

This becomes especially important in production systems, where prompts are executed thousands or millions of times.

A few unnecessary examples might not matter during experimentation.

At scale, they can become a real cost.

9. What About Modern Reasoning Models?

This is another thing worth keeping in mind.

Modern reasoning-oriented models can perform substantial reasoning internally.

That means I don’t always need to explicitly provide a long Chain-of-Thought prompt.

Sometimes this:

Solve this problem.

is already enough.

Adding:

Let's think step by step.

may not provide a meaningful improvement.

And adding several detailed examples may make the prompt much more expensive without making the answer better.

So today, I don’t treat Chain-of-Thought as something I should automatically add to every prompt.

I treat it as something I should test when the task requires reasoning.

10. What If One Reasoning Path Is Wrong?

There’s another technique I found interesting: Self-Consistency.

The basic idea is simple.

Instead of asking the model once, ask it multiple times and compare the answers.

For example, imagine we ask the same question five times:

Run 1 → A
Run 2 → B
Run 3 → A
Run 4 → A
Run 5 → B

We have:

A → 3 votes
B → 2 votes

So we choose:

A

The idea is that different runs can produce different reasoning paths.

Instead of trusting one path, we look for the answer that appears most consistently.

This is called Self-Consistency.

11. But Again, There Is a Trade-Off

If one model call costs:

$X

and I run it five times, I may now be spending roughly:

5 × $X

There can also be additional latency.

So Self-Consistency can potentially improve reliability, but it comes with a price.

The trade-off becomes:

More runs
   ↓
More reasoning paths
   ↓
Potentially better reliability
   ↓
But...
   ↓
More cost + latency

This is why I don’t think there is a single prompting technique that is “the best.”

It depends on what I’m optimizing for.

12. How I Decide Which Technique to Use

After experimenting with these approaches, this is the mental model I find most useful.

I start with the simplest option.

               Direct Prompt
                     ↓
              Is accuracy good?
                /          \
              Yes           No
              ↓              ↓
             Done      Zero-Shot CoT
                            ↓
                     Is accuracy good?
                       /          \
                     Yes           No
                     ↓              ↓
                    Done      Few-Shot CoT
                                   ↓
                          Still inconsistent?
                                   ↓
                           Self-Consistency

This prevents me from unnecessarily making every prompt complicated.

13. The Three Metrics I Would Watch

When I evaluate an LLM prompting strategy, I wouldn’t look at accuracy alone.

I’d track at least three things.

1. Accuracy

How often does the model give the correct answer?

2. Cost

How many tokens and API calls are required?

3. Latency

How long does the user have to wait?

Imagine I have two approaches:

Approach A
Accuracy: 95%
Cost: Low
Latency: Low

Approach B
Accuracy: 96%
Cost: 5× higher
Latency: 4× higher

Is B automatically better?

Not necessarily.

If that extra 1% accuracy isn’t important for my application, A may be the better engineering decision.

This is why prompting is not only an AI problem.

It’s also an engineering trade-off problem.

14. My Practical Prompting Workflow

If I were building an LLM feature today, I’d probably follow something like this.

Step 1 — Start With Direct Prompting

Don’t over-engineer the first version.

Question → Answer

Step 2 — Create a Small Evaluation Set

Take real examples from the problem you’re solving.

For example, if I’m building a customer-support classifier, collect real types of questions users might ask.

Step 3 — Measure

Track:

Accuracy
Tokens
Latency
Cost

Step 4 — Try Zero-Shot CoT

If the task requires multi-step reasoning and direct prompting isn’t performing well, try a simple reasoning instruction.

Step 5 — Try Few-Shot Prompting

If the model needs help understanding the expected behavior or output format, provide a few high-quality examples.

Step 6 — Try Few-Shot CoT

If the reasoning structure itself matters, demonstrate that structure through examples.

Step 7 — Consider Self-Consistency

If reliability is extremely important and the additional cost is acceptable, generate multiple responses and compare them.

15. The Bigger Lesson I Took Away

Before experimenting with these techniques, I thought prompt engineering was mainly about finding clever instructions.

Now I think about it differently.

Prompt engineering is closer to optimization.

I’m trying to find the smallest amount of guidance that gives me the reliability I need.

Something like:

             More Guidance
                   ↑
                   │
          Few-Shot CoT
                   │
          Zero-Shot CoT
                   │
             Direct
                   │
                   └────────────→
                     Cost

As I add more guidance, I may get better results.

But I also increase:

• Prompt size
• Token usage
• Cost
• Latency
• Complexity

So the goal isn’t:

“Use the most advanced prompting technique.”

The goal is:

“Use the simplest technique that gives you good enough results.”

16. Final Takeaways

After experimenting with these approaches, this is how I remember them.

Direct Prompting

Just ask the question.

Question → Answer

Best when the task is simple.

Zero-Shot CoT

Don’t provide examples.

Just encourage reasoning.

Question
+
"Let's think step by step."

Useful for multi-step reasoning.

Few-Shot Prompting

Give the model examples.

Example
Example
Example
↓
New Question

Useful when the model needs to understand a pattern or expected output.

Few-Shot CoT

Give examples that include reasoning.

Example
↓
Reasoning
↓
Answer

Useful when you want the model to follow a particular reasoning structure.

Self-Consistency

Run the problem multiple times and compare the answers.

Run 1 → A
Run 2 → B
Run 3 → A
Run 4 → A
Run 5 → B

Majority → A

Useful when reliability matters enough to justify the extra cost.

The One Thing I’d Remember

If I had to reduce everything in this article to one principle, it would be this:

Don’t make your prompt complicated just because you can.

Start with a simple prompt.

Measure the result.

If it isn’t good enough, add reasoning.

If that’s still not enough, add examples.

If consistency is still a problem and the cost is acceptable, consider multiple runs.

In other words:

Start Simple
     ↓
Measure
     ↓
Improve
     ↓
Measure Again

Because ultimately, a prompt isn’t good because it looks sophisticated.

It’s good because it reliably gets the job done at an acceptable cost and latency.