How to Write Prompts in LangChain

Bad prompts are the reason LLMs give wrong or unpredictable answers. Here is how to write them properly.

If you have ever asked an LLM a question and got back a confusing or wrong answer, your first thought was probably “the model is bad.” Most of the time, that is not the case. The model answered exactly what you asked, you just did not ask it clearly.

This is the most important thing to understand about prompting: the model is not reading your mind. It is reading your words. If those words are unclear, the output will be unclear too.

This article covers everything about prompting from the ground up, what a good prompt looks like, how to structure it, how to use variables, when to give the model examples, and how to keep prompts clean and maintainable as your project grows.

What Is a Prompt, Really?

A prompt is not just a question or a sentence. It is an instruction that tells the model four things:

• What it should do
• How should it think about doing it
• What content should it work on
• What the answer should look like

Think of it like giving instructions to a new employee. If you say “write a report,” they will have dozens of questions. What kind of report? For who? How long? What format? The more specific your instructions, the better the output.

A prompt works the same way.

The Four Parts of a Good Prompt

Every well-written prompt has up to four parts. You do not always need all four, but knowing each one helps you figure out why a prompt is not working.

1. Instruction: Tell the model what to do

This is the actual task. Summarize this text. Translate this sentence. Classify this review as positive or negative.

A good instruction is direct and specific. It should not leave any room for the model to guess what you want.

Bad instruction: “Do something with this text.” Good instruction: “Summarize this text in three bullet points.”

2. Context: Tell the model how to do it

Context gives the model extra information about how to approach the task.

For example:

• What role should it play: “You are a customer support agent.”
• Who the audience is: “Explain this to someone with no technical background.”
• What tone to use: “Keep the tone professional and friendly.”
• What limits to respect: “Do not mention competitor products.”

Context does not change the task. It changes how the task gets done. Without context, the model makes its own assumptions — and those assumptions may not match what you want.

3. Input: The actual content to work on

Input is the text, question, document, or data that the model needs to process. This is the part that changes every time. The instruction and context mostly stay the same, but the input is different for each request.

For example, if you are building a tool that summarizes customer emails, the instruction and context stay fixed, but the email changes with every call.

In LangChain, inputs are handled as variables — not pasted directly into the template. More on this below.

4. Output expectation: Tell the model what the answer should look like

If you need the model to return JSON, say so. If you want a one-sentence answer, say so. If you want a numbered list, say so.

If you do not specify this, the model will format the answer however it wants. That is fine for casual use, but a problem when you are parsing the output in code.

Why Prompts Fail?

When a prompt gives you a bad result, it is almost always because one of these four things went wrong:

• The instruction was vague, and the model did not know exactly what to do
• The context was missing, and the model made its own assumptions
• The input was mixed in with the instruction, and the model got confused about what was a command and what was content
• The output format was not defined, and the result came back in an unexpected shape

The model is not making a mistake in these cases. It is just responding to an unclear spec.

Keep Instruction, Context, and Input Separate

This is the most common mistake anyone makes. When you write everything as one big paragraph, the model has to figure out which part is the instruction and which part is the content. That leads to inconsistent results.

Here is an example of what not to do:

“You are an expert. The user is a beginner. Please summarize this machine learning article in simple language and keep it short: [article text].”

Everything is blended. Now here is the same prompt, separated properly:

• Instruction: Summarize the article in simple language.
• Context: You are writing for someone with no technical background. Keep it under 100 words.
• Input: [article text]

The second version is easier to read, test, and update. If you want to change the word limit, you change one line in the context. The instruction and the article stay untouched.

This separation also makes prompts reusable. The same template works for any article. You swap the input.

Prompt Variables: Stop Copy-Pasting Prompts

Once you understand that input should be separate from instruction, the next step is making that input a variable.

A prompt variable is a placeholder inside your template that gets filled with real content when the prompt runs. Instead of writing:

Summarize this article: [paste article text here]

Every time, you write:

Summarize this article: {article}

and pass the article text as a variable when you run the chain. The template never changes. Only the content changes.

This is the same idea as a function in programming. You define the logic once. You pass different values each time you call it. You would never hard-code a value inside a function, and you should not hard-code content inside a prompt either.

In LangChain, this is exactly how PromptTemplate works. You put {variable_name} in your template, and LangChain fills it in at runtime. It also validates that all required variables are present, so if you forget to pass something, you get an error right away instead of sending a broken prompt to the model.

Name your variables clearly. A variable called {text} tells you nothing. A variable called {customer_review} tells you exactly what goes there. Good names make prompts much easier to maintain when others read your code, or when you come back to it three months later.

Simple rule: if a value changes between runs, make it a variable. Instructions and context stay fixed in the template. Only the content varies.

Few-Shot Learning: Show the Model What You Want

Sometimes a clear instruction is not enough. The task might be hard to describe in words. The output format might be tricky. Or the model keeps drifting slightly on different inputs.

In these cases, instead of writing a longer instruction, you show the model a few examples of what a correct answer looks like. This is called few-shot learning.

You are not training the model. You are not changing it in any way. You are just showing two or three solved examples before asking your actual question. The model recognizes the pattern and follows it.

Zero-shot vs few-shot

Zero-shot means you give the model only an instruction — no examples. This works fine for simple, clear tasks.

Few-shot means you include a small number of examples before the real question. This works better when:

• The output needs a very specific format
• The task is ambiguous and hard to describe
• The model keeps giving inconsistent answers
• Instructions alone do not pin down the behavior

What makes a good example

The quality of your examples matters a lot more than how many you have. A few good examples will always beat many bad ones.

• Use examples that look like your real inputs. If you are building a tool that classifies support tickets, your examples should be support tickets, not general questions.
• Keep the format consistent across all examples. If your first example has the answer on one line and the second has it spread across three lines, the model will get confused.
• Show the reasoning, not just the answer. For tasks that need logical thinking, include the intermediate steps in the answer. This teaches the model how to think through the problem, not just what to output at the end.
• Start with two or three examples. Adding more is not automatically better. Too many examples increase the token cost and can actually dilute the pattern. Only add more if two or three are not working.

When not to use a few-shot

If a clear instruction already gives you stable, consistent output, do not add examples. They add cost with no benefit. Few-shot is a tool for when instructions alone are not enough — not a default setting.

FewShotPromptTemplate: The Right Way to Handle Examples in LangChain

Writing examples directly as text inside a prompt string works for quick testing. It falls apart quickly in a real project. Examples get mixed with instructions, formatting drifts, and the whole thing becomes hard to manage.

LangChain gives you FewShotPromptTemplate to handle this properly. Examples become structured components, not raw text embedded in a string.

Here is how to build it from scratch.

Step 1: Load environment and set up the model

from dotenv import load_dotenv
load_dotenv()

from langchain_google_genai import ChatGoogleGenerativeAI

llm = ChatGoogleGenerativeAI(
    model="gemini-2.5-flash",
    temperature=0,
)

Temperature is set to 0 here on purpose. When you are using few-shot examples to guide the model, you want it to follow the pattern — not explore variations. Low temperature keeps results consistent.

Before building anything else, verify the model works:

response = llm.invoke("What is the capital of Canada?")
print(response.content)

Always check this first. If the model is not responding here, the problem is environment or credentials — not your few-shot setup.

Step 2: Define your examples

Each example should show how to reason through the answer, not just the final result.

examples = [
    {
        "question": "Who lived longer, Steve Jobs or Albert Einstein?",
        "answer": """Do we need additional questions? Yes.
Steve Jobs died at age 56.
Albert Einstein died at age 76.
Final answer: Albert Einstein."""
    },
    {
        "question": "When was Google's founder born?",
        "answer": """Do we need additional questions? Yes.
Google was founded by Larry Page.
Larry Page was born on March 26, 1973.
Final answer: March 26, 1973."""
    },
]

See how the answers walk through intermediate steps before giving the final answer? This teaches the model the thinking process — not just what to write at the end. This makes a big difference for reasoning-heavy questions.

Step 3: Define how each example should be formatted

from langchain_core.prompts import PromptTemplate

example_prompt = PromptTemplate.from_template(
    "Question:\n{question}\nAnswer:\n{answer}"
)

This template controls how every example gets rendered in the final prompt. Every single example goes through this same format. Do not vary it, even small formatting differences between examples will confuse the model.

Step 4: Build the FewShotPromptTemplate

from langchain_core.prompts.few_shot import FewShotPromptTemplate

prompt = FewShotPromptTemplate(
    examples=examples,
    example_prompt=example_prompt,
    suffix="Question:\n{question}\nAnswer:",
    input_variables=["question"],
)

The suffix is where the real question goes — clearly after the examples. The input_variables list tells LangChain what needs to be provided at runtime. This is now a proper prompt system, not a string.

Step 5: Always inspect the prompt before running it

final_prompt = prompt.format(
    question="How old was Bill Gates when Microsoft was founded?"
)
print(final_prompt)

Print it and read it. This catches formatting errors, missing variables, and cases where examples accidentally bleed into each other. Takes ten seconds. Saves a lot of debugging.

Step 6: Run the chain

from langchain_core.output_parsers import StrOutputParser

chain = prompt | llm | StrOutputParser()

result = chain.invoke({
    "question": "How old was Bill Gates when Microsoft was founded?"
})

print(result)

Prompt goes to model. Model output goes to the parser. Clean, no glue code.

Example Selectors: When Your Example Library Gets Large

Once you have more than a handful of examples, putting all of them in every prompt is wasteful. More examples mean bigger prompts, higher token cost, and sometimes worse results because the model gets confused by too many patterns.

Example selectors solve this by automatically picking only the most relevant examples for each question at runtime.

The most useful one is SemanticSimilarityExampleSelector. It uses embeddings to measure how similar each example is to the current question, and picks the closest ones.

from langchain_core.example_selectors import SemanticSimilarityExampleSelector
from langchain_google_genai import GoogleGenerativeAIEmbeddings
from langchain_chroma import Chroma

embeddings = GoogleGenerativeAIEmbeddings(model="models/embedding-001")

vectorstore = Chroma(
    collection_name="fewshot_examples",
    embedding_function=embeddings,
)

example_selector = SemanticSimilarityExampleSelector.from_examples(
    examples=examples,
    embeddings=embeddings,
    vectorstore=vectorstore,
    k=1,
)

Then use the selector instead of the static list:

prompt = FewShotPromptTemplate(
    example_selector=example_selector,
    example_prompt=example_prompt,
    suffix="Question:\n{question}\nAnswer:",
    input_variables=["question"],
)

Now the prompt automatically picks the most relevant example for each question. Your example library can grow without the prompt growing with it.

Few-Shot Prompting for Chat Models

If you are building a chat application, LangChain has a separate class for this: FewShotChatMessagePromptTemplate. It structures examples as proper human-AI message pairs instead of raw text.

from langchain_core.prompts import (
    ChatPromptTemplate,
    FewShotChatMessagePromptTemplate,
)

example_prompt = ChatPromptTemplate.from_messages(
    [
        ("human", "{instruction}\n{input}"),
        ("ai", "{answer}"),
    ]
)

few_shot_prompt = FewShotChatMessagePromptTemplate(
    example_selector=example_selector,
    example_prompt=example_prompt,
)

final_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", "You are a helpful assistant."),
        few_shot_prompt,
        ("human", "{instruction}\n{input}"),
    ]
)

The system message, examples, and the actual user question are all kept separate. Run it like this

chain = final_prompt | llm

response = chain.invoke({
    "instruction": "Please write meeting minutes",
    "input": "On December 26, the product team reviewed project progress..."
})

print(response.content)

This is the standard pattern for few-shot prompting in chat-based LangChain applications.

Prompt Hygiene: Keeping Prompts Clean Over Time

Writing a prompt that works today is the easy part. Keeping it working three months later, after it has been edited ten times and is being used inside a bigger pipeline, that is where most projects struggle.

Prompt hygiene means keeping your prompts clean, clear, and easy to maintain. Here are the rules that matter most:

One task per prompt. If you ask the model to explain something, then analyze it, then summarize it — it has to decide what matters most. That decision is rarely what you intended. If a task is complex, break it into separate steps.

Say what you mean directly. Do not use hints, metaphors, or vague language, hoping the model will figure it out. The model takes your words literally. If something is important, state it plainly.

Keep instruction, context, and input separate. Already covered this — but worth repeating because mixing them is still the most common source of problems.

Remove what is not needed. A long prompt is not a better prompt. Every line that does not change the model’s behavior just adds noise. Cut it.

Do not rely on memory that is not there. If your prompt says “based on what we discussed earlier” but you have not set up memory management, the model has no idea what you are referring to. Everything the model needs must be present in the current prompt.

Always define the output format. If you need JSON, say you need JSON. If you need two sentences, say two sentences. Leaving this out means the format varies with every run, which breaks downstream code.

Use the same tone throughout. Switching between formal and casual, technical and simple, within the same prompt adds confusion. Pick one style and stick to it.

Test with edge cases. After writing a prompt, test it with short inputs, empty inputs, and unusual inputs. These are the cases that expose hidden assumptions you did not know you had.

Treat prompts like code. Version them. When you change a prompt, know what changed. Do not scatter prompt strings as inline text across your codebase. In LangChain, they belong in templates — centralized, named, and reusable.

Quick self-check before using any prompt:

• Is the task clear and specific?
• Are the instructions explicit?
• Is the input kept separate from the instruction?
• Is the output format defined?
• Can this same prompt handle different inputs safely?

If any answer is no, fix it before moving on.

Mistakes That Keep Coming Up

Even when you understand the concepts, a few mistakes happen repeatedly:

One variable doing too many jobs. If {data} sometimes holds a question, sometimes a document, and sometimes a username, you will get silent, confusing failures. Each variable should hold one type of content.

Too many few-shot examples. More is not better. Two or three good examples are enough to guide the model.

Inconsistent example formats. If example one has the answer on one line and example two has it across a paragraph, the model picks up the inconsistency. Keep every example formatted the same way.

Skipping the inspect step. Always print your rendered prompt before running it in a chain. Formatting errors are invisible until you look at the actual output.

Using few-shot when a simple instruction already works. If the instruction is giving you good results, adding examples adds cost for no reason.

Hard-coding content into templates. If a value changes between runs, it is a variable — not something to paste into the template.

The One Thing to Remember

Every prompt you write has a structure, whether you think about it or not. The question is whether that structure is intentional or accidental.

When it is intentional — instruction clearly written, context set, input passed as a variable, output format defined — the model gives you predictable results. You can test it, update it, and reuse it.

When it is accidental — everything blended, no clear roles, no output spec — the model guesses. Sometimes it guesses right. Often it does not. And when it goes wrong, you have no idea where to start fixing it.

LangChain’s prompting tools exist to push you toward the intentional approach. PromptTemplate, FewShotPromptTemplate, and example selectors are all designed around the same idea: keep structure separate from content, and treat prompts as proper engineering artifacts — not throwaway strings.