Stop Tweaking Prompts Manually: Let Microsoft Agent Lightning Train Your AI Agents Automatically

The frustration every AI developer knows too well

You’ve built an AI agent. It works… sometimes. You spend hours tweaking prompts:

“You are a helpful assistant…” — 60% accuracy “You are an expert assistant…” — 62% accuracy “You are a precise expert…” — 58% accuracy

What if your agent could learn from its mistakes and optimize itself?

Enter Microsoft Agent Lightning — an open-source framework that makes any AI agent learn and improve automatically.

What is Agent Lightning?

Think of it as auto-tune for AI agents. It takes your existing agent and makes it better through:

• Automatic Prompt Optimization — AI generates and tests better prompts
• Reinforcement Learning — Trains the actual model to behave smarter
• Zero rewrites — Works with LangChain, AutoGen, CrewAI, or custom code

The best part? No ML PhD required.

Let’s Build Something Real

We’ll create a customer support intent classifier that improves from 60% to 90%+ accuracy automatically.

Your Starting Point

from openai import OpenAI

def intent_classifier(message):
    client = OpenAI()

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "Classify intent: greeting, question, complaint, request, feedback"},
            {"role": "user", "content": message}
        ]
    )

    return response.choices[0].message.content

# Test
intent_classifier("I can't log in!")
# Returns: "question" (should be "complaint")

Problem: Inconsistent. Around 60% accuracy. Would take days to manually fix.

Step 1: Prepare Training Data

training_data = [
    {"message": "Hello there!", "correct_intent": "greeting"},
    {"message": "I can't log into my account!", "correct_intent": "complaint"},
    {"message": "How do I reset my password?", "correct_intent": "question"},
    {"message": "Can you help me with billing?", "correct_intent": "request"},
    {"message": "Your app is amazing!", "correct_intent": "feedback"},
    {"message": "Why is this so slow???", "correct_intent": "complaint"},
    {"message": "What are your business hours?", "correct_intent": "question"},
    {"message": "I'd like to upgrade my plan", "correct_intent": "request"},
    # Add 50-100 examples for best results
]

Step 2: Convert to Agent Lightning

Add three small changes:

from agentlightning import rollout
from openai import OpenAI

@rollout  # 1. Add decorator
def trainable_classifier(task, prompt_template):  # 2. Add prompt_template param
    client = OpenAI()

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": prompt_template.text},  # Use it here
            {"role": "user", "content": task["message"]}
        ]
    )

    predicted = response.choices[0].message.content.strip().lower()
    correct = task["correct_intent"].lower()

    # 3. Return reward (1.0 = correct, 0.0 = wrong)
    return 1.0 if predicted == correct else 0.0

That’s it. Your agent can now learn.

Step 3: Train It

import agentlightning as agl
from agentlightning.algorithms import APO

# Initial prompt (not optimal)
initial_prompt = agl.PromptTemplate(
    text="Classify intent: greeting, question, complaint, request, feedback"
)

# Configure optimizer
algorithm = APO(
    model="gpt-4o-mini",
    max_proposals_per_step=5,
    beam_search_size=3
)

# Create trainer
trainer = agl.Trainer(
    algorithm=algorithm,
    agent=trainable_classifier,
    num_workers=2
)

# Run training
print("Training started...")
trainer.run(
    initial_resources={"prompt_template": initial_prompt},
    train_tasks=training_data,
    num_iterations=3
)
print("Done!")

What Happens Behind the Scenes?

Round 1: Tests your initial prompt

Accuracy: 60% 
"Okay, this needs work..."

Round 2: GPT-4 analyzes failures

"The prompt lacks clear definitions. 
Add examples for each intent type.
Specify to return only the category name."

Round 3: Generates 5 improved prompts

Prompt 1: "You are an expert intent classifier..."
Prompt 2: "As a customer support analyzer..."
[... tests all variations ...]

Round 4: Finds the winner

✨Best prompt achieved 91% accuracy!

Round 5: Keeps refining

 Final accuracy: 94%
(Up from 60% - that's +57% improvement)

All automatic. You just wait.

Use Your Improved Agent

# Get the optimized prompt
best_prompt = trainer.get_best_resource("prompt_template")

# Deploy to production
def production_classifier(message):
    client = OpenAI()
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": best_prompt.text},
            {"role": "user", "content": message}
        ]
    )
    return response.choices[0].message.content

# Test it
print(production_classifier("I can't log in!"))
# Returns: "complaint" 

Want Even Better Results? Use RL Training

For serious improvements, train the actual model (requires GPU):

from agentlightning.algorithms import VERL

algorithm = VERL(
    model_path="meta-llama/Llama-3.2-3B",
    learning_rate=1e-6
)

trainer = agl.Trainer(algorithm=algorithm, agent=trainable_classifier)
trainer.run(...)  # Train for 4 hours on GPU

# Result: 97%+ accuracy 

RL doesn’t just improve prompts — it rewires the model’s brain to understand your task better.

Real Performance Gains

From Microsoft Research papers:

TaskBeforeAfterTimeText-to-SQL52%87%4 hoursIntent Classification60%94%1 hourRAG Q&A68%89%2 hoursMulti-Agent Tasks45%82%6 hours

Installation

# Install Agent Lightning
pip install agentlightning[apo]

# Set API key
export OPENAI_API_KEY="sk-your-key-here"

# Run training
python train.py

Requirements:

• Python 3.10+
• OpenAI API key ($0.50-$5 per training)
• No GPU needed for APO
• GPU needed for RL (use cloud: Colab Pro, Lambda Labs)

Works With Your Existing Stack

# LangChain
@rollout
def my_langchain_agent(task, prompt_template):
    # Your existing LangChain code
    return reward

# AutoGen
@rollout
def my_autogen_agent(task, prompt_template):
    # Your existing AutoGen code
    return reward

# Custom Python
@rollout
def my_custom_agent(task, prompt_template):
    # Any code you want
    return reward

Just add @rollout and define rewards. That's it.

The Bottom Line

Before Agent Lightning:

• Manual prompt testing: Days
• Results: Maybe 70% accuracy if lucky
• Cost: Your time ($$$$)

After Agent Lightning:

• Automated optimization: 1–4 hours
• Results: 90–97% accuracy
• Cost: $0.50-$5 in API calls

You transform from: “Let me try this prompt… and this one… and this one…”

To: *trainer.run() → Coffee break ☕ → Agent is 40% better*

Try It Yourself

Complete working code:

# train_agent.py
from openai import OpenAI
import agentlightning as agl
from agentlightning import rollout
from agentlightning.algorithms import APO

# Your training data
data = [
    {"message": "Hello!", "correct_intent": "greeting"},
    {"message": "I have a problem", "correct_intent": "complaint"},
    # ... add more
]

# Your agent
@rollout
def agent(task, prompt_template):
    client = OpenAI()
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": prompt_template.text},
            {"role": "user", "content": task["message"]}
        ]
    )
    predicted = response.choices[0].message.content.lower()
    correct = task["correct_intent"].lower()
    return 1.0 if predicted == correct else 0.0

# Train
algorithm = APO(model="gpt-4o-mini", max_proposals_per_step=5)
trainer = agl.Trainer(algorithm=algorithm, agent=agent, num_workers=2)

trainer.run(
    initial_resources={"prompt_template": agl.PromptTemplate(text="Classify the intent")},
    train_tasks=data,
    num_iterations=3
)

print("Your agent is now smarter!")

Resources

• GitHub: microsoft/agent-lightning
• Docs: microsoft.github.io/agent-lightning
• Paper: Agent Lightning: Train ANY AI Agents with RL

Final Thoughts

Agent Lightning is what prompt engineering should have been from the start: automated, intelligent, and actually effective.Stop manually testing prompts. Let AI optimize AI.Your agents can learn. You just need to let them.