✦In-Depth Learning Tutorial · AI Engineering

Image generated by Gemini for illustration

Building a Travel AI Agent

with LangChain, LangGraph & Streamlit

A complete end-to-end guide — from tools and architecture to a 4-layer evaluation framework aligned with RAG & Agentic AI best practices.

Cheikh Badiane

cheikhbadiane99@gmail.com · @cheikhb

⏱ 12 min read

Imagine a travel concierge available 24/7 — one that searches flights, compares hotels, builds a day-by-day itinerary, handles bookings, and answers support questions in real time, all through natural conversation. That’s Voyager AI, and in this article I’ll show you exactly how I built it.

🧭Why a Travel AI Agent?

Planning a trip today means juggling dozens of tabs: Google Flights, Booking.com, TripAdvisor, travel blogs, visa websites, currency converters… The information exists, but it’s fragmented across the entire internet.

The goal was simple: consolidate all of that intelligence into a single conversational agent — and measure every response it produces.

The agent needed to cover the full travel planning lifecycle:

• Understand traveller preferences — budget, style, dates, group type
• Search for flights and hotels in real time
• Recommend curated activities based on travel style
• Generate a personalised day-by-day itinerary
• Handle bookings with confirmation references
• Answer support questions — weather, visa requirements, currency conversion
• Evaluate every single response with a structured metrics framework

🛠️Tech Stack

ComponentTechnologyRoleLLMOpenAI GPT-4oAgent brain — reasoning & languageAgentLangGraph create_react_agentTool orchestration (ReAct loop)MemoryLangGraph MemorySaverThread-based conversation memoryToolsLangChain BaseTool8 business-logic toolsUIStreamlitChat interface + metrics dashboardLive dataSerpAPIGoogle Flights & HotelsEvaluationGPT-4o-mini (LLM Judge)Qualitative response scoringConfigpython-dotenvSecure API key management

Why LangGraph instead of the old AgentExecutor?

LangChain 1.x underwent a major architectural overhaul: AgentExecutor was removed in favour of LangGraph, a state-graph framework that offers:

• Thread-based persistent memory across conversation turns
• Fine-grained control over execution steps
• Built-in support for supervision and checkpointing
• A foundation for complex multi-agent systems

from langgraph.prebuilt import create_react_agent
from langgraph.checkpoint.memory import MemorySaver

🏗️System Architecture

User (Streamlit browser)

│

▼

app.py — Streamlit Interface

├── Sidebar : user preferences

├── Chat : conversation history

└── Dashboard : evaluation metrics

│

▼

agent.py — TravelAgent (LangGraph)

├── GPT-4o as the reasoning core

├── create_react_agent as orchestrator

└── MemorySaver for conversation memory

│

▼

tools.py — 8 LangChain Tools

├── search_flights ← SerpAPI or simulated

├── search_hotels ← SerpAPI or simulated

├── search_activities ← curated simulated data

├── build_itinerary ← day-by-day generator

├── make_booking ← confirmation reference

├── get_weather_forecast ← simulated forecast

├── convert_currency ← static exchange rates

└── get_travel_advisory ← visa & safety info

│

▼

metrics.py — Evaluation Layer (fully decoupled)

├── RetrievalMetrics

├── GenerationMetrics

├── AgenticMetrics

└── LLMJudgeMetrics

Key design principle: the evaluation layer (metrics.py) is completely decoupled from the business logic. It never modifies the agent's behaviour — it observes and measures it after each conversation turn.

🔧Step-by-Step Build Guide

1 Define the Tools

Each agent capability is encapsulated in a LangChain tool — a class inheriting from BaseTool with a precise description (the prompt the LLM uses to decide when to call it) and a _run() method.

2 Build the Agent with LangGraph

Wire the LLM, tools, and memory together using create_react_agent. The ReAct loop lets the model reason → act → observe → reason again until it produces a complete response.

3 Build the Streamlit Interface

Create a chat UI with a preference sidebar, styled message bubbles, and quick-start prompts. Load API keys automatically from .env — no manual input in the UI.

4 Integrate the Evaluation Framework

Implement 4 metric layers as a fully decoupled observer. Call the evaluator after each agent response — never inside it.

5 Wire Everything Together

Connect agent → evaluator → UI in app.py, and expose the metrics dashboard below the chat.

Step 1 in detail — Defining a Tool

from langchain.tools import BaseTool
from pydantic import BaseModel, Field
from typing import ClassVar, Dict, Optional, Type

class FlightSearchInput(BaseModel):
    origin: str = Field(description="Origin city or airport code")
    destination: str = Field(description="Destination city or airport code")
    departure_date: str = Field(description="Departure date YYYY-MM-DD")
    passengers: int = Field(default=1)
    cabin_class: str = Field(default="economy")

class FlightSearchTool(BaseTool):
    name: str = "search_flights"
    description: str = (
        "Search for available flights between two cities. "
        "Returns a list of options with prices, airlines, and schedules."
    )
    args_schema: Type[BaseModel] = FlightSearchInput
    serpapi_key: Optional[str] = None

    def _run(self, origin, destination, departure_date, **kwargs) -> str:
        if self.serpapi_key:
            return self._live_search(...)   # Real data via SerpAPI
        return self._simulated(...)         # Realistic simulated data

The 8 Tools at a Glance

ToolDescriptionData Sourcesearch_flightsFlights between two citiesSerpAPI / simsearch_hotelsHotels in a destinationSerpAPI / simsearch_activitiesTours & experiencesSimulatedbuild_itineraryDay-by-day travel planGeneratormake_bookingBooking confirmationSimulatedget_weather_forecastWeather + packing tipsSimulatedconvert_currency15+ currency pairsStatic ratesget_travel_advisoryVisa, health, safetySimulated

Step 2 in detail — The LangGraph Agent

from langgraph.prebuilt import create_react_agent
from langgraph.checkpoint.memory import MemorySaver
from langchain_openai import ChatOpenAI

class TravelAgent:
    def __init__(self, openai_api_key, serpapi_key=None):
        self.llm = ChatOpenAI(model="gpt-4o", temperature=0.4,
                              api_key=openai_api_key)
        self.tools  = build_tools(serpapi_key=serpapi_key)
        self.memory = MemorySaver()        # thread-based memory
        self.agent  = create_react_agent(  # ReAct loop
            model=self.llm,
            tools=self.tools,
            checkpointer=self.memory,
        )

    def run(self, user_message):
        config = {"configurable": {"thread_id": "session-001"}}
        result = self.agent.invoke(
            {"messages": [SystemMessage(...), HumanMessage(user_message)]},
            config=config
        )
        return self._parse_result(result)

The ReAct pattern (Reasoning + Acting) gives the LLM a structured loop: think → pick a tool → observe the result → think again → repeat until it has enough information to produce a final, grounded response.

Step 4 in detail — The Evaluation Framework

This is what separates a prototype from a production-ready system. The framework implements 4 independent metric layers, each addressing a distinct failure mode.

Layer 1: Retrieval

raw_count · selected_count · top_1_score · avg_score · compression_ratio · empty / over / under retrieval

Layer 2: Generation

grounded · has_answer · answer_length · potential_hallucination · compression_ratio

Layer 3: Agentic

pipeline_complete · execution_steps · agents_used · steps_per_agent · latency_ms · tools_sequence

Layer 4: LLM Judge

relevance · faithfulness · completeness · clarity · weighted score (0–5) · verdict + recommendations

Hallucination detection — if the agent produces a response without calling any search tool, it’s flagged as a potential hallucination:

search_tools = {"search_flights", "search_hotels", "search_activities", ...}
grounded              = bool(set(tools_used) & search_tools)
potential_hallucination = has_answer and not grounded

LLM Judge scoring — a second LLM (GPT-4o-mini) scores each response on 4 criteria with a weighted final score:

# Weighted score — relevance and faithfulness carry more weight
score_global = (
    relevance    * 0.35 +
    faithfulness * 0.30 +
    completeness * 0.20 +
    clarity      * 0.15
)

# Verdict
if   score >= 4.5: verdict = "EXCELLENT"
elif score >= 3.5: verdict = "GOOD"
elif score >= 2.5: verdict = "ACCEPTABLE"
else:             verdict = "INSUFFICIENT"

Step 5 in detail — Wiring it all together

# app.py — called on every user message
result = st.session_state.agent.run(user_input, context)

# Evaluate immediately, after the agent responds
report = evaluator.evaluate(
    question     = user_input,
    answer       = result["output"],
    tools_used   = result["tools_used_full"],
    tool_results = result["tool_results"],
    latency_ms   = result["latency_ms"],
)

st.session_state.messages.append({
    "role": "assistant",
    "content": result["output"],
    "tools_used": result["tools_used"],
    "eval_report": report.summary(),
})

🚀Installation & Setup

# 2. Create and activate virtual environment
python -m venv venv
venv\Scripts\activate      # Windows
source venv/bin/activate   # Mac / Linux

# 3. Install dependencies
pip install -r requirements.txt

# 4. Configure API keys
cp .env.example .env
# Edit .env and add your keys:
# OPENAI_API_KEY=sk-proj-...
# SERPAPI_API_KEY=...  (optional)

# 5. Launch
streamlit run app.py

Open http://localhost:8501 — the agent initialises automatically from your .env file. No API key input required in the UI.

💡What I Learned

Lesson 01: LangChain moves fast — pin your versions

The migration from AgentExecutor to LangGraph between 0.x and 1.x broke most existing tutorials. Always run pip show langchain before copying code from the internet, and test against a specific pinned version in requirements.txt.

Lesson 02: Pydantic v2 is strict about annotations

Inside any class inheriting from BaseTool, class-level constants must be typed as ClassVar. Missing this annotation causes a PydanticUserError at import time — it took me a frustrating debugging session to find this one.

Lesson 03: Keep evaluation completely decoupled from business logic

Mixing metrics into the agent code makes everything fragile. With metrics.py as a standalone observer, I could iterate on scoring logic, add new metric layers, and run A/B comparisons without touching the agent or tools at all.

Lesson 04: The LLM Judge catches what technical metrics miss

A pipeline_complete = True score doesn't mean the response is actually good. The LLM Judge detected subtle failures: a structurally correct but factually incomplete itinerary, a response that answered a different question than the one asked, and answers that were technically grounded but confusingly written.

🔮What’s Next

• Real booking APIs — Amadeus or Skyscanner integration instead of simulated confirmations
• Persistent memory — store conversation history across sessions in a database
• True multi-agent architecture — separate research, itinerary-building, and support agents
• Production monitoring — export metrics to LangSmith or a Grafana dashboard
• Automated regression tests — a reference question dataset to catch quality regressions on each code change

📎Resources

• 🐙
• Source Code github.com/cheikhb/voyager-ai
• 📚
• LangGraph Documentation langchain-ai.github.io/langgraph
• 📚
• Streamlit Documentation docs.streamlit.io
• 🔑
• OpenAI API platform.openai.com/api-keys
• 🔍
• SerpAPI — Live flight & hotel data serpapi.com

Cheikh Badiane

AI Engineer passionate about LangChain, LangGraph, and building production-ready intelligent systems. If this article helped you, consider starring the repo ⭐

✉ Email⌥ GitHub★ Star the repo