From Chatbots to Agents: Empowering LLMs with Function Calling

Modern Large Language Models (LLMs) can generate text and solve problems from natural language prompts. However, by themselves, they’re limited to “text-in/text-out.” They can’t check current stock prices, trigger business processes, or send messages unless you provide a way to connect them to external software.

Function calling (sometimes known as tool calling) provides this capability: it allows LLMs to interact with your systems, APIs, data, and actions in a safe, structured, and auditable manner.

What is Function (Tool) Calling in LLMs and How Does It Work?

Instead of outputting only text, an LLM can generate structured JSON describing:

• The tool or function to invoke (such as get_weather, check_stock_price, create_refund)
• The arguments to send (parameters and their values)

The host application receives this JSON, inspects and validates the call, performs the action (such as calling an API, querying a database, or trading a stock), and can optionally send the results back to the LLM for further processing or conversation.

Why Is This Important?

• LLMs can act as operational agents, able to retrieve stock quotes, trigger trades, run database reports, or manage workflows.
• Provides deterministic, reliable actions. No more fake API calls written in freeform text; LLMs give explicit instructions your application can execute.
• Brings auditability and security. You decide what gets run, how arguments are checked, and how results are handled.
• Enables complex workflows. Multiple calls can be chained, results can be combined, and logic can flow between tool invocations.

Step-by-Step: Tool Calling in Depth

1. Define and Register Your Tools

Expose each function or tool with:

• A name and human-friendly description
• Argument names, types, and constraints (often as JSON schemas)

Example tool definitions:

get_weather(city: string, country_code: string)
check_stock_price(ticker: string, exchange: string)
create_refund(order_id: string, reason: string)
list_orders(customer_id: string)
send_message(channel_id: string, text: string)
schedule_meeting(participants: [string], time: string)

Detailed Stock Market Tool Example:

{
  "name": "check_stock_price",
  "description": "Get the latest trading price for a given stock ticker symbol from a specified exchange.",
  "parameters": {
    "ticker": {
      "type": "string",
      "description": "The stock ticker symbol, e.g., AAPL, TSLA"
    },
    "exchange": {
      "type": "string",
      "description": "The stock exchange, e.g., NASDAQ, NYSE"
    }
  }
}

2. Describe All Tools to the LLM

When a conversation, workflow, or agent begins, send the LLM the list of available tools and their JSON schemas. This way, the LLM knows exactly what actions are available and how to call them.

3. LLM Understands Requests and Selects Tools

Given a user request, for example:

“What was Apple’s closing price today on NASDAQ?”

The LLM converts this into a tool call:

{
  "tool": "check_stock_price",
  "arguments": {
    "ticker": "AAPL",
    "exchange": "NASDAQ"
  }
}

For workflow automation:

“Send a Slack message to #alerts if TSLA drops below $700.”

The LLM chains tool calls:

1. Check TSLA stock price.
2. If result < 700, call send_message.

Each tool returns results, which the LLM can use to decide the next action.

4. The Application Executes Tools and Handles Security

• Validate the LLM’s tool calls (argument types, permissions, abuse guards).
• Run the call (fetch data, call APIs, make trades, and similar tasks).
• Log every call for compliance and debugging.
• Optionally, require human approval for sensitive operations (such as trading or refunds).

5. Return Results and Continue the Conversation

After the external action is complete:

• The application returns the results to the LLM, often in structured format such as JSON.
• The LLM uses these results to produce follow-up questions, summaries, reports, or trigger more tool calls.

Practical Benefits

• Business logic and security stay in the application code instead of being hidden in arbitrary prompts.
• Fine-grained access control: decide which tools are available in each context and to which users.
• Provides audit trails for every action the LLM proposes and every call your backend executes.
• Workflows are composable: tools can build on each other to aggregate data, summarize, run analytics, or even trade stocks.

Example: Stock Price Retrieval

Suppose the request is, “Show me the price of Tesla (TSLA) on NASDAQ.”

Step 1: Your application sends the check_stock_price tool schema to the LLM.

Step 2: LLM generates:

{
  "tool": "check_stock_price",
  "arguments": {
    "ticker": "TSLA",
    "exchange": "NASDAQ"
  }
}

Step 3: The backend validates the input, queries a stock price API (such as Yahoo Finance), and receives:

{"ticker": "TSLA", "price": 698.54, "currency": "USD", "timestamp": "2024-06-13T16:00:00Z"}

Step 4: This result is delivered to the LLM, which then provides a plain-language explanation: “Tesla (TSLA) closed at $698.54 USD today on NASDAQ.”

Chaining Example:

“If Google drops more than 5% in a day, send me an SMS alert and a summary of news.”

Tool Chain:

1. check_stock_price(ticker="GOOGL", exchange="NASDAQ")
2. get_stock_change_percentage(ticker="GOOGL", period="1d")
3. If greater than 5% drop, then:
4. get_stock_news(ticker="GOOGL")
5. send_sms(number="...", text="Google dropped by X%. News: ...")

Tool-Centric Applications: Best Practices

• Design granular, reusable tools. Each tool should do one thing well.
• Use JSON Schema for validation. Enforce argument types, required parameters, and value ranges.
• Add permission layers as needed. For sensitive tools, consider extra checks or human approval.
• Log every invocation for compliance and troubleshooting.
• Test new tools with both real and adversarial prompts to ensure the LLM cannot exploit them.

Unlocking True Agency With LLMs

Function calling is an essential pattern for advancing from simple conversation to intelligent, reliable, and actionable AI. By registering tools like weather APIs, stock market data, order systems, and messaging platforms, you provide your LLMs with the ability to:

• Observe live information such as market data, news, or customer status
• Take actions securely such as sending notifications, executing trades, or updating orders
• Compose and automate workflows, with full control and auditability

The key is to combine LLM creativity with precise, secure, application-controlled tool invocation. This approach leads to trustworthy operational AI agents, ready to solve your most complex and impactful use cases.