Using LangChain CSV Agent for Performing Analytical Tasks

Learn how to make use of the CSV agent to analyze your CSV file and generate Python code

Photo by Flipsnack on Unsplash

Large Language Models (LLMs) are designed to understand and generate human-like natural languages. They excel in tasks involving natural languages and have very impressive capabilities in language-related tasks. However, they are not very good at analytical tasks, such as analyzing and summarizing your own dataset. Generally, if you want to use LLM to analyze your data, you have to specify the schema of your dataset to the LLM and ask the LLM to write the code for you to query your data. Once the code is generated, you have to manually run the code to obtain the result.

[embed]AI Access Made Simple: LangChain’s Shortcut to OpenAI and Hugging Face Hub Models An introduction on how to use LangChain to access LLMs provided by OpenAI and Hugging Face Hublevelup.gitconnected.com

However, all these changed with the introduction of LangChain agents. LangChain agents are systems that use language models to interact with other tools and are designed to perform well-designed tasks. They can be used for tasks such as question/answer, interfacing with APIs, generating text, generating images, language translation, summarizing text, etc.

Some examples of LangChain agents are:

• CSV Agent
• Pandas DataFrame Agent
• Python Agent
• Spark DataFrame Agent
• Xorbits Agent

In this article, I will show you a particular example of a LangChain agent — the CSV Agent, and how it helps you to perform analytical tasks on your CSV file.

What is the LangChain CSV Agent?

The CSV Agent is a LangChain agent that reads data from a CSV file, and then performs different types of operations on the data. Behind the scene, the CSV Agent calls another agent — the Pandas DataFrame Agent, which in turn calls the Python Agent, which generates LLM generated Python code.

For the following sections, we will make use of the CSV Agent to analyze the Titanic dataset (Titanic_train.csv).

Source of Data: The data source for this article is from *https://www.kaggle.com/datasets/tedllh/titanic-train.*

Licensing — Database Contents License (DbCL) v1.0 https://opendatacommons.org/licenses/dbcl/1-0/

First of all, you need to install the following libraries:

$ pip install langchain
$ pip install langchain_openai
$ pip install langchain_experimental

As usual, before you can use the OpenAI models, you need to set your OpenAI API key:

import os
os.environ['OPENAI_API_KEY'] = "OPENAI_API_Key"

Let’s create a CSV Agent using the create_csv_agent() function:

from langchain_experimental.agents.agent_toolkits import create_csv_agent
from langchain_openai.chat_models import ChatOpenAI

agent = create_csv_agent(
    ChatOpenAI(temperature = 0, 
               model = "gpt-3.5-turbo-0613"),
    "Titanic_train.csv",
    verbose = True,
)

Here, we use the ChatOpenAI class and specify the gpt-3.5-turbo-0613 model. We are using the Titanic dataset (Titanic_train.csv). Behind the scene, the CSV Agent will load the CSV file into a Pandas DataFrame.

Let’s now ask the CSV Agent a question regarding the content of the CSV file using the invoke() method:

response = agent.invoke("Split the name based on Title. and return the unique list of titles in the name")

Note that the older run() method has been deprecated and will be removed in version 0.2.0 of LangChain.

When you run the code, you will see the thought process that the agent went through:

You can print out the response returned by the agent:

print(response)

You will see the following output(formatted for clarity):

{
  'input': 'Split the name based on Title. and return the unique list of titles in the name', 
  'output': "The unique list of titles in the name column is [' Mr', ' Mrs', ' Miss', ' Master', ' Don', ' Rev', ' Dr', ' Mme', ' Ms', ' Major', ' Lady', ' Sir', ' Mlle', ' Col', ' Capt', ' the Countess', ' Jonkheer']"
}

The answer you want is the value of the **output** key.

Let’s try another question:

response = agent.invoke("Plot a chart showing age distribution")

This time, the agent directly plots the chart that you want.

If your print out the response from the model, you will see this:

{
  'input': 'Plot a chart showing age distribution', 
  'output': 'The chart showing the age distribution has been plotted.'
}

Let’s try another question:

agent.invoke("Plot a pie chart showing embarkation")

You will now see the pie chart as well as the output returned by the agent:

Amazing! Observe that there is no need for you to write any code and the CSV Agent automatically loads the CSV file for you and generated the answers or charts that you want.

Agent Types

Earlier, when we created the CSV agent, we did not specify the agent_type parameter:

agent = create_csv_agent(
    ChatOpenAI(temperature = 0, model = "gpt-3.5-turbo-0613"),
    "Titanic_train.csv",
    verbose = True,
)

By default, the agent_type is set to AgentType.ZERO_SHOT_REACT_DESCRIPTION:

agent = create_csv_agent(
    ChatOpenAI(temperature = 0, model = "gpt-3.5-turbo-0613"),
    "Titanic_train.csv",
    verbose = True,
    agent_type = AgentType.ZERO_SHOT_REACT_DESCRIPTION, # <-- default
)

AgentType.ZERO_SHOT_REACT_DESCRIPTION means the agent functions on the current action only — it has no memory.

Another agent type that you can use is the OPENAI_FUNCTIONS. As explained in my previous article, OpenAI supports a feature known as function calling. OpenAI function calling allows you to instruct the model to return the response in a particular JSON format. This agent-type is very useful here if you want the OpenAI model to return the response back to you in a particular format, especially if you want the Python code to be sent back to you in the response.

In the previous section, the Python code is shown when you set the verbose parameter to True and it was not returned as part of the response from the invoke() method.

To get the model to return you Python code, set the agent_type parameter to AgentType.OPENAI_FUNCTIONS, like this:

from langchain.agents.agent_types import AgentType

agent = create_csv_agent(
    ChatOpenAI(temperature = 0, model = "gpt-3.5-turbo-0613"),
    "Titanic_train.csv",
    verbose = True,
    agent_type = AgentType.OPENAI_FUNCTIONS,
)

Let’s now try to ask a question:

agent.invoke("Split the name based on Title. and return the unique list of titles in the name")

However, you will get an error:

ValueError: An output parsing error occurred. In order to pass this error
back to the agent and have it try again, pass `handle_parsing_errors=True` 
to the AgentExecutor. This is the error: Could not parse tool input: 
{'arguments': "import pandas as pd\n\n# Split the name based on Title\ndf
['Title'] = df['Name'].str.split(',').str[1].str.split('.').str[0].str.
strip()\n\n# Get the unique list of titles\nunique_titles = df['Title'].
unique().tolist()\n\nunique_titles", 'name': 'python'} because the 
`arguments` is not valid JSON.

This is because the model is now returning you the result in JSON and the agent is trying to parse the JSON result (but were not able to do so due to incorrect format). To fix this, you need to pass in a dictionary containing the following:

    {
        "input": {
            "name": "python",
            "arguments": "Split the name based on Title. and return the unique list of titles in the name"
        }
    }

The value for the **arguments** key is the question that you want to pose to the model. You can now call the invoke() function using this dictionary:

from langchain.agents.agent_types import AgentType

agent = create_csv_agent(
    ChatOpenAI(temperature = 0, model = "gpt-3.5-turbo"),
    "Titanic_train.csv",
    verbose = True,
    agent_type = AgentType.OPENAI_FUNCTIONS,
)

try:
    #---format the user's input---
    function_input = {
        "input": {
            "name": "python",
            "arguments": "Split the name based on Title. and return the unique list of titles in the name"
        }
    }
    response = agent.invoke(function_input)
    #------------------------------
    print(response)
except Exception as e:
    print(f"Error: {e}")

When you run the above code snippet, the response returned by the model now is a JSON string:

{
  'input': 
    {
      'name': 'python', 
      'arguments': 'Split the name based on Title. and return the unique 
                    list of titles in the name'
    }, 
   'output': 'The unique list of titles in the `Name` column after splitting 
              based on title is:\n- Mr\n- Mrs\n- Miss\n- Master\n- Don\n- 
              Rev\n- Dr\n- Mme\n- Ms\n- Major\n- Lady\n- Sir\n- Mlle\n- 
              Col\n- Capt\n- the Countess\n- Jonkheer'
}

Let’s try another question:

    function_input = {
        "input": {
            "name": "python",
            "arguments": "Plot a bar chart showing embarkation points"
        }
    }

This time round, the response value is:

{
  'input': 
    {
      'name': 'python', 
      'arguments': 'Plot a bar chart showing embark'
    }, 
  'output': "To plot a bar chart showing the embarkation points, you can 
             use the `matplotlib` library in Python. Here's an example 
             of how you can do it:\n\n```python\nimport matplotlib.pyplot 
             as plt\n\n# Count the number of passengers for each 
             embarkation point\nembark_counts = df['Embarked'].
             value_counts()\n\n# Plot the bar chart\nplt.bar(
             embark_counts.index, embark_counts.values)\n\n# Add labels 
             and title\nplt.xlabel('Embarkation Point')\nplt.ylabel(
             'Number of Passengers')\nplt.title('Passenger Count by 
             Embarkation Point')\n\n# Show the plot\nplt.show()\n```\n\n
             This code assumes that you have already imported the `pandas`
             and `matplotlib.pyplot` libraries and that your dataframe is 
             named `df`."
}

Notice that there is Python code in the value for the **output** key above. We can extract the Python code and then run the Python code programmatically using the exec() function. The following code snippet looks for the Python code in the response and runs it if it is available:

from langchain.agents.agent_types import AgentType
#------------------------
import re
import pandas as pd
#------------------------

agent = create_csv_agent(
    ChatOpenAI(temperature = 0, model = "gpt-3.5-turbo"),
    "Titanic_train.csv",
    verbose = True,
    agent_type = AgentType.OPENAI_FUNCTIONS,
)

try:
    #---format the user's input---
    function_input = {
        "input": {
            "name": "python",
            "arguments": "Plot a bar chart showing embarkation points"
        }
    }
    response = agent.invoke(function_input)
    print(response)

    #---------------------------------------
    # extract the result from the "output" key
    response = response['output']

    # search for Python code in the response
    python_code_pattern = r"```python(.*?)```"
    match = re.search(python_code_pattern, response, re.DOTALL)

    if match:  # if Python code is found
        df = pd.read_csv('Titanic_train.csv')
        # extract the matched Python code and strip any leading/trailing whitespaces
        python_code = match.group(1).strip()
        print(python_code)
        exec(python_code)
    else:
        print(response)
    #------------------------

except Exception as e:
    print(f"Error: {e}")

Based on the above question (“Plot a bar chart showing embarkation points”), the response contains the following Python code (extracted using RegEx):

import matplotlib.pyplot as plt

# Count the number of passengers for each embarkation point
embarkation_counts = df['Embarked'].value_counts()

# Plot the bar chart
plt.bar(embarkation_counts.index, embarkation_counts.values)
plt.xlabel('Embarkation Point')
plt.ylabel('Number of Passengers')
plt.title('Embarkation Points')
plt.show()

When you execute the above Python code using the exec() function, you will see a chart:

Let’s try another question:

    function_input = {
        "input": {
            "name": "python",
            "arguments": "Plot a bar chart the distribution of age"
        }
    }

The Python code returned (and extracted) is:

import matplotlib.pyplot as plt

# Filter out missing values in the 'Age' column
age_data = df['Age'].dropna()

# Plot the bar chart
plt.hist(age_data, bins=20, edgecolor='black')
plt.xlabel('Age')
plt.ylabel('Count')
plt.title('Distribution of Age')
plt.show()

And here is the chart generated by the above Python code:

One final question:

    function_input = {
        "input": {
            "name": "python",
            "arguments": "Plot a pie chart showing embarkation"
        }
    }

Here is the Python code generated:

import pandas as pd
import matplotlib.pyplot as plt

# Assuming 'df' is your dataframe
df['Embarked'].value_counts().plot.pie(autopct='%1.1f%%')
plt.title('Embarkation')
plt.ylabel('')
plt.show()

And here is the chart generated by the above Python code:

From the above demonstrations, take note that the value of the *output* key does not always contain the Python code.

Summary

In this article, I have explained what is LangChain Agent, what it does, and how it works using the CSV Agent as an example. I have also explained the different between the two agent types — ZERO_SHOT_REACT_DESCRIPTION and OPENAI_FUNCTIONS and they affect the responses returned by the model. Hopefully, you now have a better idea of the roles played by a LangChain agent and can utilize them in your applications.