How to Use Phi-3 Mini Model for Text Generation in Google Colab: A Step-by-Step Guide

Artificial Intelligence continues to revolutionize how we interact with technology, especially in natural language processing (NLP). With the release of advanced models like Phi-3 mini, developed by Microsoft, the power of text generation and language understanding is now accessible to developers worldwide. In this guide, we’ll show you how to run the Phi-3 mini model in Google Colab and perform a simple text classification task.

What is Phi-3 Mini and Why Should You Use It?

Phi-3 mini is a powerful language model designed to generate contextually accurate text based on user input. It can perform various tasks such as text generation, translation, summarization, and more. The model is fine-tuned to respond with precise outputs, making it ideal for applications like chatbots, AI assistants, and intelligent categorization systems.

In this tutorial, we will focus on setting up Phi-3 mini in Google Colab to categorize user queries, which is a practical and efficient application of the model.

Step 1: Installing Required Libraries in Google Colab

Before you can start using Phi-3, you’ll need to install a few essential libraries. The most important one is the transformers library from Hugging Face, which provides tools to interact with the Phi-3 model. In a new Google Colab cell, enter the following command to install the required library:

!pip install accelerate

This will install accelerate, which optimizes the model’s performance, especially when using a GPU.

Step 2: Import Necessary Libraries for Phi-3 Model

After installing the required dependencies, the next step is to import the necessary libraries. These libraries will allow you to load the Phi-3 model and tokenizer, which are essential for processing text data.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

Here, we are using the AutoModelForCausalLM class to load the Phi-3 model, and AutoTokenizer will help us prepare the input data for the model.

Step 3: Load the Phi-3 Model and Tokenizer

The next step is to load the Phi-3 mini model and tokenizer. This is where we specify the model version and make sure it’s set up to run on a GPU for faster processing. Here’s how you can load the model and tokenizer:

torch.random.manual_seed(0)

model = AutoModelForCausalLM.from_pretrained(
    "microsoft/Phi-3-mini-4k-instruct",
    device_map="cuda",
    torch_dtype="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-128k-instruct")

We’re loading the Phi-3 mini-4k-instruct model, setting it to use the GPU (device_map='cuda'), and configuring it to work with optimal data types (torch_dtype='auto').

Step 4: Create the Input Messages for Text Classification

Now that the model is loaded, we need to set up the input messages that the model will process. In this case, we are building a simple AI categorization assistant that identifies the category of a user’s query. For instance, a user might ask about EMI charges for cards, and the assistant will identify the query as belonging to the “Cards” category.

messages = [
    {"role": "assistant", "content": "You are an AI category identification assistant. Your task is to identify all the categories of the user query. These are the following categories: 1.General, 2.CardsIf category not identified then return None. Convert the user query in English. return the response in json .{Identified_category:'<Identified_category>',User_query_in_english:'<User_query_in_english>'}"},
    {"role": "user", "content": "எனது கார்டுக்கான EMI கட்டணங்கள் என்ன?"}
]

Here, the assistant is trained to categorize queries and translate non-English inputs into English. The user’s query is in Tamil, asking about EMI charges for their card.

Step 5: Generate Text Using the Phi-3 Model

We can now use the pipeline feature from Hugging Face’s transformers library to generate text based on our input. The pipeline makes it easy to feed input data into the model and retrieve the output.

pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
)

generation_args = {
    "max_new_tokens": 500,
    "return_full_text": False,
    "temperature": 0.0,
    "do_sample": False,
}

output = pipe(messages, **generation_args)
print(output[0]['generated_text'])

In this code, we specify the settings for text generation, including the number of tokens, whether we want to sample different outputs, and the temperature (which controls the randomness of the model’s responses).

Step 6: Understanding the Output

The model will generate output based on the input query. In this case, the user’s query in Tamil is translated into English, categorized, and returned in a structured JSON format.

Here’s an example of the expected output:

{
    "Identified_category": "Cards",
    "User_query_in_english": "What are the EMI charges for my card?"
}

The model successfully identifies the category (Cards) and translates the user’s query to English.

Why Use Phi-3 in Google Colab?

Running Phi-3 in Google Colab offers several advantages:

1. Access to GPUs for Faster Processing: Google Colab provides free access to powerful GPUs, allowing you to run resource-intensive models like Phi-3 without worrying about hardware limitations.
2. Easy Setup: Colab’s integration with Python makes it incredibly easy to set up and experiment with advanced models like Phi-3 without any complex configurations.
3. Cost-Effective: Colab offers free access to GPUs, making it a cost-effective solution for running AI models without the need for expensive hardware.

Github: https://github.com/kukretinishtha/medium_blog/blob/main/microsoft_phi3.ipynb Notebook : https://colab.research.google.com/drive/11DJe3QaxriAGPSSNRmO38FxglYI7oDGP?usp=sharing