How to use Fast API to deploy your NLP project

Photo by Aaron McLean on Unsplash

Introduction

Deploying a machine learning model can sometimes be tricky, often becoming one of the most time-consuming steps in development. That’s where FastAPI comes in. It’s a lightweight Python framework that makes it easy to build and deploy APIs quickly.

In this 5 minute tutorial, we’ll walk through on how to take a functional NLP pipeline and deploy it as a web API so that it is usable as a HTTP endpoint making it easy to incorporate in any website.

Why FastAPI?

There are several similar Python frameworks out there such as Flask, Django, Tornado — so why use FastAPI?

FastAPI is asynchronous, meaning it can handle multiple requests at once without blocking the server. This is ideal for NLP applications where you may be working with external APIs, databases, or large file processing — allowing your backend to remain responsive and efficient.

FastAPI also provides automatic request validation using Pydantic models, i.e., it checks if incoming data (like JSON payloads) is the right format and data types before the request is even handled. This embedded validation makes your API more robust and error-free.

The pipeline

Let’s assume you’ve got the model, it can be anything but in our instance it’ll be a sentiment analysis pipeline using a transformer model from HuggingFace.

# Imports
from helper_functions import video_functions, predict_functions, translate_functions
import pandas as pd

# ---------------------------------------------------------------------------------------------------------------------------------------
# Load the video
# ---------------------------------------------------------------------------------------------------------------------------------------

# Load the video and convert to audio
samples, sample_rate = video_functions.youtube_converter('YouTube Link goes here')

print('Video loaded')

# ---------------------------------------------------------------------------------------------------------------------------------------
# Transcribe, language between 'el' and 'en' or None for auto-detect and model selection
# ---------------------------------------------------------------------------------------------------------------------------------------

print('Beginning transcription')

# Transcribe the audio
transcription_df = video_functions.transcribe_and_save_from_array(samples, sample_rate=44100, chunk_length=30, language=None, model="large-v3-turbo")

# Drop any NaNs
transcription_df = transcription_df.dropna(subset=['Sentence'])

print('Audio Transcribed')

# ---------------------------------------------------------------------------------------------------------------------------------------
# Predict
# ---------------------------------------------------------------------------------------------------------------------------------------

print(transcription_df.head(10))

print('Beginning emotion prediction')

# Predict the dataset
emotion_df = predict_functions.predict_dataset(transcription_df, 'Sentence', 
                                            api_key="API Key would go here...",
                                            model_dir=r"emotion_classifier_model_v10.7_Billingual")

print('Emotions predicted')

Now that we have the pipeline, I want to deploy it, but how do I do that?

Let’s first install FastAPI:

pip install "fastapi[standard]"

You’ll also need uvicorn which allows server production:

pip install uvicorn

Now that we have the packages, we need to make a few modifications on our pipeline as well as what it returns so that it’s sent to our front-end website which can be a React.JS website.

Integrating FastAPI

Let’s begin by making some adjustments:

# Imports
from helper_functions import video_functions, predict_functions, translate_functions
import pandas as pd

# Define the imports for fastapi
from fastapi import FastAPI, HTTPException

# We can use this to validate the data inputted
from pydantic import BaseModel

# ---------------------------------------------------------------------------------------------------------------------------------------
# Load the video
# ---------------------------------------------------------------------------------------------------------------------------------------

# Assign the link variable 
link = youtube_link.link
print('YouTube link:', link)

# Define our app
app = FastAPI()

# App CORS configurations
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"], # Allows for requests from any origin, you'll need to restrict this to your domain eventually.
    allow_credentials=True, # Allows credentials like cookies.
    allow_methods=["GET", "POST", "OPTIONS"], # Specifies which HTTP methods are allowed.
    allow_headers=["Content-Type","Set-Cookie"], # Specifices which HTTP headers can be used.

)

# Create a youtube link class
class YoutubeLink(BaseModel):
    link: str

# Now we wrap the pipeline into a FastAPI route handler
@app.post("/process/")
async def process_video(youtube_link: YoutubeLink):
    try:
        # Load the video and convert to audio, changed to the link assigment
        samples, sample_rate = video_functions.youtube_converter(link)

        print('Video loaded')

        # ---------------------------------------------------------------------------------------------------------------------------------------
        # Transcribe, language between 'el' and 'en' or None for auto-detect and model selection
        # ---------------------------------------------------------------------------------------------------------------------------------------

        print('Beginning transcription')

        # Transcribe the audio
        transcription_df = video_functions.transcribe_and_save_from_array(samples, sample_rate=44100, chunk_length=30, language=None, model="large-v3-turbo")

        # Drop nan's
        transcription_df = transcription_df.dropna(subset=['Sentence'])

        print('Audio Transcribed')

        # ---------------------------------------------------------------------------------------------------------------------------------------
        # Predict
        # ---------------------------------------------------------------------------------------------------------------------------------------

        # Verify pipeline process
        print(transcription_df.head(10))

        print('Beginning emotion prediction')

        # Predict the dataset
        emotion_df = predict_functions.predict_dataset(transcription_df, 'Sentence', 
                                                    api_key="API Key would go here...",
                                                    model_dir=r"emotion_classifier_model_v10.7_Billingual")

        print('Emotions predicted')

        # Now we return a JSON which will be used to pass to the website
        result = emotion_df.to_dict(orient='records')
        return {"status": "success", "data": result} 

    # Return error 500
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

What just happened?

Let’s go through the changes we made, first we installed ‘fastapi’ as well as ‘uvicorn’ so that server production is allowed, we then imported our new packages and wrote our link variable where it takes in YouTube links. We use ‘pydantic’ to check the input data in a class and then we wrap the entire pipeline into route handler where the entire pipeline process occurs. We also need to define the CORS app configurations where we can change a few parameters. It is recommended that you change the origin once you deploy to your domain because in this instance, it’ll accept requests from any origin.

The rest is quite simple and you can find more about what configurations you can implement at FastAPI’s documentation website. Once the pipeline finishes, it will return a JSON file which can be used to display data in a dashboard for example.

Running the API

To run the pipeline locally, you can simply execute this command in your anaconda terminal. Make sure you are also running your correct environment.

uvicorn main:app --reload

If you need to truly deploy, there are several other services which provide cloud computing and easy hosting such as:

• Render
• DigitalOcean
• Railway
• Amazon Web Hosting

These also include free hosting tiers but may not suit your computational power needs…

Once the server is up, you can copy the API endpoint and paste it in your website project.

const youtubeLink = 'https://www.youtube.com/watch?v=dQw4w9WgXcQ&ab_channel=RickAstley'

// Here you put the FastAPI endpoint link
fetch("http://127.0.0.1:8000/process/", {
  method: "POST",
  headers: {
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    link: youtubeLink
  })
})
.then(response => {
  if (!response.ok) {
    throw new Error(`HTTP error! Status: ${response.status}`);
  }
  return response.json();
})
.then(data => {
  // Display each sentence and emotion
  data.data.forEach(entry => {
    console.log(`Sentence: ${entry.Sentence}`);
    console.log(`Emotion: ${entry.Emotion}`);
    console.log(`Confidence: ${entry.Confidence}`);
  });
})
.catch(error => {
  console.error("API call failed:", error);
});

To conclude…

The entire process is fairly simple, this tutorial is aimed at implementing a simple pipeline. This could be used to build portfolio’s or even dashboard websites to showcase data using different statistical components. What’s next is up to you and there’s lot’s of room for extending it further like integrating the frontend, adding logging and error tracking or even moving the pipeline from a local server to production using the listed services above.

References

• https://fastapi.tiangolo.com/
• https://www.uvicorn.org/
• https://docs.pydantic.dev/
• https://huggingface.co/docs/transformers/index
• https://github.com/openai/whisper
• https://render.com/
• https://realpython.com/fastapi-python-web-apis/
• https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS
• https://huggingface.co/blog/emotion