Launch a Free Public LLM Server with vLLM and Ngrok

Today, I’d like to share how you can create a public LLM server to which you can send requests from anywhere in the world- for free!

We need:

• A free GPU provider (Google Colab)
• A free proxy provider (Ngrok)

Here is a step by step guide:

• Create a free Ngrok account and get an Authtoken.
• Open a Google Colab notebook and select one of the available GPUs. As of now, they offer T4 GPUs with 15GB of VRAM for free users.
• Run these commands in the given order.

!pip install vllm  #when prompted, click restart session

!wget https://bin.equinox.io/c/bNyj1mQVY4c/ngrok-v3-stable-linux-amd64.tgz
!sudo tar xvzf ./ngrok-v3-stable-linux-amd64.tgz -C /usr/local/bin

!ngrok config add-authtoken YOUR_AUTHTOKEN  #replace with your Authtoken

!nohup ngrok http 8000 > ngrok.log 2>&1 &

Next, choose which LLM to run based on your requirements and the available GPU memory. For this example Qwen/Qwen3–4B-FP8

Run it with vllm serve command.

!vllm serve Qwen/Qwen3-4B-FP8

After a few minutes you should see that the server is running.

Now you need to find your public Ngrok URL. Go to here:

https://dashboard.ngrok.com/agents/

Click the three dots and “See Agent details”

Here, you can find the endpoint for your public LLM server.

Now we can send requests to this server from anywhere on the world. Here is an example code:

from openai import OpenAI
client = OpenAI(
    base_url="http://2190e99a4797.ngrok-free.app/v1", 
                #change with your Endpoint, do not for /v1 at the end!
    api_key="",
)

completion = client.chat.completions.create(
    model="Qwen/Qwen3-4B-FP8",
    messages=[
        {
            "role": "user",
            "content": "What is 2+2",
        }
    ]
)
print(completion.choices[0].message.content)
#Output: The sum of 2 and 2 is **4**.

That’s it, now you can use this LLM server for various applications!

Some notes:

• This approach suitable for quick demos but not for production. The Colab server will timeout after a couple of hours. You can subscribe to Colab Pro for longer sessions and access to better GPUs (A100 with 40GB of RAM) but even that won’t last and be reliable enough for production
• Here, I run the model using a simple vllm serve command, but there are many additional details (max model length, KV cache quantization…) that can improve performance. These aspects are a whole new topic that I may cover in a future article.

Thanks for reading!

Contact: atahanuz1@gmail.com