Add Free, Local AI to Your Web Apps, Directly in Browser

Introduction

It sounds almost too good to be true, but you can now run AI models to your deployed apps directly in the browser, completely free. The catch for now is hardware: you’ll need a decent GPU or enough RAM to get something close to ChatGPT performance.

However this is important to watch, as models and browser runtimes become more optimized, high-quality, local, free, and private AI will soon run smoothly on laptops and even phones.

What’s really impressive is how simple this is to set up and integrate into your projects. This is possible due to WebLLM, allowing you to retrieve and load open source LLM models directly in the browser, so no server, API key, or backend needed.

Here is an example implementation in React and TailwindCSS here on CodePen, and here is the original JavaScript demo I modified.

[embed]Start with TinyLlama before trying more intensive models

[embed]local-ai-reactwebllm.zip Downloaddrive.google.com

Considerations

Before continuing there are some considerations to be aware of:

• Hardware specs matter: RAM, VRAM, and even disk space can affect model performance
• Mobile devices may struggle due to heavy hardware requirements
• Start small, begin with TinyLlama models and gauge from there The blunt truth: you may not get any meaningful benefits if your goal is to deploy at mass scale, but as models and hardware improve, this simple interface will only become more powerful.

Features

Let’s summarize the top features:

• Load 100+ open source models directly in the browser
• No extra setup needed, works completely on the client, no API keys, no server
• Free, private, and offline capability
• Integrates easily into your React, Next, and JavaScript web apps.
• Limited vision support for now, but likely to have big advancements in future. Only Phi 3.5 vision instruct available currently.

Code

Below is a simplified walkthrough of how the demo works, so you can set it up in minutes inside any React app. See the whole code on CodePen.

Steps

Instantiate WebLLM once — create a single MLCEngine instance outside the component to persist across renders. You can import WebLLM either via CDN (quick test) or NPM (recommended for production).

Option 1 — CDN (Quick Demo)

import * as webllm from "https://esm.run/@mlc-ai/web-llm";

// Create a single MLCEngine instance globally
const engine = new webllm.MLCEngine();

Option 2 — NPM (Recommended)

npm install @mlc-ai/web-llm

Then import it normally in your React app:

import * as webllm from "@mlc-ai/web-llm";

// Create a single MLCEngine instance globally
const engine = new webllm.MLCEngine();

Model loading flow — engine.reload(selectedModel) handles initialization. Progress events come from setInitProgressCallback to update UI and initialize model.

const [selectedModel, setSelectedModel] = useState("TinyLlama-1.1B-Chat-v0.4-q4f32_1-MLC-1k");
const [isEngineInitialized, setIsEngineInitialized] = useState(false);
const [downloadStatus, setDownloadStatus] = useState({ text: "", progress: 0 });

useEffect(() => {
  engine.setInitProgressCallback((report) => {
    setDownloadStatus({
      text: report.text || "Loading...",
      progress: report.progress || 0,
    });
    if (report.progress === 1) setIsEngineInitialized(true);
  });
}, []);
const initializeModel = async () => {
  if (isEngineInitialized) return;
  await engine.reload(selectedModel, { temperature: 1.0, top_p: 1 });
};

Chat streaming loop — engine.chat.completions.create({ stream: true, messages }) yields token chunks and update the last assistant message incrementally for real-time output.

const [messages, setMessages] = useState([]);

const streamChat = async (currentMessages) => {
  setMessages((prev) => [...prev, { role: "assistant", content: "typing..." }]);
  let curMessage = "";
  const completion = await engine.chat.completions.create({
    stream: true,
    messages: currentMessages,
  });
  for await (const chunk of completion) {
    const delta = chunk.choices[0].delta.content;
    if (delta) curMessage += delta;
    setMessages((prev) => {
      const updated = [...prev];
      updated[updated.length - 1] = { role: "assistant", content: curMessage };
      return updated;
    });
  }
};

State-driven message list — store all messages (user + assistant) in state, filter out the system prompt, and auto-scroll with a ref whenever messages change.

const chatBoxRef = useRef(null);

useEffect(() => {
  if (chatBoxRef.current) {
    chatBoxRef.current.scrollTop = chatBoxRef.current.scrollHeight;
  }
}, [messages]);
return (
  <div ref={chatBoxRef} className="overflow-y-auto h-[500px] p-4">
    {messages
      .filter((msg) => msg.role !== "system")
      .map((msg, i) => (
        <div key={i} className={msg.role === "user" ? "text-right" : "text-left"}>
          <p className="inline-block p-3 rounded-2xl bg-gray-200">{msg.content}</p>
        </div>
      ))}
  </div>
);

Runtime stats + UX polish — call engine.runtimeStatsText() post-generation, display stats and progress, and disable UI while loading or generating.

const [statsText, setStatsText] = useState("");
const [isGenerating, setIsGenerating] = useState(false);

const handleChat = async (msgs) => {
  setIsGenerating(true);
  await streamChat(msgs);
  const stats = await engine.runtimeStatsText();
  setStatsText(stats);
  setIsGenerating(false);
};
return (
  <div className="text-xs text-gray-600 p-2 border-t bg-blue-50">
    {statsText || (isGenerating ? "Generating..." : "Idle")}
  </div>
);

Integration Demo: Offline Chatbot

In one of my last posts, I described a project I worked on for a fully offline chatbot as a lightweight, private alternative to hosted LLMs. You can read more about it here. It is a ChatGPT like interface that connects to open source models from Ollama, using the Ollama JS API. After learning of WebLLM, I noticed it used many of the same models and even a very similar method for chat generation, which made it fit right into the vision I had for this project.

Demo for the offline AI chatbot

Live Demo: offline-chatbot.netlify.app

Source & Docs: GitHub Repository

Conclusion

Browser-based AI may not be a game changer for now, but it is worth keeping an eye on. As open source models become smaller and more efficient, they will become more lightweight and practical to use. Not only will this simplify development logistics (eliminating the need for an AI provider, API keys, and costs), but will allow for truly free, private, and even offline AI use.

Thanks for reading! I’m a full-stack developer specializing in React, TypeScript, and Web3 technologies.

Check out more of my work at mrmendoza.dev Find my open-source projects on GitHub Connect with me on LinkedIn

Let me know if this article was helpful or anything you’d like to see next.

References

• CodePen WebLLM React Example
• CodePen WebLLM JavaScript Example
• WebLLM Docs