Micro Epiphanies 💡: AI Is Quietly Leaving the Cloud

For the first time in decades, computing is starting to move back toward the edge.

This is not because cloud failed. But because modern AI workloads are becoming too frequent, too personal, and too expensive to send remotely every single time…

Non-members can read here. Also subscribe to stay updated whenever I publish…

For the past few years, most people experienced AI through APIs:

• First OpenAI’s ChatGPT
• Then Anthropic’s Claude
• Google’s Gemini and Perplexity
• Perhaps a bit of xAI’s Grok as well…

You typed a prompt, the request traveled across the internet. Some distant GPU cluster generated a response. Then the answer came back.

That model worked but it also created a hidden assumption:

“AI must live in the cloud.” That assumption is starting to break now…

Two developments are accelerating this shift:

1. Ollama on desktops
2. Google AI Edge Gallery on phones

Together, they point toward something much larger: LLMs and VLMs are not just cloud services but…

They are becoming democratized infrastructure.

First: What Is Ollama?

Ollama is essentially a local AI runtime. Think:

“Docker for LLMs.”

It allows you to download and run large language models directly on your own machine, Windows laptop, MacBook or Mac Studio cluster, Linux workstation etc. without relying on external APIs.

Instead of “Send everything to OpenAI” you can now run models locally and expose them as APIs, even connect them to analytical workflows like KNIME or automations like n8n. Even you can build agents on top of them while keeping your data private…

[embed]

Installation is absurdly simple.

• Download , drag the Ollama icon into your applications folder.
• Launch through spotlight search, will get a prompt asking to install the command-line tools. Grant it permission.
• Then run:

ollama run qwen3:8b

And suddenly, you have a local reasoning model running on your laptop. Doesn’t matter if you have internet connection or not.

No cloud dependency, no need for a GPU server, no complicated CUDA nightmare.

That simplicity is why Ollama exploded!

Why Ollama Actually Matters

Most people think Ollama is about privacy. That’s only a small part of the story unless you are in a regulated business like a bank or in a secret government organization.

The bigger story is cost structure. Because right now, many companies are doing something economically irrational: sending every tiny task to frontier APIs.

Imagine asking Claude Opus to:

• Summarize a short note
• Classify a support ticket
• Rewrite an email in a more professional tone
• Transcribe a small audio or a youtube video
• Extract invoice fields
• Perform lightweight reasoning

Millions of times per month.

That becomes expensive very quickly.

Especially once AI gets embedded into Salesforce CRMs, SAP ERPs, WhatsApp workflows, call center customer support and internal company copilots…

Local models change the economics completely. Because a surprisingly large amount of AI work does NOT require frontier reasoning.

🚀 Quick Start Commands

Run these directly in your terminal to grab the popular models:

• Qwen 2.5/3 7B/8B: ollama run qwen2.5:7b (or qwen3 if using the latest release)
• Phi-4: ollama run phi4
• DeepSeek-R1 Distilled (8B/14B): ollama run deepseek-r1:8b
• Gemma 2 (2B/9B): ollama run gemma2:2b

Qwen is the daily driver

• Pro-Tip: Use Qwen specifically for JSON outputs. Its instruction-following for strict schema formatting rivals much larger cloud models.
• Hardware Fit: Runs flawlessly on base M-series Macs with 8GB or 16GB of RAM.

Phi-4 is the logic engine

• Pro-Tip: Deploy Phi-4 for Chain-of-Thought (CoT) prompts. It excels at breaking down complex corporate compliance or data analysis before giving a final answer.
• Hardware Fit: Best on 16GB+ RAM Macs, as 14B models require around 9GB of VRAM to run smoothly.

DeepSeek Distilled is the developer’s copilot

• Pro-Tip: Pair the DeepSeek-R1 distilled models (like the 8B or 14B variants) with IDE extensions like Continue.dev or Llama Coder. It is highly optimized for code completion and syntax debugging.
• Hardware Fit: The 8B version is lightning fast on standard hardware; the 14B offers a massive jump in coding logic if you have the RAM.

Gemma Edge is the offline native

• Pro-Tip: Use the 2B parameter version for embedding generation or simple text classification tasks that need to happen instantly, completely offline, with zero battery drain.
• Hardware Fit: Runs on practically anything, including base iPads and older laptops via community ports.

💡 Pro-Tip for Agent Orchestration

If you are using these for agents (like AutoGen, CrewAI, or LangChain), use Qwen for the Router/Manager agent because of its tool-calling capabilities, and route coding tasks specifically to DeepSeek.

Then Google Did Something Bigger

Google AI Edge Gallery is one of the most underrated AI releases of 2026. Most people still think: “AI on a phone means chatbot.” That era is ending.

The Google AI Edge Gallery demonstrates something much more important.

Agentic AI running locally on mobile devices and offline!

[embed]Google AI Edge Gallery App - App Store Download Google AI Edge Gallery by Google on the App Store. See screenshots, ratings and reviews, user tips, and more…apps.apple.com

Or the android verison is here download right now! https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery

What Is Google AI Edge Gallery?

It’s an open-source mobile app from Google that allows you to:

• Run Gemma models locally
• Execute prompts, process images offline
• Perform speech transcription
• Use AI-powered mobile actions
• Experiment with agent skills

directly on your phone.

[embed]

No cloud required. No API required. Even no internet connection required…

Why This Changes Everything

Because the smartphone is no longer just:

“an interface to the cloud.”

It becomes an inference device. That changes latency (it’s closest thing to your brain), privacy (works in airplane mode) and operating costs (it’s your hardware).

And eventually disrupts the software architecture itself. That means AI on your phone is no longer just “chat.” Now your phone can:

• understand intent
• trigger workflows
• call tools
• manipulate apps
• execute actions locally

Almost like the old TV series Quantum Leap: an intelligence layer continuously jumping between systems, contexts, and actions in real time.

https://www.imdb.com/title/tt0096684/

Which Mobile Models Matter?

Gemma 4 E2B

It’s tiny, fast, efficient. Great for:

• Offline summaries, lightweight tasks
• Phone-native interactions

Gemma 4 E4B

More capable reasoning. Good for:

• multimodal tasks, vision-based (camera) or screen-aware workflows
• mobile agents

Whisper Models

Still incredibly important. Because voice becomes the dominant interface for local AI. Especially:

• Transcription -> meeting summaries
• Multilingual interaction and
• Conversational workflows

The Real Future Is Hybrid

Of course, this is not: “local AI replaces cloud AI.” That’s the wrong framing.

The future is hybrid!

Like local AI handles; realtime tasks or private data and on device action or lightweight reasoning.

Then cloud AI handles; huge context windows, deep reasoning and advanced multimodal workloads.

Applications themselves are changing. In old model, humans navigated software, learnt interfaces, clicked buttons.

In new models, humans express intent, AI orchestrates software, interfaces become secondary.

That is the real shift happening underneath all of this. And for the first time:

you can already feel it running

— directly on your own laptop, iPad, or

— inside your truly “smart” phone this time…

All the best

Altan