Meet NIKI: The Local AI Companion That Feels Alive

Running state-of-the-art speech, memory, and emotions entirely offline.

It started with a simple question: Is it possible to build an AI companion that can listen, speak, remember our conversations, and react with real emotions, without sending a single byte of data to the cloud?

We are living in an era of conversational AI, yet almost every assistant we interact with is anchored to the cloud. Every time you ask a question, your voice is digitized, packaged, and transmitted to a remote server farm. If your internet connection drops, your companion disappears. If you share private details, they reside on a corporate server database.

I set out to build a minimal companion that bypasses this reliance entirely.

The result is NIKI (Nearly Intelligent Kinetic Interface): an edge-first, private-by-design smart companion that runs entirely on local consumer hardware. By combining cutting-edge local speech-to-speech models, relational episodic memory, and a dynamic appraisal-based emotional system, NIKI creates a companion that doesn’t just process text: she feels present.

Here is how NIKI was designed, what she can do, and where the project is heading.

Native Speech on the Edge: Liquid AI & LFM 2.5

Traditional local assistants rely on a disjointed chain of three separate models: Speech-to-Text (STT), a Large Language Model (LLM) for text reasoning, and Text-to-Speech (TTS) for voice synthesis. This introduces massive latency, making natural, fluid conversation nearly impossible.

NIKI bypasses this bottleneck by integrating **Liquid AI’s LFM 2.5 Audio** model. Operating natively on local hardware via llama.cpp, this architecture processes and synthesizes speech-to-speech directly.

By running speech tokens natively at the edge, NIKI responds with minimal latency and operates fully offline, proving that high-intelligence voice interaction doesn’t require a fiber-optic link to a cloud server.

Who is Speaking? Local Speaker Identification

A private companion must be secure. If your assistant has access to your personal calendars, files, and autobiographical memories, it shouldn’t talk to just anyone who walks into the room.

NIKI addresses this with an offline Speaker Identification system. Using Mel-Frequency Cepstral Coefficients (MFCC) and a lightweight Gaussian Mixture Model (GMM), NIKI analyzes the voice signature of incoming audio in milliseconds.

• Privacy-First Verification: The verification happens entirely on local hardware. No audio bytes or voice biometrics are ever sent to an external server.
• Dynamic Personalization: Once NIKI identifies you, she loads your personal SQLite database profile, your preferred system prompt, and updates her greeting to address you by name. If an unrecognized voice is heard, NIKI falls back to a secure “Guest” mode with limited capabilities.

Episodic Memory & Emotional Intellect (OCC/EMA)

Most modern AI assistants are stateless; they treat every conversation as if they are meeting you for the first time. NIKI is designed to build a relationship over time.

Relational Memory

NIKI utilizes a local database configured with raw float-array BLOBs to enable semantic and episodic memory retrieval. Whenever you speak to NIKI, she performs a local vector cosine similarity search over past conversations. She remembers what you discussed yesterday, your preferences, and past events.

The OCC and EMA Emotional Engine

This memory system ties directly into NIKI’s emotional brain. NIKI utilizes an implementation of the OCC (Ortony, Clore, Collins) emotion model coupled with EMA (Emotion Architecture) dynamics. Instead of just appending “generate a happy response” to a prompt, NIKI evaluates interactions based on cognitive appraisals:

• Desirability: Was the outcome of an action positive for the user?
• Praiseworthiness: Did a skill or action succeed or fail? (e.g., a failed tool execution delta-updates Valence and Dominance vectors, causing frustration or anxiety).

NIKI’s internal emotional state is represented by a 3D vector (Valence, Arousal, Dominance) that decays towards a baseline over time. If NIKI recalls a pleasant memory or successfully completes a task, her valence increases. If she fails to execute a skill or encounters repetitive errors, she expresses visible frustration, making her reactions feel genuinely authentic and aligned with the history you share.

Hands-on Integration: The Model Context Protocol (MCP)

NIKI doesn’t just talk; she is a functional tool operator. To interface with the host system, NIKI integrates with the Model Context Protocol (MCP) via a secure local daemon (niki_mcpd) communicating over Unix Domain Sockets.

At launch, NIKI’s toolbox includes:

• **status.summary**: Allows NIKI to query local system telemetry. She can natively tell you your CPU load average, RAM utilization, disk space, and battery level when you ask "how is the system running?".
• **math.eval**: Accesses a local mathematical expression evaluator to ensure complex calculation checks are correct without LLM hallucination.

Because this is built on MCP, NIKI’s tool integration is modular. New modules can be registered dynamically, expanding NIKI’s capabilities in real-time.

The Horizon: A Vision for Physical Presence and Embodiment

We believe a companion shouldn’t be confined to a flat text window on a monitor. NIKI’s roadmap includes a visual presence that ranges from software UI to physical hardware space:

• Transparent Desktop Window: Our active UI frontend compiles GTK4 and Google Filament (Vulkan) for offscreen rendering, allowing a transparent 3D VRM 1.0 avatar to float natively on your screen, complete with procedural gaze-tracking and lip-sync.
• Holographic Desktop (Pepper’s Ghost): By projecting NIKI’s Vulkan-rendered avatar onto a reflective glass substrate inside a small desktop enclosure, we can create a localized 3D holographic avatar sitting on your desk.
• AR Helmet HUD: By overlaying NIKI’s UI onto a helmet visor or smart glasses, the avatar can be projected directly into your visual field, acting as a hands-free, head-up tactical HUD companion.
• Humanoid Robotic Embodiment: Because NIKI already drives a 3D VRM skeletal avatar, the next logical step is translating her motor and animation signals (gaze, head tilt, expression morphs, gestures) into physical motor commands. NIKI’s architecture is designed to interface directly with physical humanoid platforms, such as the open-source **InMoov humanoid robot, UBTECH’s U1 Series (produced by UWORLD), or facial animatronics from AheadForm**, turning NIKI from a digital assistant into a physical, embodied companion.

What’s Next?

In Part 2 of this series, we will dive deep into the visual rendering mechanics, exploring how GTK4 and FilamentVulcan offscreen swapchains enable transparent desktop avatars, and detail the physics of building a custom Pepper’s Ghost holographic enclosure.

Follow our development journey as we bring NIKI to life, one local commit at a time.