#latency

Measuring Availability Properly: Percentiles, Tail Latency, and the Production Traps

Observability is a key part of any infrastructure but I’ve watched teams repeat the same mistakes around measuring availability. For…

What Happens When Go’s Garbage Collector Pauses Your Latency-Sensitive Service

How default GOGC=100 creates p99 latency spikes, why most benchmarks miss it, and why GOMEMLIMIT might be the correct answer you haven’t…

16× smaller vectors in Qdrant: memory, recall, and latency results.

Vector search is memory-hungry. A million OpenAI embeddings is ~6 GB of RAM just for the raw floats and RAM is the expensive part of any…

Latency Explained: What It Is & Why It Matters

If you’ve ever blamed your internet provider for laggy games or delayed video calls, you may have been looking at the wrong metric. While…

AXI for a noob

AXI in simple words for the world

Impact of Latency During Inference

Latency during inference is the time it takes a model to process data and deliver a result. It is a critical system constraint that…

The Ultimate Guide to Understanding Key Concepts in System Design and AI

System design and AI are complex fields that involve a multitude of concepts, each with its own unique characteristics and applications. In…

Distributed Process Consistency: The Real Problem Isn’t “Can It Run” — It’s “Run Once, On Schedule…

Distributed System

Why Your Rust Microservice Is Slower Than Go (And How to Fix It)

We migrated our API gateway from Go to Rust expecting a 3x latency improvement. Instead, our P99 went from 8ms to 42ms. The median looked…

Building a Lightning-Fast Search Relevance Ranker

Teaching a Tiny Model to Think Like a Giant

Does Retrieval Strategy Determine Hallucination? A Controlled Comparison of Four RAG Architectures

Abstract

How We Solved API Latency and PostgreSQL CPU Spikes Using Dedicated HikariCP Pools and Read/Write…

Introduction

Deno vs Node.js for a Hello World API

Detailed Comparison

Cold Starts Compound: Why Serverless + AI Agent Chains Create Latency Chaos

From Bottleneck to Breakthrough: How We Rebuilt Product Enrichment at Scale

What Is Product Enrichment?

From Bottlenecks to Benchmarks: How We Scale LLM Inference on Limited Hardware

When teams first put an LLM into production, the story usually sounds simple:

Beyond the Speed of Light: Why Jitter is the Real Enemy in O-RAN Fronthaul

Introduction: The Death of the “Dumb” Pipe

⏱️ Why Latency Spikes Hurt More Than Slow Averages

Most performance dashboards proudly show averages:

🧊 The Cold Pod That Triggered a Latency Spike

(Spring Boot Production Reality)

P99 Latency Tells You What Your Users Actually Feel

Why P95 looks great while users complain, and what you measure differently when tail latency matters

Why Your LLM Latency Spikes at Scale

Memory Fragmentation, Batch Variance, and Scheduler Effects

Low Latency: The Millisecond Advantage of Agora’s Conversational AI

Why conversational AI succeeds or fails comes down to latency. Learn how millisecond delays break natural flow, kill adoption, and stall AI

Go Production Bug: defer Inside Loops Causing Latency Spikes

At some point, every Go developer learns:

Implementing Cluster-Wide TLS Rotation Without Reload

We enable TLS Session Resumption to reduce latency, a standard practice for improving HTTPS performance.

Infiniband vs. Ethernet: Which Protocol Wins the Latency War for NVIDIA AI?

This deep dive compares InfiniBand’s native ultra-low latency against Ethernet RoCEv2 and reveals the optimal interconnect choice.

Voice Quality Optimization Explained: Jitter, Latency, Packet Loss and How Teler Fixes It

Because a broken voice experience breaks trust faster than any bad UI.

🧠 Complete Guide: Asterisk AI Voice Bot Integration Methods — 2025

From AGI to AudioSocket — tested integration methods for real-time AI voice performance under 50 ms.

Building a Debounced Autocomplete in Swift with Combine and MapKit

A “Latest Wins” pattern you can unit-test and ship with confidence 🚀

Inter-GPU Communication

Introduction

Go Netpoller Secrets: Keeping P99 From Exploding

When our P99 latency spiked to 2.8 seconds under load, the culprit wasn’t the database or network — it was Go’s single epoll thread…

The fast path/slow path mirage

Dividing processing into a fast path and slow path is a common design technique to optimize the common case —  we show it’s not all roses

Efektifkah Memperluas Jaringan via Kabel LAN Dibanding dengan Extender?

Banyak orang yang menghadapi masalah sinyal WiFi lemah di sudut rumah atau kantor. Solusi cepat yang sering digunakan adalah memasang WiFi…

Understanding the P95/P99 Latency Principle: Why the Slowest Requests Matter Most

When we talk about system performance, the first number people usually quote is the average response time.

Antrean Kopi dan Performa Sistem: Memahami P95 & P99 Latency dengan Analogi Sederhana

Dalam konteks performa sistem, terutama untuk latency (waktu tunda atau waktu respons), P95 dan P99 adalah metrik yang sangat penting dan…

Managing LLM Response Latency in Production Using Celery & Celery Beat

Large Language Models (LLMs) like GPT-4 — sure, they’re powerful. No question. But fast? Not always. Response lags — often 8, 15, 30+…

Important Latency Numbers for Programmers

Introduction

GeoDNS vs Anycast DNS

You can skip the DNS section if you are well aware of what DNS does.

Overclocking monitor in Linux

I made 60 Hz monitor to run at 80 Hz at max resolution

How the Communication Network affects Federated Learning Performance?

Recently, Federated Learning (FL) has attracted more attention due to user privacy and the nature of distributed training data. FL is a…

Enable Mac Bluetooth APTX Without Tools

so there’s a latency problem between your mac and your bluetooth headphone, and almost all of the methods requires “bluetooth explorer”…

Data Races and Latency Numbers

Pada story yang lalu saya sempat membahas keunggulan immutable data dalam mencegah terjadinya data races (here), tetapi apa sih data races…

Latency Numbers

Some latency numbers every developer should know about!