Orchestrating JioHotstar Traffic: The Difference Between a Loading Spinner and a Winning Six

If you’ve ever streamed a high-stakes match on JioHotstar, the experience probably felt like a simple “tap and play”. It’s a seamless transition from the app icon to the stadium; a flick of a finger and you’re right there, cheering for your players to hit another six. Beneath that simple play button lies one of the most aggressive engineering challenges in the world.

At Hotstar, we navigate a “**Tsunami**” of traffic across one of the most complex network landscapes in the world. During events like the IPL or a high-stakes ODI match, we don’t just manage millions of users - we manage millions of unique network realities. If you thought just placing a Content Delivery Network (CDN) makes the magic happen — read on.

The Illusion of the Monolithic Network

Traditionally, CDN traffic management happens at the network provider or state level. For years, this was enough. But at our scale, “good enough” is the enemy of the audacious. A live event requires upwards of 60–80 Tbps of network bandwidth for streaming. To put things into perspective, it’s like instantly grabbing the entire 4K movie collection from JioHotstar (hundreds of blockbusters). Now imagine finishing all those downloads in just one second ….and then doing it again the next second and the next. That is the relentless pace of the tidal wave we face during a major event.

However, India’s network isn’t a monolith. It is a mosaic of fiber, 4G, 5G and fluctuating bandwidth that changes from one street to the next. A user on a 5G connection in South Delhi faces a vastly different network topology than a user on a local ISP in rural Rajasthan.

When you are only as good as the video you deliver, you realize that macro-level routing hits a ceiling. To provide the best Quality of Service (QoS), we needed to treat every state, every city, and eventually every cohort, as a unique routing decision.

The Solution: The QoS Routing Manager

To solve the “Last Mile” problem, we built a real-time observability and orchestration engine: the QoS Routing Manager.

The mission of this service is simple but massive: Observe crucial video metrics at a granular level and adjust traffic weights dynamically to ensure every user is mapped to the best possible CDN for their specific location.

The Granular Cohort

Instead of routing by “Maharashtra” or “Jio,” we segment users into Cohorts. A cohort is a group of people who share a common characteristic over a given period. Currently for us cohort is a specific combination of geographical, network and business categorisation i.e ASN-Country-State-City-UserType.

This allows us to detect if a specific provider is facing issues in a specific city, even if their national health looks perfect. By slicing the data this way, we can bypass localized congestion before it affects the broader user base

The Power of The Scoring Logic

The QoS Routing Manager pulls real-time performance data from our sophisticated in-house OLAP analytical processing beast - called ARGUS, which serves as our eyes and ears when it comes to video performance across the platform. It collects heartbeat data from the devices, processes them and provides the telemetry to our service.

We map every metric - Playback Failure Rate (PFR), Rebuffering, and **RTT Latency - into normalized scores using a [Linear Piecewise Scoring** function](https://en.wikipedia.org/wiki/Piecewise_linear_function). This allows us to define “Severity Buckets” (Ideal, Baseline, Sev3, Sev2, Sev1) based on direct business impact.

To determine which CDN “wins” for a specific cohort, we calculate a Cumulative Health Score based on a specific precedence order:

Cumulative Score= ( X PFR{score} ) + ( Y Rebuffer{score} ) + ( Z * RTT{score} )

We weigh PFR most heavily to ensure that “reachability” is the absolute priority, followed closely by the “fluidity” of the stream (Rebuffering) and the “snappiness” of the connection (RTT).

Filtering the Noise: The Power of EWMA

Raw network telemetry is inherently noisy. A momentary 4G tower hand-off in Jammu or transient packet loss in Bangalore can look like a critical failure in a 10-second window.

If our routing engine reacted to every micro-spike, we would introduce dangerous volatility, creating a “jittery” experience where users are constantly bounced between CDNs. We needed a way to smooth out the true performance trend from the momentary noise.

CDN Scores for different cohorts — observe the intermittent drops and spikes

To achieve this, we don’t pass raw metrics directly into our scoring engine. Instead, we pass all incoming telemetry through an **Exponentially Weighted Moving Average (EWMA)** filter.

The formula we use is:

EWMA{t} = α x_t + ( 1 — α ) EWMA{t-1}

where EWMA{t} is the new smoothed value, x_t is the raw input score, and EWMA{t-1} is the previous smoothed history.

We tune our smoothing factor α to approximately 0.6. In practical terms, this means our engine ensures recent metrics have a stronger influence on the final score while keeping the last 5 scores to have significance

Only once the metrics are smoothed by EWMA do we move to the routing.

EWMA CDN Scores for same cohorts on network — smoother spikes

Two-Phase Capacity Steering

A high score isn’t the only requirement for routing. We must balance “Customer Joy” with “Infrastructure Dynamics”. For a media streaming entity, the biggest tradeoff is quality and available network bandwidth.

It’s like a bridge where every viewer wants to drive a wide luxury bus (High Quality), but the physical lanes (Bandwidth) are finite; during a peak surge, there simply isn’t enough pavement to let everyone drive a bus at once without the bridge failing, so you have to balance the vehicle size just to keep everyone moving.

Our engine keeps track of the used bandwidth on the CDNs and employs two distinct strategies:

• Pre-Threshold Guardrail (< X% Utilization): We preemptively throttle CDNs that are on track to hit their capacity limits too early, even if they are performing well.
• Uniform Exhaustion (>= X% Utilization): During a surge, we shift logic to ensure all CDNs exhaust their capacity at the same time, squeezing every possible megabit out of our infrastructure.

Safety Gates: Resilience Over Risk

In a system of this scale, “no update” is better than a “bad update.” We built a defense-in-depth approach covering both data integrity and routing logic.

1. ASN Constraints (Hard Binding): Physics and contracts still matter. Some CDNs only have presence on specific networks. Before any scoring happens, the system applies hard constraints to ensure we never route a cohort to a CDN that physically cannot serve it.
2. Volatility Dampening (Max Deviation): To prevent wild swings in traffic that could de-stabilize the network, we cap the maximum percentage change allowed in a single iteration (e.g., a CDN cannot gain or lose more than y% share in one minute).
3. The “Warm-Up” Floor (Minimum Weight): We never let a functional CDN drop to 0% traffic. We enforce a configurable floor (typically 5%). This keeps CDN caches warm and DNS paths active, ensuring that if we need to fail-back to them instantly during a crisis, they are ready to take the load immediately.

The Audacious Impact

The results of moving to granular, cohort-based management has been significant. During a recent T20 Match, we ran a A/B rollout of the QoS Routing Manager with only ASN-State cohorts. For the treatment group:

• Playback Failure Rate (PFR) improved by a staggering 11%.
• Rebuffering and RTT Latency both saw a 2% improvement.

In our world of 50M+ concurrent users, an 11% improvement in PFR represents millions of users who stayed connected to the game instead of seeing a loading spinner. This is how we ensure that whether you are in a high-rise in Chennai or a village in Sikkim, you enjoy each and every six with best in class quality.

The work continues to build on the millions of datapoints that stream in that allow us to steer all our customer sessions to a stable viewing experience!

Are you interested in solving high-concurrency challenges at the edge? Do check out open roles if you want to build for millions of customers, who will use features that you build!