SSAI as an encoding challenge?

Learnings from 10 years SSAI on live linear signals.

No matter wether you work at a broadcaster, a streaming service, or an ad tech company, wether you’ve ever dealt with Server-Side Ad Insertion (SSAI) on live linear signals, you know that everyone makes it sound easy. Replace some content, make money, done. But after 10 years of running SSAI in production at Zattoo (being a OTT distributor of live broadcast TV content), one thing is clear: the devil is in the details, and some of those details are actually encoding challenges. Here’s the story:

Once upon a time…

Back in 2016, walking around at IBC, every second booth was talking about SSAI on live linear signals. The idea was simple: substitute the linear ad break with unique-per-user ads. Finally fully personalized commercial breaks in broadcast content. In the US, some companies were already deploying that to some degree. At IBC, you could only find good demos. But when you asked around, no one seemed to really run it in production, in front of actual users.

But it is the easiest thing ever:

Figure 1: Linear signal with SCTE-35 markers

Step 1: You get your linear signal, and it contains SCTE-35 markers pointing to the exact frames where the ad break starts and the original content begins again.

Figure 2: Personalized ad replacement per user

Step 2: When creating a stream for a specific user, you fetch some ads and replace the old ad break with new, personalized ads.

Step 3: Money for everyone!

So let’s do it!

The protocol zoo

As we are still in 2016, let’s quickly revisit how the technical OTT landscape looked like for streaming protocols:

+ - - - - - - - - - + - - - - - - - - - - - - - - - - - - - - - + - - - - +
| Protocol          | Platforms                                 | Format  |
+ - - - - - - - - - + - - - - - - - - - - - - - - - - - - - - - + - - - - +
| HLS               | Apple devices, some Smart TVs             | MPEG-TS |
| DASH              | Web platforms, gaining on Android         | fMP4    |
| HDS               | Flash Player (yes, still around in 2016)  | fMP4    |
| Smooth Streaming  | Xbox, Windows Apps, Smart TV (for DRM)    | fMP4    |
+ - - - - - - - - - + - - - - - - - - - - - - - - - - - - - - - + - - - - +

So there was a bit of a zoo of devices and protocols to support if you wanted to cover all your users. Additionally on the HLS part, there were new versions coming up regularly, that we had to support in parallel to continue serving legacy platforms in addition to introducing new features. There was a time with three HLS versions to be operated in parallel, v3 (legacy), v5 (Fairplay) and v7 (fMP4 fragments).

Let’s talk about “the other” SSAI

When we started thinking about doing ads with streaming in general, client-side ad insertion was of course an option and in place for pre-roll ads. But doing it on client-side for ad break substitution as well would mean implementation effort on all the different applications. Additionally, it would not be frame-accurate, which wasn’t acceptable in a linear TV experience that tries to convince users that OTT streaming is a proper alternative to traditional DVB broadcast TV.

On the server-side, there were two approaches flying around: Manifest Manipulation and Fragment Stitching.

Manifest manipulation sounded promising. You splice your content on a timeline, separate the ad break from the rest of the content, and then exchange the content period of ads with personalized ads. But when we looked at the player landscape in 2016, we saw a different picture. Support for it only existed on some MSE (W3C Media Source Extension) players, which left out the majority of our revenue potential (basically all native apps). Nowadays this is practically the default way of doing SSAI.

But times were different back then, so we had a closer look at a fragment stitching approach.

The implementation details

We already had on-the-fly packaging in place, which we did directly on the edge nodes. This brought us to the idea that we could also substitute the content on the edge nodes before we created our streaming fragments.

Figure 3: High-level Fragment stitching architecture

The architecture we ended up: The Transcoder produces the encoded content. On the Edge Node, the Provider component fetches from upstream Transcoder and content arrives in an intermediate MKV format. Provider also contains all the stitching logic. It knows about SCTE-35 markers, which ad to substitute at which time for each specific request. And we actually did this frame by frame, replacing one frame of content with exactly one frame of new ads. The Packager then takes this merged fragment and packages it into whatever streaming format the client needs (HLS, DASH, Smooth Streaming, etc.). The beauty of this approach is that both the Packager and the Player are completely SSAI-agnostic.

But was it really that easy? Of course not.

Learning 1: The SCTE-35 marker myth

Everyone loves SCTE-35 markers! They point exactly to the first frame of the ad break, right?

We thought so too. Until we started to run our solution with the first broadcasters. Somehow it was stitching content a bit too early, or a bit too late. The first thought was of course: where is the bug in our stitching solution? But after looking at it for a while, we accepted that it was totally free of bugs ;)

Figure 4: SCTE-35 markers that point to an early frame

So we checked the incoming signal again in order to verify the markers themselves, which is a really tedious act:

• Recording a snippet of live source content where we expect the presence of a marker
• Searching for markers and storing them somewhere
• Parsing SCTE-35 to figure out the PTS that it is signalling
• Rendering the frame at that PTS to verify if it’s the correct one (first of ad break or first of content after ad break)

We had to introduce an easier solution.

Figure 5: Visual marker

On the ingest side we extended our encoder options. When configuring to scan for SCTE markers, we added a flag that will add a visual marker directly on the frame that is signalled in SCTE-35. We used a small red square on the top left corner. The person configuring the ingest for a channel could then easily verify if we are off from the intended ad break start.

After confirming that the marker is already off in the source signal, we could either solve it by talking to the broadcaster to let them fix it. Or we were lucky and it’s a static offset for all markers. Then we could configure an offset in the marker processing to correct it.

The truth is, we have only seen one, exactly one, broadcaster so far that really delivered exact markers. So it’s a general issue that lies somewhere in the production and tooling pipelines of broadcasters, between scheduling systems and encoders, between SCTE-104 and SCTE-35. The SCTE-35 standard being “exact” is more of a theoretical promise than a practical reality.

Learning 2: The decoder artefact problem

We are at a point where we merged the fragments properly and also the markers are working now. Now we can see the ad break is starting with personalized ads and playing well. As a user we are very drawn to what we see and immediately want to buy whatever is advertised to us.

Then the ad break ends. We switch back into the original live linear content and get… decoder artefacts.

That’s a clear case for everyone, I guess?

Figure 6: GOP structure and decoder error

Let me walk you through what’s happening here. In the diagram, you can see the frame sequence: the Content GOP2 (starting with an I-frame, continuing with P and B frames), then AD GOP101 starting with an I-frame, followed by its P and B frames. AD GOP102 also starts cleanly with an I-frame. But look at what happens when we cut back to the live content, it resumes at a P-frame. There’s no previous-frame reference for the decoder to work with.

The decoder sees this P-frame and tries to reconstruct it based on a reference frame it never received. The result in practice: this shows up as visual artefacts and blocky corruption until the next keyframe arrives.

What can we do here? We don’t know when the ad break will end for a specific user on the ingest side. Every user might have a different ad break length depending on ad sales, availability and targeting of ads on the ad server. We can’t just add one keyframe at a point derived from the SCTE marker into the content during the initial encoding step.

To our advantage, we didn’t only have on-the-fly packaging in place; we also had on-the-fly transcoding. So we extended the on-the-fly transcoder API with a keyframe PTS parameter to inject a keyframe exactly where needed for each user’s stream.

You might wonder if that added load to our transcoder pool? Yes, it did.

It didn’t really overload from the keyframe injection, but we saw an impact, of course. We have multiple caching layers in this whole on-the-fly packager/transcoder architecture. So we tried to minimize the impact by adding the keyframe PTS to the various cache keys.

While investigating the keyframe issue, we found another issue that was causing additional transcoder load. As we piped all the ads through our VOD encoding pipeline, they were naturally encoded with the VOD profile, which comes with a lower frame rate, typically 24fps or 25fps, while our live profiles target 50fps.

Figure 7: Mismatch in profiles of live vs. ads caused transcoder load

The diagram shows what happens: Live content at 50fps flows through the transcoder pool cleanly. But ad content at 25fps with a VOD profile? It needs to be re-encoded from 25fps to 50fps to match the live stream, causing again some extra load on the transcoder pool. As our transcoders implicitly did the conversion on every request, it worked well from the beginning. It only showed up as an issue, when looking at the transcoder load in detail.

The fix was easy enough: we tweaked the ingest path for ad assets to use the live encoding profiles from the start, so they match the frame rate and encoding parameters of the live content. No more re-encoding at stitching time due to that.

Learning 3: The neglected friend strikes again

In video streaming, we often care a lot about video. But as I mentioned in my previous article about picking the right fragment length, let’s not forget about audio. And here we are again, audio is attracting attention :)

Chapter 1: Silence

When everything worked on the video side, we let it run for a bit and monitored how it was going. But then we got user complaints: no audio during the ad break. We tried to reproduce it, but everything was fine. As an engineer you try to test “efficiently”, quickly grabbing Chrome on a random debug player you have at hand. But everything works, sound is coming out also during the ad break.

Digging deeper, we figured out that this was happening on channels that came with an E-AC3 5.1 track, which users could choose instead of stereo. The diagram shows the problem clearly: Main content audio has six channels (Front L, Front R, Center, LFE, Surround L, Surround R), but the ad audio? Stereo only — just Left and Right. This was not reproducible on Chrome, as there is no 5.1 support available. Our tests always defaulted to the Stereo track.

Figure 8: Audio channel mismatch between content and ads

Advertisers don’t produce too much surround sound these days. So there was suddenly no audio content during the ad break — the 5.1 track simply didn’t exist in the ad. Which led our stitching implementation to, more or less silently, add silence on the 5.1 track.

Stereo-only ad breaks are a normal thing, but with an SSAI solution that only kicks in after the initial encode, you skip all existing measures to deal with it. So again, we had to touch our ingest pipeline to enforce an upmix from stereo to 5.1 for all ads in order to continue audio during the stitched ad breaks.

Chapter 2: Loudness

After no audio at all, yet another problem occurred: Too much audio!

Advertisement has the annoying habit of being too loud too often. The visualization shows it: the linear stream has a normal loudness level, but the moment an ad starts — the levels spike way above normal.

Figure 9: Loudness spikes during ad breaks

Merging it into the linear stream was creating peaks in loudness for the user. So we thought, let’s apply loudness normalization, as we do for the linear signals. How does it look now?

Figure 10: Loudness normalization with spike zone in the beginning

Almost. There is just a small problem with small files like ads, being only 20 or 30 seconds long. The loudness normalizer needs a bit of buffer for its loudnorm logic to be effective. FFmpeg officially needs 300ms, but practically still takes a few seconds to stabilize.

You can see in the diagram what happens: even with normalization applied, there’s a “spike zone” of about 2–3 seconds at the start of the ad where the loudness is still elevated. The normalizer needs time to analyze the incoming audio and settle into its target level. For a 30-second ad that often starts with sound immediately, giving the first 2–3 seconds to a loudness spike is noticeable. And if it happens with every ad during the ad break, it’s annoying!

To fix that, we touched our ad ingest pipeline yet another time. We duplicated some audio content and prepended it at the beginning of the input buffer to give the normalizer time to settle. Then we read the normalized audio content only after the duplicated part. A bit of a hack, but it works.

Finally, we can start montetizing now!

Would we build it the same way today?

It’s now 2026 — ten years since we started this journey. Would we build it the same way today?

There is still a certain beauty in doing a fragment stitching approach. It works across all our streaming protocols, doesn’t require client-side changes, and handles the edge cases we’ve discovered over the years. No ad blocker is able to differentiate ads from original content. The fragment stitching approach even has proven to be robust and scalable for us.

But manifest manipulation has matured since 2016, most players are supporting multi-period DASH, and there is now agreement in the industry as to how to create manifests for it. Scalability with manifest manipulation is definitely higher, as we move the stitching burden from the “heavy” part of stitching fragments to a “lightweight” part of stitching manifest files. Still there is some need for custom-length fragments to start and end different content periods. Additionally we could even extend our use cases of SSAI to cover classic client-side ad insertion cases like pre-rolls for live content as well as pre/mid/post rolls for non-linear/VOD content.

The encoding challenges I’ve described? They’re not specific to our approach. Independent of whether you do manifest manipulation or fragment stitching, you’ll run into the same issues:

• SCTE-35 markers are rarely frame-accurate in practice: build tooling to verify and compensate.
• Keyframe injection is needed when cutting back to live content: your architecture needs to support this. For manifest manipulation this means creating custom-length fragments to have a new fragment starting with a keyframe at that point.
• Encoding profiles must match between ads and live content to avoid potential side effects on the client side, we can’t control the decoder!
• Audio channel configurations must be consistent, plan for upmixing stereo to 5.1.
• Loudness normalization requires special handling for short-form content — the normalizer needs time to settle.

Take away?

What else to take away from this post? It turned out that SSAI has a huge impact on the encoding pipeline. But of course there is much more to it than the encoding aspects, something for a future blog post. If you’re just starting out: don’t trust those SCTE-35 markers. Build the red square.

Other than that, let’s raise a glass:

Happy 10th anniversary to Zattoo SSAI 🎉

A technology formerly known as DAI / Dynamic Ad Insertion, more recently known as DAS / Dynamic Ad Substition, industry also calls it SSAI / Server-side Ad Insertion. Pick your battles :)

Acknowledgement

This blog post was inspired by my tech talk at the Montevideo Tech Day 2026, which was part of the Montevideo Summer Week organised by Qualabs. Juan Pablo Saibene was asking me to talk about something in the area of Encoding, which sparked my idea of talking about the encoding challenges in SSAI, which are typically not emphasised widely. Thanks for the opportunity to do this talk! I don’t want to leave anyone without this nice view over Montevideo while watching a tech talk, great place :)

Montevideo Tech Days 2026