50% Bandwidth Savings: Intel’s WebRTC H.265/HEVC implementation landed in Chrome

Authors: Jianlin Qiu, Jonathan Ding

In today’s digital landscape, the demand for high-quality video streaming in applications such as video conferencing, cloud gaming, and content services is greater than ever. However, achieving optimal performance and power efficiency has been a persistent challenge for developers and service providers. The introduction of WebRTC H.265/HEVC addresses this critical issue, offering a solution that enhances the codec portfolio for Independent Software Vendors (ISVs) and service providers.

This eagerly awaited feature promises significant improvements in power and performance, enabling smoother and more efficient streaming experiences. As a vital component of modern video conferencing platforms like Microsoft Teams®, Google Meet®, and Cisco Webex®, WebRTC plays a key role in negotiating media capabilities between user endpoints, ensuring high-quality streaming of video, audio, and text/binary data. With the addition of H.265/HEVC, the ecosystem can now leverage advanced capabilities to meet the growing demands of users and deliver superior streaming quality.

As shown in the above graph, according to the Comprehensive Video Codec Comparison published by Cambridge University Press, H.265/HEVC offers compression efficiency (consumed bits for the same quality measured in peak signal-to-noise ratio, AV1 as the baseline) comparable to VP9 and benefits from widespread hardware acceleration for encoding and decoding on both older and newer platforms. Additionally, most IP cameras support H.265/HEVC, making it an attractive transcoding-free option for reducing video streaming bandwidth.

Intel’s implementation of H.265/HEVC in Chromium for WebRTC is a collaborative effort with Google, Microsoft, and other industry partners. This feature has been enabled by default since Chrome browser version 136 for Windows, macOS, and Android.

Architecturally, the H.265/HEVC WebRTC pipeline in Chromium mirrors other codecs, with most video processing components having H.265/HEVC-specific implementations:

Compared to other WebRTC codecs in Chromium, H.265/HEVC stands out in terms of software/hardware compatibility, quality of service, and coding efficiency.

Improved Software/Hardware Compatibility

Drawing from industry experiences with other WebRTC codecs, our implementation started with standardization efforts in W3C and IETF. Contributions to the specifications include, but are not limited to:

• The signaling protocol improvement to RFC 7798 in AVTCore-HEVC-WebRTC, which regulates how profiles, tiers, levels, and packetization modes of H.265/HEVC should be negotiated across different endpoints, including browsers. These standards are now established in the IETF to prevent interoperability issues similar to those found in codecs like H.264/AVC.
• The WebRTC Web API improvement for asymmetrical encode/decode capabilities. Unlike other codecs, H.265/HEVC offers the flexibility for endpoints to support only encoding or decoding during video streaming. Enhancements to the WebRTC specification ensure that WebRTC sessions are established with well-defined behavior, even when the encoding/decoding capabilities of H.265/HEVC vary end-to-end.

The comparison of current hardware support of H.265/HEVC with AV1, according to Google’s statistics, with Chrome browser on Windows and macOS:

By leveraging H.265/HEVC, ISVs and developers benefit from the enhanced software and hardware compatibility integrated into WebRTC.

Improved Quality of Service (QoS)

Compared to H.264/AVC, the H.265/HEVC WebRTC implementation in the Chromium runtime offers improved quality of service (QoS) under various network conditions. These improvements include the retransmission bandwidth reduction with temporal scalability on packet-loss networks, and elimination of broken video on key-frames with newly designed packetization/depacketization modules for H.265/HEVC.

Temporal Scalability

Our initial QoS enhancement over H.264/AVC in Chromium is the support for temporal scalability with H.265/HEVC. The concept of temporal scalability is illustrated in the graph below:

When encoding and transmitting H.265/HEVC video over WebRTC in Chromium, video frames are organized into up to three temporal layers. Higher temporal layer frames can reference lower temporal layer frames, but not vice versa. This design allows WebRTC H.265/HEVC endpoints to selectively decode lower temporal layer frames in the event of network frame loss, eliminating the need to retransmit lost higher temporal layer frames.

Emulation results indicate that on a 100Mbps network with a round-trip time (RTT) of 100ms and 5% random packet loss, at a resolution of 1280x720 (720p), H.265/HEVC temporal scalability provides an 11% reduction in retransmission bandwidth and an 18% reduction in jitter of received video frames.

Packetization and Depacketization

Packetization involves splitting compressed video frames into RTP fragments to fit the maximum transmission unit (MTU) size during video transmission. Depacketization is the reverse process, assembling compressed video frames from RTP fragments.

Our implementation of H.265/HEVC packetization/depacketization is fully compatible with RFC 7798, ensuring better interoperability with existing media servers that offer H.265/HEVC streaming services.

One common issue with H.264/AVC-based video conferencing systems during depacketization is the inconsistent key-frame assembly behavior across different endpoints. In networks with packet loss, this often results in key-frames being assembled with missing parameter sets, leading to a poor user experience with broken video if not properly managed.

Our approach to this issue starts with consolidating the key-frame definition for H.265/HEVC for WebRTC into the RFC specification with the community, thereby avoiding semantic ambiguity during depacketization. Additionally, a new frame assembling module has been designed and implemented in Chromium to simplify the assembly logic. This eliminates the need for the WebRTC stack to fix key-frames, reducing the complexity of the software stack on receiving endpoints.

Improved Coding Efficiency over H.264/AVC

The coding efficiency of H.265/HEVC in WebRTC is comparable to VP9 on Windows, representing a significant improvement over its H.264/AVC counterpart. This enhancement is attributed to 1) inclusion of a newly introduced software bitrate controller in Chromium, and 2) Low level reference control and corresponding RTP session level metadata added for WebRTC H.265/HEVC in Chromium.

Software Bitrate Controller

WebRTC dynamically adjusts transmission bitrate based on runtime-measured network metrics such as packet loss ratio and RTT. Before H.265/HEVC integration, H.264/AVC encoding bitrate control was managed by the graphics driver using constant bit rate (CBR) mode, with Chromium runtime lacking control over compressed frame sizes. This led to inconsistent output bitrates from different hardware encoders and large frames causing network transmission spikes.

Our implementation addresses this by incorporating a software bitrate controller (SW-BRC) into Chromium, which is contributed by Microsoft for H.264, and switching the bitrate mode from constant bit rate (CBR) to constant quantization parameter (CQP) mode for the H.265/HEVC hardware encoder. In this new mode, the driver receives only the quantization parameter (QP), pre-calculated by the SW-BRC, on a per-frame basis during encoding. The benefits for WebRTC include:

• Significantly reduces key-frames on dynamic networks: Frequent changes in WebRTC’s framerate and bitrate often trigger key-frame generation, which can impair the network and degrade user experience. With SW-BRC, Chromium no longer generates key-frames for each change in framerate and bitrate. This reduces key-frame requests to the hardware encoder from one per second in extreme cases to once every 100 seconds.
• Improved accuracy of requested vs. actual encoding bitrate: The bias is reduced from nearly 5% in CBR mode to below 1% according to our test. When temporal scalability is enabled, Chromium can control bitrates on different temporal layers, providing better video quality at lower temporal layers.

Low-level Frame Reference Control and Dependency Descriptors

Another key improvement to the encoder is the addition of low-level frame reference control in Chromium. Prior to our implementation, hardware-accelerated encoding for all codecs in Chromium on Windows was managed through the Windows Media Foundation API, which did not support Chromium runtime control over which previous frames could be used as inter-frame references for delta-frames (frames coded by referencing previously encoded frames). This led to two major issues for video conferencing systems:

• Media servers cannot determine reference structures among frames when endpoints encode at multiple temporal layers. This prevents the selective forwarding of video frames at different layers when temporal scalability is enabled.
• In packet-loss networks, the inability to control reference structures on-demand prevents Chromium from utilizing codec features like long-term reference (LTR). This limitation hinders the dynamic adjustment of reference pictures for subsequent frames based on frame-loss indications from receiving endpoints. As a result, Chromium must request key-frame encoding from the sending side, which degrades the streaming user experience.

Our implementation enhances frame reference control on Windows by transitioning the encoding pipeline from Media Foundation to the D3D12 Video Encoder API. The D3D12 API provides low-level access to encoder hardware, enabling explicit programming with various coding tools, including the ability to specify per-frame reference pictures. Additionally, it allows Chromium to encode sub-regions of the captured video frame with different parameters, improving regional quality.

We also added support for the dependency descriptor RTP extension for H.265/HEVC, originally designed for AV1. This extension conveys the reference structure of frames at the session layer to media servers and receiving clients, enabling precise selective forwarding of video frames based on whether the current frame’s dependencies have already been forwarded. This enhancement further reduces retransmission and key-frame requests in packet-loss networks.

Benchmarking: Encode Quality Assessments

A quantitative quality comparison of different WebRTC codecs in Chromium, encoded at 2000kbps and 500kbps, using Peak Signal-to-Noise Ratio (PSNR) as the indicator (higher values indicate better quality). The measurements were conducted on an Intel Core Ultra 258V (Lunar Lake) system running Windows 11 24H2 with Chromium’s video encode accelerator perf test suites.

At high bitrate settings (2000kbps), the H.265/HEVC implementation in Chromium provides nearly a 4dB improvement in PSNR compared to H.264/AVC. At low bitrate settings (500kbps), the PSNR improvement increases to 5.7dB. This quality enhancement means that Chrome can achieve similar visual quality with H.265/HEVC at 1000kbps as H.264/AVC at 2000kbps, resulting in a 50% bandwidth reduction for video conferencing applications.

Summary

We have contributed the H.265/HEVC WebRTC pipeline into Chromium, with improvements over its precedent H.264/AVC:

• Improved compatibility by enhancing W3C and IETF standards
• Improved QoS through temporal scalability and packetization/depacketization optimizations, and corresponding RTP session level metadata in Chromium
• Improved efficiency through the adoption of a new software bitrate controller, and new D3D12 based video encode pipeline.

Taken together these deliver a 50% reduction in bandwidth required for equivalent video quality compared to H.264/AVC, and longer battery life at the same video quality level.

Future Work

Intel’s Web Platform team is collaborating with various ISVs to enable WebRTC H.265/HEVC in their video conferencing applications. This effort involves close cooperation with the Google WebRTC team, which has provided significant support for implementation, review, and experimentation of this feature in Chrome.

As we continue to promote the adoption of H.265/HEVC for WebRTC, focusing on improved coding efficiency and performance, collaboration with ISVs is crucial for feature enhancement and ecosystem growth. We welcome your feedback during your experiments with this feature and encourage you to share your thoughts and experiences by contacting us at open.ecosystem.communications@intel.com.

About the Authors

Jianlin Qiu: Web media specialist and principal engineer at Intel. Focuses on the performance and power efficiency of media stack of web engines on Intel client platforms. He is part of the Intel Web Platform Engineering team.

Jonathan Ding: Senior principal engineer at Intel. As chief architect of Web platforms, he oversees Intel’s engineering efforts including performance optimizations and innovations for Chrome/Chromium runtime, JavaScript V8 compilers, and Web based media and streaming, etc.

© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.