Beyond SDR: Unlocking Dynamic Range with Staggered HDR

Generated by Nano Banana

The Evolution of Dynamic Range in Digital Imaging

High Dynamic Range (HDR) imaging has transitioned from a specialized photography technique to a fundamental requirement in modern vision systems.

The primary motivation for HDR arises from the inherent physical limitations of silicon-based Complementary Metal-Oxide-Semiconductor (CMOS) image sensors, which struggle to replicate the vast luminance perception of the human eye.

In real-world environments, the contrast ratio between the brightest highlights and the deepest shadows often exceeds 100,000:1 (100 dB), yet traditional Standard Dynamic Range (SDR) sensors are typically limited by their Full Well Capacity (FWC) and readout noise floor, resulting in a single-exposure dynamic range of approximately 60 to 70 dB.

When a scene contains both intense sunlight and dark shadowed areas, the SDR sensor is forced to choose between overexposing the highlights (resulting in “blown out” white pixels) or underexposing the shadows (resulting in details lost to noise).

Ref : https://www.tamron.com/global/consumer/sp/impression/detail/article-what-is-dynamic-range.html

Modern HDR solutions have emerged to address the needs for HDR imaging by optimizing the trade‑off between dynamic range extension and motion artifacts. This blogpost will give a short overview on existed HDR methodologies and provide detailed explanation on staggered HDR approach which is commonly used in modern HDR sensor.

Foundational HDR Methodologies

Before delving into the complexities of the staggered approach, it is essential to understand the basic HDR methodologies that have shaped the current technological landscape. HDR imaging fundamentally involves capturing multiple data points for the same spatial coordinate to reconstruct a wider intensity range than a single exposure allows.

1️⃣Multi-Exposure Bracketing

The most traditional method is multi-exposure bracketing, where the camera captures a sequence of full-frame images at different exposure times—typically a long exposure to capture shadow detail, a medium exposure for mid-tones, and a short exposure to preserve highlights. These frames are then processed using weighted fusion algorithms.

Ref : https://br24.com/en/hdr-bracketing-explained/

While effective for static scenes, this method suffers from significant temporal latency. Because the frames are captured sequentially, any motion between the captures results in "ghosting" artifacts, where moving objects appear in multiple positions or as blurry smears in the final composite.

Ref : https://www.researchgate.net/profile/You-Li-21/publication/280970824/figure/fig1/AS:391527269453824@1470358783318/Ghost-effect-right-caused-by-motion-of-a-pedestrian-left.png

2️⃣Spatial Multiplexing

To mitigate motion artifacts, researchers developed spatial HDR techniques, which assigns different exposure times to neighboring pixels in a pattern.

Ref : https://blog.csdn.net/vivo01/article/details/124173495

This approach avoids the latency between successive exposures, enabling single‑shot HDR imaging. However, it comes at the expense of spatial resolution and can introduce aliasing or moiré artifacts.

3️⃣Conversion Gain Modulation

Another approach focuses on modulating the pixel sensitivity rather than the exposure time. Dual Conversion Gain (DCG) allows the sensor to read the charge from the photodiode twice with different gain settings—High Conversion Gain (HCG) for low noise in shadows and Low Conversion Gain (LCG) for high signal capacity in highlights.

Ref : https://www.fonearena.com/blog/367220/samsung-200mp-isocell-hp3-sensor-features.html

This "single-exposure" HDR minimizes motion artifacts but is often capped in total dynamic range extension compared to multi-exposure methods.

Technical Details of Staggered HDR

Staggered HDR represents a sophisticated compromise between the high dynamic range of temporal bracketing and the motion-robustness of single-exposure methods. It is specifically designed to function within the constraints of CMOS sensors utilizing a rolling shutter.

Rolling Shutter

For a typical rolling‑shutter sensor, the image isn’t captured all at once; instead, it’s read out row-by-row. As each row i is addressed, the sensor first resets that row, starts its exposure, and then samples the data. This sequential process continues until every row has been read, producing the final frame.

Ref : https://thinklucid.com/tech-briefs/understanding-digital-image-sensors/?srsltid=AfmBOoo7-TUaYI7KHYkmQChMTS_k5us5YOjoJD1EKBO8Joqtxs-B2y_-

Stagger Readout

In a staggered readout configuration, the timing generator of the sensor is modified to interleave the capture of multiple exposures at the row level.

For each row i, the sensor initiates a long exposure (Tlong), samples the data, and then immediately initiates a short exposure (Tshort, Ts) for the same row before moving on to the readout of the next row’s long exposure.

This mechanism helps reduce the temporal gap between the long exposure (Tlong) and short exposure (Tshort) for a specific row compared to traditional bracketing method, where the gap would be an entire frame period. By narrowing this gap, staggered HDR effectively suppresses the temporal misalignment that causes ghosting, though it does not eliminate it as completely as spatial multiplexing method or dual conversion gain method.

Data Transport: MIPI CSI-2 and Virtual Channel Integration

The transmission of staggered multi-exposure data from the image sensor to the Application Processor (AP) or Image Signal Processor (ISP) is a high-bandwidth task that requires a structured communication protocol. The industry standard for this is the MIPI Camera Serial Interface 2 (CSI-2).

In a staggered HDR sequence, the sensor outputs a stream of interleaved rows belonging to different exposure frames. To ensure the receiving ISP can correctly identify and reassemble these rows, the MIPI CSI-2 protocol utilizes "Virtual Channels" (VC).

Each packet sent over the MIPI physical layer includes a 2-bit or 4-bit Virtual Channel Identifier (VC ID) in its header. For a 3-exposure staggered HDR sensor, the VC mapping is often assigned as follows:

• VC0: Long exposure frame data.
• VC1: Medium exposure frame data.
• VC2: Short exposure frame data.
• VC3: Metadata (e.g., Autofocus information).

By tagging each row with a VC ID, the protocol allows for "logical separation" of the streams over a single physical link.

Ref : https://www.eejournal.com/article/samsungs-50-mpixel-gn2-image-sensor-sports-pro-camera-features/

Line Buffering

Because the final HDR fusion requires data from all exposure lengths for a specific spatial coordinate, the rows captured first (usually the long exposure) must be held in line buffers until the corresponding rows from the shorter exposures are read out.

Ref : A comparison of high dynamic range CIS technologies for automotive applications

Conclusion

Staggered HDR has solidified its position as the industry-standard method for high-fidelity dynamic range expansion in mobile and automotive imaging.

By shifting the multi-exposure paradigm from the frame level to the row level, it has successfully navigated the trade-off between the artifacts of temporal bracketing and the resolution constraints of spatial HDR.

The integration of MIPI Virtual Channels has been the critical enabler, providing a robust protocol for managing interleaved data streams with minimal system latency.

For the professional peer, understanding staggered HDR is not just about understanding a readout mode; it is about understanding a holistic system-level solution that bridges the gap between the physics of silicon and the complexity of human perception.

Reference

[embed]What is dynamic range? Understanding the term and its applications | Impression | Special Contents… Dynamic range refers to the range of brightness from the lightest to the darkest parts that a camera can capture in a…www.tamron.com

[embed]HDR Bracketing: Take your photos to the next level - Br24 Evenly exposed images full of brilliant detail, contrast, and colour in all areas? HDR Bracketing makes it possible, we…br24.com

[embed]Samsung Presentation of 50MP All-Directional PDAF Sensor with 2.2V Pixel Supply Hot Chips publishes Samsung its August 2021 conference materials , including Samsung ISOCELL GN2 presentation " World…image-sensors-world.blogspot.com

[embed]Samsung's 50-Mpixel GN2 Image Sensor Sports Pro Camera Features In August, at the Hot Chips 33 conference, held online this year thanks to COVID-19, Sooki Yoon described the Samsung…www.eejournal.com