From RGB to YUV: What I Learned While Implementing a Traffic Sign Recognition Project

From RGB to YUV: What I Learned While Implementing a Traffic Sign Recognition Project

As part of the Deep Learning course at ITI, I worked with two teammates on a project implementing a classic paper on traffic sign recognition.

While the project itself was centered around a convolutional network architecture, one of the things that stood out to me personally was something much more basic: image representation.

Until now, most of my experience with image data has been limited to the following:

• RGB
• or grayscale

But during this project, I came across YUV color space for the first time.

That pushed me to ask a few simple questions:

• What exactly is YUV?
• How is it different from RGB?
• Why would we use it instead of grayscale or RGB?
• And how can we convert an image from RGB to YUV in practice?

This article is my attempt to answer those questions clearly and practically.

1. The familiar starting point: RGB and grayscale

In most computer vision projects, image data is usually represented in one of two common forms:

RGB RGB stands for:

• R: Red
• G: Green
• B: Blue

This is the standard way digital images are stored and displayed. Each image is represented using three color channels.

RGB preserves full color information, which is useful in many tasks. However, brightness and color are mixed together across the three channels.

Grayscale Grayscale reduces the image to a single channel that represents intensity only.

This makes the input simpler and smaller, but it removes all color information.

So if we compare them quickly:

• RGB: keeps all color information
• Grayscale: keeps only intensity
• YUV: gives us something in between, but in a more structured way

2. What is YUV?

YUV is a color representation that separates an image into:

• Y: luminance

This represents brightness or intensity.

• U: chrominance

This carries part of the color information.

• V: chrominance

This carries the remaining color information.

The key idea is simple:

YUV separates brightness from color

This is what makes it different from RGB.

In RGB, brightness and color are entangled across the red, green, and blue channels. In YUV, brightness is explicitly isolated in the Y channel, while color information is stored separately in U and V.

3. Why is this separation useful?

This matters because not every computer vision task needs raw color in the same way.

In many tasks, the most useful cues come from:

• edges
• contrast
• shapes
• illumination patterns
• object structure

These are often more closely related to brightness than to raw color values.

By separating luminance from chrominance, YUV gives us a representation where:

• Y channel: emphasizes visual structure and intensity
• U/V channels: preserve color information without mixing it directly with brightness

This can be helpful when:

• lighting conditions vary
• shapes and edges are strong signals
• color is useful, but not the only important feature
• we want the model to process brightness and color differently

4. Why might YUV help in traffic sign recognition?

Traffic sign recognition is a good example of a task where this idea can make sense.

A traffic sign is not identified only by color:

• its shape matters
• its border and contrast matter
• its symbol structure matters
• and its visibility under different lighting conditions also matters

Color still helps, of course, but separating luminance from chrominance may allow the model to focus more clearly on the sign structure while still preserving color cues.

That was one of the interesting ideas I encountered while working on this project.

5. How do we convert RGB to YUV?

At a mathematical level, YUV is computed as a transformation of the RGB channels.

A common form of the conversion is Y = 0.299R + 0.587G + 0.114B U = -0.14713R — 0.28886G + 0.436B V = 0.615R — 0.51499G — 0.10001B

The exact coefficients can vary slightly depending on the convention or library being used, but the main principle stays the same:

• Y is a weighted combination representing brightness
• U and V encode color differences

6. Converting RGB to YUV in Python using Pytorch

import torch

def rgb_to_yuv(img):
    """
    img: Tensor of shape (3, H, W) with values in [0, 1]
    returns: Tensor of shape (3, H, W)
    """
    r, g, b = img[0], img[1], img[2]

    y = 0.299 * r + 0.587 * g + 0.114 * b
    u = -0.14713 * r - 0.28886 * g + 0.436 * b
    v = 0.615 * r - 0.51499 * g - 0.10001 * b

    return torch.stack([y, u, v], dim=0)

7. When should we use YUV?

There is no universal rule, but YUV can be worth trying when:

• the task depends heavily on shape and contrast
• color matters, but not as much as intensity structure
• illumination variation is expected
• you want to experiment with alternative input representations

That said, whether YUV helps or not should always be validated experimentally.

9. One important takeaway

This project reminded me that in deep learning, performance is not only about choosing a model architecture.

Sometimes, the way we represent the input image can be just as important.

It is easy to focus only on the following:

• CNN layers
• optimizers
• loss functions
• training schedules

But preprocessing decisions, such as whether we use RGB, grayscale, or YUV, can also shape what the model learns.

For me, that was one of the most useful lessons from this project.

10. Project context

This insight came from a team project completed as part of the Deep Learning track at ITI, where I worked with two teammates on implementing the following:

Traffic Sign Recognition with Multi-Scale Convolutional Networks by Pierre Sermanet and Yann LeCun

If you are interested, you can check out the project repository here:

GitHub Repo: [https://github.com/ahmednashatnoaman-svg/Traffic-Sign-Recognition-Multiscale-CNN]

Final thoughts

YUV was new to me, but learning it helped me better appreciate a simple idea:

image representation is not just a preprocessing detail — it can be part of the modeling decision itself.

If you have worked with YUV, HSV, Lab, or other color spaces in computer vision, I would be interested to know when and why you used them.