How Cameras Really See the World

Do you know which lenses you should actually use for image recognition?

Photo by Gleb Lucky on Unsplash

If you’ve been following my journey (and this new mini-series here on Medium), you probably read the article:

[embed]Do You Think AI Can Really See? Let’s Talk About Computer Vision Today I want to share the first base of Image Analysis — or rather, what’s behind the fancy term Computer Vision.blog.stackademic.com

Before we jump into deep computer vision topics — like image pre-processing or neural network classification — we need to build a solid foundation.

Because first we have to understand:

How does light become data?

Or, in other words — how does a camera turn photons into pixels?

From Light to Pixels

To get from physical light to numerical data for computers, we rely on image sensors — the digital eyes of our cameras.

The magic happens right there: sensors convert photons into electrical signals, which are then transformed into the pixel values that make up your image.

Types of Sensors

There are several types of sensors, each with different roles and use cases:

Area Sensor (e.g. CCD, CMOS chips)

Used for capturing object images (like forms, rectangular scenes, etc.)

Rectangle — image by author

Line sensor

Similar to area sensors, but the pixels are arranged in a line — perfect for industrial applications like scanners.

Rectangle — image by author

Point sensor

Records just a single intensity value. And they’re not limited to visible light — there are sensors for infrared (IR), ultraviolet (UV), X-ray, and even radio waves (radar sensors).

Rectangle — image by author

CCD Sensors

The CCD (Charge Coupled Device) is a classic example.

It consists of light-sensitive pixels separated by potential barriers and shift registers.

These registers move the captured charges during readout — that’s how the image data gets collected and transferred.

There are different types:

• Full-frame sensors: use almost the entire surface as a sensor (mechanical shutter required).
• Interline sensors: have alternating light-sensitive and masked areas for faster readout.

image by cambridge.org

First integrate then save and then shoft vertical (Step 1) and then cell by cell shift into output register until the row is empty then again shift vertically and so on. The needed time called shutter log

Each sensor type has its own strengths and weaknesses.

Ultimately, image quality depends on how clean and noise-free the signal remains during this conversion.

image by engineersgarage

Common CCD Problems

If you’ve ever seen strange vertical lines or overexposed streaks in an image — that’s not your lens; that’s physics.

Blooming: happens during exposure when a pixel gets overfilled with light and spills into its neighbors.

Smear: occurs during readout when bright light leaks along the readout direction.

Even though CCDs are light and cheap, they can be sensitive to these effects.

image by nikon-fotografie

The second one:

image by www.teledynevisionsolutions.com

From Black & White to Color

Light sensors in cameras mainly detect intensity, not color — so initially, they only produce grayscale images.

To add color, we can:

• Split light optically into separate wavelengths (using 3 sensors for R, G, and B) — but that’s complex.
• Or use a Color Filter Array (CFA) — tiny color filters on each pixel.

The most famous CFA is the Bayer pattern (RGGB) — where green appears twice as often because it’s the most perceptually sensitive color.

If a pixel is missing a certain color (like red or blue), we interpolate it from its neighbors — averaging is fast, but interpolation gives better results.

There’s also the RGBW pattern, which adds white pixels for better low-light performance (like at night).

image by astrobasics.de

the gray is the sensor and the color ones are the color filter.

image by researchgate

Optical Imaging — Seeing the World Through a Lens

Now that we understand how sensors capture light — how does that light even reach the sensor?

Enter the Pinhole Camera Model — or Lochkameramodell.

It’s the simplest model for projecting 3D reality onto a 2D plane.

Fun fact: there are glasses that invert the world completely, but after a few minutes your brain flips it back — our visual system literally re-learns the projection.

image by pngegg

Lens Types: Entocentric, Telecentric, Hypercentric

• Entocentric: the “normal” perspective (what we’re used to seeing).
• Telecentric: a parallel projection used in machine vision for precise measurements.
• Hypercentric: captures multiple sides of an object at once (used for inspecting bottles, labels, etc.).

Each of these defines how the world gets projected onto the sensor — and that projection deeply affects how algorithms later interpret shapes and distances.

image by Beyerer

Example:

Scene:

image by thorlabs

Telecentric:

image by thorlabs

Example of a Hypercentric Perspective

image by researchgate

image by computergraphics.stackexchange

Selecting the Right Lens

When choosing a lens for a computer vision setup, consider:

• Perspective and focal length (fixed or zoom?)
• Sensor compatibility (size, mount, etc.)
• Working distance and aperture
• Resolution (measured in line pairs per mm)
• Optical quality — geometric and chromatic distortions
• Filter threads — for special use cases (e.g. IR cut, ND filters)

In short: your lens defines how the world looks to your model.

Why this Matters

Understanding these basics isn’t just “camera theory” — it’s what separates a working model from a failing one.

If you want reliable object detection or depth estimation, your data source (the camera) must be clean, calibrated, and well-understood.

Coming up next:

In the next part of this series, we’ll explore **how camera calibration works and how images are pre-processed for neural networks** — from denoising to normalization and feature extraction.

About me

I’m Dustin — an AI engineer and founder currently building intelligent healthcare and digital systems.

Here on Medium, I’m writing a small hands-on series about applied AI and the way machines “see” the world — from pixels to perception.

If you enjoy these posts, stick around — the next one will be more hands-on.

Until then: keep your lenses clean and your models sharp.

— Dustin

A message from our Founder

Hey, Sunil here. I wanted to take a moment to thank you for reading until the end and for being a part of this community.

Did you know that our team run these publications as a volunteer effort to over 3.5m monthly readers? We don’t receive any funding, we do this to support the community. ❤️

If you want to show some love, please take a moment to follow me on LinkedIn, TikTok, **Instagram. You can also subscribe to our [weekly newsletter](https://newsletter.plainenglish.io/)**.

And before you go, don’t forget to clap and follow the writer️!