CV-1 (Basics -RGB, HSV, Processing Techniques, Histogram Equalization)

1. Image processing vs Computer vision?

Image Processing — Processing of an image at a low level such as extracting edges, inverting, and transforming to extract information. Technique to enhance an existing image or to extract important information from it to boost the performance of a model.

Computer Vision (CV) is more higher-level abstraction or understanding of the world. It is a subset of AI that enables machines to derive information from images, videos, and other inputs.

2. What are the types of images?

a) Binary image — Images that have only two unique values of pixel intensity- 0 (representing black) and 1 (representing white). It is commonly used for image segmentation.

b) Grayscale image — Grayscale or 8-bit images are composed of 256 unique colors. A pixel intensity of 0 (black) and a pixel intensity of 255 (white) color, remaining 254 values (different shades of gray).

c) RGB image — RGB or colored images are 16-bit (65,536 different colors) matrices to computers that are possible for each pixel. “RGB”: Red (255, 0, 0), Green (0, 255, 0), and Blue (0, 0, 255) “channels” of an image. pixel value is (0, 0, 0) — black, (255, 255, 255) — white

d) RGBA images are colored RGB images with an extra channel known as “alpha” that depicts the opacity of the RGB image. Opacity ranges from a value of 0% to 100% and is a “see-through” property. Cellophane paper is transparent (100% opacity), frosted glass is translucent, and wood is opaque.

3. What are forward (computer graphics) and inverse (computer vision) models?

Computer graphics — produces images from 3D models with the help of parameters such as light source at a certain angle, color of light, and object placement.

In Computer vision, the inverse model approach takes place. Meaning, that the real world needs to be modeled from a camera (partial/noisy information) which can have lens noise or processing noise as we do not have the information of the real world of which the image is taken.

4. What are the challenges in the CV?

a) Viewpoint variation, b) Illumination, c) Scale, d) Occlusion, e) Deformation, f) Background clutter, g) Intra-class variation, h) Lack of annotated data, i) Inverse model (noise in the pipeline), j) High computational cost, k) Perceive and Cognizant, l) Need human help to solve problems, m) Contextual understanding

5. How are images formed?

When light hits the object’s surface and gets reflected onto an image plane, this gets captured through optics onto a sensor plane, i.e., the photon hits the detector, the detector gets charged and reads out as brightness. The image formation depends on several factors such as the strength, color, and direction of the light source, surface geometry, nearby objects, and sensor properties.

6. CCD vs CMOS

In CCD (charged-coupled device) — It has high sensitivity and power. It cannot be individually addressed. The photogenerated charge is moved from pixel to pixel, and at the output node, it gets converted to voltage. At the end, ADC converts each pixel’s value into a digital value.

CMOS — It has lower sensitivity and power. It can be individually addressed. It converts the charge to voltage inside each element. Several transistors are used in each pixel to amplify and move the charge using more traditional wires. It does not require ADC (signals are digital).

7. Sampling and Aliasing:

If the maximum frequency in the dataset is fmax, then the sampling rate must be fs >= 2fmax -> Nyquist frequency. If this is not followed, then an aliasing effect will occur. Aliasing causes issues when down/upsampling the image.

8. Color space representation: RGB, CMYK (printers), XYZ, YUV, YCbCr, Lab.

Additive colors — R,G,B

Subtractive colors — C, M, Y

Bayer pattern –Arrangement of color filters on the camera sensor. Green (50%) i.e., there are more green filters as they carry more luminance information. Luminance describes the amount of light that passes through, is emitted from, or is reflected from a particular area, and falls within a given solid angle.

9. What is HSV?

Hue: It refers to colors (red, blue).

Saturation: It describes the intensity of a color. Higher saturation means a more vibrant and pure color, while lower saturation brings the color closer to gray.

Value/luminance: It describes the darkness or lightness of a particular color i.e., from black (no brightness) to white (full brightness).

Removing saturation can indirectly remove the luminance/brightness from the image.

10. What is contrast?

The contrast of an image depicts the range of brightness values taken by all its pixels. It is the difference between the lightest and darkest areas.

11. What are cones and rods?

Human eyes are made up of rods (detect intensity) and cones (capture colors). There are mainly 3 kinds of cones with specific sensitivities at specific wavelengths. These cones peak (not exactly) at R (Large), B (Medium), and G (Small). The RGB components of a colored image are already extracted in the retina. The rest of the brain combines this information to form complex colors.

12. Image as a function is an intensity at a given position (x,y). f: R2-> R. A digital image is a discrete and quantized version of it.

13. Different image processing techniques.

1. Point operations — Change each pixel in a way that depends on its value and does not depend on its location or other pixel values i.e., output depends on 1 input of the same coordinates (complexity — constant). Examples — inverting images, modifying contrast, color transforms, histogram equalization.

2. Local operation — Output depends on the neighboring (k x k) input values of the same coordinate (complexity — k2).

3) Global operation — Output depends on all input values of the N x N image (complexity — N2).

14. Disadvantages of a single-point operation

a) Sensitive to outliers

b) Not much information

15. What is image averaging?

Image averaging reduces noise without compromising detail because it increases the signal-to-noise ratio (SNR) of the image.

16. Histogram Equalization

A histogram of pixel intensities tells us how many pixels are present for each pixel intensity. The histogram equalization flattens the distribution of pixel intensities by computing cumulative distributive function (CDF).

CDF is a graph between intensity and cumulative proportion i.e., what is the cumulative distribution at a particular pixel intensity value, say 40? or what percentage of pixels that are darker than 40. Say, the answer is 0.3, then multiply this value by Imax (say, 255) to get the new intensity value for 40.

This improves the global contrast by effectively spreading out the most frequent intensity values.

17. Disadvantages of Histogram equalization.

a) Can amplify noise in the dark areas, b) Can flatten the range in bright areas

18. How to resolve the disadvantages of Histogram Equalization?

Locally adaptive histogram equalization (local operations). The new value of a pixel depends on itself and the neighboring blocks. It is computationally expensive. Example, CLAHE.

19. Histogram normalization scales all the pixel intensities to flatten the range, but it does so by applying the point operation (pixel value is modified locally) which does not work with high-contrast images.