Introduction to Neural Network Quantization

This blog will go over Quantization’s fundamentals and these will touch the basics we need to know to learn the advanced matrial in next blog,

What is Quantization

Quantization is the process of representing a continuous or high precision value using a smaller set of discrete values. In deep learning, model quantization means converting model parameters and/or activations from high-precision formats like FP32 into lower-precision formats like INT8, INT4, or sometimes FP16/BF16. Think of it has berriers we store inside one of the sections of refrigerator. Similarly, these can be weights inside the neural nets stored in the memory. The data type will be discussing shows how weights or anyhting smilar are stored inside memory.

What is FP32?

FP32 means 32-bit Full Precision. is a number format used to store decimal/real values in neural networks.

Example: 0.1234,−2.71,15.89,0.000034

Neural networks usually store weights and activations as decimal numbers, so they often use floating-point formats.

What is BF16?

BF16 = BFloat16 = 16-bit half precision. BF16 is also 16-bit, but different from FP16.

Integer formats used in quantization

Now we move from floating point to integer. No decimal directly.

Integer means values like: −128,−3,0,7,127

What is INT8?

NT8 = 8-bit integer. This is the most common quantization format.

What is UINT8?

UINT8 = unsigned 8-bit integer. Unsigned means no negative values.

Clean comparison table

Clean Comparision Table

Which ones matter most for quantization?

The goal is: reduce precision without destroying model performance

Q: What things can be quantized ?

1: Weight/Parameter

These includes Linear layer weight matrix, Convolution filters, Embedding, table, Attention projection matrices, MLP weight, etc..

2: Activations

Activations are the intermediate outputs produced while the model runs.

a = f(wx+b)

Activations cane be quantized, this helps reduce memory movement and can speed up inference. But activation quantization is harder than weight quantization because activations change depending on the input.

3: Biases

Bias terms can be quantized too, but often they are kept in higher precision. Bias values are sensitive because they are added after multiplication, so many systems keep them in higher precision.

4: Gradients

During training, neural networks compute gradients.

Gradient: thetaL / thetaW

But for normal inference quantization, you do not care about gradients because inference does not use backpropagation

5: Optimizer States

During training, optimizers like Adam store extra tensors. Some memory-efficient training methods quantize optimizer states to reduce GPU memory. This is training-focused, not inference-focused.

6: KV Cache

In large language models, during generation, the model stores attention keys and values. For long-context generation, KV cache memory becomes very large.

KV cache FP16 → INT8 / INT4

This helps LLMs generate longer sequences with less memory

Lets see the first type of Quantization

LINEAR QUANTIZATION

Linear quantization means mapping a real-valued FP32 range to a lower-precision integer range using a straight-line relationship.

Lets make up such formula and try to apply linear quantization

r = s(q-z)
Therefore: q = r/s+z

TERM r

r = orignal real value

s = scale, How much real value one integer step represents?

s = (rmax - rmin)/ (qmax - qmin)

Example:

r range is 1 to 10
q range is 1 to 5 
s = (rmax - rmin)/ (qmax - qmin)
s = 2.25 
MEANS: Every time the quantized integer q increases by 1, the real value r 
       increases by 2.25

TERM z

z = zero-point, Which integer code represents real zero?

z = round(qmin -(rmin / s ))

Example:

r range is -1 to 3
q range is 0 to 255
s = (rmax - rmin)/ (qmax - qmin)
s = 0.00157\
z = round(qmin -(rmin / s ))
z = 64
MEANS: Integer value q=64 represents real value r=0.
This serves the formula to maintain the midpoint or 0 value for both r and q 

Use case of z

1. ReLU outputs

After ReLU:

ReLU(x)=max⁡(0,x)

Many activation values are exactly zero. If zero is represented exactly, quantized inference is more accurate.

2. Padding

In CNNs, Transformers, and sequence models, padding often uses zeros.

If real zero is exactly representable, padding remains truly zero after quantization.

3. Sparse tensors

Many models have lots of zeros.

If zero maps exactly to an integer code z, sparse behavior is preserved better.

TERM q

q = quantized value

q = r/s + z 

Result:

Depth into Root of Model Quantization

There are scenerios where we see after adding quantization, there is major loss of information. But the root cause can be mis-matched scaling. To avoid these issues we must understand Symmetric and Asymmetric Quantization.

Symmetric Quantization

In symmetric quantization, the real range is forced to be symmetric around zero. Symmetric quantization is simple and efficient. It is very common for weights, because neural network weights are often distributed around zero:

negative weights  ←  0  →  positive weights

Weights naturally have both negative and positive values, so symmetric quantization makes sense.

Example

Range: 0 to 1000
Quantized range: 0 to 255 
scale = s = (rmax - rmin)/ (qmax - qmin) = 3.92

If we get a value of 194 from the Range, means after quantization we recieve
194/3.92 =49.49 

Asymmetric Quantization

In asymmetric quantization, the real range does not need to be symmetric around zero.

Example

ange: -20 to 1000
Quantized range: 0 to 255 
scale = s = (rmax - rmin)/ (qmax - qmin) = 4

If we get a value of -5 from the Range, means after quantization we recieve
-5/4=-1.25 : THESE DOES NOT EXIST 

WE ADD ZERO POINT WHICH IS "Which integer code represents real zero?"
we add the zero point to make it 0 even.

we get -1.25 + 1.25 = 0

Next: There are two Modes of Quantization

• Post Training Quantization (PTQ)
• Quantization-Aware Training (QAT)

Will continue this……

Refrences

Fundamentals of QuantizationStanford Universityhttps://ee.stanford.edu › ~gray › shortcourse

[embed]Quantization in Depth - DeepLearning.AI Customize model compression with advanced quantization techniques. Try out different variants of Linear Quantization…learn.deeplearning.ai

Quantization: Post Training Quantization, …Towards AIhttps://towardsai.net › Publication › Latest