Tensor by Tensor

MODULE 03 · LESSON 04

Quantization

Quantization maps many floating-point values onto a smaller integer set.

PLAIN-LANGUAGE INTRODUCTION

What is this?

Quantization maps many floating-point values onto a smaller integer set.

One simple example

With scale 0.1 and zero point 0, floats [0,0.4,1.2] map to integers [0,4,12].

What goes in?

Floating-point values plus a scale, zero point, and integer range.

What comes out?

Integer values that decode to nearby floating-point values.

Why does it matter?

Smaller numbers can reduce model size and improve speed on suitable hardware.

What is it not?

Quantization does not guarantee faster inference or unchanged accuracy.

WORK THROUGH THE IDEA

See the idea in more detail

  1. Scale says how much one integer step represents. Here, one step represents 0.1.
  2. Zero point says which integer represents real zero. Here, that integer is 0.
  3. Divide each float by 0.1: [0,0.4,1.2] becomes [0,4,12].
  4. Decode by multiplying by 0.1. These three chosen values return exactly in this example.
  5. Common mistake: values between representable steps must be rounded, which introduces error.
Open the detailed notes ↗