We have said "the model" for six classes. Today we open it: a neuron, a layer, a network. Then we count its numbers, and decide whether it fits on the board.
This class carries the neural-network minimum for a cohort that has not studied them yet. Go slowly in the first half; Units III and IV assume it.
last class's exit ticket
3 axes × 50 Hz × 2 bytes × 2 s =
600 bytes
With 50% overlap a new window starts every second: about 60 windows a minute, so 60 inferences a minute. Today: what happens inside each one.
Take two answers before revealing. Common mistake: forgetting the 2 bytes per sample.
How big is a model? Guess first.
Logarithmic scale: each grid line is 1,000 times the one before. Sizes are approximate, for 32-bit weights except where the model is usually shipped smaller.
Reveal the three device lines first, then the models one by one, asking "which side of the line?" each time. Figures are rough public estimates; the point is the nine orders of magnitude.
part 1 · neural networks in 25 minutes
A neuron: multiply, add, squash.
y = ReLU(w₁x₁ + w₂x₂ + b)
step
value
0.5 × 0.8
0.40
0.2 × (−0.4)
−0.08
+ bias 0.1
0.10
sum
0.42
ReLU: max(0, sum)
0.42
Do the arithmetic on the board before each row. Then ask them to change w₂ to −2 and recompute: sum = 0.4 − 0.4 + 0.1 = 0.1. Then −3: sum becomes negative and ReLU outputs 0.
Play with it. Can you make the output exactly zero?
input x₁0.5
input x₂0.2
weight w₁0.8
weight w₂−0.4
bias b0.1
sum
ReLU(sum)
Training is exactly this, done automatically: nudge the weights and the bias until the outputs are right for thousands of examples.
Let a student drive. Challenge: get the sum negative using only w₁ (it needs w₁ ≤ −0.1). Then: which knobs can training change? (Weights and bias. The inputs come from the sensor.)
The activation is what makes depth worth it.
Without it, a stack of layers is just multiply-and-add again and again, and that collapses into one single multiply-and-add. Depth would buy nothing.
ReLU is the simplest bend: max(0, x). One comparison. Cheap on a microcontroller.
Others exist (sigmoid, tanh, softmax at the output). ReLU and its cousins dominate edge models because they are cheap.
Do not prove the collapse formally; one sentence and the picture are enough here.
A layer: many neurons reading the same inputs.
Every line is one weight. Count them: 3 × 4 = 12. Plus one bias per neuron: 4. This layer has 16 numbers to learn.
All four neurons together are one matrix multiplication. That is why GPUs, and the S3's vector instructions, are good at this.
parameters = inputs × neurons + neurons
Have them count the lines on the picture. The formula on the last line is the one they will use for the rest of the course.
A network: layers stacked, each feeding the next.
This is the network we count on the next slides. Ask: how many layers of weights? (Two: 100→32 and 32→4.)
A convolution reuses one small set of weights everywhere.
A 3 × 3 filter has 9 weights. It slides across the whole image, looking for the same pattern in every position.
A fully connected layer on a 96 × 96 image needs 9,216 weights per neuron. The filter needs 9.
That reuse is why almost every vision and audio model on a microcontroller is a CNN. Unit III covers them properly; today, just the name and the reason.
Name only. Do not teach convolution arithmetic today.
Training is a loop. Inference is one pass.
Inference needs no error, no nudging, no loop: just multiply-adds with fixed weights. That is why a microcontroller can do it.
This closes the neural-network minimum. Ask one student to explain training versus inference in their own words before moving on.
part 2 · counting
Count the gesture network: 100 → 32 → 4.
quantity
calculation
result
layer 1 parameters
100 × 32 + 32
3,232
layer 2 parameters
32 × 4 + 4
132
total parameters
3,232 + 132
3,364
as 32-bit floats
3,364 × 4 bytes
≈ 13 KB
as 8-bit integers
3,364 × 1 byte
≈ 3.3 KB
multiply-adds per inference
100 × 32 + 32 × 4
3,328
Storing weights in 8 bits instead of 32 makes the model four times smaller. That trick is called quantisation; Unit IV is about doing it without losing accuracy.
Students compute each row before it appears. Multiply-adds: one per weight; biases are additions and are usually left out of the count.
try it live
Design a network. Will it fit?
input
width
layers
parameters
float32 weights
int8 weights
multiply-adds
int8 weights vs SRAM 512 KB
float32 weights vs flash 16 MB
Sequence: gesture, 32, 1 layer (tiny). Then image, 256, 3 layers: over 2 million parameters, too big for SRAM even in int8. Then ask why the image models we actually ship are CNNs (slide 9). Fully connected networks only; outputs are 4, 12 and 2 classes.
Fits in flash, fails in SRAM: the working memory decides.
A small CNN for a 96 × 96 grey image. While each layer runs, its input and its output must both sit in SRAM. Bar = that pair.
float32 peaks at about 432 KB: it cannot run, although its weights fit easily in flash. int8 peaks at about 108 KB: it runs.
Reveal float32, then the budget line, then int8. Peak = (48×48×16 + 48×48×32) × 4 bytes = 442,368 bytes ≈ 432 KB in float32, ÷ 4 ≈ 108 KB in int8. The 300 KB budget is illustrative. This is the "fits in flash, fails in SRAM" sentence from Class 4.
Three circles, one inside the other.
Embedded ML: machine learning running inside a dedicated product, on any size of chip.
TinyML: the microcontroller end. Models of kilobytes to a few megabytes, power in milliwatts, often on a battery for months.
Typical TinyML jobs: wake words, gestures, vibration anomalies, simple person detection. Exactly the products from Class 3.
All TinyML is embedded ML; all embedded ML is edge AI. Not the other way round.
The TinyML workflow is a loop, not a line.
Steps 1 to 5 on the laptop (blue), 6 and 7 on the board (green). Compare with Class 4's toolchain: today we added "collect data" at the start and closed the loop.
Why TinyML is harder than it looks.
KB
memory
every activation and buffer must be planned in advance
no OS
or a tiny one
no files, no processes, no pip install
int
arithmetic
some chips have no floating-point hardware at all; 8-bit integers are fastest
blind
debugging
no notebook, no plots: a serial print and an LED
And yet a few kilobytes are enough to hear a wake word. That is the whole course in one sentence.
The ESP32-S3 does have a floating-point unit; many smaller microcontrollers do not. Either way, int8 is usually faster and smaller.
exit ticket · on paper, before you leave
A model has 200,000 parameters. How big are its weights in float32, and in int8? Which of the ESP32-S3's three memories could hold each?
Next class: digital signal processing. How 16,000 numbers a second become the 49 × 40 the model actually sees.
Answer: 800 KB in float32 (flash or PSRAM, not SRAM); 200 KB in int8 (flash, or it would even fit in SRAM). Then ask: what else needs SRAM while it runs? (Activations, slide 13.)