class 04 · unit I · hardware and software components
Open the box
For three classes we treated the device as a black box. Today we take it apart: the chips, the memory, and the seven layers of software between a sensor and a prediction.
Hand one ESP32-S3 board to each team before you start. Ask them not to plug anything in yet.
The board in your hands.
One label per click; teams find each part on their own board before you reveal it. The labels are a simplified drawing, not an exact layout of every revision of the board.
Every edge device is the same five parts.
Build it in order. Point at the BrickML specs from Class 1: five sensors, a processor, memory, BLE, and a 1 µA standby current. Power sits above everything because it constrains everything.
Three kinds of memory, drawn to scale.
The smallest square is the one that matters most. Every calculation the model does happens in SRAM.
At this scale, your laptop's 16 GB of RAM would be a square 9.6 metres wide.
A model can fit in flash and still fail to run, because its working memory does not fit in SRAM. Remember that sentence; it comes back in Class 7.
Squares are drawn with area proportional to capacity. The 9.6 m figure: √(16,384 MB ÷ 16 MB) × 300 px ≈ 9,600 px, roughly the width of the classroom. Walk to the back wall to show it.
ESP32-S3 DevKitC-1 · N16R8
Your board, by the numbers.
2
cores
Xtensa LX7
240 MHz
clock
up to
512 KB
SRAM
on the chip
8 MB
PSRAM
the “R8”
16 MB
flash
the “N16”
Wi-Fi + BLE 5
radio
2.4 GHz
vector
instructions
many multiply-adds at once
Read the part name now: N16R8 = 16 MB of flash, 8 MB of PSRAM. It is why this course bought this variant: the PSRAM holds camera frames.
Figures from the Espressif ESP32-S3 datasheet. Ask: which of these numbers limits the model most? (SRAM.) Which limits the camera? (PSRAM.)
Four rungs of edge compute.
Figures are typical and approximate. Ask where the BrickML sits (first rung) and where a Tesla's computer sits (third). Unit II spends a whole class on each rung.
A neural network is mostly one operation, repeated millions of times.
multiply-accumulate: sum = sum + weight × input
Squares are illustrative, not exact counts. The idea: do more multiply-adds per tick of the clock, and the same model runs faster on less energy.
Ask the room to count roughly how many multiply-adds a tiny network does (Class 7 does this properly). Then: why does it matter how many happen per clock tick? Speed and battery.
1Bootloader and firmwarethe program flashed onto the chip
2RTOS / OSFreeRTOS inside ESP-IDF · Linux on a Pi
3Driverscode that talks to the mic and camera
4DSP / featuresraw signal → spectrogram, statistics
5Inference runtimeruns the model: TFLite Micro, ESP-DL, ONNX Runtime
6The modela file of weights + a graph of operations
7Application logicchime, log, alert, decide what to do
Seven layers between a sensor and a decision.
The model is one layer of seven.
Most of the engineering effort in edge AI sits in the other six. That is why this course is not just a machine learning course.
The stack builds from the bottom up, layer 1 first. For each layer ask: who writes this? (The chip vendor, the OS, you, the ML engineer, the runtime authors.)
The inference runtime: the engine that runs your model.
Runtimes you will meet: TensorFlow Lite for Microcontrollers (now part of Google's LiteRT), Espressif's ESP-DL, ONNX Runtime on phones and PCs, NVIDIA TensorRT on Jetson.
The key idea: the same model file can run on many chips because the runtime and its kernels change per chip, not the model.
On a microcontroller, a model is just an array of bytes.
It is compiled into the firmware like any other constant and stored in flash.
No file system. No download. It is part of the program.
The bytes 54 46 4c 33 spell "TFL3" in ASCII: the model format's signature. The length here is an example.
This demystifies "deploying a model". Ask: so how do you update the model on ten thousand devices? (Flash new firmware, over the air.) That is the "updating is hard" row from Class 2.
The toolchain: from a notebook to a blinking LED.
Stress the loop: nobody gets it right the first time. Every unit of this course is one trip round this loop, done more carefully.
our running example
The doorbell, part by part.
part
our choice
sensors
OV2640 camera · INMP441 microphone
processor
ESP32-S3, using its vector instructions
memory
model in flash · frames in PSRAM · activations in SRAM
connectivity
Wi-Fi, for clips and notifications
power
USB from the existing doorbell wiring
runtime
TFLite Micro with ESP-NN kernels
models
person detector + keyword spotter
application
ring the chime; send a clip when someone rings
One possible design, not the only one. Ask a team to argue for a different processor rung, and what it would cost in power and price.
Where does each one live? Vote, then click.
the model's weightskept when power is off; read-only while running
the firmware itselfthe program lives where the model lives
one 640×480 colour camera frameabout 600 KB: bigger than all of SRAM
the current one-second audio window32 KB, needed fast, again and again
activations while the model runsthe fastest memory, used every operation
the last 100 alerts, kept after a rebootmust survive power-off
Take a show of hands for each row before clicking. A wrong click turns red and still shows the reason; the right button turns green.
exit ticket · on paper, before you leave
A doorbell model has 300 KB of weights and needs 150 KB of working memory while it runs. Where does each part live on the ESP32-S3, and does it fit?
Next class: architecture and design patterns. Five ways to arrange these parts into a working system.
Answer for Class 5's recall: weights in flash (300 KB of 16 MB, easily); activations in SRAM (150 KB of 512 KB, fits, but Wi-Fi and the rest of the firmware also need SRAM, so it is tighter than it looks).