class 15 · unit II · hardware architecture, microcontrollers and DSPs
What actually does the multiply-add?
Five classes on sensors. Now the other half of Unit II: the chips that turn those numbers into a decision. We start inside a microcontroller, and end by timing one.
Theory runs to slide 13; the lab is one block at the end (slides 14 to 17). The lab needs PSRAM enabled in the board settings (OPI PSRAM on the N16R8): check before class.
last class's exit ticket
A patient leaving bed, without cameras.
sensor
fooled by
mmWave radar
a fan, a moving curtain
pressure mat or load cell
the patient shifting; a visitor sitting on the bed
PIR
a nurse walking past
The best answers combined two. Today: once the data is in, what does the arithmetic, and how fast?
Two minutes. That closes the sensor half of the unit.
guess first
Class 7's gesture network needs 3,328 multiply-adds. How long on the ESP32-S3?
3 s
hands up
30 ms
hands up
3 ms
hands up
30 µs
hands up
Write your guess down. You will measure the real answer yourself in the lab.
Do not reveal. Count hands for each option and write the counts on the board; return to them at the end of the lab.
try it live · one term of sum = sum + w × x
Fetch, decode, execute, one instruction per tick.
A simplified, made-up instruction set, one instruction per tick. Real cores pipeline instructions and some fuse multiply and add, but the lesson holds: of six instructions, two only move data in from memory, and two are loop bookkeeping. Only two do the maths.
The clock sets the ticks. 240 MHz is 240 million ticks a second.
4.2 ns
one tick at 240 MHz
1.25 m
how far light travels in one tick
≤ 240 M
simple instructions a second, per core, at best
If a multiply-add took six instructions, one core would manage about 40 million a second. Two ways to beat that: do more per instruction (SIMD, slide 10), or spend fewer ticks waiting for memory (slide 7).
1 ÷ 240 × 10⁶ ≈ 4.17 ns; 3 × 10⁸ m/s × 4.17 ns ≈ 1.25 m. 240 ÷ 6 = 40. The S3 has two cores; most inference code uses one.
Fetching the next instruction and the next number: one road or two?
DSPs were built Harvard-style so a multiply-add never waits for its own instruction. Most modern microcontrollers mix the two: separate paths and caches near the core, one shared memory further out.
Name it, do not dwell. The useful idea is that the path to memory is a shared, limited resource, which sets up the next slide.
The most important chart in this unit: moving a number costs more than using it.
Energy per operation, log scale. Figures from Mark Horowitz's widely cited ISSCC 2014 keynote, for a 45 nm process. Newer chips are lower across the board; the ratios are what to remember.
Reveal top to bottom. Two ratios to say aloud: an 8-bit add is over 100× cheaper than one SRAM read; a DRAM read costs over 20,000× an 8-bit add. This chart explains int8, caches and accelerators in one picture. Source: M. Horowitz, "Computing's energy problem (and what we can do about it)", ISSCC 2014.
Small and fast at the top, big and slow at the bottom.
On the S3, PSRAM and flash are reached through a cache, so small hot data from them can still be quick; large data streamed through them is not. The lab measures exactly that. Speeds are qualitative; exact cycle counts depend on the access pattern.
Three things this explains.
int8 is faster
not just smaller
Four times fewer bytes to move per weight, and cheaper arithmetic. Class 7's quantisation teaser, explained.
where data lives
changes the speed
The same loop over the same numbers runs slower when they sit in PSRAM. You will measure it.
accelerators
put memory next to maths
The real trick of an NPU is not more multipliers; it is fewer trips to memory. Class 16.
Ask students to predict the lab's two results before they run it: how much faster is int8, and how much slower is PSRAM?
Do more per instruction: one instruction, many pairs.
MAC unit
Hardware that does multiply-and-accumulate as one step, instead of a multiply then an add.
SIMD · single instruction, multiple data
Wide registers holding many small numbers, all multiplied at once. This is what Class 4 called "vector instructions".
The narrower the number, the more fit in one register: another reason int8 wins.
The ESP32-S3's vector extension works on 128-bit registers, i.e. 16 int8 values at a time; check the S3 technical reference manual for the exact instruction set. Espressif's ESP-NN and ESP-DSP libraries use it so you do not have to write it by hand.
Four kinds of small processor, from tiny to TinyML-ready.
kind
examples
strength
for ML
basic MCU
Arm Cortex-M0+, small 8-bit parts
lowest power and cost
only the tiniest models
MCU + DSP/SIMD
Cortex-M4, M7, M55; ESP32-S3
many MACs per instruction
the TinyML sweet spot
MCU + micro-NPU
Cortex-M55 with an Ethos-U55 class NPU
a dedicated ML block
bigger vision and audio models
dedicated DSP
TI C6000 family; audio DSPs in earbuds
very fast fixed signal maths
audio front-ends, FFTs, filters
Ask which row the BrickML from Class 1 likely sits in, and which row a smartwatch's always-on wake-gesture chip might use.
A DSP is built for one loop: filters and FFTs, forever.
single-cycle MAC
a multiply-add every tick
circular buffers
Class 6's ring buffer, in hardware: the address wraps for free
zero-overhead loops
no "increment, compare, branch" instructions
fixed-point maths
integers standing in for fractions: cheap, and close to int8 quantisation
Look back at slide 4: a DSP removes the bookkeeping instructions and overlaps the loads. Six instructions become, in effect, one.
Your earbuds almost certainly contain an audio DSP running noise cancellation. The keyword front-end from Class 8 (FFT, mel bands) is classic DSP work.
our running example
Can the S3 run both of the doorbell's models?
keyword model · yes
Small, int8, run on the S3's vector instructions through ESP-NN. Comfortably real-time.
person detector · it depends
Tens of millions of multiply-adds per frame. Is the S3 fast enough at the frame rate we want? That is your lab's measured number, and Class 16's question.
End of theory. Boards out; the lab needs only the board and a USB cable.
lab · teams of four · board only
Measure your board's multiply-adds per second.
25:00
A
float32 vs int8
The same dot product, two number types. Which is faster, and by how much?
B
SRAM vs PSRAM
The same int8 loop, arrays in two different memories. Does where the data lives matter?
C
answer the hook
Put your measured rate into the calculator: how long does Class 7's network take?
Board settings: ESP32S3 Dev Module, PSRAM: OPI PSRAM (needed for Part B).
Optional extension for fast teams: replace the int8 loop with an ESP-DSP or ESP-NN optimised kernel and compare. Only if you have set the library up and tested it beforehand.
// Part A · one dot product, two number typesconst int N = 1000, REPS = 1000; // one million MACsfloat af[N], bf[N];
int8_t a8[N], b8[N];
voidsetup() {
Serial.begin(115200); delay(1000);
for (int i = 0; i < N; i++) {
af[i] = random(-100, 100) / 100.0; bf[i] = random(-100, 100) / 100.0;
a8[i] = random(-100, 100); b8[i] = random(-100, 100);
}
uint32_t t = micros(); volatile float sf = 0;
for (int r = 0; r < REPS; r++) { float s = 0;
for (int i = 0; i < N; i++) s += af[i] * bf[i]; sf += s; }
uint32_t tf = micros() - t;
t = micros(); volatile int32_t si = 0;
for (int r = 0; r < REPS; r++) { int32_t s = 0;
for (int i = 0; i < N; i++) s += a8[i] * b8[i]; si += s; }
uint32_t ti = micros() - t;
double macs = (double)N * REPS;
Serial.printf("float32: %.1f M MAC/s\n", macs / tf);
Serial.printf("int8 : %.1f M MAC/s\n", macs / ti);
}
voidloop() {}
lab · part A
Run it. Press reset and run it again.
MACs ÷ microseconds = millions of MACs per second. Write both numbers down.
Is int8 faster here? By how much? Is that less than the 4× you might expect? Why might it be?
Plain C loops; the compiler does not automatically use the S3's vector instructions here, so the int8 gain is smaller than a hand-optimised kernel would give. That gap is the reason ESP-NN exists, and a good discussion point. The volatile accumulators stop the compiler deleting the loops.
// Part B · the same int8 loop, arrays in two different memories#include"esp_heap_caps.h"const int N = 100000, REPS = 20; // 100 KB per arrayuint32_ttimeIt(int8_t* a, int8_t* b) {
uint32_t t = micros(); volatile int32_t acc = 0;
for (int r = 0; r < REPS; r++) { int32_t s = 0;
for (int i = 0; i < N; i++) s += a[i] * b[i]; acc += s; }
return micros() - t;
}
voidsetup() {
Serial.begin(115200); delay(1000);
int8_t *aS = (int8_t*)heap_caps_malloc(N, MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT);
int8_t *bS = (int8_t*)heap_caps_malloc(N, MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT);
int8_t *aP = (int8_t*)heap_caps_malloc(N, MALLOC_CAP_SPIRAM);
int8_t *bP = (int8_t*)heap_caps_malloc(N, MALLOC_CAP_SPIRAM);
if (!aS || !bS || !aP || !bP) { Serial.println("alloc failed: PSRAM on?"); return; }
for (int i = 0; i < N; i++) { aS[i] = aP[i] = random(-100, 100);
bS[i] = bP[i] = random(-100, 100); }
double macs = (double)N * REPS;
Serial.printf("SRAM : %.1f M MAC/s\n", macs / timeIt(aS, bS));
Serial.printf("PSRAM: %.1f M MAC/s\n", macs / timeIt(aP, bP));
}
voidloop() {}
lab · part B
Same numbers, same loop, different address.
The arrays are 100 KB each: bigger than the cache in front of PSRAM, so every pass goes out to the external chip.
Try N = 1,000 as well. Does the gap shrink? Why?
heap_caps_malloc with MALLOC_CAP_INTERNAL forces internal SRAM; MALLOC_CAP_SPIRAM forces PSRAM. Plain malloc in the Arduino core may place large blocks in PSRAM automatically, which is why the code is explicit. With N = 1,000 the arrays fit in the cache and the gap mostly disappears: slide 8 made measurable.
lab · part C and worksheet
Type in your measured rate.
your int8 rate, M MAC/s
workload
multiply-adds
time at your rate
A · float32
M MAC/s
A · int8
M MAC/s; speed-up over float32
B · SRAM vs PSRAM
M MAC/s each; how much slower?
B · with N = 1,000
does the gap shrink? why?
C · the hook
your measured time for 3,328 MACs; whose guess was closest?
The default 20 is a placeholder; students type their own. The keyword CNN and person-detector counts are illustrative orders of magnitude, not a specific model's published figures. Compare the gesture-network time with the hands-up guesses from slide 3.
exit ticket · on paper, before you leave
Explain in two sentences why converting a model from float32 to int8 can make it faster, not just smaller. Use one number you measured today.
Next class: when the CPU is not enough. System-on-chip, NPUs, and how to read a TOPS number honestly.
Good answers mention both effects: fewer bytes to move (slide 7) and more values per instruction (slide 10), plus their own measured speed-up.