class 05 · unit I · architecture and design patterns
Five ways to arrange the parts
Last class we took the box apart. Today we put boxes together: one blueprint every edge AI system follows, and five patterns for splitting the work between device and cloud.
Start with the recall slide; the answer to last class's exit ticket is on it.
last class's exit ticket
300 KB of weights, 150 KB of working memory. Does it fit?
The red segment is deliberately unlabelled in size: it depends on the firmware. The lesson is that "fits" means fits alongside everything else.
One blueprint. Every edge AI system.
Blocks 1 to 5 run on the device every time. Block 6 is used only when the product needs it. Compare with Class 4's software stack: block 2 is the DSP layer, block 3 the runtime and model, blocks 4 and 5 the application.
the doorbell hears someone say "open"
Follow one second of sound through the five blocks.
One block per click. The numbers are a worked example in the style of the TFLite Micro speech demo; Class 8 explains where 49 × 40 comes from.
Never act on a single prediction.
Thresholds, "k in a row", averaging over a few windows: cheap tricks in block 4 that cut false alarms a lot.
Illustrative scores. Ask: what does it cost to raise the threshold to 0.95? (More missed real commands.) That trade-off returns when we evaluate systems in Unit V.
Five patterns for splitting the work.
Real products often combine two. The doorbell, by the end of today, uses two of these at once.
Just the shapes for now. Ask them to guess what each one does from the picture before the detailed slides.
pattern 1
Fully on-device: sense, decide and act in one box.
real examples
a fitness band counting reps; a voice-controlled toy; a smoke alarm with a learned sensor
use it when
privacy is strict, there is no network, or the product must be cheap and simple
The simplest pattern and the one TinyML is built for. Ask: what do you lose? (No updates without a cable; no fleet-wide learning.)
pattern 2
Edge filter, cloud analysis: the device decides what is worth sending.
real examples
CCTV that uploads only clips with people; a factory camera that sends only defects for review
use it when
bandwidth or cost is the problem, but some deeper analysis still needs the cloud
Point at the arrow widths: fat in, thin out. That ratio is the bandwidth saving, the B in BLERP.
pattern 3
Cascade: a tiny model wakes a bigger one only when needed.
real examples
"Hey Google" waking the assistant; the smartwatch from Class 3 asking for an ECG; a camera woken by motion
use it when
most of the time nothing interesting happens, and battery life matters
You have now seen this pattern three times: the smart speaker in Class 1, the watch in Class 3, and here. The power trace is illustrative.
pattern 4
Gateway aggregation: many simple sensors, one shared brain.
real examples
vibration sensors across a factory feeding one edge box; soil sensors on a farm reporting to one hub
use it when
each sensor is too small or too cheap to run a model, or the decision needs several sensors together
The sensors here only measure and transmit. Ask: what happens if the gateway fails? (Everything goes blind: a single point of failure.)
pattern 5
Split computation: cut the model in two.
real examples
mostly research and some phone-to-cloud systems today; an active area
use it when
the full model is too big for the device, but the raw data should not leave it
This is a preview of "segmentation of AI models" and "sharing of AI computation" in Unit IV. Ask: where would you cut, and why there? (Where the features are small.)
A rough guide to choosing.
A teaching aid, not a law: several answers can be yes, and then patterns combine. Run two products from Class 3's cards through it.
Whatever the pattern, plan for three things.
When unsure
graceful degradation
What does the product do when the model is not confident, or the network is down? A safe default beats a guess.
Ask a human
human in the loop
Send low-confidence cases to a person. Their answers become labelled data for the next model.
Plan v2
over-the-air updates
The model you ship on day one will not be the last. Design the update path before you need it.
Ask each team to say which of the three their Class 3 product would need most, and why.
team activity · your Class 3 product
Draw its architecture.
15:00
1 · the blocks
Draw sense → prepare → infer → decide → act, with the real sensor and model names.
2 · the pattern
Which of the five? Or which two? Mark what runs on the device and what does not.
3 · the stress test
What happens when the battery is at 10%? When Wi-Fi drops? When the model is unsure?
Same teams and cards as Class 3, so they build on their own worksheet. Walk round with the box-3 questions; one team presents in the last five minutes.
our running example
The doorbell uses two patterns at once.
Class 2 split the doorbell by BLERP, Class 3 gave it sensors and tasks, Class 4 picked its parts. Today it gets its architecture. Ask why the camera sleeps: power, and fewer chances for a false "person".
exit ticket · on paper, before you leave
Draw the architecture of a smart speaker in five blocks. Which two patterns does it use?
Next class: the data pipeline. What actually flows along those arrows, and how much memory it takes.
Expected answer: cascade (wake word wakes the rest) and edge filter, cloud analysis (only the question after the wake word goes to the cloud).