For the next hour, forget the data centre. We are going to look at AI that runs on a chip smaller than your thumbnail, right where the data is born.
Do not explain anything yet. Go straight to the video. The definitions only land after they have seen it working.
before we start
Watch a short demo. Keep one question in your head.
Where is the AI running?
Reveal the question before playing. Tell them you will ask it at the end and you want a precise answer, not "in AI".
Edge Impulse · BrickML predictive maintenance demo · Embedded Vision Summit 2023 · open on YouTube
Click the video to play. Afterwards click the cream margin outside the video once, so the arrow keys work again. If the hall network blocks YouTube, download the clip beforehand.
So where did the AI run?
Not in a data centre. Not on a laptop. Not on a phone.
In a small box stuck on the machine itself.
Ask the question and then stop talking. Wait for at least three answers. Someone will say "the cloud" or "the laptop". Do not correct them; reveal the first line and let the room adjust.
Meet the box.
BrickML is a reference design from Edge Impulse and Reloc. You mount it on a motor or a pump. It listens to the machine and decides, by itself, whether the machine is healthy.
It learns what normal looks like. Anything that does not match is an anomaly: an early sign of a fault, weeks before a breakdown.
This is called predictive maintenance: fix the machine before it fails, not after.
40×23×5
size in millimetres
1 µA
standby current
5
kinds of sensing: vibration, temperature, humidity, sound, voltage
BLE
a low-power radio, for results only
Hold up an ESP32-S3 next to this slide. "The board in your kit is in the same family." Point at 1 µA: a typical phone draws hundreds of thousands of times more.
Everything in that demo needs words we have not defined yet.
"Hey Google" is heard on the phone; your question is answered in the cloud
Two minutes, pairs. Then reveal. The "Hey Google" split is the most important one: a tiny always-on model on the phone, a big model in the cloud. That pattern comes back in Class 5.
part two
Twelve words, defined properly
Every later unit assumes these. If one of them is fuzzy now, everything built on it will be fuzzy too.
Slow down from here. Each term gets an example from the demo or their phones.
Edgewhere data is born
The place where data is produced and the physical world is sensed, as opposed to a central data centre.
A factory floor. A farm. A wrist. A car. A motor with a BrickML on it.
edge device
Any computing device at the edge: a microcontroller, a phone, a gateway, a local server.
"Edge" is a position, not a size. A GPU box in a hospital basement is still an edge device.
Ask: is a phone an edge device? Yes. Is a hospital's own server? Yes, if it sits near the data rather than in a remote data centre.
Embedded systema computer you never see
A computer built into a larger product to do one dedicated job.
The washing machine's controller. The car's ABS unit. The BrickML.
microcontroller (MCU)
A single chip holding a processor, memory and input/output pins. Usually no operating system. The ESP32-S3 in your kit is one.
512 KB
ESP32-S3 SRAM
16 GB
your laptop's RAM
~30,000×
less working memory
Pass a board around while you talk. Let the 30,000× sit for a moment: the whole course is about doing useful AI inside that gap.
IoTinternet of things
Networks of physical devices that sense and exchange data.
classic IoT
Sensor measures.
Raw data goes to the cloud.
The cloud thinks.
IoT with edge AI
Sensor measures.
The device thinks.
Only the decision travels.
IoT is about connection. Edge AI is about where the thinking happens. A device can be one without the other.
Common confusion. A smart meter that uploads readings every 15 minutes is IoT with no AI on it. A toy that responds to voice offline is edge AI with no internet.
AI and machine learning: who writes the rule?
a rule written by an engineer
if vibration > 2.5 g → alarm
Simple and cheap. But who picked 2.5? It misses faults that change the pattern of vibration without making it louder.
a rule learned from data
record normal → learn it → flag what differs
This is machine learning: the rule comes from examples, not from a person. It is what the BrickML did.
AI is the goal: machines doing tasks we associate with intelligence. ML is today's main way of getting there. The learned rule is called a model.
The 2.5 g threshold is illustrative, not from the video. Say so. The point is the contrast between a hand-picked number and a learned pattern.
The most important distinction of the day.
training · learning the rule
Show the model thousands of examples and adjust it until it is right most of the time.
hours
on a GPU
rarely
once, or now and then
inference · using the rule
Give the finished model one new input and get one answer back.
ms
on the device
always
every second, for years
Train in the cloud. Infer at the edge. That one sentence is why edge AI is possible at all.
Almost every student mixes these up. Ask: in the demo, which part was training and which was inference? Training happened on a computer through Edge Impulse; inference ran on the box.
definition
Edge AI is running AI inference on, or near, the device that collects the data.
Sometimes training happens there too, but that is rarer and harder. We will see it in Unit III.
Make them write this one down word for word. The exit ticket asks them to say it without the word "edge".
TinyMLedge AI at the smallest end
Machine learning on microcontrollers: kilobytes of memory, milliwatts of power, often running on a battery for months.
KB
model size
mW
power budget
24 / 7
always listening
Wake words in earbuds. Gesture detection in a smartwatch. The BrickML watching a motor.
All TinyML is edge AI; not all edge AI is TinyML. A phone running a vision model is edge AI but not TinyML.
DSPdigital signal processing
Turning raw sensor signals into something compact and meaningful before a model sees them.
Smaller input, smaller model, faster inference. Class 8 is all about this.
The feature values are illustrative. The idea to land: the model never sees the raw wiggle; it sees a handful of numbers that describe it.
Twelve terms, one line each.
term
meaning
edge
Where data is produced, away from the data centre
edge device
Any computer at the edge: MCU, phone, gateway, local server
embedded system
A computer inside a product, doing one job
microcontroller
CPU, memory and I/O on one chip, usually no OS
IoT
Connected devices that sense and share data
AI
Machines doing tasks we associate with intelligence
machine learning
Getting the rule from examples instead of writing it
model
The learned rule: a file of numbers plus a recipe
training
Finding the model's numbers from data; heavy, rare
inference
Using the model on a new input; light, constant
edge AI
Inference on or near the device that collects the data
TinyML / DSP
Edge AI on microcontrollers / turning signals into features
Photograph-worthy slide. Give them thirty seconds, then move on.
It is not edge versus cloud. It is a spectrum.
First click: the edge AI and cloud AI brackets. Second click: TinyML at the far left. Ask where the BrickML sits (sensor node) and where ChatGPT sits (cloud).
Think, pair, share: where on the spectrum?
device
where the AI runs
why
Smartwatch fall detection
sensor node
must work with no network, on a battery
Gallery grouping your photos by face
phone
your photos never need to leave the phone
Car lane detection
in-car computer
a network delay is a safety risk
CCTV that sends only alerts
camera or edge server
video is too heavy to upload all day
"Hey Google, what's the weather?"
phone + cloud
wake word local, question in the cloud
Three minutes in pairs before revealing. Reveal column by column per row. Do not explain the "why" in depth; that is Class 2's BLERP. Just say "we will give these reasons names on Wednesday".
Three things people get wrong.
"edge AI means no cloud at all"
Most real products are hybrids. The BrickML still sends results out over its radio.
"edge AI is weaker AI"
It is right-sized AI. A tiny model that answers in milliseconds beats a huge one that answers too late.
"IoT and edge AI are the same thing"
IoT is about connection. Edge AI is about where the thinking happens.
Ask the room to vote true or false on each before revealing the correction.
back to the demo
Now describe the box in one sentence.
An embedded system with sensors uses signal processing to turn vibration into features, runs inference with a TinyML model that was trained elsewhere, and sends only the decision over the network.
That is edge AI. Forty-five hours from now you will be able to build one.
Ask a student to read the sentence aloud. Then ask if any word in it is still unclear. If one is, it is the first thing to fix next class.
exit ticket · on paper, before you leave
Define edge AI in one sentence without using the word "edge".
Next class: why would anyone bother? Five reasons, one word to remember them by.
Collect the slips. Read two good ones aloud at the start of Class 2 as the recall.