One second of audio is sixteen thousand numbers. The keyword model sees under two thousand, and they are better numbers. Today: how, and why.
Recall first: 800 KB in float32, 200 KB in int8, and "what else needs SRAM?" (the activations).
Fewer numbers, and more meaningful ones.
About 8× fewer inputs means a smaller first layer, a smaller model, faster inference.
And the features describe what matters: which pitches are present, and when. The raw wiggle hides that.
DSP is cheap, well understood, and often done by hardware or tuned libraries. Every edge AI pipeline you build will have some.
Areas to scale. Link to Class 7's calculator: the keyword input there was exactly 1,960.
How fast is fast enough? Twice the highest frequency.
sampling rate > 2 × highest frequency
This is the Nyquist rule. Below it, high frequencies do not just disappear: they come back disguised as low ones.
Voice at 16 kHz captures everything up to 8 kHz, which covers what we need to tell words apart.
The filter before sampling is the anti-aliasing filter: analog, built into most digital microphones. Next slide shows what goes wrong without it.
try it live · sampling at 10 per second
Sample too slowly and a fast signal pretends to be a slow one.
signal
Grey: the real signal. Dots: what the sampler records. Red: the slowest wave through those dots, which is what the system "sees". Below 5 Hz red and grey agree; above it they do not. 9 Hz looks like 1 Hz; 11 Hz looks exactly like 1 Hz. Mention the wagon wheel turning backwards in films.
Each sample is stored with a limited number of levels.
5 levels
17 levels
8-bit
256 levels · 1 byte
16-bit
65,536 levels · audio standard
same idea
quantising weights, Unit IV
The word "quantisation" returns in Unit IV for model weights: same idea, fewer levels, fewer bytes, a small loss of precision.
filtering
The simplest filter: average each point with its neighbours.
window of 3
average
2, 8, 3
4.33
8, 3, 9
6.67
3, 9, 4
5.33
Input: 2, 8, 3, 9, 4. Jumpy.
Output: 4.33, 6.67, 5.33. Calmer. The fast jitter is gone; the slow trend stays.
That is a low-pass filter: it lets slow changes through and blocks fast ones.
High-pass does the opposite (removes gravity from an accelerometer). Band-pass keeps a range (a motor's hum). All cost a few multiply-adds per sample.
Students compute each row before it appears. Ask what happens with a window of 5 (smoother, but slower to react).
try it live
Make the window wider. What do you gain, and what do you lose?
window
The signal is a slow wave plus noise, with a sudden step marked by the dashed line. Small windows leave noise; large windows smooth it but blur the step. Every filter trades noise against reaction time.
time → frequency
Any signal can be built from simple waves.
Your ear does the reverse: it hears the messy sum and picks out the pitches. The Fourier transform does the same with arithmetic.
Draw the two sines on the board, then add them point by point at three or four places. The messy curve is "just" two waves.
try it live · the FFT
Switch waves on and off. Watch both views.
waves
TIME · WHAT THE SENSOR RECORDS · 0.1 s
FREQUENCY · WHAT THE FFT REVEALS · 0 TO 1 kHz
Start with 50 Hz alone: one clean wave, one spike. Add 120 Hz: the wave gets messy, the spectrum just gains a second spike. Add noise: the time view becomes unreadable, the spikes still stand out. That is why models are fed frequency features. Computed live with a discrete Fourier transform of 200 samples at 2 kHz.
A spectrogram: one FFT per short slice, stacked side by side.
Time runs left to right. Pitch runs bottom to top. Darker means louder.
The vowel sits low with a few bands moving. The "s" at the end is a smear of high frequencies.
It turns sound into a picture, so the image tools from Class 7, small CNNs, apply directly.
Computed from a synthetic signal, not a real recording; the demo later in class uses your voices.
This image was computed with the same settings as the keyword-spotting example (next slide). Ask the room to point at the vowel and the "s".
Where 49 × 40 comes from.
step
number
slice length
30 ms · 480 samples
stride
20 ms
slices: (1000 − 30) ÷ 20 + 1
49
frequency bands per slice
40
features per second
49 × 40 = 1,960
why overlapping slices?
Exactly Class 6's windowing: overlap means a short sound is never cut in half at a slice boundary.
why 40 bands, not 241?
A 480-sample FFT gives about 241 frequency bins. Grouping them into 40 bands keeps what matters for speech and throws away detail the model does not need.
These settings match the TensorFlow Lite Micro speech example. (1000 − 30) ÷ 20 = 48.5, so 48 full strides after the first slice: 49 slices.
Bands shaped like human hearing.
Your ear tells 100 Hz from 200 Hz easily, but not 7,000 from 7,100. Mel bands are narrow at low pitches and wide at high ones, so the 40 bands go where the information is.
MFCCs go one step further and compress the mel bands again. Name only today; Unit III treats them as features.
Name and intuition only. No mel formula today.
The same idea for motion: describe the window, do not copy it.
feature, per axis
what it captures
for example
mean
the average orientation
is the wrist up or down?
standard deviation
how much it moves
sitting versus walking
peak-to-peak
the biggest swing
a fall, a tap
strongest frequency
how fast it repeats (FFT)
walking near 2 steps a second, running faster
4 features × 3 axes = 12 numbers instead of 600. This is how the gesture network in Class 7 got its inputs.
Link back to Class 6 (600 → 12). The step-rate example is approximate.
# Colab · record "yes" and "no" on a phone, upload the WAVsimport librosa, librosa.display
import matplotlib.pyplot as plt
y, sr = librosa.load('yes.wav', sr=16000)
fig, ax = plt.subplots(2, 1, figsize=(10, 6))
librosa.display.waveshow(y, sr=sr, ax=ax[0])
S = librosa.feature.melspectrogram(
y=y, sr=sr, n_fft=512, win_length=480,
hop_length=320, n_mels=40)
librosa.display.specshow(librosa.power_to_db(S), sr=sr,
hop_length=320, x_axis='time',
y_axis='mel', ax=ax[1])
plt.show()
live demo · 10 minutes
Two waveforms that look alike. Two pictures that do not.
Record two students saying "yes" and "no". Plot both.
Can the room tell the words apart from the waveforms? From the spectrograms?
Same settings as slide 11: 30 ms windows (480 samples), 20 ms hop (320), 40 mel bands.
Usually the spectrograms are easy to tell apart and the waveforms are not. That is the argument for DSP in one picture.
Good edge engineers try the cheapest thing first.
If step 1 works, ship step 1. A threshold never needs retraining, never runs out of SRAM, and is easy to explain.
Ask which step the BrickML probably needs (step 2: learned "normal" rather than a single threshold), and which a smoke alarm uses (step 1).
our running example
The doorbell's ear, end to end.
Every arrow here is cheap arithmetic that runs before the model. Remove it and the model would need eight times more inputs and still learn less.
The doorbell has now been through every class: BLERP (2), recipe (3), parts (4), architecture (5), pipeline (6), model size (7), and DSP today.
exit ticket · on paper, before you leave
A motor hums at 50 Hz. A failing bearing adds a tone at 300 Hz. What is the slowest sampling rate that can see the fault? If you sample at 400 Hz, where does the 300 Hz tone appear?
Next class: challenges and opportunities, and the whole of Unit I in one picture.
Answers: faster than 600 Hz. At 400 Hz, the 300 Hz tone aliases to |300 − 400| = 100 Hz, where it could be mistaken for something harmless.