class 17 · unit II · FPGAs, ASICs and edge servers
Rewire the chip, design the chip, or share a bigger one.
Last class you bought an accelerator off the shelf. Today: hardware you can rewire after it is made, hardware made for exactly one job, and one powerful box shared by a whole building.
No board lab today. Theory runs to slide 15; the exercise block is slides 16 to 18. Print the six scenario cards from slide 16, one set per team.
last class's exit ticket
A has twice the TOPS, at four times the power. Battery camera: which first?
investigate B first
Half the TOPS for a quarter of the power: twice the TOPS per watt.
then ask
Is B fast enough for the frame rate at all? At what precision, with what utilisation, on which layers? Is there a measured benchmark?
Two minutes. The best answers noticed that "most efficient" is useless if it is too slow for the job.
try it live · always listening for "Hey…" on two AA cells
Why would anyone build a chip just to hear one word?
always listening on
battery
The 200 µW figure is Syntiant's announced active power for its NDP100 neural decision processor in always-on use (2018). The S3 figure is a rough estimate for a board with its CPU awake. Battery energies are typical nominal values; real batteries also self-discharge over years, so very long lifetimes are an upper bound. The point: a chip designed for one job wins by three orders of magnitude.
try it live · click a point on the line
Every processor sits somewhere between flexible and efficient.
Most NPUs from Class 16 are ASIC blocks inside an SoC: fixed hardware for neural-network layers, sitting next to flexible CPU cores. The ESP32-S3 is a CPU with a little help from vector instructions.
try it live · load a different bitstream
An FPGA is a grid of blank logic. A file decides what it becomes.
FPGA: field-programmable gate array. "Field-programmable" means you can change it after it leaves the factory, even after it is installed. The bitstream is loaded from flash at power-up. Logic blocks, memory blocks and DSP slices (hardware multipliers) are the three ingredients. Layouts here are schematic.
try it live · four bits make a logic gate
Inside each block: a tiny memory that is the truth table.
configure as
inputs
A 2-input lookup table (LUT) stores four bits, one per input combination; the inputs simply select which bit comes out. Change the four bits and the same hardware becomes a different gate. Real FPGAs use 4- to 6-input LUTs, thousands to millions of them. This is the whole trick behind "rewiring".
Where FPGAs earn their place at the edge.
fixed latency
every sample takes exactly the same time, no interrupts
odd interfaces
talk to any sensor at its own speed, even ones no MCU supports
change after shipping
new model or new sensor: send a new bitstream
small runs
custom hardware without making a custom chip
the costs
Harder to program: hardware description languages, or high-level synthesis tools.
Usually less efficient, and dearer per unit, than an ASIC doing the same job.
examples, by name only
Small, low-power FPGAs (Lattice iCE40 class) for always-on presence detection at a few milliwatts.
Larger AMD (Xilinx) and Intel (Altera) parts in industrial cameras and radar.
No hardware description language in this course; the aim is to know when to reach for one. Lattice markets its sensAI stack for always-on AI at "a few milliwatts"; check current reference designs for exact numbers.
try it live
A CPU is fast on average. An FPGA is fast every single time.
Simulated, illustrative latencies for 60 samples against a 1 ms deadline. Ask: if a bottle is rejected by an air jet as it passes, which matters more, the average or the worst case? Hard deadlines are where FPGAs (and careful real-time MCU code) earn their keep.
An ASIC is a chip designed for one job. They exist at every scale.
data centre
Google TPU
Racks of chips built only to train and run neural networks.
your pocket
Apple Neural Engine
An NPU block inside the phone's SoC: Face ID, photos, dictation.
your ear
always-on audio chips
Keyword spotting in microwatts: the hook's chip.
ASIC: application-specific integrated circuit. Nothing wasted on flexibility, so the most work per watt and the lowest cost per chip. Once you have paid to design it.
Ask: why does Google bother designing its own chip when it could buy GPUs? (Scale: at millions of chips, efficiency and unit cost dominate.)
The catch: you pay a fortune before the first chip exists.
NRE
non-recurring engineering: paid once
Design team, verification, software tools and licences, and the photomasks used to print the chip.
1–2 years
from idea to working silicon
Your model may be out of date before the chip ships.
frozen
no changes after manufacture
A bug in the hardware means a new set of masks: a "respin".
Mask costs rise steeply with newer, smaller process nodes; at the leading edge, total design cost runs into many millions of dollars. Older nodes are far cheaper, which is why small always-on chips are often made on them.
try it live · illustrative rupee figures, invented for the arithmetic
Buy a microcontroller, or design an ASIC? It depends how many you sell.
MCU / unit
ASIC / unit
ASIC NRE
Worked example: ₹30 crore ÷ (₹400 − ₹50) ≈ 8.6 lakh units (₹30 crore is 30 × 10⁷ = 3 × 10⁸). Below that, buy; above, design. The rupee figures are invented to make the arithmetic clear; real ASIC costs depend heavily on the process node and are often far higher. Drag the NRE to ₹100 crore and watch the crossover move off the chart.
The break-even is necessary, not sufficient.
question
why it can overturn the sum
will you really sell that many?
The NRE is spent whether the product sells or not.
can you wait two years?
A competitor using an off-the-shelf part ships first.
will the model change?
A frozen ASIC cannot run next year's better network unless it was designed with room to spare.
does efficiency sell the product?
For a hearing aid or earbud, battery life is the product. For a mains-powered box, maybe not.
This is the bridge to the scenario exercise: teams will need both the sum and these questions.
An edge server: data-centre muscle, in the building.
Also at the telecom base station: multi-access edge computing (MEC), where the operator runs your workload one hop from the phone. This is Class 5's gateway, grown up.
What it adds over devices: models too big for any camera, one place to update them, and a view across all cameras (tracking a person from camera 3 to camera 7). Over the cloud: low latency, and video never leaves the building (the P in BLERP).
try it live · a mall's internet connection
Forty cameras streaming to the cloud: will the uplink cope?
cameras
stream quality
all video to the cloud
edge server, events only
Bitrates are typical H.264 values; the 100 Mbps uplink is an illustrative business connection, so ask what the college's actual uplink is. The events-only line assumes about 20 kbps per camera for alerts and snapshots, also illustrative.
our running example
One doorbell, a hundred doorbells, ten million doorbells.
1
your prototype
An off-the-shelf board like the S3. Never an ASIC.
100
a housing society
One shared edge server in the society office instead of a hundred cloud subscriptions. Someone must power, cool and look after it.
10,000,000
a manufacturer
Now a custom chip may pay for itself, and many mass-market smart devices do use one.
End of theory. Hand out the scenario cards.
exercise · teams of four · six scenario cards
MCU, GPU, FPGA, ASIC or edge server? One reason each.
12:00
hearing aid
always listening for its owner's voice commands; tiny battery; millions sold
bottle line
camera rejects faulty bottles at 1,000 a minute; a decision within 1 ms, every time
soil sensor pilot
200 units in farm trials; the model will change after the first season
hospital ward
30 cameras watching for falls; video must not leave the building
satellite
radar images processed in orbit; the algorithm will be updated after launch
classroom counter
one prototype that counts students at the door; built this semester
Use the spectrum, the break-even and the four hidden questions. More than one answer can be defended; the reason is what earns the marks.
Circulate. Push teams who pick ASIC for the soil sensor pilot back to the break-even; push teams who pick an MCU for the bottle line to the 1 ms deadline.
click a scenario after teams have reported
These are defensible answers, not the only ones. Give credit for a different choice backed by a good reason, for example an MCU with careful real-time code for the bottle line if the model is tiny.
exercise · worksheet
Four answers per team.
1 · break-even
MCU ₹250/unit; ASIC ₹30/unit after ₹50 crore NRE. How many units?
2 · time
Selling 1 lakh units a month, how many months to break even?
3 · hidden questions
Which of the four would worry you most here, and why?
4 · FPGA
Name one scenario from the cards where an FPGA beats both an MCU and an ASIC.
Answers: ₹50 crore ÷ (250 − 30) = 50,00,00,000 ÷ 220 ≈ 22,72,727 units, about 22.7 lakh; at 1 lakh a month, about 23 months of sales (plus the 1–2 years of design before any sales). Figures are invented for the arithmetic.
exit ticket · on paper, before you leave
For each, pick CPU/MCU, FPGA, ASIC or edge server, and give one reason: (a) 10 million fitness bands; (b) a research radar whose processing changes monthly; (c) 40 CCTV cameras in a mall.
Next class: multi-device architectures, devices and workloads. Putting the whole unit together.
Expected: (a) ASIC, volume and battery life; (b) FPGA, reconfigurable after deployment; (c) edge server, many streams, bandwidth and privacy.