Complete series · 10 chapters · reads like a book, start at part one
How did machines learn to read? Not with transformers. Long before attention, there was a small, honest chain of ideas: telegraph codes, a matrix of counts, a dot product, softmax, and one absurdly clean gradient. These ten chapters build the whole chain from scratch, with interactive toys to poke at in every one. If you've ever nodded along to the word "embeddings" without quite believing yourself, start at chapter one.
01 Text was a problem long before AIhow text got into machines at all read 02 From characters to conceptswhy 'cat' and 'cap' look nearly identical to a computer read 03 From words to spacesthe matrix that notices dog and cat are twins read 04 From counting to learningevery word wears two hats read 05 What the dot product really measuresdog and cat never touch, yet end up neighbours read 06 Why raw dot products aren't enoughhow scores learn to compete read 07 Two distributions, one goalthe corpus is the distribution read 08 Why negative log P?the only penalty with the right shape read 09 Creating the learning signalthe entire gradient is p minus q read 10 From signal to geometrywatch the vectors find each other read