Lai Learns Neural Networks

A course in 30 lessons

Neural Networks

From a chessboard to the stars.

All 30 lessons written

Why I’m learning this

I’m curious about how neural networks work. Not just what they can do, but what is actually going on inside them. So I’m starting with the basics and going slowly.

Every lesson is anchored on a theme: a chessboard, a termite mound, a morning shower. Something familiar to hang each new idea on.

Begin with lesson 1 →

Contents

  1. 01ChessOf the Thinking Engine, and How It Learneth Its GameWhat a neural network is: a machine that learns patterns from examples instead of following written rules
  2. 02TermitesOf the Humble Neuron, Small of Wit Yet Mighty in MultitudeThe artificial neuron: one tiny, simple unit; intelligence comes from many working together
  3. 03SoccerWherein Each Voice Is Weighed, and Some Weigh MoreInputs and weights: how a neuron decides whose signal to trust most
  4. 04MusicHow the Note Is Struck, and When It Sounds NotBias and the activation function: the threshold that decides whether a neuron “fires”
  5. 05AviationOf Layers Upon Layers, from Runway Unto the CloudsLayers and the forward pass: how signals travel from input, through hidden layers, to an answer
  6. 06ArcheryOf the Arrow’s Errand, and How Far It StrayedThe loss function: measuring how wrong a guess was
  7. 07Hiking in fogOf the Wanderer Who Findeth the Valley by Feel AloneGradient descent: finding better answers by always stepping downhill
  8. 08The morning showerTurn Not the Tap Too Boldly, Lest Thou Be ScaldedThe learning rate: how big each correcting step should be
  9. 09Detective workOf Blame Traced Backward Through Every DoorBackpropagation: working out which weights caused the mistake
  10. 10DominoesHow One Small Push Doth Topple the Whole ProcessionThe chain rule: how a change ripples through many layers
  11. 11Marathon trainingOf Laps Run, and Laps Run AgainThe training loop and epochs: why networks learn by repetition
  12. 12The classroomOf the Scholar Who Must Not Glimpse the ExaminationTraining, validation and test data: keeping the final exam honest
  13. 13TailoringOf a Coat Cut So Close It Fitteth No Other ManOverfitting: when a network memorises instead of learning
  14. 14TheatreShould the Player Fall Ill, Let the Understudy RiseRegularisation and dropout: making a network sturdy by practising without some of its parts
  15. 15BakingOf Loaves Baked by the Trayful, Not the CrumbBatches and stochastic gradient descent: learning from small handfuls of examples
  16. 16The whisper gameOf a Message Whispered Down a Long and Weary LineVanishing and exploding gradients: why very deep networks once struggled to learn
  17. 17GardeningHow the Seed Be Sown Determineth the HarvestWeight initialisation: why where you start matters
  18. 18CyclingOf the Rider Who Keepeth His Speed Through the HollowsOptimisers (momentum, Adam): smarter ways of stepping downhill
  19. 19PhotographyOf Light Made Even, That No Face Be Lost in ShadowNormalisation: keeping signals on a steady scale
  20. 20The post officeOf Letters Sorted Each Unto Its Proper BagClassification and softmax: turning scores into confident choices
  21. 21Map-makingWherein Words Are Given a Place Upon the MapEmbeddings: representing words and things as points in space
  22. 22BirdwatchingOf the Watcher Who Scanneth the Hedgerow Patch by PatchConvolutional networks: spotting patterns in images
  23. 23ArchitectureFrom Brick to Wall, from Wall to CathedralFeature hierarchies and pooling: building big ideas out of small ones
  24. 24Campfire storiesOf the Tale That Remembereth What Came BeforeRecurrent networks: handling sequences and memory
  25. 25The libraryOf What the Keeper Shelveth, and What She Casteth OutLSTMs and gates: choosing what to remember and what to forget
  26. 26The dinner partyTo Whom Shall the Host Lend Her Ear?Attention: focusing on what matters most in the moment
  27. 27The newsroomOf Many Desks Reading All Pages at OnceTransformers: the architecture behind modern language models
  28. 28Moving abroadOf the Traveller Who Carrieth His Old Skills to Foreign ShoresTransfer learning and fine-tuning: reusing what a network already knows
  29. 29The novelist’s deskOf the Pen That Guesseth Ever the Next WordGenerative models: how networks write, draw and create
  30. 30StargazingOf All the Stars Now Joined in One ConstellationBringing it all together: what networks can and cannot do, and where to go next