Videos / AI

Episode 1 of 22 · AI, end to end

How we got here: from the perceptron to the transformer

Chapters

1. Three eras0:00
2. The founding bet0:17
3. A cabinet of rules0:36
4. Expert systems0:52
5. A rule per fact1:11
6. Invert the whole thing1:28
7. One line on a page1:44
8. The arithmetic inside it2:02
9. Nudge and repeat2:18
10. What a line can do2:35
11. XOR breaks it2:52
12. The first winter3:11
13. Blame, flowing backwards3:31
14. The chain rule, applied3:46
15. Invented more than once4:05
16. The layers invent their own4:25
17. Then it stalls again4:44
18. Early, not wrong5:03
19. A leaderboard that will not move5:19
20. The hardware happens to fit5:38
21. Twenty twelve5:58
22. Old ideas, new size6:17
23. Off a cliff6:38
24. Scale becomes a strategy6:57
25. Words as points7:17
26. Directions that mean something7:36
27. A loop that remembers7:55
28. Gates that hold on8:11
29. One vector for a whole sentence8:30
30. Let the decoder look back8:49
31. Delete the loop9:07
32. Query, key and value9:24
33. Position, and many heads9:44
34. Why parallel wins anyway10:03
35. Pretrain, then adapt10:21
36. One block, everything10:40
37. The whole story10:58

How we got here: from the perceptron to the transformer. Artificial intelligence has meant three completely different things. First, rules that a human wrote down. Then weights that a machine learned from examples. And now scale, applied

Mark as watchedSaved in this browser only.