Videos / AI

Episode 6 of 22 · AI, end to end

Attention from scratch: the mechanism that ate the field

Chapters

1. One word, two meanings0:00
2. One vector per word0:15
3. The dependency that moves0:31
4. A window cannot work0:48
5. One state, in order1:04
6. The requirement1:22
7. The lost property counter1:40
8. Three projections1:56
9. Query is not key2:14
10. Match on one, pass on another2:32
11. The shapes of it2:49
12. Self, and cross3:06
13. Faders and a mix3:23
14. Step one: the dot3:40
15. Step two: the scale3:56
16. Step three: the softmax4:14
17. Step four: the weighted sum4:32
18. All four, in one line4:49
19. The bill5:07
20. One pen, three jobs5:23
21. Split the width5:39
22. Different patterns5:58
23. Concatenate, then project6:16
24. Eight for the price of one6:34
25. Do not oversell heads6:51
26. A bag, not a line7:09
27. Position, added in7:27
28. Rotate instead7:45
29. Reading the answer8:03
30. The triangle8:20
31. The mask is the difference8:39
32. The other half of it8:56
33. Add the input back9:14
34. Normalise it too9:32
35. The block, in order9:51
36. Stack the block10:10
37. The whole thing10:28

Attention from scratch: the mechanism that ate the field. Read this sentence. The bank was steep and muddy. Now read this one. The bank refused the loan. Same four letters, two unrelated meanings, and nothing inside the word itself told y

Mark as watchedSaved in this browser only.