Videos / AI

Episode 22 of 22 · AI, end to end

Voice, image and video: diffusion and the other modalities

Chapters

1. Three files, one model0:00
2. The same thing is a sequence0:17
3. The machinery does not look back0:33
4. Two ways to cut0:50
5. Meaning becomes a place1:06
6. In is cheap, out is not1:24
7. Cut it into tiles1:42
8. Flatten, project, remember where1:59
9. Every patch, first layer2:17
10. Pull together, push apart2:34
11. Text can point at a picture2:51
12. How a chat model sees3:07
13. The answer starts as static3:24
14. Destroying it on purpose3:41
15. Labels nobody had to write3:58
16. It predicts the noise4:14
17. Run it backwards4:31
18. Sampling, not undoing4:47
19. Something, not your something5:04
20. The words as vectors5:19
21. Cross-attention5:35
22. Run it twice, and lean5:53
23. Faithful or interesting6:10
24. Diffuse somewhere smaller6:28
25. A wiggling line6:45
26. Sound becomes a picture7:01
27. Transcription is translation7:20
28. Nobody labelled where the letters are7:38
29. Back out as a waveform7:56
30. You cannot un-say a syllable8:14
31. Images, plus a promise8:31
32. A flicker reel8:49
33. Time as a dimension9:05
34. Why it costs that much9:23
35. Three bargains9:40
36. Drift, not detail10:00
37. The whole thing10:16

Voice, image and video: diffusion and the other modalities. A photograph, a voice note, and a paragraph of text. Three completely different files. Three completely different shapes on disk. And one model answers questions about all three of

Mark as watchedSaved in this browser only.