Convolutional Networks

The Teaching It To See board draws a photograph as a machine receives it, a grid of pixels 224 × 224 × 3, then cuts it into squares. This lesson is what read that grid before the squares: a small window, slid across it.

Before you start. From What a Neural Network Is: a neuron multiplies each input by a weight, adds the results and a bias, and passes zero if the total is negative; a row of neurons is a layer, and all the weights and biases together are the network’s parameters. From Vectors, Dot Products and Matrix Multiply: the one sum under all of it is “multiply two lists pair by pair, then add”. From How It Learns (loss, gradient descent, backprop): training nudges every weight until the answers come out right, and overfitting is a network that memorises its examples and fails on new ones.

The idea in 30 seconds

Wiring every pixel to every neuron fails on pictures twice: too many weights for the examples, and no sense that a moved shape is the same shape.
A kernel is a small grid of weights slid across an image, multiplying and adding at every position; doing that is a convolution.
The result is a feature map, a grid saying where the kernel’s pattern was found. Move the input and the map moves with it.
Weight sharing, the same few weights at every position, lets a convolutional network (CNN) learn a shape once and find it anywhere, with a tiny fraction of the parameters.
Pooling keeps the largest value in each small block of a feature map, trading exact position for tolerance to small shifts.
Stacked, a CNN’s layers learn edges, then textures, parts and objects. Vision transformers, models that compare every patch of the image with every other, drop the built-in assumption that only nearby pixels matter, and beat CNNs only with far more data.

Read the story · Skip to the numbers