AI
gradient descent
step downhill to reduce loss
Source board
This definition comes from the board below. Open it to see the term in context.
How It LearnsAIMore terms from this board
- automatic differentiationthe framework doing it for you
- backpropagationthe chain rule computing every weight's gradient in one backward pass
- batch / mini-batcha subset processed per step
- convergencethe loss stops dropping
- epochone pass over all data
- generalizationworking on unseen data (the goal)
- gradientslope of the loss w.r.t. the weights
- learning ratestep size
- loss / cost functionhow wrong the prediction is (cross-entropy, MSE)
- optimizer (SGD/Adam)the downhill-stepping algorithm
- overfittingmemorizing train data, failing on new
- parameters/weightswhat's being tuned
- regularizationdropout / weight decay to prevent overfitting
- stepone weight update
- validation set / early stoppingmonitor & stop in time