AI
embedding
a token's learned vector (d dims)
Source board
This definition comes from the board below. Open it to see the term in context.
Tokens and EmbeddingsAIAlso defined on RAG.
More terms from this board
- analogy arithmeticking−man+woman≈queen
- BPEbyte-pair encoding (merge frequent pairs → vocab)
- cosine similaritycloseness by angle between vectors
- dense vectorall-numbers representation (vs one-hot)
- dimension dsize of the vector (768…12288)
- embedding matrixthe vocab×d lookup table (learned)
- positional encodingadded so order is known
- semantic searchnearest-vector retrieval (RAG, l.08)
- static vs contextual embeddingfixed per word vs context-shifted
- tokenthe atomic unit a model reads (a subword)
- token IDan integer index
- tokenizertext ↔ token IDs (a fixed pre-step)
- vector databasestores/searches embeddings
- vector spacewhere meaning lives geometrically
- vocabularythe ~50-100k known tokens