vocabulary

Source board

This definition comes from the board below. Open it to see the term in context.

Tokens and EmbeddingsAI

More terms from this board

• analogy arithmeticking−man+woman≈queen
• BPEbyte-pair encoding (merge frequent pairs → vocab)
• cosine similaritycloseness by angle between vectors
• dense vectorall-numbers representation (vs one-hot)
• dimension dsize of the vector (768…12288)
• embeddinga token's learned vector (d dims)
• embedding matrixthe vocab×d lookup table (learned)
• positional encodingadded so order is known
• semantic searchnearest-vector retrieval (RAG, l.08)
• static vs contextual embeddingfixed per word vs context-shifted
• tokenthe atomic unit a model reads (a subword)
• token IDan integer index
• tokenizertext ↔ token IDs (a fixed pre-step)
• vector databasestores/searches embeddings
• vector spacewhere meaning lives geometrically