Embeddings

An embedding converts a word, sentence, or document into a list of numbers, typically hundreds to thousands of them. These numbers are not arbitrary; they are learned during training so that texts with similar meanings end up with similar numbers. 'Car' and 'automobile' will have embeddings that are very close together. 'Car' and 'philosophy' will be far apart.

This mathematical structure enables semantic search, where you find content by meaning rather than keyword matching. It is the engine that powers RAG: you embed your documents once, store them in a vector database, and then at query time you embed the user's question and retrieve the documents whose embeddings are closest. The result is retrieval that actually understands what the user is asking rather than just matching words.

Embedding models are separate from the main LLM. OpenAI, Cohere, and open-source projects like Nomic and Sentence Transformers all offer embedding models optimized for this task. Choosing the right embedding model, and keeping it consistent between when you index documents and when you query, is a detail that catches many builders early in building their first RAG pipeline.