Vector Database
A vector database stores embeddings and finds the closest ones by meaning, fast. It is the technical basis of RAG: it answers the question of which fragments to place into the model's context.
How it works
An ordinary database looks for exact matches: a string, a number, a range. A vector database looks for nearest neighbours in a space of hundreds of dimensions, meaning records similar in meaning rather than in spelling.
Exhaustive comparison across millions of records is too slow, so approximate search is used: it gives up a small share of accuracy for an answer in milliseconds. That trade-off is configurable.
A dedicated database is not always necessary. At modest volumes, extensions to ordinary relational systems cope, and a separate one earns its place when fragments run into the hundreds of thousands.
What decides quality
- How documents are chunkedChunks too small lose context, chunks too large blur retrieval. This is the main knob in the design.
- One embedding modelIndexing and querying must use the same model. Change it and the store has to be rebuilt.
- Metadata filtersA date, an author or a section stored alongside the vector narrows retrieval to what is current instead of the whole archive.
- Hybrid search usually winsCombining vector search with plain keyword search catches both meaning and exact terms like part numbers.