Vector Database

A vector database is the storage layer that makes semantic search possible. Traditional databases index rows by exact values and string matches. A vector database indexes rows by their position in a high-dimensional embedding space, then answers queries with the rows whose vectors sit closest to the query vector. Closest is measured with cosine similarity or dot product. The result is a system that finds documents by meaning rather than by keyword. A user asks about "reducing churn" and the database returns chunks that talk about retention, win-back, and account expansion, even when none of those exact words appear in the query.

Vector databases sit at the core of every production RAG system. The flow looks the same across deployments. Documents get chunked, each chunk gets converted to an embedding by a model like text-embedding-3-large or bge-large, and the embedding gets written to the vector database along with metadata pointing back to the source. At query time the user question gets embedded with the same model, the database returns the top-k most similar chunks, and those chunks get passed to the language model as grounded context. Common production choices include Pinecone, Weaviate, Qdrant, Milvus, and pgvector inside Postgres. Choice depends on document volume, query throughput, and whether you need self-hosted infrastructure.

For sensitive workloads the vector database stays inside your perimeter. Healthcare clients run pgvector on their own Postgres or self-hosted Qdrant on-prem so PHI never enters a third-party SaaS. Funded teams running the AI Ops Department and AI Support Department get the vector database deployed against their knowledge base as part of standard delivery. The database is not the hard part. The hard part is choosing chunk sizes, embedding models, and reranking strategies that produce retrieval quality good enough for production traffic instead of demo-grade results.