A vector database stores high-dimensional numerical vectors and retrieves the most similar ones to a query vector in milliseconds. It is the storage layer that makes semantic search, RAG, and recommendation systems practical at scale - the piece that connects embedding models to the LLM context window.
What vector databases do
Traditional databases search for exact matches or range conditions on structured fields. A vector database searches for approximate similarity in a high-dimensional geometric space. Each document, image, or chunk of text is represented as a vector of floating-point numbers (an embedding), and the database indexes those vectors so it can answer "find the K vectors most similar to this query vector" in sub-linear time.
The naive approach - computing the distance from the query to every stored vector - is exact but O(n) per query. At 10 million documents this takes seconds, not milliseconds. Vector databases use approximate nearest-neighbour (ANN) algorithms that trade a small amount of recall for orders-of-magnitude faster search.
Similarity metrics
"Similar" has a precise mathematical definition that depends on which metric you choose. The metric must be configured at index creation time and affects both index quality and query results.
| Metric | Formula (simplified) | When to use |
|---|---|---|
| Cosine similarity | dot(A, B) / (|A| * |B|) → range [-1, 1] | Default for text embeddings. Measures angle, not magnitude - normalises for document length. |
| Dot product (inner product) | sum(Aᵢ * Bᵢ) → unbounded | Faster than cosine when vectors are pre-normalised (unit length). Used in many production systems. |
| Euclidean distance (L2) | sqrt(sum((Aᵢ - Bᵢ)²)) → [0, ∞) | Image embeddings and some structured embeddings. Sensitive to magnitude. |
ANN search
Approximate nearest-neighbour (ANN) search returns the K most similar vectors without scanning the entire dataset. The key parameter is the tradeoff between recall (how often the true nearest neighbour is in the result) and latency (how long the search takes). Most production systems target 95-99% recall at less than 50 ms p99 latency.
Two families of ANN algorithms dominate production systems: graph-based (HNSW) and quantisation-plus-partition-based (IVF + PQ). They have fundamentally different memory and latency profiles.
HNSW and IVF
HNSW (Hierarchical Navigable Small World) builds a multi-layer graph where each layer is a subset of the full index. At query time the search starts at the top (coarse, few nodes), greedily navigates to the approximate region of interest, then descends to the bottom layer for precise neighbourhood search. Think of it as a geographic zoom: country level, then city, then street.
IVF (Inverted File Index) partitions the vector space into Voronoi cells (clusters), stores an inverted list of vectors per cell, and at query time only searches the most relevant cells. IVF is typically combined with product quantisation (PQ) to compress vectors and reduce memory footprint dramatically.
| Algorithm | Memory | Build time | Query speed | Recall | Best for |
|---|---|---|---|---|---|
| HNSW | High (full vectors in RAM) | Slower (graph build) | Very fast | Very high | Default choice; low latency, high recall |
| IVF + PQ | Low (compressed vectors) | Faster (cluster partitioning) | Fast | Good (tunable) | Billion-scale indexes where RAM is the constraint |
| ScaNN (Google) | Medium | Medium | Very fast | Very high | Large-scale GCP deployments |
HNSW is the default in almost every managed vector database (Pinecone, Weaviate, Qdrant, Milvus). pgvector supports both HNSW and IVF-Flat. For most RAG applications you will never need to tune the algorithm - the defaults are well-calibrated.
pgvector vs dedicated vector DBs
This is the most common architectural decision teams face when adding vector search. pgvector is a Postgres extension that adds a vector column type and ANN index support. Dedicated vector databases are purpose-built systems with vector-first architecture.
Metadata filtering
Pure vector search returns the globally nearest vectors. In practice you almost always want to filter by metadata first: "find the most similar documentsowned by this user" or "find the most similar chunksfrom this product category".
There are two approaches, and choosing the wrong one degrades recall severely:
- Pre-filter (filter then search) - apply the metadata filter first to reduce the candidate set, then run ANN search within it. Fast when the filter removes most documents. Recall collapses when the filter returns a small subset, because ANN was built on the full index.
- Post-filter (search then filter) - run ANN to get top-1000, then apply the filter. Recall is high, but you may need a large over-fetch if the filter is selective, increasing latency.
Production vector databases (Weaviate, Qdrant, Pinecone) use a hybrid approach - a payload index for metadata combined with ACORN (Weaviate) or filtered graph traversal (Qdrant) - to maintain recall across all filter selectivities. Understand which strategy your chosen database uses before designing your schema.
Comparing your options
| Database | Hosting | Index | Hybrid search | Filtering | Notes |
|---|---|---|---|---|---|
| pgvector | Self-hosted (Postgres) | HNSW, IVF-Flat | Manual (combine with pg_bm25 / ParadeDB) | SQL WHERE clause | Best for teams already on Postgres; no extra infra |
| Pinecone | Managed SaaS | Proprietary (HNSW-based) | Yes (built-in sparse + dense) | Metadata filters | Zero ops; serverless tier; strong enterprise SLAs |
| Weaviate | Self-hosted or managed | HNSW | Yes (BM25 + vector hybrid, ACORN filtering) | GraphQL + metadata index | Open-source; rich module ecosystem; good multimodal support |
| Qdrant | Self-hosted or managed | HNSW | Yes (sparse vectors + dense) | Payload index + filtered HNSW | High-performance Rust core; excellent filtering recall |
| Milvus | Self-hosted or managed (Zilliz) | HNSW, IVF+PQ, DiskANN | Yes | Scalar index | Battle-tested at billion-scale; most algorithm choices |
For how the vectors stored in these databases are produced, see Embeddings Explained. For how vector databases plug into a full retrieval pipeline, see RAG Explained.
Sources & further reading
- 1Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs — Malkov & Yashunin, 2018 - arXiv 1603.09320
- 2pgvector - Open-source vector similarity search for Postgres — pgvector on GitHub
- 3Pinecone Learn - Vector Database Guide — Pinecone
- 4Weaviate Vector Database Documentation — Weaviate