Vector Databases

Retrieval needs the $k$ nearest vectors to a query. Exactly, that is a dot product against every vector in the corpus — fine at ten thousand, slow at ten million, and the entire reason this category of software exists.

HNSW, which is what they all run

Hierarchical Navigable Small World graphs are the index behind almost every current system. Vectors become nodes in a layered proximity graph: sparse long edges at the top for coarse navigation, dense short edges at the bottom for precision. A search enters at the top, greedily walks towards the query, and descends.

Logarithmic in corpus size, high recall, and expensive to build — roughly $O(n \log n)$ with a large constant, and the graph must be in memory.

$$M = \text{edges per node}, \qquad \mathit{ef} = \text{candidate list size}$$

$M$ around 16 and ef_construction around 200 are reasonable defaults. At query time ef_search sets how hard to look — raise it for recall, lower it for latency. It is the only one of the three you can change after building.

Whether you need one

Vectors	Reasonable choice
< 100 k	numpy, brute force
< 10 M	pgvector, or SQLite + an index
> 10 M	Qdrant, Milvus, a managed service
research / offline	FAISS

The first row is not a joke. A brute-force scan of 100 000 × 768 floats is a single matmul and takes a few milliseconds — faster than a network round trip to a dedicated service, exact rather than approximate, and with no system to operate.

The landscape

FAISS — Meta’s library, not a database. Every index type, GPU support, excellent for offline and research work. You handle persistence and serving.

pgvector — Postgres extension. Transactions, joins, backups, permissions: everything a database already does, plus HNSW.

Qdrant, Milvus, Weaviate — purpose-built, with filtering, sharding and replication. Worth the operational cost above a few tens of millions of vectors.

Pinecone, and managed offerings — the same, without running it yourself.

The improvement most worth making. Dense embeddings capture meaning and miss exact tokens — a part number, an error code, a surname. BM25 does the reverse. Run both and fuse the rankings:

$$\mathrm{RRF}(d) = \sum_{r \in \text{rankers}} \frac{1}{k + \mathrm{rank}_r(d)}, \qquad k \approx 60$$

RRF combines ranks rather than scores, so the two systems’ incomparable similarity numbers never have to be reconciled. It is a few lines of code and reliably beats either retriever alone.