Vector Databases — Recall, Latency & Scale

Topics We'll Cover

• The Vector DB Landscape
• ANN Algorithms — IVF, HNSW, PQ, Scalar & Binary Quantisation
• HNSW Deep Dive — Layers, ef_construction, ef_search, M
• IVF + PQ — Compression for 100M+ Vectors
• Filtered Search — Pre/Post-Filter, Metadata Indexing
• Updates, Deletes & Snapshots
• pgvector in Depth — IVFFLAT vs HNSW on Postgres
• Choosing a Vector DB — Decision Matrix
• What to Take Away

The Vector DB Landscape

Vector databases are not a single category. They differ fundamentally on four axes: storage architecture, indexing algorithm, filter capability, and operational model. Before choosing, classify your workload:

ANN Algorithms — IVF, HNSW, PQ, Scalar & Binary Quantisation

All ANN algorithms trade recall (fraction of true nearest neighbours returned) against latency and memory. The tradeoff surface has a characteristic knee; good production systems sit just above the knee.

HNSW Deep Dive — Layers, ef_construction, ef_search, M

HNSW (Hierarchical Navigable Small World, Malkov & Yashunin 2016) is a layered proximity graph. Understanding its three key parameters is essential for production tuning.

IVF + PQ — Compression for 100M+ Vectors

Inverted File Index (IVF) partitions the vector space into Voronoi cells via k-means. Product Quantisation (PQ, Jégou et al. 2011) compresses each vector by splitting it into M sub-vectors and quantising each sub-vector against a codebook of 256 centroids, yielding M bytes per vector.

Filtered Search — Pre/Post-Filter, Metadata Indexing

Most production RAG queries include at least one metadata filter: user_id, doc_type, date_range, tenant_id. The strategy for applying these filters determines recall and latency far more than the ANN algorithm choice.

from qdrant_client import QdrantClient, models

client = QdrantClient("localhost", port=6333)

results = client.query_points(
    collection_name="docs",
    query=query_vector,          # 1024-dim float list
    query_filter=models.Filter(
        must=[
            models.FieldCondition(
                key="tenant_id",
                match=models.MatchValue(value="acme-corp"),
            ),
            models.FieldCondition(
                key="doc_date",
                range=models.Range(gte=1704067200),  # 2024-01-01 Unix
            ),
        ]
    ),
    limit=20,
    with_payload=True,
)

Updates, Deletes & Snapshots

Most vector DBs are append-optimised. Understanding the mutation model prevents data-loss incidents and painful re-index operations.

pgvector in Depth — IVFFLAT vs HNSW on Postgres

pgvector (v0.7, 2024) added HNSW support, making it a serious option for <5 M vector workloads already running Postgres. The two index types behave very differently under load.

-- Extension + table
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE documents (
    id       bigserial PRIMARY KEY,
    content  text,
    metadata jsonb,
    embedding vector(768)
);

-- Build HNSW index after bulk load
SET maintenance_work_mem = '4GB';
CREATE INDEX docs_embedding_hnsw
    ON documents
    USING hnsw (embedding vector_cosine_ops)
    WITH (m=16, ef_construction=128);

-- Query: top-10 nearest with metadata filter
SET hnsw.ef_search = 80;
SELECT id, content, 1 - (embedding <=> '[...]'::vector) AS score
FROM  documents
WHERE metadata->>'tenant' = 'acme'
ORDER BY embedding <=> '[...]'::vector
LIMIT 10;

Choosing a Vector DB — Decision Matrix

There is no universally best choice. The right system depends on scale, latency SLA, filter complexity, and what you already run in production.

What to Take Away

• HNSW dominates query latency at the cost of slow build and large memory. IVF+PQ dominates memory at the cost of recall and build complexity.
• Key HNSW parameters: M controls connectivity and memory; ef_construction controls build quality; ef_search is the query-time recall knob.
• Binary quantisation + rescoring is the best recall-per-byte deal for >10 M vector collections (Qdrant's implementation is production-ready).
• Pre-filter vs hybrid-filter: never naive post-filter for high-selectivity predicates. Use Qdrant's payload indexes or pgvector's SQL WHERE clause.
• Deletions degrade HNSW recall silently. Monitor tombstone fraction; trigger compaction before it exceeds 20%.
• pgvector is sufficient for <5 M vectors if you already run Postgres. HNSW with ef_search=80, M=16, ef_construction=128 is a solid starting configuration.
• Milvus for >100 M; Qdrant for 1 M–100 M with complex filters; LanceDB for serverless/S3 analytics workloads.