Vector Search skill for Claude Code

A vector index is tuned when you have measured its recall@k against exact (brute-force) search on a sample of real queries, with the filters production uses, and latency and memory fit the budget. ANN parameters copied from a blog post without that measurement are a guess.

Ground rules

• Pick the store by operational fit first: if Postgres is already the system of record and the corpus is under ~10-50M vectors, start with pgvector.
• One embedding model per index. Record the model name and version alongside the vectors; changing models means a full re-embed.
• Normalize vectors when using cosine similarity, and use the metric the embedding model was trained for (most text models: cosine or dot product on normalized vectors).
• Always measure ANN recall against exact search with production filters applied.
• Store the source text or a pointer next to each vector so results are explainable.

Workflow

- [ ] 1. Size the problem (vector count, dimensions, QPS, filter selectivity, update rate)
- [ ] 2. Choose embedding model and dimension
- [ ] 3. Choose the store
- [ ] 4. Create schema and index with explicit parameters
- [ ] 5. Build an exact-search ground truth for 200-1000 sample queries
- [ ] 6. Tune search-time parameters to the recall/latency target
- [ ] 7. Load test with filters and concurrent writes

1. Size it

Raw memory = vectors x dimensions x bytes per component. 10M x 1024 x 4 bytes (float32) = ~41 GB before index overhead (HNSW adds roughly 1.1-1.5x depending on m). Halving precision or dimensions halves that.

2. Embedding model and dimension

Need	Choice
General English/multilingual text, hosted	A current hosted embedding model (e.g. Voyage, OpenAI, Cohere, Google); check the MTEB leaderboard for your language and domain
Self-hosted / data cannot leave	Open models served locally (BGE, E5, Nomic, GTE families) via sentence-transformers or a TEI server
Code search	A code-specialized embedding model
Cost/memory pressure	Models with Matryoshka support: truncate to 256-512 dims and re-normalize; verify recall loss

Anthropic does not provide its own embeddings endpoint; pair Claude with any embedding provider. Benchmark 2-3 models on your own labeled queries; leaderboard rank does not transfer reliably to a specific domain.

3. Choose the store

Situation	Store
Already on Postgres, need joins/transactions with app data, < ~50M vectors	pgvector
Heavy filtered search, high QPS, self-hosted or managed, rich quantization	Qdrant
Zero-ops managed service, serverless scaling, namespaces per tenant	Pinecone
Existing OpenSearch/Elasticsearch deployment with BM25	Its kNN/vector field, keep hybrid in one engine

4. Schema and index

pgvector (0.8+):

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunks (
  id          bigserial PRIMARY KEY,
  tenant_id   bigint NOT NULL,
  doc_id      bigint NOT NULL,
  content     text NOT NULL,
  embedding   vector(1024) NOT NULL,     -- halfvec(1024) halves memory
  embed_model text NOT NULL,
  updated_at  timestamptz NOT NULL DEFAULT now()
);

-- Build after bulk load; raise memory for the build session.
SET maintenance_work_mem = '8GB';
SET max_parallel_maintenance_workers = 7;
CREATE INDEX CONCURRENTLY chunks_embedding_hnsw
  ON chunks USING hnsw (embedding vector_cosine_ops) WITH (m = 16, ef_construction = 64);
CREATE INDEX CONCURRENTLY chunks_tenant ON chunks (tenant_id);

-- Query: operator must match the index opclass (<=> cosine, <-> L2, <#> negative inner product)
SET hnsw.ef_search = 100;                -- per session or SET LOCAL in a transaction
SET hnsw.iterative_scan = relaxed_order; -- 0.8+: keep scanning when filters discard results
SELECT id, content, embedding <=> $1 AS distance
FROM chunks
WHERE tenant_id = $2
ORDER BY embedding <=> $1
LIMIT 10;

pgvector notes:

• vector indexes support up to 2,000 dimensions; halfvec up to 4,000. Above that, reduce dimensions or use binary quantization with re-ranking.
• HNSW: better recall/latency, slower build, more memory. IVFFlat: faster build, needs data present at build time (lists ~ rows/1000 up to 1M rows, sqrt(rows) above) and ivfflat.probes tuning; rebuild after large distribution shifts.
• Check the plan: EXPLAIN ANALYZE must show an index scan on the HNSW index. A missing ORDER BY ... LIMIT or a mismatched operator silently falls back to a sequential scan.
• Very selective filters (e.g. one tenant of thousands) often do better with a partial index per large tenant, or a B-tree filter first and exact distance on the small set.

Qdrant (Python client):

from qdrant_client import QdrantClient, models

qc = QdrantClient(url="http://localhost:6333")
qc.create_collection(
    collection_name="chunks",
    vectors_config=models.VectorParams(size=1024, distance=models.Distance.COSINE),
    hnsw_config=models.HnswConfigDiff(m=16, ef_construct=100),
    quantization_config=models.ScalarQuantization(
        scalar=models.ScalarQuantizationConfig(type=models.ScalarType.INT8, always_ram=True)
    ),
)
qc.create_payload_index("chunks", field_name="tenant_id", field_schema="keyword")

hits = qc.query_points(
    collection_name="chunks",
    query=query_vec,
    query_filter=models.Filter(must=[models.FieldCondition(key="tenant_id", match=models.MatchValue(value="t42"))]),
    search_params=models.SearchParams(hnsw_ef=128),
    limit=10,
).points

Create payload indexes for every filtered field before loading data; Qdrant uses them to plan filtered HNSW traversal.

Pinecone:

from pinecone import Pinecone

pc = Pinecone()                      # reads PINECONE_API_KEY
index = pc.Index("chunks")
index.upsert(vectors=[{"id": "doc42#3", "values": vec, "metadata": {"doc_id": 42, "lang": "en"}}],
             namespace="tenant-42")
res = index.query(vector=qvec, top_k=10, namespace="tenant-42",
                  filter={"lang": {"$eq": "en"}}, include_metadata=True)

Use one namespace per tenant for hard isolation and cheaper filtered queries; keep metadata small (IDs and filter fields, not full text) to stay within per-record limits.

5. Ground truth

Run the sample queries with exact search to get true top-k:

• pgvector: SET enable_indexscan = off; in a session, or query a copy without the index.
• Qdrant: search_params=models.SearchParams(exact=True).
• Pinecone: compute exact top-k offline with numpy over an exported sample.

recall@k = |ANN top-k intersect exact top-k| / k, averaged over queries. Target 0.95+ for RAG retrieval unless latency forces lower.

6. Tune

Knob	Effect	Typical range
ef_search / hnsw_ef	Higher = better recall, slower queries	40-400; must be >= k
m	Graph degree; higher = better recall, more memory, slower build	12-48
ef_construction	Build quality; higher = slower build	64-200
Quantization (int8 / binary)	4x / 32x less memory; recall drop recovered by rescoring with full vectors	Enable rescoring/oversampling
Dimensions (Matryoshka truncation)	Linear memory and speed gain	Measure recall per size

Sweep ef_search in steps (40, 80, 160, 320), plot recall vs p95 latency, and pick the knee. Rebuild with higher m only if max ef_search cannot reach the target.

7. Load test

Run concurrent queries with realistic filters while ingesting at the expected write rate. Watch p95/p99 latency, memory, and (Postgres) autovacuum on the vector table; HNSW indexes bloat under heavy update/delete churn and need periodic REINDEX CONCURRENTLY.

Gotchas

• Post-filtering: ANN returns top-k, then filters remove most of them, leaving 0-2 results. Use filter-aware search (Qdrant payload indexes, pgvector iterative scans, Pinecone metadata filters) and test the most selective filter.
• Mixing vectors from two embedding models or versions in one index yields garbage similarity scores without any error.
• Unnormalized vectors with inner product rank long documents higher; normalize or use cosine.
• Similarity scores are not calibrated probabilities. Do not hard-code a global threshold without checking its precision on labeled data.
• Building HNSW before bulk load is far slower than loading first and indexing after.
• Deletes in HNSW are often tombstones; heavy churn degrades recall until rebuild or optimization runs.

Output contract

**Store:** <pgvector | Qdrant | Pinecone> — chosen because <reason>
**Embeddings:** <model>, <dims>, <metric>, normalized: <yes/no>
**Index:** <HNSW m=, ef_construction= | IVFFlat lists=>, quantization <type + rescoring>
**Search:** ef_search=<n>, filters <fields + indexes>
**Measured:** recall@10 <x> vs exact on <n> queries; p95 <ms> at <QPS>; memory <GB>