Tensor TownISSUE #99 of 120

vector dbs · pinecone · weaviate · chroma

NeuraVSThe Overfit Ogre

Neura saysVector databases (Pinecone, Weaviate, Chroma) store embeddings and search them fast.

Searching millions of vectors by brute force is slow. Vector databases use approximate-nearest-neighbor (ANN) indexes (HNSW, IVF) to find similar vectors in milliseconds, with metadata filtering and scaling built in. Managed (Pinecone), self-hostable (Weaviate, Qdrant), and embedded (Chroma) options exist. They're the storage/retrieval backbone of RAG and semantic search — pick by scale, filtering needs, and ops appetite.

Power-ups you unlock

Store + fast-search embeddings (ANN)
HNSW/IVF indexes → millisecond search
Metadata filtering + scaling
Backbone of RAG and semantic search

The Overfit Ogre attacks — common mistakes

Brute-force search at scale (too slow)
Ignoring metadata filtering needs
Choosing a DB without considering ops/scale

Boss battleExplain why ANN indexing beats exact search at million-vector scale.

Example code

<!doctype html><html><head><meta charset="utf-8"></head>
<body style="background:#06040d;color:#e6e0ff;font-family:monospace;padding:20px"><pre>1M vectors: brute force = slow
ANN index (HNSW) → nearest in milliseconds
+ metadata filters → production RAG</pre></body></html>

▶ Open the interactive comic issue

‹ Embeddings Models · Openai · Cohere · Bge Mlops · Pipelines · Monitoring · Drift ›