Vector Databases for RAG Compared: pgvector, Qdrant, Chroma, Weaviate

Vector stores are the filing cabinet behind RAG. They index embedding vectors, return nearest neighbors, and attach metadata filters so agents do not stuff irrelevant chunks into the prompt. The right choice is mostly an ops and query-shape decision, not a leaderboard score.

TL;DR comparison

pgvector: vectors inside Postgres

Vector database comparison visualized as indexed clusters on storage shelves

pgvector adds a vector type and ANN indexes (HNSW, IVFFlat) to PostgreSQL. Your application already uses SQL transactions, row-level security, and backups — vectors become another column beside user IDs and billing fields.

Strengths: one backup stream, JOIN embeddings with relational data, mature tooling, easy compliance story for teams that must not export rows to a SaaS index.

Tradeoffs: very large billion-vector tiers need careful partitioning; heavy hybrid search may push you toward dedicated engines or Postgres extensions plus full-text search maintained by your DBAs.

Use pgvector when your embedding model guide points at a modest corpus (docs, tickets, internal wiki) and latency targets sit in tens of milliseconds on tuned HNSW.

Qdrant: filter-first vector search

Qdrant is an open-source engine built around collections, payloads, and rich filtering. It targets production similarity search where metadata (tenant_id, doc_class, timestamp) is as important as cosine distance.

Strengths: expressive JSON filters, horizontal scaling story, gRPC performance, optional cloud control plane.

Tradeoffs: another system to secure and monitor; relational reports still need ETL back to a warehouse.

Pair Qdrant with our reranker pipeline guide when top-50 ANN hits are too noisy for coding agents.

Chroma: developer speed, migrate later

Chroma optimizes the first afternoon: pip install chromadb, embed, query. Local persistent mode suits notebooks and demos.

Strengths: frictionless Python ergonomics, tight Hugging Face tutorial compatibility, fast hackathon loops.

Tradeoffs: enterprise RBAC, cross-region DR, and strict multi-tenant isolation require planning — note industry moves such as Chroma joining Replit and winding down Chroma Cloud; self-hosted ChromaDB remains Apache 2.0 but operators should read migration guides before relying on deprecated cloud paths.

Start on Chroma, export collections to Qdrant or pgvector when SLAs harden.

Weaviate: modules and hybrid retrieval

Hybrid search for RAG shown as lexical and vector signals merging in one lens

Weaviate packages vector search with optional sparse modules for keyword components, generative modules, and multi-tenancy patterns. Teams chasing hybrid RAG (dense + lexical) without gluing two products sometimes standardize here.

Strengths: schema-first classes, hybrid queries, modular inference connectors.

Tradeoffs: operational surface area; GraphQL learning curve for engineers used to SQL only.

Compare mentally with RAG vs agentic RAG patterns — graph-assisted retrieval may still sit above whichever store you pick.

Index types builders actually tune

Approximate nearest neighbor (ANN) indexes trade recall for speed. HNSW graphs are the default choice on pgvector and Qdrant for many 2026 deployments: fast queries, RAM hungry builds. IVFFlat clusters vectors first — cheaper builds, more recall tuning via lists and probes on pgvector. Weaviate exposes multiple vector index configs per class; read their capacity planning page before picking replication factors.

Dimensionality matters as much as algorithm. A 3072-dim embedding from a frontier model doubles storage versus 1536-dim models. Our Perplexity embed v2 posts show how contextual embeddings change chunking — your index stores more vectors per doc when each chunk gets multiple contextualized vectors.

Reindex playbook: freeze writes, snapshot old collection, embed with the new model in a shadow table, run offline recall tests against labeled Q&A, cut traffic over, delete legacy index. Skipping offline eval is how teams ship silent retrieval regressions.

Agent and MCP access patterns

Agents should not receive superuser database credentials. Pattern that works:

1. Expose search_docs(query, k, filters) on an MCP server backed by the vector store SDK.
2. Return chunk text plus citation metadata only — no arbitrary SQL.
3. Log queries for abuse monitoring; rate-limit per user.

Long-horizon agent memory sometimes duplicates vector stores (episodic memory product) beside RAG corpora (static knowledge). Keep collections separate so a poisoned chat memory cannot overwrite handbook embeddings.

For AWS-centric teams, Bedrock Knowledge Bases hide index choice behind managed OpenSearch or vector backends — compare managed cost against self-hosted Qdrant if you already operate Kubernetes.

What people ask before they commit

Can one store serve realtime and batch? Yes with separate collections or partitions; do not mix interactive agent queries with nightly bulk reindex on the same HNSW without maintenance windows.

Do I still need Elasticsearch? Only if lexical features or log analytics already live there. Hybrid-heavy news or legal search often keeps Elasticsearch or OpenSearch while vectors sit in Weaviate or Qdrant. PageIndex-style retrieval (RAG vs agentic RAG) may bypass vectors entirely for code repos — do not buy a vector DB before you validate retrieval mode.

How does this interact with inference cost? Retrieval cheapens LLM bills by shrinking prompts, but bad recall increases retry loops — more tokens than a smaller, accurate context. Pair store choice with rerankers.

Is pgvector “enough” for coding agents? Often for single-tenant internal tools under a few million chunks. Multi-tenant SaaS with per-org ACLs and aggressive metadata filters frequently outgrows naive SQL filters — benchmark before launch.

Migration paths that do not trap you

Export scripts should preserve stable chunk IDs so you can diff recall before cutover. Store embedding model name and dimension in metadata always.

Security and compliance notes

Vector rows may contain PII pasted into tickets or HR PDFs. Encrypt at rest, restrict backup access, and apply tenant filters in the query path — not only in the UI. Universal geometry embeddings research is a reminder that “private” vectors can still leak semantic neighbors if ACL bugs exist.

For regulated teams, pgvector keeps data residency inside an existing Postgres compliance boundary. SaaS vector clouds require DPAs and key management review like any other subprocessors.

How to decide in one pass

1. Where do authoritative rows live? If Postgres is already source of truth, try pgvector first.
2. How nasty are filters? Multi-tenant SaaS with compound filters often lands on Qdrant or Weaviate.
3. How fast must you ship v0? Chroma or embedded stores, with a written migration trigger (vector count, QPS, compliance).
4. Do agents need MCP tools? Wrap queries behind a narrow MCP server per our MCP security guide instead of handing agents raw SQL.
5. Measure end-to-end. Index tuning beats database religion; log recall@k before swapping engines.

Latency, cost, and hosted tiers

Self-hosting wins on per-query marginal cost once your team already runs Kubernetes or Postgres. Managed SaaS (Qdrant Cloud, Weaviate Cloud, various “vector inside Aurora” offerings) wins when you pay for compliance, patching, and on-call instead of metal.

Latency budgets split into ANN query time, reranker milliseconds, and LLM prefill once chunks return. A store that saves 5 ms on ANN but forces cross-AZ traffic may lose to colocated pgvector beside your API pods. Measure p95 retrieval-to-first-token, not isolated ANN benchmarks.

Storage math for planners: points × dimensions × 4 bytes for fp32 embeddings (often less with quantization on the index side). Add HNSW graph overhead — rule of thumb 1.5–2× raw vector bytes. This is separate from LLM KV cache sizing when chunks land in the prompt.

Operational checklist

• Embedding version pins: reindex when you change models (read model cards).
• Chunk strategy: store chunk hash and source URL in metadata for audit trails.
• Access control: map API keys to collections; never share one global service token across customers.
• Cost: ANN RAM scales with dimension × point count; KV cache is separate LLM memory — do not conflate the two budgets.
• Observability: export query latency, empty-result rate, and embedding version per collection — alert when recall@10 drops on golden questions.

Worked scenario: internal support bot

Imagine 40k Notion pages, 512-token chunks, text-embedding-3-large class dimensions, 200 internal users. pgvector on RDS handles this on a medium instance with HNSW if filters are team_id via SQL RLS. Add a cross-encoder reranker on the top 20 hits.

If the same product becomes external multi-tenant with 2k organizations, payload filters multiply. Qdrant collections per tenant or strict payload indexes often beat giant SQL WHERE chains. Re-evaluate when p95 retrieval exceeds 80 ms or when vacuum/autovacuum on Postgres struggles with churn.

Feature names and docs paths reflect November 8, 2026 releases; verify index types and cloud SKUs before production commits.

Related on explainx.ai

• What are embeddings?
• Top embedding models
• Rerankers in RAG pipelines
• Grounding vs fine-tuning RAG decisions
• MCP security guide
• Crawl4ai + Bedrock knowledge prep