How to Choose a Vector Database for RAG Without Religious Wars

A vector database is an implementation choice inside a RAG system, not the system itself. It stores vector representations and usually metadata so a query can retrieve plausible source passages. It does not make sources current, decide what a caller may see, or prove that an answer is supported.

This guide answers a narrower question: how should a team choose the vector-storage layer for a bounded RAG workload? It complements the broader RAG systems guide, where delivery scope and architecture are discussed. The choice should follow the retrieval contract, source lifecycle and operating constraints—not a vendor slogan or a benchmark copied from another corpus.

1. Start with the workload, not a shortlist

Before comparing products, write down what the retrieval layer must do. A support assistant over versioned product documentation has different needs from a multi-tenant internal knowledge base, a recommendation feature, or a prototype with a few thousand public pages.

The smallest useful workload brief includes:

1. Corpus and change pattern. How many passages exist now, how quickly they grow, which documents are replaced or deleted, and whether historical material remains searchable.
2. Query pattern. Natural-language questions, exact identifiers, filters, multilingual queries, batch jobs, or a mix. An exact order number is often a system-of-record lookup, not a vector-search problem.
3. Scope and permissions. Tenant, role, product, language and lifecycle conditions that must apply before a passage reaches model context.
4. Service objective. Expected availability, latency budget, recovery expectation, evaluation gate and named owner.
5. Operating boundary. Who runs backups, updates, access policies, monitoring and incident response after the pilot.

If these answers are unknown, a small reversible prototype is usually a better decision than an elaborate platform commitment. A database cannot compensate for an unowned corpus, missing metadata, or no way to tell whether retrieval is correct. The article on source quality for RAG covers that upstream gate; metadata and filters explains why admissibility belongs before answer generation.

2. Use VECTOR-6 as the comparison contract

VECTOR-6 is a deliberately vendor-neutral matrix. Score each candidate against the same six questions, then record the evidence and trade-offs rather than declaring a universal winner.

The matrix separates “fast enough in a demo” from “appropriate for this system.” A managed vector service may reduce infrastructure work but add data-location, access or cost questions. A self-hosted engine can provide more control while making patching, capacity and recovery your responsibility. A vector extension in an existing relational store may simplify consistency and operations at one scale, while a specialised engine can be a better fit for a different query pattern. Those are conditional trade-offs, not a league table.

type VectorChoiceEvidence = {
  candidate: string;
  corpus: { passages: number; updatePattern: "batch" | "continuous" };
  requiredFilters: string[];
  querySet: string[];
  recoveryOwner: string;
  decision: "pilot" | "adopt" | "reject";
  unresolvedRisk: string[];
};

3. Compare architecture shapes before brands

Most teams are choosing between a few shapes. A relational database with vector support can be practical when source metadata, application data and transactional updates already live together and the workload is bounded. It can reduce moving parts, but it still needs index, query and maintenance tests.

A dedicated vector engine can fit retrieval-heavy systems that need purpose-built indexing, similarity operations or particular hybrid-search behaviour. The added component must still receive lifecycle events, permission filters and backups. “Dedicated” does not mean the retrieval policy is built automatically.

A managed service can make provisioning and baseline operations easier. It does not remove the need to understand data residency, account boundaries, export paths, deletion semantics, rate limits, observed cost and incident ownership. Check those facts in the candidate's current documentation and contract rather than assuming them from a launch announcement.

A search platform with vector capability may be sensible when lexical search, filtering and existing operational practice are already central. It can be especially useful when exact terms and semantic phrasing must cooperate. Whether hybrid search is worthwhile is an evaluation question: test cases should show that combining signals improves the desired evidence, not merely changes rankings.

4. Three scenarios, three reasonable choices

Scenario A: a controlled first pilot. The corpus is small, sources are public or tightly owned, one team operates the application, and the purpose is to test whether retrieval helps at all. Prefer the smallest path that preserves source IDs, metadata, versioning and deletion. The decision is reversible: validate retrieval against a small test set before adding scale or vendor complexity.

Scenario B: a multi-tenant internal assistant. The main risk is not only relevance but disclosure. The choice must demonstrate tenant and role filtering, versioned source removal, auditability and a recovery owner. Reject a candidate that requires application code to remember security filters without a test that proves every retrieval call carries scope. The database is only one part of the boundary; identity and policy contracts remain separate dependencies.

Scenario C: a document search product with changing content. New documents, revisions, exact terms and natural-language questions coexist. Choose a shape that can trace each answer candidate to a source locator, remove superseded content predictably and compare lexical, vector and hybrid retrieval against named questions. Add reranking only when the evaluation shows that the extra stage changes useful outcomes enough to justify its operating cost.

The same product can be an appropriate choice in one scenario and an unnecessary liability in another. That is why a decision record should name the workload and exclusions instead of saying “we selected the best vector database.”

5. Failure modes that a comparison page often hides

A benchmark becomes a promise. Benchmark results depend on dataset, language, hardware, index parameters, filters and measurement method. They may be useful input, but they do not predict the quality or cost of a different corpus. Run a representative test and keep the conditions with the result.

Filtering happens after retrieval context is assembled. If a result can enter prompt context before tenant, role or lifecycle checks, the boundary already failed. Build trusted filters from authenticated identity and apply them before the model sees text.

Deletion means only removing a document row. A source may still exist in chunks, index replicas, caches or queued jobs. Define the removal event, expected propagation, verification query and recovery route. Test it with a superseded or access-revoked document.

Similarity thresholds are borrowed. There is no universal safe score. A threshold depends on the embedding model, language, chunking, query distribution and competing candidates. Evaluate precision, unsupported-answer routes and failure cases in your own system.

Operations are deferred to “later.” A pilot becomes production with no backup, no alert, no access review and no owner. If the service matters enough to answer employees or customers, its recovery and accountability need a named path before expansion.

6. A practical decision gate

Make the first decision an evidence-backed pilot, not a permanent declaration. Prepare representative sources and questions, including normal retrieval, exact identifiers, source updates, denied access, mixed-language inputs if they are in scope, and questions that should receive no answer. Record expected evidence before tuning.

require(workload.owner && workload.failureCost && workload.changePattern);
require(scope.tenantId || workload.isPublic);
require(testSet.normal && testSet.changed && testSet.denied && testSet.noAnswer);
require(recovery.backupOwner && recovery.restorePath);

const adopt = evaluation.meetsAcceptance && operations.haveNamedOwner;
const route = adopt ? "bounded-rollout" : "revise-or-keep-pilot";

The useful outcome is a retrieval layer whose limits are known: which sources it searches, which filters it enforces, how it changes when documents change, how a reviewer can inspect a result, and who restores it when something goes wrong. For an architecture review of that full path, start with the RAG systems guide or the engineering evidence in the case studies.