Enterprise RAG Is Harder Than the Model: Designing a Multi-Tenant Architecture

Enterprise RAG Often Fails on Data Boundaries Before Model Quality

In enterprise RAG, the architecture question is not only which model answers best. The system must make tenant boundaries, permissions, retrieval, and operational evidence verifiable.

Model quality sets an upper bound on answers, but data isolation, auditability, and the ability to evolve safely determine whether a prototype can become a production service.

What Does a Headless RAG Architecture Connect?

A Headless architecture separates client experience from reusable knowledge and AI services while keeping identity, authorization, audit, and data scope in one governance layer.

Which Multi-Tenant Data Architecture Should You Choose?

Higher isolation generally increases infrastructure, upgrade, and operating costs.

How should the isolation model be chosen?

Choose the isolation layer from sensitivity, customization, backup and restore requirements, data-residency obligations, and operating cost—not from a single universal rule.

How PostgreSQL RLS Establishes Tenant Isolation

PostgreSQL Row-Level Security can restrict reads and writes according to the tenant context of the current database transaction. It adds a database-level control when application code misses a tenant filter.

1CREATE EXTENSION IF NOT EXISTS vector;
2
3CREATE TABLE knowledge_chunks (
4  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
5  tenant_id UUID NOT NULL,
6  document_id UUID NOT NULL,
7  content TEXT NOT NULL,
8  embedding vector(1536),
9  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
10);
11
12ALTER TABLE knowledge_chunks ENABLE ROW LEVEL SECURITY;
13ALTER TABLE knowledge_chunks FORCE ROW LEVEL SECURITY;
14
15CREATE POLICY tenant_select_policy
16ON knowledge_chunks FOR SELECT
17USING (tenant_id = NULLIF(current_setting('app.current_tenant_id', true), '')::uuid);
18
19CREATE POLICY tenant_insert_policy
20ON knowledge_chunks FOR INSERT
21WITH CHECK (tenant_id = NULLIF(current_setting('app.current_tenant_id', true), '')::uuid);

What can RLS protect, and what can it not replace?

How do connection pools and roles keep RLS effective?

An RLS policy does not create a trustworthy tenant boundary on its own. Every request must set tenant context and run tenant-scoped reads, writes, and retrieval in the same explicit transaction with a runtime role that cannot bypass RLS.

1BEGIN;
2SET LOCAL app.current_tenant_id = 'tenant-uuid';
3
4SELECT id, content
5FROM knowledge_chunks
6ORDER BY embedding <=> :query_embedding
7LIMIT 8;
8
9COMMIT;

SET LOCAL only applies inside the current transaction. Transaction pooling can support this pattern when BEGIN, SET LOCAL, and all queries stay together; statement pooling cannot. Superusers and BYPASSRLS roles always bypass RLS, while table owners bypass it by default unless FORCE ROW LEVEL SECURITY is enabled.

How Should a Headless RAG API Behave?

A Headless API should remain stateless and derive tenant and permission scope from trusted credentials. The client must never choose or override tenant_id.

Document Upload API

1POST /api/v1/documents
2Authorization: Bearer <access_token>
3Content-Type: multipart/form-data

1{
2  "document_id": "d3b07384-d113-4956-a5cc-9c60012443d3",
3  "status": "processing",
4  "message": "The document is queued for parsing and indexing"
5}

Conversation and Retrieval API

1POST /api/v1/chat/completions
2Authorization: Bearer <access_token>
3Content-Type: application/json

1{
2  "messages": [{ "role": "user", "content": "How is annual leave calculated?" }],
3  "stream": true
4}

Enterprise responses should include source documents, versions, updated time, Request ID, and explicit states for no evidence or insufficient permission.

How Do Documents Enter the RAG Knowledge Base?

Ingestion should be asynchronous so parsing, OCR, chunking, and embedding do not occupy the original HTTP request.

Chunking Should Follow Document Structure, Not a Fixed Character Count

Use headings, clauses, semantic boundaries, and document type to create chunks that preserve source and version traceability.

How Should Embedding Dimensions Evolve?

vector(1536) is a fixed data contract, not a placeholder for arbitrary future models. A new model can change both semantic space and output dimensions, so model name and version alone are not a migration plan.

1-- Example: a v2 model outputs 1024 dimensions
2CREATE TABLE knowledge_chunk_embeddings_v2 (
3  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
4  tenant_id UUID NOT NULL,
5  chunk_id UUID NOT NULL,
6  embedding_model TEXT NOT NULL,
7  embedding vector(1024) NOT NULL,
8  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
9);
10
11CREATE INDEX CONCURRENTLY knowledge_chunk_embeddings_v2_hnsw
12ON knowledge_chunk_embeddings_v2
13USING hnsw (embedding vector_cosine_ops);

How Does RAG Retrieve and Generate an Answer?

Each complete query establishes identity and tenant scope before hybrid retrieval, reranking, model generation, citations, and audit logging.

Why Does Enterprise Knowledge Need Hybrid Retrieval?

Enterprise knowledge mixes semantic questions with product codes, regulations, clause numbers, and exact values. Hybrid retrieval combines vector search, keyword search, metadata filters, and reranking.

How Should Vector Indexes Be Maintained After Tenant Deletion?

Complete tenant deletion is a compliance requirement, but high-churn deletion also changes index workload. Monitor index size, latency, recall, and maintenance time—especially for HNSW—rather than checking only that rows disappeared.

1REINDEX INDEX CONCURRENTLY knowledge_chunks_embedding_hnsw;
2VACUUM (ANALYZE) knowledge_chunks;

What Should Be Checked Before Enterprise RAG Goes Live?

A convincing demo is not a production acceptance test. Keep evidence for the following controls before launch.

Frequently Asked Questions

1. What is a multi-tenant enterprise RAG architecture?

It allows multiple organizations to use one RAG application while isolating their documents, vectors, permissions, and query results across the database, storage, queues, caches, logs, and model inputs.

2. Is PostgreSQL suitable for enterprise RAG?

It is suitable for many small to large RAG workloads because it combines relational data, metadata, full-text search, transactions, and pgvector. The right choice still depends on scale, latency, and operating complexity.

3. What does PostgreSQL RLS protect?

RLS can limit rows according to transaction-scoped tenant context. It reduces the risk of missing application filters, but it must be paired with explicit transactions, least-privilege roles, and connection-pool tests.

4. Does Shared Schema automatically cause data leakage?

No. Leakage risk usually comes from incorrect roles, missing tenant context, or missing negative tests. RLS, least privilege, and cross-tenant tests reduce that risk.

5. When is Shared Schema a poor fit?

Evaluate stronger isolation when customers require independent backup and restore, data residency, different schemas, extensive customization, or strict compliance.

6. Why should ingestion be asynchronous?

Parsing, OCR, chunking, and embedding can take seconds or minutes. Queues and workers prevent request timeouts and improve retries, recovery, and status tracking.

7. How large should RAG chunks be?

There is no universal chunk size. Use document structure, semantic boundaries, clauses, and real questions, then tune with retrieval and answer-quality tests.

8. What is hybrid retrieval?

It combines vector search, keyword search, metadata filtering, and reranking so the system can handle both semantic questions and exact terms, models, clauses, and values.

9. Can RAG eliminate hallucinations?

No. Retrieval can improve grounding, but retrieval failures, stale documents, prompt design, and generation errors remain. Use citations, refusal conditions, and review controls.

10. What is the business value of Headless RAG?

It turns retrieval and generation into governed APIs that websites, apps, messaging bots, and internal systems can reuse with the same data and permission model.

11. Does changing an embedding model require re-embedding?

Usually. Different models can change semantic space and dimensions. Plan a new table or column, parallel indexes, re-embedding, and a controlled read cutover.

12. What proves that enterprise RAG is ready for production?

Evidence should cover tenant isolation, pool transaction scope, role bypass, document versions, real question sets, citations, refusal behavior, performance, dimension migration, and deletion-index operations.

Conclusion: Architecture Is Not About One Universal Technical Answer

For many SaaS RAG products, a Headless API, Shared Schema, PostgreSQL RLS, pgvector, and asynchronous workers are a cost-effective starting architecture.

The important decision is not a single preferred technology. It is whether the complete system can isolate safely, operate reliably, evolve without hidden data risk, and provide evidence for every answer.