The Trust Layer for Federal Compliance AI

Executive Case Study

Problem. A hypothetical compliance assistant scenario: surfacing accurate, auditable answers across four NIST frameworks — SP 800-53, AI RMF 1.0, AI 600-1, and FedRAMP Moderate Baseline — to demonstrate governed RAG architecture under zero-tolerance conditions for PII exposure and hallucinated policy citations.

Constraints. No proprietary data leaves the boundary. Every response must be traceable to a source document. Guardrails fire after PII scrub but before retrieval — compliance assertions never bypass safety controls at any stage.

Decisions. Hybrid retrieval (pgvector HNSW + BM25/RRF fusion) over keyword-only search — captures semantic and lexical signal across compliance corpus. Dual Bedrock Guardrail gates at input and output — compliance assertions never bypass safety controls. Cohere cross-encoder rerank narrows top-10 to top-5 before generation fires.

Risk mitigations.

• Presidio PII scrubbing on input and output.
• MIN_RRF_SCORE=0.0150 post-RRF quality gate filters noise before Cohere rerank fires.
• Langfuse trace-level observability per query.
• RAGAs golden-set evaluation — faithfulness, context precision, and answer relevancy scored against curated compliance QA pairs. Faithfulness 0.90 is the governing metric.
• Answer relevancy intentionally low (0.56) — hedging compliance assertions is correct domain behavior.

Architecture

Governed RAG pipeline — offline ingestion + online query path with dual guardrail gates.

Related

Companion article: Beyond Retrieval — Architecting the Trust Layer for Enterprise AI