Portfolio reference implementation discussed throughout this piece (IRSS) is built on public U.S. Census Bureau ACS PUMS microdata. Not affiliated with any actual federal program or agency. Views and architectural choices are my own.
Building a working AI/ML system is only part of getting to production in a federal environment. Authorization introduces a different set of architectural questions — especially when the system includes AI-specific governance concerns around fairness, drift, explainability, and human oversight.
For the Income Risk Scoring System (IRSS) — a reference implementation built on 2023 U.S. Census Bureau ACS PUMS data — the AI/ML-specific governance and authorization layer became part of the system architecture, not documentation added afterward. This article walks through what changed, what that looked like in practice, and what defensible AI authorization looks like when governance becomes system behavior rather than policy documentation.
What AI Systems Add to the Authorization Conversation
Traditional authorization frameworks primarily establish trust in the operating environment: access control, encryption, logging, configuration management. These questions map cleanly onto NIST SP 800-53 Rev. 5 control families — AC, AU, CM, SC, SI.
AI systems introduce an additional governance layer around model behavior — fairness, drift, explainability, human oversight, and lifecycle controls. That layer is distinct from infrastructure trust, and it requires different controls, different evidence, and different expertise to address.
Figure 1. Traditional vs. governed AI authorization controls. Infrastructure trust and model trust are distinct governance problems.
Building IRSS made this distinction concrete. The infrastructure controls were necessary but not sufficient. The authorization question was not just “can I trust the environment?” It was also “can I trust the model?” — and 800-53 alone does not answer that second question.
800-53 does not explicitly address model fairness, drift governance, explainability, or AI lifecycle risk in the way AI systems require. This is not a criticism of 800-53 — it was designed for information systems, not machine learning pipelines. It is a gap that NIST AI RMF 1.0 exists to fill, organized across GOVERN, MAP, MEASURE, and MANAGE functions.
The Federal AI Authorization Stack
The relationship between these frameworks is a stack, not a parallel set of documents:
Figure 2. Federal AI authorization framework stack. System trust + model trust = defensible AI authorization.
FIPS 199 determines the impact level — the high water mark across Confidentiality, Integrity, and Availability. That level drives which 800-53 controls apply. 800-37 defines the RMF lifecycle: Prepare, Categorize, Select, Implement, Assess, Authorize, and Monitor. 800-53 provides the security control baseline. The AI RMF sits as a governance overlay, adding the layer that applies specifically to AI systems, where model behavior introduces governance concerns traditional controls do not cover.
The stack produces an Authorization Evidence Package — SSP, RAR, POA&M, SAP, PIA, and AI governance artifacts — assembled from both layers. The bottom line is the architectural point the diagram makes: System Trust + Model Trust = Defensible AI Authorization.
For IRSS: Confidentiality Low (public ACS PUMS microdata, no PII), Integrity Moderate (model tampering produces biased predictions with downstream mission impact), Availability Low (decision-support tool, local inference fallback available). Overall: Moderate — driven by Integrity. That single determination drove which controls applied and what the risk assessment needed to cover.
Three Architectural Decisions That Became Governance Controls
What became clear building IRSS: governance cannot live only in policy documents. It has to exist in architecture. Policy-stated controls depend on humans following procedures. Governance embedded in code is structural — it cannot be bypassed without a code change, a pull request, and a CI/CD rerun.
Three decisions in IRSS illustrate this.
Sensitive feature separation. Race, sex, and nativity are physically removed from the training dataset before the train/test split in preprocess.py. They never reach the model. They are rejoined only in the evaluation script for post-prediction fairness auditing. This supports the fairness governance objectives of NIST AI RMF GOVERN 1.7 — and it does so through code enforcement, not policy statement. A reviewer cannot read a policy and verify it is followed. A reviewer can inspect implementation evidence and confirm the separation exists.
The fairness gate as a CI/CD hard stop. evaluate.py exits with code 1 if any demographic group’s Positive Prediction Rate deviates from the overall rate by more than ±0.20. GitHub Actions treats exit code 1 as a pipeline failure — deployment is blocked. This is NIST AI RMF MEASURE 2.7 implemented as system behavior. The gate is not a monitoring dashboard that someone might review and override. It is a structural hard stop. A model that fails the gate cannot reach production without changing the code.
No automated model promotion. The MLflow registry requires explicit human action to promote a model from staging. A GitHub Environment gate requires named reviewer approval before the deployment job executes. No code path exists for automated production deployment. This is NIST AI RMF MANAGE 1.3 — enforced structurally. The job cannot start without the approval.
The pattern across all three is the same: the governance control is verifiable because it is embedded in the system, not stated in a document. For a reviewer asking “how do I know this actually happens?” — the answer is verifiable implementation evidence, not policy assertion.
What the Decision Log Actually Is
One thing building IRSS reinforced: the decision log is not a documentation artifact. It is a governance artifact.
The IRSS decision log (DL-001 through DL-020) was maintained throughout development alongside the code. Every architectural decision — why XGBoost over logistic regression, why person_weight is a sample weight and not a feature, why Virginia data as the initial scoped sample, why the fairness threshold is ±0.20 — is documented with explicit rationale and tradeoffs at the time the decision was made.
In a federal context, this maps to the Prepare step of NIST SP 800-37. It is also the most credible response to the question that comes up in any serious authorization review: “why did you make this decision?” The answer is not a verbal explanation. It is a dated log entry with documented rationale, written at the time of the decision rather than reconstructed afterward.
When the decision log and the SSP are written from the same architectural reality, the authorization package reads as evidence.
The POA&M as Honest Accounting
The Plan of Action and Milestones is often where package credibility is tested — either by claiming no gaps, or by listing gaps without remediation plans. The POA&M is not a list of failures. It is structured accountability for known weaknesses.
The IRSS POA&M has seven open items. The most significant: no ground truth monitoring post-deployment. Evidently AI detects input distribution drift — feature distributions shifting away from training baseline. But model performance drift, the actual degradation of prediction accuracy, requires ground truth labels: knowing what the actual income outcome was for a given prediction. Without that feedback loop, model degradation is undetectable until it surfaces in fairness metrics or downstream outcomes.
This is documented as a Medium-severity open item and a blocking condition for national deployment. Documenting it explicitly is more credible than claiming complete coverage — a reviewer who sees honest gap accounting with specific remediation timelines has more reason to trust the rest of the package.
The Architectural Point
Building IRSS reinforced a clear architectural distinction: traditional security controls establish trust in the environment. AI systems introduce a second trust problem around model behavior — fairness, drift, explainability, promotion controls, and lifecycle governance.
Defensible AI authorization requires both layers, and the AI governance layer has to be implemented in system architecture, not just described in policy documentation — that is what turns AI governance from policy intent into authorization evidence.
The IRSS reference implementation — including the ATO-aligned reference package — is in the responsible-mlops-risk-engine repository. The responsible-mlops-risk-engine is a production-grade reference implementation built on public U.S. Census Bureau ACS PUMS microdata. It is not affiliated with any federal agency or live authorization package — but every architectural decision, governance control, and documentation artifact reflects the standards that production federal AI systems require.
Related
Reference implementation: Responsible MLOps Risk Engine
Companion article: Beyond Predictions: From Model Accuracy to System Accountability