AI-generated code is the agentic use case most banks already run and the one with the thinnest rulebook. No regulator has written a rule for software produced by a model, but the expectations are not empty: New York DFS told regulated entities in May 2026 to put human review on AI-generated code before deployment, validate inputs and restrict script execution; NIST profiled the secure software development life cycle for AI models and AI-integrated systems in SP 800-218A; the April 2026 model risk guidance decides whether something the agent produced counts as a model; and third-party guidance covers the coding platform and the foundation model behind it. The design problem is therefore enforcement, not policy. A bank's architecture rules — approved libraries and services, data-classification boundaries, logging, authentication patterns, no direct writes to the core — already exist for people. The system that survives an examination applies them to the agent mechanically: the rules are encoded as checks the agent cannot bypass, every change carries the prompt, the model, the diff and the reviewer, and a person with authority approves anything that reaches production. The agent accelerates the work inside the guardrails; it never moves them.
AI-generated code & coding agents: The model prepares and may act within limits; a person approves anything that reaches a customer or cannot be undone. Several model calls orchestrated by your code, in a sequence or a graph you designed. Material but recoverable. The model may prepare and, within limits, act, with a person approving anything that leaves the bank or touches a customer.
Which steps belong to a person, the model and a system?
| # | Step | Owner | Note |
|---|---|---|---|
| 1 | Encode the architecture rules as executable checks | A system | Linters, policy-as-code, dependency allow-lists, secret and data-boundary scanners, run in CI on every change regardless of author; the rules a reviewer used to hold in their head. |
| 2 | Scope what the agent can reach | A system | Repository, branch and tool permissions per task; no production credentials, no core-system endpoints, an allow-listed dependency registry; the constraint NIST's agent-security work describes. |
| 3 | Generate the change | The model | From a ticket and the codebase, in a branch; the prompt, model version and retrieved context are recorded with the diff. |
| 4 | Run the gates | A system | Build, tests, static analysis, the architecture checks and a licence and provenance scan; a failed gate returns the work to the model or the engineer, never to production. |
| 5 | Review and approve | A person | An engineer with authority over that system reads the diff and the gate results and approves; the record names them — the DFS expectation stated in May 2026. |
| 6 | Classify what was built | A person | If the change implements or alters a complex quantitative method, it enters the model inventory and validation under the 2026 guidance; otherwise it is software under change management. |
| 7 | Deploy under change management | A system | The existing release process, with the AI-assisted origin flagged so incident review and audit can trace it. |
| 8 | Monitor and learn | A system | Defect, vulnerability and rollback rates for AI-assisted changes tracked against the baseline; rules tightened where the agent keeps failing a gate. |
Which control layers carry the weight?
Each layer is described, with its controls and documents, on the control plane page.
- Governance and accountability: core for this design.
- Identity and entitlements: core for this design.
- Action gateway: core for this design.
- Data and knowledge: core for this design.
- Models and vendors: core for this design.
Which rules and guidance does this design answer to?
| Document | Authority | Why it applies here | Status |
|---|---|---|---|
| DFS Frontier AI Models Industry Letter (May 2026) | NY DFS | Human review of AI-generated code before deployment, input validation and restricted script execution — the one US expectation stated by name. | In force |
| NIST SP 800-218A (SSDF profile for generative AI) | NIST | The secure-development life cycle extended to AI models and AI-integrated systems; the standard to cite for the review gate. | In force |
| SR 26-2 | Federal Reserve | Decides whether what the agent produced is a model: complex quantitative methods are validated, deterministic code is not — but the boundary must be documented. | In force |
| SR 23-4 | Federal Reserve | The coding platform, the foundation model and any hosted tools are third-party relationships with due diligence, contract and monitoring duties. | In force |
| NIST AI 600-1 (Generative AI Profile) | NIST | Confabulation, information security and value-chain risks for the generative system writing the code. | In force |
| CAISI RFI on AI agent security (2026) | NIST | Constraining and monitoring what a coding agent can reach; indirect prompt injection through repositories and dependencies. | Proposed |
| 23 NYCRR Part 500 | NY DFS | For New York-regulated banks, the change-management, access-control and certification obligations the code gate sits inside. | In force |
| SB 26-189 | Colorado AI Act | Colorado: notice, explanation and human review for consequential automated decisions from Jan 1, 2027. | Final |
| BCBS Third-Party Risk Principles (Dec 2025) | Basel Committee | Nth-party supply chains and concentration on cloud providers. | In force |
| BCBS ICT Risk Management Report (June 2026) | Basel Committee | How supervisors look at ICT and cloud dependencies. | Final |
| OCC Bulletin 2026-13 | OCC | For a national bank, the OCC's copy of the 2026 model-risk guidance. | In force |
| NIST AI RMF 1.0 | NIST | The voluntary Govern, Map, Measure, Manage frame for everything model-risk guidance leaves out. | In force |
How will you know it works, before and after launch?
- A golden dataset of at least 150 real cases with expected outputs, including adversarial inputs: wrong documents, unusual formats, prompts that try to change the task.
- Code-based checks on every output: schema conformance, required fields, reconciliations against the system of record. Threshold: 99% or better before launch, every run in production.
- Citation checks: every factual claim resolves to a passage in the governed set; unsupported claims below 2% of answers.
- Trace evals per step, not only end to end: which step fails, how often, at what cost, so a prompt or model change can be judged step by step.
- The same suite reruns on every prompt change, model version change and retrieval change; a regression blocks the release. That is what ongoing monitoring and outcomes analysis mean in model-risk terms.
Where must a person be in the loop?
- Per-action approval by a competent reviewer for anything customer-facing or irreversible; sampled review for the rest.
- Validation proportionate to materiality, with monitoring for drift on inputs and outputs.
- An escalation route to a person that the customer can reach in one step.
- Monthly review of the evals and the exception log by the accountable owner.
- Trusting review to a second model: DFS asked for human oversight of AI-generated code, and an examiner will ask who the human was.
- Letting the agent hold credentials or reach the core directly; the blast radius of a wrong action is the whole reason for the permission scope.
- Treating every generated artefact as 'just code': a scoring formula or a monitoring threshold the agent wrote is a model under the 2026 definition and needs validation.
- Forgetting the platform is a vendor: the coding assistant, the model and any hosted tools sit under third-party guidance, including what the provider does with the bank's code and data.
- Skipping provenance: without prompt, model version and context in the record, the change cannot be explained at an incident review or reproduced for an audit.
What will a validator or an examiner ask?
- Where is this system in your inventory, what tier did you assign, and who signed it off?
- What counts as a model here, and what does your validation cover for the parts that are not?
- Show me the data lineage behind the retrieval set and the training or tuning data.
- What can the system do without a person, and where is that written down?
- How do you know it is still working: which evals run, how often, and what happened the last time one failed?
- What did you do about the vendor: due diligence, contract, exit plan, concentration?
- Walk me through one wrong output from production and what the customer, if any, saw.
- Who can switch it off, and has that been tested?
What it does: writes, reviews and ships code with AI assistance under the bank's architecture rules; the model prepares and may act within limits; a person approves anything that reaches a customer or cannot be undone.
Pattern: workflow (prompt chaining); tier 2: act with approval.
Rules it answers to: 4 documents across US, each linked in the brief.
How we know it works: a golden dataset, automated checks on every release, and human review at the level the tier demands.
What could go wrong and who answers: the accountable owner, the kill switch, the escalation route.
Every rule this brief cites, the morning it changes.
agent deployments, regulator positions and the day's six stories · in your inbox by 7 am ET · free
plus every tracker, bank and agent page update, the morning after · leave any morning