AI-generated code is the agentic use case most banks already run and the one with the thinnest rulebook. No regulator has written a rule for software produced by a model, but the expectations are not empty: New York DFS told regulated entities in May 2026 to put human review on AI-generated code before deployment, validate inputs and restrict script execution; NIST profiled the secure software development life cycle for AI models and AI-integrated systems in SP 800-218A; the April 2026 model risk guidance decides whether something the agent produced counts as a model; and third-party guidance covers the coding platform and the foundation model behind it. The design problem is therefore enforcement, not policy. A bank's architecture rules — approved libraries and services, data-classification boundaries, logging, authentication patterns, no direct writes to the core — already exist for people. The system that survives an examination applies them to the agent mechanically: the rules are encoded as checks the agent cannot bypass, every change carries the prompt, the model, the diff and the reviewer, and a person with authority approves anything that reaches production. The agent accelerates the work inside the guardrails; it never moves them.
AI-generated code & coding agents: The model prepares and may act within limits; a person approves anything that reaches a customer or cannot be undone. Several model calls orchestrated by your code, in a sequence or a graph you designed. Material but recoverable. The model may prepare and, within limits, act, with a person approving anything that leaves the bank or touches a customer.
Which steps belong to a person, the model and a system?
| # | Step | Owner | Note |
|---|---|---|---|
| 1 | Encode the architecture rules as executable checks | A system | Linters, policy-as-code, dependency allow-lists, secret and data-boundary scanners, run in CI on every change regardless of author; the rules a reviewer used to hold in their head. |
| 2 | Scope what the agent can reach | A system | Repository, branch and tool permissions per task; no production credentials, no core-system endpoints, an allow-listed dependency registry; the constraint NIST's agent-security work describes. |
| 3 | Generate the change | The model | From a ticket and the codebase, in a branch; the prompt, model version and retrieved context are recorded with the diff. |
| 4 | Run the gates | A system | Build, tests, static analysis, the architecture checks and a licence and provenance scan; a failed gate returns the work to the model or the engineer, never to production. |
| 5 | Review and approve | A person | An engineer with authority over that system reads the diff and the gate results and approves; the record names them — the DFS expectation stated in May 2026. |
| 6 | Classify what was built | A person | If the change implements or alters a complex quantitative method, it enters the model inventory and validation under the 2026 guidance; otherwise it is software under change management. |
| 7 | Deploy under change management | A system | The existing release process, with the AI-assisted origin flagged so incident review and audit can trace it. |
| 8 | Monitor and learn | A system | Defect, vulnerability and rollback rates for AI-assisted changes tracked against the baseline; rules tightened where the agent keeps failing a gate. |
Which control layers carry the weight?
Each layer is described, with its controls and documents, on the control plane page.
- Governance and accountability: core for this design.
- Identity and entitlements: core for this design.
- Action gateway: core for this design.
- Data and knowledge: core for this design.
- Models and vendors: core for this design.
Which rules and guidance does this design answer to?
| Document | Authority | Why it applies here | Status |
|---|---|---|---|
| DFS Frontier AI Models Industry Letter (May 2026) | NY DFS | Human review of AI-generated code before deployment, input validation and restricted script execution — the one US expectation stated by name. | In force |
| NIST SP 800-218A (SSDF profile for generative AI) | NIST | The secure-development life cycle extended to AI models and AI-integrated systems; the standard to cite for the review gate. | In force |
| SR 26-2 | Federal Reserve | Decides whether what the agent produced is a model: complex quantitative methods are validated, deterministic code is not — but the boundary must be documented. | In force |
| SR 23-4 | Federal Reserve | The coding platform, the foundation model and any hosted tools are third-party relationships with due diligence, contract and monitoring duties. | In force |
| NIST AI 600-1 (Generative AI Profile) | NIST | Confabulation, information security and value-chain risks for the generative system writing the code. | In force |
| CAISI RFI on AI agent security (2026) | NIST | Constraining and monitoring what a coding agent can reach; indirect prompt injection through repositories and dependencies. | Proposed |
| 23 NYCRR Part 500 | NY DFS | For New York-regulated banks, the change-management, access-control and certification obligations the code gate sits inside. | In force |
| SB 26-189 | Colorado AI Act | Colorado: notice, explanation and human review for consequential automated decisions from Jan 1, 2027. | Final |
| BCBS Third-Party Risk Principles (Dec 2025) | Basel Committee | Nth-party supply chains and concentration on cloud providers. | In force |
| BCBS ICT Risk Management Report (June 2026) | Basel Committee | How supervisors look at ICT and cloud dependencies. | Final |
| OCC Bulletin 2026-13 | OCC | For a national bank, the OCC's copy of the 2026 model-risk guidance. | In force |
| NIST AI RMF 1.0 | NIST | The voluntary Govern, Map, Measure, Manage frame for everything model-risk guidance leaves out. | In force |
How will you know it works, before and after launch?
- A golden dataset of at least 150 real cases with expected outputs, including adversarial inputs: wrong documents, unusual formats, prompts that try to change the task.
- Code-based checks on every output: schema conformance, required fields, reconciliations against the system of record. Threshold: 99% or better before launch, every run in production.
- Citation checks: every factual claim resolves to a passage in the governed set; unsupported claims below 2% of answers.
- Trace evals per step, not only end to end: which step fails, how often, at what cost, so a prompt or model change can be judged step by step.
- The same suite reruns on every prompt change, model version change and retrieval change; a regression blocks the release. That is what ongoing monitoring and outcomes analysis mean in model-risk terms.
Where must a person be in the loop?
- Per-action approval by a competent reviewer for anything customer-facing or irreversible, with authority to approve, modify or decline and escalation volume sized to reviewer capacity; sampled review for the rest.
- Validation proportionate to materiality, with monitoring for drift on inputs and outputs.
- An escalation route to a person that the customer can reach in one step.
- Monthly review of the evals and the exception log by the accountable owner.
- Trusting review to a second model: DFS asked for human oversight of AI-generated code, and an examiner will ask who the human was.
- Letting the agent hold credentials or reach the core directly; the blast radius of a wrong action is the whole reason for the permission scope.
- Treating every generated artefact as 'just code': a scoring formula or a monitoring threshold the agent wrote is a model under the 2026 definition and needs validation.
- Forgetting the platform is a vendor: the coding assistant, the model and any hosted tools sit under third-party guidance, including what the provider does with the bank's code and data.
- Skipping provenance: without prompt, model version and context in the record, the change cannot be explained at an incident review or reproduced for an audit.
What will a validator or an examiner ask?
- Where is this system in your inventory, what tier did you assign, and who signed it off?
- What counts as a model here, and what does your validation cover for the parts that are not?
- Show me the data lineage behind the retrieval set and the training or tuning data.
- What can the system do without a person, and where is that written down?
- How do you know it is still working: which evals run, how often, and what happened the last time one failed?
- What did you do about the vendor: due diligence, contract, exit plan, concentration?
- Walk me through one wrong output from production and what the customer, if any, saw.
- Who can switch it off, and has that been tested?
What it does: writes, reviews and ships code with AI assistance under the bank's architecture rules; the model prepares and may act within limits; a person approves anything that reaches a customer or cannot be undone.
Pattern: workflow (prompt chaining); tier 2: act with approval.
Rules it answers to: 4 documents across US, each linked in the brief.
How we know it works: a golden dataset, automated checks on every release, and human review at the level the tier demands.
What could go wrong and who answers: the accountable owner, the kill switch, the escalation route.
Which of the 120 largest US banks have put this use case on the record?
Every rule this brief cites, the morning it changes.
agent deployments, regulator positions and the day's six stories · in your inbox by 7 am PT · free
plus every regulation, bank and agent page update, the morning after · leave any morning