# Building a system that writes, reviews and ships code with AI assistance under the bank's architecture rules inside a bank

Source: https://www.bankingnewsai.com/agentic-banking/build/ai-generated-code
Last updated: Sep 17, 2026

AI-generated code is the agentic use case most banks already run and the one with the thinnest rulebook. No regulator has written a rule for software produced by a model, but the expectations are not empty: New York DFS told regulated entities in May 2026 to put human review on AI-generated code before deployment, validate inputs and restrict script execution; NIST profiled the secure software development life cycle for AI models and AI-integrated systems in SP 800-218A; the April 2026 model risk guidance decides whether something the agent produced counts as a model; and third-party guidance covers the coding platform and the foundation model behind it. The design problem is therefore enforcement, not policy. A bank's architecture rules — approved libraries and services, data-classification boundaries, logging, authentication patterns, no direct writes to the core — already exist for people. The system that survives an examination applies them to the agent mechanically: the rules are encoded as checks the agent cannot bypass, every change carries the prompt, the model, the diff and the reviewer, and a person with authority approves anything that reaches production. The agent accelerates the work inside the guardrails; it never moves them.

## The brief (default answers)

| Choice | Answer |
| --- | --- |
| Who is affected | Staff only |
| Reversibility | With cost |
| Stakes | Material |
| Knowledge | Both |
| Verifiability | By a rule or a test |
| Steps | Partly |
| Jurisdiction | United States |
| Delivery route | Through a cloud platform |

- **Pattern:** Workflow — Several model calls orchestrated by your code, in a sequence or a graph you designed.
- **Tier:** 2, Tier 2: act with approval — Material but recoverable. The model may prepare and, within limits, act, with a person approving anything that leaves the bank or touches a customer.
- **Workflow shapes:** Prompt chaining
- **Human involvement:** The model prepares and may act within limits; a person approves anything that reaches a customer or cannot be undone.
- **Knowledge:** Retrieval over the governed document set, with a citation on every answer and a refusal when nothing relevant is found.; Tool calls to the system of record for anything live. Never retrieval for a balance, a case status or a limit.; Documents to reason with, systems to check against: the workflow decides which question goes where before the model answers.

## Decomposition

| Step | Owner | Note |
| --- | --- | --- |
| Encode the architecture rules as executable checks | system | Linters, policy-as-code, dependency allow-lists, secret and data-boundary scanners, run in CI on every change regardless of author; the rules a reviewer used to hold in their head. |
| Scope what the agent can reach | system | Repository, branch and tool permissions per task; no production credentials, no core-system endpoints, an allow-listed dependency registry; the constraint NIST's agent-security work describes. |
| Generate the change | model | From a ticket and the codebase, in a branch; the prompt, model version and retrieved context are recorded with the diff. |
| Run the gates | system | Build, tests, static analysis, the architecture checks and a licence and provenance scan; a failed gate returns the work to the model or the engineer, never to production. |
| Review and approve | human | An engineer with authority over that system reads the diff and the gate results and approves; the record names them — the DFS expectation stated in May 2026. |
| Classify what was built | human | If the change implements or alters a complex quantitative method, it enters the model inventory and validation under the 2026 guidance; otherwise it is software under change management. |
| Deploy under change management | system | The existing release process, with the AI-assisted origin flagged so incident review and audit can trace it. |
| Monitor and learn | system | Defect, vulnerability and rollback rates for AI-assisted changes tracked against the baseline; rules tightened where the agent keeps failing a gate. |

## Control layer weights (0–4)

| Layer | Weight |
| --- | --- |
| governance | 1 |
| identity | 1 |
| actions | 0.9 |
| data | 1 |
| models | 1 |
| runtime | 0.8 |
| observability | 0.7 |
| oversight | 0.6 |

## Documents that apply

- [DFS Frontier AI Models Industry Letter (May 2026)](https://www.bankingnewsai.com/ai-regulation/documents/ny-dfs-industry-letter-2026-05-21-frontier-ai-models) — Human review of AI-generated code before deployment, input validation and restricted script execution — the one US expectation stated by name.
- [NIST SP 800-218A (SSDF profile for generative AI)](https://www.bankingnewsai.com/ai-regulation/documents/nist-sp-800-218a) — The secure-development life cycle extended to AI models and AI-integrated systems; the standard to cite for the review gate.
- [SR 26-2](https://www.bankingnewsai.com/ai-regulation/documents/fed-sr-26-2) — Decides whether what the agent produced is a model: complex quantitative methods are validated, deterministic code is not — but the boundary must be documented.
- [SR 23-4](https://www.bankingnewsai.com/ai-regulation/documents/fed-sr-23-4) — The coding platform, the foundation model and any hosted tools are third-party relationships with due diligence, contract and monitoring duties.
- [NIST AI 600-1 (Generative AI Profile)](https://www.bankingnewsai.com/ai-regulation/documents/nist-ai-600-1) — Confabulation, information security and value-chain risks for the generative system writing the code.
- [CAISI RFI on AI agent security (2026)](https://www.bankingnewsai.com/ai-regulation/documents/nist-caisi-rfi-ai-agent-security-2026) — Constraining and monitoring what a coding agent can reach; indirect prompt injection through repositories and dependencies.
- [23 NYCRR Part 500](https://www.bankingnewsai.com/ai-regulation/documents/ny-dfs-23-nycrr-part-500) — For New York-regulated banks, the change-management, access-control and certification obligations the code gate sits inside.
- [SB 26-189](https://www.bankingnewsai.com/ai-regulation/documents/co-sb26-189) — Colorado: notice, explanation and human review for consequential automated decisions from Jan 1, 2027.
- [BCBS Third-Party Risk Principles (Dec 2025)](https://www.bankingnewsai.com/ai-regulation/documents/bcbs-third-party-risk-principles-2025) — Nth-party supply chains and concentration on cloud providers.
- [BCBS ICT Risk Management Report (June 2026)](https://www.bankingnewsai.com/ai-regulation/documents/bcbs-ict-risk-management-range-of-practices-2026) — How supervisors look at ICT and cloud dependencies.
- [OCC Bulletin 2026-13](https://www.bankingnewsai.com/ai-regulation/documents/occ-bulletin-2026-13) — For a national bank, the OCC's copy of the 2026 model-risk guidance.
- [NIST AI RMF 1.0](https://www.bankingnewsai.com/ai-regulation/documents/nist-ai-100-1) — The voluntary Govern, Map, Measure, Manage frame for everything model-risk guidance leaves out.

## Evals

- A golden dataset of at least 150 real cases with expected outputs, including adversarial inputs: wrong documents, unusual formats, prompts that try to change the task.
- Code-based checks on every output: schema conformance, required fields, reconciliations against the system of record. Threshold: 99% or better before launch, every run in production.
- Citation checks: every factual claim resolves to a passage in the governed set; unsupported claims below 2% of answers.
- Trace evals per step, not only end to end: which step fails, how often, at what cost, so a prompt or model change can be judged step by step.
- The same suite reruns on every prompt change, model version change and retrieval change; a regression blocks the release. That is what ongoing monitoring and outcomes analysis mean in model-risk terms.

## Human gates

- Per-action approval by a competent reviewer for anything customer-facing or irreversible; sampled review for the rest.
- Validation proportionate to materiality, with monitoring for drift on inputs and outputs.
- An escalation route to a person that the customer can reach in one step.
- Monthly review of the evals and the exception log by the accountable owner.

## What an examiner will ask

- Where is this system in your inventory, what tier did you assign, and who signed it off?
- What counts as a model here, and what does your validation cover for the parts that are not?
- Show me the data lineage behind the retrieval set and the training or tuning data.
- What can the system do without a person, and where is that written down?
- How do you know it is still working: which evals run, how often, and what happened the last time one failed?
- What did you do about the vendor: due diligence, contract, exit plan, concentration?
- Walk me through one wrong output from production and what the customer, if any, saw.
- Who can switch it off, and has that been tested?

## What the board should hear

- What it does: writes, reviews and ships code with AI assistance under the bank's architecture rules; the model prepares and may act within limits; a person approves anything that reaches a customer or cannot be undone.
- Pattern: workflow (prompt chaining); tier 2: act with approval.
- Rules it answers to: 4 documents across US, each linked in the brief.
- How we know it works: a golden dataset, automated checks on every release, and human review at the level the tier demands.
- What could go wrong and who answers: the accountable owner, the kill switch, the escalation route.

## Pitfalls

- Trusting review to a second model: DFS asked for human oversight of AI-generated code, and an examiner will ask who the human was.
- Letting the agent hold credentials or reach the core directly; the blast radius of a wrong action is the whole reason for the permission scope.
- Treating every generated artefact as 'just code': a scoring formula or a monitoring threshold the agent wrote is a model under the 2026 definition and needs validation.
- Forgetting the platform is a vendor: the coding assistant, the model and any hosted tools sit under third-party guidance, including what the provider does with the bank's code and data.
- Skipping provenance: without prompt, model version and context in the record, the change cannot be explained at an incident review or reproduced for an audit.

Change any answer on the interactive map: https://www.bankingnewsai.com/agentic-banking/build/ai-generated-code

---

Canonical page: https://www.bankingnewsai.com/agentic-banking/build/ai-generated-code
Part of [BankingNewsAI](https://www.bankingnewsai.com/) — a free daily brief on AI in banking, an AI regulation tracker (19 authorities, 166 documents) and AI-strategy profiles of the 100 largest US banks. Markdown versions of every reference page: append `.md` to the page URL; index at https://www.bankingnewsai.com/llms.txt.