Fraud is the most permissive use case in regulatory terms and the most time-critical operationally. Detection itself is usually a validated model on live data; language models earn their place in triage, investigator assistance and customer contact, where speed and clarity matter and where the threat itself is increasingly generated by AI. The rule to design around: a block or a decline that touches credit is an adverse action, and a customer contact must not be mistakable for a scam.
Fraud detection: The model prepares and may act within limits; a person approves anything that reaches a customer or cannot be undone. One model call with retrieval, tools and a structured output, inside code you control. Material but recoverable. The model may prepare and, within limits, act, with a person approving anything that leaves the bank or touches a customer.
Which steps belong to a person, the model and a system?
| # | Step | Owner | Note |
|---|---|---|---|
| 1 | Detect | A system | The validated scoring model on live transaction data. |
| 2 | Triage and enrich | The model | Summarise the case, pull related activity via tools, propose a disposition. |
| 3 | Act on the account | A system | Holds and blocks by rule, within limits; logged. |
| 4 | Contact the customer | A person | Or a verified channel with a scripted, clearly identified AI assistant. |
| 5 | Investigate and report | A person | With the model's file, and the SAR where required. |
Which control layers carry the weight?
Each layer is described, with its controls and documents, on the control plane page.
- Governance and accountability: core for this design.
- Identity and entitlements: core for this design.
- Action gateway: core for this design.
- Models and vendors: core for this design.
- Human oversight and escalation: core for this design.
Which rules and guidance does this design answer to?
| Document | Authority | Why it applies here | Status |
|---|---|---|---|
| FIN-2024-Alert004 (Deepfake Media) | FinCEN | The threat picture: synthetic documents, voices and video at onboarding. | In force |
| Regulation (EU) 2024/1689 | EU AI Act | Fraud detection is carved out of the high-risk credit category. | In force |
| SR 26-2 | Federal Reserve | Fraud models validated in proportion to materiality. | In force |
| ECOA / Regulation B adverse action (15 U.S.C. 1691(d); 12 CFR 1002.9) | CFPB | A fraud-driven decline of a credit application is still an adverse action. | In force |
| DFS AI Cybersecurity Industry Letter (Oct 2024) | NY DFS | AI-enabled social engineering must appear in risk assessments and MFA design. | In force |
| CFPB Chatbots in Consumer Finance (issue spotlight, 2023) | CFPB | Customer-facing AI must not block access to a person or give wrong answers about rights. | Final |
| SR 23-4 | Federal Reserve | US third-party risk management, including the model provider. | In force |
| SB 26-189 | Colorado AI Act | Colorado: notice, explanation and human review for consequential automated decisions from Jan 1, 2027. | Final |
| BCBS Third-Party Risk Principles (Dec 2025) | Basel Committee | Nth-party supply chains and concentration on cloud providers. | In force |
| BCBS ICT Risk Management Report (June 2026) | Basel Committee | How supervisors look at ICT and cloud dependencies. | Final |
| OCC Bulletin 2026-13 | OCC | For a national bank, the OCC's copy of the 2026 model-risk guidance. | In force |
| NIST AI RMF 1.0 | NIST | The voluntary Govern, Map, Measure, Manage frame for everything model-risk guidance leaves out. | In force |
How will you know it works, before and after launch?
- A golden dataset of at least 150 real cases with expected outputs, including adversarial inputs: wrong documents, unusual formats, prompts that try to change the task.
- Code-based checks on every output: schema conformance, required fields, reconciliations against the system of record. Threshold: 99% or better before launch, every run in production.
- Escalation evals: the cases that must reach a person do, on a held-out set, with precision and recall both reported.
- The same suite reruns on every prompt change, model version change and retrieval change; a regression blocks the release. That is what ongoing monitoring and outcomes analysis mean in model-risk terms.
Where must a person be in the loop?
- Per-action approval by a competent reviewer for anything customer-facing or irreversible; sampled review for the rest.
- Validation proportionate to materiality, with monitoring for drift on inputs and outputs.
- An escalation route to a person that the customer can reach in one step.
- Monthly review of the evals and the exception log by the accountable owner.
- The customer is told they are dealing with an AI system and can reach a person in one step.
- A fraud decline on a credit product with no adverse-action reasons.
- Customer outreach that trains customers to trust unverified AI voices.
- Training data assembled outside the bank's data governance.
What will a validator or an examiner ask?
- Where is this system in your inventory, what tier did you assign, and who signed it off?
- What counts as a model here, and what does your validation cover for the parts that are not?
- Show me the data lineage behind the retrieval set and the training or tuning data.
- What can the system do without a person, and where is that written down?
- How do you know it is still working: which evals run, how often, and what happened the last time one failed?
- What did you do about the vendor: due diligence, contract, exit plan, concentration?
- Walk me through one wrong output from production and what the customer, if any, saw.
- Who can switch it off, and has that been tested?
- Show me how a customer reaches a person, and how long it took the last ten who tried.
What it does: scores and triages suspected fraud, and helps investigators and customers respond; the model prepares and may act within limits; a person approves anything that reaches a customer or cannot be undone.
Pattern: augmented call; tier 2: act with approval.
Rules it answers to: 4 documents across US, each linked in the brief.
How we know it works: a golden dataset, automated checks on every release, and human review at the level the tier demands.
What could go wrong and who answers: the accountable owner, the kill switch, the escalation route.
Which of the 100 largest US banks have put this use case on the record?
Every rule this brief cites, the morning it changes.
agent deployments, regulator positions and the day's six stories · in your inbox by 7 am ET · free
plus every tracker, bank and agent page update, the morning after · leave any morning