Block (Square, Cash App) · Institution's own AI · severity 2 of 5
A current Block employee told The Guardian in March 2026 that, according to internal surveys, the chatbots Block uses for initial customer-support requests had "made incredible mistakes", including telling customers to cancel or close their accounts. The claim was reported after Block cut about 4,000 jobs, citing AI; Block did not give the Guardian a statement. In April 2026 Square's head of product told VentureBeat that hallucinations are "never going to be zero". Separately, in August 2026 Block's engineering blog described Cash App's AI assistant Moneybot telling a customer their account type without checking, and said an automated repair system had produced more than 40 verified fixes. No customer loss or regulator action has been reported.
Record last reviewed Oct 21, 2026 · Markdown
Date
Institution
Country
United States
AI system
Customer-support chatbots; Moneybot (Cash App AI assistant) · LLM-based customer-facing chatbots and assistant · vendor: Block (built on third-party frontier models)
AI?
Described as AI by the institution or regulator
The institution, a regulator or a court called the system AI.
Category
Status
Reported
Severity
2 of 5
Impact
Not quantified. The Guardian's source cited internal surveys but gave no number of affected customers; no customer is reported to have lost money. · Block's engineering blog said more than 100 reported Moneybot failures no longer reproduce and more than 40 fixes were verified; it did not say how many customers saw the failures.
What happened?
On 8 March 2026 The Guardian published interviews with seven current and recently laid-off Block employees, a week after chief executive Jack Dorsey cut about 4,000 jobs, almost half the workforce, citing gains in AI productivity. One current employee, speaking anonymously, said some customers had become angry about Block outsourcing initial support requests to chatbots: "We've noticed [from internal surveys] that [these bots have] made incredible mistakes," including telling customers to cancel or close their existing accounts as potential solutions. Block did not provide a statement to the Guardian.[1]
On 7 April 2026, in a VentureBeat interview about Square's new Managerbot agent, Block's head of product at Square, Willem Avé, was asked about the Guardian's report of chatbots telling customers to cancel or close accounts. He pointed to the agent harness's role in reducing hallucinations and said: "It's never going to be zero. Obviously, these are probabilistic systems, and we have guidance and call-outs in the tool to provide that." He added that in regulated domains such as banking, lending and payments there are "strict guardrails" on what the product can say.[2]
Separately, on 28 August 2026 Block's engineering blog described failures in Moneybot, Cash App's LLM-powered financial assistant. In one backlog ticket, after a question about wire transfers, a customer asked whether they had a personal or business account and Moneybot answered from inference, without calling a tool, presenting a guess as account-specific knowledge; Block said the dialogue it published is a synthetic reconstruction. Block said its automated repair system had produced more than 40 verified fixes and established that more than 100 reported failures no longer reproduce. This post is not about the support chatbots the Guardian's source described, and Block has not linked the two.[3]
When did it happen?
- Mar 8, 2026
The Guardian reports a current Block employee's account that customer-support chatbots told customers to cancel or close their accounts; Block gives no statement.[1]
- Apr 7, 2026
Square's head of product tells VentureBeat that AI hallucinations are "never going to be zero" when asked about the Guardian's report.[2]
- Aug 28, 2026
Block's engineering blog describes a Moneybot failure and an automated system that produced more than 40 verified fixes.[3]
What was the impact?
- Customers: Not quantified. The Guardian's source cited internal surveys but gave no number of affected customers; no customer is reported to have lost money.[1][3]
- Other: Block's engineering blog said more than 100 reported Moneybot failures no longer reproduce and more than 40 fixes were verified; it did not say how many customers saw the failures.[1][3]
Severity 2 of 5: Limited harm: employees reported wrong account-closure advice from support chatbots, but no customer loss, number of affected customers or regulator action has been reported.
How did Block (Square, Cash App) respond?
Block did not provide a statement to the Guardian. When VentureBeat raised the chatbot errors, Square's head of product Willem Avé said hallucinations are reduced through tuning, prompt engineering and tool-call loops but are "never going to be zero", and that regulated domains have strict guardrails. Block's engineering blog said engineers stay in the loop to review fixes and decide what ships.[1][2][3]
What was the outcome, and did regulators act?
Block has not publicly confirmed or quantified the support-chatbot errors described to the Guardian. For Moneybot, Block said it added a rule that the assistant must never state a customer-specific account fact unless a tool result or account context establishes it, and turns each reproduced failure into a permanent regression test. No regulator action over Block's chatbots has been reported as of 21 October 2026.[1][3]
What did Block's AI chatbots get wrong?
A current Block employee told The Guardian in March 2026 that internal surveys showed the chatbots handling initial customer-support requests had made serious mistakes, including telling customers to cancel or close their existing accounts. Block did not comment to the Guardian and has not confirmed or quantified the errors.
Did Block admit its AI chatbots make mistakes?
Not about the specific claim. Square's head of product told VentureBeat in April 2026 that hallucinations are "never going to be zero", and Block's engineering blog in August 2026 described Cash App's Moneybot asserting a customer's account type without checking, which Block said it fixed with a new prompt rule.
Was any money lost because of Block's chatbot errors?
No customer loss has been reported. The Guardian's source described wrong advice to close or cancel accounts but gave no figures, and Block's engineering post did not report any customer harm from the Moneybot failures it described. No regulator has acted over Block's chatbots as of October 2026.
Is this linked to Block's AI layoffs?
The claims surfaced in The Guardian's reporting on Block's February 2026 decision to cut about 4,000 jobs, which Jack Dorsey attributed to AI productivity gains. Employees interviewed questioned whether AI tools could replace staff at that scale; the chatbot errors were one example they gave.
When AI goes wrong at a bank, the next morning's brief says so.
when one of these regulators moves, the next morning's brief says so · six sourced stories · 7 am PT · free
plus every regulation, bank and agent page update, the morning after · leave any morning