Power & systems

Meta's AI agent bumped a dog's insurance from $200 to $888

Who pays for this kind of mistake? US banks just answered.

6 min read
Meta's AI agent bumped a dog's insurance from $200 to $888

Four times the insurance bill, and nobody asked for it

A Meta employee is testing Muse, the company's in-house AI agent, the kind that can send emails, book things and buy things on your behalf. The agent switches the employee's dog insurance to a premium plan by mistake: $888 a month instead of $200. The employee reports there was no way to undo it.

The story comes from internal conversations obtained by Reuters, which we're following here through the write-ups by Ars Technica and Next. There's more in them. In the middle of a chat about fishing, Muse slips an employee the balances of two accounts, at Capital One and Charles Schwab, that aren't theirs. Elsewhere, according to Next, the agent books a non-refundable Airbnb without permission, flags transactions that never happened, and quotes wrong prices for purchases that very much did.

Think of an overeager intern who copies the neighbor's bank statement into your file, then signs your purchase order. Except this intern has been downloaded more than ten million times since it launched in the US in September.

What Meta says, and what nobody could establish

Meta calls these isolated incidents. Internally, staff were given an explanation: hallucination. Built-in instructions pushed the agent to nudge the user between sessions, and it would then make up messages. One member of the AI team lamented that the made-up content leaned "unfortunately" toward the touchiest topics, finance and security. According to Meta, the flaws have been fixed.

The company adds a point that matters: employees use versions of Muse that differ from the public one, with features you won't find elsewhere. And Reuters itself admits it couldn't determine whether Muse was revealing other people's balances or inventing them.

That doesn't leave the public version in the clear. Since September, users have complained that Muse read private messages they said they hadn't given it access to, and a columnist at Inc. described his own case. Meta acknowledged "a few reports" from people who believed their agent was leaking other people's data, and called it a bug that has been fixed. For its investigation, Reuters ran its own test with Signal and the AI Now Institute: on Android and on Mac, Muse could read Signal messages by indirect routes, bypassing the connections the user picks.

So the line is clear. Message access affects the public product. The money errors are documented only on internal versions, and none of the sources we've read shows such an error at a customer. But the question they raise doesn't depend on the version.

What Meta put in its terms

Meta has already answered that question, in the Muse terms in force since November 17. The user is solely responsible for the agent's actions, contracts, commitments and transactions included. If you give Muse a payment method, you bear the full risk of financial loss: it's on you to set limits, watch your accounts and dispute errors with your bank.

The help page puts it plainly: your Muse can get things wrong or do unexpected things, and fixing that, including with third parties, is up to you. As for Meta's own liability, it's capped, except for gross negligence, intentional misconduct or fraud, at $250 or what the user paid over twelve months, and the clause explicitly covers any "unintended, erroneous, or unauthorized action" by Muse, unintended transactions included. For a user on the free tier, the dog insurance alone would have blown past that in the first month.

Which leaves the bank. It has just shut the door too.

What banks put in theirs

Citi, the third-largest commercial bank in the US, added an entire section on AI agents to its online banking agreement in November. We've read it. It first advises against handing your credentials to a third-party agent. If the customer does it anyway, everything the agent does is treated as authorized by the customer. If the credentials given to the agent leak to other people or other agents, the bank will treat that the same way: as if the customer had acted.

The clause goes further. Citi has no obligation to verify a transaction, even when it contradicts the instructions the customer gave their agent. Misinterpretation, unintended action, overstepping the mandate: all of it falls on the customer, unless the law says otherwise. And monitoring the agent is "your responsibility and not Citi's."

At JPMorgan, the country's biggest bank, the Chase terms require an agent to identify itself as one and answer honestly if asked whether it's human. The user stays responsible if the agent misunderstands, acts unintentionally or exceeds the authority it was given. The text doesn't mention hacking; Citi's clause is the one that goes that far. According to TechCrunch, Bank of America took a similar position in October, PNC changed its terms on November 18, and Citizens warned customers by email. Those three texts, we haven't read.

Why they can write it

The principle isn't new. The US federal rule on electronic transfers protects a customer against an unauthorized transfer, but it excludes one made by someone the customer gave their access to. Lend a friend your card and PIN, and you've accepted what they do with it until you tell the bank. The new clauses read as that exception stretched to AI agents: hand over your credentials, and you've lent over your keys. That's our reading, not a legal finding.

There's a nuance TechCrunch points out, and the text confirms it: at Citi, these rules only apply to other people's agents. The ones the bank provides itself are exempt. TechCrunch also sees a way of keeping hold of the customer just as AI labs want to run their accounts for them.

Between two contracts, nobody

Set side by side, the two texts draw the same picture. Meta writes that the agent acts for you and you answer for it. The bank writes that the agent acts as you and you answer for it. One sells the agent, the other holds the account, and neither signs for what happens in between.

Muse has only launched in the US. But agents that order on our behalf are multiplying, as we saw with the few hundred orders placed by AIs at Deliveroo's owner, and this isn't Meta's first trust question with Muse: two months ago we looked at what it does with your conversations. Meta's testers, meanwhile, couldn't even tell where the balance on their screen came from.

Knowing what your assistant knows about you, and being able to read it back, is why Déclic relies on manence.ai, which keeps the assistant's memory in files on the user's side that they own and can read. Déclic is published by Manence.

Was this article useful?

Topics covered:

EconomyMetaAnalysis

Frequently asked questions

What happened with Meta's Muse AI agent?
According to internal conversations obtained by Reuters, and relayed by Ars Technica and Next, Muse mistakenly switched a tester's dog insurance from $200 to $888 a month, with no way back, and showed another employee balances from accounts that weren't theirs. Meta calls them isolated incidents and blames hallucination, and says its employees use different versions of Muse from the public one. Reuters says it couldn't determine whether those balances were real or invented.
Have Muse customers actually lost money to these errors?
The money errors are documented only on internal versions. None of the sources we've read shows such an error at a customer. The public product does have a different problem, access to messages: users complained as early as September, and Reuters, working with Signal and the AI Now Institute, found that Muse could read Signal messages on Android and Mac through indirect routes.
Who is liable if Muse makes a transaction by mistake?
Muse's terms, in force since November 17, make the user solely responsible for what the agent does, transactions included. If you hand it a payment method, you carry the full risk of financial loss. Meta's own liability is capped, except for gross negligence, intentional misconduct or fraud, at $250 or what the user paid over twelve months.
What do US banks say about AI agents?
In November, Citi added a section to its online banking agreement that advises against giving your credentials to a third-party agent. If a customer does, everything the agent does is treated as authorized by the customer, including when those credentials leak to others. Chase's terms also leave the user responsible for their agent's mistakes, without mentioning hacking. According to TechCrunch, Bank of America, PNC and Citizens have taken similar positions; we haven't read those three texts.
Why can banks push responsibility onto the customer?
The US federal rule on electronic transfers protects customers against unauthorized transfers, but it excludes one made by someone the customer gave their access to, until the bank is told. The new clauses read as that exception stretched to cover an AI agent. At Citi, they only apply to other companies' agents: the ones the bank provides itself are exempt.
Alexandre Noto

Alexandre Noto

Co-founder & Tech Expert

Alexandre has been in tech for over 20 years. Entrepreneur, software architect and AI enthusiast, he translates complex concepts into accessible explanations. At Declic Media, he is the technical voice that makes AI understandable for everyone.

All articles by Alexandre →
The free AI newsletter