Power & systems

An OpenAI agent got into an Australian health insurance portal

The company took three months to tell anyone. What does the rule announced this Tuesday change?

5 min read
An OpenAI agent got into an Australian health insurance portal

An email in a public inbox

On September 10, an email lands in a public inbox run by the Australian government. In it, OpenAI explains that one of its AI agents, a program that chains actions together on its own to complete a task, broke into the Medicare statistics portal on June 18. Medicare here is Australia's public health insurance system, not the American program of the same name. The agent reached non-public files. According to Prime Minister Anthony Albanese, three other public health systems may have been affected.

Between June 18 and September 10, 84 days went by. According to the Guardian, the company knew by August. OpenAI's explanation is that its models were looking for statistics about Australia during an internal evaluation. Its own words: "our models took actions we did not intend."

The day before that email, on September 9, OpenAI had published a public policy paper. In it, the company asked the US Congress to require firms to provide "prompt written notice" to organizations whose protections their models get around during development or evaluation. We took that paper apart at the time. The Australian case falls squarely inside its scope.

What Canberra announced

On Tuesday, November 10, the Albanese government published the recommendations of the rapid review it commissioned in September. Eighteen recommendations, all accepted. The report itself is being held back: it will be released later, with national security material protected, the Prime Minister promises.

Alongside it, the government announced a dedicated AI safety regulator and six obligations for companies building the most advanced models. One of them: notify the Australian government of incidents "rapidly." Another: accept independent evaluation of their models.

The first question is whether the rule would have covered Medicare. On paper, yes. Recommendation 3, which proposes that the government consider mandatory reporting, explicitly includes "misaligned" behavior (an AI doing something other than what it was asked to do) and cyber incidents caused by agents. It sets no condition about personal data leaking. Government sources confirmed to the Guardian that the future law would target incidents like this one.

How fast is "rapidly"?

What the documents don't say is the deadline. The press release says "rapidly." The review recommends considering mandatory, "time-bound" reporting, without putting a number on it. The details of the law will come "in due course," the government writes.

It's a speed limit sign that just says "not too fast." Everyone agrees with it, and it will be hard to ticket anyone.

Yet Home Affairs and Cyber Security Minister Tony Burke did give a figure at the press conference that makes the question urgent. According to him, the time between discovering a vulnerability and exploiting it has dropped from roughly two and a half years to about ten hours with AI. He wants reporting to occur "immediately."

Two questions, two answers that miss

Then there's the past. At the same press conference, a journalist asks whether OpenAI will face criminal proceedings. The question contains a second one, about copyright and the singer Jimmy Barnes. Albanese answers only the second.

Another reporter presses: OpenAI, it seems, broke no law. The Prime Minister talks about the future notification rule. Pushed on the intrusion itself, he says the law has to "get ahead" of a fast-changing world. He calls the direct consequences "minor," since no personal data was obtained, while adding that this didn't make it a trivial matter.

Guardian Australia understands that OpenAI is not expected to face further proceedings. The review's criminal track, a "failure to prevent" offence (failing to take reasonable steps to stop harm caused by one's model), looks forward, not back. And it is only an option for the Justice Ministry to "consider." OpenAI says it is studying the recommendations.

The day before, in Paris

On Monday, November 9, the Élysée Palace brought together heads of state and government and the heads of several AI labs, including Sam Altman (OpenAI), Dario Amodei (Anthropic) and Arthur Mensch (Mistral AI). Out of it came a "Paris Call for the Safety of Artificial Intelligence."

Eight states signed: Kenya, Bahrain, Canada, Finland, France, Norway, the Netherlands and Cyprus. On the company side, OpenAI, Anthropic, Mistral AI, Microsoft, Hugging Face and others. The United Kingdom, Japan and the United States, which the Élysée says were represented at the meeting, are not on the published list, which "remains open." Neither is Australia.

On incidents, the text is cautious. Signatories "recognize the value" of strengthening existing arrangements to agree on how incidents are classified and to share information on serious incidents "in a timely manner." They agree to "examine whether a cooperation mechanism would be useful." No legal obligation, no penalty, no deadline. (Quotes from the Paris Call are from the French-language text.)

So in two months, three documents have promised speed: OpenAI's in September, Paris on Monday, Canberra on Tuesday. "Prompt," "timely," "rapidly." OpenAI wrote the first, signed the second, and its incident is what triggered the review behind the third.

The delay wasn't only OpenAI's

Fairness requires saying it: part of the slowness was bureaucratic. Once the email reached a public inbox, it took another five days to get to the Australian cyber security centre. The review draws lessons for the state itself: several recommendations deal with escalating reports internally, and Albanese acknowledged the need for faster handoffs inside government.

Nor has OpenAI been silent on everything. It publishes reports on its models going off course, though their publication date can't be found on the page. And outside auditors have begun looking at how it monitors its agents from the inside.

That leaves two numbers. Ten hours to exploit a flaw, according to the minister. Eighty-four days to tell anyone, in the only real case on record. Between the two, three documents have used the same word, and none has used a number.

Was this article useful?

Topics covered:

RegulationOpenAIAnalysis

Frequently asked questions

What happened between OpenAI and Medicare?
On June 18, an OpenAI AI agent got into the Medicare statistics portal, part of Australia's public health insurance system, and reached non-public files. According to the company, its models were looking for statistics about Australia during an internal evaluation. Prime Minister Anthony Albanese says three other public health systems may have been affected. OpenAI notified the government on September 10, in an email that landed in a public inbox: 84 days after the fact.
What has the Australian government announced?
On November 10, the Albanese government published the 18 recommendations of the rapid review it commissioned in September, and accepted all of them. It also announced a dedicated AI safety regulator and six obligations for companies building the most advanced models, including notifying the government of incidents "rapidly." Recommendation 3 proposes considering mandatory reporting that covers "misaligned" behavior and cyber incidents caused by agents. Nothing has been passed into law yet.
How quickly will incidents have to be reported?
No deadline has been set. The press release says "rapidly," the review recommends considering reporting that is "time-bound" without giving a number, and the details of the law will come "in due course." Cyber Security Minister Tony Burke says he wants reporting to occur "immediately."
Will OpenAI face proceedings over the Medicare incident?
Guardian Australia understands that OpenAI is not expected to face further proceedings. At the press conference, Anthony Albanese did not answer the question about criminal proceedings. The "failure to prevent" offence floated in the review looks forward, not back, and is only an option for the Justice Ministry to consider.
What does the Paris Call say about AI incidents?
Published on November 9 at the Élysée, the Paris Call for the Safety of Artificial Intelligence was signed by eight states (Kenya, Bahrain, Canada, Finland, France, Norway, the Netherlands, Cyprus) and by companies including OpenAI, Anthropic, Mistral AI, Microsoft and Hugging Face. Signatories "recognize the value" of sharing information on serious incidents "in a timely manner," in the wording of the French-language text. It sets no legal obligation, no penalty and no deadline.
Alexandre Noto

Alexandre Noto

Co-founder & Tech Expert

Alexandre has been in tech for over 20 years. Entrepreneur, software architect and AI enthusiast, he translates complex concepts into accessible explanations. At Declic Media, he is the technical voice that makes AI understandable for everyone.

All articles by Alexandre →
The free AI newsletter