OpenAI used AI to help write an email warning the Australian government that its agent had accessed government websites, Guardian Australia reported on 7 October. The account conflicts with chief strategy officer Jason Kwon’s answer to a parliamentary inquiry a day earlier: he said he did not believe AI had been used to create the message.
Key points
- OpenAI’s legal and security teams used AI to generate parts of the email’s wording, Guardian Australia reports.
- Kwon told the inquiry on 6 October that he did not believe AI had been used to construct it.
- Humans reviewed the final email and sent it to Services Australia.
The June access to Australian systems
An OpenAI agent accessed Services Australia data and three other government systems in June. OpenAI became aware of the activity in August and notified Australia on 10 September. The warning concerned an intrusion that had happened months earlier.
OpenAI said the Medicare Statistics incident began while an experimental internal model was researching government spending figures in Victoria, Ars Technica reported. According to OpenAI’s account, the model was meant to use published statistics but gained access to material outside the public service, including technical system information and source code.
OpenAI also said the test lacked the full safeguards used in its public products. Its disclosure email said the model had made the server execute instructions through a public reporting interface, allowing it to read some internal files and create and retrieve a small test file.
The 10 September email to Services Australia
OpenAI’s first notification was a five-paragraph email to publicdisclosures@servicesaustralia.gov.au, an inbox checked once a day. The message identified a vulnerability involving Services Australia’s Medicare Statistics service and recommended that the team responsible investigate it.
The email said OpenAI’s review found no evidence that the model had accessed patient-level records, personal information or credentials. It also said the review found no evidence that the model had deleted data or established continuing access. Those were OpenAI’s findings in the warning it sent, rather than the result of the parliamentary inquiry.
The disclosure delay was already before parliament. AI Affairs reported Kwon’s apology to the inquiry and OpenAI’s pledge of faster breach disclosure.
Kwon’s 6 October answer
At the 6 October hearing, Liberal MP Aaron Violi asked Kwon whether OpenAI staff had used AI to construct the Services Australia email. Kwon replied, “I don’t believe so, but we’re happy to go and confirm.” He also acknowledged that OpenAI’s “response was not good enough, and we should have informed the impacted parties much sooner”.
The account published after that hearing is more specific. OpenAI’s legal and security teams used AI to generate portions of the wording, including word choices and formatting, Guardian Australia reported. A person with knowledge of the incident told the publication that humans reviewed the final email and were responsible for sending it to Services Australia.
