Gov proposes AI safety regulator

Detailed summary

Following incident where an AI agent accessed a Medicare data portal.

The federal government is proposing to set up an AI safety regulator to manage risks posed by frontier models and AI agents, after an incident that affected Medicare and state government authorities.

The regulator’s existence and remit is to be enabled by forthcoming “Australian AI standards legislation”, a group of ministers said in a joint statement.

“Strong mandatory safety and security protections will be legislated through the AI Standards and apply to all frontier AI companies that wish to operate in Australia,” the ministers said.

“The government will propose these protections be enforced by a new targeted regulator for AI safety, which will work with the Australian AI Safety Institute to provide technical expertise and oversight as global standards evolve.

“The regulator would be able to enforce significant penalties for AI companies that fail to comply with mandatory obligations that will help keep Australia safe.”

The proposal for a new regulator is the latest measure to come out of an investigation into an incident in June - disclosed late September - where an OpenAI agent accessed public and non-public files associated with an older Medicare data portal.

In addition to new rules for the developers of frontier AI models and agents, the incident has also cast a light on government departments and older systems they still run.

Home Affairs has ordered a government-wide stocktake of systems by March next year, with the aim of identifying configuration weaknesses that could be susceptible to exploitation by AI agents.

Now, under rules that would be enforceable by the AI safety regulator, the developers of AI models and agents would have to “allow independent evaluation of their models” and offer “rapid” disclosure of incidents of rogue behaviour.

Opening of the article, excerpted from iTnews. To read the rest, open the article on the publisher's site.