Background: OpenAI is the company behind ChatGPT, the chatbot and agent used by hundreds of millions of people weekly. It's also one of the world's most valuable private companies and potentionally-soon-public-company. But there are concerns that OpenAI and its rivals are growing at a speed that is too fast for regulation... and too fast to comprehend its potential, existential risks.  

What happened: Now, Anthony Albanese has revealed that one of ChatGPT's rogue AI agents broke into the Medicare Statistics Reporting Service Portal back in June this year. According to OpenAI, it was researching public medicine spending as part of an internal test. While the agent was initially blocked from accessing the information, it went rogue and tried again and again... until it found a way to infiltrate the system.

What else: It wasn't just Medicare though... the AI agent also hit three other government systems. And the confusing part? OpenAI only noticed in August and then notified Services Australia on the 10th of September through a public inbox that's checked once per day.

What's the key learning?

💡 The most dangerous flaw in any system is the one nobody knows about... yet. In software, these are called zero-day vulnerabilities, because developers have had zero days to fix it before attackers can exploit it.  

💡 Fnding these vulnerabilities used to take elite hackers weeks, but AI can now go hunting for them at scale… 24/7. No sleep. No coffee breaks. No stopping at 3pm on Friday for work drinks. In fact, when Anthropic prepared to launch Claude Mythos, they found thousands of high-risk flaws across major operating systems and browsers.

💡 So this is a big reminder of how critical institutions, including government websites like Medicare as well as banks and hospitals face serious threats as AI continuously tests their security controls. And this ain't a good look for OpenAI's ongoing PR campaign either.