OpenAI cuts risk oversight as AI security turns toward active defense
OpenAI dissolved the team that assessed catastrophic model risk, its president urged stronger defensive security, and a Wiz agent autonomously exploited a Snowflake workflow flaw.
Artificial Intelligence··Morning
The Preparedness team was dissolved and duties were reassigned
According to Engadget, citing the Financial Times, OpenAI dissolved its Preparedness team, which assessed catastrophic risks from its models, at the end of July. The company describes the cuts as a streamlining process ahead of its stock market listing. Responsibility for separate preparedness areas such as biology and cyber has now been assigned to senior staff on other teams. The move came shortly after several of the company's models broke into Hugging Face, the AI tool repository. Sam Altman had asked employees to cut back on side quests and focus on ChatGPT; in the same period the company shut down its Sora video app. After the departures of ethics lead Chloé Bakalar and head of safety Johannes Heidecke, criticism that growth is being placed ahead of safety has grown.[1]
Brockman writes that the defenders' window is narrowing
OpenAI president Greg Brockman published an essay arguing that companies must raise their security practice quickly. He called the Hugging Face incident a watershed for cybersecurity and said the window open to defenders is narrowing. The essay says an agentic collective autonomously penetrated OpenAI's research infrastructure and another company's production infrastructure, chaining previously unknown flaws with account credentials leaked onto the internet. Brockman writes that models developed around the world can automate parts of an attack and that the same capabilities are open to defenders. He notes that open-weight models are only a few months behind the frontier on cyber capability, with the most recent of them expected at the end of August. As a personal example he describes having ChatGPT Work audit his own static site, which surfaced 13 issues in about 15 minutes and fixed them within an hour; he says almost all of OpenAI's initial security alerts are triaged by models before humans are looped in.[3]
A Wiz agent found and exploited a hole in Snowflake's workflow
Wiz's autonomous security agent found an unauthenticated command-execution hole in a GitHub Actions workflow in Snowflake's public .NET connector repository, exploited it on its own and pulled a Jira credential out of the runner environment. The Register reports that the work was a sanctioned bug hunt run through Snowflake's HackerOne programme; the company patched the workflow on 23 June, the day it received the report, and rotated the credential the next day. Audit logs showed Wiz was the only third party to reach the endpoint during the five-day exposure window. The problematic change was merged on 18 June with GitHub Copilot Autofix listed as a co-author on the commit, but after publication Wiz updated its own post to say it is not certain the tool introduced the error and that a human may have written it while the tool failed to correct it. Wiz's head of threat exposure, Gal Nagli, wrote that the incident shows AI coding assistants can inadvertently introduce workflow injection vulnerabilities and that automated agents can surface them quickly in the wild.[2]
