AI Agents Are Getting Better at Writing Code—and Hacking It as Well

The latest artificial intelligence models are not only remarkably bully astatine package engineering—new investigation shows they are getting ever-better astatine uncovering bugs successful software, too.

AI researchers astatine UC Berkeley tested really good nan latest AI models and agents could find vulnerabilities successful 188 ample unfastened root codebases. Using a new benchmark called CyberGym, nan AI models identified 17 caller bugs including 15 antecedently unknown, aliases “zero-day,” ones. “Many of these vulnerabilities are critical,” says Dawn Song, a professor astatine UC Berkeley who led nan work.

Many experts expect AI models to go formidable cybersecurity weapons. An AI instrumentality from startup Xbow presently has crept up nan ranks of HackerOne’s leaderboard for bug hunting and presently sits successful apical place. The institution precocious announced $75 cardinal successful caller funding.

Song says that nan coding skills of nan latest AI models mixed pinch improving reasoning abilities are starting to alteration nan cybersecurity landscape. “This is simply a pivotal moment,” she says. “It really exceeded our wide expectations.”

As nan models proceed to amended they will automate nan process of some discovering and exploiting information flaws. This could thief companies support their package safe but whitethorn besides assistance hackers successful breaking into systems. “We didn't moreover effort that hard,” Song says. “If we ramped up connected nan budget, allowed nan agents to tally for longer, they could do moreover better.”

The UC Berkeley squad tested accepted frontier AI models from OpenAI, Google, and Anthropic, arsenic good arsenic unfastened root offerings from Meta, DeepSeek, and Alibaba mixed pinch respective agents for uncovering bugs, including OpenHands, Cybench, and EnIGMA.

The researchers utilized descriptions of known package vulnerabilities from nan 188 package projects. They past fed nan descriptions to nan cybersecurity agents powered by frontier AI models to spot if they could place nan aforesaid flaws for themselves by analyzing caller codebases, moving tests, and crafting proof-of-concept exploits. The squad besides asked nan agents to hunt for caller vulnerabilities successful nan codebases by themselves.

Through nan process, nan AI devices generated hundreds of proof-of-concept exploits, and of these exploits nan researchers identified 15 antecedently unseen vulnerabilities and 2 vulnerabilities that had antecedently been disclosed and patched. The activity adds to increasing grounds that AI tin automate nan find of zero-day vulnerabilities, which are perchance vulnerable (and valuable) because they whitethorn supply a measurement to hack unrecorded systems.

AI seems destined to go an important portion of nan cybersecurity manufacture nonetheless. Security master Sean Heelan recently discovered a zero-day flaw successful nan wide utilized Linux kernel pinch thief from OpenAI’s reasoning exemplary o3. Last November, Google announced that it had discovered a antecedently chartless package vulnerability utilizing AI done a programme called Project Zero.

Like different parts of nan package industry, galore cybersecurity firms are enamored pinch nan imaginable of AI. The caller activity so shows that AI tin routinely find caller flaws, but it besides highlights remaining limitations pinch nan technology. The AI systems were incapable to find astir flaws and were stumped by particularly analyzable ones.