The AI Era Is Creating a Bug Hunting Arms Race
“Nation-state issues are very serious and very real, but criminal actors still make up the vast majority of incidents that organizations face and many of those incidents are quite serious,” adds Hultquist. “The use of zero-day by criminal actors has been quite limited, and those who do use them tend to be very successful. So I think we shouldn’t underestimate the impact of more criminals with zero-day on their hands.”
For researchers who make money from bug hunting, times are changing. Command-line tool Curl ended its bug bounty program (run through third-party service HackerOne) in January after being flooded with low-quality AI-generated submissions.
“We concluded the hard way that a bug bounty provides too strong an incentive for people to find and fix ‘issues’ in bad faith that cause overload and abuse,” the group wrote at the time, adding that “we always appreciate and value valid vulnerability reports.”
Last week, Linux creator and lead developer Linus Torvalds wrote that the popular Linux security mailing list has become “almost entirely unmanageable” due to high volume and duplicate AI bug reports.
In April, however, Daniel Stenberg, founder and lead developer of Curl, said in a LinkedIn post that the quality of submissions had improved. “Over the past few months, we have stopped receiving security reports about AI errors in the curl project,” he wrote. “Instead, we are receiving an ever-increasing number of very good safety reports, almost all of them made with the help of AI. They are being submitted at a frequency never before seen and are putting us under a heavy load.”
And in late April, Google announced that it was overhauling its vulnerability reward programs for Chrome and Android and reducing payouts for some classes of bugs, while increasing others.
“As the security research landscape evolves with AI, we are making changes to our programs to ensure we reward the most difficult and impactful vulnerabilities in our products,” the company wrote.
“I think 90th percentile bug hunters with special skills will still be able to get results and get payments from big companies,” says Jonathan Dunn, a cardiologist and also a bug hunter. “But even with AI, we also need to strongly incentivize ethical researchers to seek information about public infrastructure and other critical systems that otherwise would not receive enough attention from advocates. »
For now, most organizations seem willing to offer every solution imaginable to solve the problem (and benefits) of accelerated bug discovery. “It changes the dynamics of the bug-hunting industry, but it still requires human time,” says Alex Zenla, chief technology officer at cloud security company Edera.
Earlier this month, Anthropic launched a HackerOne bug bounty for researchers to submit their findings on the company’s own systems and Claude AI models. However, more and more researchers say that structural defenses are necessary to address the accelerated discovery of vulnerabilities. In other words, they design digital solutions for different classes of vulnerabilities that eliminate them or make them significantly less exploitable in practice.
“There’s no fixing your way out of this,” says Niels Provos, a longtime engineer and security researcher. “You need to create an infrastructure that makes as many bugs as unnecessary as possible.”
