Immunefi spammed by ChatGPT-generated web3 bug bounty reports

Web3 bug bounty platform Immunefi has been inundated with ChatGPT-generated web3 security reports since OpenAI launched the tool in November, according to a new report.

Immunefi said the bug reports seemed genuine initially, but on closer inspection, none of the submissions managed to discover any real vulnerabilities. The underlying claims in the reports were “nonsensical,” submitted by individuals “totally lacking in web3 security skills who were hoping that web3 bug bounty hunting would be as easy as entering in some ChatGPT prompts,” Immunefi added.

Permanent bans that Immunefi placed on any accounts found using the tool now account for 21% of all banned accounts.

“The industry must thoroughly assess every tool it plans on including in its security arsenal. At the moment, ChatGPT is not a reliable one. For web3 security, namely vulnerability discovery, the technology is just not there,” Immunefi founder and CEO Mitchell Amador said.

Web3 ChatGPT survey

Immunefi conducted a broader ChatGPT web3 security survey as part of the report, finding that 76.4% of whitehats have used the tool in their web3 security practices, with 36.7% using it as part of their daily workflow. 

Some 52.1% of respondents had a positive sentiment toward ChatGPT, 38.8% neutral and 9.1% negative, with some 68.4% recommending the tool to web3 security colleagues. Some 73.9% of the whitehats saw ChatGPT as suitable for education, 60.6% for smart contract auditing and 46.7% for vulnerability discovery. 

However, concerns arose among 64.2% of the group regarding the technology's limited accuracy in identifying vulnerabilities, with 61.2% highlighting its lack of specific knowledge and challenges in handling large-scale audits.

While 52.1% of whitehats said the general use of ChatGPT presents security concerns, with its potential for phishing, scams and social engineering, the majority of the community (75.2%) still believes that it has the potential to improve web3 security research. To mitigate the risks, the community said there was a need for strong governance frameworks, strict access controls and ongoing monitoring.

Immunefi claims to have paid out more than $80 million in bounties and saved over $25 billion in user funds across protocols like Chainlink, The Graph, Synthetix and MakerDAO. The highest bounty facilitated by Immunefi being a $10 million award for a vulnerability discovered in Wormhole’s cross-chain protocol. 

Earlier this month, Immunefi found there were 63% more crypto attacks last quarter compared to a year ago.