CoinDesk Consensus

Coinbase says ChatGPT, a popular artificial intelligence tool, does not meet accuracy requirements and will not be integrated into its security review process.

Coinbase used ChatGPT to test the security standards of 20 unnamed ERC-20 tokens. Testing showed the tool to be “promising in its ability to quickly assess smart contract risk.”

However, when comparing the ChatGPT results to the Coinbase security team’s manual review, the machine returned 8 incorrect answers. Five of which he was the worst case.

A breakdown of these errors showed that ChatGPT incorrectly labeled high-risk assets as low-risk. According to Coinbase, “underestimating the risk score is far more harmful than overestimating it.”

Coinbase Security Team Said We first taught ChatGPT how to do security analysis using that format. However, the machine incorrectly categorized these risks due to its inability to recognize “when you lack the context to perform robust security analysis.”

On top of that, ChatGPT’s responses were also inconsistent when asked the same question over and over again. Coinbase says AI tools are “affected by comments in the code, sometimes appearing to default to comments rather than functional logic.”

Coinbase concluded:

“ChatGPT has high hopes for its ability to quickly assess the risk of smart contracts, but does not meet the accuracy requirements for integration into Coinbase’s security review process.”

Meanwhile, this experiment represents another example of potential applications for ChatGPT and its latest version, GPT-4. AI tools have become popular for their human-like responses and high scores on major exams.

Crypto enthusiasts also highlight their ability to review Ethereum smart contracts and identify vulnerabilities and ways to exploit the code.Coinbase Director Connor Grogan I got it This was a Twitter thread in which Machine “highlighted a number of security vulnerabilities and pointed out surface areas where contracts could be exploited.”

Some blockchain developers believe this tool will help them in their work, but do not believe it will replace humans.

By Jules

Leave a Reply