Neural Features to Tackle AI Hallucinations

New neural features in AI systems could be the key to solving AI hallucinations and provide developers with a deeper understanding of neural networks.

Unidentified neural characteristics in artificial intelligence (AI) systems may be vital in resolving one of the industry’s most significant challenges: AI hallucinations.

As an unintended consequence of training complex language models to function as generative AI systems, “hallucinations” occur. Hallucinations ensue when AI models generate new, frequently erroneous information that is not present in their data set.

Dario Amodei, co-founder and chief executive officer of Anthropic, argues that the issue surrounding AI hallucinations may be remedied with the aid of the newly identified neural characteristics.

During a speech at Viva Tech Paris 2024, Amodei stated:

“Modern AI systems have hundreds of millions of neurons… We found 30 million features in our model. Features for security vulnerabilities or bias against certain kinds of groups… We’re able to now intervene in them and change them.”

One difficulty associated with hallucinations is that AI protocols remain oblivious to the erroneous output. Amodei explained, however, that the newly discovered features may render AI hallucinations obsolete.

“Reducing [artificial intelligence] hallucinations is among the main benefits of this new discovery.”

Anthropic has initiated research into the consequences of modifying the internal characteristics of its AI assistant to influence its behavior. Due to creating a prototype model with a specific feature, the AI “focused intensely on the Golden Gate Bridge.” As stated by Anthropic in a May 23 X post:

“The fact that we can find and alter these features within Claude makes us more confident that we’re beginning to understand how large language models really work.”

In May, Anthropic, a prominent AI development company, introduced Claude, an AI chatbot, to the European market. Since May 14, the AI assistant has been accessible in multiple European languages, including French, German, Spanish, and Italian, via its web application and iOS app.

AI development will depend on talent, not data: Amodei

Insufficiency of data has been the principal obstacle to the advancement of AI.

However, since the creation of synthetic data sets, the scarcity of developer talent has become the most significant obstacle rather than the volume of data. Amodei declared:

“Data will not be a bottleneck as some firms are starting to create synthetic data… AI advancement will mainly come down to talent…”

Synthetic data is promising for the scalability of artificial intelligence, according to Amodei. However, the ratio of synthetic to real-world data cannot be infinite, as this would produce synthetic data sets that are weaker and more exaggerated.

Investor interest in the AI sector has increased since the start of 2024. Elon Musk’s company successfully obtained a Series B funding round of $6 billion on May 27. By the end of 2024, this funding intends to position Grok as a competitor to ChatGPT.