The latest iteration of Meta’s cutting-edge open-source large language model (LLM), Llama 3, has been unveiled.
According to the IT giant, Llama 3 breaks past industry-leading models such as GPT-3.5 in real-world circumstances, setting new performance records.
In a blog post announcing the release, Meta stated, “With Llama 3, we set out to build the best open models that are on par with the best proprietary models available today.”
Versions with 8 billion and 70 billion parameters are the first Llama 3 models available. According to Meta, scientific papers summarizing the work will be released in the following months, and the company’s researchers are continuing to train larger models with more than 400 billion parameters.
Substantial resources have been devoted to gathering high-quality training data, scaling up distributed training, refining the model architecture, and developing novel techniques for instruction fine-tuning. Llama 3 has been in development for more than two years.
Meta’s instruction fine-tuned model with 70 billion parameters fared better in human evaluations across 12 significant usage situations, such as coding, reasoning, and creative writing than GPT-3.5, Claude, and other LLMs of similar scale. Additionally, the company’s pre-trained model with 8 billion parameters breaks previous records for everyday LLM evaluation tasks:
According to Meta, “We think these are the best open source models of their class, period.”
The tech company is promoting an open ecosystem around AI development by providing the models through an “open by default” strategy. Llama 3 will be available to all leading cloud service providers, model hosts, hardware producers, and AI platforms.
“With the global shift towards AI regulation, the launch of Meta’s Llama 3 model is notable,” stated Victor Botev, CTO and co-founder of Iris.ai. By embracing transparency through open sourcing-, Meta is in line with the increasing focus on ethical development and responsible AI methods.
Additionally, open models allow for broader community education by allowing for the examination of different approaches and facilitating insights into progress. This transparency also feeds back into the creation and implementation of regulations.
A revised set of AI safety tools, such as CyberSec Eval for evaluating possible abuse and Llama Guard for risk classification—accompanies Meta’s most recent models. A brand-new element dubbed Code Shield has also been added to filter unsafe code recommendations during inference.
“But it’s crucial to keep things in perspective—just because a model is open-source doesn’t mean it’s ethical AI,” Botev said.
AI’s challenges necessitate a comprehensive strategy to address problems like algorithmic bias, data privacy, and societal effects—all of which are major areas of concern for newly developing AI policies around the globe.
Although open projects such as Llama 3 encourage examination and cooperation, their actual influence depends on a comprehensive strategy for adhering to AI governance and integrating ethics into AI systems’ lifecycles. While Meta’s ongoing work on the Llama model is a positive step, ethical AI necessitates a consistent commitment from all parties involved.
With Llama 3, Meta claims to have taken a “system-level approach” to the responsible development and application of AI. The company emphasizes that developers should apply their own input/output filtering in accordance with their application’s requirements, even though the models have passed comprehensive safety testing.
Meta AI, the company’s end-user product that integrates Llama 3, asserts that the new models have made it the best AI assistant in the world. Through Facebook, Instagram, WhatsApp, Messenger, and the internet, users can utilize Meta AI for general questions, learning, creativity, and productivity.
An early look at multimodal Meta AI versions with vision capabilities will soon be available for Meta’s Ray-Ban smart glasses.
Even if Llama 3 has accomplished a great deal, some in the AI community are dubious of Meta’s open approach and its claim to be “for the good of society.”
However, Meta’s release raises the bar for publicly available LLMs once again, coming only a day after Mistral AI’s Mixtral 8x22B set a new standard for open-source models.
