How Meta’s new AI Models Will Shake up the Global AI Race

The landscape of AI development is highly competitive, with prominent corporations striving to dominate a forecasted US$1.3tn sector by 2032.

Amid this dynamic environment, Meta takes an innovative step forward to solidify its leading position by unveiling Llama 4, a significant advancement in its line of open-source large language models (LLMs).

This release introduces Scout and Maverick, two models designed to enhance enterprise-level digital transformation initiatives.

Initially launched in 2023, the Llama family comprises LLMs with parameter sizes ranging from 1 billion to 405 billion.

Parameters, in this context, are crucial as they dictate the transformation of input data into output predictions, forming the backbone of the neural network’s learning capability.

The Llama series — including the newly introduced models — targets capabilities crucial for digital innovation such as content creation, visual understanding, and multimodal processing.

The development of Llama models represents a substantial evolution in AI systems, enabling the integration and conversion of an array of data formats — text, video, images and audio — seamlessly within a single model.

Advancements in model architecture

Meta's commitment to the open-source movement is further strengthened with Llama 4 Scout and Maverick, empowering the larger tech community to contribute toward and leverage these cutting-edge technologies.

According to Meta, Scout “dramatically increases the supported context length from 128K in Llama 3 to an industry leading 10 million tokens,” – context length referring to the amount of text a model can process in a single operation, measured in tokens, which are word fragments used by AI systems.

The innovation doesn't stop at Scout and Maverick.

Meta also offers a glimpse of Llama 4 Behemoth, a larger model that not only showcases the scale of Meta's AI ambitions but is set to underpin future models with its educational capacity.

Llama 4 Scout stands with 17 billion active parameters supported by 16 experts — specialised components dedicated to processing specific tasks or data types — resulting in 109 billion total parameters.

This configuration allows Scout to operate efficiently on a single Nvidia H100 GPU, aided by Int4 quantization that conserves computational resources.

Breaking boundaries with contextual processing

Scout revolutionises the context length capabilities seen in its predecessor, Llama 3, expanding from 128K to a groundbreaking 10 million tokens.

In AI terms, context length is the measure of text that can be processed in a single operation, fundamentally enhancing the model’s application in complex data processing tasks.

On the other hand, Llama 4 Maverick includes an imposing 400 billion total parameters, supported by 128 experts, all while utilising a sophisticated mixture-of-experts (MoE) architecture.

This design ensures that only the necessary parameters are engaged per operation, enhancing computational efficiency during both training and inference stages.

“In MoE models, a single token activates only a fraction of the total parameters,” the company says.

“MoE architectures are more compute efficient for training and inference and, given a fixed training FLOPs budget, delivers higher quality compared to a dense model”.

Enabling enhanced multimodality

Meta has incorporated native multimodality within the Llama 4 series, leveraging early fusion techniques to amalgamate text, image and video data seamlessly into the model's framework.

This approach not only enhances pre-training but also aligns with Meta’s commitment to robust AI model innovation.

Using its proprietary MetaCLIP technology, the company has boosted the model’s vision encoding capability, training it separately with a static Llama model for optimal adaptation.

Meta has extended the Llama models for public access via llama.com and Hugging Face, broadening availability across diverse tech platforms.

This openness, as Meta suggests, is essential for fostering innovation globally.

“We continue to believe that openness drives innovation and is good for developers, good for Meta and good for the world,” Meta says in its announcement.

In the midst of growing competition from noteworthy tech companies like OpenAI and Google, who have also introduced multimodal systems, Meta distinguishes itself by prioritising open-source frameworks and implementing robust safeguards throughout the model development lifecycle.

“We aim to develop the most helpful and useful models while protecting against and mitigating the most severe risks,” Meta says.

“We built Llama 4 with the best practices outlined in our Developer Use Guide: AI Protections.

“As more people continue to use AI to enhance their daily lives, it's important that the leading models and systems are openly available so everyone can build the future of personalised experiences.”

Explore the latest edition of Technology Magazine and be part of the conversation at our global conference series, Tech & AI LIVE.

Discover all our upcoming events and secure your tickets today.

Technology Magazine is a BizClik brand