Llama 4 brings 10M token context and MoE architecture with 3 new models

Meta has officially launched Llama 4, its latest series of AI models, on April 5, 2025. This release includes four models: Scout, Maverick, Behemoth, and L4emoth. These models are designed with a focus on multimodal capabilities and computational efficiency, leveraging a mixture-of-experts (MoE) architecture. The MoE approach allows the models to divide tasks into subtasks handled by specialized "expert" components, enhancing performance while reducing computational costs.

Scout and Maverick are currently available through Meta's platforms and partners like Hugging Face, while Behemoth remains in training. Scout operates with 109 billion total parameters and excels in processing long-context documents with a context window of up to 10 million tokens. Maverick, with 400 billion parameters, is optimized for general assistant applications, including creative writing and multilingual tasks. Behemoth, once released, is expected to be Meta's most powerful model yet, targeting STEM-related applications.

Meta has integrated Llama 4 into its AI assistant across WhatsApp, Messenger, and Instagram in 40 countries. However, its multimodal features are currently limited to English-speaking users in the U.S. The company also faces licensing restrictions in the EU due to regulatory concerns and requires special licenses for enterprises with over 700 million monthly active users.

Llama 4 represents Meta's response to increasing competition from companies like OpenAI and China's DeepSeek. While internal benchmarks suggest improvements over some competitors in specific tasks, the models still lag behind advanced offerings from Google and Anthropic in certain areas like reasoning. Meta has emphasized that this release marks the beginning of a new phase for the Llama ecosystem.

This launch highlights Meta's commitment to advancing open-source AI while navigating challenges posed by regulatory frameworks and competition from global AI leaders.

Source