MiniMax unveils MiniMax-01 Series 2 with 456B-parameter AI model

· 2 min read
MiniMax

MiniMax has announced the release of the MiniMax-01 Series 2, which includes advancements in their AI model lineup, particularly the MiniMax-Text-01. This update highlights their commitment to pushing boundaries in AI technology, focusing on large-scale language models and practical applications.

The MiniMax-Text-01 is a state-of-the-art Mixture of Experts (MoE) language model with 456 billion parameters, 45.9 billion of which are activated per token. It employs a hybrid attention mechanism combining Lightning Attention and Softmax Attention to optimize performance. The model supports extensive context lengths, with a training capacity of up to 1 million tokens and inference capabilities reaching 4 million tokens. This makes it suitable for tasks requiring deep contextual understanding and long-form text processing. Additionally, its use of Rotary Position Embedding (RoPE) enhances positional encoding for half the attention head dimensions, ensuring efficient processing of complex data structures.

Key specifications include:

  1. 80 layers with alternating attention mechanisms.
  2. 32 experts in the MoE framework, utilizing top-2 routing.
  3. A hidden size of 6144 and a vocabulary size of 200,064 tokens.

The MiniMax-01 Series 2 models demonstrate competitive performance against other leading AI systems like Qwen and DS3, excelling in benchmarks for long-context comprehension. However, some challenges remain in instruction adherence and coding tasks, areas where further refinement is anticipated.

The company behind this innovation, MiniMax, is a Chinese AI startup that has gained significant traction with its consumer-facing app "Talkie," a virtual companion platform that has surpassed competitors like Character.AI in popularity. MiniMax's focus on both enterprise-level AI solutions and consumer applications underscores its versatility in addressing diverse market needs.

💡
Try MiniMax-01 on Hailuo AI

This release positions MiniMax as a formidable player in the AI landscape, particularly as it continues to innovate in large-scale models and practical AI deployments. Industry observers will be watching closely to see how these advancements impact both technical applications and user experiences globally.

Source