Google has announced the release of two updated production-ready Gemini models: Gemini-1.5-Pro-002 and Gemini-1.5-Flash-8B-Exp-0924. These models are designed for general performance across a wide range of text, code, and multimodal tasks, such as synthesizing information from large PDFs, answering questions about extensive code repositories, and creating content from hour-long videos.
Key improvements in these models include:
- A 7% increase in MMLU-Pro benchmark performance
- A 20% improvement in MATH and HiddenMath benchmarks
- Better performance in vision and code use cases (ranging from 2-7% improvement)
The models also offer more concise responses, with default output lengths 5-20% shorter than previous models, making them easier to use and more cost-efficient.
Additionally, Google is reducing the pricing for Gemini 1.5 Pro, with:
- A 64% price reduction on input tokens
- A 52% price reduction on output tokens
- A 64% price reduction on incremental cached tokens for prompts less than 128K tokens
These changes will be effective October 1st, 2024. The paid tier rate limits for 1.5 Flash and 1.5 Pro are also being increased to 2,000 RPM and 1,000 RPM, respectively.
The models are available for free via Google AI Studio and the Gemini API, and for larger organizations and Google Cloud customers, they are also available on Vertex AI. Developers can access these models to build a variety of applications, leveraging the 2 million token long context window and multimodal capabilities of Gemini 1.5 Pro.
Logan Kilpatrick
Google emphasizes its focus on building safe and reliable models, with improvements to the model's ability to follow user instructions while balancing safety. The company will continue to offer a suite of safety filters that developers may apply to Google’s models, allowing them to determine the configuration best suited for their use case.
