Mistral AI and NVIDIA have unveiled Mistral NeMo, a cutting-edge 12 billion parameter language model that boasts a 128,000 token context window and claims state-of-the-art performance in reasoning, world knowledge, and coding accuracy for its size category.
Mistral NeMo Release
On July 18, 2024, Mistral AI and NVIDIA jointly announced the release of Mistral NeMo, a state-of-the-art language model developed through their collaborative efforts. This 12 billion parameter model represents a significant advancement in AI technology, combining Mistral AI’s expertise in training data with NVIDIA’s optimized hardware and software ecosystem. The model was trained on the NVIDIA DGX Cloud AI platform, utilizing 3,072 H100 80GB Tensor Core GPUs, which showcases the cutting-edge infrastructure behind its development.
Key Features Overview
Designed for high performance across various natural language processing tasks, the model excels in generating text, summarizing content, translating languages, and analyzing sentiment. Its 128K token context window allows for processing extensive and complex information more coherently. The introduction of Tekken, a new tokenizer based on Tiktoken, offers approximately 30% more efficient compression for source code and several major languages, with even greater gains for Korean and Arabic.Additionally, the model’s quantization awareness during training enables FP8 inference without compromising performance, crucial for efficient deployment in enterprise settings.
Comparison with Other Models
Mistral NeMo 12B demonstrates impressive performance compared to other models in its size range. According to benchmarks, it outperforms both Gemma 2 (9B) and Llama 3 (8B) in accuracy and efficiency. The model’s pricing is competitive at $0.3 per 1 million input & output tokens, positioning it favorably against larger models like GPT-4 (32k context) and Mixtral 8x22B, which are significantly more expensive. Mistral NeMo’s 128K context window and advanced tokenization with Tekken give it an edge in handling long-form content and multilingual tasks, surpassing the Llama 3 tokenizer in text compression for about 85% of all languages.
Accessibility and Deployment
The model weights for Mistral NeMo are available on HuggingFace for both base and instruct versions, allowing developers to easily access and implement the technology. It can be used with mistral-inference and adapted with mistral-finetune tools. For enterprise deployment, Mistral NeMo is packaged as an NVIDIA NIM inference microservice, accessible through ai.nvidia.com. Designed to run on a single NVIDIA L40S, GeForce RTX 4090, or RTX 4500 GPU, the model brings powerful AI capabilities directly to business desktops, making it highly accessible for various organizations.
Potential Applications
Designed for versatility, the model can be applied to a wide range of tasks including enterprise-grade AI solutions, chatbots, and conversational AI systems. Its multilingual capabilities make it particularly useful for global businesses and organizations dealing with diverse language requirements. Additionally, the model’s strong performance in coding accuracy positions it as a valuable tool for software development and code generation tasks. The combination of a large context window and advanced reasoning capabilities also makes Mistral NeMo well-suited for complex text analysis, summarization, and research applications across various industries.
Source: Perplexity
