BLOOM: The World's Largest Open Multilingual Language Model

Large language models (LLMs) have revolutionized AI research, offering powerful, general-purpose tools capable of handling diverse language tasks. However, access to these models has been largely restricted to a few industrial labs with the resources and exclusive rights to develop and utilize them. This has created a significant barrier for academic researchers, nonprofits, and smaller companies.

BLOOM aims to change this. It's the first multilingual LLM trained in complete transparency, representing the culmination of a massive collaborative effort. With 176 billion parameters, BLOOM generates text in 46 natural languages and 13 programming languages. For many of these languages, including Spanish, French, and Arabic, BLOOM is the first model of its scale ever created.

The Collaborative Effort:

The development of BLOOM involved over 1000 researchers from 70+ countries and 250+ institutions. The training process, lasting 117 days (March 11 - July 6), utilized the Jean Zay supercomputer in France, thanks to a significant compute grant from CNRS and GENCI.

Accessibility and Openness:

BLOOM's source code, weights, and training data are publicly available. Researchers can download, run, and study the model to understand its performance and internal workings. Anyone agreeing to the Responsible AI License can use and build upon the model, leveraging the Hugging Face ecosystem for easy integration.

Future Developments:

The BLOOM project is ongoing. Future work includes improving its instruction-following capabilities, adding more languages, creating a more compact version without sacrificing performance, and exploring more complex architectures. The goal is to foster a thriving community around BLOOM, continuously expanding its capabilities and applications.

Key Features:

• Multilingual: Supports 46 natural languages and 13 programming languages.
• Open-Source: Code, weights, and training data are publicly available.
• Large-Scale: 176 billion parameters.
• Collaborative: Developed by a global team of researchers.
• Accessible: Easy to use via Hugging Face and a planned inference API.

BLOOM represents a significant step towards democratizing access to advanced language models, empowering researchers and developers worldwide to push the boundaries of AI research and development.