Baidu has made ERNIE 5.5 Flash Base downloadable as a starting point for developers training their own systems. The release contains the checkpoint reached at the end of pretraining, before instruction tuning and other later stages. Its mixture-of-experts design activates only part of the model for each token, and use is governed by a custom licence.
Baidu, the technology company, made ERNIE 5.5 Flash Base weights available on 5 November. The download gives developers a base for domain adaptation, instruction tuning and reinforcement learning research. Baidu identifies it as the checkpoint at the end of pretraining. Intermediate and subsequent training have been left for later stages, and the company plans to release a further-trained Flash version afterwards.[1], [2]
The architecture is a mixture of experts: a routing component chooses which specialist parts process each token. Its 280 billion parameters include 14 billion activated per token. Baidu describes 512 routed experts, with 10 selected alongside a shared expert. Most attention layers compress groups of tokens, while layers at regular intervals keep a more detailed view of earlier input.[1]
Pretraining used 33.5 trillion tokens, according to Baidu. Code material included repositories, commits, issue discussions and tests. The documentation describes serving through SGLang and vLLM, software used to run language models. The downloadable checkpoint is governed by ERNIE 5.5 Flash Model License v1.0; developers receive weights for further work under those terms.[1]
