Finally, a Replacement for BERT Note This is a cross-post of the announcement blog post posted on the 🤗 HuggingFace blog. TL;DR This blog post introduces ModernBERT, a family of state-of-the-art encoder-only models representing improvements over older generation encoders across the board, with a 8192 sequence length, better downstream performance and much faster processing. ModernBERT is available as a slot-in replacement for any BERT-like models, with both a base (149M params) and large (...
