Hugging Face shows how to finetune multi-vector embedding models, beating general retrievers in 14.5 hours on one GPU

Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers

Sentence Transformers v6.0 introduces MultiVectorEncoder, a new model type for ColBERT-style late interaction retrieval. This blog walks through finetuning a multi-vector model that beats general-purpose retrievers on your own data. The author trained mLateOn-medical on a single RTX 3090 in 14.5 hours, and it outperformed every general-purpose retrieval model (dense, sparse, lexical) on a medical retrieval benchmark. The post covers model initialization, dataset format, loss functions, training arguments, evaluators, and the Trainer class, including multi-dataset training.