mxbai-embed-large-v1

Σχετικά με Mixedbread AI

Ιδρύθηκε 2023 · Berlin, Germany

Mixedbread AI (also stylised mxbai) was founded in 2023 in Berlin by Sean Lee, Aamir Shakir, Julius Lipp and Rui Huang with the goal of building best-in-class open-source retrieval and embedding models. The team released several iterations of the mxbai-embed and mxbai-rerank series under Apache 2.0 licence on Hugging Face and is widely cited as one of the few well-funded open-weights embedding labs alongside Jina AI and Nomic AI. mxbai-embed-large-v1 launched in March 2024 and immediately ranked at the top of the MTEB English leaderboard among models under 1B parameters, while remaining fully open under Apache 2.0. The company raised a seed round in 2024 from BlueYard Capital and angel investors and offers a hosted API as a paid product complementing the free open weights.

Αρχιτεκτονική

Transformer bi-encoder with AnglE loss and Matryoshka representation learning

mxbai-embed-large-v1 is a 335M-parameter Transformer bi-encoder built on top of the bert-large-uncased backbone (24 layers, 1024 hidden dim, 16 heads). It outputs a 1024-dim vector and is trained for English text embedding using the AnglE loss (Angle-Optimized Text Embedding) plus contrastive InfoNCE and a curated mix of supervised text pairs from NLI, MS MARCO, HotpotQA, FEVER, NQ and SQuAD. The model supports Matryoshka representation learning, meaning the leading 64 / 128 / 256 / 512 / 768 dimensions are independently meaningful and can be truncated for storage savings with minimal quality loss. Maximum input length is 512 tokens, and inputs longer than this must be chunked. The training emphasised generalisation rather than benchmark over-fitting, and the model is reported to remain competitive without any prompt engineering. Weights are Apache-2.0 licensed and the model runs efficiently on consumer GPUs.

Παράμετροι

335M

Περιεχόμενο

512 tokens

Δυνατότητες

• Top-tier MTEB English score for an open-weights 335M model
• 1024-dim vectors with Matryoshka truncation to 64/128/256/512 dims
• AnglE loss for improved similarity isotropy
• Open weights under Apache 2.0 with no use restriction
• Runs on a single 8 GB GPU or in CPU mode for low-volume tasks
• Strong on retrieval, clustering and STS benchmarks
• Best for: open-source RAG, on-premise embedding pipelines, cost-sensitive SaaS

Εκπαίδευση & άδεια

Supervised training on a curated mix of English text pairs from NLI, MS MARCO, HotpotQA, FEVER, Natural Questions and SQuAD, plus contrastive negatives mined from large web corpora.

Άδεια: Apache 2.0 for code and weights; commercial use permitted without restriction.

Δοκιμές ασφάλειας: No formal red-team report; embeddings are not subject to content filters.