Cohere Command R+ (08-2024)

О Cohere

Основана в 2019 · Toronto, Canada

Cohere is an enterprise-focused LLM lab founded in 2019 by Aidan Gomez (co-author of 'Attention Is All You Need'), Ivan Zhang and Nick Frosst. Cohere targets enterprise customers with deployment options spanning Cohere's hosted API, AWS Bedrock, Azure, Oracle Cloud and on-prem (Cohere North). Command R+ is Cohere's flagship enterprise LLM, scaling the Command R recipe up to 104B parameters and positioned against GPT-4, Claude 3 Opus and Mistral Large. The 08-2024 refresh shipped August 2024 with major gains on math, code, reasoning, citation accuracy and reduced refusals. Investors include Nvidia, Cisco, Salesforce and Oracle, with total funding above $1B and a 2024 valuation around $5.5B.

Архитектура

Decoder-only Transformer optimised for RAG and tool use

Command R+ 08-2024 is a 104B parameter dense decoder-only transformer — the larger sibling of Command R, scaled up with the same architectural recipe. It uses grouped-query attention (8 KV heads), RoPE positional embeddings (extended for 128K context), SwiGLU activations, RMSNorm and no biases, with the same 256,000-token multilingual BPE tokeniser as Command R and the Aya family. The 08-2024 refresh kept the architecture identical to the April 2024 release but delivered substantial improvements in math, code, reasoning, instruction following and structured outputs while reducing spurious refusals at unchanged pricing. Post-training emphasises citation-grounded RAG (structured `<co: 0>` citation tokens), parallel multi-step tool calling with chain-of-thought planning, and multilingual instruction following across 10+ business languages. Open weights released on Hugging Face under CC-BY-NC 4.0 for research; commercial production via Cohere API or marketplaces.

Параметры

104B (dense)

Контекст

128 000 токенов

Возможности

• 104B dense parameters — Cohere's frontier enterprise model
• Top-tier RAG quality with structured inline citations
• Highly capable multi-step tool use with parallel calls and error recovery
• 128K context window
• Strong multilingual: 10+ business languages including Arabic, Japanese, Korean
• Open weights (CC-BY-NC) at 104B — largest open RAG-tuned model at release
• Hosted on Cohere API, AWS Bedrock, Azure, Oracle Cloud
• Best for: production enterprise RAG, complex tool-using agents, multilingual content ops, regulated-industry deployments.

Обучение и лицензия

Trained on trillions of tokens of multilingual web data, code, math, books, and Cohere-curated synthetic RAG and tool-use traces. Knowledge cutoff approximately early 2024. Post-training combines supervised fine-tuning with preference optimisation; specific methods and data are proprietary.

Лицензия: Cohere Terms of Service for the hosted API. Open weights on Hugging Face under CC-BY-NC 4.0 (research only). Commercial production via Cohere API, AWS Bedrock, Azure, Oracle Cloud, or licensed on-prem deployment under Cohere North.

Тестирование безопасности: Internal Cohere safety evaluation against the Cohere Acceptable Use Policy with model card disclosures. The 08-2024 refresh specifically targeted reduced over-refusal on benign questions.