Cohere Command R+ (08-2024)

À propos de Cohere

Fondée 2019 · Toronto, Canada

Cohere is an enterprise-focused LLM lab founded in 2019 by Aidan Gomez (co-author of 'Attention Is All You Need'), Ivan Zhang and Nick Frosst. Cohere targets enterprise customers with deployment options spanning Cohere's hosted API, AWS Bedrock, Azure, Oracle Cloud and on-prem (Cohere North). Command R+ is Cohere's flagship enterprise LLM, scaling the Command R recipe up to 104B parameters and positioned against GPT-4, Claude 3 Opus and Mistral Large. The 08-2024 refresh shipped August 2024 with major gains on math, code, reasoning, citation accuracy and reduced refusals. Investors include Nvidia, Cisco, Salesforce and Oracle, with total funding above $1B and a 2024 valuation around $5.5B.

Architecture

Decoder-only Transformer optimised for RAG and tool use

Command R+ 08-2024 is a 104B parameter dense decoder-only transformer — the larger sibling of Command R, scaled up with the same architectural recipe. It uses grouped-query attention (8 KV heads), RoPE positional embeddings (extended for 128K context), SwiGLU activations, RMSNorm and no biases, with the same 256,000-token multilingual BPE tokeniser as Command R and the Aya family. The 08-2024 refresh kept the architecture identical to the April 2024 release but delivered substantial improvements in math, code, reasoning, instruction following and structured outputs while reducing spurious refusals at unchanged pricing. Post-training emphasises citation-grounded RAG (structured `<co: 0>` citation tokens), parallel multi-step tool calling with chain-of-thought planning, and multilingual instruction following across 10+ business languages. Open weights released on Hugging Face under CC-BY-NC 4.0 for research; commercial production via Cohere API or marketplaces.

Paramètres

104B (dense)

Contexte

128 000 tokens

Capacités

• 104B dense parameters — Cohere's frontier enterprise model
• Top-tier RAG quality with structured inline citations
• Highly capable multi-step tool use with parallel calls and error recovery
• 128K context window
• Strong multilingual: 10+ business languages including Arabic, Japanese, Korean
• Open weights (CC-BY-NC) at 104B — largest open RAG-tuned model at release
• Hosted on Cohere API, AWS Bedrock, Azure, Oracle Cloud
• Best for: production enterprise RAG, complex tool-using agents, multilingual content ops, regulated-industry deployments.

Entraînement et licence

Trained on trillions of tokens of multilingual web data, code, math, books, and Cohere-curated synthetic RAG and tool-use traces. Knowledge cutoff approximately early 2024. Post-training combines supervised fine-tuning with preference optimisation; specific methods and data are proprietary.

Licence: Cohere Terms of Service for the hosted API. Open weights on Hugging Face under CC-BY-NC 4.0 (research only). Commercial production via Cohere API, AWS Bedrock, Azure, Oracle Cloud, or licensed on-prem deployment under Cohere North.

Tests de sécurité: Internal Cohere safety evaluation against the Cohere Acceptable Use Policy with model card disclosures. The 08-2024 refresh specifically targeted reduced over-refusal on benign questions.