Qwen 2.5 72B

Σχετικά με Alibaba Cloud (Qwen team)

Ιδρύθηκε 2009 · Hangzhou, China

The Qwen (通义千问, Tongyi Qianwen) team is the LLM research group inside Alibaba Cloud, the cloud-computing arm of Alibaba Group founded in 2009. Alibaba started large-language-model research at the Damo Academy and Tongyi Lab; the Qwen series became publicly available in 2023. Releases include Qwen-7B (Aug 2023, first Chinese open-weight foundation model from a major cloud), Qwen-14B/72B (late 2023), Qwen 1.5 (Feb 2024), Qwen2 family (Jun 2024), Qwen2.5 family (Sep 2024 with 0.5B/1.5B/3B/7B/14B/32B/72B sizes plus Qwen2.5-Coder, Qwen2.5-Math, Qwen2.5-VL), Qwen3 family in 2025 introducing thinking mode, and Qwen2.5-Max as the closed-weight flagship. The Qwen team is led by Junyang Lin and has published over a dozen widely cited technical reports. Models are released under the Tongyi Qianwen LICENSE (commercial-friendly for most use cases but with a >100M MAU clause requiring a separate license). Qwen has become the dominant open-weight model family for Chinese-language tasks and has the largest derivative-model ecosystem on HuggingFace by download volume.

Αρχιτεκτονική

Decoder-only Transformer (dense, Grouped Query Attention)

Qwen2.5-72B was released by the Alibaba Qwen team in September 2024 as the flagship dense model of the Qwen2.5 family. It is a decoder-only Transformer with 72.7B parameters, 80 layers and Grouped Query Attention (GQA) with 64 query heads and 8 KV heads. The model was pretrained on a 18-trillion-token multilingual corpus emphasising Chinese, English and 27 other languages, plus heavy concentrations of math, code and long-form documents. Compared to Qwen2-72B (7T tokens), the 18T-token pretraining and improved data filtering pipeline meaningfully lifted knowledge, math and code performance. Qwen2.5-72B supports a 128K token (131,072) context window with YaRN scaling extension and ships in both base and Instruct variants. Post-training applied a multi-stage SFT pipeline on over 1M curated examples followed by Direct Preference Optimisation (DPO) and Group Relative Policy Optimisation (GRPO) for reasoning data. Qwen2.5-Coder-32B and Qwen2.5-Math-72B specialised siblings push code and math frontiers, and Qwen2.5-VL adds vision. Weights are released under the Tongyi Qianwen LICENSE (free commercial use unless >100M MAU). The Qwen ecosystem also publishes GGUF, AWQ, GPTQ quantised variants and is supported by vLLM, SGLang, llama.cpp, Ollama, MLX and Apple MLX.

Παράμετροι

72.7B (dense)

Περιεχόμενο

131.072 tokens

Δυνατότητες

• 72.7B dense parameters with Grouped Query Attention
• Pretrained on 18T multilingual tokens
• 128K context window with YaRN scaling
• Strong multilingual performance: Chinese, English, Japanese, Korean and 27 more
• Specialised siblings: Qwen2.5-Coder, Qwen2.5-Math, Qwen2.5-VL
• Function calling and JSON mode supported
• Open weights under Tongyi Qianwen LICENSE (commercial-friendly)
• Massive ecosystem: vLLM, SGLang, llama.cpp, Ollama, MLX, HuggingFace
• GGUF/AWQ/GPTQ quantised variants officially published
• Strong math performance (Qwen2.5-Math variant beats GPT-4o on competition math)
• Best for: open-weight Chinese/English chat, coding, on-prem enterprise, RAG.

Εκπαίδευση & άδεια

Pretrained on 18 trillion tokens of multilingual web text, code, books and scientific papers, with strong Chinese and English coverage. Post-training uses 1M+ curated SFT examples followed by DPO and GRPO. Data cutoff approximately mid-2024.

Άδεια: Tongyi Qianwen LICENSE: free commercial use for products with fewer than 100M monthly active users. >100M MAU requires a separate commercial license from Alibaba.

Δοκιμές ασφάλειας: Qwen applies SFT-based safety alignment plus DPO with safety preferences. The model filters Chinese politically sensitive topics; third-party safety evaluations have flagged below-average refusal rates on Western dual-use prompts.