Qwen 3 235B Instruct

Σχετικά με Alibaba Cloud (Qwen team)

Ιδρύθηκε 2009 · Hangzhou, China

The Qwen team inside Alibaba Cloud has shipped one of the most prolific open-weight model lines in industry, starting with Qwen-7B (Aug 2023, first Chinese open-weight foundation model from a hyperscaler), through Qwen 1.5 (Feb 2024), Qwen2 (Jun 2024), Qwen2.5 (Sep 2024, sizes 0.5B-72B plus Coder/Math/VL), Qwen2.5-Max closed-weight flagship (Jan 2025) and the Qwen3 family in April-May 2025. Qwen3 introduced a uniform 'hybrid thinking' approach across the family, letting developers toggle between fast direct answers and long chain-of-thought reasoning in the same checkpoint. The 235B-A22B MoE flagship anchors the family alongside dense variants from 0.6B to 32B. The team is led by Junyang Lin and has published over a dozen technical reports. Models ship under the Apache 2.0 license starting with Qwen3, a substantial liberalisation over the earlier Tongyi Qianwen LICENSE. Alibaba Cloud, founded in 2009, is the largest cloud provider in China and hosts the Qwen models on its Model Studio service while also distributing weights freely on HuggingFace, ModelScope and GitHub.

Αρχιτεκτονική

Sparse Mixture-of-Experts Transformer (hybrid thinking)

Qwen3-235B-A22B is the flagship Mixture-of-Experts model of the Qwen3 family, released by Alibaba's Qwen team on 29 April 2025 with weights under Apache 2.0. The architecture is a Sparse MoE Transformer with 235 billion total parameters and 22 billion active per token (128 experts, 8 selected per token), 94 layers, and Grouped Query Attention (64 query heads / 4 KV heads). It supports a 128K native context window extended to 256K via YaRN scaling. The model was pretrained on approximately 36 trillion tokens spanning 119 languages with strong Chinese, English and code coverage, more than doubling the 18T-token corpus used for Qwen2.5. Pretraining was performed in three stages with progressively longer context lengths and improved data filtering. Post-training applied a four-stage pipeline: long-CoT cold start, RL on reasoning tasks, integration of thinking and non-thinking modes via mixed SFT, and a final general-purpose RL stage. The resulting model exposes hybrid thinking, toggled via the chat template, where the same checkpoint can produce either an R1-style chain-of-thought before the answer or a fast direct response. Qwen3-235B leads several open-weight benchmarks including AIME 2025, LiveCodeBench, ArenaHard and BFCL agent-eval as of release.

Παράμετροι

235B total, 22B active per token

Περιεχόμενο

262,144 tokens

Δυνατότητες

• 235B-parameter MoE with 22B active per token (128 experts, 8 selected)
• Hybrid thinking mode toggle (CoT or direct answer in one checkpoint)
• Pretrained on ~36T tokens across 119 languages
• 256K context window with YaRN scaling
• Apache 2.0 license, fully commercial
• Top open-weight scores on AIME 2025, LiveCodeBench, ArenaHard, BFCL
• Function calling, MCP server support, parallel tool calls
• Specialised siblings: Qwen3-Coder, Qwen3-Math, Qwen3-VL
• Compatible with vLLM, SGLang, llama.cpp, Ollama, MLX, HuggingFace
• Broad multilingual coverage with strong Chinese/English/Japanese/Korean performance
• Best for: open-weight reasoning, agentic workloads, multilingual chat, on-prem enterprise.

Εκπαίδευση & άδεια

Pretrained on approximately 36 trillion tokens covering 119 languages with strong Chinese and English emphasis, code repositories and scientific content. Post-training uses a four-stage pipeline: long-CoT cold start, reasoning RL, mixed SFT integrating thinking/non-thinking modes, and a final general-purpose RL stage.

Άδεια: Apache 2.0. Open weights, fully commercial use permitted including for >100M MAU products (a notable liberalisation versus the earlier Tongyi Qianwen LICENSE).

Δοκιμές ασφάλειας: Standard SFT+DPO safety alignment plus general RL safety stage. Filters Chinese politically sensitive topics. Limited third-party safety evaluations published.