DeepSeek R1

Σχετικά με DeepSeek

Ιδρύθηκε 2023 · Hangzhou, China

DeepSeek (formally DeepSeek AI) was founded in July 2023 by Liang Wenfeng, who is also the co-founder of the quantitative hedge fund High-Flyer (founded 2015). High-Flyer initially funded DeepSeek's research and provided access to thousands of NVIDIA A100 and H800 GPUs accumulated before US export controls tightened. DeepSeek's research output rapidly became influential: DeepSeek Coder (Nov 2023), DeepSeek LLM 67B (Jan 2024), DeepSeekMath (Feb 2024) which introduced GRPO reinforcement learning, DeepSeek V2 with Multi-head Latent Attention and MoE (May 2024), DeepSeek V3 (Dec 2024) and DeepSeek R1 (Jan 2025), the open-weight reasoning model that matched OpenAI o1 on many benchmarks. R1's release in January 2025 triggered a significant US stock-market re-rating of AI infrastructure spending given its training cost reportedly under $6M for the V3 base. DeepSeek publishes detailed technical reports and releases weights under the MIT license, making it one of the most transparent frontier labs. The company employs roughly 200 researchers, mostly recent graduates from top Chinese universities, and has stated it is not currently raising external venture capital.

Αρχιτεκτονική

Sparse Mixture-of-Experts Transformer (reasoning model trained with pure RL)

DeepSeek R1 was released on 20 January 2025 with weights under MIT license, a publicly downloadable technical report, and pricing roughly 1/30th of OpenAI o1 at the API. Architecturally R1 inherits from DeepSeek V3: a Sparse Mixture-of-Experts Transformer with 671B total parameters and 37B active per token, using Multi-head Latent Attention (MLA) and DeepSeekMoE routing. The breakthrough is the training recipe. DeepSeek R1-Zero was trained from the V3 base via pure large-scale reinforcement learning (GRPO) on verifiable math, code and reasoning tasks, with no supervised fine-tuning at all - the model spontaneously developed long chains-of-thought, reflection and self-verification behaviours, the so-called 'aha moment' phenomenon. R1-Zero suffered from readability issues, so DeepSeek R1 added a cold-start SFT step using a small set of curated long-CoT examples, followed by a multi-stage pipeline alternating between RL, rejection sampling and SFT. The team also distilled the reasoning behaviour into smaller dense models (DeepSeek-R1-Distill-Qwen-1.5B/7B/14B/32B and DeepSeek-R1-Distill-Llama-8B/70B), demonstrating that reasoning capability can be transferred to compact models. R1 supports a 128K context window and is widely deployed via vLLM, SGLang, Ollama and HuggingFace Inference.

Παράμετροι

671B total, 37B active per token

Περιεχόμενο

128.000 tokens

Δυνατότητες

• Open weights under MIT license, weights freely downloadable
• 671B-parameter MoE with 37B active per token
• Long chain-of-thought reasoning learned via pure RL (GRPO)
• Matches or beats OpenAI o1 on AIME, MATH-500 and Codeforces benchmarks
• 128K context window
• Distilled reasoning variants from 1.5B to 70B (Qwen and Llama bases)
• Pricing approximately 1/30th of OpenAI o1 at the DeepSeek API
• Strong code generation on LiveCodeBench and HumanEval
• Self-verification and reflection emerge from training
• Compatible with vLLM, SGLang, Ollama, llama.cpp and HuggingFace
• Best for: cost-sensitive reasoning workloads, on-prem deployment, research, reproducible chains-of-thought.

Εκπαίδευση & άδεια

Built on the DeepSeek V3 base (14.8T high-quality tokens of multilingual web text, code, books and scientific papers). R1 post-training uses cold-start SFT on a small curated long-CoT dataset, followed by multi-stage GRPO reinforcement learning against verifiable rewards on math, code and reasoning tasks plus rule-based language-consistency rewards.

Άδεια: MIT license (model weights, code, distilled variants). Commercial use permitted with no usage restrictions.

Δοκιμές ασφάλειας: Limited published safety evaluations; the model has been observed to refuse politically sensitive topics in line with Chinese regulations while having a relatively low refusal rate on Western dual-use queries.