Grok 3

About xAI

Founded 2023 · Palo Alto, USA

xAI was founded by Elon Musk in March 2023 alongside engineers and researchers recruited from DeepMind, OpenAI, Google Research, Microsoft Research and Tesla, including Igor Babuschkin, Tony Wu, Christian Szegedy, Jimmy Ba and Yuhuai Wu. The lab's stated mission is to 'understand the true nature of the universe' and to build maximally truth-seeking AI. xAI's flagship product is Grok, an LLM-based assistant integrated into the X (formerly Twitter) platform; Grok 1 was released as open weights under Apache 2.0 in November 2023 with 314B parameters in a Mixture-of-Experts configuration. Grok 1.5 introduced long context (128K) and vision, Grok 2 launched in mid-2024, and Grok 3 was unveiled in February 2025 trained on the Colossus supercluster of ~100,000-200,000 NVIDIA H100 GPUs in Memphis, Tennessee. xAI raised $6 billion in May 2024 at a $24 billion valuation and another reported $6 billion later in 2024, with investors including Andreessen Horowitz, Sequoia and Saudi Arabian funds. The company also operates the Grok API and integrates Grok into the Premium+ tier of X.

Architecture

Decoder-only Transformer (frontier reasoning, trained on Colossus)

Grok 3 was unveiled by xAI on 17 February 2025. It is a decoder-only Transformer trained on the company's Colossus supercluster in Memphis, which xAI reports as the largest single training cluster in the world at launch (~100,000 NVIDIA H100 GPUs at first, scaling to ~200,000). The training run reportedly used roughly 10x the compute of Grok 2. Pretraining used a multi-trillion-token mixture of web text, code, mathematical content, scientific papers and a continuously updated stream of public posts from the X platform, which gives Grok unusually fresh real-world data. Post-training combined supervised fine-tuning, RLHF and large-scale reinforcement learning against verifiable rewards on math, code and reasoning tasks. Grok 3 introduced two reasoning modes: 'Think' which produces an internal chain-of-thought and 'Big Brain' (later renamed) which spends additional test-time compute on complex problems. The model family also ships a Grok 3 mini variant tuned for low-latency and cost. Grok 3 supports a 1,000,000 token context window in the API, vision input, function calling and DeepSearch agent mode which crawls X and the public web with retrieval-augmented reasoning.

Parameters

Undisclosed (estimated multi-hundred billion dense parameters)

Context

1,000,000 tokens

Capabilities

• 1,000,000 token context window in the API
• Two reasoning modes: Think (chain-of-thought) and high-effort variants
• DeepSearch agent that searches X and the public web with retrieval-augmented reasoning
• Trained on the Colossus 100K-200K H100 supercluster
• Vision input for images and screenshots
• Function calling and structured JSON output
• Real-time access to X (Twitter) posts via integrated tooling
• Top scores on AIME 2025, GPQA and Codeforces benchmarks at launch
• Available in X Premium+ and via the standalone Grok app
• Grok 3 mini variant for low-cost workloads
• Best for: real-time information chat, reasoning over X data, math/code benchmarks, edgy/uncensored content.

Training & license

Multi-trillion-token pretraining corpus combining web text, code, scientific literature and a uniquely large continuous feed of public X (Twitter) posts. Knowledge cutoff is approximately late 2024 with live retrieval available via DeepSearch. Post-training uses RLHF and reinforcement learning against verifiable rewards.

License: Proprietary commercial license via the xAI API and X Premium+ subscription. Grok 1 weights were open under Apache 2.0; Grok 3 weights are not released.

Safety testing: xAI publishes a safety overview and risk-evaluation summary on x.ai; the model has fewer hard refusals than competitors by design, with a stated 'maximum truth-seeking' policy.