Whisper Large V3

Over OpenAI

Opgericht 2015 · San Francisco, California, USA

OpenAI was founded in December 2015 by Sam Altman, Elon Musk, Greg Brockman, Ilya Sutskever, Wojciech Zaremba and John Schulman, restructured to capped-profit OpenAI LP in 2019. Whisper was first released as an open-weights speech recognition model in September 2022 by Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey and Ilya Sutskever, and quickly became the de-facto open ASR baseline thanks to its zero-shot multilingual quality. Whisper Large v3 was released in November 2023 alongside the GPT-4 Turbo launch as the final open checkpoint in the original Whisper line. OpenAI later released Whisper Large v3 Turbo (October 2024), a distilled faster variant. All Whisper weights remain available under MIT licence on GitHub and Hugging Face and are widely used in production by competitors.

Architectuur

Encoder-decoder Transformer for multitask speech recognition

Whisper Large v3 is a 1.55B-parameter encoder-decoder Transformer trained for multitask audio understanding. Input is a 30-second log-mel spectrogram with 128 mel bins (up from 80 in v2) processed by a 32-layer audio encoder; the decoder is a 32-layer Transformer that emits special task tokens for language identification, transcription, translation-to-English and timestamp prediction. Whisper was trained on 5 million hours of audio in total: 680k hours of weakly supervised multilingual web audio for v1/v2 plus an additional 4 million hours of pseudo-labelled audio generated by Whisper Large v2 for v3, yielding approximately 1 million hours of weakly supervised and 4 million hours of pseudo-labelled data. The model covers 99 languages, with very strong performance on English long-form (~5% WER on LibriSpeech test-clean) and significantly improved low-resource coverage. Audio longer than 30 seconds is processed with a sliding window. The model is released under MIT licence.

Parameters

1.55B (Large v3)

Context

30 tokens

Mogelijkheden

• 99-language transcription with automatic language detection
• Translation of any supported language directly to English text
• Word-level and segment-level timestamps
• 30-second context window with sliding window for long-form audio
• Open weights under MIT licence, runs on a single 16 GB GPU in fp16
• Strong robustness to accents, background noise and technical jargon
• Best for: open-source ASR pipelines, research baselines, on-premise transcription

Training & licentie

5 million hours total: ~680k hours of weakly supervised multilingual audio scraped from the public web (subtitles aligned with audio) plus ~4 million hours of pseudo-labelled audio generated by Whisper Large v2. 17% of the supervised set is non-English speech.

Licentie: MIT licence for code and weights; commercial use permitted.

Veiligheidstests: OpenAI published bias and hallucination analyses; the paper documents over-confident hallucination on silent or non-speech audio. No formal red-team report beyond the original paper.