Dataset
aka Colossal Clean Crawled Corpus
C4
Google's filtered 750GB English web-text dataset, used to train T5.
Definition
C4 is a 750GB clean version of Common Crawl created for T5: dedup, language filter, removed gibberish and offensive content. It seeded many later filtering pipelines and remains a common public pre-training corpus on HuggingFace.
Common use cases
- LLM pre-training
- Tokenizer training
- Filtering research