FineWeb-Edu
Hugging Face (FineWeb team)
1.3 trillion tokens of web text kept because a classifier judged it educational.
A subset of FineWeb filtered with a classifier trained on Llama-3-70B-Instruct annotations of educational quality. One row is one web page with its text, URL, and an educational score. The card also describes a larger 5.4 trillion token version with a lower score cutoff.
Access
- Cost
- FreeNo charge; Parquet files on the Hub.
- License
- Open licenseODC-By 1.0
- Sign-up
- No sign-up
- How you get it
- API and Bulk downloadParquet
What's in it
- One row is
- One row per web document
- Covers
- Global web, English only
- Years
- Common Crawl snapshots through 2024
- Updated
- Irregular
- Size
- 1.3 trillion tokens (5.4 trillion in the lower-threshold variant)
Quality
Documented in a paper, and models trained on it score well on knowledge benchmarks for their size. The filter reflects an LLM's idea of educational content.
- Quality labels come from an LLM classifier, so its biases carry through
- Heavy tilt toward school-style explanatory prose
- Benchmark questions may leak because the classifier favors textbook-like pages
Connects to
- FineWeb: Parent corpus this was filtered from
- SmolLM-Corpus: Includes FineWeb-Edu as one of its three subsets
Questions it could answer
- Which domains contribute most to the educational subset compared with the full FineWeb?
- How does the educational score distribute across crawl years?