FineMath
Hugging Face (SmolLM team)
34 billion tokens of math-teaching web pages, chosen by a classifier trained on Llama annotations.
Math-focused educational content filtered from Common Crawl by a classifier using Llama-3.1-70B-Instruct labels. One row is a web page with its text and score. Variants add a larger set combined with InfiMM-WebMath.
Access
- Cost
- FreeNo charge.
- License
- Open licenseODC-By 1.0
- Sign-up
- No sign-up
- How you get it
- API and Bulk downloadParquet
What's in it
- One row is
- One row per web page
- Covers
- Global web, mostly English
- Years
- Common Crawl snapshots
- Updated
- One-time release
- Size
- 34 billion tokens (FineMath-3+); 54 billion with InfiMM-WebMath-3+
Quality
Documented filtering with measured gains on math benchmarks when used in pretraining.
- Selected by an LLM's judgment of educational value
- Equation rendering and extraction can be lossy
Connects to
- FineWeb-Edu: General educational counterpart from the same group
- Natural Reasoning: Questions there were back-translated partly from FineMath
Questions it could answer
- How do math-heavy sites rank by tokens retained?
- What score threshold separates tutorials from problem lists?