FinePDFs
Hugging Face (FineWeb team)
About 3 trillion tokens extracted from PDFs, a source most web corpora skip because extraction is costly.
A pretraining corpus built only from PDF documents found in Common Crawl, covering many languages. One row is one extracted document with its text and metadata. It targets content such as reports, papers, and manuals that rarely appears in ordinary HTML text.
Access
- Cost
- FreeNo charge; Parquet files on the Hub.
- License
- Open licenseODC-By 1.0
- Sign-up
- No sign-up
- How you get it
- API and Bulk downloadParquet
What's in it
- One row is
- One row per extracted PDF document
- Covers
- Global web, many languages
- Years
- Common Crawl snapshots (see card)
- Updated
- Irregular
- Size
- About 3 trillion tokens
Quality
Newer corpus with a clear method, but PDF text extraction is noisy and the long-term track record is short.
- Extraction errors in tables, equations, and multi-column layouts
- PDFs often carry copyright that the dataset license does not clear
Connects to
- FineWeb2: HTML-based multilingual counterpart from the same team
- Common Corpus: Open-license text corpus that also includes documents and scientific articles
Questions it could answer
- How do languages rank by tokens in PDFs versus HTML pages?
- What share of documents come from academic or government sources?