What’s Out There
← Back to swiping
Language & textFreeCompany

FinePDFs

Hugging Face (FineWeb team)

About 3 trillion tokens extracted from PDFs, a source most web corpora skip because extraction is costly.

A pretraining corpus built only from PDF documents found in Common Crawl, covering many languages. One row is one extracted document with its text and metadata. It targets content such as reports, papers, and manuals that rarely appears in ordinary HTML text.

Access

Cost
FreeNo charge; Parquet files on the Hub.
License
Open licenseODC-By 1.0
Sign-up
No sign-up
How you get it
API and Bulk downloadParquet

What's in it

One row is
One row per extracted PDF document
Covers
Global web, many languages
Years
Common Crawl snapshots (see card)
Updated
Irregular
Size
About 3 trillion tokens
texturllanguagetoken countextraction method

Quality

3 of 5

Newer corpus with a clear method, but PDF text extraction is noisy and the long-term track record is short.

  • Extraction errors in tables, equations, and multi-column layouts
  • PDFs often carry copyright that the dataset license does not clear

Connects to

  • FineWeb2: HTML-based multilingual counterpart from the same team
  • Common Corpus: Open-license text corpus that also includes documents and scientific articles

Questions it could answer

  • How do languages rank by tokens in PDFs versus HTML pages?
  • What share of documents come from academic or government sources?

Similar data sets

  • FineWeb2

    Pretraining text for over 1,000 languages, with the pipeline tuned and validated language by language.

    Language & textHugging Face (FineWeb team)Free

  • FineWeb

    Trillions of tokens of cleaned English web text, with every filtering step published.

    Language & textHugging FaceFree

  • FineWeb-Edu

    1.3 trillion tokens of web text kept because a classifier judged it educational.

    Language & textHugging Face (FineWeb team)Free

  • Common Corpus

    2.27 trillion tokens of text that is either uncopyrighted or openly licensed, with per-document provenance.

    Language & textPleiasFree

  • FineMath

    34 billion tokens of math-teaching web pages, chosen by a classifier trained on Llama annotations.

    EducationHugging Face (SmolLM team)Free

  • RedPajama-Data-V2

    Raw Common Crawl text in five languages with precomputed quality signals attached to each document.

    Language & textTogether AIFree