What’s Out There
← Back to swiping
Language & textFreeCompany

Common Corpus

Pleias

2.27 trillion tokens of text that is either uncopyrighted or openly licensed, with per-document provenance.

A multilingual collection of books, newspapers, scientific articles, government and legal documents, and code. Each document carries information about its source and license. It was built so models can be trained without relying on content of uncertain legal status.

Access

Cost
FreeNo charge.
License
Open licenseMixed per-document; the card says only uncopyrighted or freely licensed data is included
Sign-up
No sign-up
How you get it
API and Bulk downloadParquet

What's in it

One row is
One row per document, with provenance metadata
Covers
Mostly Europe and North America; English and French dominate
Years
Spans centuries (digitized books) to recent web and code
Updated
Irregular
Size
2.27 trillion tokens
textsource collectionlanguagelicensedatetoken count

Quality

4 of 5

Documented provenance and an ICLR 2026 paper. Old OCR text and uneven source coverage limit its use as a general corpus.

  • OCR errors in historical texts
  • Per-document license must be checked for specific uses
  • English and French dominate other languages

Connects to

  • FinePDFs: Another document-heavy corpus, but without the open-license restriction
  • Dolma: Open pretraining mix that includes web text of less clear rights

Questions it could answer

  • How much of the corpus is public-domain versus explicitly licensed?
  • Which collections supply the non-English text?

Similar data sets

  • FinePDFs

    About 3 trillion tokens extracted from PDFs, a source most web corpora skip because extraction is costly.

    Language & textHugging Face (FineWeb team)Free

  • Dolma

    The three-trillion-token corpus behind OLMo, with the recipe and tooling published alongside it.

    Language & textAllen Institute for AIFree

  • OPUS Parallel Corpora

    Over 100 billion aligned sentence pairs across more than 1,000 languages.

    Language & textUniversity of Helsinki (Language Technology Research Group)Free

  • Corpus of Contemporary American English (COCA)

    A billion words of American English, balanced across genres from TV scripts to academic journals.

    Language & textEnglish-Corpora.org (Mark Davies)Free tier, paid upgrades

  • The Pile

    The 825 GB open training set that taught the field how LLM data is assembled, and got sued over it.

    Language & textEleutherAIFree

  • Project Gutenberg

    Tens of thousands of public-domain books, hand-proofed by volunteers since 1971.

    Language & textProject Gutenberg Literary Archive FoundationFree