What’s Out There
← Back to swiping
Language & textFreeCompany

Cosmopedia

Hugging Face (SmolLM team)

Synthetic textbooks, blog posts, and stories written by Mixtral to mimic what web pretraining text teaches.

An open synthetic pretraining dataset: Mixtral-8x7B-Instruct generated documents from prompts seeded with web samples and curated educational sources. One row is a prompt, the generated text, and the audience and format it targeted. Millions of rows in the v0.1 release.

Access

Cost
FreeNo charge.
License
Open licenseApache 2.0
Sign-up
No sign-up
How you get it
API and Bulk downloadParquet

What's in it

One row is
One row per generated document with its prompt
Covers
Not geographic
Years
2024
Updated
One-time release
Size
10 to 100 million rows
prompttextseed sourceaudienceformat

Quality

3 of 5

Documented and reproducible, but all text is model output and can contain factual errors and repetitive style.

  • Hallucinated facts presented in textbook voice
  • Topic coverage follows seed data, not the real distribution of knowledge
  • Superseded by v0.2 inside SmolLM-Corpus

Connects to

  • SmolLM-Corpus: Contains the improved Cosmopedia v2
  • TinyStories: Earlier, much smaller synthetic corpus built on a similar idea

Questions it could answer

  • Which prompt styles yield the most varied text?
  • How repetitive are documents generated from the same seed topic?

Similar data sets

  • SmolLM-Corpus

    The mix of synthetic textbooks, educational web text, and Python code that trained small SmolLM models.

    Language & textHugging Face (SmolLM team)Free

  • DiffusionDB

    2 million Stable Diffusion images paired with the prompts and settings real users typed.

    Images & videoPolo Club of Data Science (Georgia Tech)Free

  • FineWeb

    Trillions of tokens of cleaned English web text, with every filtering step published.

    Language & textHugging FaceFree

  • Synthetic Text-to-SQL

    105,851 synthetic text-to-SQL records spanning dozens of domains and SQL task types.

    Internet & techGretel AIFree

  • PersonaHub

    Short persona descriptions that steer an LLM into generating varied synthetic data.

    Language & textTencent AI Lab (proj-persona)Free

  • FineMath

    34 billion tokens of math-teaching web pages, chosen by a classifier trained on Llama annotations.

    EducationHugging Face (SmolLM team)Free