What’s Out There
← Back to swiping
Language & textFreeAcademic

SQuAD (Stanford Question Answering Dataset)

Stanford University (Stanford NLP Group)

150,000 crowd-written questions on Wikipedia passages that defined a decade of reading benchmarks.

Reading-comprehension questions written by crowdworkers about paragraphs from 500+ Wikipedia articles, where each answer is a span of the passage. Version 2.0 added over 50,000 questions that cannot be answered from the text, to test whether systems know when to abstain. It is a landmark benchmark and a ready-made question-answer training set.

Access

Cost
FreeFree download; hidden test set is scored through the leaderboard.
License
Open licenseCC BY-SA 4.0
Sign-up
No sign-up
How you get it
Bulk downloadJSON

What's in it

One row is
One row per question, tied to a Wikipedia paragraph and its answer span
Covers
Not geographic
Years
Released 2016 (v1.1) and 2018 (v2.0)
Updated
One-time release
Size
About 150,000 questions in v2.0; training file about 40 MB
article titlecontext paragraphquestionanswer textanswer start positionis impossible

Quality

4 of 5

Carefully built and extremely widely used; now saturated, with models exceeding the human baseline.

  • Benchmark is saturated; top scores beat the human baseline
  • Questions often reuse the passage's wording, making them easier than real questions
  • Text has almost certainly leaked into LLM training data

Connects to

  • Wikipedia Database Dumps: Source articles for every passage
  • GLUE Benchmark: Sibling benchmark suite covering other language-understanding tasks

Questions it could answer

  • Which question types (who, when, why) remain hardest for models?
  • How often do answerable questions copy words directly from the passage?

Similar data sets

  • Humanity's Last Exam

    Thousands of expert-written questions designed to be too hard for current AI, across many fields.

    EducationCenter for AI Safety and Scale AIFree

  • MMMLU (Multilingual MMLU)

    The MMLU test set translated into 14 languages by professional human translators.

    EducationOpenAIFree

  • SWE-bench Verified

    500 real GitHub issues, human-checked, used to score whether AI can fix bugs in popular Python projects.

    Internet & techPrinceton NLP (with OpenAI)Free

  • MMLU-Pro

    A harder MMLU with ten answer options and 12,000 reasoning-heavy questions.

    EducationTIGER-Lab (University of Waterloo and collaborators)Free

  • Synthetic Text-to-SQL

    105,851 synthetic text-to-SQL records spanning dozens of domains and SQL task types.

    Internet & techGretel AIFree

  • Mozilla Common Voice

    Volunteers reading sentences aloud in 100+ languages, donated to the public domain.

    Audio & musicMozilla FoundationFree