What’s Out There
← Back to swiping
Language & textFreeCompany

Google Books Ngram Datasets

Google

How often every word and phrase appeared in millions of scanned books, year by year.

Counts of one- to five-word sequences in the Google Books scanning corpus, broken out by publication year and by language corpus. Each row gives an n-gram, a year, how many times it appeared, and in how many books. It powers the Ngram Viewer and a whole field of 'culturomics' studies of word use over time.

Access

Cost
FreeFree download of compressed tab-separated files.
License
Open licenseCreative Commons Attribution 3.0 Unported
Sign-up
No sign-up
How you get it
Bulk downloadTSV (gzip)

What's in it

One row is
One row per n-gram per publication year
Covers
Books in English (with American and British subsets) and several other languages
Years
Centuries of publication years; the 2020 release runs through 2019
Updated
Irregular
Size
Terabytes uncompressed across all n-gram orders; released in 2009, 2012, and 2020 versions
n-gramyearmatch countvolume countpart-of-speech tagslanguage corpus

Quality

4 of 5

Enormous and consistently built, but the book sample changes over time and OCR errors are common in older years.

  • Scientific and academic books grow as a share of the corpus over the 20th century
  • OCR errors, such as long-s read as f, distort pre-1800 counts
  • Only n-grams appearing more than 40 times are included

Connects to

Questions it could answer

  • When did 'the United States is' overtake 'the United States are' in American books?
  • How fast do new technology words spread into print?
  • Which words declined fastest in English books between 1950 and 2019?

Similar data sets

  • Corpus of Contemporary American English (COCA)

    A billion words of American English, balanced across genres from TV scripts to academic journals.

    Language & textEnglish-Corpora.org (Mark Davies)Free tier, paid upgrades

  • Project Gutenberg

    Tens of thousands of public-domain books, hand-proofed by volunteers since 1971.

    Language & textProject Gutenberg Literary Archive FoundationFree

  • xd Crossword Corpus

    Over 6 million crossword clue-answer pairs, from the project that caught a crossword plagiarist.

    Odditiesxd.saul.pw (Saul Pwanson)Free

  • UCSD Goodreads Datasets

    228 million shelvings, ratings, and reviews from 876,000 Goodreads readers, scraped in 2017.

    Film, TV, & booksUniversity of California San Diego (McAuley Lab)Free

  • Chronicling America

    Millions of digitized pages of historic American newspapers, OCR text included, free to mine.

    HistoryLibrary of Congress and National Endowment for the HumanitiesFree

  • Discogs Data Dumps

    A crowd-built catalog of recorded music, down to pressing plants and matrix numbers, all CC0.

    Audio & musicDiscogsFree