What’s Out There
← Back to swiping
Science of scienceFreeAcademic

arXiv Metadata (OAI-PMH)

arXiv, Cornell University

Metadata for every arXiv preprint since 1991, updated daily, the raw record of physics, math, and AI.

arXiv offers bulk metadata for all its articles through OAI-PMH, which it calls the preferred way to keep an up-to-date copy. Each record is a preprint with title, authors, abstract, categories, versions, and license. Full-text PDFs and sources are available separately from Amazon S3 and Kaggle.

Access

Cost
FreeMetadata is free via OAI-PMH, the API, and a Kaggle snapshot; S3 full-text bulk access may incur AWS transfer charges.
License
Public domainMetadata CC0; full text under each article's own license
Sign-up
No sign-up
How you get it
API and Bulk downloadXML, JSON, PDF

What's in it

One row is
One record per preprint, with version history
Covers
Worldwide authors
Years
1991-present
Updated
Daily
Size
More than 2 million preprints
arXiv IDtitleauthorsabstractcategoriessubmission dateversionslicense

Quality

5 of 5

Clean, authoritative, and complete for its scope, with daily updates.

  • Author names are free text, not disambiguated
  • Category labels are self-chosen by submitters

Connects to

Questions it could answer

  • How fast did machine learning submissions grow relative to physics?
  • How often are preprints revised, and how much do abstracts change?

Similar data sets

  • Semantic Scholar Academic Graph (S2AG)

    214 million papers and 2.49 billion citations, with AI-extracted influential citations and paper embeddings.

    Science of scienceAllen Institute for AI (Ai2)Free

  • PubMed Baseline and Update Files

    Every PubMed citation, more than 36 million, as one yearly XML dump plus daily changes.

    Science of scienceNational Library of MedicineFree

  • Crossref Public Data File

    An annual snapshot of every Crossref DOI's metadata, well over 150 million records, free as a torrent.

    Science of scienceCrossrefFree

  • Association of Religion Data Archives (ARDA)

    More than 1,000 survey and census data sets on religion, from congregations to world religions.

    Arts & cultureAssociation of Religion Data Archives, Penn State UniversityFree

  • OpenAlex

    A free, open map of the world's research: hundreds of millions of papers, authors, and citations, all CC0.

    Science of scienceOurResearchFree tier, paid upgrades

  • Pushshift Reddit Archive

    The archive of nearly every Reddit post and comment, until Reddit shut researchers out in 2023.

    Social mediaPushshift (Network Contagion Research Institute)By application