GH Archive
GH Archive (Ilya Grigorik)
Every public push, star, fork, and issue comment on GitHub, archived hour by hour since 2011.
Hourly files of the public GitHub event stream (15+ event types, from pushes and pull requests to stars and new members) as reported by GitHub's API, also loaded into a public BigQuery dataset. Researchers use it to measure open-source activity, project popularity, developer behavior, and the spread of tools and languages.
Access
- Cost
- FreeFree file downloads; BigQuery queries are billed by Google after its free tier.
- License
- License not statedNot stated; events come from GitHub's public API
- Sign-up
- No sign-up
- How you get it
- Bulk downloadJSON (gzip), BigQuery
What's in it
- One row is
- One row per public GitHub event
- Covers
- Global (GitHub users)
- Years
- February 2011-present
- Updated
- Real time
- Size
- Billions of events; one gzipped JSON file per hour
Quality
Complete for public events and easy to use; event formats changed when GitHub's API did.
- 2011-2014 data came from a deprecated Timeline API with a different schema
- Private repository activity is absent
- Bot accounts generate a large share of events
Connects to
- PyPI Download Statistics (BigQuery): Compare a Python package's GitHub activity with how often it is installed
- Stack Exchange Data Dump: Link questions about a library to its repository's activity
Questions it could answer
- Which programming languages gained the most new repositories each year?
- How long do pull requests take to merge in large versus small projects?
- What share of GitHub events now come from bots?