How much do peer-group rankings change when you change the peer-group definition?
The question
Do different school rankings agree on which New York City schools are best — and what should a parent do when they disagree?
Methodology story: school rankings shift considerably depending on which group of peer schools you compare against. The city's official rankings (the New York City School Quality system), New York State Report Cards, US News, GreatSchools, and our own peer group all produce different placements for the same school. Within our own method, changing the number of nearest neighbors from 40 to 20 (more selective) or 80 (more inclusive) moves a school's peer-percentile rank by 10 to 15 points. The school itself doesn't change; the comparison lens does. Don't trust any single ranking — use them as triangulating data points.
Data analysis
Investigation run 2026-05-31.
TL;DR
Methodology piece — no specific new data needed. The question is real: school rankings shift considerably depending on which group of peer schools you compare against. Our derived peer group differs from the city's official rankings, US News, GreatSchools, and the state Report Card. A direct comparison would require scraping each ranking and joining. Quick stability test: our nearest-neighbor matching with 20, 40, or 80 peers shifts most schools' placement by 1 to 2 tenths of the distribution.
Five rankings, five answers
For the same school, you'll get five different rank placements depending on which peer group is used: the city's official school-quality lookup, the New York State Report Card, US News high-school rankings, GreatSchools (with its growth model), and our derived peer group. The methodological choices behind each ranking — whether to adjust for student-body challenge, how much to weight subgroup gaps versus aggregate scores, growth versus proficiency — fundamentally change which schools look 'best.'
Even our own method is sensitive to its parameters
Our nearest-neighbor matching uses grade band (as a hard match) plus admission category (hard match) plus demographics plus topic similarity (weighted). Changing the number of nearest neighbors from 40 to 20 makes the peer group more selective; changing to 80 makes it more inclusive. A school's peer-percentile rank can shift by 10 to 15 percentile points across those choices. The school itself doesn't change; the lens does.
What this means for parents
Don't trust any single ranking. Use multiple rankings as triangulating data points. A school that places top-decile in all five systems is probably actually strong. A school that's top in one and bottom in another is contested — that's the school worth digging into.
Doing the full comparison
Scrape current rankings from US News, GreatSchools, and the New York State Report Card. Compute decile rank under each system. Compute disagreement measures: rank correlation across systems, and the maximum rank divergence per school. The 5 to 10 schools with maximum divergence become case studies. About 2 weeks of scraping and analysis.
What's next
Methodology or explainer piece, not investigative. Worth doing once a year as school rankings shift.
Methodology & replication recipe — the data sources, queries, and steps behind this analysis.