Perception gapsFull briefing
How much do peer-group rankings change when you change the peer-group definition?
The longer form of the briefing for this story: the original thesis, the supporting findings the data team flagged, and the reporting directions a journalist could follow. For the short version and the data analysis, see the main story page.
Thesis
School rankings are sensitive to peer-group choice. NYCENET, NYS, US News, GreatSchools, and our own derivation all produce different rankings for the same school. Quantify the volatility: pick 100 schools and show how their 'how good is this school' answer ranges across methodologies.
Supporting findings
- Five rankings to compare (NYCENET, NYS, US News, GreatSchools, our derived peer percentiles).
- For each school, compute its decile rank under each methodology.
- Distribution of decile ranges (how many schools are 'top-decile' in one methodology and 'bottom-decile' in another?).
- Highlight the most-divergent schools — pick 5 for case studies.
- Explain the methodology differences: NYS uses statewide cohorts, US News weights AP / SAT, GreatSchools weights test growth, etc.
- Our peer-group methodology is now multi-axis (grade band + admission + demographics + topic). How much does adding admission category vs not change rankings?
- Stability test: re-run our k-NN with different K values (10, 20, 40, 80) — how much does decile rank shift?
- What does this mean for parents? 'Three different rankings, three different stories — here's how to read them.'
- What does it mean for accountability? If a single rating moves the principal's bonus, you'd want a stable rating.
- Frame: 'There is no single right ranking. There are right questions and useful answers.'
Reporting directions
- Scrape current rankings from US News, GreatSchools, NYS report card.
- Build the joint decile-rank table for 100 schools.
- Pick 5 high-divergence schools and write paragraph-length comparisons.
- Interview the lead methodologists at each ranking org.