How much of school outcomes is just demographic composition?
The question
How much of the difference between New York City schools' test scores is explained by demographics, and how much by what the school itself does?
We sort New York City elementary schools into 10 equal-sized groups by how much challenge their student body faces — the share of students from low-income households, the share still learning English, and the share with disabilities. This 1-to-10 ranking (we call it the student-body challenge group) tracks with the overall elementary-school score at a correlation of −0.63, which means demographics account for about 40% of the differences in scores between schools. The easiest-to-serve group averages 86% proficient; the hardest-to-serve averages 42%. The other 60% of the variation is instruction, leadership, culture, and noise — and the 50 standout schools profiled elsewhere show that the ranking isn't a ceiling.
Data analysis
Investigation run 2026-05-30.
TL;DR
Across 779 New York City elementary schools, the 1-to-10 student-body challenge group ranking correlates with the overall elementary-school score at −0.63. Roughly 40% of the differences in scores between schools come from demographics alone. The other 60% is everything else: instruction, leadership, culture, principal stability, peer effects, year-to-year noise. The challenge-group ranking is the single most predictive aggregate measure we have, but it leaves a lot of room for the schools themselves to matter.
The numbers
| Measure | Value |
|---|---|
| Correlation between challenge-group ranking and overall elementary score (2024-25) | −0.63 |
| Share of school-to-school variation explained by the ranking | 39% |
| Schools included | 779 |
| Average score, challenge group 1 (easiest to serve) | 85.8% |
| Average score, challenge group 10 (hardest to serve) | 42.3% |
| Gap, group 1 to group 10 | 43.5 pts |
What this means
If a school sits in challenge group 1, you'd predict its overall elementary score at about 86%, give or take 15 points. If it sits in group 10, you'd predict about 42%, give or take 15 points. About 60% of the school-to-school variation in scores isn't predicted by the ranking — that's the 'school effect.' Sean Reardon's national research finds that school effects typically account for 10-25% of school-level variation; our 60% is higher because the ranking doesn't include teacher quality, curriculum, building, or district as separate inputs.
Reading school rankings, with this in mind
Two schools scoring 85% and 65% on the overall elementary measure aren't directly comparable if the first is in challenge group 1 and the second is in challenge group 6 — both are scoring about where the ranking would predict. The school in the second case is the one whose outcomes most exceed the prediction. New York City's official rankings publish raw scores, which include both who the school admits and what the school adds. A ranking adjusted for the challenge group separates the second from the first. Each kind of ranking answers a different question.
Why 60% isn't explained
Some of the unexplained variation reflects genuine differences in what schools do — instruction, leadership, school climate. Some is year-to-year noise (small grades have noisier averages). Some reflects gaps in the ranking itself, which doesn't capture parental education, neighborhood stability, what language families speak at home, and other inputs. A more sophisticated model would push the explained share above 50%, leaving roughly 40% as the genuine school effect.
Caveats
A simple linear correlation can understate the relationship at the top and bottom of the scale, where most scores cluster near the ceiling or the floor. A logistic or quantile regression would capture those effects better, but the basic finding — that the ranking explains roughly 40% of variation — holds across model choices.
What's next
Publish a methodology piece comparing challenge-group-adjusted rankings with the New York City Department of Education's raw-score rankings. Show which schools move up and which move down once the student-body adjustment is applied.
Methodology & replication recipe — the data sources, queries, and steps behind this analysis.