The 50 NYC schools punching highest above demographic expectation
A reproducibility log: the data this analysis touched, the queries it ran, the outside sources it leaned on, and the steps a third party would follow to re-run it.
Summary
Identified the 50 New York City schools that score furthest above their student-body-challenge expectation by ranking the residual between a school's actual academic composite and the average outcome at its peer group (k=40 nearest neighbors on grade band + admission category + demographics + topic similarity).
Data sources
- Database tableschools
One row per New York City public school (DBN, name, district, borough, school_type, grade_band, latest_enrollment, proclivity_decile, proclivity_score, closed_at, admission_category).
- Database tableschool_year_metrics
Long-format per-school per-year metric facts (school_dbn, year, metric_key, subgroup, value, suppressed). Loaded from DOE/NYSED public files via scripts/loaders/*.
- Database tablepeer_groups + school_peers
K-nearest-neighbor peer groups derived from grade band + admission category (hard match) plus demographics + topic similarity (weighted). Default k=40. Built in scripts/derive/peer-groups.ts.
- Database tableproclivity_score / proclivity_decile
Composite student-body-challenge score (lower = lower-need, higher = higher-need) built from % economically disadvantaged, % English Language Learners, % students with disabilities. Decile 1 = lowest-need; decile 10 = highest-need.
- Loader script
scripts/loaders/_story_deepdive_queries.ts
Steps
For every school, compute peer-group average academic composite (composite_es_academic for elementary; composite_ms_academic for middle; composite_hs_academic for high school) using its k=40 peer set.
Subtract peer average from school's value to get the residual. Rank descending; take the top 50.
Pull each top-50 school's proclivity decile, latest enrollment, and trajectory (city-percentile rank 2019-20 → 2024-25).
Caveats
Peer group composition is sensitive to k (40 vs 20 vs 80 changes ranks ~10-15 percentile points). 'Outperforming peers' is a relative measure: a school in decile 8 punching above can still be far below citywide average. We can't distinguish leadership effects from selection effects without longitudinal individual-student data.
References
Reproduce
Use the /ratings/outliers page to filter by metric=composite_es_academic, proclivity=8-10, direction=top. Or query directly: see the steps. Underlying peer group is built by scripts/derive/peer-groups.ts; tweak k there to test sensitivity.
The recipe lives at data/stories/recipes.ts in the repo. Corrections welcome.