How much of the 2024-25 'recovery' is a test-recalibration artifact?
A reproducibility log: the data this analysis touched, the queries it ran, the outside sources it leaned on, and the steps a third party would follow to re-run it.
Summary
Compared the average citywide proficiency change between contiguous years to detect the state-test recalibration effect. Identified 2022-23 as the recalibration year by the sudden citywide jump that doesn't track school-level changes proportionally.
Data sources
- Database tableschool_year_metrics
Long-format per-school per-year metric facts (school_dbn, year, metric_key, subgroup, value, suppressed). Loaded from DOE/NYSED public files via scripts/loaders/*.
- External datasetNYSED — 3-8 ELA/Math Test Score Recalibration documentation
Steps
Citywide averages of ela_all_proficiency and math_all_proficiency by year, 2018-19 onward.
Per-school year-over-year change; identify whether the jump is uniform (recalibration signature) or concentrated at low-scoring schools (real change).
Caveats
Without the state's raw scale-score crosswalk, we can only infer the recalibration's size from the population shape, not subtract it cleanly.
References
Reproduce
Clone the repo, set DATABASE_URL to a Postgres with the project schema loaded, run `npx tsx scripts/loaders/<source>.ts` for any not-yet-loaded data, then issue the queries in the Steps section. The story page also lists the exact `metric_key`/`subgroup`/`year` filters used. Searchable by the answer's headline number — every figure is recomputable from the queries shown.
The recipe lives at data/stories/recipes.ts in the repo. Corrections welcome.