Find What Works: Schools

Methodology

How the data is sourced, joined, and framed. The full machine-readable spec lives in SPECS.md.

Data quality

Every metric goes through a four-check validation program (completeness, base case vs the source file, spot check vs NYC's public reports, computed values), plus cross-publisher reconciliation against NYSED. The findings ledger, agreement figures, and per-metric reports live in Data quality →

Comparison groups

NYC schools serve very different student populations. Citywide averages flatten that variation and make outlier detection misleading: a school in a high-poverty neighborhood compared to the citywide average will almost always look worse — but compared to schools serving similar populations, it might be doing exceptionally well.

The fix is a peer group: ~40 schools in the same grade band, serving demographically similar students. Every metric on the site is framed against this group. The Ratings outlier page surfaces schools in the top or bottom decile of their peer group — not of the city — so peer context is the lens.

How v1 builds the group: NYCENET publishes authoritative comparison groups for DOE schools via a per-school lookup tool — no bulk download. Rather than scrape ~1,900 lookups, v1 computes the peer group ourselves: k-nearest neighbors (K=40) with hard filters and a weighted similarity score.

  • Hard filter: grade band. ES schools peer only with ES, MS with MS, K8 with K8, HS with HS, etc. — so the comparison is age-appropriate.
  • Hard filter: admission bucket. Schools peer only with others recruiting from the same applicant pool. Buckets: Zoned / open (standard DOE enrollment), Selective (screened, composite-score, specialized exam), Audition, Charter (lottery), D75 inclusive, Transfer, Other.
  • Weighted similarity: demographics (primary). Euclidean distance on % Black, % Hispanic, % Asian, % White, % ELL, % students with disabilities, % economically disadvantaged.
  • Weighted similarity: topic tags (secondary). Jaccard distance over school focus areas (STEM, Arts, Dual Language, etc.) — schools with matching focus get a small bonus.

Admission category comes from NYC Open Data HS + MS directory fields (method1 / admissionsmethod_prog1). For ES schools without directory entries, we default to Zonedunless the name suggests G&T. Topic tags are derived via keyword matching on the school name + DOE description + our internal InsideSchools narrative. Tagged as source = DERIVED; a future pass will reconcile with NYCENET groups for DOE schools.

Click peers on any school × metric row to see the peer group: the ~40 schools, their value on that metric, and the peer-group distribution (10th / 50th / 90th percentile).

Proclivity decile

A school's proclivity decile(1–10) is a predicted-outcome bucket based on student-body characteristics, not on actual results. It answers the question: "before looking at any outcome, how advantaged is this school's student population?"

  • Decile 1 = most-advantaged student population (lowest % economically disadvantaged, ELL, SWD). Outcomes expected to be high.
  • Decile 10 = most-challenged student population. Outcomes expected to be lower.

The interesting story is when those expectations break. A school in decile 9 or 10 with top-decile outcomes is punching well above what its student composition would predict — worth a closer look. Conversely, a decile-1 school with bottom-decile outcomes is an underperformer for its profile.

The score is the mean of percentile ranks across % econ disadvantaged, % ELL, and % SWD (using the most recent demographic snapshot per school). Schools without demographics are unranked. D75 / D79 / alt programs are excluded by default since their mandate isn't comparable.

Composite metrics

For each grade band we publish a composite metric alongside the individual ones, so you can ask "how is this school doing academically" without picking sides between ELA and math.

  • Elementary academic composite = mean of 3rd-grade ELA proficiency and 3rd-grade math proficiency.
  • Middle school academic composite = mean of 7th-grade ELA and math proficiency.
  • High school outcomes composite = 4-year graduation rate (will fold in college-and-career readiness when that source is ingested).

Composites are flagged with a composite badge in tables. They require all underlying inputs to be present (non-null, non-suppressed) for that school × year. Survey-based composites (bullying, teacher trust) will appear once the NYC School Survey loader lands.

Outlier definition (v1)

A school is a positive (or negative) outlier on a metric in a given year if it falls in the top (or bottom) decile of its comparison group's distribution for that metric. Same rule across every metric in v1. Per-metric tuning (z-score for bounded proportions, Poisson tail for low-count incidents) is a v2 problem once we've inspected the distributions.

Suppression

Cells suppressed by NYCDOE / NYSED for student privacy show up as "suppressed (low N)" rather than being silently dropped. Listing a school as an outlier on the basis of a suppressed cell isn't fair, so the Ratings outlier list hides suppressed rows by default. A toggle exposes them, badged.

Survey rollups

We ingest the full NYC School Survey — all three respondent groups (student, teacher, parent), every question, every year. NYCDOE publishes rolled-up domain scores, which we store and surface by default. We also compute our own rollups from the raw response distributions for transparency, exposed via a toggle.

School universe

DOE public schools + NYC public charter schools. District 75 (special-ed), District 79 (transfer), and alternative programs are included in the database but tagged so they don't pollute default Ratings outlier lists. A toggle opts them in.

Year normalization

Canonical form is YYYY-YY (e.g., 2023-24). All loaders normalize at the source boundary.

Sources

Have a methodology question or spot a bug? Open an issue on GitHub.