Find What Works: Schools
Philadelphia

Methodology

How the data gets from the publishers (SDP, PDE, the Future Ready PA Index, the Philadelphia School Experience Survey) to the rankings shown on this site, plus the validation steps each metric passes before it's served.

The data layers

  1. Source files are fetched from publishers, recorded with sha256 + retrieved-at provenance, and inspected. Source registry: docs/cities/philly/sources.yaml.
  2. Loaders normalize the wide / long / zip-archived source rows into long-format facts in philly_school_year_metrics. Most metrics carry a population_cut of n/a; PSSA and Keystone carry both acct (accountability cohort) and actual cuts.
  3. Peer groups are derived as the 40 nearest neighbors on demographic features (race composition, ELL, IEP, economically disadvantaged), with hard filters on school level, admission category, and sector. 287 of 330 open schools get a peer group; 43 are dropped because their universe has fewer than 5 candidate peers.
  4. Percentiles are computed both citywide (every default-included school) and within peer group. Direction-adjusted: for lower-better metrics (out-of-school suspension rate, chronic absenteeism, etc.), a high percentile always means a good outcome.

What “validated” means

Each loaded metric is run through a four-check taxonomy before it's eligible to be served:

  • (i) Completeness — does every (school, year) cell in the source appear in the served DB?
  • (ii) Base case — does the served value match an independent re-read of the exact ingested file at ±1.0 pp tolerance? The binding gate.
  • (iii-a) SPREE spot check— vs SDP's published family-facing per-school report. 616 of 616 paired cells match within tolerance.
  • (iii-b) Future Ready cross-check — vs the independent-publisher PDE Future Ready PA Index file.
  • (iv) Computed values — derived metrics (out-of-school suspension distribution rollups, PSES topic rollups, demographic shares) against their raw components. OSS 8,034/8,034 match; PSES 8,157/8,157 match.

Plus an Acct vs Actual population-cut diff specific to Pennsylvania's dual reporting cuts. The Acct cut is canonical for the rankings; the Actual cut is loaded alongside so the diff stays inspectable.

Reports written by the harness live under docs/qa_reports/philly/. Current results: (i) PASS, (ii) PASS at ±1.0 pp across 6 components (837 paired cells, zero mismatches). Acct vs Actual is byte-identical across all 4,535 paired cells for 2024-25 at the school-rollup grain (the harness keeps both load paths so future divergence surfaces immediately).

Decisions of record

Eight design decisions (O.1 through O.8) are settled and documented in docs/cities/philly/00_overview.md. Highlights: SDP is the canonical publisher for outcomes when available; PSES teacher and student instruments are both first-class; PVAAS growth is a first-class metric alongside proficiency; the Acct cohort cut is canonical for PSSA.

Source URLs

← Philly home