PhiladelphiaMethodologyeasy
Computed-value (iv) validation: OSS + PSES rollups agree to the cell with the raw source
Thesis
Philly's (iv) computed-value validation: re-derive the metrics from raw inputs, compare to the served DB. OSS distribution derivations (any/multiple/chronic) match the raw at 8,034/8,034 cells. PSES topic rollups (mean of subtopic scores per respondent) match at 8,157/8,157. Demographic shares (aggregated from grade × school rows) match at 3,879/4,127 (94%, with the 248 'mismatches' being NaN-handling differences in pct_econ_disadv).
Supporting findings
- OSS pct_any: 8,034 cells, 100% agreement.
- OSS pct_multiple + pct_chronic: same.
- PSES rollups: 8,157 cells, 100% agreement.
- Demographics: 94% agreement, with the 6% gap being a NaN-vs-null re-derivation logic difference on pct_econ_disadv (CEP), not a data issue.
Reporting directions
- Fix the CEP NaN handling in pipeline_philly/verify/reconcile_computed.py for 100% on demographics too.
- Add the (iv) gate to CI so any future regression is caught.
Methodology & replication recipe — the queries + sources behind the findings above.
More on methodology
- Philly publishes both Acct and Actual PSSA cuts. They're byte-identical.
- SPREE agrees with our DB on 616/616 cells — Philly's data pipeline is structurally clean
- Philly PSSA: 2023-24 vs 2024-25 year-over-year — which schools moved most
- With 287 peer-grouped schools and k=40, the rankings barely move at k=20