One entry per finding from the data-validation program, never deleted — status changes are edited in place. A finding is verified (established, evidence linked), fixed (was a defect; resolution linked), open (real, cause or resolution not established), or ruled out (investigated, not the case).
The attendance loader silently dropped five source categories ("Not Poverty", "Not ELL", "Not STH", "Other", "Neither male nor female") via exact-string mismatch — ~39k school-year cells, including the comparison sides of the poverty/ELL/housing gaps.
2026-06-09. PR #9. Evidence: PR #9
148 attendance school-years exist in the source file but not the warehouse: DBNs missing from the schools directory (stale 2019-20 vintage); loaders skip unknown DBNs. Same root cause confirmed for the test-metric gaps.
2026-06-11. fixed locally (branch kx/bakeoff-prep, dev replica) — pending prod re-ingest. Directory seed now unions wg9x-4ke6 with the live LCGMS export: 141/148 school-years resolve (residual: 15K418, dual-coded with D75 75K372), 33/34 unmatched charter BEDS match; 0 previously-served metric rows changed.. Evidence: PR #9
The source file carries per-grade rows (PK–12) and average-daily-attendance columns that were never ingested. Grade-level All-Students cells are only 3.9% suppressed (per-school grade trajectories and pseudo-cohorts viable); grade×subgroup runs 23–50% suppressed.
PR #12 ingests all of it (~2.25M cells). Evidence: PR #12
pct_econ_dis in school_year_demographics is top-coded at 95% and "Above 95%" is stored as NULL for 8–18% of schools per year — and those schools have higher absenteeism, so dropping them biases any SES adjustment. The demographics loader also hard-codes pct_temp_housing to null, and no Economic Need Index is ingested.
2026-06-11. fixed locally (branch kx/bakeoff-prep, dev replica) — pending prod re-ingest. "Above 95%" now stored as 0.95 with pct_econ_dis_topcoded flag (0 NULLs remain); the file's Economic Need Index ingested as eni; pct_temp_housing derived from the attendance STH sheet by sth-shares.ts (full series in data/analysis/sth-shares.json)..
Base case is exact for chronic_absenteeism_rate (10,639 cells, 0 mismatches) and now for average_daily_attendance (10,639 cells, 0 mismatches) + per-grade components.
2026-06-09. validation reports; PR #13. Evidence: chronic_absenteeism_rate__validation.md · PR #13
The Snapshot's chronic-absenteeism figure is the same-year GRADE-BAND rate, not the school-wide rate — grades K–8 (kindergarten-inclusive) on the EMS report, grades 9–12 on the HS report — and a band-spanning school has BOTH report pages, each carrying its own band's figure. Established by the exhaustive census (6,570 district school-year pairs, SY 2018-19 → 2022-23; replaces the 300-school sample): school-wide agreement within ±1.5pp is 0.6534 (the sample's "~67%" confirmed; full tail −26.84 to +15.83pp); against the matching band it rises to 0.8695 pooled (6,964 type-matched report-band pairs; both report types fetched for all 201 band-spanning schools 2026-06-10), 0.9261 EMS-report, ≥0.94 for EMS report-years 2020–2023. The entire worst-15 list is band-explained, including the single worst case (28Q157 SY 2021-22, −26.84pp school-wide: its HS page's 54% is its 35-student grade-9 rate, 19/35, exactly, while its EMS page's 26% matches its grades-6–8 rate 26.56%). Supersedes the "suspected accountability-cohort denominator" for district schools. Residual not established — and UNCHANGED by type-matching: HS-report values in SY 2019-20 and 2021-22 (band agreement 0.4834/0.4974 with a one-sided +1.5pp median shift, served above Snapshot) and both report types in SY 2018-19 (0.80–0.83).
2026-06-10. largely explained by the exhaustive census, re-censused type-matched 2026-06-10; residual = HS reports SY 2019-20/2021-22 + both report types SY 2018-19 (persists under type-matching). Evidence: snapshot/CENSUS.md · snapshot/census-stats.json · Verification card
Snapshot report-years 2024/2025 carry chronic absenteeism under renamed varnames (val_chronic_absent_ems/hs_all, with peer averages pavg_*). Decoded (2026-06-10): the value is the COMPLEMENT — round(100 − same-report-year grade-band chronic-absenteeism rate) (EMS: grades K–8, K-inclusive; HS: grades 9–12) — i.e., the share NOT chronically absent. Decoded agreement at ±1.5pp: 0.9710/0.9740 (ry2024 EMS/HS) and 0.9665/0.9814 (ry2025); every raw-value candidate maxes out at 0.042, so the earlier 4-school "matches neither adjacent InfoHub year" result was an artifact of comparing the raw value instead of its complement. The series changed sign convention in 2024, not population or period. Human review (2026-06-10) found the rendered Snapshot page does not visibly display chronic absenteeism for 2024-25 — the field lives in the data feed; what label the page attaches to it remains unestablished (the one open thread).
2026-06-10. open thread: page-label semantics. Evidence: snapshot/CENSUS.md §5 · snapshot/census-stats.json
Charters DO appear in the NYC Snapshot tool with school-specific chronic-absenteeism values and comparison-group averages — despite being absent from the End-of-Year file (zero charter rows, full-file scan). Settled at full scale (2026-06-10): the values are computed, not copied — only 17.5–19.8% match whole-percent roundings of NYSED's published band rates (vs ~100% expected for republication), values exist in 39 school-years where NYSED's public file suppresses, and the enrollment basis is K-inclusive at scale (95.8% of K-serving charters show an enrollment surplus over NYSED's grades-1–8 band, median +9.8% ≈ one grade; non-K charters −2.4%). They track NYSED only loosely (median |diff| 1.79pp on the matched band, r 0.73–0.83 every year, vs district r 0.99 — why not established). Reporting outages co-move perfectly with the state chain: 82/82 DATA_REP_FLAG=N charter school-years (2021–2025) have no Snapshot CA value — zero counterexamples, ≥5 networks (all 31 SA schools wink out exactly when SIRS reporting stops, the Snapshot report itself still published). Practical rating: PARTIALLY independent — can catch gross serving errors (independently computed, r ~0.7–0.8) and coverage gaps; cannot catch failures originating in the state attendance-reporting chain (it goes dark in lockstep); cannot arbitrate differences under ~2pp. Not established: the exact computation basis (denominator rules, thresholds, as-of date); whether DOE receives charter attendance through SIRS itself or a parallel submission failing simultaneously; why charter correlation is so much weaker than district.
2026-06-10. unestablished sub-points named in the verdict (computation basis; SIRS-vs-parallel submission; why charter correlation is weaker). Evidence: snapshot/PROVENANCE.md · snapshot/provenance-stats.json
NYSED publishes school-level chronic absenteeism for 2017-18, 2018-19, 2020-21, 2021-22 → 2024-25 (SRC2025 released 2026-04-30), including NYC charters — so an all-schools universe holds for every analysis year. 2019-20 was never published (COVID). Format: Access-only Report Card Database zips; tables ACC EM/HS Chronic Absenteeism.
2026-06-09. Evidence: nysed/FORMAT_NOTES.md
BEDS→DBN is deterministic for district schools (county→borough letter + CSD + last-3-of-building-number; zero collisions across all panels). Charter BEDS codes do not encode the DBN → crosswalk needed (NYC OD wg9x-4ke6 + current LCGMS).
2026-06-09. Evidence: nysed/RECONCILIATION.md
Ten of twelve (year × grade-band) panels reconcile across publishers at the median — all comparable years (2018-19, 2020-21 → 2024-25 × grades 1–8 / 9–12; 2019-20 is not comparable: NYSED never published it and the DOE year is truncated; 2017-18 is NYSED-only): median per-school diff ≤0.4pp in the ten; r 0.987–0.997 in every grades-1–8 panel except 2024-25. Still the strongest external validation of the attendance data. The two failures: 2024-25 EM (median −1.84pp → A.11) and 2022-23 HS (median −0.60pp, only 47% within ±1pp, r 0.779 → A.20). Median agreement hides tails: the 2020-21/2021-22 HS panels pass at the median yet carry the same one-sided tail (A.20; weighted aggregates −2.1/−3.4pp, r 0.94/0.92), and 2020-21 carries the remote-attendance caveat. NYSED-only HS entities are transfer/alternative schools absent from the DOE file's school-level rows.
2026-06-10. expanded to 12 panels 2026-06-10, RECONCILIATION.md §9. Evidence: nysed/RECONCILIATION.md · nysed/stats.json · Cross-publisher page
2024-25 grades 1–8 diverges systematically: DOE shows EM chronic absenteeism falling (30.5→29.0% weighted) while NYSED shows it rising (30.0→30.8%) on the same ~1,115 schools; numerator-side; broad-based; not our aggregation; not a NYSED revision. Document review rules out any documented 2024-25 definition change on either side, and NYSED's standing suspension/medical exclusion predicts the opposite sign. One documented candidate remains: SIRS defaults missing/unpaired attendance records to "absent, unexcused". Implication: 2024-25 EM chronic absenteeism is publisher-dependent — carry source attribution. The expanded panels (2026-06-10) show the break has no precursor: EM medians in time order +0.39 → −0.13 → +0.09 → +0.13 → +0.38 → −1.84pp — a clean single-year break, broad-based (474 of ~1,115 joined schools below −2pp vs 51–61 in the COVID-era years), different in shape from the tail-concentrated A.20 phenomenon.
2026-06-09. NYSED inquiry drafted. Evidence: nysed/NARROWING.md · nysed/RECONCILIATION.md §9 · Cross-publisher page
D75 and D79 have zero rows in NYSED's CA tables in any year — NYSED cannot extend coverage there (D75 stays DOE-only; D79 has no public school-level data from either publisher).
2026-06-09. Evidence: nysed/RECONCILIATION.md
Success Academy stopped reporting attendance to NYSED network-wide in 2023-24 (DATA_REP_FLAG=N; 35/35 EM + 3/3 HS schools in 2024-25; zero flagged in 2018-19/2021-22). Charter-universe analyses must disclose this hole.
2026-06-09. Evidence: nysed/stats.json
62 / 112 / 129 of ~414 joined high schools are >5pp below NYSED in 2020-21 / 2021-22 / 2022-23 (vs 0 / 0 / 3 in 2018-19 / 2023-24 / 2024-25), largely on byte-identical denominators; 86 schools persist across 2021-22 → 2022-23; extremes reach NYSED 99.8% vs DOE 15.3% on identical N. Verified publisher-level, not a reconstruction artifact: DOE's own published school totals match the per-grade reconstruction exactly at tail schools. Median-based panel summaries hide this tail (the 2020-21/2021-22 HS panels pass at the median; 2022-23 HS fails — median −0.60pp, r 0.779). Cause not established. Charter implication: charter values are served from NYSED, so charter HS chronic absenteeism for 2020-21 → 2022-23 inherits whatever affected NYSED's HS data in those years — unverifiable directly for charters (no second publisher).
2026-06-10. Evidence: nysed/RECONCILIATION.md §9 · nysed/stats.json · Cross-publisher page
These are working analyses pending validation and a checkpoint with the newsroom. Results are published here after editorial review — the working figures are browsable, clearly marked as unverified, on the working-analysis page.
The stack could not run against a local/dev Postgres (hard-coded SSL in three pg pools; percentile scripts missing DATABASE_URL fallback; Supabase auth threw on every route when env unset).
PR #11 (local replica + dev recipe in PR body). Evidence: PR #11
The canonical ledger lives in the repository at docs/design/08_absenteeism_findings.md — this page is a rendering of it. Validation methodology: verify/METHODOLOGY.md; evidence: validation reports.