← Data quality

Two publishers, one metric: the NYSED reconciliation

New York State (NYSED) and NYC DOE independently publish school-level chronic absenteeism for the same schools. We compared them, school by school, across every comparable year — six years and two grade bands — the first check of our attendance data against a genuinely different publisher (check iii-b). Ten of twelve panels reconcile at the median. Two do not, and a third pattern — a COVID-era high-school tail that passing medians hide — sits beneath them. This page documents exactly what is and is not known about why. For the metric's full picture — definition, datasets, and every validation check — see the chronic absenteeism deep dive.

The metric being checked: chronic absenteeism, citywide

A student is chronically absent if they miss 10% or more of their enrolled days (with at least 10 enrolled days and 1 day present). The mean school's rate jumped from ~30% before the pandemic to 44% in 2021-22 and stood at 36.6% in 2024-25 — every trend claim on this site rides on this series, which is exactly why an independent check matters.

Chronic absenteeism across NYC schools (school-level, unweighted), from the NYC DOE end-of-year file we ingest. ~1,510–1,525 schools per year.
COVID disruption0%15%30%45%60%2018-192019-202020-212021-222022-232023-242024-25
Mean school rateMedian school rate
The full school-level distribution by year (p10–p90 whiskers, p25–p75 box, median line).
0.0%p10 — p25 — median — p75 — p9070.0%
2018-19
median 28.7%
2019-20
median 26.6%
2020-21
median 30.8%
2021-22
median 44.4%
2022-23
median 39.2%
2023-24
median 37.8%
2024-25
median 36.4%

Panel-by-panel agreement

Each panel is one school year × grade band, ~410–1,130 NYC district schools joined per school via the deterministic BEDS→DBN rule (zero collisions). diff = our DOE-derived rate − NYSED's rate, in percentage points. Every comparable year is paneled: 2019-20 is not comparable (NYSED never published it under the COVID waiver, and the DOE year ends at the March 2020 closure), and 2017-18 is NYSED-only (the DOE file starts in 2018-19).

Ten of twelve panels reconcile at the median (|median per-school diff| ≤ 0.5pp; ≤ 0.4pp in all ten). Two do not: 2024-25 grades 1–8 (median −1.84pp, only 22% within ±1pp) and 2022-23 grades 9–12 (median −0.60pp, only 47% within ±1pp, correlation 0.78). The median is also not the whole story: the 2020-21 and 2021-22 grades-9–12 panels pass at the median but hide a heavy one-sided tail of high schools far below NYSED — see the COVID-era section below. And 2020-21 carries its own documented caveat: DOE counted remote and blended attendance as present, while NYSED published 2020-21 as an informational-only year under the federal accountability waiver — agreement that year is weaker evidence than in any other panel.

Agreement per panel (NYC district schools, All Students). Source: docs/qa_reports/nysed/stats.json.

PanelJoinedMedian diff≤ ±1.0pp≤ ±1.5ppCorr.NYSED wtd.DOE wtd.
2018-19 grades 1–81,118+0.39pp74%86%0.99520.7%21.2%
2018-19 grades 9–12414+0.09pp75%86%0.99731.6%31.7%
2020-21 grades 1–81,128-0.13pp77%87%0.99624.9%24.6%
2020-21 grades 9–12414-0.26pp64%67%0.94436.5%34.3%
2021-22 grades 1–81,127+0.09pp76%85%0.99534.8%34.9%
2021-22 grades 9–12415-0.15pp63%66%0.91547.1%43.7%
2022-23 grades 1–81,123+0.13pp77%86%0.98731.3%31.5%
2022-23 grades 9–12414-0.60pp47%53%0.77941.5%36.5%
2023-24 grades 1–81,119+0.38pp74%85%0.99730.0%30.5%
2023-24 grades 9–12410+0.30pp80%90%0.99835.4%35.6%
2024-25 grades 1–81,115-1.84pp22%35%0.99330.8%29.0%
2024-25 grades 9–12410-0.02pp89%91%0.99735.0%34.6%
Median per-school difference (DOE − NYSED, percentage points). Ten cells near zero; 2024-25 grades 1–8 and 2022-23 grades 9–12 diverge — and the near-zero 2020-21/2021-22 HS medians hide the COVID-era tail documented below.
Grades 1–8 (EM)Grades 9–12 (HS)
2018-19+0.4+0.1
2020-21-0.1-0.3
2021-22+0.1-0.2
2022-23+0.1-0.6
2023-24+0.4+0.3
2024-25-1.8-0.0

The 2024-25 divergence, in one picture

On the same ~1,115 joined schools, the two datasets move in opposite directions in 2024-25: NYSED shows elementary/middle chronic absenteeism rising (30.0% → 30.8%) while the DOE file shows it falling (30.5% → 29.0%). The denominators barely move (enrollment ratio ~1.03 both years) — the disagreement is on how many students were counted absent, not how many students there were.

The expanded panels show this break has no precursor: the grades-1–8 median difference stays within ±0.4pp in every earlier panel (+0.39, −0.13, +0.09, +0.13, +0.38 in time order) and snaps to −1.84pp in 2024-25 — broad-based, with 474 of ~1,115 joined schools more than 2 points below NYSED (vs 51–61 in the COVID-era years).

Enrollment-weighted chronic absenteeism, grades 1–8, joined NYC district schools only. Note: the x-axis skips 2019-20 (never published by NYSED; the DOE year was truncated by the March 2020 closure).
15%20%25%30%35%2018-192020-212021-222022-232023-242024-25
NYSED Report CardNYC DOE end-of-year file (ours)

The full divergence distributions, panel by panel

Each strip buckets the per-school difference (NYC DOE − NYSED, percentage points) for one panel — cool blues where NYC DOE's value sits below NYSED's, neutral within ±1pp, warm above. The two failing panels and the COVID-era high-school tail are visible at a glance; counts per bucket are printed beneath each strip. Source: stats.json.

2018-19 grades 1–8 schools by gap between NYC DOE's value and NYSED's (percentage points, NYC DOE − NYSED) · n = 1,118
< −5−5 to −2−2 to −1±1+1 to +2+2 to +5> +5

green = agreement (within ±1pp) · left of it: NYC DOE runs lower than NYSED · right: higher

2018-19 grades 9–12 schools by gap between NYC DOE's value and NYSED's (percentage points, NYC DOE − NYSED) · n = 414
< −5−5 to −2−2 to −1±1+1 to +2+2 to +5> +5

green = agreement (within ±1pp) · left of it: NYC DOE runs lower than NYSED · right: higher

2020-21 grades 1–8 schools by gap between NYC DOE's value and NYSED's (percentage points, NYC DOE − NYSED) · n = 1,128
< −5−5 to −2−2 to −1±1+1 to +2+2 to +5> +5

green = agreement (within ±1pp) · left of it: NYC DOE runs lower than NYSED · right: higher

2020-21 grades 9–12 schools by gap between NYC DOE's value and NYSED's (percentage points, NYC DOE − NYSED) · n = 414
< −5−5 to −2−2 to −1±1+1 to +2+2 to +5> +5

green = agreement (within ±1pp) · left of it: NYC DOE runs lower than NYSED · right: higher

2021-22 grades 1–8 schools by gap between NYC DOE's value and NYSED's (percentage points, NYC DOE − NYSED) · n = 1,127
< −5−5 to −2−2 to −1±1+1 to +2+2 to +5> +5

green = agreement (within ±1pp) · left of it: NYC DOE runs lower than NYSED · right: higher

2021-22 grades 9–12 schools by gap between NYC DOE's value and NYSED's (percentage points, NYC DOE − NYSED) · n = 415
< −5−5 to −2−2 to −1±1+1 to +2+2 to +5> +5

green = agreement (within ±1pp) · left of it: NYC DOE runs lower than NYSED · right: higher

2022-23 grades 1–8 schools by gap between NYC DOE's value and NYSED's (percentage points, NYC DOE − NYSED) · n = 1,123
< −5−5 to −2−2 to −1±1+1 to +2+2 to +5> +5

green = agreement (within ±1pp) · left of it: NYC DOE runs lower than NYSED · right: higher

2022-23 grades 9–12 schools by gap between NYC DOE's value and NYSED's (percentage points, NYC DOE − NYSED) · n = 414
< −5−5 to −2−2 to −1±1+1 to +2+2 to +5> +5

green = agreement (within ±1pp) · left of it: NYC DOE runs lower than NYSED · right: higher

2023-24 grades 1–8 schools by gap between NYC DOE's value and NYSED's (percentage points, NYC DOE − NYSED) · n = 1,119
< −5−5 to −2−2 to −1±1+1 to +2+2 to +5> +5

green = agreement (within ±1pp) · left of it: NYC DOE runs lower than NYSED · right: higher

2023-24 grades 9–12 schools by gap between NYC DOE's value and NYSED's (percentage points, NYC DOE − NYSED) · n = 410
< −5−5 to −2−2 to −1±1+1 to +2+2 to +5> +5

green = agreement (within ±1pp) · left of it: NYC DOE runs lower than NYSED · right: higher

2024-25 grades 1–8 schools by gap between NYC DOE's value and NYSED's (percentage points, NYC DOE − NYSED) · n = 1,115
< −5−5 to −2−2 to −1±1+1 to +2+2 to +5> +5

green = agreement (within ±1pp) · left of it: NYC DOE runs lower than NYSED · right: higher

2024-25 grades 9–12 schools by gap between NYC DOE's value and NYSED's (percentage points, NYC DOE − NYSED) · n = 410
< −5−5 to −2−2 to −1±1+1 to +2+2 to +5> +5

green = agreement (within ±1pp) · left of it: NYC DOE runs lower than NYSED · right: higher

What we know about the 2024-25 divergence

We reviewed every authoritative document we could find on both calculations — NYSED's database ReadMes, report-card glossaries, the SIRS reporting manual and report guides, Regents materials, DOE's file notes, the Mayor's Management Report, and Chancellor's Regulation A-210. The honest summary:

Verified.No documented definition change on either side for 2024-25: NYSED's 2023-24 and 2024-25 definitions are word-for-word identical, and NYSED's own guide dates its only chronic-absenteeism definition change (a 10-day → 30-day enrollment window) to 2025-26. DOE's file documents one uniform methodology for all years, and DOE's separate publication channel (the Mayor's Management Report) shows the same falling direction — so the disagreement is between the DOE and NYSED data, not an artifact of one DOE file.

Ruled out.Our own aggregation (DOE's published school totals match our per-grade reconstruction exactly); a NYSED data revision (the overlapping 2023-24 rows are byte-identical across releases); denominator differences (at Brooklyn Tech the two publishers count the exact same 5,840 students — NYSED counts 979 of them chronically absent, the DOE file implies 829). The one standing, documented difference between the measures — NYSED excludes suspensions and extended medical absences — predicts NYSED lower than DOE, the opposite of what 2024-25 shows, and cannot explain a single-year, single-band divergence anyway.

Not established.Which pipeline changed. One documented mechanism remains that could inflate NYSED's numerator with no definition change on either side: under SIRS rules, a missing or incompletely paired daily-attendance record "will default to absent, unexcused absence." That is a mechanism, not a finding that it occurred — no public document shows whether it did. An inquiry to NYSED's data office is drafted. There is also no third public source of 2024-25 school-level data to break the tie.

What this means for numbers on this site. Until the cause is established, 2024-25 chronic absenteeism for grades 1–8 is publisher-dependent: the citywide level differs by ~1.7 points and the year-over-year direction differs. Our served values come from the NYC DOE file (base-case exact against it) and should carry that source attribution; claims about whether elementary/middle absenteeism improved in 2024-25 should say which publisher they rest on. See ledger findings A.11 and A.10.

The COVID-era high-school tail (2020-21 → 2022-23)

The expanded panels surfaced a second, differently-shaped disagreement that median-based summaries hide. The verified facts (full detail in RECONCILIATION.md §9):

  • 62 / 112 / 129 of ~414 joined high schools sit more than 5 points below NYSED in 2020-21 / 2021-22 / 2022-23 — vs 0 in 2018-19, 0 in 2023-24 and 3 in 2024-25. The median high school still nearly agrees each year, so this is a tail phenomenon, not a level shift.
  • It is numerator-side: denominators are byte-identical at 174 / 298 / 272 of ~414 schools, and the worst cases pair equal student counts with hugely different absent counts — extremes in 2020-21 include NYSED 99.8% vs DOE 15.3%, and NYSED 100.0% vs DOE 32.7%, on identical denominators.
  • It is publisher-level, not our reconstruction: at tail schools, DOE's own published school-total chronic absenteeism matches our per-grade reconstruction exactly (e.g. Brooklyn Tech 2021-22: NYC DOE 20.147% (the value we serve); NYSED 32.8% on a denominator that differs by one student).
  • It persists school-by-school: 86 schools are in the >5pp tail in both 2021-22 and 2022-23. The 2022-23 grades-1–8 deep tail consists entirely of secondary schools that also appear in the high-school panel — elementary-only schools are unaffected.

Cause not established.No document on hand shows what produced the tail or which schools it hit; the shape is consistent with a school-level reporting failure into the state's data system, but that is a candidate mechanism, not a finding that it occurred. Because our charter values are served fromNYSED, charter high-school chronic absenteeism for 2020-21 → 2022-23 inherits whatever affected NYSED's high-school data in those years — and cannot be verified directly for charters, since no second publisher exists. See ledger finding A.20.

Coverage limits of the check

  • District 75 and District 79 have zero rows in NYSED's chronic-absenteeism tables in any year — NYSED cannot extend coverage there.
  • Success Academy stopped reporting attendance to NYSED network-wide in 2023-24 (35/35 elementary-middle and 3/3 high schools flagged as non-reporting in 2024-25), so charter-universe comparisons have a disclosed hole.
  • Charters were excluded from this join (charter BEDS codes do not encode the DBN; a crosswalk is the follow-up).

Full write-ups on GitHub: RECONCILIATION.md (method + per-panel results) · NARROWING.md (the document review behind the section above) · FORMAT_NOTES.md (NYSED source scouting) · stats.json (machine-readable numbers rendered here).