How much of the 2024-25 'recovery' is a test-recalibration artifact?

Citywide averages of the percentage of students proficient on the New York State tests, year by year, for math and English Language Arts in grades 3, 5, and 8. The pattern repeats across every test: an upward jump in 2022-23 (the first year of the new test) and a bigger one in 2024-25. The years in between move 1 to 3 points — the size of a real (small) annual change. The jumps don't.

Whether a student is labeled 'proficient' depends on whether their raw test score (the scale score) clears a cutoff line. Raw scores are the actual underlying measure: scaled numbers from 100 or so on up. If kids were really learning more or less, raw scores would shift gradually. They don't. In 2022-23, New York State released a new test on a new scale — and the underlying average raw score dropped about 150 points all at once, across every grade and subject, on the same day. The students didn't suddenly get worse; the ruler changed.

Since the new test launched in 2022-23, raw test scores have stayed nearly flat. Third-grade math went from 452.6 in 2022-23 to 451.7 the next year to 457.8 in 2024-25 — only about 5 points of climb over two years. Meanwhile, the percentage of students proficient over the same span jumped from 53% to 54% to 62% — a 9-point climb. Students' measured skill barely changed. What changed was the share of students scoring above the proficient cutoff. The simplest explanation: the cutoff itself moved lower.

One way to size whether a jump is unusual is to compare it with how much scores normally move year to year. Across the six grade-and-subject combinations and the four 'normal' year-to-year transitions (skipping the COVID disruption), the typical change is about 1.7 percentage points. The 2022-23 and 2024-25 transitions are three to five times that — well outside the normal range.

Between 2018-19 and 2024-25, two distinct things happened to the New York State tests:

• 2022-23 — the test was rebuilt. New York State released a new test aligned to new learning standards (the 'Next Generation Learning Standards'). The whole scoring scale was rebuilt; what had been a roughly 600-point scale became a roughly 450-point scale. This was publicly disclosed in technical documents from the state education agency but rarely shows up in news coverage of test results.

• 2024-25 — the proficient cutoff appears to have moved. Proficiency rates jumped 5 to 15 points across grades while raw test scores stayed nearly flat. That's the signature of a moved cutoff. The state hasn't publicly described it as a cutoff change, so this is something we're inferring from the data rather than a stated policy action. But the simplest explanation that fits the data is that the cutoff moved.

Between those two events, year-to-year scores moved by 1 to 3 points — normal. Across the period, almost all of the city's reported 'recovery' from pandemic-era lows comes from these two test changes, not from teaching changes.

Within a single year (like 2024-25), comparing one school with another is fine: every school takes the same test on the same scale at the same time. Comparing different years is the problem. Without adjusting for the test changes, year-over-year score comparisons across 2022-23 or 2024-25 will look like big movements that aren't real. The best outside benchmark is NAEP — the federally run test, called in full the National Assessment of Educational Progress, which is given the same way in every state and isn't subject to New York State's changes. NAEP gives the test to a sample of students rather than every student, so it carries its own measurement noise. But it's the cleanest available outside check. NAEP's 2024 results for New York City fourth-graders: reading down 5 points, math up 2 points, both compared with 2022. Much smaller movements than the state-test 'recovery' implies — which is what we'd expect if the state-test movement is mostly the test changing.