Darin Nielsen knows a bit about how headwinds negatively impact a runner’s performance.
“Those of you who are familiar with running recognize that when a headwind is in your face, it requires an increased level of exertion. It results in a decreased pace and decreases the distance you can travel in a given amount of time,” Nielsen told members of the Utah Legislature’s Education Interim Committee this week.
Nielsen, state assistant superintendent of student learning and an avid runner, uses the example to illustrate how the COVID-19 pandemic has affected student achievement in the Beehive State.
“There appears to be a pandemic effect on student achievement across all grades, subject areas, student groups with declines slightly larger in math compared to English language arts. We can think of the pandemic as an ‘academic headwind’ of sorts,” he said.
While standard test data provides insights, Nielsen urged restraint in drawing conclusions from the information “because we don’t have equal representation of student groups across each of these number sets.”
How did pandemic affect students’ test scores?
Student participation rates among the various standardized tests the state uses to measure student achievement for its school accountability system dropped in 2021 compared to 2019, and many underserved students participated at lower rates than other students.
State assessments were suspended in 2020 due to the pandemic so that data is not available for comparison and analysis.
Among high school juniors, 7% fewer took the ACT college admissions exam in 2021 compared to 2019. The statewide average composite score declined by 0.27 point, which is comparable to one month of lost instruction, Nielsen said.
The Utah Aspire Plus test, administered to students in grades nine and 10, had the sharpest decline in participation with 10% fewer students taking it in 2021. Mathematics performance dropped 46% in 10th grade and 37% in 9th grade. English language arts performance worsened by 14% in both grades, according to the analysis.
RISE assessments, short for Readiness, Improvement, Success and Empowerment, that are administered to students in grades three through eight, also reflected sharp declines in performance levels such as a 57% decline in sixth grade math and a 45% drop in fifth grade math.
‘Sobering and concerning’
Rep. Lowry Snow, R-Santa Clara, co-chairman of the Utah Legislature’s Education Interim Committee, called the data “sobering and concerning.”
An executive report provided to lawmakers by the state school board and the National Center for the Improvement of Educational Assessment suggests the pandemic effects documented in their study of Utah students’ performance “is likely an underestimate of the true pandemic effects because it is based only on students who participated in 2020-2021. This speaks to the necessity of helping all students in Utah with learning recovery going forward.”
The board’s and center’s analysis followed a method recommended by Harvard University professor Andrew Ho, who focuses on assessment. They applied Ho’s method to their comparisons of RISE and Utah Aspire Plus assessment data.
Two of the suggested metrics included a “fair trend adjustment” to account for changes in the testing population and an “equity check” that attempts to estimate the best-case academic performance of students who did not test in 2020-2021, “providing a gauge of the impact of missing students on the overall academic outcomes,” the executive summary states.
Results from student surveys administered in spring 2021 as part of RISE and the Utah Aspire Plus assessments show most students attended school in-person most of the school year.
However, about half of the students who responded to the surveys indicated they experienced quarantine protocols at least once during the school year. More than a quarter of students who took the Utah Aspire Plus assessment reported they experienced two or more quarantine interruptions while nearly 20% of those who took the RISE assessment reported multiple quarantines.
The survey also found that more frequent in-person learning and fewer instances of quarantine were associated with higher student achievement on the two assessments.
The executive document notes while there are “several potential threats to the validity of the comparisons presented in this report, we argue that the signal is loud enough to be heard through any noise. These results must be interpreted as a call to action from the statehouse to the schoolhouse.
“The learning acceleration necessary cannot be left to teachers and principals alone. School leaders, educators, and local communities will need support and resources to sustain the necessary interventions well beyond the time when the federal Elementary and Secondary School Emergency Relief funds run out.”