Educators, parents and students in Utah are wondering how the state intends to deal with scores from the 2019 RISE assessment season considering last spring’s technology platform disruptions. As a result of Questar’s failure to meet multiple contractual obligations, the Utah State Board of Education canceled its contract with Questar, the vendor that provided the technological platform to administer statewide RISE assessments to students in grades 3-8 in the 2018-19 school year. The board also launched a review process to determine whether assessment data from this year is valid.
The data was analyzed by three entities: Educational Testing Service (which owns Questar but was validated by an independent third party), the Center for Assessment (an independent party), and staff from the Utah State Board of Education and Salt Lake City and Jordan school districts. Those results were then reviewed by the state’s Assessment and Accountability Policy Advisory Committee (consisting of local advisers) and the Technical Advisory Committee (consisting of five national experts in the area of psychometrics and assessment).
All review groups determined that available evidence suggests the margin of error for the data is within the expected range and results are reportable. This is largely because the interruptions happened between active testing sections and not while students were in the middle of a section.
The review found there were fewer statistical anomalies this year than a couple of years ago when the writing portion of the test was adjusted. These consistent outcomes are most certainly a result of the extraordinary efforts of our teachers and principals to navigate the delivery irregularities and the support of our students and parents despite the interruptions.
Having said that, we as State Board of Education members have heard many reports from educators questioning how valid scores can be amid so much testing frustration. One superintendent told us this month that while he is willing to own his district’s testing data, he has serious concerns this year about its validity. I have heard from board members from six school districts and a charter school representative who expressed similar concerns.
At our board meeting this month, I reiterated to our members and guests that we have solid data to be used for macro-level analyses, for teachers to compare year-to-year trends and for all to take a wide look at the achievement of student groups. This data is also helpful for schools wanting to collaborate with each other.
However, we also have myriad qualitative data and anecdotal evidence from our teachers that the testing interruptions truly were disruptive, interfering with our students’ ability to demonstrate all that they know and can do. Results from said disruptions are certainly hard to quantify. We can’t disregard that information even as we consider the quantitative analyses and reports.
As a board, we have unanimously asked our staff to seek legislative flexibility on RISE test accountability. This could include a suspension of school letter grades and a process for ensuring schools are not erroneously identified for the School Turnaround program. We also plan to include a disclaimer on the reporting webpage “Data Dashboard” that lets the public know there were testing interruptions with the technology platform.
As parents, teachers, administrators, local and state board members and legislators, we are on the same team. We need to listen to each other and seek understanding. As parents, practitioners and policymakers, we can and should work together for students and schools.
Cindy Davis represents District 9 on the Utah State Board of Education. She has worked as a teacher, administrator and district area special education director in Utah and now works as an adjunct professor at Utah Valley University.