Steady Courses in a Sea of Change: Analysis of Student Ratings Pre-Post Pandemic

Nearly a decade after designing my courses based on Quality Matters (QM) Standards and teaching fully online, the March 2020 pivot to emergency remote instruction was business as usual for my students while I assisted my colleagues with the transition online. As the subsequent semesters passed, I got the impression that my student ratings of instruction were rising though I had not made significant changes to my courses because, like many working adults, I was overwhelmed with homeschooling my children, coordinating schedules with my spouse who was also working from home, and coping with life changes the pandemic brought. As a sense of normalcy returned, I gained the ability to analyze my students’ ratings to address my curiosity about whether the increase I suspected was real or merely perceived. This session will describe how I leveraged my end-of-semester student ratings of instruction as longitudinal data to examine pre-post pandemic changes. The analysis revealed large discrepancies in student ratings when the institution shifted measures despite transforming the data to the same scale. Implications of the shift will be discussed. The analysis of the currently used measure pre- to post-pandemic indicated that student ratings of my courses increased after the pivot to emergency remote instruction as suspected though ratings of the instructor and progress on course objectives were unchanged. That my courses were designed with QM in mind may be associated with the increase in ratings as online courses that were not designed based on QM Standards flooded students’ schedules. Attendees will leave the session able to discuss research on student ratings of instruction that includes cautions about using them to generalize across faculty members. And, they will have examples of how they can analyze students’ ratings of instruction in a manner that controls for faculty individual differences. When data is collected longitudinally, it provides a way to examine the impact of course changes despite small sample sizes that can be used to inform continuous improvement.