Authored by Andria F. Schwegler, Associate Professor of Psychology in the Counseling and Psychology Department at Texas A&M University - Central Texas
Andria teaches a range of fully online courses at the graduate and undergraduate levels, including statistics, research methods, history of psychology, and social psychology in addition to courses in the psychology of learning and educational technology. She is the Graduate Coordinator for the Master of Science in Educational Psychology program.
Given the absence of a widely accepted definition of academic rigor, an erosion of academic expectations in the learning context is a threat to student learning. An objective, measurable definition of academic rigor can facilitate high expectations by prompting a review and implementation of research-based practices. Evidential support for decisions made in crafting and maintaining the learning context provides a rationale for educators to uphold standards even when confronted with resistance or pressure to lower expectations. The first in a three-part series (Paper 2 and Paper 3) from this invited author, this QM white paper provides the reader with background on the concept and use of the term academic rigor, along with a comprehensive definition of the term.
The Role of Learning in Higher Education
Discussions of academic rigor assume that the purpose of obtaining a higher education is to advance student learning, and academic rigor is a mechanism to do so. Today, however, learning as the primary goal of higher education is being questioned as many struggle to identify what the purpose of higher education is or how it applies to real world and work contexts (Francis, 2018; Jaschik & Lederman, 2018; Labaree, 1997). Concerns are supported by evidence revealing that institutions of higher education are marketing experiences unrelated to learning (Hartley & Morphew, 2008; Saichaie & Morphew, 2014) and that most adults report attending college to get credentials for jobs instead of to learn or gain knowledge (Strada-Gallup Education Consumer Survey, 2018, January). These perceptions suggest that higher education serves a social and job credentialing function instead of promoting student learning, undermining the value of academic rigor in the learning context.
Empirical Evidence on Student Learning
Recent research evidence indicates that students accrue small learning gains in the first two years of college (Arum & Roksa, 2011; Blaich & Wise, 2011; Roohr, Liu, & Liu, 2016) and continue on this trajectory of small gains during the last two years of college (Arum & Roksa, 2014; Blaich & Wise, 2011; Roohr et al., 2016). These results are based on measures assessing general proficiencies (e.g., writing, critical thinking) on standardized assessments. Larger estimates of student learning have been obtained when assessments are based directly on the curriculum to which students have been exposed (Hathcoat, Sundre, & Johnston, 2015; Mathers, Finney, & Hathcoat, 2018). However, gains in learning that can be attributed to the college academic curriculum remain clouded by a lack of comparison groups and a lack of clarity in curriculum alignment. Assessments that are not aligned with the curriculum weaken the ability to document student learning, and poor curricular planning may undermine strategies to support rigor.
Academic Rigor As a Negotiable Standard
Utilizing heuristics (e.g., an institution’s prestige or selectivity) as a proxy measure for determining what a college graduate learned and believing that a college degree is sufficient regardless of the learning underlying it (Labaree, 1997) pose threats to academic rigor in the learning context, making it appear as an unnecessary impediment of students’ progress. This perception is supported by evidence of pressure from students to lower academic expectations and teachers’ compliance (Schnee, 2008). Definitions of rigor that are based on comparisons to idiosyncratic personal experiences (Schnee, 2008) and reports of educators assigning higher grades than students earned (Schutz, Drake, & Lessner, 2013) support the concern that what is considered academically rigorous is highly malleable. In a social context in which lowering academic expectations may be rewarded (i.e., an emphasis on completion instead of learning, Jaschik & Lederman, 2018), a wide variety of activities could be considered academically rigorous depending on the unique individuals in — and circumstances that surround — a learning context, qualities that are highly variable. Such definitions of rigor lack generalizability and create conditions to perpetuate educational inequalities (Keller, 2018; Schnee, 2008). While students’ life experiences are integral in learning (Choy, 2014; Keller, 2018), these characteristics should not limit the learning context; they should be used to interpret the curriculum, not replace it (Schnee, 2008). A definition of academic rigor is needed that is based on elements of the learning context that are crafted to foster learning, a definition that can be substantiated by research and generalized across contexts. Valuing students’ experiences but relying on research evidence regarding teaching and learning in crafting the educational context prioritizes learning for all students, a shared value across institutional mission statements (Morphew & Harley, 2006).
Qualities of Rigor Derived from Experience with Teaching and Learning
Despite the widely shared goal of providing a liberal arts education via rigorous educational experiences, finding consensus on a definition of academic rigor and its application has been problematic (e.g., Hechinger Institute, 2009). A review of existing descriptions based on experience with teaching and learning reveals important characteristics for defining academic rigor. Common themes include setting and enforcing high expectations and standards for academic performance (Draeger et al., 2013; Graham & Essex, 2001; Schnee, 2008; Whitaker, 2016); crafting learning experiences that require active cognitive engagement (Draeger et al., 2013; Graham & Essex, 2001; Schnee, 2008; Whitaker, 2016; Wraga, 2010); grounding learning experiences in the knowledge, skills, and abilities that learners will need in their personal and professional lives (Draeger et al., 2013; Whitaker, 2016; Wraga, 2010); requiring learners to spend time engaging with academic content beyond time spent in class (Draeger et al., 2013; Draeger et al., 2015); and providing academic support for learners as they engage with content and in learning experiences (Graham & Essex, 2001; Schnee, 2008; Whitaker, 2016). These characteristics provide insight into setting and assessing the conditions for academic rigor and situate it in a context that extends beyond the physical and/or virtual walls of a classroom.
Qualities of Rigor Derived from Psychological Research on Human Learning
In addition to the insights of educators and learners, psychological research on human learning must be integrated into our understanding of academic rigor. As Bjork and Bjork (2011) argued, “we can be misled by our subjective impressions” when we seek to identify ways that we learn best (p. 57). Though counterintuitive, difficulties imposed in the learning context that hamper initial learning can promote long-term learning (Bjork & Bjork, 2011). Desirable difficulties demonstrated through empirical research to improve learning include varying the context in which learning occurs (Smith & Vela, 2001), spreading learning activities out over time (Donovan & Radosevich, 1999), simultaneously learning information on separate concepts (Taylor & Rohrer, 2010), and testing frequently (Roediger & Karpicke, 2006). Because these elements may slow initial learning, our perceptions may lead us to erroneously conclude that they should be avoided. But, in doing so, we are not setting the conditions to promote learning, and we are not likely to realize it. Subjective judgements of ability (Kruger & Dunning, 1999) and the degree memory can change due to studying (Kornell & Bjork, 2009) are not objectively accurate. As such, empirical research on human learning can reveal techniques to improve learning that our perceptions and experiences cannot.
An Integrated Definition of Academic Rigor
Research on human learning provides recommendations that are not fully captured by faculty members’ and students’ descriptions of academic rigor. Each type of information provides unique insights. Thus, to best facilitate learning, an interdisciplinary approach that is open to new information as it develops is needed. As our knowledge of how to facilitate learning expands, the conditions to promote rigor will increase as well. As such, academic rigor is an ongoing process of setting the conditions to promote learning. Though it is associated with desirable difficulties, cognitive effort, and time dedicated to academic tasks, academic rigor is a positive quality of the learning environment because its goal is to promote student learning, the purpose of higher education. Conceptualized as conditions that are set to facilitate learning allows these conditions to be objectively observed and evaluated along with their relationship to artifacts students produce as evidence of learning. Further, these conditions apply to any learning context; they are not limited to only higher-level courses or elite student samples (e.g., graduate work or other advanced study). Though higher-level courses will differ in curriculum and the types of cognitive effort expected, all learning contexts can be examined in terms of the conditions set to foster learning, and such an examination of context is necessary to document academic rigor. Merely stating that academic rigor exists or offering as evidence of learning static, indirect artifacts (e.g., a list of curriculum or course learning objectives, summative course grade, grade point average, standardized test score) is insufficient to demonstrate academic rigor. Instead, educators need opportunities to reveal how the process of academic rigor plays out in specific learning contexts. Just as academic rigor requires multiple perspectives to inform its definition, it also requires multiple lines of evidence to demonstrate its existence. Defining academic rigor as intentionally crafted and sequenced learning activities and interactions that are supported by research and provide students the opportunity to create and demonstrate their own understanding or interpretation of information and support it with evidence allows for the consideration of multiple factors that can facilitate or undermine rigor.
Conclusion
The proposed definition of academic rigor accommodates research supporting the efficacy of teaching techniques on student learning from a variety of disciplines. It does not prescribe specific techniques as it is inclusive of practices that are supported by research to document their efficacy, and this definition accommodates new research on practices yet unstudied. As such, educators have the flexibility to utilize teaching practices that suit them that are supported by research evidence without limiting faculty members to particular techniques. This perspective acknowledges that not all techniques are equally effective at promoting student learning and calls for research to enable educators to gauge efficacy of techniques to best facilitate student learning.
References
Arum, R., & Roksa, J. (2011). Academically adrift: Limited learning on college campuses. Chicago, IL: University of Chicago Press.
Arum, R., & Roksa, J. (2014). Aspiring adults adrift: Tentative transitions of college graduates. Chicago, IL: University of Chicago Press.
Bjork, E. L., & Bjork, R. A. (2011). Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning. In M. A. Gernsbacher, R. W. Pew, L. M. Hough, & J. R. Pomerantz (Eds.) Psychology and the real world: Essays illustrating fundamental contributions to society (pp. 56-64). New York, NY: Worth.
Blaich, C., & Wise, K., (2011, January). From gathering to using assessment results: Lessons from the Wabash National Study. (Occasional Paper #8). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment. Retrieved from http://www.learningoutcomeassessment.org/documents/wabash_001.pdf
Chen, J. C. (2014). Teaching nontraditional adult students: Adult learning theories in practice. Teaching in Higher Education, 19(4), 406-418. doi: 10.1080/13562517.2013.860101
Donovan, J. J., & Radosevich, D. J. (1999). A meta-analytic review of the distribution of practice effect: Now you see it, now you don’t. Journal of Applied Psychology, 84(5), 795-805.
Draeger, J., del Prado Hill P., Hunter, L. R., & Mahler, R. (2013). The anatomy of academic rigor: The story of one institutional journey. Innovative Higher Education, 38, 267-279.
Draeger, J., del Prado Hill, P., & Mahler, R. (2015). Developing a student conception of academic rigor. Innovative Higher Education, 40(3), 215-228. doi: 10.1007/s1075 5-014-9308-1
Francis, C. (2018). Academic rigor in the college classroom: Two federal commissions strive to define rigor in the past 70 years. New Directions for Higher Education, 181, 25-34. doi: 10.1002/he.20268
Graham, C., & Essex, C. (2001). Defining and ensuring academic rigor in online and on-campus courses: Instructor perspectives. Annual Proceedings National Convention of the Association for Educational Communications and Technology, 1-2, 330-337.
Harding, L. M. (2018). Students of a feather “flocked” together: A group assignment method for reducing free-riding and improving group and individual learning outcomes. Journal of Marketing Education, 40(2), 117-127. doi: 10.1177/0273475317708588
Hartley, M., & Morphew, C. C. (2008). What’s being sold and to what end? A content analysis of college viewbooks. Journal of Higher Education, 79(6), 671-691.
Hathcoat, J. D., Sundre, D. L., & Johnston, M. M. (2015). Assessing college students’ quantitative and scientific reasoning: The James Madison University story. Numeracy, 8(1), Article 2. doi: http://dx.doi.org/10.5038/1936-4660.8.1.2 Retrieved from https://scholarcommons.usf.edu/numeracy/vol8/iss1/art2/
Hechinger Institute. (2009). Understanding and reporting on academic rigor. Retrieved from http://hechinger.tc.columbia.edu/primers/Hechinger_Institute_Rigor_Primer.pdf
Hutchings, P., Huber, M. T., & Ciccone, A. (2011). The scholarship of teaching and learning reconsidered: Institutional integration and impact. San Francisco, CA: Jossey-Bass.
Jaschik, S., & Lederman, D. (2018). 2018 survey of community college presidents: A study by Inside Higher Ed and Gallup. Retrieved from https://www.insidehighered.com/booklet/2018-survey-community-college-presidents
Keller, C. M. (2018). Reframing rigor: Implication for institutional practice and policy. New Directions for Higher Education, 181, 89-96. Retrieved from https://onlinelibrary.wiley.com/doi/full/10.1002/he.20273
Kornell, N., & Bjork, R. A. (2008). Learning concepts and categories: Is spacing the “enemy of induction”? Psychological Science, 19(6), 585-592.
Kornell, N., & Bjork, R. A. (2009). A stability bias in human memory: Overestimating remembering and underestimating learning. Journal of Experimental Psychology: General, 138(4), 449-468.
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121-1134.
Labaree, D. F. (1997). Public goods, private goods: The American struggle over educational goals. American Educational Research Journal, 34(1), 39-81.
Mathers, C. E., Finney, S. J., & Hathcoat, J. D. (2018). Student learning in higher education: A longitudinal analysis and faculty discussion. Assessment & Evaluation in Higher Education, 43(8), 1211-1227.
Morphew, C. C., & Hartley, M. (2006). Mission statements: A thematic analysis of rhetoric across institutional type. Journal of Higher Education, 77(3), 456-471.
Pleitz, J. D., MacDougall, A. E., Terry, R. A., Buckley, M. R., & Campbell, N. J. (2015). Great expectations: Examining the discrepancy between expectations and experiences on college student retention. Journal of College Student Retention: Research, Theory & Practice, 17(1), 88-104.
Roediger, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181-210. doi: 10.1111/j.1745-6916.2006.00012.x
Roohr, K. C., Liu, H., & Liu, O. L. (2017). Investigating student learning gains in college: A longitudinal study. Studies in Higher Education, 42(12), 2284-2300. doi: 10.1080/03075079.2016.1143925
Ryan, J. J., Anderson, J. A., & Birchler, A. B. (1980). Student evaluation: The faculty responds. Research in Higher Education, 12(4), 317-333.
Saichaie, K., & Morphew, C. C. (2014). What college and university websites reveal about the purposes of higher education. Journal of Higher Education, 85(4), 499-530.doi: 10.1353/jhe.2014.0024
Schnee, E. (2008). “In the real world no one drops their standards for you”: Academic rigor in a college worker education program. Equity & Excellence in Education, 41(1), 62-80. doi: 10.1080/10665680701764502
Schutz, K. R., Drake, B. M., & Lessner, J. (2013). Do community college full-time and adjunct faculties differ in their perceptions of rigor in assigning grades? American Journal of Educational Studies, 6(2), 59-77.
Smith, S. M., & Vela, E. (2001). Environmental context-dependent memory: A review and meta-analysis. Psychonomic Bulleting & Review, 8(2), 203-220.
Stanny, C. J. (2016). Reevaluating Bloom’s Taxonomy: What measurable verbs can and cannot say about student learning. Education Sciences, 6(37), 1-12. doi: 10.3390/edusci6040037
Strada Education Network and Gallup (2018, January). Why higher ed? Top reasons U.S. consumers choose their educational pathways. Retrieved from https://news.gallup.com/reports/226457/why-higher-ed.aspx
Taylor, K., & Rohrer, D. (2010). The effects of interleaved practice. Applied Cognitive Psychology, 24, 837-848.
Whitaker, M. (2016). (Re)defining academic rigor: From theory to praxis in college classrooms. Currents in Teaching & Learning, 8(1), 4-17.
Wraga, W. G. (2010). What’s the problem with a “rigorous academic curriculum”? Paper presented at the meeting of the Society of Professors of Education/American Educational Research Association, Denver, Colorado. Retrieved from https://eric.ed.gov/?id=ED509394
Yan, V. X., Thai, K. P., & Bjork, R. A. (2014). Habits and beliefs that guide self-regulated learning: Do they vary with mindset? Journal of Applied Research in Memory and Cognition, 3, 140-152. doi: 10.1016/j.jarmac.2014.04.003
