Do Examinees Have Similar Test-Taking Effort? A High-Stakes Question for Low-Stakes Testing

“2011), quality of studying and test-taking ability (Barry et al. 2010), and motivation for learning (Wolters and Taylor 2012).”

“Examinee effort is on average not only lower in low-stakes assessments compared to high-stakes assessments, but it is also likely to be more variable (Barry et al., 2010).”

“…(e.g., DeMars, 2000; Wise, 2006; Wolf et al., 1995), but to our knowledge, only one study has examined effort across multiple tests within a single testing session (Barry et al., 2010), and there has been no study explicitly examining change in test-level effort across tests in a testing session.”

“Recent research in the domain of test-taking motivation suggests the influence of the personality characteristics of conscientiousness and agreeableness on expended test-taking effort (Barry et al. 2010; Barry & Finney, 2014; DeMars et al., 2013).”

“As typically found, the results from this assessment suggested that students were moderately to highly motivated on average (e.g., Barry et al. 2010; Thelk, Sundre, Horst, & Finney, 2009).”

“Motivational changes within a single test based on self-report scales The results from Barry and Finney (2016) and Barry et al. (2010) primarily illuminate how testtaking motivation changes across tests within a single testing session, but do not address how motivation changes across items during a…”

“For example, examinees who put forth less effort on low-stakes tests may be lower in openness, conscientiousness, and agreeableness than examinees who put forth more effort, as found in previous studies (Barry & Finney, 2016; Barry et al., 2010).”

“It has been found by several researchers that, in high-stakes tests, students are more motivated and get better test results than in low-stakes tests (Barry et al. 2010, Baumert and Demmrich 2001, Eklöf 2006, 2010, Sundre and Kitsantas 2004, Wise and DeMars 2005, 2010).”

“The influence of test-taking motivation on educational test performance is clearly documented (Barry, 2010; Baumert & Demmrich, 2001; Eklöf, 2010; Sundre & Kitsantas, 2004; Wise & DeMars, 2005, 2010).”

“…2013), age groups (e.g., Wise & DeMars, 2010), items within a test (e.g., Wise, 2006; Wise et al., 2009), or tests within a testing session (e.g., Barry et al., 2010), these results do not imply variation in change in effort across distal pretest and posttest occasions, hence the need for the…”

“In low-stakes testing, in contrast, the test result has no consequences for testtakers [9], which may decrease the subjective task value, resulting in lower motivation or test-taking effort and, in turn, lower performance [10, 11].”

“Low pro fi ciency examinees have been previously shown to engage less with items in low-stakes assessments (Azzolini et al., 2019; Barry et al., 2010; Wise & Gao, 2017).”

“Low test-taking effort can be a significant threat to the validity in both highstakes and low-stakes testing contexts, but it occurs more frequently with low-stakes assessments (Barry et al., 2010; Simzar et al., 2015; Wise, 2017).”

“…previous research found that conscientiousness is positively related to effort (Barry & Finney, 2016; Barry, Horst, Finney, Brown, & Kopp, 2010; Richardson et al., 2012; Yeo & Neal, 2008), performance (Finney, Sundre, Swain, & Williams, 2016; Poropat, 2009), and importance (Barry et al., 2010).”

“In the absence of external pressure and motivational drivers, variability in motivation is higher than in the presence of external incentives and the influence of PSEC on performance tends to be stronger (Barry et al., 2010; Cole et al., 2008; Wise and DeMars, 2010).”

“The SOS items have been used in this manner to gauge changes in test importance and effort during a testing session (e.g., Barry & Finney, 2016; Barry et al., 2010) and during a test (e.”

“Would changes in our grading policy affect TOSLS scores or self-reported student effort? While it is impossible to predict changes in TOSLS student scores as a response to further increases in stakes or value, previous literature in the field shows that, in the context of low-stakes testing, the cognitive demand of a…”

“…raised a number of methodological concerns about existing ILSAs, including concerns about variation in the motivation of students to take the tests (Barry et al, 2010), criticisms of the use of unidimensional summaries of student achievement (Goldstein, 1995), and the concentration on…”

“If a small number of points are in play, many students reduce their effort and test and writing assignments scores fall (Barry et al. 2010; Wise and Demars 2005; Elbow 1997; Frank and Pearce 2016).”

“These PTs have no direct consequences (i.e. no ‘stakes’) for study progress, which may lead to a lower perceived value, which in turn may result in less test-taking motivation and effort put in these tests (Barry et al., 2010 ; Schüttpelz-Brauns et al., 2020 ).”

“In addition, previous research into online learning has repeatedly demonstrated that some students will submit answers to problems in unusually short amounts of time, which are often attributed to either random guessing or answer copying [20–22].”

“Thus, the scores may partly reflect differences in effort levels rather than variation in domain scores of interests (Barry et al., 2010), which may also have some influence on response styles.”

“Studies reported that unmotivated students might show non-effortful response behavior in both summative and formative assessments (Barry et al., 2010; Simzar et al., 2015).”

“Test-taking effort could be expected to be higher in high-stakes assignments, which may have personal consequences for the test takers, than in low-stakes contexts, where no such consequences exist (Barry et al., 2010).”

“…on different cognitive processes (Haladyna & Rodriguez, 2013), the different response strategies employed by students (Bridgeman, 1992), or their test-taking motivation, which is likely lower when facing items that require text production in low-stakes settings such as PISA (Barry et al., 2010).”

“Therefore, examining students’ test-taking efforts in large-scale assessments has become an important topic in recent years (Barry et al., 2010; Silm et al., 2019).”

“So far, evidence was provided for single predictors (Ackerman and Kanfer 2009; Asseburg and Frey 2013; Barry et al. 2010; Boe et al. 2002; Duckworth et al. 2011; Giermann 2012; Pekrun 1993; Pohlmann et al. 2005; Tucker-Drob et al. 2016; Yeo and Neal 2008).”

“Furthermore, we did not assess TTM for the completion of the student questionnaire like previous studies (Barry and Finney 2016; Barry et al. 2010; Horst 2010).”

“Previous research has shown that disengagement tends to increase over the course of a testing session, with later tests in a series often resulting in more disengagement (Barry & Finney, 2016; Barry et al., 2010; Finney & McFadden, 2023; McFadden & Finney, 2024).”

“However, some evidence indicates that effort on a low-stakes test may be independent of its perceived importance, and thus should be unaffected by the lack of consequences resulting from testing in a low-stakes context (Barry & Finney, 2016; Barry et al., 2010).”

“Research conducted by Barry et al. (2010) shows that test scores may not truly reflect the abilities of examinees in cases when there is little or no personal consequence for the test taker.”

“Barry et al. (2010) identified three groups of participants in a low-stakes, postsecondary testing environment: a) people who were generally low in effort, but who reported even lower effort on a demanding cognitive test, b) people who were generally high in effort but who also reported lower…”

“Openness has been found to be related to test-taking effort (e.g., Barry et al., 2010; Borghans et al., 2024; Schaefer & Finney, 2025) and test performance (e.g., Finney, Sundre et al., 2016; Schaefer & Finney, 2025).”

“…& Kitsantas, 2004; Wolf et al., 1995), and the individual features that can impact test-taking motivation and test performance (Barry & Finney, 2015; Barry et al., 2010; Brown & Finney, 2011; Kopp & Finney, 2013; Kopp et al., 2011; Swerdzewski et al., 2011), the demand for further studies in this…”

“Indeed, test-taking effort is commonly more variable in low-stakes contexts (Barry et al., 2010), which in combination with their lower level of standardisation implies that FA ability estimates are normally less reliable than those derived from SAs.”

“Barry et al. (2010) showed that item durations were correlated with Big 5 personality characteristics, including for conscientiousness, which is related to persistence.”

“Several studies have found that test-taking efforts can rise or fall during testing sessions (Barry et al., 2010; Barry & Finney, 2016; Pastor et al., 2019; Penk & Richter, 2017).”

“…Wichtigkeit des Tests innerhalb einer dreistündigen Testsitzung, in der Studierende einen Leistungstests und vier Einstellungsfragebögen (im Original nichtkognitive Tests genannt, wie z. B. Einstellung zur Universität) bearbeiteten (Barry & Finney, im Druck; Barry et al., 2010; Horst, 2010).”