The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Addressing Response-shift Bias: Retrospective Pretests in Recreation Research and Evaluation
Abstract
The self-reported pretest/posttest has been commonly used to assess change in recreation research and evaluation efforts. The viability of comparing pre and post measures relies on the assumption that the scale of measurement, or metric, is the same before and after an intervention. With self-report measures, the metric resides within the study participants and, thus, can be directly affected by the intervention. If participants' levels of self-knowledge change as the result of a recreation program, then this metric may also shift, making comparisons between measures from before and after the program problematic. This article aims to both synthesize the theory and literature surrounding this problem and to offer a mixed-methods, data-based example, which illustrates the problem in a recreation context and posits possible reasons for differences in reported precourse attribute levels by reporting time. Results generally supported using a retrospective pretest as a way to address changing metrics with self-report measures. This article further discusses when and how it is appropriate to use retrospective pretests in recreation research and evaluation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Especially when assessing skills prior to educational intervention, learners tend to overestimate their competency because they often cannot yet fully grasp the depth of the field, an effect commonly referred to as response-shift bias [29,30].”
From Evaluating AI Courses: A Valid and Reliable Instrument for Assessing Artificial-Intelligence Learning through Comparative Self-Assessment (2023) · cites this paper as [30]
Citations
Cites
Hierarchical Linear Models: Applications and Data Analysis Methods.
Longford, N. T., Bryk, A. S., & Raudenbush, S. W. (1993) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2075823 · Cited by 18,773
(1963) · Medical Entomology and Zoology · Cited by 15,085
Hierarchical Linear Models: Applications and Data Analysis Methods
Davidian, M. (2003) · Journal of the American Statistical Association · doi:10.1198/jasa.2003.s288 · Cited by 13,820
How we should measure "change": Or should we?
Cronbach, L. J., & Furby, L. (1970) · Psychological Bulletin · doi:10.1037/h0029382 · Cited by 2,456
Therapeutic Songwriting in Music Therapy
Baker, F. A., Wigram, T., Stott, D., & McFerran, K. S. (2008) · Nordic Journal of Music Therapy · doi:10.1080/08098130809478203 · Cited by 1,140
A growth curve approach to the measurement of change.
Rogosa, D., Brandt, D., & Zimowski, M. F. (1982) · Psychological Bulletin · doi:10.1037/0033-2909.92.3.726 · Cited by 1,084
Measuring Program Outcomes: Using Retrospective Pretest Methodology
Pratt, C. C., McGuigan, W. M., & Katzev, A. R. (2000) · American Journal of Evaluation · doi:10.1177/109821400002100305 · Cited by 611 · cited 1×
“Pratt et al. (2000) posit that memoir-is a greater problem when the questions are more genet-al in focus.”
BackgroundResponse-shift bias: A source of contamination of self-report measures.
Howard, G. S., & Dailey, P. R. (1979) · Journal of Applied Psychology · doi:10.1037/0021-9010.64.2.144 · Cited by 565 · cited 1×
“…in an effort to please tbe investigator or program (i,e,, acqtiiescence) or to justify-tbe efTort expended in tbe program, a false negative result migbt be presented in tbe bope of gaining access to additional training or treatment (e.g., Howard & Dailey, 1979; Spangers & Hoogsiraten, 1988).”
BackgroundHoward (1980) · doi:10.1177/0193841x8000400105 · Cited by 531
Howard, G. S., Ralph, K., Gulanick, N. A., Maxwell, S. E., Nance, D. W., & Gerber, S. K. (1979) · Applied Psychological Measurement · doi:10.1177/014662167900300101 · Cited by 482
Underreporting by cancer patients: The case of response-shift
Breetvelt, I. S., & van Dam, F. S. A. M. (1991) · Social Science & Medicine · doi:10.1016/0277-9536(91)90156-7 · Cited by 309
Lam, T. C. M., & Bengo, P. (2003) · American Journal of Evaluation · doi:10.1177/109821400302400106 · Cited by 280
INTERNAL INVALIDITY IN STUDIES EMPLOYING SELF‐REPORT INSTRUMENTS: A SUGGESTED REMEDY
Howard, G. S., Schmeck, R. R., & Bray, J. H. (1979) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1979.tb00094.x · Cited by 151
Caldwell, L. L., Baldwin, C. K., Walls, T. A., & Smith, E. (2004) · Journal of Leisure Research · doi:10.1080/00222216.2004.11950026 · Cited by 137
Cox, S. (1991) · Nursing Standard · doi:10.7748/ns.6.3.52.s63 · Cited by 128
Enhancing a Sense of Independence and Psychological Well-Being Among the Elderly: A Field Experiment
Searle, M. S., Mahon, M. J., Iso‐Ahola, S. E., Adam, H., & van Dyck, J. (1995) · Journal of Leisure Research · doi:10.1080/00222216.1995.11949737 · Cited by 116
Personal recall and the limits of retrospective questions in surveys.
Pearson, R. W., Ross, M. A., & Dawes, R. M. (1992) · Cited by 113
Pretesting effects in retrospective pretest^posttest designs.
Sprangers, M. A. G., & Hoogstraten, J. (1989) · Journal of Applied Psychology · doi:10.1037//0021-9010.74.2.265 · Cited by 107
The Feasibility of Informed Pretests in Attenuating Response-Shift Bias
Howard, G. S., Dailey, P. R., & Gulanick, N. A. (1979) · Applied Psychological Measurement · doi:10.1177/014662167900300406 · Cited by 99
Rohs, F. R. (1999) · Journal of Agricultural Education · doi:10.5032/jae.1999.04028 · Cited by 92
Pretesting effects in retrospective pretest-posttest designs.
Sprangers, M. A. G., & Hoogstraten, J. (1989) · Journal of Applied Psychology · doi:10.1037/0021-9010.74.2.265 · Cited by 88
Methods of Analysis with Response-Shift BIAS
Bray, J. H., Maxwell, S. E., & Howard, G. S. (1984) · Educational and Psychological Measurement · doi:10.1177/0013164484444002 · Cited by 79
Influence of Objective Measures on Self-Reports in a Retrospective Pretest-Posttest Design
Hoogstraten, J. (1985) · The Journal of Experimental Education · doi:10.1080/00220973.1985.10806383 · Cited by 66
Townsend, M. A. R., & Wilton, K. M. (2003) · British Journal of Educational Psychology · doi:10.1348/000709903322591190 · Cited by 55
Cantrell, P. (2003) · School Science and Mathematics · doi:10.1111/j.1949-8594.2003.tb18116.x · Cited by 52
Increased Accuracy in Measuring Leadership Impacts
Rohs, F. R., & Langone, C. A. (1997) · Journal of Leadership Studies · doi:10.1177/107179199700400113 · Cited by 48
Using Retrospective Pre-Ratings to Counteract Response-Shift Confounding
Pohl, N. F. (1982) · The Journal of Experimental Education · doi:10.1080/00220973.1982.11011826 · Cited by 47
The Retrospective Pretest and the Role of Pretest Information in Evaluative Studies
Goedhart, H., & Hoogstraten, J. (1992) · Psychological Reports · doi:10.2466/pr0.1992.70.3.699 · Cited by 46
Green, G. T., Kleiber, D. A., & Tarrant, M. A. (2000) · Journal of Park and Recreation Administration · Cited by 36
Lee, T. M. C., Paterson, J. G., & Chan, C. C. H. (1994) · American Journal of Occupational Therapy · doi:10.5014/ajot.48.7.633 · Cited by 29
The Retrospective Pretest: An Alternative Approach in Evaluating Drug Prevention Programs
Rhodes, J. E., & Jason, L. A. (1987) · Journal of Drug Education · doi:10.2190/1lnk-bh22-f201-8je3 · Cited by 26
Benefits-based programming: making an impact on youth.
Hurtes, K. P., Allen, L. R., Stevens, B. W., & Lee, C. (2000) · Journal of Park and Recreation Administration · Cited by 25
On Delay and Reassessment of Retrospective Preratings
Sprangers, M. A. G., & Hoogstraten, J. (1988) · The Journal of Experimental Education · doi:10.1080/00220973.1988.10806480 · Cited by 24
Response-Style Effects, Response-Shift Bias, and a Bogus-Pipeline: A Replication
Sprangers, M. A. G., & Hoogstraten, J. (1988) · Psychological Reports · doi:10.2466/pr0.1988.62.1.11 · Cited by 18
Implications of the response‐shift bias for management
Mann, S. (1997) · Journal of Management Development · doi:10.1108/02621719710174516 · Cited by 14
A method for analyzing retrospective pretest/posttest designs: I. Theory
Koele, P., & Hoogstraten, J. (1988) · Bulletin of the Psychonomic Society · doi:10.3758/bf03334859 · Cited by 14
The response-shift bias in a counsellor education programme
Manthei, R. J. (1997) · British Journal of Guidance and Counselling · doi:10.1080/03069889708253804 · Cited by 11
If Leisure Research is to Matter II
Witt, P. A. (2000) · Journal of Leisure Research · doi:10.1080/00222216.2000.11949912 · Cited by 11
Pre‐then‐post testing: A tool to improve the accuracy of management training program evaluation
Mezoff, B. (1981) · Performance & Instruction · doi:10.1002/pfi.4150200808 · Cited by 10
Leadership Development and Youth Camping: Determining a Relationship.
Toupence, R. H., & Townsend, C. D. (2000) · Cited by 8
Statistical and methodological issues in leisure research.
Christensen, J. L., & Barnett, L. (1988) · Cited by 2
Hopkins, K. D. (1986) · The Journal of Special Education · doi:10.1177/002246698602000105 · Cited by 2
Research works printed but not linked: 3
- Christensen J., 1995, Research about leisure: Past, present, and future (2nd ed.), 231
- Green G., 2000, Journal of Park and Recreation Administration, 18, 76
- Hurtes K. P., 2000, Journal of Park and Recreation Administration, 18, 34
Cited by
Sorted by citations
Duerden, M. D., & Witt, P. A. (2010) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2010.03.007 · Cited by 520
Validation of the interprofessional collaborative competency attainment survey (ICCAS)
Archibald, D., Trumpower, D. L., & Macdonald, C. J. (2014) · Journal of Interprofessional Care · doi:10.3109/13561820.2014.917407 · Cited by 289
Revisiting the utility of retrospective pre-post designs: The need for mixed-method pilot data
Geldhof, G. J., Warner, D. A., Finders, J. K., Thogmartin, A. A., Clark, A., & Longway, K. A. (2018) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2018.05.002 · Cited by 116
Creating Outcomes Through Experiential Education: The Challenge of Confounding Variables
Ewert, A., & Sibthorp, J. (2009) · Journal of Experiential Education · doi:10.1177/105382590803100305 · Cited by 110
Self-citationVitone, T., Stofer, K. A., Steininger, M. S., Hulcr, J., Dunn, R. R., & Lucky, A. (2016) · Journal of Science Communication · doi:10.22323/2.15010203 · Cited by 67
Metadata
- Authors
- Sibthorp, J., Paisley, K., Gookin, J., & Ward, P.
- Affiliations
- University of Utah (US)
- The National Outdoor Leadership School
- Brigham Young University (US)
- Corresponding author
- Sibthorp, Jim
- Venue
- Journal of Leisure Research
- Publisher
- Informa UK Limited
- Published
- 2007-06
- Volume
- 39
- Issue
- 2
- Pages
- 295-315
- ISSN
- 0022-2216 (ISSN-L)
- 2159-6417
- OpenAlex
- W1747140384
- Cited by
- 111
- References
- 41
- Citations by year
- 2026: 5 · 2025: 6 · 2024: 4 · 2023: 10 · 2022: 8 · 2021: 8 · 2020: 4 · 2019: 8 · 2018: 7 · 2017: 8 · 2016: 10 · 2015: 4 · 2014: 7 · 2013: 6 · 2012: 6
- FWCI
- 4.30
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Leisure Research (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Social Sciences › Business, Management and Accounting › Tourism, Leisure and Hospitality Management › Recreation, Leisure, Wilderness Management
- Keywords
- response-shift bias, self-report measures, pre-post test comparison, measurement scales
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review