The DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index.
Optimal recall periods for patient-reported outcomes: challenges and potential solutions
Abstract
OBJECTIVES: As the role and importance of patient-reported outcomes (PROs) increase, the validity and reliability of PRO measures come under greater scientific and regulatory scrutiny. One key issue is selecting the 'most appropriate' recall period for capturing PROs in clinical trials. This paper draws on survey research, health-specific literature, and results from clinical trials to summarize factors that can influence recall and provide guidance on selecting an optimal recall period. METHODS: We conducted a systematic review of six databases and additional literature drawn from bibliographies of the selected articles. RESULTS: Six major factors can influence recall; these can be classified into two broad areas: characteristics of the recalled phenomenon (recency, attributes, complexity) and context or meaning of the recalled phenomenon (salience, patient experience, mood). Results of different recall periods for three classes of PROs are presented: health behaviors, symptoms, and health-related quality of life. We present findings on the effect of alternative recall periods for three commonly used PROs. Finally, we propose a heuristic model to link the concept under investigation with an optimal recall period. CONCLUSIONS: No single recall period is best for all measures or all phenomena. The recall period must correspond to the characteristics of the phenomenon of interest and the purpose of the assessment. Recall period is an issue of internal validity. An incorrect recall period introduces measurement error that may reduce the chances of detecting a treatment effect. Researchers should consider recall period as seriously as they do other measurement properties.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Development and validation of brief measures of positive and negative affect: The PANAS scales.
Watson, D., Clark, L. A., & Tellegen, A. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.6.1063 · Cited by 32,433
The Psychology of Survey Response
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000) · doi:10.1017/cbo9780511819322 · Cited by 4,734
The PANAS-X: Manual for the Positive and Negative Affect Schedule - Expanded Form
Watson, D., & Clark, L. A. (1994) · University of Iowa · doi:10.17077/48vt-m4t2 · Cited by 3,700
Memory: A Contribution to Experimental Psychology
Ebbinghaus, H. (2013) · Annals of Neurosciences · doi:10.5214/ans.0972.7531.200408 · Cited by 2,827
How we should measure "change": Or should we?
Cronbach, L. J., & Furby, L. (1970) · Psychological Bulletin · doi:10.1037/h0029382 · Cited by 2,454
Integrating response shift into health-related quality of life research: a theoretical model
Sprangers, M. A. G., & Schwartz, C. E. (1999) · Social Science & Medicine · doi:10.1016/s0277-9536(99)00045-3 · Cited by 2,228
Blaney, P. H. (1986) · Psychological Bulletin · doi:10.1037//0033-2909.99.2.229 · Cited by 1,773
(1985) · Cited by 1,630
When More Pain Is Preferred to Less: Adding a Better End
Kahneman, D., Fredrickson, B. L., Schreiber, C. A., & Redelmeier, D. A. (1993) · Psychological Science · doi:10.1111/j.1467-9280.1993.tb00589.x · Cited by 1,502
Answering Autobiographical Questions: The Impact of Memory and Inference on Surveys
Bradburn, N. M., Rips, L. J., & Shevell, S. K. (1987) · Science · doi:10.1126/science.3563494 · Cited by 1,183
Improving Survey Questions: Design and Evaluation
Fowler, F. J. (1995) · Cited by 1,123
Redelmeier, D. A., & Kahneman, D. (1996) · Pain · doi:10.1016/0304-3959(96)02994-6 · Cited by 1,078
Measurement Error in Survey Data
Bound, J., Brown, C., & Mathiowetz, N. (2001) · Handbook of Econometrics · doi:10.1016/s1573-4412(01)05012-7 · Cited by 893
Schwartz, C. E., & Sprangers, M. A. G. (1999) · Social Science & Medicine · doi:10.1016/s0277-9536(99)00047-7 · Cited by 828
Patient non-compliance with paper diaries
Stone, A. A., Shiffman, S., Schwartz, J. E., Broderick, J. E., & Hufford, M. R. (2002) · BMJ · doi:10.1136/bmj.324.7347.1193 · Cited by 810
Survey Questions: Handcrafting the Standardized Questionnaire
Quarm, D., Converse, J. M., & Presser, S. (1988) · Teaching Sociology · doi:10.2307/1317544 · Cited by 766
Handbook of Research Methods. in Social and Personality Psychology
Lai, H. L. T. (2000) · Cited by 629
Converse, J., & Presser, S. (1986) · Sage eBooks · doi:10.4135/9781412986045 · Cited by 586
Response-shift bias: A source of contamination of self-report measures.
Howard, G. S., & Dailey, P. R. (1979) · Journal of Applied Psychology · doi:10.1037/0021-9010.64.2.144 · Cited by 565
Howard (1980) · doi:10.1177/0193841x8000400105 · Cited by 531
Memories of colonoscopy: a randomized trial
Redelmeier, D. A., Katz, J., & Kahneman, D. (2003) · Pain · doi:10.1016/s0304-3959(03)00003-4 · Cited by 460
Event-sampling and other methods for studying everyday experience.
Reis, H. T., & Gable, S. L. (2000) · Cited by 457
Simple pain rating scales hide complex idiosyncratic meanings
de Williams, A. C., Davies, H. T. O., & Chadury, Y. (2000) · Pain · doi:10.1016/s0304-3959(99)00299-7 · Cited by 452
Ecological momentary assessment research in behavioral medicine
Smyth, J. M., & Stone, A. A. (2003) · Journal of Happiness Studies · doi:10.1023/a:1023657221954 · Cited by 439
Memory accuracy in the recall of emotions.
Thomas, D. L., & Diener, E. (1990) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.59.2.291 · Cited by 424
Concordance of Self-reported Data and Medical Record Audit for Six Cancer Screening Procedures
Gordon, N. P., Hiatt, R. A., & Lampert, D. I. (1993) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/85.7.566 · Cited by 415
Remember that? A comparison of real-time versus retrospective recall of smoking lapses.
Shiffman, S., Hufford, M., Hickcox, M., & Paty, J. A. (1997) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.65.2.292.a · Cited by 379
Memory for pain: Relation between past and present pain intensity
Eich, E., Reeves, J. L., Jaeger, B., & Graff-Radford, S. B. (1985) · Pain · doi:10.1016/0304-3959(85)90007-7 · Cited by 321
The accuracy of pain and fatigue items across different reporting periods
Broderick, J. E., Schwartz, J. E., Vikingstad, G., Pribbernow, M., Grossman, S., & Stone, A. A. (2008) · Pain · doi:10.1016/j.pain.2008.03.024 · Cited by 298
Stone, A. A., Broderick, J. E., Schwartz, J. E., Shiffman, S., Litcher-Kelly, L., & Calvanese, P. (2003) · Pain · doi:10.1016/s0304-3959(03)00040-x · Cited by 293
Patient recall and recall bias of health state and health status
Schmier, J. K., & Halpern, M. T. (2004) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1586/14737167.4.2.159 · Cited by 269
A Study of Response Errors in Expenditures Data from Household Interviews
Neter, J., & Waksberg, J. (1964) · Journal of the American Statistical Association · doi:10.1080/01621459.1964.10480699 · Cited by 255
Stone, A. A., Broderick, J. E., Shiffman, S. S., & Schwartz, J. E. (2004) · Pain · doi:10.1016/j.pain.2003.09.020 · Cited by 252
Mammography use among sociodemographically diverse women: the accuracy of self-report.
Zapka, J. G., Bigelow, C., Hurley, T., Ford, L. D., Egelhofer, J., Cloud, W. M., & Sachsse, E. (1996) · American Journal of Public Health · doi:10.2105/ajph.86.7.1016 · Cited by 249
Confirmatory factor analyses and reliability of the modified cigarette evaluation questionnaire
Cappelleri, J. C., Bushmakin, A. G., Baker, C. L., Merikle, E., Olufade, A. O., & Gilbert, D. G. (2007) · Addictive Behaviors · doi:10.1016/j.addbeh.2006.06.028 · Cited by 241
Dorman, P., Slattery, J., Farrell, B., Dennis, M., & Sandercock, P. (1998) · Stroke · doi:10.1161/01.str.29.1.63 · Cited by 234
Stone, A. A., Schwartz, J. E., Broderick, J. E., & Shiffman, S. S. (2005) · Personality and Social Psychology Bulletin · doi:10.1177/0146167205275615 · Cited by 206
Optimal recall length in survey design
Clarke, P. M., Fiebig, D. G., & Gerdtham, U.-G. (2008) · Journal of Health Economics · doi:10.1016/j.jhealeco.2008.05.012 · Cited by 204
Reilly, M. C., Bracco, A., Ricci, J. ‐., Santoro, J., & Stevens, T. (2004) · Alimentary Pharmacology & Therapeutics · doi:10.1111/j.1365-2036.2004.02091.x · Cited by 196
American translation, modification, and validation of the St. George's Respiratory Questionnaire
Barr, J. T., Schumacher, G. E., Freeman, S., LeMoine, M., Bakst, A. W., & Jones, P. W. (2000) · Clinical Therapeutics · doi:10.1016/s0149-2918(00)80089-2 · Cited by 180
Peters, M. L., Sorbi, M. J., Kruise, D. A., Kerssens, J. J., Verhaak, P. F. M., & Bensing, J. M. (2000) · Pain · doi:10.1016/s0304-3959(99)00206-7 · Cited by 177
A summary of studies of interviewing methodology.: (429132004-001)
Cannell, C. F., Marquis, K. H., & Laurent, A. (1977) · PsycEXTRA Dataset · doi:10.1037/e429132004-001 · Cited by 169
Signaling does not adequately improve diary compliance
Broderick, J. E., Schwartz, J. E., Shiffman, S. S., Hufford, M. R., & Stone, A. A. (2003) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm2602_06 · Cited by 169
Anxiety and Anxiety Disorders: (555372011-001)
(2013) · PsycEXTRA Dataset · doi:10.1037/e555372011-001 · Cited by 164
Social memory in everyday life: Recall of self-events and other-events.
Skowronski, J. J., Betz, A. L., Thompson, C. P., & Shannon, L. (1991) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.60.6.831 · Cited by 163
Time Frames for Mood: Relations between Momentary and Generalized Ratings of Affect
Parkinson, B., Briner, R. B., Reynolds, S., & Totterdell, P. (1995) · Personality and Social Psychology Bulletin · doi:10.1177/0146167295214003 · Cited by 159
Litwin, M. S., & McGuigan, K. A. (1999) · Journal of Clinical Oncology · doi:10.1200/jco.1999.17.9.2882 · Cited by 158
Measuring clinical pain in chronic widespread pain: selected methodological issues
Gendreau, M., Hufford, M. R., & Stone, A. A. (2003) · Best Practice & Research Clinical Rheumatology · doi:10.1016/s1521-6942(03)00031-7 · Cited by 156
Femoral neck fractures in the elderly: Functional outcome and quality of life according to EuroQol
Tidermark, J., Zethraeus, N., Svensson, O., Törnkvist, H., & Ponzer, S. (2002) · Quality of Life Research · doi:10.1023/a:1015632114068 · Cited by 144
Cai, B., Elmer, M. E., Lindley, C. M., Pearson, J. D., Martin, A. R., & Horgan, K. J. (2003) · Supportive Care in Cancer · doi:10.1007/s00520-003-0482-4 · Cited by 139
The Accuracy of Self-Reported Disability Days
Revicki, D. A., IRWIN, D., Reblando, J. A., & SIMON, G. E. (1994) · Medical Care · doi:10.1097/00005650-199404000-00008 · Cited by 137
Patient Self-Reports in Pharmacoeconomic Studies: Their Use and Impact on Study Validity
Evans, C., & Crawford, B. (1999) · PharmacoEconomics · doi:10.2165/00019053-199915030-00004 · Cited by 136
Stone, A. A., Broderick, J. E., Kaell, A. T., DelesPaul, P. A. E. G., & Porter, L. E. (2000) · The Journal of Pain · doi:10.1054/jpai.2000.7568 · Cited by 133
Impact of nausea and vomiting on quality of life in cancer patients during chemotherapy.
Ballatori, E., & Roila, F. (2003) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-1-46 · Cited by 120
Response shift influenced estimates of change in health-related quality of life poststroke
Ahmed, S., Mayo, N. E., Wood-Dauphinée, S., Hanley, J. A., & Cohen, S. R. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2003.11.003 · Cited by 116
Comparison of responses to SF-36 Health Survey questions with one-week and four-week recall periods.
Keller, S., Bayliss, M. S., Ware, J. E., Hsu, M.-A., Damiano, A. M., & Goss, T. F. (1997) · Europe PMC (PubMed Central) · Cited by 113
Harrison, G. G., Galal, O., Stormer, A., Leslie, J., Ibrahim, N., Khorshid, A., & Saleh, N. T. (2000) · Journal of Nutrition · doi:10.1093/jn/130.8.2049 · Cited by 110
Lingard, E. A., Wright, E. A., & Sledge, C. B. (2001) · Journal of Bone and Joint Surgery · doi:10.2106/00004623-200108000-00003 · Cited by 101
Recency affects reporting accuracy of children's dietary recalls
Baxter, S. D., Smith, A. F., Litaker, M. S., Guinn, C. H., Shaffer, N. M., Baglio, M. L., & Frye, F. H. A. (2004) · Annals of Epidemiology · doi:10.1016/j.annepidem.2003.07.003 · Cited by 100
REPORTING OF HOSPITALIZATION IN THE HEALTH INTERVIEW SURVEY.
Biering-Sørensen, U. (1965) · PubMed · Cited by 97
Best practice in symptom assessment: a review
McColl, E. (2004) · Gut · doi:10.1136/gut.2003.034355 · Cited by 94
Low Back Pain Recollection Versus Concurrent Accounts: Outcomes Analysis
Dawson, E. G., Kanim, L. E. A., Sra, P., Dorey, F. J., Goldstein, T. B., Delamarter, R. B., & Sandhu, H. S. (2002) · Spine · doi:10.1097/00007632-200205010-00020 · Cited by 73
Interview Data on Chronic Conditions Compared with Information Derived from Medical Records.
Madow, W. G. (1967) · Annals of Internal Medicine · doi:10.7326/0003-4819-67-5-1128_2 · Cited by 65
On the Accuracy of Retrospective Attitudinal Data
Gutek, B. A. (1978) · Public Opinion Quarterly · doi:10.1086/268462 · Cited by 63
Williams, D. A., Gendreau, M., Hufford, M. R., Groner, K. H., Gracely, R. H., & Clauw, D. J. (2004) · Clinical Journal of Pain · doi:10.1097/00002508-200409000-00010 · Cited by 59
Raselli, C., & Broderick, J. E. (2007) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2006.10.001 · Cited by 57
Contributions of Cognitive Psychology to Advertising Research
Sudman, S., & Schwarz, N. (1989) · Journal of Advertising Research · doi:10.1080/00218499.1989.12467703 · Cited by 50
Optimum recall period for reporting persons injured in motor vehicle accidents.
Cash, W., & Moss, A. J. (1972) · PubMed · Cited by 49
Howell, J. R., Xu, M., Duncan, C. P., Masri, B. A., & Garbuz, D. S. (2008) · The Journal of Arthroplasty · doi:10.1016/j.arth.2007.07.020 · Cited by 45
MEMORY FOR CHRONIC PAIN INTENSITY: CORRELATES OF ACCURACY
Linton, S. J. (1991) · Perceptual and Motor Skills · doi:10.2466/pms.1991.72.3c.1091 · Cited by 44
Amjadi, S., Amjadi, S., Khanna, D., Park, G. S., Bulpitt, K. J., Wong, W. K., & Paulus, H. E. (2004) · PubMed · Cited by 41
Perneger, T., Etter, J., & Rougemont, A. C. (1997) · Journal of Epidemiology & Community Health · doi:10.1136/jech.51.3.320 · Cited by 41
Sohl, S. J., & Friedberg, F. (2008) · Behavioral Medicine · doi:10.3200/bmed.34.1.29-38 · Cited by 37
Friedberg, F., & Sohl, S. J. (2008) · International Journal of Behavioral Medicine · doi:10.1007/bf03003071 · Cited by 30
A blueprint for symptom scales and responses: measurement and reporting
Wyrwich, K. W., & Tardino, V. M. S. (2004) · Gut · doi:10.1136/gut.2003.034348 · Cited by 24
Booth, M. I., Stratford, J. M., & Dehn, T. C. B. (2001) · Scandinavian Journal of Gastroenterology · doi:10.1080/003655201750313298 · Cited by 20
Cited by
Basch, E., Reeve, B. B., Mitchell, S. A., Clauser, S. B., Minasian, L. M., Dueck, A. C., Mendoza, T. R., Hay, J., Atkinson, T. M., Abernethy, A. P., et al. (2014) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/dju244 · Cited by 1,151
Trends in Ambulatory Self-Report: The Role of Momentary Experience in Psychosomatic Medicine
Conner, T. S., & Barrett, L. F. (2012) · Psychosomatic Medicine · doi:10.1097/psy.0b013e3182546f18 · Cited by 363
Electronic patient‐reported outcome systems in oncology clinical practice
Bennett, A. V., Jensen, R. E., & Basch, E. (2012) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.21150 · Cited by 353
Ecological momentary assessment methodology in chronic pain research: a systematic review
May, M., Junghaenel, D. U., Ono, M., Stone, A. A., & Schneider, S. (2018) · The Journal of Pain · doi:10.1016/j.jpain.2018.01.006 · Cited by 247
Developing a Valid Patient-Reported Outcome Measure
Rothrock, N. E., Kaiser, K. A., & Cella, D. (2011) · Clinical Pharmacology & Therapeutics · doi:10.1038/clpt.2011.195 · Cited by 222
Metadata
- Authors
- Stull, D. E., Leidy, N. K., Parasuraman, B., & Chassany, O.
- Venue
- Current Medical Research and Opinion
- Publisher
- Informa Healthcare
- Published
- 2009-02-27
- Volume
- 25
- Issue
- 4
- Pages
- 929-942
- ISSN
- 0300-7995 (ISSN-L)
- 1473-4877
- OpenAlex
- W2171228644
- Cited by
- 404
- References
- 76
- FWCI
- 29.31
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.