Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: A meta-analytic review
Open accessAbstract
Objectives Patient-reported outcomes (PROs; self-report assessments) are increasingly important in evaluating medical care and treatment efficacy. Electronic administration of PROs via computer is becoming widespread. This article reviews the literature addressing whether computer-administered tests are equivalent to their paper-and-pencil forms.Methods Meta-analysis was used to synthesize 65 studies that directly assessed the equivalence of computer versus paper versions of PROs used in clinical trials. A total of 46 unique studies, evaluating 278 scales, provided sufficient detail to allow quantitative analysis.Results Among 233 direct comparisons, the average mean difference between modes averaged 0.2% of the scale range (e.g., 0.02 points on a 10-point scale), and 93% were within +/-5% of the scale range. Among 207 correlation coefficients between paper and computer instruments (typically intraclass correlation coefficients), the average weighted correlation was 0.90; 94% of correlations were at least 0.75. Because the cross-mode correlation (paper vs. computer) is also a test-retest correlation, with potential variation because of retest, we compared it to the within-mode (paper vs. paper) test-retest correlation. In four comparisons that evaluated both, the average cross-mode paper-to-computer correlation was almost identical to the within-mode correlation for readministration of a paper measure (0.88 vs. 0.91).Conclusions Extensive evidence indicates that paper- and computer-administered PROs are equivalent.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Previous studies highlight no differences between the use of paper-pencil or online formats.
“Previous studies highlight the no differences between the use of paper–pencil or online formats (e.g., Dodou & de Winter, 2014; Gwaltney et al., 2008).”
From Online Sexual Harassment Perpetration Among Peer Adolescents: A Cross-National and Cross-Gender Study (2025) · cites this paper as Gwaltney et al., 2008
The literature confirms that the use of online or paper questionnaires is largely indifferent.
“the literature confirms that the use of online or paper questionnaires is largely indifferent.”
From Online Sexual Harassment Perpetration Among Peer Adolescents: A Cross-National and Cross-Gender Study (2025) · cites this paper as Gwaltney et al., 2008
Cites
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 23,433 · see citing works
Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646 · see citing works
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,347 · see citing works
Statistical Methods for Rates and Proportions.
Everitt, B. S., & Fleiss, J. L. (1981) · Biometrics · doi:10.2307/2530193 · Cited by 9,034 · see citing works
Forming inferences about some intraclass correlation coefficients.
McGraw, K. O., & Wong, S. P. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.1.30 · Cited by 6,903 · see citing works
Jaeschke, R., Singer, J., & Guyatt, G. H. (1989) · Controlled Clinical Trials · doi:10.1016/0197-2456(89)90005-6 · Cited by 4,982 · see citing works
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,832 · see citing works
Models for Longitudinal Data: A Generalized Estimating Equation Approach
Zeger, S. L., Liang, K., & Albert, P. S. (1988) · Biometrics · doi:10.2307/2531734 · Cited by 4,505 · see citing works
Statistical Methods for Rates and Proportions, 2nd ed
GROVE, W. A. (1981) · American Journal of Psychiatry · doi:10.1176/ajp.138.12.1644-a · Cited by 3,756 · see citing works
Fleiss, J. L., & Cohen, J. (1973) · Educational and Psychological Measurement · doi:10.1177/001316447303300309 · Cited by 3,354 · see citing works
Wilson, I. B. (1995) · JAMA · doi:10.1001/jama.1995.03520250075037 · Cited by 3,027 · see citing works
(1981) · PubMed · Cited by 2,253 · see citing works
A Fail-Safe N for Effect Size in Meta-Analysis
Orwin, R. G. (1983) · Journal of Educational Statistics · doi:10.2307/1164923 · Cited by 2,064 · see citing works
Cognitive Interviewing: A Tool for Improving Questionnaire Design
Willis, G. (2004) · Cited by 1,934 · see citing works
Ecological Momentary Assessment (Ema) in Behavioral Medicine
Stone, A. A., & Shiffman, S. (1994) · Annals of Behavioral Medicine · doi:10.1093/abm/16.3.199 · Cited by 1,931 · see citing works
Self-citationPatient non-compliance with paper diaries
Stone, A. A., Shiffman, S., Schwartz, J. E., Broderick, J. E., & Hufford, M. R. (2002) · BMJ · doi:10.1136/bmj.324.7347.1193 · Cited by 810 · see citing works
Self-citationCognitive Interviewing. A Tool for Improving Questionnaire Design
McColl, E. (2006) · Quality of Life Research · doi:10.1007/s11136-005-5263-8 · Cited by 551 · see citing works
Comprehensive handbook of psychological assessment
(2004) · Choice Reviews Online · doi:10.5860/choice.42-1239 · Cited by 482 · see citing works
Rosenthal, R. (1991) · Psychosomatic Medicine · doi:10.1097/00006842-199105000-00001 · Cited by 380 · see citing works
Palermo, T. M., Valenzuela, D., & Stork, P. P. (2003) · Pain · doi:10.1016/j.pain.2003.10.005 · Cited by 346 · see citing works
Velikova, G., Wright, E. P., Smith, A. B., Cull, A., Gould, A., Forman, D., Perren, T., Stead, M., Brown, J., & Selby, P. J. (1999) · Journal of Clinical Oncology · doi:10.1200/jco.1999.17.3.998 · Cited by 339 · see citing works
Willke, R. J., Burke, L. B., & Erickson, P. (2004) · Controlled Clinical Trials · doi:10.1016/j.cct.2004.09.003 · Cited by 304 · see citing works
Electronic diaries for monitoring chronic pain: 1-year validation study
Jamison, R. N., Raymond, S. A., Levine, J. G., Slawsby, E. A., Nedeljković, S. S., & Katz, N. P. (2001) · Pain · doi:10.1016/s0304-3959(00)00450-4 · Cited by 249 · see citing works
A Computer-Administered Telephone Interview to Identify Mental Disorders
Kobak, K. A. (1997) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1997.03550110043034 · Cited by 238 · see citing works
Jamison, R. N., Gracely, R. H., Raymond, S. A., Levine, J. G., Marino, B., Herrmann, T. J., Daly, M., Fram, D. H., & Katz, N. P. (2002) · Pain · doi:10.1016/s0304-3959(02)00178-1 · Cited by 209 · see citing works
Wiklund, I. E. (2004) · Fundamental and Clinical Pharmacology · doi:10.1111/j.1472-8206.2004.00234.x · Cited by 205 · see citing works
Computer-Based Interview for Screening Blood Donors for Risk of HIV Transmission
Locke, S. E. (1992) · JAMA · doi:10.1001/jama.1992.03490100099034 · Cited by 203 · see citing works
Test-Retest Reliability of the Short-Form McGill Pain Questionnaire
Grafton, K., Foster, N. E., & Wright, C. C. (2004) · Clinical Journal of Pain · doi:10.1097/00002508-200501000-00009 · Cited by 198 · see citing works
Guidelines for computer-based tests and interpretations
(1986) · PsycEXTRA Dataset · doi:10.1037/e502052009-001 · Cited by 190 · see citing works
Bellamy, N., Campbell, J., Stevens, J., Pilch, L., Stewart, C., & Mahmood, Z. (1997) · PubMed · Cited by 163 · see citing works
Bliven, B. D., Kaufman, S. E., & Spertus, J. A. (2001) · Quality of Life Research · doi:10.1023/a:1016740312904 · Cited by 146 · see citing works
Theiler, R., Spielberger, J., Bischoff, H. A., Bellamy, N., Huber, J. F., & Kroesen, S. (2002) · Osteoarthritis and Cartilage · doi:10.1053/joca.2002.0807 · Cited by 130 · see citing works
Drummond, H. E., Ghosh, S., Ferguson, A., Brackenridge, D., & Tiplady, B. (1995) · Quality of Life Research · doi:10.1007/bf00434379 · Cited by 119 · see citing works
Stratton, R. J., Stubbs, R. J., Hughes, D., King, N. A., Blundell, J. E., & Elia, M. (1998) · European Journal of Clinical Nutrition · doi:10.1038/sj.ejcn.1600636 · Cited by 119 · see citing works
Structure of the Computer-Assisted Beck Anxiety Inventory With Psychiatric Inpatients
Steer, R. A., Rissmiller, D. J., Ranieri, W. F., & Beck, A. T. (1993) · Journal of Personality Assessment · doi:10.1207/s15327752jpa6003_10 · Cited by 114 · see citing works
Ryan, J., Corry, J. R., Attewell, R. G., & Smithson, M. J. (2002) · Quality of Life Research · doi:10.1023/a:1014415709997 · Cited by 113 · see citing works
Electronic pain diary: a randomized crossover study
Gaertner, J., Elsner, F., Pollmann-Dahmen, K., Radbruch, L., & Sabatowski, R. (2004) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2003.12.017 · Cited by 113 · see citing works
Fortner, B., Okon, T. A., Schwartzberg, L. S., Tauer, K. W., & Houts, A. C. (2003) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2003.04.003 · Cited by 104 · see citing works
(1997) · PubMed · Cited by 100 · see citing works
Pouwer, F., Snoek, F. J., van der Ploeg, H. M., Heine, R. J., & Brand, A. N. (1997) · Quality of Life Research · doi:10.1023/a:1008832821181 · Cited by 100 · see citing works
Comparing web and mail responses in a mixed mode survey in college alcohol use research
McCabe, S. E., Diez, A., Boyd, C. J., Nelson, T. F., & Weitzman, E. R. (2006) · Addictive Behaviors · doi:10.1016/j.addbeh.2005.12.009 · Cited by 95 · see citing works
Performance of health status measures with a pen based personal digital assistant
Kvien, T. K., Mowinckel, P., Heiberg, T., Dammann, K. L., Dale, Ø., Aanerud, G. J., Alme, T. N., & Uhlig, T. (2005) · Annals of the Rheumatic Diseases · doi:10.1136/ard.2004.030437 · Cited by 92 · see citing works
Palmblad, M., & Tiplady, B. (2004) · Quality of Life Research · doi:10.1023/b:qure.0000037501.92374.e1 · Cited by 90 · see citing works
Comparing computerized versus traditional psychological assessment
Lukin, M. E., Dowd, E. T., Plake, B. S., & Kraft, R. G. (1985) · Computers in Human Behavior · doi:10.1016/0747-5632(85)90006-8 · Cited by 87 · see citing works
Electronic Versus Paper Questionnaires: A Further Comparison in Persons with Asthma
Bushnell, D. M., Martin, M. L., & Parasuraman, B. M. (2003) · Journal of Asthma · doi:10.1081/jas-120023501 · Cited by 86 · see citing works
Cook, A. J., Roberts, D. A., Henderson, M. D., Winkle, L. C. V., Chastain, D. C., & Hamill-Ruth, R. J. (2004) · Pain · doi:10.1016/j.pain.2004.04.012 · Cited by 85 · see citing works
Bushnell, D. M., Reilly, M. C., Galani, C., Martin, M. L., Ricci, J., Patrick, D. L., & McBurney, C. R. (2006) · Value in Health · doi:10.1111/j.1524-4733.2006.00087.x · Cited by 85 · see citing works
Bischoff-Ferrari, H. A., Vondechend, M., Bellamy, N., & Theiler, R. (2004) · Annals of the Rheumatic Diseases · doi:10.1136/ard.2003.019307 · Cited by 84 · see citing works
Greenwood, M., Hakim, A. J., Carson, E. R., & Doyle, D. (2005) · Lara D. Veeken · doi:10.1093/rheumatology/kei100 · Cited by 77 · see citing works
Tseng, H., Tiplady, B., MACLEOD, H. A., & Wright, P. (1998) · British Journal of Psychology · doi:10.1111/j.2044-8295.1998.tb02705.x · Cited by 76 · see citing works
Kleinman, L., Leidy, N. K., Crawley, J. A., Bonomi, A. E., & Schoenfeld, P. (2001) · Medical Care · doi:10.1097/00005650-200102000-00008 · Cited by 75 · see citing works
Jaime, J., Caro, I., Caro, J., Wouters, F., & Juniper, E. F. (2001) · Quality of Life Research · doi:10.1023/a:1013811109820 · Cited by 73 · see citing works
Wilson, A. S., Kitas, G. D., Carruthers, D. M., Reay, C. A., Skan, J., Harris, S. K., Treharne, G. J., Young, S. P., & Bacon, P. A. (2002) · Lara D. Veeken · doi:10.1093/rheumatology/41.3.268 · Cited by 64 · see citing works
Comparison of commonly used orthopaedic outcome measures using palm‐top computers and paper surveys
Saleh, K. J., Radosevich, D. M., Kassim, R. A., Moussa, M., Dykes, D., Bottolfson, H., Gioe, T. J., & Robinson, H. (2002) · Journal of Orthopaedic Research® · doi:10.1016/s0736-0266(02)00059-1 · Cited by 63 · see citing works
Validity and sensitivity of a pen computer battery of performance tests
Cameron, E., Sinclair, W. A., & Tiplady, B. (2001) · Journal of Psychopharmacology · doi:10.1177/026988110101500207 · Cited by 60 · see citing works
The effects of computerized versus paper-and-pencil administration on measures of negative affect
George, C. E., Lankford, J. S., & Wilson, S. E. (1992) · Computers in Human Behavior · doi:10.1016/0747-5632(92)90004-x · Cited by 55 · see citing works
A web-compatible instrument for measuring self-reported disease activity in arthritis.
Athale, N., Sturley, A., Skoczen, S., Kavanaugh, A., & Lenert, L. A. (2004) · PubMed · Cited by 50 · see citing works
Schulenberg, S. E., & Yutrzenka, B. A. (1999) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03207726 · Cited by 48 · see citing works
Glaze, R., & Cox, J. (1991) · Journal of Affective Disorders · doi:10.1016/0165-0327(91)90086-8 · Cited by 40 · see citing works
Computerized Depression Screening and Awareness
Ogles, B. M., France, C. R., Lunnen, K. M., Bell, T., & Goldfarb, M. (1998) · Community Mental Health Journal · doi:10.1023/a:1018760128239 · Cited by 37 · see citing works
Stubbs, R. J., Hughes, D. A., Johnstone, A. M., Rowley, E., Ferris, S., Elia, M., Stratton, R. J., King, N. A., & Blundell, J. E. (2001) · Physiology & Behavior · doi:10.1016/s0031-9384(00)00440-6 · Cited by 34 · see citing works
Schmitz, N., Hartkamp, N., Brinschwitz, C., Michalek, S., & Tress, W. (2000) · Acta Psychiatrica Scandinavica · doi:10.1034/j.1600-0447.2000.102002147.x · Cited by 34 · see citing works
Schaeren, S., Bischoff-Ferrari, H. A., Knupp, M., Dick, W., Huber, J. F., & Theiler, R. (2005) · Journal of Bone and Joint Surgery - British Volume · doi:10.1302/0301-620x.87b2.15548 · Cited by 31 · see citing works
Test-retest reliability in psychiatric patients of the sf-36 health survey
Burke, J. D., Burke, K. C., Baker, J. H., & Hillis, A. (1996) · International Journal of Methods in Psychiatric Research · Cited by 29 · see citing works
Ford, B. D., Vitelli, R., & Stuckless, N. (1996) · Computers in Human Behavior · doi:10.1016/0747-5632(95)00026-7 · Cited by 26 · see citing works
Theiler, R., Bischoff-Ferrari, H. A., Good, M., & Bellamy, N. (2004) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2004.08.006 · Cited by 24 · see citing works
Schmitz, N., Hartkamp, N., Brinschwitz, C., & Michalek, S. (1999) · Psychiatry Research · doi:10.1016/s0165-1781(99)00060-8 · Cited by 22 · see citing works
Computer-assisted assessment of depression and function in older primary care patients
Kurt, R., Bogner, H. R., Straton, J. B., Tien, A. Y., & Gallo, J. J. (2003) · Computer Methods and Programs in Biomedicine · doi:10.1016/s0169-2607(03)00026-9 · Cited by 21 · see citing works
Effect of an Educational Multimedia Prostate Program on the International Prostate Symptom Score
van Schaik, P., Ahmed, T., Suvakovic, N., & Hindmarsh, J. R. (1999) · European Urology · doi:10.1159/000019924 · Cited by 13 · see citing works
New technology in quality of life research: Are all computer-assisted approaches created equal?
Litaker, D. G. (2003) · Quality of Life Research · doi:10.1023/a:1023457927406 · Cited by 11 · see citing works
Bent, H., Ratzlaff, C., Goligher, E. C., Kopec, J. A., & Gillies, J. H. (2005) · PubMed · Cited by 11 · see citing works
Mosley‐Williams, A., & Williams, C. (2004) · Arthritis & Rheumatism · doi:10.1002/art.11497 · Cited by 10 · see citing works
The value of computer-administered self-report data in central nervous system clinical trials.
Byrom, B., & Mundt, J. C. (2005) · PubMed · Cited by 8 · see citing works
Izquierdo‐Porrera, A. M., Manchanda, R., Powell, C. C., Sorkin, J. D., & Bradham, D. D. (2002) · Journal of the American Geriatrics Society · doi:10.1046/j.1532-5415.2002.50365.x · Cited by 4 · see citing works
Flagged first, then by citations
2 of the 76 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Ecological momentary assessment
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.3.022806.091415 · Cited by 6,454 · see citing works
Self-citationReddel, H. K., Taylor, D. R., Bateman, E. D., Boulet, L., Boushey, H. A., Busse, W. W., Casale, T. B., Chanez, P., Enright, P., Gibson, P. G., et al. (2009) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.200801-060st · Cited by 1,918 · see citing works
Registries for Evaluating Patient Outcomes: A User’s Guide
(2020) · doi:10.23970/ahrqepcregistries4 · Cited by 968 · see citing works
Experience sampling research in psychopathology: opening the black box of daily life
Myin-Germeys, I., Oorschot, M., Collip, D., Lataster, J., Delespaul, P., & van Os, J. (2009) · Psychological Medicine · doi:10.1017/s0033291708004947 · Cited by 817 · see citing works
Coons, S. J., Gwaltney, C. J., Hays, R. D., Lundy, J. J., Sloan, J. A., Revicki, D. A., Lenderking, W. R., Cella, D. F., & Basch, E. (2008) · Value in Health · doi:10.1111/j.1524-4733.2008.00470.x · Cited by 533 · see citing works
Self-citationState Mindfulness Scale (SMS): Development and initial validation.
Tanay, G., & Bernstein, A. (2013) · Psychological Assessment · doi:10.1037/a0034044 · Cited by 486 · see citing works
Basch, E., Abernethy, A. P., Mullins, C. D., Reeve, B. B., Smith, M. L., Coons, S. J., Sloan, J., Wenzel, K., Chauhan, C., Eppard, W., et al. (2012) · Journal of Clinical Oncology · doi:10.1200/jco.2012.42.5967 · Cited by 476 · see citing works
Anker, S. D., Agewall, S., Borggrefe, M., Calvert, M., Jaime Caro, J., Cowie, M. R., Ford, I., Paty, J. A., Riley, J. P., Swedberg, K., et al. (2014) · European Heart Journal · doi:10.1093/eurheartj/ehu205 · Cited by 362 · see citing works
Electronic patient‐reported outcome systems in oncology clinical practice
Bennett, A. V., Jensen, R. E., & Basch, E. (2012) · CA A Cancer Journal for Clinicians · doi:10.3322/caac.21150 · Cited by 356 · see citing works
Benefits and Disadvantages of Electronic Patient-reported Outcome Measures: Systematic Review
Meirte, J., Hellemans, N., Anthonissen, M., Denteneer, L., Maertens, K., Moortgat, P., & Daele, U. V. (2020) · JMIR Perioperative Medicine · doi:10.2196/15588 · Cited by 309 · see citing works
Patient-Reported Outcomes In Performance Measurement
Cella, D. F., Hahn, E. A., Jensen, S. E., Butt, Z. A., Nowinski, C. J., Rothrock, N. E., & Lohr, K. N. (2015) · doi:10.3768/rtipress.2015.bk.0014.1509 · Cited by 326 · see citing works
Ecological Momentary Assessment
Kirchner, T., & Shiffman, S. S. (2013) · Wiley eBooks · doi:10.1002/9781118384404.ch20 · Cited by 286 · see citing works
Self-citationMoore, R. C., Depp, C. A., Wetherell, J. L., & Lenze, E. J. (2016) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2016.01.011 · Cited by 274 · see citing works
Muehlhausen, W., Doll, H., Quadri, N., Fordham, B., O'Donohoe, P., Dogar, N., & Wild, D. (2015) · Health and Quality of Life Outcomes · doi:10.1186/s12955-015-0362-x · Cited by 259 · see citing works
Patient-reported outcomes in cancer care — hearing the patient voice at greater volume
LeBlanc, T. W., & Abernethy, A. P. (2017) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2017.153 · Cited by 246 · see citing works
Measuring patients' experiences and outcomes
Black, N. A., & Jenkinson, C. (2009) · BMJ · doi:10.1136/bmj.b2495 · Cited by 240 · see citing works
Etkind, S. N., Daveson, B. A., Kwok, W., Witt, J., Bausewein, C., Higginson, I. J., & Murtagh, F. E. (2014) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2014.07.010 · Cited by 239 · see citing works
Atkinson, T. M., Mendoza, T. R., Sit, L., Passik, S., Scher, H. I., Cleeland, C., & Basch, E. (2010) · Pain Medicine · doi:10.1111/j.1526-4637.2009.00774.x · Cited by 224 · see citing works
Reliability, validity and sensitivity of a computerized visual analog scale measuring state anxiety
Abend, R., Dan, O., Maoz, K., Raz, S., & Bar-Haim, Y. (2014) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2014.06.004 · Cited by 217 · see citing works
Belisario, J. S. M., Jamšek, J., Huckvale, K., O’Donoghue, J., Morrison, C. P., & Car, J. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000042.pub2 · Cited by 216 · see citing works
Coons, S. J., Eremenco, S. L., Lundy, J. J., O’Donohoe, P., O’Gorman, H., & Malizia, W. (2014) · Patient · doi:10.1007/s40271-014-0090-z · Cited by 213 · see citing works
Eskildsen, A., Dalgaard, V. L., Nielsen, K. J., Andersen, J. H., Zachariae, R., Olsen, L. R., Jørgensen, A., & Christiansen, D. H. (2015) · Scandinavian Journal of Work, Environment & Health · doi:10.5271/sjweh.3510 · Cited by 195 · see citing works
Holzner, B., Giesinger, J. M., Pinggera, J., Zugal, S., Schöpf, F., Oberguggenberger, A., Gamper, E., Zabernigg, A. F., Weber, B., & Rumpold, G. (2012) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-12-126 · Cited by 188 · see citing works
Gibbons, C., Hopkins, M., Beaulieu, K., Oustric, P., & Blundell, J. E. (2019) · Current Obesity Reports · doi:10.1007/s13679-019-00340-6 · Cited by 172 · see citing works
Rose, M., & Bezjak, A. (2009) · Quality of Life Research · doi:10.1007/s11136-008-9436-0 · Cited by 154 · see citing works
Chang, E. M., Gillespie, E. F., & Shaverdian, N. (2019) · Patient Related Outcome Measures · doi:10.2147/prom.s178344 · Cited by 155 · see citing works
Psychometric evaluation of self-report outcome measures for prosthetic applications
Hafner, B. J., Morgan, S. J., CPO, Askew, R. L., & Salem, R. (2016) · The Journal of Rehabilitation Research and Development · doi:10.1682/jrrd.2015.12.0228 · Cited by 143 · see citing works
Abernethy, A. P., Herndon, J. E., Wheeler, J. L., Patwardhan, M. B., Shaw, H., Lyerly, H. K., & Weinfurt, K. P. (2008) · Health Services Research · doi:10.1111/j.1475-6773.2008.00887.x · Cited by 141 · see citing works
Orwelius, L., Nilsson, M., Nilsson, E., Wenemark, M., Walfridsson, U., Lundström, M., Taft, C., Palaszewski, B., & Kristenson, M. (2018) · Journal of Patient-Reported Outcomes · doi:10.1186/s41687-018-0030-0 · Cited by 140 · see citing works
Equivalence of electronic and paper-based patient-reported outcome measures
Campbell, N., Ali, F. M., Finlay, A. Y., & Salek, S. (2015) · Quality of Life Research · doi:10.1007/s11136-015-0937-3 · Cited by 135 · see citing works
Bennett, A. V., Dueck, A. C., Mitchell, S. A., Mendoza, T. R., Reeve, B. B., Atkinson, T. M., Castro, K. M., Denicoff, A. M., Rogak, L. J., Harness, J. K., et al. (2016) · Health and Quality of Life Outcomes · doi:10.1186/s12955-016-0426-6 · Cited by 122 · see citing works
Bjørner, J. B., Rose, M., Gandek, B., Stone, A. A., Junghaenel, D. U., & Ware, J. E. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.07.016 · Cited by 116 · see citing works
Mode of administration does not cause bias in patient-reported outcome results: a meta-analysis
Rutherford, C., Costa, D. S. J., Mercieca‐Bebber, R., Rice, H. E., Gabb, L., & King, M. T. (2015) · Quality of Life Research · doi:10.1007/s11136-015-1110-8 · Cited by 115 · see citing works
The use of EORTC measures in daily clinical practice—A synopsis of a newly developed manual
Wintner, L. M., Sztankay, M. J., Aaronson, N. K., Bottomley, A., Giesinger, J. M., Grøenvold, M., Petersen, M. A., van de Poll-Franse, L. V., Velikova, G., Leeuw, I. M. V., et al. (2016) · European Journal of Cancer · doi:10.1016/j.ejca.2016.08.024 · Cited by 115 · see citing works
Developing Adolescent-Specific Health State Values for Economic Evaluation
Ratcliffe, J., Flynn, T. N., Terlich, F., Stevens, K. J., Brazier, J. E., & SAWYER, M. G. (2012) · PharmacoEconomics · doi:10.2165/11597900-000000000-00000 · Cited by 112 · see citing works
An assessment of the construct validity of the CHU9D in the Australian adolescent general population
Ratcliffe, J., Stevens, K. J., Flynn, T. N., Brazier, J. E., & SAWYER, M. G. (2011) · Quality of Life Research · doi:10.1007/s11136-011-9971-y · Cited by 108 · see citing works
Smartphone Versus Pen-and-Paper Data Collection of Infant Feeding Practices in Rural China
Zhang, S., Wu, Q., van Velthoven, M. H., Chen, L., Car, J., Rudan, I., Zhang, Y., Li, Y., & Scherpbier, R. W. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2183 · Cited by 104 · see citing works
Hambly, K., & Griva, K. (2010) · The American Journal of Sports Medicine · doi:10.1177/0363546509359678 · Cited by 102 · see citing works
Basch, E., Rogak, L. J., & Dueck, A. C. (2016) · Clinical Therapeutics · doi:10.1016/j.clinthera.2016.03.011 · Cited by 101 · see citing works
Kivi, M., Eriksson, M. C. M., Hange, D., Petersson, E.-L., Vernmark, K., Johansson, B., & Björkelund, C. (2014) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2014.921834 · Cited by 92 · see citing works
Brod, M. I., Hammer, M., Christensen, T., Alolga, S. L., & Bushnell, D. M. (2009) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-7-83 · Cited by 97 · see citing works
Patient-Reported Outcomes in Performance Measurement
Cella, D. F., Hahn, E. A., Jensen, S. E., Butt, Z. A., Nowinski, C. J., Rothrock, N. E., & Lohr, K. N. (2015) · Cited by 96 · see citing works
Abernethy, A. P., Zafar, S. Y., Uronis, H. E., Wheeler, J. L., Sykes, A. D., Rowe, K., Shelby, R. A., Fowler, R., & Herndon, J. E. (2010) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2010.01.017 · Cited by 96 · see citing works
Graves, T. D., Cha, Y., Hahn, A. F., Barohn, R. J., Salajegheh, M. K., Griggs, R. C., Bundy, B. N., Jen, J. C., Baloh, R. W., & Hanna, M. G. (2014) · Brain · doi:10.1093/brain/awu012 · Cited by 95 · see citing works
Papuga, M. O., Dasilva, C., McIntyre, A. W., Mitten, D. J., Kates, S. L., & Baumhauer, J. F. (2017) · Health Systems · doi:10.1057/s41306-016-0016-1 · Cited by 95 · see citing works
Benze, G., Nauck, F., Alt‐Epping, B., Gianni, G., Bauknecht, T., Ettl, J., Munte, A., Kretzschmar, L., & Gaertner, J. (2017) · Annals of Palliative Medicine · doi:10.21037/apm.2017.07.05 · Cited by 94 · see citing works
A Systematic Review of Momentary Assessment Designs for Mood and Anxiety Symptoms
Hall, M., Scherner, P. V., Kreidel, Y., & Rubel, J. A. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.642044 · Cited by 100 · see citing works
Interformat Reliability of Digital Psychiatric Self-Report Questionnaires: A Systematic Review
Alfonsson, S., Maathz, P., & Hursti, T. J. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3395 · Cited by 94 · see citing works
Norman, R., King, M. T., Clarke, D., Viney, R. C., Cronin, P., & Street, D. J. (2010) · Quality of Life Research · doi:10.1007/s11136-010-9609-5 · Cited by 92 · see citing works
Garcia‐Palacios, A., Herrero, R., Belmonte, M. A., Castilla, D., Guixeres, J., Molinari, G., Baños, R. M., & Botella, C. (2014) · European Journal of Pain · doi:10.1002/j.1532-2149.2013.00425.x · Cited by 90 · see citing works
Maier, A., Holm, T., Wicks, P. J., Steinfurth, L., Linke, P., Münch, C., Meyer, R., & Meyer, T. (2012) · Amyotrophic Lateral Sclerosis · doi:10.3109/17482968.2011.633268 · Cited by 87 · see citing works
Monitoring activity of patients with bipolar disorder using smart phones
Osmani, V., Maxhuni, A., Grunerbl, A., Lukowicz, P., Haring, C., & Mayora, O. (2013) · Advances in Mobile Multimedia · doi:10.1145/2536853.2536882 · Cited by 89 · see citing works
Validation of the ANCA-associated vasculitis patient-reported outcomes (AAV-PRO) questionnaire
Robson, J. C., Dawson, J., Doll, H., Cronholm, P. F., Milman, N., Kellom, K. S., Ashdown, S., Easley, E., Gebhart, D. E., Lanier, G. E., et al. (2018) · Annals of the Rheumatic Diseases · doi:10.1136/annrheumdis-2017-212713 · Cited by 88 · see citing works
Bos, F. M., Schoevers, R. A., & aan het Rot, M. (2015) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2015.08.008 · Cited by 81 · see citing works
Chih, M., DuBenske, L. L., Hawkins, R. P., Brown, R., Dinauer, S., Cleary, J. F., & Gustafson, D. H. (2012) · Palliative Medicine · doi:10.1177/0269216312457213 · Cited by 83 · see citing works
Belisario, J. S. M., Huckvale, K., Saje, A., Porcnik, A., Morrison, C. P., & Car, J. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000042 · Cited by 80 · see citing works
Hauth, F., Bizu, V., App, R., Lautenbacher, H., Tenev, A., Bitzer, M., Malek, N. P., Zips, D., & Gani, C. (2019) · JMIR mhealth and uhealth · doi:10.2196/12345 · Cited by 82 · see citing works
Buckner, J. D., & Heimberg, R. G. (2010) · Psychology of Addictive Behaviors · doi:10.1037/a0020968 · Cited by 79 · see citing works
Eremenco, S. L., Coons, S. J., Paty, J. A., Coyne, K. S., Bennett, A. V., & McEntegart, D. J. (2014) · Value in Health · doi:10.1016/j.jval.2014.06.005 · Cited by 79 · see citing works
Metabolic Phenotyping Guidelines: studying eating behaviour in humans
Gibbons, C., Finlayson, G., Dalton, M., Caudwell, P., & Blundell, J. E. (2014) · Journal of Endocrinology · doi:10.1530/joe-14-0020 · Cited by 74 · see citing works
Fritz, F., Balhorn, S., Riek, M., Breil, B., & Dugas, M. (2012) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2011.12.008 · Cited by 78 · see citing works
Fathers' Stress in a Neonatal Intensive Care Unit
Noergaard, B., Ammentorp, J., Garne, E., Fenger‐Grøn, J., & Kofoed, P. (2018) · Advances in Neonatal Care · doi:10.1097/anc.0000000000000503 · Cited by 75 · see citing works
Bracing for the worst, but behaving the best: Social anxiety, hostility, and behavioral aggression
DeWall, C. N., Buckner, J. D., Lambert, N. M., Cohen, A. S., & Fincham, F. D. (2009) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2009.12.002 · Cited by 73 · see citing works
Bangerter, A., Ness, S. L., Aman, M. G., Esbensen, A. J., Goodwin, M. S., Dawson, G., Hendren, R. L., LEVENTHAL, B. L., Khan, A., Opler, M. G. A., et al. (2017) · Journal of Child and Adolescent Psychopharmacology · doi:10.1089/cap.2017.0018 · Cited by 72 · see citing works
Barr, P. J., Forcino, R. C., Thompson, R., Ozanne, E. M., Arend, R., Castaldo, M. G., O’Malley, A. J., & Elwyn, G. J. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-014681 · Cited by 69 · see citing works
A Comparison of Tablet-Based and Paper-Based Survey Data Collection in Conservation Projects
Leisher, C. (2014) · Social Sciences · doi:10.3390/socsci3020264 · Cited by 67 · see citing works
Mode differences in a mixed-mode health interview survey among adults
Hoebel, J., von der Lippe, E., Lange, C., & Ziese, T. (2014) · Archives of Public Health · doi:10.1186/2049-3258-72-46 · Cited by 67 · see citing works
Nanivadekar, A. C., Bose, R., Petersen, B. A., Okorokova, E. V., Sarma, D., Madonna, T. J., Barra, B., Farooqui, J., Dalrymple, A. N., Levy, I., et al. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01153-8 · Cited by 65 · see citing works
Hjermstad, M. J., Lie, H. C., Caraceni, A. T., Currow, D. C., Fainsinger, R. L., Gundersen, O. E., Haugen, D. F., Heitzer, E., Radbruch, L., Stone, P. C., et al. (2012) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2011.10.025 · Cited by 66 · see citing works
Anxiety Sensitivity Prospectively Predicts Posttraumatic Stress Symptoms Following a Campus Shooting
Boffa, J. W., Norr, A. M., Raines, A. M., Albanese, B. J., Short, N. A., & Schmidt, N. B. (2016) · Behavior Therapy · doi:10.1016/j.beth.2016.02.006 · Cited by 66 · see citing works
Paty, J. A., Turner‐Bowker, D. M., Elash, C. A., & Wright, D. D. (2015) · Phlebology The Journal of Venous Disease · doi:10.1177/0268355515595193 · Cited by 65 · see citing works
Bjørner, J. B., Rose, M., Gandek, B., Stone, A. A., Junghaenel, D. U., & Ware, J. E. (2013) · Quality of Life Research · doi:10.1007/s11136-013-0451-4 · Cited by 64 · see citing works
Social anxiety and alcohol problems: The roles of perceived descriptive and injunctive peer norms
Buckner, J. D., Ecker, A. H., & Proctor, S. L. (2011) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2011.02.003 · Cited by 64 · see citing works
White, M. K., Maher, S. M., Rizio, A. A., & Bjørner, J. B. (2018) · Quality of Life Research · doi:10.1007/s11136-018-1851-2 · Cited by 63 · see citing works
Impact of COVID-19 Lockdown Measures on Spanish People with Chronic Pain: An Online Study Survey
Nieto, R., Pardo, R., Sora, B., Feliu-Soler, A., & Luciano, J. V. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9113558 · Cited by 61 · see citing works
The Current State and Validity of Digital Assessment Tools for Psychiatry: Systematic Review
Martin-Key, N. A., Spadaro, B., Funnell, E. L., Barker, E., Schei, T. S., Tomasik, J., & Bahn, S. (2021) · JMIR Mental Health · doi:10.2196/32824 · Cited by 61 · see citing works
College drinking problems and social anxiety: The importance of drinking context.
Terlecki, M. A., Ecker, A. H., & Buckner, J. D. (2014) · Psychology of Addictive Behaviors · doi:10.1037/a0035770 · Cited by 61 · see citing works
van Kollenburg, R. A. A., de Bruin, D. M., & Wijkstra, H. (2019) · Journal of Medical Internet Research · doi:10.2196/13490 · Cited by 61 · see citing works
Bächinger, D., Röösli, C., Ditzen, B., & Huber, A. M. (2016) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-016-3915-7 · Cited by 60 · see citing works
Utility-Based Instruments for People with Dementia: A Systematic Review and Meta-Regression Analysis
Li, L., Nguyen, K., Comans, T. A., & Scuffham, P. (2017) · Value in Health · doi:10.1016/j.jval.2017.09.005 · Cited by 58 · see citing works
Bird, M., Callisaya, M. L., Cannell, J., Gibbons, T., Smith, S., & Ahuja, K. (2016) · Interactive Journal of Medical Research · doi:10.2196/ijmr.4910 · Cited by 56 · see citing works
Maillart, É., Labauge, P. M., Cohen, M., Maarouf, A., Vukusic, S., Donzé, C., Gallien, P., Seze, J. D., Bourre, B., Moreau, T., et al. (2019) · European Journal of Neurology · doi:10.1111/ene.14091 · Cited by 56 · see citing works
Bratan, T., Aichinger, H., Brkic, N., Rueter, J., Apfelbacher, C., Boyer, L., & Loss, J. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-049086 · Cited by 54 · see citing works
Electronic‐Diary for Recording Headaches, Triggers, and Medication Use: Development and Evaluation
Bandarian‐Balooch, S., Martin, P. R., McNally, B., Brunelli, A., & Mackenzie, S. (2017) · Headache The Journal of Head and Face Pain · doi:10.1111/head.13184 · Cited by 54 · see citing works
The Role of Social Anxiety in a Brief Alcohol Intervention for Heavy-Drinking College Students
Terlecki, M. A., Buckner, J. D., Larimer, M. E., & Copeland, A. L. (2011) · Journal of Cognitive Psychotherapy · doi:10.1891/0889-8391.25.1.7 · Cited by 53 · see citing works
Heller, S. R., Lawton, J., Amiel, S. A., Cooke, D. D., Mansell, P., Brennan, A., Elliott, D., Boote, J. D., Emery, C., Baird, W. O., et al. (2014) · Programme Grants for Applied Research · doi:10.3310/pgfar02050 · Cited by 53 · see citing works
Gerritsen, R. T., Koopmans, M., Hofhuis, J. G. M., Curtis, J. R., Jensen, H. I., Zijlstra, J. G., Engelberg, R. A., & Spronk, P. E. (2016) · CHEST Journal · doi:10.1016/j.chest.2016.09.003 · Cited by 53 · see citing works
Byrom, B., Elash, C. A., Eremenco, S. L., Bodart, S., Muehlhausen, W., Platko, J. V., Watson, C., & Howry, C. (2022) · Therapeutic Innovation & Regulatory Science · doi:10.1007/s43441-022-00376-2 · Cited by 52 · see citing works
Byrom, B., Doll, H., Muehlhausen, W., Flood, E. M., Cassedy, C., McDowell, B., Sohn, J., Hogan, K., Belmont, R., Skerritt, B., et al. (2017) · Value in Health · doi:10.1016/j.jval.2017.10.008 · Cited by 52 · see citing works
Braekman, E., Berete, F., Charafeddine, R., Demarest, S. S., Drieskens, S., Gisle, L., Molenberghs, G., Tafforeau, J., der Heyden, J. V., & Hal, G. F. V. (2018) · PLoS ONE · doi:10.1371/journal.pone.0197434 · Cited by 52 · see citing works
Weber, M., Groulx, M. W., Lemieux, C. J., Scott, D. J., & Dawson, J. (2019) · Journal of Sustainable Tourism · doi:10.1080/09669582.2019.1620754 · Cited by 51 · see citing works
Richter, J., Nannen, C., Chehab, G., Acar, H., Becker, A., Willers, R., Huscher, D., & Schneider, M. (2021) · Arthritis Research & Therapy · doi:10.1186/s13075-021-02500-3 · Cited by 51 · see citing works
Zhang, X., Zhang, Q., Fan, Q.-X., Zhang, C., You, Y., Zhang, C., Lin, T., Xu, L., Moutari, S., Moore, J. E., et al. (2021) · International Journal of Ophthalmology · doi:10.18240/ijo.2021.06.07 · Cited by 51 · see citing works
Laudenslager, M. L., Simoneau, T. L., Mikulich‐Gilbertson, S. K., Natvig, C. L., Brewer, B. W., Sannes, T. S., Kilbourn, K. M., Gutman, J., & McSweeney, P. A. (2019) · Psycho-Oncology · doi:10.1002/pon.5126 · Cited by 49 · see citing works
Buckner, J. D., Silgado, J., & Schmidt, N. B. (2010) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2010.07.005 · Cited by 49 · see citing works
Oak, S. R., O’Rourke, C., Strnad, G. J., Andrish, J. T., Parker, R. D., Saluan, P. M., Jones, M. H., Stegmeier, N., & Spindler, K. P. (2015) · The American Journal of Sports Medicine · doi:10.1177/0363546515589108 · Cited by 49 · see citing works
on behalf of the PROMs, P. A. E. P., O'Connell, S. M., Palmer, R. I., Withers, K. L., Saha, N., Puntoni, S., & Carolan‐Rees, G. (2018) · Pilot and Feasibility Studies · doi:10.1186/s40814-018-0282-8 · Cited by 49 · see citing works
Hengen, K. M., & Alpers, G. W. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.01676 · Cited by 49 · see citing works
Li, Q., Xu, Y., Zhou, H., & Loke, A. Y. (2016) · BMC Palliative Care · doi:10.1186/s12904-016-0142-3 · Cited by 47 · see citing works
Automation of Stress Recognition Using Subjective or Objective Measures
Goyal, A., Singh, S. N., Vir, D., & Pershad, D. (2016) · Psychological Studies · doi:10.1007/s12646-016-0379-1 · Cited by 48 · see citing works
- Authors
- Gwaltney, C. J., Shields, A. L., & Shiffman, S.
- Affiliations
- Brown University (US)
- East Tennessee State University (US)
- University of Pittsburgh (US)
- Venue
- Value in Health
- Publisher
- Elsevier BV
- Published
- 2008-03
- Volume
- 11
- Issue
- 2
- Pages
- 322-333
- ISSN
- 1098-3015 (ISSN-L)
- OpenAlex
- W2146545407
- Cited by
- 623
- References
- 76
- Citations by year
- 2026: 18 · 2025: 23 · 2024: 20 · 2023: 43 · 2022: 36 · 2021: 42 · 2020: 49 · 2019: 45 · 2018: 56 · 2017: 44 · 2016: 51 · 2015: 35 · 2014: 44 · 2013: 28 · 2012: 25
- FWCI
- 5.19
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Value in Health (publishedVersion)
- PubMed (publishedVersion)
- Digital Commons - East Tennessee State University (East Tennessee State University) (submittedVersion)
- Topics
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- patient-reported outcomes, paper-and-pencil administration, mode effects, measurement invariance, test-retest reliability, clinical trials
- MeSH terms
- Data Collection, Humans, Statistics as Topic, Patient Satisfaction, Outcome Assessment, Health Care, Computers, Handheld
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
