Cited by 623Health Systems, Economic Evaluations, Quality of LifeSurvey Methodology and NonresponsePsychometric Methodologies and Testing

The DOI matches its Crossref record; 74 of the 76 references it cites were found in a registry or an index.

Equivalence of Electronic and Paper-and-Pencil Administration of Patient-Reported Outcome Measures: A Meta-Analytic Review

Open access

Gwaltney, Chad J., Shields, Alan L., Shiffman, Saul·2008·Value in Health

DOI 10.1111/j.1524-4733.2007.00231.x

Open this paper in CoChat — free account

Abstract

Objectives Patient-reported outcomes (PROs; self-report assessments) are increasingly important in evaluating medical care and treatment efficacy. Electronic administration of PROs via computer is becoming widespread. This article reviews the literature addressing whether computer-administered tests are equivalent to their paper-and-pencil forms.Methods Meta-analysis was used to synthesize 65 studies that directly assessed the equivalence of computer versus paper versions of PROs used in clinical trials. A total of 46 unique studies, evaluating 278 scales, provided sufficient detail to allow quantitative analysis.Results Among 233 direct comparisons, the average mean difference between modes averaged 0.2% of the scale range (e.g., 0.02 points on a 10-point scale), and 93% were within +/-5% of the scale range. Among 207 correlation coefficients between paper and computer instruments (typically intraclass correlation coefficients), the average weighted correlation was 0.90; 94% of correlations were at least 0.75. Because the cross-mode correlation (paper vs. computer) is also a test-retest correlation, with potential variation because of retest, we compared it to the within-mode (paper vs. paper) test-retest correlation. In four comparisons that evaluated both, the average cross-mode paper-to-computer correlation was almost identical to the within-mode correlation for readministration of a paper measure (0.88 vs. 0.91).Conclusions Extensive evidence indicates that paper- and computer-administered PROs are equivalent.

Integrity signals

Registry

Crossref · 10.1111/j.1524-4733.2007.00231.x

Cites

74 of 74 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646
Book Reviews: Statistical Methods for Meta-Analysis Larry V. Hedges and Ingram Olkin New York: Academic Press, 1985. xxii + 369 pp
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,377
Statistical Methods for Rates and Proportions.
Everitt, B. S., & Fleiss, J. L. (1981) · Biometrics · doi:10.2307/2530193 · Cited by 9,034
Forming inferences about some intraclass correlation coefficients.
McGraw, K. O., & Wong, S. P. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.1.30 · Cited by 6,903
Measurement of health status
Jaeschke, R., Singer, J., & Guyatt, G. H. (1989) · Controlled Clinical Trials · doi:10.1016/0197-2456(89)90005-6 · Cited by 4,982
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,840
Models for Longitudinal Data: A Generalized Estimating Equation Approach
Zeger, S. L., Liang, K., & Albert, P. S. (1988) · Biometrics · doi:10.2307/2531734 · Cited by 4,505
Statistical Methods for Rates and Proportions, 2nd ed
GROVE, W. A. (1981) · American Journal of Psychiatry · doi:10.1176/ajp.138.12.1644-a · Cited by 3,756
The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability
Fleiss, J. L., & Cohen, J. (1973) · Educational and Psychological Measurement · doi:10.1177/001316447303300309 · Cited by 3,354

Show all 76 references

Research works printed but not linked: 3

Food and Drug Administration, 2006, Guidance for industry: Patient Reported Outcome measures: use in medical product development to support labeling claims, Fed Regis, 71, 1
Hufford, 2002, Electronic subject diaries: an examination of applications and what works in the field, Appl Clin Trials, 11, 46
Hufford MR, Shiffman S. Correspondence between paper and electronic visual analog scales among adult asthmatics. Paper Presented at the Drug Information Association Statistics Conference. Hilton Head, South Carolina, 2002.

Sorted by citations

Items 101–200 of 622

Use of daily electronic patient-reported outcome (PRO) diaries in randomized controlled trials for rheumatoid arthritis: rationale and implementation
Bingham, C. O., Gaich, C. L., DeLozier, A. M., Engstrom, K. D., Naegeli, A. N., de Bono, S., Banerjee, P., & Taylor, P. C. (2019) · Trials · doi:10.1186/s13063-019-3272-0 · Cited by 48
Toward Electronic Health Recording: Evaluation of Electronic Patient-reported Outcome Measures System for Remote Monitoring of Early Rheumatoid Arthritis
El Miedany, Y., El Gaafary, M., Youssef, S., Bahlas, S., Almedany, S., Ahmed, I., & Palmer, D. (2016) · The Journal of Rheumatology · doi:10.3899/jrheum.151421 · Cited by 47
Adaptação Transcultural Brasil-Portugal da Escala de Satisfação com o Suporte Social para Estudantes do Ensino Superior
Marôco, J. P., Campos, J. Á. D. B., Vinagre, M. D. G., & Ribeiro, J. L. P. (2014) · Psicologia Reflexão e Crítica · doi:10.1590/1678-7153.201427205 · Cited by 57
The usability of a WeChat-based electronic questionnaire for collecting participant-reported data in female pelvic floor disorders: a comparison with the traditional paper-administered format
Sun, Z., Zhu, L., Liang, M., Xu, T., & Lang, J. (2016) · Menopause The Journal of The North American Menopause Society · doi:10.1097/gme.0000000000000690 · Cited by 45
Electronic forms for patient reported outcome measures (PROMs) are an effective, time-efficient, and cost-minimizing alternative to paper forms
Yu, J. Y., Goldberg, T., Lao, N., Feldman, B. M., & Goh, Y. I. (2021) · Pediatric Rheumatology · doi:10.1186/s12969-021-00551-z · Cited by 46
The Four-Dimensional Symptom Questionnaire (4DSQ) in the general population: scale structure, reliability, measurement invariance and normative data: a cross-sectional survey
Terluin, B., Smits, N., Brouwers, E. P. M., & de Vet, H. C. W. (2016) · Health and Quality of Life Outcomes · doi:10.1186/s12955-016-0533-4 · Cited by 45
Outcome measurement in plastic surgery
Wormald, J. C. R., & Rodrigues, J. N. (2017) · Journal of Plastic Reconstructive & Aesthetic Surgery · doi:10.1016/j.bjps.2017.11.015 · Cited by 46
Integrating cancer survivors' experiences into UK cancer registries: design and development of the ePOCS system (electronic Patient-reported Outcomes from Cancer Survivors)
Ashley, L., Jones, H., Thomas, J., Forman, D., Newsham, A., Morris, E. J. A., Johnson, O. A., Velikova, G., & Wright, P. (2011) · British Journal of Cancer · doi:10.1038/bjc.2011.424 · Cited by 44
The True Colours Remote Symptom Monitoring System: A Decade of Evolution
Goodday, S., Atkinson, L. Z., Goodwin, G. M., Saunders, K. E. A., South, M., Mackay, C. E., Denis, M., Hinds, C., Attenburrow, M.-J., Davies, J., et al. (2019) · Journal of Medical Internet Research · doi:10.2196/15188 · Cited by 43
Electronic Data Capture Versus Conventional Data Collection Methods in Clinical Pain Studies: Systematic Review and Meta-Analysis
Jibb, L., Khan, J. S., Seth, P., Lalloo, C., Mulrooney, L., Nicholson, K., Nowak, D. A., Kaur, H., Chee-A-Tow, A., Foster, J., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/16480 · Cited by 42
Assessing self-regulation strategies: development and validation of the tempest self-regulation questionnaire for eating (TESQ-E) in adolescents
de Vet, E., de Ridder, D. T. D., Stok, F. M., Brunsø, K., Băban, A., & Gaspar, T. (2014) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-014-0106-z · Cited by 46
Internet Surveys as Part of a Mixed-Mode Design
de Leeuw, E. D. (2018) · Routledge eBooks · doi:10.4324/9780203844922-3 · Cited by 43
Stability and change in patterns of concerns related to eating, weight, and shape in young adult women: A latent transition analysis.
Cain, A. S., Epler, A. J., Steinley, D., & Sher, K. J. (2010) · Journal of Abnormal Psychology · doi:10.1037/a0018117 · Cited by 53
Randomized controlled trial of brief alcohol screening and intervention for college students for heavy-drinking mandated and volunteer undergraduates: 12-month outcomes.
Terlecki, M. A., Buckner, J. D., Larimer, M. E., & Copeland, A. L. (2015) · Psychology of Addictive Behaviors · doi:10.1037/adb0000056 · Cited by 42
Best Practices for the Electronic Implementation and Migration of Patient-Reported Outcome Measures
Mowlem, F. D., Elash, C. A., Dumais, K. M., Haenel, E., O’Donohoe, P., Olt, J., Kalpadakis-Smith, A. V., James, B., Balestrieri, G., Becker, K., et al. (2023) · Value in Health · doi:10.1016/j.jval.2023.10.007 · Cited by 41
Remote Collection of Patient-Reported Outcomes Following Outpatient Hand Surgery: A Randomized Trial of Telephone, Mail, and E-Mail
Schwartzenberger, J., Presson, A. P., Lyle, A., O'Farrell, A. L., & Tyser, A. R. (2017) · The Journal Of Hand Surgery · doi:10.1016/j.jhsa.2017.05.002 · Cited by 41
A comparison of data quality and practicality of online versus postal questionnaires in a sample of testicular cancer survivors
Smith, A. B., King, M. T., Butow, P. N., & Olver, I. (2011) · Psycho-Oncology · doi:10.1002/pon.2052 · Cited by 41
Validity and Responsiveness of the Knee Injury and Osteoarthritis Outcome Score: A Comparative Study Among Total Knee Replacement Patients
Gandek, B., & Ware, J. E. (2017) · Arthritis Care & Research · doi:10.1002/acr.23193 · Cited by 40
Fitting in and feeling fine: Conformity and coping motives differentially mediate the relationship between social anxiety and drinking problems for men and women
Buckner, J. D., & Shah, S. M. (2014) · Addiction Research & Theory · doi:10.3109/16066359.2014.978304 · Cited by 44
The State of Assessment in Human-Animal Interaction Research
Rodriguez, K. E., Guérin, N. A., Gabriels, R. L., Serpell, J., Schreiner, P., & O’Haire, M. (2018) · Human-animal interaction bulletin · doi:10.1079/hai.2018.0022 · Cited by 39
Updated Recommendations on Evidence Needed to Support Measurement Comparability Among Modes of Data Collection for Patient-Reported Outcome Measures: A Good Practices Report of an ISPOR Task Force
O’Donohoe, P., Reasner, D. S., Kovacs, S. M., Byrom, B., Eremenco, S. L., Barsdorf, A. I., Arnera, V., & Coons, S. J. (2023) · Value in Health · doi:10.1016/j.jval.2023.01.001 · Cited by 38
A within-subjects trial to test the equivalence of online and paper outcome measures: the Roland Morris Disability Questionnaire
Bishop, F. L., Lewis, G., Harris, S., McKay, N. D., Prentice, P. M., Thiel, H. W., & Lewith, G. T. (2010) · BMC Musculoskeletal Disorders · doi:10.1186/1471-2474-11-113 · Cited by 39
11 Computer-based competence tests in the national educational panel study: The challenge of mode effects
Kröhne, U., & Martens, T. (2011) · Zeitschrift für Erziehungswissenschaft · doi:10.1007/s11618-011-0185-4 · Cited by 36
Collection of patient-reported outcomes; - text messages on mobile phones provide valid scores and high response rates
Christie, A., Dagfinrud, H. S., Dale, Ø., Schulz, T. W., & Hagen, K. B. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-52 · Cited by 36
Comparing the measurement equivalence of EQ-5D-5L across different modes of administration
Mulhern, B. J., O’Gorman, H., Rotherham, N., & Brazier, J. E. (2015) · Health and Quality of Life Outcomes · doi:10.1186/s12955-015-0382-6 · Cited by 36
Refinement and Validation of the Social Participation Restrictions Questionnaire: An Application of Rasch Analysis and Traditional Psychometric Analysis Techniques
Heffernan, E., Maidment, D. W., Barry, J. G., & Ferguson, M. (2018) · Ear and Hearing · doi:10.1097/aud.0000000000000618 · Cited by 38
Psychometric performance assessment of Malay and Malaysian English version of EQ-5D-5L in the Malaysian population
Shafie, A. A., Thakumar, A. V., Lim, C. J., & Luo, N. (2018) · Quality of Life Research · doi:10.1007/s11136-018-2027-9 · Cited by 36
Developing an International Standard Set of Patient-Reported Outcome Measures for Psychotic Disorders
McKenzie, E., Matkin, L., Fialho, L. S., Emelurumonye, I. N., Gintner, T., Ilesanmi, C., Jagger, B., Quinney, S., Anderson, E. L., Baandrup, L., et al. (2021) · Psychiatric Services · doi:10.1176/appi.ps.202000888 · Cited by 39
User satisfaction is influenced by the interval between a health care service and the assessment of the service
Jensen, H. I., Ammentorp, J., & Kofoed, P. (2010) · Social Science & Medicine · doi:10.1016/j.socscimed.2010.02.035 · Cited by 36
Toward a better understanding of patient-reported outcomes in clinical practice.
Bitton, A., Onega, T., Tosteson, A. N. A., & Haas, J. S. (2014) · PubMed · Cited by 37
Can electronic web-based technology improve quality of life data collection? Analysis of Radiation Therapy Oncology Group 0828
Movsas, B., Hunt, D. L., Bruner, D. W., Lee, W. R., Tharpe, H., Goldstein, D. S., Moore, J. V., Dayes, I. S., Parise, S., & Sandler, H. M. (2013) · Practical Radiation Oncology · doi:10.1016/j.prro.2013.07.014 · Cited by 35
Cannabis Use Behaviors and Social Anxiety: The Roles of Perceived Descriptive and Injunctive Social Norms
Ecker, A. H., & Buckner, J. D. (2014) · Journal of Studies on Alcohol and Drugs · doi:10.15288/jsad.2014.75.74 · Cited by 39
Is it too early to move to full electronic PROM data collection?
Palmen, L. N., Schrier, J. C. M., Scholten, R., Jansen, J., & Koëter, S. (2015) · Foot and Ankle Surgery · doi:10.1016/j.fas.2015.05.001 · Cited by 35
Outcome Measures in Myasthenia Gravis: Incorporation Into Clinical Practice
Muppidi, S. (2017) · Journal of Clinical Neuromuscular Disease · doi:10.1097/cnd.0000000000000156 · Cited by 35
Measurement Equivalence of Patient-Reported Outcome Measures Migrated to Electronic Formats: A Review of Evidence and Recommendations for Clinical Trials and Bring Your Own Device
Byrom, B., Gwaltney, C. J., Slagle, A. F., Gnanasakthy, A., & Muehlhausen, W. (2018) · Therapeutic Innovation & Regulatory Science · doi:10.1177/2168479018793369 · Cited by 35
Self-citation
Registries for Evaluating Patient Outcomes
Gliklich, R. E., Dreyer, N. A., & Leavy, M. B. (2014) · Cited by 36
How does the context of physical activity influence perceived mood and wellbeing after exercise?
White, R. L., Ryan, D., Young, C., Elston, R., & Rossi, T. (2023) · Mental health and physical activity · doi:10.1016/j.mhpa.2023.100504 · Cited by 40
Development of the web-based Spanish and Catalan versions of the Euroqol 5D-Y (EQ-5D-Y) and comparison of results with the paper version
Robles, N., Rajmil, L., Rodríguez-Arjona, D., Azuara, M., Codina, F. N., Raat, H., Ravens‐Sieberer, U., & Herdman, M. (2015) · Health and Quality of Life Outcomes · doi:10.1186/s12955-015-0271-z · Cited by 38
Pain palliation measurement in cancer clinical trials: The US Food and Drug Administration perspective
Basch, E., Trentacosti, A. M., Burke, L. B., Kwitkowski, V. E., Kane, R. C., Autio, K. A., Papadopoulos, E. J., Stansbury, J. P., Kluetz, P. G., Smith, H., et al. (2013) · Cancer · doi:10.1002/cncr.28470 · Cited by 33
A Qualitative Review of the Psychometric Properties and Feasibility of Electronic Headache Diaries for Children and Adults: Where We Are and Where We Need to Go
Stinson, J., Huguet, A., McGrath, P. J., Rosenbloom, B. N., Soobiah, C., White, M., & Coburn, G. (2013) · Pain Research and Management · doi:10.1155/2013/369541 · Cited by 33
Psychometric properties of the Japanese version of the Fear-Avoidance Beliefs Questionnaire (FABQ)
Matsudaira, K., Kikuchi, N., Murakami, A., & Isomura, T. (2013) · Journal of Orthopaedic Science · doi:10.1007/s00776-013-0471-5 · Cited by 33
Digital audio recordings improve the outcomes of patient consultations: A randomised cluster trial
Wolderslund, M., Kofoed, P., Holst, R., Axboe, M. K., & Ammentorp, J. (2016) · Patient Education and Counseling · doi:10.1016/j.pec.2016.08.029 · Cited by 33
Routine psychosocial distress screening in radiotherapy: implementation and evaluation of a computerised procedure
Dinkel, A., Berg, P., Pirker, C., Geinitz, H., Sehlen, S., Emrich, M., Marten‐Mittag, B., Henrich, G., Book, K., & Herschbach, P. (2010) · British Journal of Cancer · doi:10.1038/sj.bjc.6605930 · Cited by 32
Comparison of the paper‐based and electronic versions of the Dermatology Life Quality Index: evidence of equivalence
Ali, F. M., Johns, N., Finlay, A. Y., Salek, S., & Piguet, V. (2017) · British Journal of Dermatology · doi:10.1111/bjd.15314 · Cited by 32
Test–retest reliability and comparability of paper and computer questionnaires for the Finnish version of the Tampa Scale of Kinesiophobia
Koho, P., Aho, S., Kautiainen, H., Pohjolainen, T., & Hurri, H. (2014) · Physiotherapy · doi:10.1016/j.physio.2013.11.007 · Cited by 32
The Psoriatic Arthritis Impact of Disease 12-item questionnaire: equivalence, reliability, validity, and feasibility of the touch-screen administration versus the paper-and-pencil version
Salaffi, F., Carlo, M. D., Carotti, M., Farah, S., & Gutiérrez, M. (2016) · Therapeutics and Clinical Risk Management · doi:10.2147/tcrm.s101619 · Cited by 31
Evaluation of the Swedish Web-Version of Quality of Recovery (SwQoR): Secondary Step in the Development of a Mobile Phone App to Measure Postoperative Recovery
Dahlberg, K., Jaensson, M., Eriksson, M. H., & Nilsson, U. G. (2016) · JMIR Research Protocols · doi:10.2196/resprot.5881 · Cited by 31
Computerized Adaptive Testing—Ready for Ambulatory Monitoring?
Rose, M., Bjørner, J. B., Fischer, F. H., Anatchkova, M., Gandek, B., Klapp, B. F., & Ware, J. E. (2012) · Psychosomatic Medicine · doi:10.1097/psy.0b013e3182547392 · Cited by 31
Evaluation of the SCA instrument for measuring patient satisfaction with cancer care administered via paper or via the Internet
Kamo, N., Dandapani, S. V., Miksad, R. A., Houlihan, M. J., Kaplan, I. D., Regan, M. M., Greenfield, T. K., & Sanda, M. G. (2010) · Annals of Oncology · doi:10.1093/annonc/mdq417 · Cited by 31
Collection of daily patient reported outcomes is feasible and demonstrates differential patient experience in chronic kidney disease
Pittman, Z., John, S. G., & McIntyre, C. W. (2016) · Hemodialysis International · doi:10.1111/hdi.12470 · Cited by 30
Norms and validation of the online and paper-and-pencil versions of the Abbreviated Math Anxiety Scale (AMAS) for Polish adolescents and adults
Cipora, K., Willmes, K., Szwarc, A., & Nuerk, H. (2018) · Journal of Numerical Cognition · doi:10.5964/jnc.v3i3.121 · Cited by 30
Mode Equivalence of Health Indicators Between Data Collection Modes and Mixed-Mode Survey Designs in Population-Based Health Interview Surveys for Children and Adolescents: Methodological Study
Mauz, E., Hoffmann, R. G., Houben, R., Krause, L., Kamtsiuris, P., & Gößwald, A. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.7802 · Cited by 30
An electronic patient-reported outcomes measurement system in paediatric orthopaedics
Sabatino, M. J., Gans, C. V., Zynda, A. J., Chung, J. S., Miller, S. M., Wilson, P. L., Jo, C.-H., & Ellis, H. B. (2019) · Journal of Children s Orthopaedics · doi:10.1302/1863-2548.13.190053 · Cited by 30
Clinical Outcome Assessment in Cancer Rehabilitation and the Central Role of Patient-Reported Outcomes
Lehmann, J., Rothmund, M., Riedl, D., Rumpold, G., Grote, V., Fischer, M. J., & Holzner, B. (2021) · Cancers · doi:10.3390/cancers14010084 · Cited by 30
Electronic Versus Paper-Based Assessment of Health-Related Quality of Life Specific to HIV Disease: Reliability Study of the PROQOL-HIV Questionnaire
Duracinský, M., Lalanne, C., Goujard, C., Herrmann, S., Cheung-Lung, C., Brosseau, J.-P., Schwartz, Y., & Chassany, O. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3330 · Cited by 30
A Web site–based reporting system for monitoring home treatment during oral immunotherapy for food allergy
Nachshon, L., Goldberg, M. R., Elizur, A., Levy, M. B., Schwartz, N., & Katz, Y. (2015) · Annals of Allergy Asthma & Immunology · doi:10.1016/j.anai.2015.04.007 · Cited by 30
Management of gastrointestinal symptoms in advanced cancer patients: the rapid learning cancer clinic model
Abernethy, A. P., Wheeler, J. L., & Zafar, S. Y. (2010) · Current Opinion in Supportive and Palliative Care · doi:10.1097/spc.0b013e32833575fd · Cited by 30
Feel My Pain
Price, B. A., Kelly, R., Mehta, V., McCormick, C., Ahmed, H., & Pearce, O. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173743 · Cited by 30
An interactive voice response diary for patients with non‐dystrophic myotonia
Statland, J. M., Wang, Y., Richesson, R., Bundy, B., Herbelin, L., Gomes, J., Trivedi, J., Venance, S., Amato, A., Hanna, M., et al. (2011) · Muscle & Nerve · doi:10.1002/mus.22007 · Cited by 30
Distress tolerance as a mechanism of mindfulness for depression and anxiety: Cross-sectional and diary evidence
Li, Y., Ju, R., Hofmann, S. G., Chiu, W., Ye, G., Leng, Y., & Liu, X. (2023) · International Journal of Clinical and Health Psychology · doi:10.1016/j.ijchp.2023.100392 · Cited by 33
Detailing of gastrointestinal symptoms in cancer patients with advanced disease: new methodologies, new insights, and a proposed approach
Abernethy, A. P., Wheeler, J. L., & Zafar, S. Y. (2009) · Current Opinion in Supportive and Palliative Care · doi:10.1097/spc.0b013e32832531ce · Cited by 29
Ecological momentary assessment for rehabilitation of chronic illness and disability
McKeon, A. B., McCue, M. P., Skidmore, E. R., Schein, M. L., & Kulzer, J. L. (2017) · Disability and Rehabilitation · doi:10.1080/09638288.2017.1280545 · Cited by 27
Improving domain definition and outcome instrument selection: Lessons learned for OMERACT from imaging
D’Agostino, M. A., Beaton, D., Maxwell, L. J., Cembalo, S. M., Hoens, A. M., Hofstetter, C., Zăbălan, C., Bird, P., Christensen, R., de Wit, M. P. T., et al. (2021) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2021.08.004 · Cited by 29
A Novel Digital Patient-Reported Outcome Platform for Head and Neck Oncology Patients–-A Pilot Study
Peltola, M., Lehikoinen, J. S. J., Sippola, L. T., Saarilahti, K., & Mäkitie, A. A. (2016) · Clinical Medicine Insights Ear Nose and Throat · doi:10.4137/cment.s40219 · Cited by 29
Sexual body esteem and mindfulness in college women.
Fink, S., Foran, K. A., Sweeney, A. C., & O’Hea, E. L. (2009) · Body Image · doi:10.1016/j.bodyim.2009.07.003 · Cited by 28
Validity and reproducibility of the Italian version of the patient determined disease steps scale in people with multiple sclerosis
Lavorgna, L., Sparaco, M., Esposito, S., Motl, R. W., Gallo, A., Bisecco, A., Tedeschi, G., & Bonavita, S. (2017) · Multiple Sclerosis and Related Disorders · doi:10.1016/j.msard.2017.09.027 · Cited by 28
Content validity of the Migraine-Specific Quality of Life Questionnaire version 2.1 electronic patient-reported outcome
Speck, R. M., Shalhoub, H., Ayer, D. W., Ford, J. H., Wyrwich, K. W., & Bush, E. N. (2019) · Journal of Patient-Reported Outcomes · doi:10.1186/s41687-019-0138-x · Cited by 28
Smartphone-Based Self-Reports of Depressive Symptoms Using the Remote Monitoring Application in Psychiatry (ReMAP): Interformat Validation Study
Goltermann, J., Emden, D., Leehr, E. J., Dohm, K., Redlich, R., Dannlowski, U., Hahn, T., & Opel, N. (2021) · JMIR Mental Health · doi:10.2196/24333 · Cited by 28
Measurement equivalence and feasibility of the EORTC QLQ-PR25: paper-and-pencil versus touch-screen administration
Chang, Y., Chang, C., Peng, C., Wu, H., Lin, H.-C., Wang, J., Lin, C., Yeh, Y., & Liang, W.-M. (2014) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-12-23 · Cited by 28
Quality of life before and after sinonasal surgery: a population-based matched cohort study
Alakärppä, A. I., Koskenkorva, T. J., Koivunen, P. T., & Alho, O. (2016) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-016-4272-2 · Cited by 27
Standards for Instrument Migration When Implementing Paper Patient-Reported Outcome Instruments Electronically: Recommendations from a Qualitative Synthesis of Cognitive Interview and Usability Studies
Muehlhausen, W., Byrom, B., Skerritt, B., Carthy, M. M., McDowell, B., & Sohn, J. (2017) · Value in Health · doi:10.1016/j.jval.2017.07.002 · Cited by 27
Measurement properties of the musculoskeletal health questionnaire (MSK-HQ): a between country comparison
Christiansen, D. H., McCray, G., Winding, T. N., Andersen, J. H., Nielsen, K. J., Karstens, S., & Hill, J. C. (2020) · Health and Quality of Life Outcomes · doi:10.1186/s12955-020-01455-4 · Cited by 29
Estimates of Minimally Important Differences ( MID s) for Two Patient‐Reported Outcomes Measurement Information System ( PROMIS ) Computer‐Adaptive Tests in Chronic Pain Patients
Swanholm, E., McDonald, W. C., Makris, U. E., Noe, C. E., & Gatchel, R. J. (2014) · Journal of Applied Biobehavioral Research · doi:10.1111/jabr.12026 · Cited by 27
Effect of Test-Taking Venue and Response Format on Political Knowledge Tests
Shulman, H. C., & Boster, F. J. (2014) · Communication Methods and Measures · doi:10.1080/19312458.2014.937526 · Cited by 27
Improving Patient Satisfaction Through Computer-Based Questionnaires
Smith, M. J., Reiter, M. J., Crist, B. D., Schultz, L. G., & Choma, T. J. (2015) · Orthopedics · doi:10.3928/01477447-20151218-07 · Cited by 27
Cross-cultural adaptation and multi-centric validation of the Italian version of the Achilles tendon Total Rupture Score (ATRS)
Vascellari, A., Spennacchio, P., Combi, A., Grassi, A., Patella, S., Bisicchia, S., Canata, G. L., & Zaffagnini, S. (2016) · Knee Surgery Sports Traumatology Arthroscopy · doi:10.1007/s00167-016-4152-8 · Cited by 26
CHILD AND PARENT REPORTED OUTCOME MEASURES: A SCOPING REPORT FOCUSING ON FEASIBILITY FOR ROUTINE USE IN THE NHS
Gibbons, E. J., Morris, C., & Fitzpatrick, R. M. (2009) · Cited by 27
Incorporating Patient-Reported Outcome Measures into Breast Surgical Oncology: Advancing Toward Value-Based Care
Lagendijk, M., Mittendorf, E. A., King, T. A., Sidey-Gibbons, C. J., Pusic, A. L., & Dominici, L. S. (2019) · The Oncologist · doi:10.1634/theoncologist.2019-0355 · Cited by 27
Validation of a new hand-held electronic appetite rating system against the pen and paper method
Almiron‐Roig, E., Green, H., Virgili, R., Aeschlimann, J., Moser, M., & Erkner, A. (2009) · Appetite · doi:10.1016/j.appet.2009.09.014 · Cited by 27
Applying PRO Reference Values to Communicate Clinically Relevant Information at the Point-of-care
Jensen, R. E., & Bjørner, J. B. (2019) · Medical Care · doi:10.1097/mlr.0000000000001113 · Cited by 26
Paper-Based Versus Web-Based Versions of Self-Administered Questionnaires, Including Food-Frequency Questionnaires: Prospective Cohort Study
Zazpe, I., Santiago, S., de la Fuente‐Arrillaga, C., Núñez-Córdoba, J. M., Bes‐Rastrollo, M., & Martínez‐González, M. Á. (2019) · JMIR Public Health and Surveillance · doi:10.2196/11997 · Cited by 27
Mean scores for self-report surveys completed using paper-and-pencil and computers: A meta-analytic test of equivalence
Weigold, A., Weigold, I. K., & Natera, S. N. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.04.038 · Cited by 26
Electronic capturing of patient-reported outcome measures on a touchscreen computer in clinical diabetes practice (the DiaPROM trial): a feasibility study
Hernar, I., Graue, M., Richards, D. A., Strandberg, R. B., Nilsen, R. M., Tell, G. S., & Haugstvedt, A. (2019) · Pilot and Feasibility Studies · doi:10.1186/s40814-019-0419-4 · Cited by 26
Activity Measures in Pediatric Athletes: A Comparison of the Hospital for Special Surgery Pediatric Functional Activity Brief Scale and Tegner Activity Level Scale
Wagner, K. J., Sabatino, M. J., Zynda, A. J., Gans, C. V., Chung, J. S., Miller, S. M., Wilson, P. L., & Ellis, H. B. (2020) · The American Journal of Sports Medicine · doi:10.1177/0363546520904009 · Cited by 26
Mother’s Fruit Preferences and Consumption Support Similar Attitudes and Behaviors in Their Children
Groele, B., Głąbska, D., Gutkowska, K., & Guzek, D. (2018) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph15122833 · Cited by 25
Challenges for Child and Adolescent Psychiatric Research in the Era of COVID-19
Aman, M. G., & Pearson, D. A. (2020) · Journal of Child and Adolescent Psychopharmacology · doi:10.1089/cap.2020.0081 · Cited by 25
Applying multiverse analysis to experience sampling data: Investigating whether preprocessing choices affect robustness of conclusions
Weermeijer, J. D. M., Lafit, G., Kiekens, G., Wampers, M., Eisele, G. V., Kasanova, Z., Vaessen, T., Kuppens, P., & Myin‐Germeys, I. (2022) · Behavior Research Methods · doi:10.3758/s13428-021-01777-1 · Cited by 24
Digital Follow-Up and the Perspective of Patient-Centered Care in Oncology: What’s the PROblem?
Giordano, F. A., Welzel, G., Siefert, V., Jahnke, L., Ganslandt, T., Wenz, F. K., Grosu, A., Heinemann, F. E., & Nicolay, N. H. (2018) · Oncology · doi:10.1159/000495294 · Cited by 26
Psychometric Validation of the Role Function Restrictive Domain of the Migraine Specific Quality‐of‐Life Questionnaire Version 2.1 Electronic Patient‐Reported Outcome in Patients With Episodic and Chronic Migraine
Speck, R. M., Shalhoub, H., Wyrwich, K. W., Yu, R., Ayer, D. W., Ford, J. H., Bush, E. N., & Lipton, R. B. (2019) · Headache The Journal of Head and Face Pain · doi:10.1111/head.13497 · Cited by 24
Paper and electronic versions of HM-PRO, a novel patient-reported outcome measure for hematology: an equivalence study
Goswami, P., Olíva, E. N., Ionova, T., Else, R., Kell, J., Fielding, A. K., Jennings, D. M., Καρακάντζα, Μ., Al‐Ismail, S., Lyness, J., et al. (2019) · Journal of Comparative Effectiveness Research · doi:10.2217/cer-2018-0108 · Cited by 24
Comparison of In-Situ and Retrospective Self-Reports on Assessing Hearing Aid Outcomes
Wu, Y.-H., Stangl, E., Chipara, O., Gudjonsdottir, A., Oleson, J., & Bentler, R. A. (2020) · Journal of the American Academy of Audiology · doi:10.1055/s-0040-1719133 · Cited by 25
Outcome Assessment via Handheld Computer in Community Mental Health: Consumer Satisfaction and Reliability
Goldstein, L. A., Gibbons, M. B. C., Thompson, S. M., Scott, K., Heintz, L., Green, P. A., Thompson, D., & Crits-Christoph, P. F. (2010) · The Journal of Behavioral Health Services & Research · doi:10.1007/s11414-010-9229-4 · Cited by 24
Psychometric validation of the French version of the Supportive Care Needs Survey for Partners and Caregivers of cancer patients
Baudry, A., Anota, A., Bonnetain, F., Mariette, C., & Christophe, V. (2018) · European Journal of Cancer Care · doi:10.1111/ecc.12896 · Cited by 24
Patient-reported outcome measurement in drug discovery: a tool to improve accuracy and completeness of efficacy and safety data
Basch, E., & Dueck, A. C. (2016) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2016.1193148 · Cited by 24
A randomized trial found online questionnaires supplemented by postal reminders generated a cost-effective and generalizable sample but don’t forget the reminders
Loban, A., Mandefield, L., Hind, D., & Bradburn, M. J. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.003 · Cited by 23
A phase III randomized-controlled, single-blind trial to improve quality of life with stereotactic body radiotherapy for patients with painful bone metastases (ROBOMET)
Mercier, C., Dirix, P. R., Ost, P., Billiet, C., Joye, I., Vermeulen, P. B., Lievens, Y. N., & Verellen, D. (2019) · BMC Cancer · doi:10.1186/s12885-019-6097-z · Cited by 23
Evaluation of point-of-care PRO assessment in clinic settings: integration, parallel-forms reliability, and patient acceptability of electronic QOL measures during clinic visits
Sharma, P., Dunn, R. L., Wei, J. T., Montie, J. E., & Gilbert, S. M. (2015) · Quality of Life Research · doi:10.1007/s11136-015-1113-5 · Cited by 23
Assessing the Effects of Participant Preference and Demographics in the Usage of Web-based Survey Questionnaires by Women Attending Screening Mammography in British Columbia
Mlikotic, R., Parker, B., & Rajapakshe, R. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5068 · Cited by 23
Estudio comparativo de los formatos en lápiz y papel y electrónicos de los cuestionarios GHQ-12, WHO-5 y PHQ-9
Barrigón, M. L., Rico-Romano, A. M., Ruiz-Gómez, M., Delgado‐Gómez, D., Barahona, I., Aroca, F., & Baca‐García, E. (2017) · Revista de Psiquiatría y Salud Mental · doi:10.1016/j.rpsm.2016.12.002 · Cited by 24
Electronic Version of the EQ-5D Quality-of-Life Questionnaire: Adaptation to a Brazilian Population Sample
Bagattini, Â. M., Camey, S. A., Miguel, S. R. P. D. S., Andrade, M. V., Noronha, K., Teixeira, M. A. D. C., Lima, A. F. B. D. S., Santos, M., Polanczyk, C. A., & Cruz, L. N. (2018) · Value in Health Regional Issues · doi:10.1016/j.vhri.2017.11.002 · Cited by 23

Authors: Gwaltney, C. J., Shields, A. L., & Shiffman, S.
Affiliations: Brown University (US)
East Tennessee State University (US)
University of Pittsburgh (US)
ORCID: Gwaltney, Chad J.
Shields, Alan L.
Shiffman, Saul
Venue: Value in Health
Publisher: Elsevier BV
Published: 2008-03
Volume: 11
Issue: 2
Pages: 322-333
ISSN: 1098-3015 (ISSN-L)
DOI: 10.1111/j.1524-4733.2007.00231.x
OpenAlex: W2146545407
Cited by: 623
References: 76
Citations by year: 2026: 18 · 2025: 23 · 2024: 20 · 2023: 43 · 2022: 36 · 2021: 42 · 2020: 49 · 2019: 45 · 2018: 56 · 2017: 44 · 2016: 51 · 2015: 35 · 2014: 44 · 2013: 28 · 2012: 25
FWCI: 5.19
Type: journal article
Language: en
Open access: bronze
doi.org
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
www.elsevier.com/open-access/userlicense/1.0/ (vor, 1963 d)
Locations: Value in Health (publishedVersion)
PubMed (publishedVersion)
Digital Commons - East Tennessee State University (East Tennessee State University) (submittedVersion)
Topics: Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Keywords: patient-reported outcomes, paper-and-pencil administration, mode effects, measurement invariance, test-retest reliability, clinical trials
MeSH terms: Data Collection, Humans, Statistics as Topic, Patient Satisfaction, Outcome Assessment, Health Care, Computers, Handheld

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Equivalence of Electronic and Paper-and-Pencil Administration of Patient-Reported Outcome Measures: A Meta-Analytic Review

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks