The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Internet Versus Mailed Questionnaires: A Randomized Comparison
Open accessAbstract
BACKGROUND: The use of Internet-based questionnaires for collection of data to evaluate patient education and other interventions has increased in recent years. Many self-report instruments have been validated using paper-and-pencil versions, but we cannot assume that the psychometric properties of an Internet-based version will be identical. OBJECTIVES: To look at similarities and differences between the Internet versions and the paper-and-pencil versions of 16 existing self-report instruments useful in evaluation of patient interventions. METHODS: Participants were recruited via the Internet and volunteered to participate (N=397), after which they were randomly assigned to fill out questionnaires online or via mailed paper-and-pencil versions. The self-report instruments measured were overall health, health distress, practice mental stress management, Health Assessment Questionnaire (HAQ) disability, illness intrusiveness, activity limitations, visual numeric for pain, visual numeric for shortness of breath, visual numeric for fatigue, self-efficacy for managing disease, aerobic exercise, stretching and strengthening exercise, visits to MD, hospitalizations, hospital days, and emergency room visits. Means, ranges, and confidence intervals are given for each instrument within each type of questionnaire. The results from the two questionnaires were compared using both parametric and non-parametric tests. Reliability tests were given for multi-item instruments. A separate sample (N=30) filled out identical questionnaires over the Internet within a few days and correlations were used to assess test-retest reliability. RESULTS: Out of 16 instruments, none showed significant differences when the appropriate tests were used. Construct reliability was similar within each type of questionnaire, and Internet test-retest reliability was high. Internet questionnaires required less follow-up to achieve a slightly (non-significant) higher completion rate compared to mailed questionnaires. CONCLUSIONS: Among a convenience sample recruited via the Internet, results from those randomly assigned to Internet participation were at least as good as, if not better than, among those assigned mailed questionnaires, with less recruitment effort required. The instruments administered via the Internet appear to be reliable, and to be answered similarly to the way they are answered when they are administered via traditional mailed paper questionnaires.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“All measures were administered online, a method which has been shown to be reliable and as valid as traditional paper-and-pencil administration [40,41].”
From Acceptability, effectiveness, and cost-effectiveness of internet-based exposure treatment for irritable bowel syndrome in a clinical sample: A randomized controlled trial (2011) · cites this paper as [41]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Measurement of patient outcome in arthritis
Fries, J. F., Spitz, P. W., Kraines, R. G., & Holman, H. R. (1980) · Arthritis & Rheumatism · doi:10.1002/art.1780230202 · Cited by 4,404 · cited 1×
“• Health Assessment Instrument measures disability and is used in the National Health Survey (20 items) (Fries et al, 1980) [15].”
MethodGosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004) · American Psychologist · doi:10.1037/0003-066x.59.2.93 · Cited by 2,932 · cited 1×
“However, Gosling et al [19] have argued that Internet samples may actually be more representative than traditional samples.”
BackgroundOutcome Measures for Health Education and Other Health Care Interventions
Lorig, K., Stewart, A., Ritter, P., González, V., Laurent>, D., & Lynch, J. (1996) · doi:10.4135/9781452232966 · Cited by 1,132 · cited 8×
“• Self-reported exercise measures minutes/week of aerobic (5 items) and minutes/week stretching and strengthening exercise (1 item) (Lorig et al, 1996) [12].”
BackgroundMethodInfluentialSelf-citationSelf-rated health and mortality in the NHANES I Epidemiologic Follow-up Study
Idler, E. L., & Angel, R. J. (1990) · American Journal of Public Health · doi:10.2105/ajph.80.4.446 · Cited by 864 · cited 1×
“This item comes from the National Health Survey and has been found to be predictive of future health status (Idler & Angel, 1990) [14].”
MethodSocial desirability, anonymity, and internet-based questionnaires
Joinson, A. (1999) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03200723 · Cited by 818 · cited 2×
“Joinson (1999) reported that both anonymity and Web usage (compared to paper-and-pencil) resulted in lower scores on a social desirability measure [8].”
BackgroundUsing the Internet for Surveys and Health Research
Eysenbach, G., & Wyatt, J. (2002) · Journal of Medical Internet Research · doi:10.2196/jmir.4.2.e13 · Cited by 790 · cited 2×
“A recent overview entitled “Using the Internet for Surveys and Health Research” (Eysenbach & Wyatt 2002, [10]), barely touched on instruments (referring readers to the Quality of Life Instruments Database at Quality of Life Instruments database [11]) and did not discuss validity or reliability of…”
BackgroundUsing the Internet for psychological research: Personality testing on the World Wide Web
Buchanan, T., & Smith, J. L. (1999) · British Journal of Psychology · doi:10.1348/000712699161189 · Cited by 596 · cited 1×
“Buchanan and Smith (1999) [5] compared a Web-based personality assessment to a paper-and-pencil version, and using confirmatory factor analyses, found similar psychometric properties in the two tests.”
MethodSelf-reports of health care utilization compared to provider records
Ritter, P. L., Stewart, A. L., Kaymaz, H., Sobel, D. S., Block, D. A., & Lorig, K. R. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00261-4 · Cited by 401 · cited 1×
“• Health care utilization (MD visits, hospitalization, hospital days, ER visits) (Lorig et al, 1996; Ritter et al, 2001) [12,18].”
Self-citationThe Use of the Internet in Psychological Research: Comparison of Online and Offline Questionnaires
Riva, G., Teruzzi, T., & Anolli, L. (2003) · CyberPsychology & Behavior · doi:10.1089/109493103321167983 · Cited by 304 · cited 2×
“Joinson (1999) reported that both anonymity and Web usage (compared to paper-and-pencil) resulted in lower scores on a social desirability measure [8].”
BackgroundWeb-based administration of a personality questionnaire: Comparison with traditional methods
Davis, R. N. (1999) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03200737 · Cited by 226 · cited 2×
“Davis (1999) [6] also compared Web and paper-and-pencil versions of a personality measure (rumination), and concluded that “findings from Web-based questionnaire research are comparable with results obtained using standard procedures.””
Result
Research works printed but not linked: 1
- Lin, C, 2003, Proc AMIA Symp, 911
Cited by
Sorted by citations
Eysenbach, G. (2004) · Journal of Medical Internet Research · doi:10.2196/jmir.6.3.e34 · Cited by 6,480
“In this issue of the Journal of Medical Internet Research we publish two methodological studies exploring the characteristics of Web-based surveys compared to mail-based surveys [1,2].”
BackgroundA guide for the design and conduct of self-administered surveys of clinicians
Burns, K. E. A., Duffett, M., Kho, M. E., Meade, M., Adhikari, N. K. J., Sinuff, T., Cook, D. J., & for the ACCADEMY Group (2008) · Canadian Medical Association Journal · doi:10.1503/cmaj.080372 · Cited by 1,357
Comparing Response Rates from Web and Mail Surveys: A Meta-Analysis
Shih, T.-H., & Fan, X. (2008) · Field Methods · doi:10.1177/1525822x08317085 · Cited by 1,159
The QCAE: a questionnaire of cognitive and affective empathy
Reniers, R. L. E. P., Corcoran, R., Drake, R., Shryane, N. M., & Völlm, B. A. (2011) · Journal of Personality Assessment · doi:10.1080/00223891.2010.528484 · Cited by 961
“However, research has shown that questionnaires administered via the Internet are reliable and are answered in the same way as when they are administered via traditional mailed paper questionnaires ( Ritter, Lorig, Laurent, & Matthews, 2004 ; Riva, Teruzzi, & Anolli, 2003 ).”
BackgroundBiddiss, E. A., & Chau, T. T. (2007) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/phm.0b013e3181587f6c · Cited by 647
“) have been demonstrated.(22,25) However, it is important to recognize the possible sampling biases inherent to the results presented herein.”
BackgroundPorst, H., Montorsi, F., Rosen, R. C., Gaynor, L., Grupe, S., & Alexander, J. R. (2006) · European Urology · doi:10.1016/j.eururo.2006.07.004 · Cited by 587
“A recent study compared the response rates of surgeons to a questionnaire sent via e-mail or standard mail and reported a significantly lower response rate with e-mail solicitation [18]; however, the responses to questions obtained using the two survey methods were not different.”
ResultWeb-based Questionnaires: The Future in Epidemiology?
van Gelder, M. M. H. J., Bretveld, R. W., & Roeleveld, N. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwq291 · Cited by 581
Consumer design priorities for upper limb prosthetics
Biddiss, E., Beaton, D., & Chau, T. (2007) · Disability and Rehabilitation: Assistive Technology · doi:10.1080/17483100701714733 · Cited by 519
“With some exceptions [44], response rates are generally lower for emailed surveys than for conventional methods [29].”
BackgroundRepresentativeness of the Patient-Reported Outcomes Measurement Information System Internet panel
Liu, H., Cella, D. F., Gershon, R. C., Shen, J., Morales, L. S., Riley, W. T., & Hays, R. D. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.11.021 · Cited by 454
Kongsved, S. M., Basnov, M., Holm-Christensen, K., & Hjollund, N. H. (2007) · Journal of Medical Internet Research · doi:10.2196/jmir.9.3.e25 · Cited by 395
“Response rates to Internet questionnaires reported in the literature vary a lot between studies [1,6-14].”
BackgroundWilliams, R. J., Volberg, R. A., & Stevens, R. M. G. (2012) · Open ULeth Scholarship (OPUS) (University of Lethbridge) · Cited by 400
van de Poll-Franse, L. V., Horevoorts, N. J. E., van Eenbergen, M. C. H. J., Denollet, J., Roukema, J. A., Aaronson, N. K., Vingerhoets, A. J. J. M., Coebergh, J. W. W., Vries, J. D., Essink‐Bot, M., et al. (2011) · European Journal of Cancer · doi:10.1016/j.ejca.2011.04.034 · Cited by 399
The mental health of university students in the United Kingdom
Macaskill, A. (2013) · British Journal of Guidance and Counselling · doi:10.1080/03069885.2012.743110 · Cited by 335
“…of measures and have found no significant differences in terms of internal consistency, test-retest reliability across the two modalities (Buchanan & Smith, 1999; Denscombe, 2006; Herrero & Menese, 2006; Ritter, Lorig, Laurent, & Mathews, 2004: Vallejo, Jordan, Diaz, Comech, & Ortega, 2007).”
Burnout and Depression: Two Entities or One?
Schonfeld, I. S., & Bianchi, R. (2016) · Journal of Clinical Psychology · doi:10.1002/jclp.22229 · Cited by 345
Fish, R. A., McGuire, B., Hogan, M., Morrison, T. G., & Stewart, I. (2010) · Pain · doi:10.1016/j.pain.2009.12.016 · Cited by 306
“, [8,13,23,43]) and a need to validate measures for use in this context [6,7].”
BackgroundThe Use of the Internet in Psychological Research: Comparison of Online and Offline Questionnaires
Riva, G., Teruzzi, T., & Anolli, L. (2003) · CyberPsychology & Behavior · doi:10.1089/109493103321167983 · Cited by 304
Devins, G. M. (2010) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2009.05.006 · Cited by 294
“, via the internet) without compromising reliability or biasing results [99].”
BackgroundLukens, C. T., & Linscheid, T. R. (2008) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-007-0401-5 · Cited by 278
Ljótsson, B., Falk, L., Vesterlund, A. W., Hedman, E., Lindfors, P., Rück, C., Hursti, T., Andréewitch, S., Jansson, L., Lindefors, N., et al. (2010) · Behaviour Research and Therapy · doi:10.1016/j.brat.2010.03.003 · Cited by 270
Feasibility of Using Web-based Questionnaires in Large Population-based Epidemiological Studies
Ekman, A., Dickman, P. W., Klint, Å., Weiderpass, E., & Litton, J. (2006) · European Journal of Epidemiology · doi:10.1007/s10654-005-6030-4 · Cited by 225
Ljótsson, B., Hedman, E., Andersson, E., Hesser, H., Lindfors, P., Hursti, T., Rydh, S., Rück, C., Lindefors, N., & Andersson, G. (2011) · American Journal of Gastroenterology · doi:10.1038/ajg.2011.139 · Cited by 218
“Data collection All measures were administered online, a method that has been shown to be reliable and produces results very similar to traditional paper-and-pencil administration (”
MethodInternet-based cognitive-behavioural therapy for severe health anxiety: randomised controlled trial
Hedman, E., Andersson, G., Andersson, E., Ljótsson, B., Rück, C., Asmundson, G. J. G., & Lindefors, N. (2011) · British Journal of Psychiatry · doi:10.1192/bjp.bp.110.086843 · Cited by 212
Vallejo, M., Jordán, C., Díaz, M. I., Comeche, M. I., & Ortega, J. (2007) · Journal of Medical Internet Research · doi:10.2196/jmir.9.1.e2 · Cited by 241
“In that sense, a growing number of computerized or online questionnaires related to areas such as panic/agoraphobia [3], youth independence living [7], aggression and impulsivity [8], quality of life in diabetes [9], and a battery of 16 other health-related questionnaires [10] have already been studied.”
BackgroundThe Danish National Health Survey 2010. Study design and respondent characteristics
Christensen, A. I., Ekholm, O., Glümer, C., Andreasen, A. H., Hvidberg, M. F., Kristensen, P. L., Larsen, F. B., Ortiz, B., & Juel, K. (2012) · Scandinavian Journal of Public Health · doi:10.1177/1403494812451412 · Cited by 211
“Despite the methodological concerns associated with it, a mixed-mode approach has many benefits which are largely seen to outweigh its potential challenges [20,22,26].”
BackgroundWhitehead, L. (2007) · Social Science & Medicine · doi:10.1016/j.socscimed.2007.03.005 · Cited by 221
Comparative symptomatology of burnout and depression
Bianchi, R., Boffy, C., Hingray, C., Truchot, D., & Laurent, É. (2013) · Journal of Health Psychology · doi:10.1177/1359105313481079 · Cited by 208
“To form the BOG and the CTR, we emailed the MBI (Dion and Tessier, 1994; Maslach and Jackson, 1986) to several hundred middle and high schools in France by using academic databases—past research has shown that Internet questionnaires are as valid and reliable as traditional, paper-and-pencil questionnaires (Gosling et…”
BackgroundMethodNormative data for the EORTC QLQ-C30 and EORTC-sexuality items in the general Dutch population
van de Poll-Franse, L. V., Mols, F., Gundy, C. M., Creutzberg, C. L., Nout, R. A., Leeuw, I. M. V., TAPHOORN, M. J. B., & Aaronson, N. K. (2010) · European Journal of Cancer · doi:10.1016/j.ejca.2010.11.004 · Cited by 209
Nachtegaal, J., Smit, J. H., Smits, C., Bezemer, P. D., van Beek, J. H. M., Festen, J. Μ., & Kramer, S. E. (2009) · Ear and Hearing · doi:10.1097/aud.0b013e31819c6e01 · Cited by 202
“Overall, it can be stated that there is satisfactory evidence for the reliability, validity, and feasibility of online questionnaires (Hallam et al. 2006; Kongsved et al. 2007; Ritter et al. 2004; Vallejo et al. 2007).”
BackgroundKhazaal, Y., van Singer, M., Chatton, A., Achab, S., Zullino, D., Rothen, S., Khan, R., Billieux, J., & Thorens, G. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.2759 · Cited by 201
“Some studies suggest that the quality of the data provided by Internet-based surveys is at least as good as in those collected by traditional paper-and-pencil methods on self-selected samples [6-8].”
BackgroundRasch model analysis of the Depression, Anxiety and Stress Scales (DASS)
Shea, T. L., Tennant, A., & Pallant, J. (2009) · BMC Psychiatry · doi:10.1186/1471-244x-9-21 · Cited by 203
“Some studies investigating the impact of scale administration methods have reported similar psychometric properties for pencil-and-paper surveys and their web-based counterparts [41-43].”
Is burnout a depressive disorder? A reexamination with special focus on atypical depression.
Bianchi, R., Schonfeld, I. S., & Laurent, É. (2014) · International Journal of Stress Management · doi:10.1037/a0037906 · Cited by 203
Questionnaire Survey Nonresponse: A Comparison of Postal Mail and Internet Surveys
Hoonakker, P., & Carayon, P. (2009) · International Journal of Human-Computer Interaction · doi:10.1080/10447310902864951 · Cited by 173
“Based on their review of the literature Ritter et al. (2004, p. 29) concluded, “although progress is being made, there remains a need to evaluate Internet versions of most of the health-behavior and outcome instruments useful to researchers evaluating patient intervention programs.””
SoftGIS as a Bridge-Builder in Collaborative Urban Planning
Kahila, M., & Kyttä, M. (2009) · The Geojournal library · doi:10.1007/978-1-4020-8952-7_19 · Cited by 168
“On the whole, reviews and comparisons between web and traditional surveys show significant differences in neither the characteristics of the respondents (Gosling et al. 2004) nor in the results of the surveys ( Rittel et al. 2004 ).”
ResultItem non‐response rates: a comparison of online and paper questionnaires
Denscombe, M. (2008) · International Journal of Social Research Methodology · doi:10.1080/13645570802054706 · Cited by 163
The Emotion Regulation Questionnaire: Validation of the ERQ-9 in two community samples.
Spaapen, D. L., Waters, F., Brümmer, L., Stopa, L. L., & Bucks, R. S. (2013) · Psychological Assessment · doi:10.1037/a0034474 · Cited by 177
Flanigan, T. S., McFarlane, E., & Cook, S. (2008) · Cited by 160
The Ethics of Internet Research
Eynon, R., Fry, J., & Schroeder, R. (2008) · SAGE Publications Ltd eBooks · doi:10.4135/9780857020055.n2 · Cited by 160
Survey Response Rates and Survey Administration in Counseling and Clinical Psychology
Horn, P. S. V., Green, K. E., & Martinussen, M. (2008) · Educational and Psychological Measurement · doi:10.1177/0013164408324462 · Cited by 156
Heitmann, K., Nordeng, H., Havnen, G., Solheimsnes, A., & Holst, L. (2017) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-017-1249-0 · Cited by 112
Fischer, A., Schröder, J., Vettorazzi, E., Wolf, O. T., Pöttgen, J., Lau, S., Heesen, C., Moritz, S., & Gold, S. M. (2015) · The Lancet Psychiatry · doi:10.1016/s2215-0366(14)00049-2 · Cited by 143
Liang, J., Xian, D., Liu, X., Fu, J., Zhang, X., Tang, B., & Lei, J. (2018) · JMIR mhealth and uhealth · doi:10.2196/11066 · Cited by 148
“They have been widely accepted by clinical medical informatics and consumer health informatics researchers and practitioners, and they are also widely applied in electronic health records, computerized physician order entry, and health applications [36-39].”
BackgroundRelationship between meditative practice and self-reported mindfulness: the MINDSENS composite index
Soler, J., Cebolla, A., Feliu-Soler, A., Demarzo, M. M. P., Pascual, J. C., Baños, R., & García-Campayo, J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0086622 · Cited by 138
“500) and the existence of studies that confirm the reliability of data obtained from this source [43], these samples are probably more heterogeneous and biased–due to the high non-response rate–than those obtained by other, more traditional methods.”
ResultNew times, new needs; e-epidemiology
Ekman, A., & Litton, J. (2007) · European Journal of Epidemiology · doi:10.1007/s10654-007-9119-0 · Cited by 139
Damholdt, M. F., Mehlsen, M. Y., O’Toole, M. S., Andreasen, R., Pedersen, A., & Zachariae, R. H. (2016) · Psycho-Oncology · doi:10.1002/pon.4058 · Cited by 138
“Generally, previous studies conclude that there are no differences between online and paper-and-pencil questionnaires on psychometric properties [38], particularly when sampling is controlled [39].”
BackgroundLjótsson, B., Andersson, G., Andersson, E., Hedman, E., Lindfors, P., Andréewitch, S., Rück, C., & Lindefors, N. (2011) · BMC Gastroenterology · doi:10.1186/1471-230x-11-110 · Cited by 132
“All measures were administered online, a method which has been shown to be reliable and as valid as traditional paper-and-pencil administration [40,41].”
MethodCrutzen, R., & Göritz, A. S. (2010) · BMC Public Health · doi:10.1186/1471-2458-10-720 · Cited by 137
“, in which web-based assessment is compared with, for example, paper-and-pencil assessment [26,27] or with telephone interviewing [28]) generally have relied on one of three different designs: randomization after recruitment (true experimental design), randomization before recruitment (where there may be differences i…”
MethodLinke, S., Murray, E., Butler, C., & Wallace, P. (2007) · Journal of Medical Internet Research · doi:10.2196/jmir.9.2.e10 · Cited by 127
“It is difficult to know how truthful they were in completing these questionnaires, but there is some evidence to support the suggestion that responses to online questionnaires are comparable to traditional paper-based versions [42,43].”
BackgroundPrevalence of Sexual Dysfunction and Impact of Contraception in Female German Medical Students
Wallwiener, C. W., Wallwiener, L.-M., Seeger, H., Mück, A. O., Bitzer, J., & Wallwiener, M. (2010) · The Journal of Sexual Medicine · doi:10.1111/j.1743-6109.2010.01742.x · Cited by 132
Scott, D. M., Berg, M. J., Tolhurst, B. A., Chauvenet, A. L. M., Smith, G. C., Neaves, K., Lochhead, J., & Baker, P. J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0099059 · Cited by 122
“In the modern era, web-based questionnaires potentially represent a more efficient approach for conducting large-scale surveys with significant logistical and cost advantages [45] [46].”
Hunter, L. (2012) · Nurse Researcher · doi:10.7748/nr2012.09.20.1.11.c9303 · Cited by 121
Participant recruitment and data collection through Facebook: the role of personality factors
Rife, S. C., Cate, K. L., Kosiński, M., & Stillwell, D. J. (2014) · International Journal of Social Research Methodology · doi:10.1080/13645579.2014.957069 · Cited by 135
The Role of Online Support Communities
Idriss, S. Z., Kvedar, J. C., & Watson, A. J. (2009) · Archives of Dermatology · doi:10.1001/archdermatol.2008.529 · Cited by 119
Mode of administration does not cause bias in patient-reported outcome results: a meta-analysis
Rutherford, C., Costa, D. S. J., Mercieca‐Bebber, R., Rice, H. E., Gabb, L., & King, M. T. (2015) · Quality of Life Research · doi:10.1007/s11136-015-1110-8 · Cited by 115
Moritz, S., Jahns, A. K., Schröder, J., Berger, T., Lincoln, T. M., Klein, J. P., & Göritz, A. S. (2016) · Journal of Affective Disorders · doi:10.1016/j.jad.2015.11.027 · Cited by 115
Hernández, G., Garín, O., Dima, A. L., Pont, À., Marti-Pastor, M., Alonso, J., Ganse, E. H. V., Laforest, L., de Bruin, M., Mayoral, K., et al. (2018) · Journal of Medical Internet Research · doi:10.2196/10178 · Cited by 114
“The internet era has led to implementing Web-based surveys to take advantage of the known benefits such as completeness [41,42], low expenses [43], and better data management.”
Moritz, S., Jelinek, L., Hauschildt, M., & Naber, D. (2010) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2010.12.2/smoritz · Cited by 113
Ljótsson, B., Hedman, E., Lindfors, P., Hursti, T., Lindefors, N., Andersson, G., & Rück, C. (2011) · Behaviour Research and Therapy · doi:10.1016/j.brat.2010.10.006 · Cited by 113
Martins, Y., Lederman, R. I., Lowenstein, C. L., Joffe, S., Neville, B. A., Hastings, B. T., & Abel, G. A. (2012) · British Journal of Cancer · doi:10.1038/bjc.2012.28 · Cited by 112
Evaluating Patients’ Experiences with Individual Physicians
Rodríguez, H. P., von Glahn, T., Rogers, W. H., Chang, H., Fanjiang, G., & Safran, D. G. (2006) · Medical Care · doi:10.1097/01.mlr.0000196961.00933.8e · Cited by 112
Experiences of Fecal Incontinence in People with Inflammatory Bowel Disease
Dibley, L. B., & Norton, C. S. (2013) · Inflammatory Bowel Diseases · doi:10.1097/mib.0b013e318281327f · Cited by 109
Development of a dual school climate and school identification measure–student (SCASIM-St)
Lee, E., Reynolds, K. J., Subašić, E., Bromhead, D., Lin, H., Marinov, V., & Smithson, M. J. (2017) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2017.01.003 · Cited by 122
Raat, H., Mangunkusumo, R. T., Landgraf, J. M., Kloek, G. C., & Brug, J. (2007) · Quality of Life Research · doi:10.1007/s11136-006-9157-1 · Cited by 106
“found no statistically significant score differences between internet and paper modes of administration for 16 health-related measures, but the study was conducted in an opportunity sample retrieved from the internet, which limits its generalizability [23].”
BackgroundGordon, W. V., Shonin, E., Dunn, T. J., García‐Campayo, J., & Griffiths, M. D. (2016) · British Journal of Health Psychology · doi:10.1111/bjhp.12224 · Cited by 103
Zuidgeest, M., Hendriks, M. C. P., Koopman, L., Spreeuwenberg, P. M. M., & Rademakers, J. J. D. J. M. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1241 · Cited by 103
“Other reasons were having no experience with the Internet or not trusting the Internet [31].”
Pregnancy-specific anxiety: which women are highest and what are the alcohol-related risks?
Arch, J. J. (2012) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2012.07.010 · Cited by 115
“[15,32,49]).”
Feasibility of recruiting a birth cohort through the Internet: the experience of the NINFEA cohort
Richiardi, L., Baussano, I., Vizzini, L., Douwes, J. E., Pearce, N. E., & Merletti, F. (2007) · European Journal of Epidemiology · doi:10.1007/s10654-007-9194-2 · Cited by 102
“Recently, some studies have posted surveys on websites, recruiting participants whose contact information was unknown before participation [12–14].”
BackgroundHousing strain, mortgage foreclosure, and health
Cannuscio, C. C., Alley, D. E., Pagán, J. A., Soldo, B. J., Krasny, S., Shardell, M. D., Asch, D. A., & Lipman, T. H. (2011) · Nursing Outlook · doi:10.1016/j.outlook.2011.08.004 · Cited by 98
Impact of the Cost of Cancer Treatment: An Internet-Based Survey
Markman, M., & Luce, R. (2010) · Journal of Oncology Practice · doi:10.1200/jop.091074 · Cited by 97
Horevoorts, N. J. E., Vissers, P. A. J., Mols, F., Thong, M. S. Y., & van de Poll-Franse, L. V. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3741 · Cited by 94
“Several studies also show that reliability of Web-based questionnaires and paper questionnaires is comparable [8-11].”
Emergency Physicians' Perceptions of Health Information Exchange
Shapiro, J. S., Kannry, J. L., Kushniruk, A. W., & Kuperman, G. J. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2507 · Cited by 93
The Length of Responses to Open-Ended Questions
Denscombe, M. (2007) · Social Science Computer Review · doi:10.1177/0894439307309671 · Cited by 86
Marquet, R. L., Bartelds, A. I. M., van Noort, S. P., Koppeschaar, C. E., Paget, J., Schellevis, F., & van der Zee, J. (2006) · BMC Public Health · doi:10.1186/1471-2458-6-242 · Cited by 85
“It has been argued, however, that internet samples may actually be more representative than traditional samples [3].”
BackgroundMcCambridge, J., Mitcheson, L., WINSTOCK, A. R., & Hunt, N. (2005) · Addiction · doi:10.1111/j.1360-0443.2005.001127.x · Cited by 85
A Web-Based Cross-Sectional Epidemiological Survey of Complex Regional Pain Syndrome
Sharma, A. K., Agarwal, S., Broatch, J., & Raja, S. N. (2009) · Regional Anesthesia & Pain Medicine · doi:10.1097/aap.0b013e3181958f90 · Cited by 81
Inquiry into the correlation between burnout and depression.
Schonfeld, I. S., Verkuilen, J., & Bianchi, R. (2019) · Journal of Occupational Health Psychology · doi:10.1037/ocp0000151 · Cited by 86
Whitehead, L. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1593 · Cited by 84
The Danish health and morbidity surveys: study design and participant characteristics
Jensen, H. A. R., Ekholm, O., Davidsen, M., & Christensen, A. I. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0733-9 · Cited by 77
Long-term effects of an internet-based treatment for posttraumatic stress
Knaevelsrud, C., & Maercker, A. (2010) · Cognitive Behaviour Therapy · doi:10.1080/16506070902999935 · Cited by 76
The Roles of Procedural and Distributive Justice in Logistics Outsourcing Relationships
Hofer, A. R., Knemeyer, A. M., & Murphy, P. R. (2012) · Journal of Business Logistics · doi:10.1111/j.2158-1592.2012.01052.x · Cited by 78
Burnout is associated with a depressive cognitive style
Bianchi, R., & Schonfeld, I. S. (2016) · Personality and Individual Differences · doi:10.1016/j.paid.2016.01.008 · Cited by 89
Varghese, N. E., Sabat, I., Neumann‐Böhme, S., Schreyögg, J., Stargardt, T., Torbica, A., van Exel, J. N. A., Barros, P. P., & Brouwer, W. B. F. (2021) · PLoS ONE · doi:10.1371/journal.pone.0250872 · Cited by 74
“Also, studies show internet based questionnaires to be a “suitable alternative” to more traditional survey methods [27, 28].”
BackgroundThe burden of stroke in the Netherlands: estimating quality of life and costs for 1 year poststroke
van Eeden, M., Heugten, C. M. V., van Mastrigt, G. A. P. G., van Mierlo, M., Visser‐Meily, J. M. A., & Evers, S. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008220 · Cited by 73
“Previous research had found no differences between online questionnaires and questionnaires administered on paper.(19) All questions administered on paper were checked by the research assistant to prevent missing data.”
BackgroundHardré, P. L., Crowson, H. M., Xie, K., & Ly, C. (2006) · British Journal of Educational Technology · doi:10.1111/j.1467-8535.2006.00591.x · Cited by 71
Use of a Web-based Questionnaire in the Black Women's Health Study
Russell, C. W., Boggs, D. A., Palmer, J. R., & Rosenberg, L. A. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwq310 · Cited by 71
Web-based survey attracted age-biased sample with more severe illness than paper-based survey
Klovning, A., Sandvik, H., & Hunskaar, S. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.10.015 · Cited by 70
“The instruments administered through the Internet appear to be reliable, and are to be answered similarly to the way they are answered when administered through traditional postal paper questionnaires [27].”
MethodHughes, C., Davoodi‐Semiromi, Y., Colee, J., Culpepper, T., Dahl, W. J., Mai, V., Christman, M. C., & Langkamp‐Henken, B. (2011) · American Journal of Clinical Nutrition · doi:10.3945/ajcn.111.014126 · Cited by 70
“Ritter and colleagues tested the reliability of 16 different health-related instruments administered online compared to the traditional mailed paper questionnaires (124).”
Moritz, S., Peters, M. J. V., Larøi, F., & Lincoln, T. M. (2010) · Cognitive Neuropsychiatry · doi:10.1080/13546801003783508 · Cited by 69
“In addition, mounting evidence suggests that results obtained over the Internet are comparable in reliability with studies using more conventional measures (Jones, Fernyhough, de-Wit, & Meins, 2008 ; Ritter, Lorig, Laurent, & Matthews, 2004 ; Riva, Teruzzi, & Anolli, 2003 ), even in studies including severely disturbe…”
ResultWestermann, S., Kesting, M.-L., & Lincoln, T. M. (2011) · Behavior Therapy · doi:10.1016/j.beth.2011.07.005 · Cited by 73
Holmes, S. (2008) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2008.08.004 · Cited by 68
“The effect of the Internet on response rates is, however, equivocal and research to date reports inconsistent findings; some have reported moderate to high response rates ranging between 45% and 96% (VanDenKerkhof et al., 2004; Kypri and McAnally, 2004; Leece et al., 2004; McGee and Kypri, 2004) while others have repo…”
ResultInfluentialBloem, B. R., & Stocchi, F. (2011) · European Journal of Neurology · doi:10.1111/j.1468-1331.2011.03532.x · Cited by 68
“In addition, online questionnaires achieve slightly higher completion rates than mailed questionnaires [49].”
Montero-Marín, J., Gaete, J., Demarzo, M., Rodero, B., Lopez, L. C. S., & García-Campayo, J. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01281 · Cited by 66
Characteristics of Gamblers Using a National Online Counselling Service for Problem Gambling
Rodda, S. N., & Lubman, D. I. (2013) · Journal of Gambling Studies · doi:10.1007/s10899-012-9352-7 · Cited by 67
“Self-report instruments developed for paper and pencil administration also appear to maintain their psychometric properties when delivered online (Ritter et al. 2004).”
BackgroundJanda, M., DiSipio, T., Hurst, C. P., Cella, D. F., & Newman, B. M. (2008) · Psycho-Oncology · doi:10.1002/pon.1428 · Cited by 67
Young, N. L., Varni, J. W., Snider, L., McCormick, A., Sawatzky, B., Scott, M., King, G. A., Hetherington, R., Sears, E., & Nicholas, D. B. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.06.011 · Cited by 65
“found survey response rates on the Internet to be 13% lower than those on paper [16].”
BackgroundPathways to hydrogen as an energy carrier
Sigfusson, T. I. (2007) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2006.1960 · Cited by 65
Deitz, M. F., Williams, S. L., Rife, S. C., & Cantrell, P. J. (2015) · Violence Against Women · doi:10.1177/1077801215573330 · Cited by 64
Burnout and depression: Label-related stigma, help-seeking, and syndrome overlap
Bianchi, R., Verkuilen, J., Brisson, R., Schonfeld, I. S., & Laurent, É. (2016) · Psychiatry Research · doi:10.1016/j.psychres.2016.08.025 · Cited by 63
“Online questionnaires have been shown to be as reliable and valid as traditional, paper-and-pencil questionnaires (Gosling et al., 2004; Jones et al., 2008; Ritter et al., 2004).”
BackgroundInternet- vs. telephone-administered questionnaires in a randomized trial of smoking cessation
Graham, A. L., Papandonatos, G. D., Bock, B. C., Cobb, N. K., Baskin-sommers, A. R., NIAURA, R. S., & Abrams, D. B. (2006) · Nicotine & Tobacco Research · doi:10.1080/14622200601045367 · Cited by 63
“, 2003), personality (Buchanan & Smith, 1999; Davis, 1999), and a variety of measures related to health status and health behaviors (Ritter et al., 2004).”
Backgroundvan Mierlo, M. L., Heugten, C. M. V., Post, M. W. M., Lindeman, E., de Kort, P. L. M., & Visser‐Meily, J. M. A. (2012) · International Journal of Stroke · doi:10.1111/j.1747-4949.2012.00882.x · Cited by 63
Lamberts, M. P., Oudsten, B. L. D., Gerritsen, J. J. G. M., Roukema, J. A., Westert, G. P., Drenth, J. P. H., & van Laarhoven, C. J. H. M. (2015) · British journal of surgery · doi:10.1002/bjs.9887 · Cited by 62
Metadata
- Authors
- Ritter, P., Lorig, K., Laurent, D., & Matthews, K.
- Affiliations
- Stanford Medicine (US)
- Stanford University (US)
- Corresponding author
- Ritter, Philip
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2004-09-15
- Volume
- 6
- Issue
- 3
- Pages
- e29
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC1550608
- OpenAlex
- W2072139531
- Cited by
- 427
- References
- 16
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 4 · 2023: 11 · 2022: 9 · 2021: 13 · 2020: 10 · 2019: 13 · 2018: 18 · 2017: 18 · 2016: 22 · 2015: 29 · 2014: 25 · 2013: 27 · 2012: 35
- FWCI
- 69.21
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed (publishedVersion)
- TSpace (University of Toronto) (submittedVersion, cc-by)
- Funders
- Robert Wood Johnson Foundation
- Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Health Sciences › Nursing › Research and Theory › Health Education and Validation
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Keywords
- internet-based questionnaire, mail surveys, self-report measures, psychometric properties, test-retest reliability, construct reliability, questionnaire completion rates, patient education evaluation, patient-reported outcomes
- MeSH terms
- Adult, Aged, Aged, 80 and over, Female, Humans, Male, Middle Aged, Postal Service, Surveys and Questionnaires, Random Allocation, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
