The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Internet Versus Mailed Questionnaires: A Randomized Comparison
Open accessAbstract
BACKGROUND: The use of Internet-based questionnaires for collection of data to evaluate patient education and other interventions has increased in recent years. Many self-report instruments have been validated using paper-and-pencil versions, but we cannot assume that the psychometric properties of an Internet-based version will be identical. OBJECTIVES: To look at similarities and differences between the Internet versions and the paper-and-pencil versions of 16 existing self-report instruments useful in evaluation of patient interventions. METHODS: Participants were recruited via the Internet and volunteered to participate (N=397), after which they were randomly assigned to fill out questionnaires online or via mailed paper-and-pencil versions. The self-report instruments measured were overall health, health distress, practice mental stress management, Health Assessment Questionnaire (HAQ) disability, illness intrusiveness, activity limitations, visual numeric for pain, visual numeric for shortness of breath, visual numeric for fatigue, self-efficacy for managing disease, aerobic exercise, stretching and strengthening exercise, visits to MD, hospitalizations, hospital days, and emergency room visits. Means, ranges, and confidence intervals are given for each instrument within each type of questionnaire. The results from the two questionnaires were compared using both parametric and non-parametric tests. Reliability tests were given for multi-item instruments. A separate sample (N=30) filled out identical questionnaires over the Internet within a few days and correlations were used to assess test-retest reliability. RESULTS: Out of 16 instruments, none showed significant differences when the appropriate tests were used. Construct reliability was similar within each type of questionnaire, and Internet test-retest reliability was high. Internet questionnaires required less follow-up to achieve a slightly (non-significant) higher completion rate compared to mailed questionnaires. CONCLUSIONS: Among a convenience sample recruited via the Internet, results from those randomly assigned to Internet participation were at least as good as, if not better than, among those assigned mailed questionnaires, with less recruitment effort required. The instruments administered via the Internet appear to be reliable, and to be answered similarly to the way they are answered when they are administered via traditional mailed paper questionnaires.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“All measures were administered online, a method which has been shown to be reliable and as valid as traditional paper-and-pencil administration [40,41].”
From Acceptability, effectiveness, and cost-effectiveness of internet-based exposure treatment for irritable bowel syndrome in a clinical sample: A randomized controlled trial (2011) · cites this paper as [41]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Measurement of patient outcome in arthritis
Fries, J. F., Spitz, P. W., Kraines, R. G., & Holman, H. R. (1980) · Arthritis & Rheumatism · doi:10.1002/art.1780230202 · Cited by 4,404 · cited 1×
“• Health Assessment Instrument measures disability and is used in the National Health Survey (20 items) (Fries et al, 1980) [15].”
MethodGosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004) · American Psychologist · doi:10.1037/0003-066x.59.2.93 · Cited by 2,932 · cited 1×
“However, Gosling et al [19] have argued that Internet samples may actually be more representative than traditional samples.”
BackgroundOutcome Measures for Health Education and Other Health Care Interventions
Lorig, K., Stewart, A., Ritter, P., González, V., Laurent>, D., & Lynch, J. (1996) · doi:10.4135/9781452232966 · Cited by 1,132 · cited 8×
“• Self-reported exercise measures minutes/week of aerobic (5 items) and minutes/week stretching and strengthening exercise (1 item) (Lorig et al, 1996) [12].”
BackgroundMethodInfluentialSelf-citationSelf-rated health and mortality in the NHANES I Epidemiologic Follow-up Study
Idler, E. L., & Angel, R. J. (1990) · American Journal of Public Health · doi:10.2105/ajph.80.4.446 · Cited by 864 · cited 1×
“This item comes from the National Health Survey and has been found to be predictive of future health status (Idler & Angel, 1990) [14].”
MethodSocial desirability, anonymity, and internet-based questionnaires
Joinson, A. (1999) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03200723 · Cited by 818 · cited 2×
“Joinson (1999) reported that both anonymity and Web usage (compared to paper-and-pencil) resulted in lower scores on a social desirability measure [8].”
BackgroundUsing the Internet for Surveys and Health Research
Eysenbach, G., & Wyatt, J. (2002) · Journal of Medical Internet Research · doi:10.2196/jmir.4.2.e13 · Cited by 790 · cited 2×
“A recent overview entitled “Using the Internet for Surveys and Health Research” (Eysenbach & Wyatt 2002, [10]), barely touched on instruments (referring readers to the Quality of Life Instruments Database at Quality of Life Instruments database [11]) and did not discuss validity or reliability of…”
BackgroundUsing the Internet for psychological research: Personality testing on the World Wide Web
Buchanan, T., & Smith, J. L. (1999) · British Journal of Psychology · doi:10.1348/000712699161189 · Cited by 596 · cited 1×
“Buchanan and Smith (1999) [5] compared a Web-based personality assessment to a paper-and-pencil version, and using confirmatory factor analyses, found similar psychometric properties in the two tests.”
MethodSelf-reports of health care utilization compared to provider records
Ritter, P. L., Stewart, A. L., Kaymaz, H., Sobel, D. S., Block, D. A., & Lorig, K. R. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00261-4 · Cited by 401 · cited 1×
“• Health care utilization (MD visits, hospitalization, hospital days, ER visits) (Lorig et al, 1996; Ritter et al, 2001) [12,18].”
Self-citationThe Use of the Internet in Psychological Research: Comparison of Online and Offline Questionnaires
Riva, G., Teruzzi, T., & Anolli, L. (2003) · CyberPsychology & Behavior · doi:10.1089/109493103321167983 · Cited by 304 · cited 2×
“Joinson (1999) reported that both anonymity and Web usage (compared to paper-and-pencil) resulted in lower scores on a social desirability measure [8].”
BackgroundWeb-based administration of a personality questionnaire: Comparison with traditional methods
Davis, R. N. (1999) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03200737 · Cited by 226 · cited 2×
“Davis (1999) [6] also compared Web and paper-and-pencil versions of a personality measure (rumination), and concluded that “findings from Web-based questionnaire research are comparable with results obtained using standard procedures.””
Result
Flagged first, then by citations
Research works printed but not linked: 1
- Lin, C, 2003, Proc AMIA Symp, 911
Cited by
Sorted by citations
Items 201–300 of 425
Validation of the Emotion Regulation Questionnaire in older community‐dwelling adults
Brady, B., Kneebone, I., & Bailey, P. E. (2018) · British Journal of Clinical Psychology · doi:10.1111/bjc.12203 · Cited by 27
Rosa, F., Lohuis, P. J. F. M., Almeida, J. C., Santos, M., Oliveira, J., Sousa, C. A. E., & Ferreira, M. G. (2017) · Brazilian Journal of Otorhinolaryngology · doi:10.1016/j.bjorl.2017.11.007 · Cited by 27
Loprinzi, P. D., Schary, D. P., & Cardinal, B. J. (2012) · Maternal and Child Health Journal · doi:10.1007/s10995-012-0952-8 · Cited by 27
“On-line surveys have demonstrated evidence of construct reliability, test–retest reliability, and higher completion rates than mailed questionnaires [9].”
BackgroundDevelopment and validation of a short-form Pain Medication Attitudes Questionnaire (PMAQ-14)
Elander, J., Said, O., Maratos, F., Dys, A., Collins, H., & Schofield, M. B. (2017) · Pain · doi:10.1097/j.pain.0000000000000773 · Cited by 14
“Notwithstanding the evidence that online participant recruitment and data recruitment are as valid as other methods [27,28], some selection bias is perhaps almost inevitable.”
BackgroundTherapy Outcomes for Teenage Athletes With Exercise-Induced Laryngeal Obstruction
Fujiki, R. B., Olson-Greb, B. K., Braden, M. N., & Thibeault, S. L. (2023) · American Journal of Speech-Language Pathology · doi:10.1044/2023_ajslp-22-00359 · Cited by 26
“The likelihood of this influencing the current data, however, is low as research indicates that moving paper surveys online does not significantly affect responses (Hohwü et al., 2013; Ritter et al., 2004).”
Comparison of Web-Based and Paper-Based Administration of ADHD Questionnaires for Adults
Hirsch, O., Hauschild, F., Schmidt, M. H., Baum, E., & Christiansen, H. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2225 · Cited by 28
“Several studies did not find substantial differences between Web-based and paper-based modes of administration [4-7].”
BackgroundGranello, D. H. (2007) · Counselor Education and Supervision · doi:10.1002/j.1556-6978.2007.tb00039.x · Cited by 25
Stocchi, F., & Bloem, B. R. (2012) · European Journal of Neurology · doi:10.1111/j.1468-1331.2012.03876.x · Cited by 25
“However, online surveys may receive a greater number of responses than those sent through the post [54] and are convenient for PD patients to complete at home.”
de Nooijer, J., & de Vries, N. K. (2006) · Health Promotion International · doi:10.1093/heapro/dal036 · Cited by 24
Loprinzi, P. D., Schary, D. P., Beets, M. W., Leary, J. M., & Cardinal, B. J. (2013) · American Journal of Health Education · doi:10.1080/19325037.2012.749685 · Cited by 24
“On-line surveys have demonstrated evidence of construct reliability, test–retest reliability, and higher completion rates than mailed questionnaires.(29) Descriptive characteristics of the 176 parent–child dyads are shown in Table 1.”
BackgroundZetterqvist, V., Gentili, C., Rickardsson, J., Sörensen, I., & Wicksell, R. K. (2020) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsaa060 · Cited by 23
Middleton, A., Bragin, E., Morley, K. I., & Parker, M. J. (2013) · Social Science Research · doi:10.1016/j.ssresearch.2013.12.004 · Cited by 23
Beck, K. H., Yan, Q., & Wang, M. Q. (2009) · Journal of Safety Research · doi:10.1016/j.jsr.2009.07.007 · Cited by 23
Turunen, M., Paanala, A., Villman, J., Nevalainen, A. I., & Haverinen‐Shaughnessy, U. (2010) · Environmental Health · doi:10.1186/1476-069x-9-69 · Cited by 23
“Comparisons between responses to paper and Internet questionnaires [10] have shown them to yield similar results.”
ResultGoh, A. P. S. (2006) · Digital Commons - University of South Florida (University of South Florida) · Cited by 23
Is Lifelong Knee Joint Force from Work, Home, and Sport Related to Knee Osteoarthritis?
Ratzlaff, C., Koehoorn, M. W., Cibere, J., & Kopec, J. A. (2012) · International Journal of Rheumatology · doi:10.1155/2012/584193 · Cited by 22
“Self-selection implies that the nature of the bias cannot be known with certainty [99].”
BackgroundTreweek, S. P., Barnett, K. N., MacLennan, G., Bonetti, D. L., Eccles, M., Francis, J., Jones, C., Pitts, N. B., Ricketts, I. W., Weal, M. J., et al. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.11.010 · Cited by 22
Vargas, K. G., Nathan, J. E., Qian, F., & Kupietzky, A. (2007) · PubMed · Cited by 22
Hardré, P. L., Crowson, H. M., & Xie, K. (2010) · Journal of Educational Computing Research · doi:10.2190/ec.42.1.e · Cited by 21
“Response rates often vary significantly across delivery methods, with mixed results (e.g., Bälter et al., 2005; Leece et al., 2004; Nichols & Sedivi, 1998), and they may be further influenced by recruitment contact methods (Porter & Whitcomb, 2003) and by incentives to participate (Cobanoglu &…”
BackgroundA Web-Based Computerized Adaptive Testing (CAT) to Assess Patient Perception in Hospitalization
Chien, T., Wang, W., Huang, S.-Y., Lai, W.-P., & Chow, J. C. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1785 · Cited by 21
“This module and available files can complement the limited uses for interactive voice response or Internet-like visualization online [7] to complete questionnaires and put them into clinical practice.”
Lazarevic, V., Holman, E. G., Oswald, R. F., & Kramer, K. Z. (2015) · Journal of Family and Economic Issues · doi:10.1007/s10834-015-9464-1 · Cited by 21
IJmker, S., Mikkers, J., Blatter, B. M., van der Beek, A. J., van Mechelen, W., & Bongers, P. M. (2008) · Applied Ergonomics · doi:10.1016/j.apergo.2007.12.003 · Cited by 21
“However, results from other research areas suggest that there are no relevant differences between paper and pencil and web-based questionnaires (Ritter et al., 2004).”
Validation of the five facet mindfulness questionnaire among community-dwelling older adults.
Brady, B., Kneebone, I. I., & Bailey, P. E. (2019) · Mindfulness · doi:10.1007/s12671-018-0994-0 · Cited by 20
Lupattelli, A., Picinardi, M., Cantarutti, A., & Nordeng, H. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17113830 · Cited by 16
Pharmacists counselling of pregnant women: Web-based, comparative study between Serbia and Norway
Odalović, M., Milankovic, S. S., Holst, L., Nordeng, H., Heitmann, K., & Tasić, L. (2016) · Midwifery · doi:10.1016/j.midw.2016.06.003 · Cited by 20
Validation of Online Administration of the Sexual Health Inventory for Men
Bernstein, A., Levinson, A. W., Hobbs, A. R., Lavery, H. J., & Samadi, D. B. (2012) · The Journal of Urology · doi:10.1016/j.juro.2012.10.053 · Cited by 20
“Ritter et al evaluated web based formats of several previously validated HRQOL instruments, comparing responses to mailed paper surveys.(9) Overall, they found excellent consistency between Internet and paper responses for all included instruments.”
MethodWeb-Based Assessment of Mental Well-Being in Early Adolescence: A Reliability Study
Hamann, C., Schultze‐Lutter, F., & Tarokh, L. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5482 · Cited by 19
“A number of studies have reported feasibility and acceptance of Web-based questionnaires evaluating health among children [24-27], teenagers [11,12,26-29], and adults [30-40]; however, very few studies have examined the reliability of Web-based questionnaires under unsupervised conditions in a sample of young adolesce…”
Myasthenia Symptom Burden, Fatigue, and Sleep: Are They Related?
Yang, S. T., Miglis, M. G., Jaradeh, S. S., & Muppidi, S. (2021) · Journal of Clinical Neuromuscular Disease · doi:10.1097/cnd.0000000000000321 · Cited by 19
Association between personality traits and substance use in Spanish adolescents
Gonzálvez, M. T., Espada, J. P., Guillén‐Riquelme, A., Secades‐Villa, R., & Orgilés, M. (2016) · Adicciones · doi:10.20882/adicciones.777 · Cited by 19
Condensed internet-delivered prolonged exposure provided soon after trauma: A randomised pilot trial
Bragesjö, M., Arnberg, F. K., Särnholm, J., Lauri, K. O., & Andersson, E. (2020) · Internet Interventions · doi:10.1016/j.invent.2020.100358 · Cited by 20
“In general, internet-delivered self-report measures have been found to generate acceptable psychometric properties (Ritter et al., 2004).”
New treatment approaches: integrating new media in the treatment of war and torture victims.
Knaevelsrud, C., Wagner, B., Karl, A., & Mueller, J. (2007) · PubMed · Cited by 19
KIDMAP, a web based system for gathering patients' feedback on their doctors
Chien, T., Wang, W., Lin, S.-B., Lin, C., Guo, H., & Su, S. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-38 · Cited by 19
Sandin-Bojö, A.-K., Kvist, L., Berg, M., & Larsson, B. W. (2011) · International Journal of Health Care Quality Assurance · doi:10.1108/09526861111098265 · Cited by 20
Pescatori, E., Giammusso, B., Piubello, G., Gentile, V., & Farina, F. P. (2007) · The Journal of Sexual Medicine · doi:10.1111/j.1743-6109.2007.00460.x · Cited by 19
Petersen, K. E., Prows, C. A., Martin, L. J., & Maglo, K. N. (2014) · Public Health Genomics · doi:10.1159/000362359 · Cited by 26
Khidir, A., Asad, H., Abdelrahim, H., Elnashar, M. S., Killawi, A., Hammoud, M. M., Al-Khal, A. L., Haddad, P., & Fetters, M. D. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0109-3 · Cited by 18
“Most such studies have compared mail with e-mail [8, 10–12, 34]; few have compared mailed surveys with in-person administration [7, 14].”
BackgroundMethodResultInfluentialDisability and Self-Management Practices of People with Bipolar Disorder: A Web-Based Survey
Depp, C. A., Stricker, J. L., Zagorsky, D., Goodale, L. C., Eyler, L. T., Patterson, T. L., Lebowitz, B. D., & Jeste, D. V. (2008) · Community Mental Health Journal · doi:10.1007/s10597-008-9174-3 · Cited by 18
“This scale shows high reliability in general chronic disease samples, concordance between Internet based and paper-based formats (Ritter et al. 2004), and has been used previously with bipolar disorder (Robb et al. 1997).”
BackgroundRosly, M. M., Halaki, M., Rosly, H. M., Davis, G. M., Hasnan, N., & Husain, R. (2019) · Disability and Rehabilitation · doi:10.1080/09638288.2018.1544294 · Cited by 17
“Distribution of the questionnaire was done physically or online as evidence supported that the two methods produced similar results [24,25].”
ResultHartford, K., Carey, R. G., & Mendonca, J. D. (2007) · Evaluation & the Health Professions · doi:10.1177/0163278706297344 · Cited by 18
“However, examining the reliability of Internet versus mailed questionnaires, Ritter et al. (2004) assembled a convenience sample of people with chronic disease who were computer literate.”
BackgroundDevelopments in the self-report assessment of personality and psychopathology in adults.
Holden, R. R., & Troister, T. (2009) · Canadian Psychology/Psychologie canadienne · doi:10.1037/a0015959 · Cited by 17
Beukes, E. W., Manchaiah, V. K. C., Allen, P. M., Baguley, D., & Andersson, G. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008241 · Cited by 19
“Previous research has found comparable results in terms of psychometric properties between computer and paper questionnaire delivery, with high test–retest reliability and completion rate on the internet.(46) 47 See table 3 for the schedule of outcome measure delivery.”
ResultGill, S. K., Muñoz, R. F., & Leykin, Y. (2017) · Crisis · doi:10.1027/0227-5910/a000492 · Cited by 17
Rittase, M. H., Kirkland, E. G., Dudas, D. M., & Patel, A. V. (2020) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-19-0757 · Cited by 17
Evaluating changes in gastrointestinal endoscopy training over 5 years: closing the audit loop
Haycock, A., Patel, J. H., Tekkis, P. P., & Thomas‐Gibson, S. (2010) · European Journal of Gastroenterology & Hepatology · doi:10.1097/meg.0b013e32832adfac · Cited by 17
Characteristics of participants visiting the Canada on the move website.
Plotnikoff, R. C., Spence, J. C., Tavares, L. S., Rovniak, L. S., Bauman, A. E., Lear, S. A., & McCargar, L. J. (2006) · PubMed · doi:10.1007/bf03405370 · Cited by 17
Methodological considerations in evaluating eHealth interventions.
Nguyen, H. Q., Cuenco, D., Wolpin, S. E., Benditt, J., & Carrieri‐Kohlman, V. (2007) · PubMed · Cited by 17
Skjerning, H., Hourihane, J., Husby, S., & DunnGalvin, A. (2017) · Quality of Life Research · doi:10.1007/s11136-017-1632-3 · Cited by 18
Ainembabazi, P., Castelnuovo, B., Okoboi, S., Arinaitwe, J. W., Parkes-Ratanshi, R. M., & Byakika‐Kibwika, P. (2021) · BMC Medical Ethics · doi:10.1186/s12910-021-00692-6 · Cited by 16
“This is consistent with previous studies that concluded that online surveys generally have a lower response rate compared to the in-person surveys [32, 33].”
ResultComparing responses from a paper-based survey with a web-based survey in environmental criminology
Ceccato, V. A., Gliori, G., Näsman, P., & Sundling, C. (2024) · Crime Prevention and Community Safety · doi:10.1057/s41300-024-00204-9 · Cited by 18
Factor structure of the Body Image Concern Inventory in a Japanese sample
Tanaka, M., Tayama, J., & 有村, 達. (2014) · Body Image · doi:10.1016/j.bodyim.2014.09.007 · Cited by 16
O’Brien, K. K., Solomon, P. E., Worthington, C., Ibáñez-Carrasco, F., Baxter, L. N., Nixon, S., Turje, R. B., Robinson, G., Zack, E., & The HIV, H. A. R. S. C. T. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3064 · Cited by 16
“Furthermore, evidence suggests response rates and reliability and validity of Web-based health status questionnaires are similar to questionnaires administered with pen and paper, with fewer recruitment efforts among people with chronic disease [40].”
Bot, A. G. J., Menendez, M. E., Neuhaus, V., Mudgal, C. S., & Ring, D. (2013) · Hand · doi:10.1007/s11552-013-9511-z · Cited by 16
“Ritter and colleagues documented comparable results for 16 different instruments [16].”
ResultThomée, S. (2012) · Gothenburg University Publications Electronic Archive (Gothenburg University) · Cited by 16
Health behaviours in people who respond to a web-based survey advertised on regional news media
Adams, J. M., & White, M. J. (2007) · European Journal of Public Health · doi:10.1093/eurpub/ckm100 · Cited by 15
Arndt, U., Behrens, F., Ziswiler, H. R., Kaltwasser, J. P., & Möller, B. (2007) · Rheumatology International · doi:10.1007/s00296-007-0382-x · Cited by 15
“A separate patient directed questionnaire focusing on speciWc functional limitations of RA and possibly also SpA appears useful and could be provided online [18, 19], which will probably improve the completion rate.”
Internet communities for recruitment of cancer patients into an internet survey: A discussion paper
Im, E., Chee, W., Tsai, H., Bender, M. S., & Lim, H. J. (2006) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2006.07.003 · Cited by 15
Problem gambling during Covid-19
Frisone, F., Alibrandi, A., & Settineri, S. (2020) · Mediterranean Journal of Clinical Psychology · doi:10.6092/2282-1619/mjcp-2457 · Cited by 30
Chronic Lumbar Pain and Insomnia in College-Aged Students
Hamaoka, K., Ashizawa, R., Hida, M., Suganuma, I., & Yoshimoto, Y. (2022) · Healthcare · doi:10.3390/healthcare10040701 · Cited by 14
“The web-based survey research methods used here have recently been employed in medical publications and large-scale studies; further, they represent a surveying technique that does not differ significantly from postal methods [29].”
MethodShimoda, H., Inoue, A., Tsuno, K., & Kawakami, N. (2014) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1437 · Cited by 16
“While previous research showed no significant difference in test–retest reliability among face-to-face, telephone, and the internet surveys (Ritter et al., 2004; Vallejo et al., 2007; Donker et al., 2010), the current finding may not be applicable to Int.”
ResultEquivalence of Paper and Computer Formats of a Child Self-Report Mental Health Measure
Patalay, P., Deighton, J., Fonagy, P., & Wolpert, M. (2014) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000206 · Cited by 14
“In terms of study methodology used to test psychometric equivalence between survey formats, two approaches are commonly used: same sample repeated measurement (e.g. Holländare et al., 2010) and comparison of separate demographically similar samples (e.g., Ritter et al., 2004).”
MethodAndreeva, V. A., Galan, P., Julia, C., Castetbon, K., Kesse‐Guyot, E., & Herçberg, S. (2014) · American Journal of Epidemiology · doi:10.1093/aje/kwt431 · Cited by 14
Carlesso, L. C., MacDermid, J. C., Santaguida, P. L., & Thabane, L. (2013) · Physiotherapy Canada · doi:10.3138/ptc.2012-28 · Cited by 13
Balancing health literacy about epilepsy surgery in the community
Zuccato, J., Milburn, C., & Valiante, T. A. (2014) · Epilepsia · doi:10.1111/epi.12791 · Cited by 13
“” Can effect seizure freedom 5 (11) 10 (22) 26 (58) 3 (7) 1 (2) Increases one’s independence 6 (13) 12 (26) 25 (54) 2 (4) 1 (2) Increases work productivity 6 (13) 14 (31) 22 (49) 1 (2) 2 (4) Enhances one’s social acceptance 6 (13) 10 (22) 26 (57) 3 (7) 1 (2) Enhances one’s relationships 7 (15) 5 (11) 31 (67) 2 (4) 1 (…”
Constant, A., Dulioust, J., Wazana, A. D., Shojaei, T., Pitrou, I., & Kovess–Masféty, V. (2014) · BMC Pediatrics · doi:10.1186/1471-2431-14-2 · Cited by 13
“The sample is a large-scale randomized French sample using strategies to ensure faithful estimates of population values; the association between unintentional injuries and child psychopathology symptoms was examined using both parent and child report; and the non-response rate was satisfactory and consistent with many…”
MethodNo blank slates: Pre-existing schemas about pharmaceuticals predict memory for side effects
Heller, M. K., Chapman, S., & Horne, R. (2017) · Psychology and Health · doi:10.1080/08870446.2016.1273355 · Cited by 12
The effects of severity and causal explanation on social perceptions of hair loss
Ricketts, E., Brandt, B. C., & Woods, D. W. (2012) · Journal of Obsessive-Compulsive and Related Disorders · doi:10.1016/j.jocrd.2012.07.007 · Cited by 14
Moritz, S., Dietl, C., Kersten, J. F., Aardema, F., & O’Connor, K. P. (2015) · Journal of Cognitive Psychotherapy · doi:10.1891/0889-8391.29.4.315 · Cited by 12
Can the Toronto Extremity Salvage Score Produce Reliable Results When Used Online?
Clayer, M., & Davis, A. M. (2010) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-010-1715-8 · Cited by 12
“However, there have been concerns raised regarding the reliability of Internet-based versions of questionnaires [4, 12, 22, 24, 34].”
BackgroundCherdchu, P., & Chambers, E. (2013) · Journal of Sensory Studies · doi:10.1111/joss.12075 · Cited by 12
Do patients die well in your emergency department?
Bradley, V., Burney, C., & Hughes, G. (2013) · Emergency Medicine Australasia · doi:10.1111/1742-6723.12099 · Cited by 12
“There is evidence that Internet surveys are as reliable as postal surveys and can have higher completion rates, especially when potential respondents are contacted by email.(10) Thus, in populations that already use the Internet, Internet surveys have been found to be a useful means of conducting research.”
BackgroundDainesi, S. M., & Goldbaum, M. (2012) · Revista Brasileira de Epidemiologia · doi:10.1590/s1415-790x2012000400003 · Cited by 11
Tortella‐Feliu, M., Soler, J., Burns, L. G., Cebolla, A., Elices, M., Pascual, J. C., del Hoyo, Y. L., Baños, R., & García‐Campayo, J. (2018) · Personality and Mental Health · doi:10.1002/pmh.1420 · Cited by 11
Said, M., der Leeuw, R. J.-V., Spek, B., Brand, P. L. P., & van Dijk, N. (2019) · Perspectives on Medical Education · doi:10.1007/s40037-019-00527-6 · Cited by 11
“Previous studies showed no differences in psychometric properties between paperbased and web-based administration of health measuring tools [38, 39].”
BackgroundLupattelli, A., Rønningen, E. Y. A., Krogsrud, S. K., Nordeng, H., & Ystrøm, E. (2018) · Journal of Health Psychology · doi:10.1177/1359105318775194 · Cited by 11
“Several areas of research have shown that the information provided in a web-based questionnaire is equivalent, of quality, and as reliable as that collected via traditional methods (Ritter et al., 2004; Whitehead, 2011).”
MethodLecturers’ and students’ perceptions of portfolios in an English School of Nursing
Williams, G. A., Park, J. R., Traynor, V., Nairn, S., O’Brien, E., Chapple, M., & Johnson, S. H. (2008) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.2008.02553.x · Cited by 11
“Ritter et al. (2004) have shown that research participants can reply to online and paper-based questions in a similar manner; this allowed us to treat the data from both groups of participants as comparable.”
ResultU.S.-Mexico cross-border workforce training needs: survey implementation
Rosales, C., Nuño, T., Dieke, A. C., Galvez, F. N., Dutton, R. J., Guerrero, R., Dulin, P., Jiménez, E. A., Granillo, B., & de Zapien, J. G. (2010) · Journal of Injury and Violence Research · doi:10.5249/jivr.v3i1.55 · Cited by 10
Long-Term Care Planning and Preparation Among Persons with Multiple Sclerosis
Putnam, M., & Tang, F. (2008) · Home Health Care Services Quarterly · doi:10.1080/01621420802022613 · Cited by 10
“Research indicates no difference in reliability of paper and electronic survey instruments ( Ritter, Lorig, Laurent, & Matthews, 2004 ).”
BackgroundGorrasi, I. S. R., Ferraris, C., Degan, R., Daga, G. A., Bo, S., Tagliabue, A., Guglielmetti, M., Roppolo, M., Gilli, G., Maran, D. A., et al. (2021) · Eating and Weight Disorders - Studies on Anorexia Bulimia and Obesity · doi:10.1007/s40519-021-01231-3 · Cited by 10
Wellbeing, psychiatric morbidity and psychological distress amongst medical students in Denmark
Kjær, J. N., Molodynski, A., Bhugra, D., & Lewis, T. (2022) · International Journal of Social Psychiatry · doi:10.1177/00207640221074916 · Cited by 12
“Previous research does however support that psychopathology screening questionnaires that are administered online are as reliable as paper-and-pencil comparisons (Ritter et al., 2004; Vallejo et al., 2007).”
BackgroundFeasibility and Acceptability of Alternate Methods of Postnatal Data Collection
McCormack, L. A., Friedrich, C., Fahrenwald, N. L., & SPECKER, B. L. (2013) · Maternal and Child Health Journal · doi:10.1007/s10995-013-1310-1 · Cited by 10
“While a few studies have assessed response rates in web versus mailed/ paper surveys [3–8] and web versus telephone surveys [9– 11] very little research exists that compares the feasibility, acceptability and cost-effectiveness of telephone, webbased and mailed/paper survey data collection.”
BackgroundWong, A., Heinemann, A. W., Mišković, A., Semik, P. E., & Snyder, T. M. (2014) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2013.12.024 · Cited by 10
Andersson, G., Ritterband, L. M., & Carlbring, P. (2008) · Clinical Psychologist · doi:10.1080/13284200802069027 · Cited by 12
Vogt, F., & Marteau, T. M. (2011) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntr197 · Cited by 10
Wang, A. R., McCabe, M. D., Gow-Lee, E., James, S., Austin, B., Wailes, D., Dinh, V. A., & Ramsingh, D. S. (2021) · Postgraduate Medical Journal · doi:10.1136/postgradmedj-2021-140127 · Cited by 9
Conducting Large‐Scale Surveys in Secondary Schools: The Case of the Youth On Religion (YOR) Project
Madge, N. J. H., Hemming, P., Goodman, A., Goodman, S., Kingston, S., Stenson, K., & Webster, C. (2011) · Children & Society · doi:10.1111/j.1099-0860.2011.00364.x · Cited by 9
Loprinzi, P. D., Cardinal, B. J., Kane, C., Lee, H. J., & Beets, M. W. (2014) · American Journal of Health Education · doi:10.1080/19325037.2014.916636 · Cited by 9
“Use of online surveys is acceptable, because online surveys have demonstrated evidence of construct reliability, test–retest reliability, and higher completion rates than mailed questionnaires.(54) The university’s Institutional Review Board approved this study, and parents provided informed consent prior to completin…”
BackgroundBusiness Sense or Subjective Satisfaction?
Osiyevskyy, O., Costa, S. F., & Madill, C. M. (2016) · The International Journal of Entrepreneurship and Innovation · doi:10.5367/ijei.2016.0207 · Cited by 9
“…Internetbased surveys is not lower in any way than that of traditional, mail-based studies (see, for example, the investigation of Gosling et al (2004) in psychology, Ritter et al (2004) in medicine, Ekman et al (2006) in epidemiology, and Paolacci et al (2010) in behavioural decision making).”
ResultHoque, D. M. E., Earnest, A., Ruseckaite, R., Lorgelly, P. K., Sampurno, F., Evans, M., & Evans, S. (2018) · Quality of Life Research · doi:10.1007/s11136-018-2061-7 · Cited by 9
Batista, S. A., Ginani, V. C., Stedefeldt, E., Nakano, E. Y., & Botelho, R. B. A. (2023) · Nutrients · doi:10.3390/nu15010213 · Cited by 8
Ein national gefördertes Onlinelabor als Infrastruktur für die psychologische Forschung
Bruder, M., Göritz, A. S., Reips, U., & Gebhard, R. K. (2014) · Psychologische Rundschau · doi:10.1026/0033-3042/a000198 · Cited by 8
An international study of analgesic dependence among people with pain in the general population
Said, O., Elander, J., & Maratos, F. A. (2019) · Substance Use & Misuse · doi:10.1080/10826084.2019.1577457 · Cited by 8
“Online survey methods have been shown to be valid and reliable (Andrews et al., 2003; Ramsey et al., 2016; Ritter et al., 2004), and may be an important strategy to enable people in diverse parts of the world to take part in international research.”
BackgroundAchieving and Exceeding Logistics Outsourcing Expectations in Brazil: A Replication Study
Hofer, A. R., Knemeyer, A. M., & Murphy, P. R. (2015) · Transportation Journal · doi:10.5325/transportationj.54.3.0339 · Cited by 8
Bernstein, M. T., Walker, J. R., Sexton, K. A., Katz, A., & Beatie, B. E. (2017) · JMIR Mental Health · doi:10.2196/mental.7231 · Cited by 9
Bortolon, C., & Raffard, S. (2018) · The Journal of Nervous and Mental Disease · doi:10.1097/nmd.0000000000000885 · Cited by 8
“Previous studies have shown that data obtained over the Internet are comparable in reliability with studies using more conventional measures (Jones et al., 2008; Ritter et al., 2004).”
ResultRhoden, V. (2012) · doi:10.25148/etd.fi12050245 · Cited by 8
“Although some online surveys have yielded fewer responses than more traditionally administered surveys (Leece et al., 2004; Wright, 2005), an increasing number have yielded excellent online response rates (e.”
BackgroundGiray, L., Fabros, B. G., Villarama, J. A., Gumalin, D. L., & Saavedra, J. B. (2025) · Internet Reference Services Quarterly · doi:10.1080/10875301.2025.2542212 · Cited by 9
“Moreover, online questionnaires have demonstrated comparable reliability and validity to traditional methods, with reduced recruitment effort (Ritter et al., 2004).”
Wynter, E., & Perich, T. (2018) · Clinical Psychologist · doi:10.1111/cp.12149 · Cited by 9
Holman, E. G., & Oswald, R. F. (2016) · Sexual and Gender Diversity in Social Services · doi:10.1080/10538720.2016.1191406 · Cited by 8
Manchaiah, V. K. C., Pyykkö, I. V., Zou, J., Levo, H., & Kentala, E. L. (2017) · Patient Preference and Adherence · doi:10.2147/ppa.s142257 · Cited by 7
“Also, when comparing Internet-based data collection with paper-and-pencil versions, Internet data appear to be reliable and need to be answered similarly to the way they are answered when they are administered via traditionally mailed paper questionnaires.(23) Moreover, all participants were asked to respond about the…”
MethodThe use of social networks in scientific research with questionnaires
Grossmann, S. D. M. C., Moura, M. D. G., Matias, M. D. P., Paiva, S. M., & Mesquita, R. A. (2018) · Brazilian journal of oral sciences/Brazilian Journal of Oral Sciences · doi:10.20396/bjos.v17i0.8652908 · Cited by 7
Metadata
- Authors
- Ritter, P., Lorig, K., Laurent, D., & Matthews, K.
- Affiliations
- Stanford Medicine (US)
- Stanford University (US)
- Corresponding author
- Ritter, Philip
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2004-09-15
- Volume
- 6
- Issue
- 3
- Pages
- e29
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC1550608
- OpenAlex
- W2072139531
- Cited by
- 427
- References
- 16
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 4 · 2023: 11 · 2022: 9 · 2021: 13 · 2020: 10 · 2019: 13 · 2018: 18 · 2017: 18 · 2016: 22 · 2015: 29 · 2014: 25 · 2013: 27 · 2012: 35
- FWCI
- 69.21
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed (publishedVersion)
- TSpace (University of Toronto) (submittedVersion, cc-by)
- Funders
- Robert Wood Johnson Foundation
- Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Health Sciences › Nursing › Research and Theory › Health Education and Validation
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Keywords
- internet-based questionnaire, mail surveys, self-report measures, psychometric properties, test-retest reliability, construct reliability, questionnaire completion rates, patient education evaluation, patient-reported outcomes
- MeSH terms
- Adult, Aged, Aged, 80 and over, Female, Humans, Male, Middle Aged, Postal Service, Surveys and Questionnaires, Random Allocation, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
