Methodological Issues in Internet-Mediated Research: A Randomized Comparison of Internet Versus Mailed Questionnaires
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The majority of Internet-mediated studies use measures developed as paper-and-pencil measures or face-to-face-delivered material. Previous research suggests that the equivalence between online and offline measures must be demonstrated rather than assumed. OBJECTIVE: The objective of this study was to explore the equivalence 4 measures completed in an online or offline setting. METHODS: A sample (n = 1969) were randomly assigned to complete 4 popular scales (the SF-12v2, the Hospital Anxiety and Depression Scale (HADS), the Fatigue Symptom Inventory, and a single-item fatigue measure) either online or by mail survey (pencil and paper). The response rate was 52.51% (n = 1034) and comparable between the online and offline groups. RESULTS: Significant differences were noted in fatigue levels between the online and offline group (P = .01) as measured by the Fatigue Symptom Inventory, with the online sample demonstrating higher levels of fatigue. Equivalency was noted for the SF-12v2, the Hospital Anxiety and Depression Scale, and the single-item fatigue measure. Internal consistency was high except for the SF-12v2. The SF-12v2 may not be an ideal measure to use for remote administration. CONCLUSIONS: Equivalency of the Hospital Anxiety and Depression Scale (HADS) and the Physical Component Score and Mental Component Score of the SF-12v2 for online and offline data were demonstrated. Equivalency was not demonstrated for the Fatigue Symptom Inventory. Explanations for the difference in fatigue score between the online and offline samples are unclear. Research that seeks to match samples and control for extraneous online and offline variables is called for, along with exploration of factors that may mediate the completion of questionnaires or alter the respondents' relationship with the same, to enhance progress in this area.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Comparing Response Rates from Web and Mail Surveys: A Meta-Analysis
Shih, T.-H., & Fan, X. (2008) · Field Methods · doi:10.1177/1525822x08317085 · Cited by 1,159
Hann (1998) · Quality of Life Research · doi:10.1023/a:1008842517972 · Cited by 500
Internet Versus Mailed Questionnaires: A Randomized Comparison
Ritter, P., Lorig, K., Laurent, D., & Matthews, K. (2004) · Journal of Medical Internet Research · doi:10.2196/jmir.6.3.e29 · Cited by 427
Implementing a five-factor personality inventory for use on the internet
Buchanan, T., Johnson, J. A., & Goldberg, L. R. (2005) · European Journal of Psychological Assessment · doi:10.1027/1015-5759.21.2.115 · Cited by 301
McCue, P., Buchanan, T., & Martin, C. R. (2006) · British Journal of Clinical Psychology · doi:10.1348/014466505x82379 · Cited by 58
Cited by
Khazaal, Y., van Singer, M., Chatton, A., Achab, S., Zullino, D., Rothen, S., Khan, R., Billieux, J., & Thorens, G. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.2759 · Cited by 201
“Some studies suggest that the quality of the data provided by Internet-based surveys is at least as good as in those collected by traditional paper-and-pencil methods on self-selected samples [6-8].”
Backgroundvan Ballegooijen, W., Riper, H., Cuijpers, P., van Oppen, P., & Smit, J. H. (2016) · BMC Psychiatry · doi:10.1186/s12888-016-0735-7 · Cited by 114
“Of the symptom severity scales, the HADS was investigated in 5 studies [19, 22–25].”
MethodHeller, H. M., Hoogendoorn, A. W., Honig, A., Broekman, B. F. P., & van Straten, A. (2020) · Journal of Medical Internet Research · doi:10.2196/15172 · Cited by 82
Kumar, S., Kumar, P., & Bhasker, B. (2018) · Behaviour & Information Technology · doi:10.1080/0144929x.2018.1470671 · Cited by 44
Weigold, A., Weigold, I. K., & Natera, S. N. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.04.038 · Cited by 26
Metadata
- Authors
- Whitehead, L.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2011-12-12
- Volume
- 13
- Issue
- 4
- Pages
- e109
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W2037909863
- Cited by
- 79
- References
- 5
- FWCI
- 20.36
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.