CoChat
Sign Up
Cited by 244Survey Methodology and NonresponseSurvey Sampling and Estimation TechniquesCensus and Population Estimation

The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.

A randomised trial and economic evaluation of the effect of response mode on response rate, response bias, and item non-response in a survey of doctors

Open access
Scott, Anthony, Jeon, Sung-Hee, Joyce, Catherine M, Humphreys, John S, Kalb, Guyonne, Witt, Julia, Leahy, Anne·2011·BMC Medical Research Methodology
DOI 10.1186/1471-2288-11-126PMC PMC3231767
Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Surveys of doctors are an important data collection method in health services research. Ways to improve response rates, minimise survey response bias and item non-response, within a given budget, have not previously been addressed in the same study. The aim of this paper is to compare the effects and costs of three different modes of survey administration in a national survey of doctors. Methods: A stratified random sample of 4.9% (2,702/54,160) of doctors undertaking clinical practice was drawn from a national directory of all doctors in Australia. Stratification was by four doctor types: general practitioners, specialists, specialists-in-training, and hospital non-specialists, and by six rural/remote categories. A three-arm parallel trial design with equal randomisation across arms was used. Doctors were randomly allocated to: online questionnaire (902); simultaneous mixed mode (a paper questionnaire and login details sent together) (900); or, sequential mixed mode (online followed by a paper questionnaire with the reminder) (900). Analysis was by intention to treat, as within each primary mode, doctors could choose either paper or online. Primary outcome measures were response rate, survey response bias, item non-response, and cost. Results: The online mode had a response rate 12.95%, followed by the simultaneous mixed mode with 19.7%, and the sequential mixed mode with 20.7%. After adjusting for observed differences between the groups, the online mode had a 7 percentage point lower response rate compared to the simultaneous mixed mode, and a 7.7 percentage point lower response rate compared to sequential mixed mode. The difference in response rate between the sequential and simultaneous modes was not statistically significant. Both mixed modes showed evidence of response bias, whilst the characteristics of online respondents were similar to the population. However, the online mode had a higher rate of item non-response compared to both mixed modes. The total cost of the online survey was 38% lower than simultaneous mixed mode and 22% lower than sequential mixed mode. The cost of the sequential mixed mode was 14% lower than simultaneous mixed mode. Compared to the online mode, the sequential mixed mode was the most cost-effective, although exhibiting some evidence of response bias. Conclusions: Decisions on which survey mode to use depend on response rates, response bias, item non-response and costs. The sequential mixed mode appears to be the most cost-effective mode of survey administration for surveys of the population of doctors, if one is prepared to accept a degree of response bias. Online surveys are not yet suitable to be used exclusively for surveys of the doctor population.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/1471-2288-11-126

Cites

36 of 36 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Description of mode of initial contact and follow up contact for the three arms of the trial.
Figure 1 Description of mode of initial contact and follow up contact for the three arms of the trial.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 2

  1. Millar M, O'Neill A, Dillman D: Are mode preferences real?. 2009, Pullman: Washington State University
  2. Australian Bureau of Statistics: ASGC remoteness classification: purpose and use. 2003, Canberra: ABS

Cited by

Sorted by citations

See all 244

Related

See all

Metadata

Authors
Scott, A., Jeon, S.-H., Joyce, C. M., Humphreys, J. S., Kalb, G., Witt, J., & Leahy, A.
Affiliations
  • The University of Melbourne (AU)
  • The Alfred Hospital (AU)
  • Monash University (AU)
  • University of Manitoba (CA)
Corresponding author
Joyce, Catherine M
ORCID
Venue
BMC Medical Research Methodology
Publisher
Springer Science and Business Media LLC
Published
2011-09-05
Volume
11
Issue
1
Pages
126-126
ISSN
1471-2288 (ISSN-L)
DOI
10.1186/1471-2288-11-126
PMC
PMC3231767
OpenAlex
W2056264205
Cited by
244
References
36
Citations by year
2026: 4 · 2025: 8 · 2024: 17 · 2023: 20 · 2022: 24 · 2021: 20 · 2020: 24 · 2019: 17 · 2018: 23 · 2017: 13 · 2016: 20 · 2015: 17 · 2014: 21 · 2013: 10 · 2012: 5
FWCI
39.64
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Research Methodology (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Minerva Access (University of Melbourne) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Monash University
  • University of Melbourne
  • Medical Research Council
  • National Health and Medical Research Council (454799)
Topics
  • Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
  • Physical Sciences › Mathematics › Statistics and Probability › Survey Sampling and Estimation Techniques
  • Physical Sciences › Mathematics › Statistics and Probability › Census and Population Estimation
Keywords
survey mode, online surveys, paper-based questionnaires, response rate, response bias, item nonresponse, cost-effectiveness, physicians, survey administration, general practitioners, specialists, Canada
MeSH terms
Adult, Aged, Australia, Cost-Benefit Analysis, Female, Humans, Job Satisfaction, Longitudinal Studies, Male, Middle Aged, Motivation, Physicians, Surveys and Questionnaires, Odds Ratio, Health Care Surveys

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review