Methods to increase response to postal and electronic questionnaires
Open accessAbstract
BACKGROUND: Postal and electronic questionnaires are widely used for data collection in epidemiological studies but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of health research. OBJECTIVES: To identify effective strategies to increase response to postal and electronic questionnaires. SEARCH STRATEGY: We searched 14 electronic databases to February 2008 and manually searched the reference lists of relevant trials and reviews, and all issues of two journals. We contacted the authors of all trials or reviews to ask about unpublished trials. Where necessary, we also contacted authors to confirm methods of allocation used and to clarify results presented. We assessed the eligibility of each trial using pre-defined criteria. SELECTION CRITERIA: Randomised controlled trials of methods to increase response to postal or electronic questionnaires. DATA COLLECTION AND ANALYSIS: We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random-effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity among trial odds ratios using a Chi(2) test and the degree of inconsistency between trial results was quantified using the I(2) statistic. MAIN RESULTS: PostalWe found 481 eligible trials. The trials evaluated 110 different ways of increasing response to postal questionnaires. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were at least doubled using monetary incentives (odds ratio 1.87; 95% CI 1.73 to 2.04; heterogeneity P < 0.00001, I(2) = 84%), recorded delivery (1.76; 95% CI 1.43 to 2.18; P = 0.0001, I(2) = 71%), a teaser on the envelope - e.g. a comment suggesting to participants that they may benefit if they open it (3.08; 95% CI 1.27 to 7.44) and a more interesting questionnaire topic (2.00; 95% CI 1.32 to 3.04; P = 0.06, I(2) = 80%). The odds of response were substantially higher with pre-notification (1.45; 95% CI 1.29 to 1.63; P < 0.00001, I(2) = 89%), follow-up contact (1.35; 95% CI 1.18 to 1.55; P < 0.00001, I(2) = 76%), unconditional incentives (1.61; 1.36 to 1.89; P < 0.00001, I(2) = 88%), shorter questionnaires (1.64; 95% CI 1.43 to 1.87; P < 0.00001, I(2) = 91%), providing a second copy of the questionnaire at follow up (1.46; 95% CI 1.13 to 1.90; P < 0.00001, I(2) = 82%), mentioning an obligation to respond (1.61; 95% CI 1.16 to 2.22; P = 0.98, I(2) = 0%) and university sponsorship (1.32; 95% CI 1.13 to 1.54; P < 0.00001, I(2) = 83%). The odds of response were also increased with non-monetary incentives (1.15; 95% CI 1.08 to 1.22; P < 0.00001, I(2) = 79%), personalised questionnaires (1.14; 95% CI 1.07 to 1.22; P < 0.00001, I(2) = 63%), use of hand-written addresses (1.25; 95% CI 1.08 to 1.45; P = 0.32, I(2) = 14%), use of stamped return envelopes as opposed to franked return envelopes (1.24; 95% CI 1.14 to 1.35; P < 0.00001, I(2) = 69%), an assurance of confidentiality (1.33; 95% CI 1.24 to 1.42) and first class outward mailing (1.11; 95% CI 1.02 to 1.21; P = 0.78, I(2) = 0%). The odds of response were reduced when the questionnaire included questions of a sensitive nature (0.94; 95% CI 0.88 to 1.00; P = 0.51, I(2) = 0%).ElectronicWe found 32 eligible trials. The trials evaluated 27 different ways of increasing response to electronic questionnaires. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were increased by more than a half using non-monetary incentives (1.72; 95% CI 1.09 to 2.72; heterogeneity P < 0.00001, I(2) = 95%), shorter e-questionnaires (1.73; 1.40 to 2.13; P = 0.08, I(2) = 68%), including a statement that others had responded (1.52; 95% CI 1.36 to 1.70), and a more interesting topic (1.85; 95% CI 1.52 to 2.26). The odds of response increased by a third using a lottery with immediate notification of results (1.37; 95% CI 1.13 to 1.65), an offer of survey results (1.36; 95% CI 1.15 to 1.61), and using a white background (1.31; 95% CI 1.10 to 1.56). The odds of response were also increased with personalised e-questionnaires (1.24; 95% CI 1.17 to 1.32; P = 0.07, I(2) = 41%), using a simple header (1.23; 95% CI 1.03 to 1.48), using textual representation of response categories (1.19; 95% CI 1.05 to 1.36), and giving a deadline (1.18; 95% CI 1.03 to 1.34). The odds of response tripled when a picture was included in an e-mail (3.05; 95% CI 1.84 to 5.06; P = 0.27, I(2) = 19%). The odds of response were reduced when "Survey" was mentioned in the e-mail subject line (0.81; 95% CI 0.67 to 0.97; P = 0.33, I(2) = 0%), and when the e-mail included a male signature (0.55; 9
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
People may be more prone to participate in a survey if the topic seems relevant to them personally.
“people may also be more prone to participate in a survey if the topic seems relevant to them personally.30”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Effect of Incentives and Mailing Features on Online Health Program Enrollment
Alexander, G. L., Divine, G. W., Couper, M. P., McClure, J. B., Stopponi, M. A., Fortman, K. K., Tolsma, D. D., Strecher, V. J., & Johnson, C. C. (2008) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2008.01.028 · Cited by 80 · see citing works
Flagged first, then by citations
Research works printed but not linked: 19
- P; A; Prof; Private research institute sponsored (PR) 3. P; A; personal appeal (Pers); U 4. P; A; Pers; PR 5. P; Nonanonymous (NA); Prof; U 6. P; NA; Prof; PR 7. P; NA; Pers; U 8. P; NA; Pers; PR 9. Not Personalised (Not P); A; Prof; U 10. Not P; A; Prof; PR 11. Not P; A; Pers; U 12. Not P; A; Pers; PR 13. Not P; NA; Prof; U 14. Not P; NA; Prof; PR 15. Not P; NA; Pers; U 16. Not P; NA; Pers; PR Comparisons 1. 10 cent incentive (MI); Postcard follow up 3 days after initial mailing (FU); Business reply envelope (BRE); Offer of survey results (OR) 2. MI; No follow up (No FU); BRE; OR 3. MI; FU; BRE; No offer of survey results (No OR) 4. MI; No FU; BRE; No OR 5. MI; FU; Stamped return envelope (SRE); OR 6. MI; No FU; SRE; OR 7. MI; FU; SRE; No OR 8. MI; No FU; SRE; No OR 9. No monetary incentive (NI); FU; BRE; OR 10. NI; No FU; BRE; OR 11. NI; FU; BRE; No OR 12. NI; No FU; BRE; No OR 13. NI; FU; SRE; OR 14. NI; No FU; SRE; OR 15. NI; FU; SRE; No OR 16. NI; No FU; SRE; No OR E. A and B and C and D Social Science Citation Index 1981 -1999 Science Citation Indes 1981 -1999 [(survey* or questionnair*) and (return* or respon*Social Psychological Educational Criminological Trials Register (SPECTR) 1950 -1998 EconLit 1969 -2000 Sociological Abstracts 1963 -2000 ((survey$ or questionn$) and (return$ or respon$)).ti or ((survey$ or questionn$) and (mail$ or post$).Technical Proceedings 1982 -2000 ((survey*, questionn*),
- Armstrong JS. Monetary incentives in mail surveys. Public Opinion Quarterly 1975;39:111-6.
- • Armstrong S. Return postage in mail surveys: a meta analysis. Public Opinion Quarterly 1987;51:233-48.
- • Bogen K. The effects of questionnaire length on response rates -a review of the literature. American Statistical Association 1996; 1020-5.
- • Brehm J. Stubbing our toes for a foot in the door? Prior contact, incentives and survey response. International Journal of Public Opinion Research 1994;6(1):45-63.
- • Church AH. Estimating the effect of incentives on mail survey response rates: a meta-analysis. Public Opinion Quarterly 1993; 57:62-79.
- • Cox WE. Response patterns to mail surveys. Journal of Marketing Research 1966;3:392-7. • Downs PE. Recent evidence on the relationship between anonymity and response variables for mail surveys. Journal of the Academy of Marketing Science 1986;14(1):72-82.
- • Duncan WJ. Mail questionnaires in survey research: a review of response inducement techniques. Journal of Management 1979; 5(1):39-55.
- • Erdos PL. Visible vs. disguised keying on questionnaires. Journal of Advertising Research 1977;17(1):13-8.
- • Fox RJ. Mail survey response rate. A meta-analysis of selected techniques for inducing response. Public Opinion Quarterly 1988; 52:467-91.
- • Francel EG. Mail-administered questionnaires: a success story. Journal of Marketing Research 1966;3:89-92.
- • Goyder JC. Further evidence on factors affecting response rates to mailed questionnaires. American Sociological Review 1982; 47:550-3.
- • Green KE. Reviewing the research on mail survey response rates: a meta-analysis. Paper presented at the Annual Meeting of the American Educational Research Association. April 1996. • Greenwald HP. Issues in survey data on medical practice: some empirical comparisons. Public Health Reports 1986;101(5):514- 46.
- • Guffey H. Stamps versus postal permits: a decisional guide for return postage in mail questionnaires. Journal of Academy of Marketing Science 1980;8(3): 234-42.
- • Harvey L. Factors affecting response rates to mailed questionnaires: a comprehensive literature review. Journal of the Market Research Society 1987;29:341-53.
- • Heberlein TA. Factors affecting response rates to mailed questionnaires. American Sociological Review 1978;43(4):447-62.
- • Hopkins KD. Response rates in survey research: a meta-analysis of the effects of monetary gratuities. Journal of Experimental Education 1992;61:52-62. CINAHL 2003 -2007.12 ERIC 2003 -2007.12 PsycINFO 2003 -2008.01 MEDLINE 2003 -2007.11 EMBASE 2003 -2007.10
- Science Citation Index 2003 -2008.01 Social Science Citation Index 2003 -2008.01 Social Psychological Educational Criminological Trials Register (SPECTR) 2003 -2007.12 EconLit 2003 -2007.12 Sociological Abstracts 2003 -2007.12 Dissertation & Theses 2003 -2008.01
- Index to Scientific & Technical Proceedings 2003 -2008.01
Cited by
Sorted by citations
Fogel, D. B. (2018) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2018.08.001 · Cited by 1,035 · see citing works
Strategies to improve recruitment to randomised trials
Treweek, S., Pitkethly, M., Cook, J., Fraser, C., Mitchell, E., Sullivan, F., Jackson, C., Taskila, T. K., & Gardner, H. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000013.pub6 · Cited by 521 · see citing works
Bower, P., Brueton, V., Gamble, C., Treweek, S., Smith, C. T., Young, B., & Williamson, P. (2014) · Trials · doi:10.1186/1745-6215-15-399 · Cited by 369 · see citing works
Physician Confidence in Artificial Intelligence: An Online Mobile Survey
Oh, S., Kim, J. H., Choi, S.-W., Lee, H. J., Hong, J., & Kwon, S. H. (2019) · Journal of Medical Internet Research · doi:10.2196/12422 · Cited by 387 · see citing works
Strategies to improve recruitment to randomised controlled trials
Treweek, S., Mitchell, E., Pitkethly, M., Cook, J., Kjeldstrøm, M., Johansen, M., Taskila, T. K., Sullivan, F., Wilson, S., Jackson, C., et al. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000013.pub5 · Cited by 378 · see citing works
Finch, T. L., Girling, M., May, C. R., Mair, F. S., Murray, E., Treweek, S., McColl, E., Steen, I. N., Cook, C., Vernazza, C. R., et al. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0591-x · Cited by 337 · see citing works
Strategies to improve retention in randomised trials: a Cochrane systematic review and meta-analysis
Brueton, V. C., Tierney, J. F., Stenning, S., Meredith, S., Harding, S., Nazareth, I., & Rait, G. (2014) · BMJ Open · doi:10.1136/bmjopen-2013-003821 · Cited by 253 · see citing works
Scott, A., Jeon, S.-H., Joyce, C. M., Humphreys, J. S., Kalb, G., Witt, J., & Leahy, A. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-126 · Cited by 244 · see citing works
Khazaal, Y., van Singer, M., Chatton, A., Achab, S., Zullino, D., Rothen, S., Khan, R., Billieux, J., & Thorens, G. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.2759 · Cited by 201 · see citing works
Finch, T. L., Rapley, T., Girling, M., Mair, F. S., Murray, E., Treweek, S., McColl, E., Steen, I. N., & May, C. R. (2013) · Implementation Science · doi:10.1186/1748-5908-8-43 · Cited by 177 · see citing works
The Cyberchondria Severity Scale (CSS): German Validation and Development of a Short Form
Barke, A., Bleichhardt, G., Rief, W., & Doering, B. K. (2016) · International Journal of Behavioral Medicine · doi:10.1007/s12529-016-9549-8 · Cited by 177 · see citing works
Variations in Clinical PET/CT Operations: Results of an International Survey of Active PET/CT Users
Beyer, T., Czernin, J., & Freudenberg, L. S. (2011) · Journal of Nuclear Medicine · doi:10.2967/jnumed.110.079624 · Cited by 136 · see citing works
Intention‐to‐treat in randomized controlled trials: Recommendations for a total trial strategy
Polit, D. F., & Gillespie, B. M. (2010) · Research in Nursing & Health · doi:10.1002/nur.20386 · Cited by 129 · see citing works
Wilson, G. B., Lock, C. A., Heather, N., Cassidy, P., Christie, M. M., & Kaner, E. F. S. (2011) · Alcohol and Alcoholism · doi:10.1093/alcalc/agr067 · Cited by 112 · see citing works
Skromanis, S., Cooling, N., Rodgers, B., Purton, T., Fan, F., Bridgman, H., Harris, K., Presser, J., & Mond, J. (2018) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph15061147 · Cited by 113 · see citing works
Khadjesari, Z., Murray, E., Kalaitzaki, E., White, I. R., McCambridge, J., Thompson, S. G., Wallace, P., & Godfrey, C. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1523 · Cited by 114 · see citing works
Fazel, M. T., Bagalagel, A., Lee, J. K., Martin, J. R., & Slack, M. K. (2017) · Annals of Pharmacotherapy · doi:10.1177/1060028017711454 · Cited by 99 · see citing works
Use and Views on Social Networking Sites of Pharmacy Students in the United Kingdom
Hall, M., Hanna, L.-A., & Huey, G. (2013) · American Journal of Pharmaceutical Education · doi:10.5688/ajpe7719 · Cited by 84 · see citing works
Weymann, N., Dirmaier, J., von Wolff, A., Kriston, L., & Härter, M. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3904 · Cited by 73 · see citing works
The Development and Validation of the Short Cyberchondria Scale (SCS)
Jokić-Begić, N., Mikac, U., Čuržik, D., & Sangster Jokić, C. (2019) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-019-09744-z · Cited by 82 · see citing works
Rasmussen, K., Bero, L., Redberg, R., Gøtzsche, P. C., & Lundh, A. (2018) · BMJ · doi:10.1136/bmj.k3654 · Cited by 74 · see citing works
Barriers to effective diabetes management–a survey of people with severe mental illness
Mulligan, K., McBain, H., Lamontagne-Godwin, F., Chapman, J., Flood, C., Haddad, M., Jones, J., & Simpson, A. (2018) · BMC Psychiatry · doi:10.1186/s12888-018-1744-5 · Cited by 68 · see citing works
Weston, P. S. J., & Smith, C. A. (2014) · Medical Teacher · doi:10.3109/0142159x.2013.836267 · Cited by 67 · see citing works
Sivertsen, B., Knapstad, M., Petrie, K., O’Connor, R., Lønning, K. J., & Hysing, M. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-057492 · Cited by 49 · see citing works
“9 Still, findings remain inconsistent, as a living systematic review of longitudinal studies does not suggest significant change in mental health among university students from before to during the COVID-19.”
BackgroundVlooswijk, C., Poll-Franse, L. V. V. D., Janssen, S. H. M., Derksen, E., Reuvers, M. J. P., Bijlsma, R., Kaal, S. E. J., Kerst, J. M., Tromp, J. M., Bos, M. E. M. M., et al. (2022) · Current Oncology · doi:10.3390/curroncol29080428 · Cited by 39 · see citing works
Mitchell, N., Hewitt, C. E., Lenaghan, E., Platt, E., Shepstone, L., & Torgerson, D. J. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.05.008 · Cited by 36 · see citing works
A theory-based online health behavior intervention for new university students: study protocol
Epton, T., Norman, P., Sheeran, P., Harris, P. R., Webb, T. L., Ciravegna, F., Brennan, A., Meier, P., Julious, S. A., Naughton, D., et al. (2013) · BMC Public Health · doi:10.1186/1471-2458-13-107 · Cited by 34 · see citing works
Vandelanotte, C., Short, C. E., Plotnikoff, R. C., Rebar, A., Alley, S., Schoeppe, S., Canoy, D. F., Hooker, C., Power, D., Oldmeadow, C., et al. (2021) · British Journal of Sports Medicine · doi:10.1136/bjsports-2020-102521 · Cited by 31 · see citing works
Prince, A. C., Moosa, A., Lomer, M. C. E., Reidlinger, D. P., & Whelan, K. (2015) · Health Expectations · doi:10.1111/hex.12219 · Cited by 29 · see citing works
Use of lethal means restriction counseling for suicide prevention in pediatric primary care
Bandealy, A., Herrera, N., Weissman, M., & Scheidt, P. (2020) · Preventive Medicine · doi:10.1016/j.ypmed.2019.105855 · Cited by 25 · see citing works
Lange, L., Fink, J., Bleich, C., Graefen, M., & Schulz, H. (2017) · Internet Interventions · doi:10.1016/j.invent.2017.06.001 · Cited by 22 · see citing works
Treweek, S., Ricketts, I. W., Francis, J., Eccles, M., Bonetti, D., Pitts, N. B., MacLennan, G., Sullivan, F., Jones, C., Weal, M., et al. (2011) · Implementation Science · doi:10.1186/1748-5908-6-16 · Cited by 14 · see citing works
Lynch, T., Ryan, C., Huff, C., Foster, D. E., & Cadogan, C. (2024) · Health Expectations · doi:10.1111/hex.13962 · Cited by 11 · see citing works
- Authors
- Edwards, P. J., Roberts, I., Clarke, M. J., DiGuiseppi, C., Wentz, R., Kwan, I., Cooper, R., Felix, L. M., & Pratap, S.
- Affiliations
- London School of Hygiene & Tropical Medicine; Department of Epidemiology and Population Health; Keppel Street London UK WC1E 7HT
- London School of Hygiene & Tropical Medicine (GB)
- National Institute for Health and Care Research (GB)
- Colorado School of Public Health (US)
- Chelsea and Westminster Hospital (GB)
- Royal College of Obstetricians and Gynaecologists (GB)
- Redhill, Reigate & Horley PCMHT; Blackborough Road Reigate UK RH2 7DG
- ORCID
- Venue
- Cochrane Database of Systematic Reviews
- Publisher
- Wiley
- Published
- 2009-07-08
- Volume
- 2010
- Issue
- 1
- Pages
- MR000008-MR000008
- ISSN
- 1361-6137 (ISSN-L)
- 1465-1858
- PMC
- PMC8941848
- OpenAlex
- W1566155680
- Cited by
- 1,379
- References
- 1
- Citations by year
- 2026: 10 · 2025: 52 · 2024: 73 · 2023: 80 · 2022: 95 · 2021: 84 · 2020: 108 · 2019: 90 · 2018: 86 · 2017: 91 · 2016: 94 · 2015: 110 · 2014: 105 · 2013: 88 · 2012: 88
- FWCI
- 33.83
- Type
- journal article
- Language
- en
- Update notices
- new_version (2023-11-30)
- Open access
- bronze
- doi.org
- Locations
- Cochrane Database of Systematic Reviews (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Research Portal (Queen's University Belfast) (submittedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Physical Sciences › Mathematics › Statistics and Probability › Survey Sampling and Estimation Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- questionnaire response rate, postal questionnaires, electronic questionnaires, nonresponse, financial incentives, pre-notification, questionnaire design, allocation concealment, selection bias, blinding
- MeSH terms
- Surveys and Questionnaires, Reward, Correspondence as Topic, Randomized Controlled Trials as Topic, Reminder Systems, Electronic Mail
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
