Modularization in an Era of Mobile Web: Investigating the Effects of Cutting a Survey Into Smaller Pieces on Data Quality
Open accessAbstract
With the rise of mobile surveys comes the need for shorter questionnaires. We investigate the modularization of an existing questionnaire in the Longitudinal Internet Study for the Social Sciences (LISS) Panel in the Netherlands. We randomly divided respondents into a normal length survey condition, a condition where the same survey was split into 3 parts, and a condition where the survey was split into 10 parts. Respondents received the parts consecutively at regular intervals over a 1-month period. We discuss response rates, data quality measures, and respondents’ evaluation of the questionnaire. Our results indicate higher start rates when the survey is cut into smaller parts but also higher dropout rates. However, the fraction of missing information is lower in the 3- and 10-part conditions. More respondents use their mobile phone for survey completion when the survey is shorter. We find fewer item missings and satisficing in shorter surveys. We find no effect on neutral and extreme responding nor on estimates of the validity of answers. People with low and high education and young and old evaluate shorter surveys better than the normal length survey.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Modularization of longer questionnaires in shorter ones in mobile surveys does not reduce data quality and can even lead to less missing data and reduce satisficing.
“Modularization of longer questionnaires in shorter ones in mobile surveys does not reduce data quality and can even lead to less missing data and reduce satisficing (Toepoel and Lugtig, 2018).”
From Context Stability in Habit Building Increases Automaticity and Goal Attainment (2022) · cites this paper as Toepoel and Lugtig, 2018
Cites
Flexible Imputation of Missing Data
van Buuren, S. (2012) · doi:10.1201/b11826 · Cited by 2,022 · see citing works
Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey
Galešić, M., & Bošnjak, M. (2009) · Public Opinion Quarterly · doi:10.1093/poq/nfp031 · Cited by 1,324 · see citing works
Heberlein, T. A., & Baumgartner, R. (1978) · American Sociological Review · doi:10.2307/2094771 · Cited by 999 · see citing works
Thiel, D. V. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9781139542326.008 · Cited by 792 · see citing works
Understanding Mail Survey Response Behavior: A Meta-Analysis
Yammarino, F. J., Skinner, S. J., & Childers, T. L. (1991) · Public Opinion Quarterly · doi:10.1086/269284 · Cited by 566 · see citing works
Computing Response Metrics for Online Panels
Callegaro, M., & DiSogra, C. (2008) · Public Opinion Quarterly · doi:10.1093/poq/nfn065 · Cited by 516 · see citing works
Total Survey Error: Design, Implementation, and Evaluation
Biemer, P. P. (2010) · Public Opinion Quarterly · doi:10.1093/poq/nfq058 · Cited by 452 · see citing works
Data Quality in PC and Mobile Web Surveys
Mavletova, A. (2013) · Social Science Computer Review · doi:10.1177/0894439313485201 · Cited by 196 · see citing works
A Split Questionnaire Survey Design
Raghunathan, T. E., & Grizzle, J. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476488 · Cited by 178 · see citing works
Peytchev, A. (2009) · Public Opinion Quarterly · doi:10.1093/poq/nfp014 · Cited by 131 · see citing works
The Use of PCs, Smartphones, and Tablets in a Probability-Based Panel Survey
Lugtig, P. J., & Toepoel, V. (2015) · Social Science Computer Review · doi:10.1177/0894439315574248 · Cited by 127 · see citing works
Self-citationComparison of Smartphone and Online Computer Survey Administration
Wells, T., Bailey, J. T., & Link, M. (2013) · Social Science Computer Review · doi:10.1177/0894439313505829 · Cited by 116 · see citing works
Couper, M. P., Antoun, C., & Mavletova, A. (2017) · doi:10.1002/9781119041702.ch7 · Cited by 103 · see citing works
Mobile Research Methods: Opportunities and challenges of mobile research methodologies
Toninelli, D., Pintér, R., & de Pedraza, P. (2015) · Ubiquity Press eBooks · doi:10.5334/bar · Cited by 98 · see citing works
Toepoel, V., & Lugtig, P. J. (2014) · Social Science Computer Review · doi:10.1177/0894439313510482 · Cited by 88 · see citing works
Self-citationDo online access panels need to adapt surveys for mobile devices?
Revilla, M., Toninelli, D., Ochoa, C., & Loewe, G. (2016) · Internet Research · doi:10.1108/intr-02-2015-0032 · Cited by 86 · see citing works
The Fraction of Missing Information as a Tool for Monitoring the Quality of Survey Data
Wagner, J. R. (2010) · Public Opinion Quarterly · doi:10.1093/poq/nfq007 · Cited by 85 · see citing works
Improving Response Rates and Questionnaire Design for Mobile Web Surveys
de Bruijne, M., & Wijnant, A. (2014) · Public Opinion Quarterly · doi:10.1093/poq/nfu046 · Cited by 84 · see citing works
Making Mobile Browser Surveys Smarter
Buskirk, T. D., & Andrus, C. H. (2014) · Field Methods · doi:10.1177/1525822x14526146 · Cited by 68 · see citing works
Split Questionnaire Design for Massive Surveys
Adıgüzel, F., & Wedel, M. (2008) · Journal of Marketing Research · doi:10.1509/jmkr.45.5.608 · Cited by 61 · see citing works
A Meta-Analysis of Breakoff Rates in Mobile Web Surveys
(2015) · Ubiquity Press eBooks · doi:10.5334/bar.f · Cited by 42 · see citing works
Mobile-only web survey respondents
Lugtig, P. J., Toepoel, V., & Amin, A. (2016) · Survey Practice · doi:10.29115/sp-2016-0020 · Cited by 40 · see citing works
Self-citationPanel Conditioning in Difficult Attitudinal Questions
Binswanger, J., Schunk, D., & Toepoel, V. (2013) · Public Opinion Quarterly · doi:10.1093/poq/nft030 · Cited by 36 · see citing works
Self-citationEvaluating a Modular Design Approach to Collecting Survey Data Using Text Messages
West, B. T., Ghimire, D. J., & Axinn, W. G. (2015) · Survey research methods · doi:10.18148/srm/2015.v9i2.6135 · Cited by 34 · see citing works
Can Mobile Web Surveys Be Taken on Computers? A Discussion on a Multi-Device Survey Design
de Bruijne, M., & Wijnant, A. (2013) · Survey Practice · doi:10.29115/sp-2013-0019 · Cited by 34 · see citing works
Longer Interviews May Not Affect Subsequent Survey Participation Propensity
Lynn, P. J. (2014) · Public Opinion Quarterly · doi:10.1093/poq/nfu015 · Cited by 29 · see citing works
Design and Estimation for Split Questionnaire Surveys
Chipperfield, J. O., & Steel, D. G. (2009) · Research Online (University of Wollongong) · Cited by 22 · see citing works
Poggio, T., Bošnjak, M., & Weyandt, K. W. (2015) · Survey Practice · doi:10.29115/sp-2015-0002 · Cited by 20 · see citing works
Roberts, C., Eva, G., Allum, N., & Lynn, P. J. (2010) · Open Access at Essex (University of Essex) · Cited by 17 · see citing works
On a Modular Approach to the Design of Integrated Social Surveys
Ioannidis, E. E., Merkouris, T., Zhang, L., Karlberg, M., Petrakos, M., Reis, F., & Stavropoulos, P. (2016) · Journal of Official Statistics · doi:10.1515/jos-2016-0013 · Cited by 12 · see citing works
Bansal, H. S., Eldridge, J., Halder, A., Knowles, R., Murray, M. D., Sehmer, L., & Turner, D. (2017) · International Journal of Market Research · doi:10.2501/ijmr-2017-016 · Cited by 6 · see citing works
How to run long web surveys: a real-life experiment with the European Values Study
Pollien, A., Stähli, M. E., Milbert, P., Ochsner, M., & Joye, D. (2018) · doi:10.13140/rg.2.2.23811.04647 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
te Pas, M. E., Rutten, W. G. M. M., Bouwman, R. A., & Buise, M. P. (2020) · JMIR Medical Informatics · doi:10.2196/21982 · Cited by 43 · see citing works
Using Smartphone Technology for Research on Refugees: Evidence from Germany
Keusch, F., Leonard, M. M., Sajons, C., & Steiner, S. (2019) · Sociological Methods & Research · doi:10.1177/0049124119852377 · Cited by 32 · see citing works
Stojanovic, M., Fries, S., & Grund, A. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.643753 · Cited by 28 · see citing works
Developing and validating a multidisciplinary scale of E-retailing website elements
Alnawas, I. A., & Khateeb, A. A. (2022) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2021.102905 · Cited by 29 · see citing works
How to Design, Implement, and Analyse a Survey
Arundel, A. V. (2023) · Edward Elgar Publishing eBooks · doi:10.4337/9781800376175 · Cited by 25 · see citing works
West, B. T., Zhang, S., Wagner, J. R., Gatward, R., Saw, H.-W., & Axinn, W. G. (2023) · PLoS ONE · doi:10.1371/journal.pone.0289695 · Cited by 22 · see citing works
Context Stability in Habit Building Increases Automaticity and Goal Attainment
Stojanovic, M., Grund, A., & Fries, S. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.883795 · Cited by 22 · see citing works
“Modularization of longer questionnaires in shorter ones in mobile surveys does not reduce data quality and can even lead to less missing data and reduce satisficing (Toepoel and Lugtig, 2018).”
BackgroundSilber, H., Moy, P., Johnson, T. P., Neumann, R., Stadtmüller, S., & Repke, L. (2022) · Social Science Quarterly · doi:10.1111/ssqu.13218 · Cited by 20 · see citing works
Development and validation of a measure of curvy ideals internalization
Walker, D. C., Gaither, S. E., Santos, B. D. L., Keigan, J., Schaefer, L. M., & Thompson, J. K. (2022) · Body Image · doi:10.1016/j.bodyim.2022.09.005 · Cited by 18 · see citing works
Preventing satisficing: A narrative review
Blazek, D. R., & Siegel, J. T. (2023) · International Journal of Social Research Methodology · doi:10.1080/13645579.2023.2239086 · Cited by 16 · see citing works
Building on a Sequential Mixed-Mode Research Design in the Monitoring the Future Study
Patrick, M. E., Couper, M. P., Jang, J. B., Laetz, V. B., Schulenberg, J. E., O’Malley, P. M., Bachman, J. G., & Johnston, L. D. (2020) · Journal of Survey Statistics and Methodology · doi:10.1093/jssam/smaa030 · Cited by 15 · see citing works
The Effect of Question Positioning on Data Quality in Web Surveys
Neuert, C. E. (2021) · Sociological Methods & Research · doi:10.1177/0049124120986207 · Cited by 13 · see citing works
Recent Innovations and Advances in Mixed-Mode Surveys
Coffey, S., Maslovskaya, O., & McPhee, C. B. (2024) · Journal of Survey Statistics and Methodology · doi:10.1093/jssam/smae025 · Cited by 12 · see citing works
Lower‐Hoppe, L. M., Aicher, T. J., & Baker, B. J. (2022) · World Leisure Journal · doi:10.1080/16078055.2022.2125572 · Cited by 13 · see citing works
Cabrera‐Álvarez, P., & Lynn, P. J. (2024) · Journal of Survey Statistics and Methodology · doi:10.1093/jssam/smae003 · Cited by 10 · see citing works
Breakoffs in an hour-long, online survey
Emery, T., Cabaço, S. L. F., Fadel, L., Lugtig, P. J., Toepoel, V., Schumann, A., Lück, D., & Bujard, M. (2023) · Survey Practice · doi:10.29115/sp-2023-0008 · Cited by 8 · see citing works
Self-citationRoberts, C., Herzing, J. M. E., Manjon, M. A., Abbet, P., & Gática-Pérez, D. (2022) · Journal of Official Statistics · doi:10.2478/jos-2022-0043 · Cited by 8 · see citing works
Iglesias, P. A., & Revilla, M. (2023) · Quality & Quantity · doi:10.1007/s11135-023-01670-3 · Cited by 8 · see citing works
Brenner, P. S., Hargraves, J. L., & Cosenza, C. A. (2022) · Journal of Official Statistics · doi:10.2478/jos-2022-0042 · Cited by 4 · see citing works
Optimal Integrating Learning for Split Questionnaire Design Type Data
Lin, C., Peng, J., Qin, Y., Li, Y., & Yang, Y. (2022) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2022.2118753 · Cited by 3 · see citing works
Die KODAQS-Tools – Eine neue Ressource zur Beurteilung der Qualität von Forschungsdaten
Peters, Y., Krämer, F., Stroppe, A.-K., Sun, J., Daikeler, J., Yıldız, Ç., Dehne, J., Silber, H., Jünger, S., Weller, K., et al. (2026) · Publizistik · doi:10.1007/s11616-026-00941-x · Cited by 2 · see citing works
Text Messages to Incentivise Response in a Web-First Sequential Mixed-Mode Survey
Cabrera‐Álvarez, P., & Lynn, P. J. (2023) · Social Science Computer Review · doi:10.1177/08944393231213315 · Cited by 2 · see citing works
Ong, A. R., Schultz, R., Sinozich, S., Sinibaldi, J., West, B. T., Wagner, J. R., & Finamore, J. (2022) · Journal of Official Statistics · doi:10.2478/jos-2022-0052 · Cited by 2 · see citing works
Jia, J. (2024) · Journal of Computational Methods in Sciences and Engineering · doi:10.1177/14727978241299183 · Cited by 1 · see citing works
Comparing single-sitting versus modular text message surveys in Egypt
Haas, G., Keusch, F., & Frölich, M. (2022) · International Journal of Public Opinion Research · doi:10.1093/ijpor/edac023 · Cited by 1 · see citing works
Developing a Modular Survey App Using Co-design Principles
Antoun, C., Yang, X., West, B. T., & Ong, A. R. (2024) · Field Methods · doi:10.1177/1525822x241260994 · Cited by 1 · see citing works
Smíšený mód a online dotazování: mezinárodní praxe a tuzemská připravenost na implementaci
Hubatková, B., & Lakomý, M. (2021) · Sociální studia / Social Studies · doi:10.5817/soc2021-1-13 · Cited by 1 · see citing works
Biffignandi, S., & Bethlehem, J. G. (2021) · doi:10.1002/9781119371717.ch9
Arundel, A. V. (2023) · Edward Elgar Publishing eBooks · doi:10.4337/9781800376175.00013
Antoun, C., West, B. T., Yang, X., Zaidi, S. J. M. A., & Sinibaldi, J. (2025) · Journal of Survey Statistics and Methodology · doi:10.1093/jssam/smaf008
- Authors
- Toepoel, V., & Lugtig, P.
- Affiliations
- Utrecht University (NL)
- Corresponding author
- Toepoel, Vera
- Venue
- Social Science Computer Review
- Publisher
- SAGE Publications
- Published
- 2018-07-01
- Volume
- 40
- Issue
- 1
- Pages
- 150-164
- ISSN
- 0894-4393 (ISSN-L)
- 1552-8286
- OpenAlex
- W2809732581
- Cited by
- 30
- References
- 32
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 4 · 2023: 7 · 2022: 10 · 2021: 4 · 2020: 2 · 2019: 1
- FWCI
- 3.98
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Social Science Computer Review (publishedVersion, cc-by-nc)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Physical Sciences › Mathematics › Statistics and Probability › Survey Sampling and Estimation Techniques
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Keywords
- modularization, mobile web surveys, questionnaire length, mobile surveys, response rate, dropout, data quality, satisficing, missing data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
