Effect of experience sampling schedules on response rate and recall accuracy of objective self-reports
The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
The Experience Sampling Method is widely used to collect human labelled data in the wild. Using this methodology, study participants repeatedly answer a set of questions, constructing a rich overview of the studied phenomena. One of the methodological decisions faced by researchers is deciding on the question scheduling. The literature defines three distinct schedule types: randomised, interval-based, or event-based (in our case, smartphone unlock). However, little evidence exists regarding the side-effects of these schedules on response rate and recall accuracy, and how they may bias study findings. We evaluate the effect of these three contingency configurations in a 3-week within-subjects study (N = 20). Participants answered various objective questions regarding their phone usage, while we simultaneously establish a ground-truth through smartphone instrumentation. We find that scheduling questions on phone unlock yields a higher response rate and accuracy. Our study provides empirical evidence for the effects of notification scheduling on participant responses, and informs researchers who conduct experience sampling studies on smartphones.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490 · cited 1×
“The use of p - values has faced increased scrutiny ( e.g. , (Kaptein and Robertson, 2012; Wagenmakers, 2007)), mainly due to the incorrect interpretation and application by researchers.”
MethodDefault Bayes factors for ANOVA designs
Rouder, J. N., Morey, R. D., Speckman, P. L., & Province, J. M. (2012) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2012.08.001 · Cited by 1,800 · cited 1×
“Given our within-subjects design, our analysis consists mainly of ‘Bayesian ANOVA’, as described in (Rouder et al., 2012).”
MethodFlow and the Foundations of Positive Psychology: The Collected Works of Mihaly Csikszentmihalyi
Csikszentmihalyi, M. (2014) · doi:10.1007/978-94-017-9088-8 · Cited by 1,653
Online Social Network Site Addiction: A Comprehensive Review
Andreassen, C. S. (2015) · Current Addiction Reports · doi:10.1007/s40429-015-0056-9 · Cited by 1,212 · cited 1×
“Usage of smartphones has been shown to be far from an emotionally neutral activity ( e.g. , smartphone overuse (Lee et al., 2014)), especially when it concerns the use of social media networks (Andreassen, 2015).”
BackgroundThe Experience Sampling Method
Larson, R., & Csikszentmihalyi, M. (2014) · Flow and the Foundations of Positive Psychology · doi:10.1007/978-94-017-9088-8_2 · Cited by 1,034
Local Standards for Sample Size at CHI
Caine, K. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858498 · Cited by 657
Böhmer, M., Hecht, B., Schöning, J., Krüger, A., & Bauer, G. (2011) · Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services · doi:10.1145/2037373.2037383 · Cited by 604 · cited 3×
“This difference can be accounted for by the frequent (but brief duration) of smartphone usage, as has been reported in the literature (Böhmer et al., 2011; Ferreira et al., 2014).”
BackgroundThe Experience Sampling Method on Mobile Devices
van Berkel, N., Ferreira, D., & Kostakos, V. (2018) · ACM Computing Surveys · doi:10.1145/3123988 · Cited by 432
Self-citationAWARE: Mobile Context Instrumentation Framework
Ferreira, D., Kostakos, V., & Dey, A. K. (2015) · Frontiers in ICT · doi:10.3389/fict.2015.00006 · Cited by 409 · cited 1×
“Using this application, based on the AWARE mobile instrumentation framework (Ferreira et al., 2015), we collect not only the participants’ responses but also log, inter alia, the following data: phone usage, application usage, ESM answer, and ESM question status (i.”
MethodSelf-citationBeyond self-report: tools to compare estimated and real-world smartphone use
Andrews, S., Ellis, D. A., Shaw, H., & Piwek, L. (2015) · PLOS ONE · doi:10.1371/journal.pone.0139004 · Cited by 408 · cited 2×
“…changing throughout the day), duration of smartphone usage (both diverse across users (Falaki et al., 2010) and challenging for users to estimate accurately (Andrews et al., 2015)), and number of times the phone was turned on (challenging for users to accurately estimate (Andrews et al., 2015)).”
Background
Cited by
Parry, D. A., Davidson, B. I., Sewall, C. J. R., Fisher, J. T., Mieczkowski, H., & Quintana, D. S. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01117-5 · Cited by 793
Wrzus, C., & Neubauer, A. B. (2023) · Assessment · doi:10.1177/10731911211067538 · Cited by 500
Eisele, G., Vachon, H., Lafit, G., Kuppens, P., Houben, M., Myin-Germeys, I., & Viechtbauer, W. (2022) · Assessment · doi:10.1177/1073191120957102 · Cited by 459
The accuracy and validity of self-reported social media use measures among adolescents
Verbeij, T., Pouwels, J. L., Beyens, I., & Valkenburg, P. M. (2021) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2021.100090 · Cited by 163
Meinhardt, L.-M., Elhaidary, M., Colley, M., Rietzler, M., Rixen, J. O., Purohit, A. K., & Rukzio, E. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713187 · Cited by 16
Metadata
- Authors
- van Berkel, N., Goncalves, J., Lovén, L., Ferreira, D., Hosio, S., & Kostakos, V.
- Venue
- International Journal of Human-Computer Studies
- Publisher
- Elsevier BV
- Published
- 2019-05
- Volume
- 125
- Pages
- 118-128
- ISSN
- 1071-5819 (ISSN-L)
- OpenAlex
- W2902009635
- Cited by
- 92
- References
- 14
- FWCI
- 10.47
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Effect of experience sampling schedules on response rate and recall accuracy of objective self-reports. https://app.cochat.ai/papers/10.1016/j.ijhcs.2018.12.002
- The paper
- van Berkel, N., Goncalves, J., Lovén, L., Ferreira, D., Hosio, S., & Kostakos, V. (2019). Effect of experience sampling schedules on response rate and recall accuracy of objective self-reports. International Journal of Human-Computer Studies. https://doi.org/10.1016/j.ijhcs.2018.12.002
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.