The DOI matches its Crossref record; 30 of the 31 references it cites were found in a registry or an index.
Local Standards for Sample Size at CHI
Abstract
We describe the primary ways researchers can determine the size of a sample of research participants, present the benefits and drawbacks of each of those methods, and focus on improving one method that could be useful to the CHI community: local standards. To determine local standards for sample size within the CHI community, we conducted an analysis of all manuscripts published at CHI2014. We find that sample size for manuscripts published at CHI ranges from 1 -- 916,000 and the most common sample size is 12. We also find that sample size differs based on factors such as study setting and type of methodology employed. The outcome of this paper is an overview of the various ways sample size may be determined and an analysis of local standards for sample size within the CHI community. These contributions may be useful to researchers planning studies and reviewers evaluating the validity of results.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Still, these numbers seem quite low compared with the average number of participants in human-computer interaction experiments ( ¯ c = 20, s = 12) reported by Caine [96].”
From A Review of Design and Evaluation Practices in Mobile Text Entry for Visually Impaired and Blind Persons (2023) · cites this paper as [96]
In the citing paper
“However, our sample size is closer to what is reported in HCI studies for qualitative research [9] and prior work exploring similar topics [53, 71].”
From How University Disability Services Professionals Write Image Descriptions for HCI Figures Using Generative AI (2026) · cites this paper as [9]
Citations
Cites
The Discovery of Grounded Theory: Strategies for Qualitative Research.
Wagner, H. R., Glaser, B. G., & Strauss, A. L. (1968) · Social Forces · doi:10.2307/2575405 · Cited by 46,361
The weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641
WEIRD languages have misled us, too
Majid, A., & Levinson, S. C. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x1000018x · Cited by 4,076
Ellis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824
Scientific method: Statistical errors
Nuzzo, R. (2014) · Nature · doi:10.1038/506150a · Cited by 1,536
Eby, L. T. D. T., Hurst, C. S., & Butts, M. M. (2008) · Cited by 926
(2011) · Choice Reviews Online · doi:10.5860/choice.48-5742 · Cited by 626
Usability evaluation considered harmful (some of the time)
Greenberg, S., & Buxton, B. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357074 · Cited by 598
Estimating the number of subjects needed for a thinking aloud test
Nielsen, J. (1994) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1994.1065 · Cited by 476
Research works printed but not linked: 6
- ACM SIGCHI 2015. Guide to a Successful Paper or Note Submission.
- ACM SIGCHI 2015. Papers Versus Notes Whats the Difference.
- 10.1186/1741--7015--8--17 · doi:10.1186/1741--7015--8--17
- 10.5555/1531294.1531321 · doi:10.5555/1531294.1531321
- Jenny Preece, Helen Sharp, and Yvonne Rogers. 2015. Interaction Design-beyond human-computer interaction. John Wiley & Sons.
- Albrecht Schmidt and Tovi Grossman. 2014. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems ACM, Toronto, Ontario, Canada, 4206.
Cited by
Vasileiou, K., Barnett, J., Thorpe, S., & Young, T. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0594-7 · Cited by 3,115
“Other work has sought to examine practices of sample size reporting and sufficiency assessment across a range of disciplinary fields and research domains, from nutrition [34] and health education [32], to education and the health sciences [22, 27], information systems [30], organisation and workplace studies [33], hum…”
BackgroundMcDonald, N., Schoenebeck, S., & Forte, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359174 · Cited by 1,206
Intersectional HCI: Engaging Identity through Gender, Race, and Class
Schlesinger, A., Edwards, W. K., & Grinter, R. E. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025766 · Cited by 449
The Experience Sampling Method on Mobile Devices
van Berkel, N., Ferreira, D., & Kostakos, V. (2018) · ACM Computing Surveys · doi:10.1145/3123988 · Cited by 432
Sampling in software engineering research: a critical review and guidelines
Baltes, S., & Ralph, P. (2022) · Empirical Software Engineering · doi:10.1007/s10664-021-10072-8 · Cited by 382
Metadata
- Authors
- Caine, K.
- Affiliations
- Clemson University (US)
- Corresponding author
- Caine, Kelly
- ORCID
- Caine, Kelly
- Venue
- Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2016-05-07
- Pages
- 981-992
- OpenAlex
- W2407300725
- Cited by
- 657
- References
- 31
- Citations by year
- 2026: 94 · 2025: 159 · 2024: 91 · 2023: 78 · 2022: 69 · 2021: 62 · 2020: 40 · 2019: 34 · 2018: 17 · 2017: 11 · 2016: 2
- FWCI
- 63.97
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- TigerPrints (Clemson University) (submittedVersion)
- Funders
- NSF (1513875, 1314342, 1117860)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- sample size determination, local standards, HCI research, statistical power, research design, research validity, peer review
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Local Standards for Sample Size at CHI. https://app.cochat.ai/papers/10.1145/2858036.2858498
- The paper
- Caine, K. (2016). Local Standards for Sample Size at CHI. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/2858036.2858498
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.