Researcher-Centered Design of Statistics: Why Bayesian Statistics Better Fit the Culture and Incentives of HCI
The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
A core tradition of HCI lies in the experimental evaluation of the effects of techniques and interfaces to determine if they are useful for achieving their purpose. However, our individual analyses tend to stand alone, and study results rarely accrue in more precise estimates via meta-analysis: in a literature search, we found only 56 meta-analyses in HCI in the ACM Digital Library, 3 of which were published at CHI (often called the top HCI venue). Yet meta-analysis is the gold standard for demonstrating robust quantitative knowledge. We treat this as a user-centered design problem: the failure to accrue quantitative knowledge is not the users' (i.e. researchers') failure, but a failure to consider those users' needs when designing statistical practice. Using simulation, we compare hypothetical publication worlds following existing frequentist against Bayesian practice. We show that Bayesian analysis yields more precise effects with each new study, facilitating knowledge accrual without traditional meta-analyses. Bayesian practices also allow more principled conclusions from small-n studies of novel techniques. These advantages make Bayesian practices a likely better fit for the culture and incentives of the field. Instead of admonishing ourselves to spend resources on larger studies, we propose using tools that more appropriately analyze small studies and encourage knowledge accrual from one study to the next. We also believe Bayesian methods can be adopted from the bottom up without the need for new incentives for replication or meta-analysis. These techniques offer the potential for a more user- (i.e. researcher-) centered approach to statistical analysis in HCI.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Bayesian statistics has also been proposed as a better alternative to frequentist statistics for HCI research[43].”
From Explanations Can Reduce Overreliance on AI Systems During Decision-Making (2023) · cites this paper as [43]
In the citing paper
“Within HCI, there is an emphasis on novel technology tools and artifacts, not necessarily reliable effect size estimates/causal inferences [47].”
From Agile science: creating useful products for behavior change in the real world (2016) · cites this paper as [47]
In the citing paper
“As demonstrated elsewhere, careful use of priors within studies of novel systems can be used to help mitigate problems with magnitude effect errors and thus could be a more appropriate statistical method for the generate phase [47].”
From Agile science: creating useful products for behavior change in the real world (2016) · cites this paper as [47]
Citations
Cites
<tt>emcee</tt>: The MCMC Hammer
Foreman-Mackey, D., Hogg, D. W., Lang, D., & Goodman, J. (2013) · Publications of the Astronomical Society of the Pacific · doi:10.1086/670067 · Cited by 12,414
Stan : A Probabilistic Programming Language
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddell, A. (2017) · Journal of Statistical Software · doi:10.18637/jss.v076.i01 · Cited by 7,754 · cited 1×
“These tools include modeling languages like JAGS [23] and Stan [2] (both with R packages, and Stan includes a Python interface), and Python-specific libraries like emcee [7] and PyMC [22].”
MethodJAGS: A program for analysis of Bayesian graphical models using Gibbs sampling
Plummer, M. (2003) · Cited by 4,720
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 2×
“There are many problems with this approach, particularly when a field utilizes small samples to estimate statistical significance, as many of these significant differences are likely due to chance [17].”
Gelman, A. (2006) · Bayesian Analysis · doi:10.1214/06-ba117a · Cited by 4,050 · cited 1×
“Other accessible articles have also been written about practical concerns in Bayesian analysis, including discussions of how to choose priors [9,11].”
BackgroundThe New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200
The aligned rank transform for nonparametric factorial analyses using only anova procedures
Wobbrock, J. O., Findlater, L., Gergle, D., & Higgins, J. J. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1978963 · Cited by 2,854
A weakly informative default prior distribution for logistic and other regression models
Gelman, A., Jakulin, A., Pittau, M. G., & Su, Y.-S. (2008) · The Annals of Applied Statistics · doi:10.1214/08-aoas191 · Cited by 1,790 · cited 4×
“[9]).”
BackgroundMethodInfluentialPrior distributions for variance parameters in hierarchical modelsPreprint
Gelman, A. (2004) · RePEc: Research Papers in Economics · Cited by 1,629
Bayesian estimation supersedes the t-test
Kruschke, J. K. (2013) · Journal of Experimental Psychology: General · doi:10.1037/a0029146 · Cited by 1,530
Research works printed but not linked: 3
- Joel Brandt, Mira Dontcheva, Marcos Weskamp, and Scott R. Klemmer. 2010. Example-centric programming. Proceedings of the 28th international conference on Human factors in computing systems CHI '10, ACM Press, 513. Retrieved January 7, 2016 from http://dl.acm.org/citation.cfm?id=1753326.1753402
- Zoltan Dienes. 2011. Bayesian Versus Orthodox Statistics: Which Side Are You On? Perspectives on psychological science?: a journal of the Association for Psychological Science 6, 3: 274-90. Retrieved January 7, 2016 from http://pps.sagepub.com/content/6/3/274.abstract
- 10.1037/0033-2909.88.2.359 · DOI · Paper page
Cited by
An Aligned Rank Transform Procedure for Multifactor Contrast Tests
Elkin, L. A., Kay, M., Higgins, J. J., & Wobbrock, J. O. (2021) · The 34th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3472749.3474784 · Cited by 664
“[25] and Robertson and Kaptein [38] introduced more modern statistical methods, and Wobbrock et al.”
MethodSelf-citationThe Experience Sampling Method on Mobile Devices
van Berkel, N., Ferreira, D., & Kostakos, V. (2018) · ACM Computing Surveys · doi:10.1145/3123988 · Cited by 432
“However, we do recommend researchers to use Bayesian statistics, as this allows for statistical inference on small sample sizes (Kay et al. 2016).”
MethodExplanations Can Reduce Overreliance on AI Systems During Decision-Making
Vasconcelos, H., Jörke, M., Grunde-McLaughlin, M., Gerstenberg, T., Bernstein, M. S., & Krishna, R. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579605 · Cited by 402
The Effects of Generative AI on Design Fixation and Divergent Thinking
Wadinambiarachchi, S., Kelly, R. M., Pareek, S., Zhou, Q., & Velloso, E. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642919 · Cited by 320
“We model the random effects of participants and images as being drawn from a normal distribution with mean zero and standard deviation computed from the data through partial pooling.”
Kim, N. J., Belland, B. R., & Walker, A. E. (2018) · Educational Psychology Review · doi:10.1007/s10648-017-9419-1 · Cited by 220
“Another possible reason is that BMA could resolve issues of overweighted effect size caused by smaller studies, which were included in the traditional meta-analysis (Kay et al. 2016; Kicinski 2013; Mengersen et al. 2016).”
MethodInfluentialAgile science: creating useful products for behavior change in the real world
Hekler, E. B., Klasnja, P., Riley, W. T., Buman, M. P., Huberty, J., Rivera, D. E., & Martin, C. A. (2016) · Translational Behavioral Medicine · doi:10.1007/s13142-016-0395-7 · Cited by 208
Self-citationHow Digital Natives Learn and Thrive in the Digital Age: Evidence from an Emerging Economy
Tran, T., Ho, M.-T., Pham, T.-H., Nguyen, M.-H., Nguyen, K.-L. P., Vuong, T.-T., Nguyen, T.-H. T., Nguyen, T.-D., Nguyen, T.-L., Khuc, Q., et al. (2020) · Sustainability · doi:10.3390/su12093819 · Cited by 198
“They allow researchers to facilitate new knowledge without traditional meta-analyses and yield more principled conclusions from each new study [53].”
BackgroundUncertainty Displays Using Quantile Dotplots or CDFs Improve Transit Decision-Making
Fernandes, M., Walls, L., Munson, S., Hullman, J., & Kay, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173718 · Cited by 175
“After Kay et al. 2016 [22], we use a mixed-effects Bayesian regression model with weakly-informed priors.”
MethodSelf-citationAI makes you smarter but none the wiser: The disconnect between performance and metacognition
Fernandes, D., Villa, S., Nicholls, S., Haavisto, O., Buschek, D., Schmidt, A., Kosch, T., Shen, C., & Welsch, R. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108779 · Cited by 188
Increasing the Transparency of Research Papers with Explorable Multiverse Analyses
Dragicevic, P., Jansen, Y., Sarma, A., Kay, M., & Chevalier, F. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300295 · Cited by 121
“[58] Matthew Kay, Gregory L Nelson, and Eric B Hekler.”
BackgroundSelf-citationQuarantined! Examining the Effects of a Community-Wide Moderation Intervention on Reddit
Chandrasekharan, E., Jhaver, S., Bruckman, A., & Gilbert, E. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3490499 · Cited by 129
“could reasonably be ascribed to the randomness introduced in selecting the sample. If not, we have evidence that the effect observed in the sample reflects an effect that is present in the population [34, 46]. In order to account for Reddit-wide trends leading to decreases in activity levels across all subreddits, we…”
MethodEvaluating the Impact of Physical Activity Apps and Wearables: Interdisciplinary Review
McCallum, C., Rooksby, J., & Gray, C. M. (2018) · JMIR mhealth and uhealth · doi:10.2196/mhealth.9054 · Cited by 126
“These preliminary, small-N studies are typical of iterative HCI research focused on developing novel technologies [185], and are unlikely to be sufficiently powered to test important hypotheses on mediators of effectiveness [17,186].”
BackgroundHARK No More: On the Preregistration of CHI Experiments
Cockburn, A., Gutwin, C., & Dix, A. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173715 · Cited by 95
van Berkel, N., Goncalves, J., Lovén, L., Ferreira, D., Hosio, S., & Kostakos, V. (2019) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2018.12.002 · Cited by 92
Belland, B. R., Walker, A. E., & Kim, N. J. (2017) · Review of Educational Research · doi:10.3102/0034654317723009 · Cited by 103
“Using a Bayesian approach helps address small study effects (Kay, Nelson, & Hekler, 2016; Mengersen, Drovandi, Robert, Pyne, & Gore, 2016).”
BackgroundMethodTime for Historicism in CSCW: An Invitation
Soden, R., Ribes, D., Avle, S., & Sutherland, W. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479603 · Cited by 79
Self-Experimentation for Behavior Change: Design and Formative Evaluation of Two Approaches
Lee, J., Walker, E., Burleson, W., Kay, M., Buman, M. P., & Hekler, E. B. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3026038 · Cited by 98
“We used Bayesian analysis instead of frequentist analysis, because it offers a more principled way to handle the uncertainty in small formative trials [19,25].”
MethodSelf-citationBoba: Authoring and Visualizing Multiverse Analyses
Liu, Y., Kale, A., Althoff, T., & Heer, J. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.3028985 · Cited by 73
Prabhudesai, S., Yang, L., Asthana, S., Huan, X., Liao, Q. V., & Banovic, N. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584033 · Cited by 82
Performance and Metacognition Disconnect when Reasoning in Human-AI Interaction
Fernandes, D., Villa, S., Nicholls, S., Haavisto, O., Buschek, D., Schmidt, A., Kosch, T., Shen, C., & Welsch, R. (2024) · arXiv · doi:10.48550/arxiv.2409.16708 · Cited by 71
Inform the Uninformed: Improving Online Informed Consent Reading with an AI-Powered Chatbot
Xiao, Z., Li, T. W., Karahalios, K. G., & Sundaram, H. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581252 · Cited by 81
“review of the Bayesian method’s advantages in HCI research [46].”
The Continued Prevalence of Dichotomous Inferences at CHI
Besançon, L., & Dragicevic, P. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3310432 · Cited by 70
Liu, Y., Althoff, T., & Heer, J. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376533 · Cited by 65
“In response, scholars have endorsed a number practices, including pre-registration [10, 56, 58], using estimation instead of dichotomous testing [2, 13, 14], adopting Bayesian statistics [22, 32], and increasing transparency in reporting [17, 39, 41, 53].”
BackgroundA Design Space of Vision Science Methods for Visualization Research
Elliott, M., Nothelfer, C. E., Xiong, C., & Szafir, D. A. (2020) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.3029413 · Cited by 61
“Finally, existing surveys on related topics such as crowdsourcing [4] and statistical analysis [35] provide insight that can augment and influence experimental design, but we focus on broader methodological approaches that can be applied across deployment platforms and can be analyzed using a variety of statistical te…”
BackgroundVilla, S., Kosch, T., Grelka, F., Schmidt, A., & Welsch, R. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2023.107787 · Cited by 53
“Kay, Nelson et al. (2016) explain advantages of Bayesian statistics in technological contexts that are also relevant to our study.”
BackgroundKloft, A. M., Welsch, R., Kosch, T., & Villa, S. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642633 · Cited by 53
Am I Wrong, or Is the Autograder Wrong? Effects of AI Grading Mistakes on Learning
Li, T. W., Hsu, S., Fowler, M., Zhang, Z., Zilles, C. B., & Karahalios, K. G. (2023) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3568813.3600124 · Cited by 65
Lessons Learned from Two Cohorts of Personal Informatics Self-Experiments
Daskalova, N., Desingh, K., Papoutsaki, A., Schulze, D., Sha, H., & Huang, J. J. S. (2017) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3130911 · Cited by 46
“explore the benefits of Bayesian statistics, emphasizing that they lead to more reasonable conclusions for small-n studies, and shift the question towards the strength of the intervention effect rather than a binary “does it work” [26].”
BackgroundCharacterizing the Influence of Confirmation Bias on Web Search Behavior
Suzuki, M., & Yamamoto, Y. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.771948 · Cited by 59
“Note that the GLMM can analyze small-scale data more accurately than methods that employ frequentist statistics (Kay et al., 2016).”
MethodTouchstone2: An Interactive Environment for Exploring Trade-offs in HCI Experiment Design
Eiselmayer, A., Wacharamanotham, C., Beaudouin-Lafon, M., & Mackay, W. E. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300447 · Cited by 46
“However, most HCI experiments are small scale, with few participants [15], and often include within-participant factors.”
BackgroundToward Usable Evidence: Optimizing Knowledge Accumulation in HCI Research on Health Behavior Change
Klasnja, P., Hekler, E. B., Korinek, E. V., Harlow, J., & Mishra, S. R. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3026013 · Cited by 44
“More recently, Kay et al. [22] have argued for the use of Bayesian statistics as a way to improve the level of quantitative evidence obtained from de-ployments of technologies, as many field studies that HCI researchers conduct are underpowered to provide high-quality evidence using frequentist…”
MethodSelf-citationTea: A High-level Language and Runtime System for Automating Statistical Analysis
Jun, E., Daum, M., Roesch, J., Chasins, S., Berger, E., Just, R., & Reinecke, K. (2019) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3332165.3347940 · Cited by 49
“While there are calls to incorporate other methods of statistical analysis [20, 21], Null Hypothesis Significance Testing (NHST) remains the norm in HCI and other disciplines.”
MethodMapping Out Human-Centered Data Science: Methods, Approaches, and Best Practices
Kogan, M., Halfaker, A., Guha, S., Aragon, C., Muller, M., & Geiger, S. (2020) · Companion of the 2020 ACM International Conference on Supporting Group Work · doi:10.1145/3323994.3369898 · Cited by 40
Helske, J., Helske, S., Cooper, M., Ynnerman, A., & Besancon, L. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3073466 · Cited by 38
“…among those who conduct statistical analyses as well as among those who interpret results, work has also been conducted in making it easier to avoid common pitfalls of NHST either by altering the way analyses are conducted [7], [8], [9] or how the results are presented [10], [11], [12], [13], [14].”
BackgroundCountry Differences in Social Comparison on Social Media
Cheng, J., Burke, M., & de Gant, B. (2021) · Proc. ACM Hum. Comput. Interact. · doi:10.1145/3434179 · Cited by 33
“Further, taking a Bayesian approach allows future work to directly incorporate the estimates presented in this work to obtain even more precise estimates [38].”
MethodSpecial Interest Group on Transparent Statistics in HCI
Kay, M., Haroz, S., Guha, S., & Dragicevic, P. (2016) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2851581.2886442 · Cited by 37
“, essays by advocates of estimation [5, 7] and of Bayesian methods [12].”
BackgroundMethodInfluentialSelf-citationKloft, A. M., Welsch, R., Kosch, T., & Villa, S. (2023) · arXiv · doi:10.48550/arxiv.2309.16606 · Cited by 34
Understanding How People use Search to Support their Everyday Creative Tasks
Zhang, Y., & Capra, R. (2019) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3295750.3298936 · Cited by 36
“[29] at CHI 2016 advocating the use of Bayesian statistics, all the statistical inferences in our study are based on Bayesian inference.”
MethodSarma, A., & Kay, M. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376377 · Cited by 41
Self-citationKim, Y., & Park, J. (2019) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2019.04.022 · Cited by 31
Measuring User-Moderator Alignment on r/ChangeMyView
Koshy, V., Bajpai, T., Chandrasekharan, E., Sundaram, H., & Karahalios, K. G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610077 · Cited by 39
“[26] provide a detailed discussion of why Bayesian modeling is a good fit for HCI research.”
BackgroundMoving Transparent Statistics Forward at CHI
Kay, M., Haroz, S., Guha, S., Dragicevic, P., & Wacharamanotham, C. (2017) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3027063.3027084 · Cited by 31
“He has also published work advancing the use of Bayesian statistics in VIS [8] and CHI [9].”
BackgroundMethodSelf-citationOrtloff, A.-M., Martius, F., Meier, M., Sans-Raimbault, T., Geierhaas, L., & Smith, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713671 · Cited by 30
Investigating Effects of Visual Anchors on Decision‐Making about Misinformation
Wesslen, R. S., Santhanam, S., Karduni, A., Cho, I., Shaikh, S., & Dou, W. (2019) · Computer Graphics Forum · doi:10.1111/cgf.13679 · Cited by 30
“In our results, we focus on effects sizes rather than p-values [KNH16] and follow conventions provided by Dragicevic [Dra16].”
MethodUnderstanding Multi-View Collaboration between Augmented Reality and Remote Desktop Users
Fages, A., Fleury, C., & Tsandilas, T. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555607 · Cited by 25
“The method is based on a Bayesian statistics [27] framework, but we emphasize that we do not use informative priors here.”
BackgroundMethodSupporting Novices Author Audio Descriptions via Automatic Feedback
Natalie, R., Tseng, J., Kacorri, H., & Hara, K. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581023 · Cited by 26
“Increasingly, researchers both from HCI [27] and feedback as the main efect and video type as a random efect.”
BackgroundInfluentialvan Gemert, T., Nilsson, N. C., Hirzle, T., & Bergström, J. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641974 · Cited by 25
“For an introduction, see [57, 91, 130].”
Revealing Perceptual Proxies with Adversarial Examples
Ondov, B., Yang, F., Kay, M., Elmqvist, N., & Franconeri, S. (2020) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.3030429 · Cited by 25
“Bayesian estimation For our own analyses, we followed a Bayesian estimation approach [24,43].”
MethodSelf-citationA Patient-Centered Proposal for Bayesian Analysis of Self-Experiments for Health
Schroeder, J., Karkar, R., Fogarty, J. A., Kientz, J. A., Munson, S. A., & Kay, M. (2018) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-018-0033-x · Cited by 22
Self-citationWang, Y., König, L. M., & Reiterer, H. (2021) · JMIR Formative Research · doi:10.2196/15369 · Cited by 21
“Moreover, Bayesian methods allow more principled conclusions from studies with a small sample size of novel techniques in the field of human-computer interaction [58].”
Top economics universities and research institutions in Vietnam: evidence from the SSHPA dataset
Vuong, Q., Bui, A.-T., Ho, M., Pham, T.-H., Vu, T.-H., Pham, H., Hoang, A.-D., Ho, M.-T., & La, V.-P. (2021) · Heliyon · doi:10.1016/j.heliyon.2021.e06273 · Cited by 21
Larson, C., Kaplan, D. E., Girolamo, T. M., Kover, S. T., & Eigsti, I. (2023) · Journal of Clinical Psychology · doi:10.1002/jclp.23570 · Cited by 21
Putting scientific results in perspective: Improving the communication of standardized effect sizes
Kim, Y., Hofman, J. M., & Goldstein, D. G. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502053 · Cited by 23
Navigating the Virtual Gaze: Social Anxiety's Role in VR Proxemics
Mello, B., Welsch, R., Verbokkem, M. C., Knierim, P., & Dechant, M. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642359 · Cited by 25
Special Interest Group on Transparent Statistics Guidelines
Wacharamanotham, C., Kay, M., Haroz, S., Guha, S., & Dragicevic, P. (2018) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3170427.3185374 · Cited by 18
“Practices going against transparency have been criticized within both HCI [2, 3, 5, 7, 10] and related fields [4, 11, 12].”
BackgroundInfluentialSelf-citationExploring the Potential of Physical Visualizations
Stusak, S. (2015) · International Conference on Tangible, Embedded, and Embodied Interaction (TEI) · doi:10.1145/2677199.2691604 · Cited by 18
Masson, D., Malacria, S., Casiez, G., & Vogel, D. (2023) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3586183.3606762 · Cited by 18
More than just informed: The importance of consent facets in smart homes
Chiang, Y.-S., Khan, O., Bates, A., & Cobb, C. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642288 · Cited by 22
Uncovering and Addressing Blink-Related Challenges in Using Eye Tracking for Interactive Systems
Grootjen, J. W., Weingärtner, H., & Mayer, S. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642086 · Cited by 15
Some Prior(s) Experience Necessary: Templates for Getting Started With Bayesian Analysis
Phelan, C., Hullman, J. R., Kay, M., & Resnick, P. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300709 · Cited by 20
Self-citation"I can show what I really like.": Eliciting Preferences via Quadratic Voting
Cheng, T.-C., Li, T. W., Chou, Y.-H., Karahalios, K. G., & Sundaram, H. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449281 · Cited by 16
“Third, Kay et al. [39] suggest a that Bayesian formulation shifts the conversation focus from “did it work" to “how strong is the effect."”
BackgroundInfluentialGrootjen, J. W., Thalhammer, P., & Kosch, T. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3676531 · Cited by 14
Mobile User Research: A Practical Guide
Consolvo, S., Bentley, F. R., Hekler, E. B., & Phatak, S. S. (2017) · Synthesis lectures on mobile and pervasive computing · doi:10.2200/s00763ed1v01y201703mpc012 · Cited by 18
Self-citationYang, F., Ma, Y., Harrison, L., Tompkin, J., & Laidlaw, D. H. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581111 · Cited by 18
“, [43, 44]) and are robust to random errors (e.”
BackgroundGender, Feedback, and Learners' Decisions to Share Their Creative Computing Projects
Gan, E. F., Hill, B. M., & Dasgupta, S. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274323 · Cited by 16
“In recent work, Bayesian analysis has been suggested as a promising tool for HCI research [34].”
MethodNo Such Thing as Too Much Chocolate: Evidence Against Choice Overload in E-Commerce
Moser, C., Phelan, C., Resnick, P., Schoenebeck, S. Y., & Reinecke, K. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025778 · Cited by 18
“Second, interpretation of Bayesian results is more intuitive than the results produced by frequentist methods [13,24].”
ResultThe Value of Activity Traces in Peer Evaluations: An Experimental Study
Shi, W. W., Kumaran, S. R. K., Sundaram, H., & Bailey, B. P. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579627 · Cited by 12
“provide a more detailed overview of the advantages of Bayesian inference for statistical analysis of HCI research [20].”
BackgroundThe Effects of User Comments on Science News Engagement
Williams, S., & Hsieh, G. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449106 · Cited by 16
“It was preregistered 1 in order to explicitly lay out our confirmatory hypotheses to differentiate between exploratory results [36], guard against p-hacking [54], and help to promote more open research and support better meta analyses in HCI [7].”
MethodStatistics for HCI: Making Sense of Quantitative Data
Dix, A. J. (2020) · Synthesis lectures on human-centered informatics · doi:10.2200/s00974ed1v01y201912hci044 · Cited by 11
Should We Use an Abstract Comic Form to Persuade? Experiments with Online Charitable Donation
Xiao, Z., Ho, P.-S., Wang, X., Karahalios, K. G., & Sundaram, H. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359177 · Cited by 12
“[36] to explain our decision to use Bayesian models.”
BackgroundMethodInfluentialReproducibility: A Researcher-Centered Definition
Feger, S. S., & Woźniak, P. W. (2022) · Multimodal Technologies and Interaction · doi:10.3390/mti6020017 · Cited by 11
“Examples of work that focus on researcher-centered contributions include Badiola et al. [11] and Kay et al. [12].”
BackgroundPang, R. Y., Reinecke, K., & Just, R. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517707 · Cited by 12
How Relevant are Incidental Power Poses for HCI?
Jansen, Y., & Hornbæk, K. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173588 · Cited by 10
“One way of representing these is to plot the density of these distributions together with a 95% highest density interval (HDI) as so-called eyeplots [42] (as done in Figures 9 & 11).”
Leusmann, J., Oechsner, C., Prinz, J., Welsch, R., & Mayer, S. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585884 · Cited by 10
Have a SEAT on Stage: Restoring Trust with Spectator Experience Augmentation Techniques
Capra, O., Berthaut, F., & Grisoni, L. (2020) · Designing Interactive Systems Conference · doi:10.1145/3357236.3395492 · Cited by 14
Chiossi, F., Ou, C., Putze, F., & Mayer, S. (2024) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3701571.3701579 · Cited by 8
“Particularly useful in contexts with limited sample sizes and nested data structures, Bayesian methods allow us to quantify the size and uncertainty of effects [31], offering insights into both the presence and absence of effects.”
Transparent Practices for Quantitative Empirical Research
Wacharamanotham, C., Yang, F., Pu, X., Sarma, A., & Padilla, L. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3503760 · Cited by 9
“A large body of research from the feld of HCI and visualization had also contributed to improving transparent practices in quantitative research (e.g., [7, 10, 11, 13, 21, 22, 44, 47]).”
BackgroundHow your Physical Environment Affects Spatial Presence in Virtual Reality
van Gemert, T., Knibbe, J., & Velloso, E. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714114 · Cited by 9
While, Z., & Sarvghad, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714229 · Cited by 7
“…an even greater understanding of how aging interacts with tasks and visualization type; collecting more data while using a more informed prior [42] or comparing correlations in accuracy and time between various visualizations and tasks [21], for example, could lead to further insights into…”
Li, T. W., Song, Y., Sundaram, H., & Karahalios, K. G. (2025) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3698205.3729550 · Cited by 7
“We operationalized the models with Bayesian formulations of hierarchical generalized linear models [57] because Bayesian inference in HCI [35] and educational research [42] has a number of advantages.”
Touchstone2: An Interactive Environment for Exploring Trade-offs in HCI Experiment DesignPreprint
Eiselmayer, A., Wacharamanotham, C., Beaudouin-Lafon, M., & Mackay, W. E. (2019) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/ab536 · Cited by 6
Bayesian Hierarchical Pointing Models
Zhao, H., Gu, S., Yu, C., & Bi, X. (2022) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3526113.3545708 · Cited by 5
An interaction Continuum for 3D Dataset Visualization
Besançon, L. (2017) · theses.fr (ABES) · doi:10.70675/2c660626z06afz457fzad61z7c742a806c56 · Cited by 9
Organize, Then Vote: Exploring Cognitive Load in Quadratic Survey Interfaces
Cheng, T.-C., Zhang, Y., Chou, Y.-H., Koshy, V., Li, T. W., Karahalios, K. G., & Sundaram, H. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714193 · Cited by 6
Levels of Detail in Visual Augmentation for Novice and Expert Audiences
Capra, O., Berthaut, F., & Grisoni, L. (2020) · Computer Music Journal · doi:10.1162/comj_a_00568 · Cited by 4
“The analyses were conducted under the common frequentist paradigm and were combined to Bayesian statistics (Kay, Nelson, and Hekler 2016).”
MethodUnderstanding Crowdsourcing Requesters’ Wage Setting Behaviors
Hara, K., & Tanaka, Y. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3519660 · Cited by 6
“, change in hourly wage), allowing us to draw more insights from data [23].”
BackgroundLi, T. W., Song, Y., Sundaram, H., & Karahalios, K. G. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791940 · Cited by 3
“We operationalized the models with Bayesian formulations of hierarchical generalized linear models [51] because Bayesian inference in HCI [32] and educational research [41] has a number of advantages.”
Highlighting Weasel Sentences for Promoting Critical Information Seeking on the Web
Saito, F., Shoji, Y., & Yamamoto, Y. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-34223-4_27 · Cited by 4
Jo, Y., Jameel, M., Cobb, C., & Bates, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713941 · Cited by 2
“Bayesian analysis is gaining popularity in the HCI community due to its ability to report more precise effects and draw more principled conclusions from small studies [36].”
Zent, M., Shea, K., & Yarosh, S. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757398 · Cited by 2
“Following recommendations in the field [47, 65, 80], we use Bayesian modeling to disentangle the relationship between these response variables and our experimental conditions.”
Wadinambiarachchi, S., Waycott, J., & Wadley, G. (2026) · Creativity and Cognition · doi:10.1145/3803784.3807524 · Cited by 3
Making Sense of Statistics in HCI: From P to Bayes and Beyond
Dix, A. J. (2017) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3027063.3027109 · Cited by 2
Transparent Practices for Quantitative Empirical Research
Wacharamanotham, C., Yang, F., Pu, X., & Sarma, A. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3574168 · Cited by 2
Khorasani, S., Hsu, S., Guan, R., Gonçalves, J., Irlitti, A., Knibbe, J., & Velloso, E. (2025) · Computers & Education X Reality · doi:10.1016/j.cexr.2025.100103 · Cited by 2
Yamanaka, S. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2407662 · Cited by 2
“Kay et al. (2016) argued that Bayesian statistical methods are more suitable for HCI research than the current frequentist approach.”
Kostolani, D., Pülke, D., & Schlund, S. (2025) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/3733155.3733200 · Cited by 1
“Moreover, Bayesian analysis leads to more reasonable conclusions in studies with low sample sizes as it prevents overestimation of the effects [20, 38].”
TermSight: Making Service Contracts Approachable
Huang, Z., August, T., & Sundaram, H. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791387 · Cited by 2
“This demonstrates that Summary Snippets were potentially more useful as navigational features to surface and interrogate relevant information from the original text compared to section headers.”
Tietenberg, J., Fürtges, T., & Masuch, M. (2026) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2026.3680644 · Cited by 1
“A second limiting factor lies in the relatively small sample size, which is common in HCI research [31], but still limits statistical power, external validity, and is likely to result in biased effect sizes based on the wide confidence intervals [19].”
Meinhardt, L.-M., Colley, M., BALCI, A., Degenhard, A., Jansen, P., & Rukzio, E. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798420 · Cited by 1
“Bayesian approaches are especially valuable for studies with limited sample sizes in emerging research areas as they provide probabilistic estimates rather than binary decisions and thus capture both effect strength and uncertainty more comprehensively than common frequentist statistics [15, 30].”
Turning cognitive science into game-based biomarkers: Acute stress detection with tunnel runner
Markovitch, B., Kamps, J. C. C., Orij, J. C., Markopoulos, P., & Birk, M. V. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.101176 · Cited by 1
Metadata
- Authors
- Kay, M., Nelson, G. L., & Hekler, E. B.
- Affiliations
- University of Washington (US)
- University of Washington, Seattle, WA, USA
- Arizona State University (US)
- Venue
- Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2016-05-07
- Pages
- 4521-4532
- OpenAlex
- W2404411786
- Cited by
- 113
- References
- 30
- Citations by year
- 2026: 5 · 2025: 11 · 2024: 11 · 2023: 16 · 2022: 11 · 2021: 10 · 2020: 14 · 2019: 14 · 2018: 7 · 2017: 12 · 2016: 2
- FWCI
- 12.36
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- National Science Foundation (DGE-1256082)
- Robert Wood Johnson Foundation (71995)
- Intel Corporation (Intel Science and Technology Center for Pervasive Computing)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- Bayesian analysis, frequentist statistics, meta-analysis, replication, user-centered design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.