The DOI matches its Crossref record.
The aligned rank transform for nonparametric factorial analyses using only anova procedures
Abstract
Nonparametric data from multi-factor experiments arise often in human-computer interaction (HCI). Examples may include error counts, Likert responses, and preference tallies. But because multiple factors are involved, common nonparametric tests (e.g., Friedman) are inadequate, as they are unable to examine interaction effects. While some statistical techniques exist to handle such data, these techniques are not widely available and are complex. To address these concerns, we present the Aligned Rank Transform (ART) for nonparametric factorial data analysis in HCI. The ART relies on a preprocessing step that "aligns" data before applying averaged ranks, after which point common ANOVA procedures can be used, making the ART accessible to anyone familiar with the F-test. Unlike most articles on the ART, which only address two factors, we generalize the ART to N factors. We also provide ARTool and ARTweb, desktop and Web-based programs for aligning and ranking data. Our re-examination of some published HCI results exhibits advantages of the ART.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 14
- Conover , W. J. and Iman , R. L. ( 1981 ). Rank transformations as a bridge between parametric and nonparametric statistics . The American Statistician 35 ( 3 ), 124 -- 129 . Conover, W. J. and Iman, R. L. (1981). Rank transformations as a bridge between parametric and nonparametric statistics. The American Statistician 35 (3), 124--129. · DOI · Paper page
- 10.1080/03610918408812368 · DOI
- 10.1145/1518701.1518956 · DOI
- 10.4148/2475-7772.1443 · DOI
- Higgins , J. J. and Tashtoush , S. ( 1994 ). An aligned rank transform test for interaction . Nonlinear World 1 ( 2 ), 201 -- 211 . Higgins, J. J. and Tashtoush, S. (1994). An aligned rank transform test for interaction. Nonlinear World 1 (2), 201--211.
- 10.1214/aoms/1177704575 · DOI · Paper page
- 10.1145/1753326.1753686 · DOI
- 10.2527/1998.7641216x · DOI · Paper page
- 10.1145/302979.302983 · DOI · Paper page
- 10.1080/03610918508812475 · doi:10.1080/03610918508812475
- 10.1080/03610919308813085 · DOI
- 10.3102/00346543060001091 · doi:10.3102/00346543060001091 · Paper page
- Vermunt , J. K. ( 1997 ). Log-linear Models for Event Histories . Thousand Oaks, CA : Sage Publications . Vermunt, J. K. (1997). Log-linear Models for Event Histories. Thousand Oaks, CA: Sage Publications.
- 10.1145/1294211.1294238 · DOI · Paper page
Cited by
Washington, P. M., Forcelli, P. A., Wilkins, T., Zapple, D. N., Parsadanian, M., & Burns, M. P. (2012) · Journal of Neurotrauma · doi:10.1089/neu.2012.2456 · Cited by 228
"These are not my hands!": Effect of Gender on the Perception of Avatar Hands in Virtual Reality
Schwind, V., Knierim, P., Tasci, C., Franczak, P., Haas, N., & Henze, N. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025602 · Cited by 221
Understanding Blind People's Experiences with Computer-Generated Captions of Social Media Images
MacLeod, H., Bennett, C. L., Morris, M. R., & Cutrell, E. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025814 · Cited by 200
How Older Adults Learn to Use Mobile Devices: Survey and Field Investigations
Leung, R., Tang, C., Haddad, S., Mcgrenere, J., Graf, P., & Ingriany, V. (2012) · ACM Transactions on Accessible Computing · doi:10.1145/2399193.2399195 · Cited by 162
The Placebo Effect of Artificial Intelligence in Human–Computer Interaction
Kosch, T., Welsch, R., Chuang, L., & Schmidt, A. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3529225 · Cited by 137
Kim, S., Eun, J., Oh, C., Suh, B., & Lee, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376785 · Cited by 133
Discrete and continuous mechanisms of temporal selection in rapid visual streams
Marti, S., & Dehaene, S. (2017) · Nature Communications · doi:10.1038/s41467-017-02079-x · Cited by 127
Access overlays: improving non-visual access to large touch screens for blind users
Kane, S. K., Morris, M. R., Perkins, A. Z., Wigdor, D., Ladner, R. E., & Wobbrock, J. O. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047232 · Cited by 127
Self-citationBreastScreening-AI: Evaluating medical intelligent agents for human-AI interactions
Calisto, F. M., Santiago, C., Nunes, N., & Nascimento, J. C. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102285 · Cited by 125
First Impressions in Human--Agent Virtual Encounters
Cafaro, A., Vilhjálmsson, H. H., & Bickmore, T. (2016) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2940325 · Cited by 120
PassChords: secure multi-touch authentication for blind people
Azenkot, S., Rector, K., Ladner, R., & Wobbrock, J. (2012) · Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/2384916.2384945 · Cited by 116
Self-citationSmith-Renner, A., Fan, R., Birchfield, M., Wu, T., Boyd-Graber, J., Weld, D. S., & Findlater, L. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376624 · Cited by 112
Self-citationKay, M., Nelson, G. L., & Hekler, E. B. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858465 · Cited by 113
Study of Multimodal Interfaces and the Improvements on Teleoperation
Triantafyllidis, E., Mcgreavy, C., Gu, J., & Li, Z. (2020) · IEEE Access · doi:10.1109/access.2020.2990080 · Cited by 105
“For the perceived system usability, an ART was used to allow the use of parametric tests on ordinal data.”
MethodInfluential(2012) · Graphics Interface · Cited by 103
Krupp, L., Steinert, S., Kiefer-Emmanouilidis, M., Avila, K. E., Lukowicz, P., Kuhn, J., Küchemann, S., & Karolus, J. (2024) · Frontiers in Artificial Intelligence and Applications · doi:10.3233/faia240195 · Cited by 82
Digital transformations of classrooms in virtual reality
Gao, H., Bozkir, E., Hasenbein, L., Hahn, J.-U., Göllner, R., & Kasneci, E. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445596 · Cited by 77
IntroAssist: A Tool to Support Writing Introductory Help Requests
Hui, J. S., Gergle, D., & Gerber, E. M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173596 · Cited by 76
Self-citationLevy, A., Agrawal, M., Satyanarayan, A., & Sontag, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445522 · Cited by 73
“For our multi-factor design (user mode and annotation difculty), we frst conduct an Aligned Rank Transform (ART) procedure [50].”
MethodAltered bodily self-consciousness and peripersonal space in autism
Mul, C.-L., Cardini, F., Stagg, S. D., Sadeghi Esfahlani, S., Kiourtsoglou, D., Cardellicchio, P., & Aspell, J. E. (2019) · Autism · doi:10.1177/1362361319838950 · Cited by 67
Acute effects of modafinil on brain resting state networks in young healthy subjects
Esposito, R., Cilli, F., Pieramico, V., Ferretti, A., Macchia, A., Tommasi, M., Saggino, A., Ciavardelli, D., Manna, A., Navarra, R., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0069224 · Cited by 66
Agent-User Concordance and Satisfaction with a Virtual Hospital Discharge Nurse
Zhou, S., Bickmore, T., Paasche-Orlow, M., & Jack, B. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-09767-1_63 · Cited by 57
Banovic, N., Yang, Z., Ramesh, A., & Liu, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579460 · Cited by 55
Writing with AI Lowers Psychological Ownership, but Longer Prompts Can Help
Joshi, N., & Vogel, D. (2025) · Proceedings of the 7th ACM Conference on Conversational User Interfaces · doi:10.1145/3719160.3736608 · Cited by 50
Do You Mind? User Perceptions of Machine Consciousness
Scott, A. E., Neumann, D., Niess, J., & Woźniak, P. W. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581296 · Cited by 46
Using behavioral rhythms and multi-task learning to predict fine-grained symptoms of schizophrenia
Tseng, V. W.-S., Sano, A., Ben-Zeev, D., Brian, R., Campbell, A. T., Hauser, M., Kane, J. M., Scherer, E. A., Wang, R., Wang, W., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-71689-1 · Cited by 44
Effect of distracting faces on visual selective attention in the monkey
Landman, R., Sharma, J., Sur, M., & Desimone, R. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1420167111 · Cited by 43
Monitoring Screen Time or Redesigning It?: Two Approaches to Supporting Intentional Social Media Use
Zhang, M. R., Lukoff, K., Rao, R., Baughan, A., & Hiniker, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517722 · Cited by 43
Chiang, C.-W., & Yin, M. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511121 · Cited by 42
FingerText: Exploring and Optimizing Performance for Wearable, Mobile and One-Handed Typing
Lee, D., Kim, J., & Oakley, I. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445106 · Cited by 34
Asthana, S., Im, J., Chen, Z., & Banovic, N. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642180 · Cited by 34
Epistasis between promoter activity and coding mutations shapes gene evolvability
Cisneros, A. F., Gagnon-Arsenault, I., Dubé, A. K., Després, P. C., Kumar, P., Lafontaine, K., Pelletier, J. N., & Landry, C. R. (2023) · Science Advances · doi:10.1126/sciadv.add9109 · Cited by 33
de Jong, S., Paananen, V., Tag, B., & van Berkel, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710946 · Cited by 32
Visuospatial, Visuoperceptual, and Visuoconstructive Abilities in Congenital Hypothyroidism
Simic, N., Khan, S., & Rovet, J. (2013) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617713001136 · Cited by 28
What’s the Appeal? Perceptions of Review Processes for Algorithmic Decisions
Lyons, H., Wijenayake, S., Miller, T., & Velloso, E. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517606 · Cited by 27
Chen, C., Lifset, E. T., Han, Y., Roy, A., Hogarth, M., Moore, A. A., Farcas, E., & Weibel, N. (2023) · The 25th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3597638.3608378 · Cited by 25
Facilitating Bodily Insights Using Electromyography-Based Biofeedback during Physical Activity
Karolus, J., Bachmann, F., Kosch, T., Schmidt, A., & Woźniak, P. W. (2021) · Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction · doi:10.1145/3447526.3472027 · Cited by 25
Boukarras, S., Özkan, D. G., Era, V., Moreau, Q., Tieri, G., & Candidi, M. (2022) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01834 · Cited by 24
Effects of postnatal estrogen manipulations on juvenile alloparental behavior
Perry, A. N., Sue Carter, C., & Cushing, B. S. (2015) · Hormones and Behavior · doi:10.1016/j.yhbeh.2015.07.019 · Cited by 24
A Performance Comparison of On-Hand versus On-Phone Nonvisual Input by Blind and Sighted Users
Oh, U., & Findlater, L. (2015) · ACM Transactions on Accessible Computing · doi:10.1145/2820616 · Cited by 23
Self-citationImmersive Virtual Reality Simulations of Bionic Vision
Kasowski, J., & Beyeler, M. (2022) · Augmented Humans 2022 · doi:10.1145/3519391.3522752 · Cited by 22
Sharif, A., Zhang, A. M., Reinecke, K., & Wobbrock, J. O. (2023) · 20th International Web for All Conference · doi:10.1145/3587281.3587284 · Cited by 21
“nonparametric Aligned Rank Transform procedure [23, 79] to statistically analyze the effects of SRU (levels: yes, no).”
MethodSelf-citationHybrid-Brailler: Combining Physical and Gestural Interaction for Mobile Braille Input and Editing
Trindade, D., Rodrigues, A., Guerreiro, T., & Nicolau, H. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173601 · Cited by 19
Lyman, K. A., Han, Y., Zhou, C., Renteria, I., Besing, G.-L., Kurz, J. E., & Chetkovich, D. M. (2021) · Science Translational Medicine · doi:10.1126/scitranslmed.abl4580 · Cited by 17
Ask Me or Tell Me? Enhancing the Effectiveness of Crowdsourced Design Feedback
Lekschas, F., Ampanavos, S., Siangliulue, P., Pfister, H., & Gajos, K. Z. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445507 · Cited by 14
Yang (2021) · doi:10.1145/3463524 · Cited by 13
(2020) · doi:10.1145/3382494.3422164 · Cited by 13
WristDial: An Eyes-Free Integer-Value Input Method by Quantizing the Wrist Rotation
Youn, E., Lee, S., Kim, S., Shim, Y. A., Chan, L., & Lee, G. (2021) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2021.1898848 · Cited by 10
Supporting Diverse Learners in K-8 Computational Thinking with TIPP&SEE
Salac, J., Thomas, C., Butler, C., & Franklin, D. (2021) · Proceedings of the 52nd ACM Technical Symposium on Computer Science Education · doi:10.1145/3408877.3432366 · Cited by 10
Team dynamics in human-AI collaboration: effects on confidence, satisfaction, and accountability
Yousefi, M., Shahi, A., Sharifi, M., J Jorge Romera, A., Hoermann, S., & Piumsomboon, T. (2025) · Proceedings of the 27th International Conference on Multimodal Interaction · doi:10.1145/3716553.3750776 · Cited by 7
Meinhardt, L.-M., Weilke, L. M., Elhaidary, M., von Abel, J., Fink, P. D. S., Rietzler, M., Colley, M., & Rukzio, E. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713454 · Cited by 7
Joo, S., Han, I., & Park, I. (2026) · Educational Psychology · doi:10.1080/01443410.2026.2618717 · Cited by 5
Zhu, Y., Sui, D., Wang, Z., Hu, X., Gu, L., Qi, Y., Wu, T., Wang, L., Wei, Y., Tang, W., et al. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791458 · Cited by 3
“The interview, detailed in Appen-dix F.7, explored their overall impressions of AICare, focusing on trust, usability, workflow integration, and the system’s clinical value.”
Perceptions of the Fairness Impacts of Multiplicity in Machine Learning
Meyer, A. P., Kim, Y.-S., D'Antoni, L., & Albarghouthi, A. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713524 · Cited by 1
Metadata
- Authors
- Wobbrock, J. O., Findlater, L., Gergle, D., & Higgins, J. J.
- Venue
- Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2011-05-07
- Pages
- 143-146
- OpenAlex
- W2128213565
- Cited by
- 2,854
- References
- 0
- FWCI
- 89.08
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.