CoChat
Sign Up
Cited by 66Social Robot Interaction and HRIPsychology Research and BibliometricsMeta-analysis and systematic reviews

Crisis ahead? Why human-robot interaction user studies may have replicability problems and directions for improvement

Open access
Leichtmann, Benedikt, Nitsch, Verena, Mara, Martina·2022·Frontiers in Robotics and AI
DOI 10.3389/frobt.2022.838116PMC PMC8961736
Open this paper in CoChat — free accountOpen PDF

Abstract

There is a confidence crisis in many scientific disciplines, in particular disciplines researching human behavior, as many effects of original experiments have not been replicated successfully in large-scale replication studies. While human-robot interaction (HRI) is an interdisciplinary research field, the study of human behavior, cognition and emotion in HRI plays also a vital part. Are HRI user studies facing the same problems as other fields and if so, what can be done to overcome them? In this article, we first give a short overview of the replicability crisis in behavioral sciences and its causes. In a second step, we estimate the replicability of HRI user studies mainly 1) by structural comparison of HRI research processes and practices with those of other disciplines with replicability issues, 2) by systematically reviewing meta-analyses of HRI user studies to identify parameters that are known to affect replicability, and 3) by summarizing first replication studies in HRI as direct evidence. Our findings suggest that HRI user studies often exhibit the same problems that caused the replicability crisis in many behavioral sciences, such as small sample sizes, lack of theory, or missing information in reported data. In order to improve the stability of future HRI research, we propose some statistical, methodological and social reforms. This article aims to provide a basis for further discussion and a potential outline for improvements in the field.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/frobt.2022.838116

Cites

20 of 20 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Past research in human-technology interaction, such as robotics, has shown that on-site experiments conducted at universities often have samples biased toward higher education levels, are more homogeneous, and consist of only low sample sizes.

    “Furthermore, past research in other fields of human-technology interaction, such as robotics, has shown that on-site experiments conducted at universities often have samples biased toward higher education levels, are thus more homogeneous, and consisted of only low sample sizes (Leichtmann et al., 2022).”

    From Effects of Explainable Artificial Intelligence on trust and human behavior in a high-risk decision task (2023) · cites this paper as Leichtmann et al., 2022

    Section 3.2 (page unavailable) says HRI user studies often use WEIRD or student samples and reports that their sample sizes are frequently very small. It does not establish that these patterns specifically characterize on-site experiments conducted at universities, or that higher education is the demonstrated reason the samples are more homogeneous. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Due to lack of theory and robust data, AI researchers might simply not be ready to test specific hypotheses for certain contexts.

    “Due to this lack of theory and robust data, we as AI researchers might simply not be ready to test specific hypotheses for certain contexts-a conclusion that is not unheard of in other research fields, such as psychology (Scheel et al., 2021) and human-robot interaction (Leichtmann et al., 2022).”

    From Effects of Explainable Artificial Intelligence on trust and human behavior in a high-risk decision task (2023) · cites this paper as Leichtmann et al., 2022

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 87

  1. Alves-Oliveira P. Arriaga P. Paiva A. Hoffman G. (2021). “ Children as Robot Designers” in HRI '21: Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, 399– 408.
  2. Bartneck C. Forlizzi J. (2004). “A Design-Centred Framework for Social Human-Robot Interaction,” in. RO-MAN 2004. 13th IEEE Int. Workshop Robot Hum. Interactive Commun., 591– 594.
  3. Baxter P. Kennedy J. Senft E. Lemaignan S. Belpaeme T. (2016). From Characterising Three Years of HRI to Methodology and Reporting Recommendations. HRI '16: ACM/IEEE Int. Conf. Human-Robot Interaction, 391– 398.
  4. Belpaeme T. (2020). “ Advice to New Human-Robot Interaction Researchers”. In Human-Robot Interaction: Evaluation Methods and Their Standardization. Editors Jost C. Le Pévédic B. Belpaeme T. Bethel C. Chrysostomou D. Crook N. (Cham: Springer International Publishing), 355– 369. 10.1007/978-3-030-42307-0_14 · DOI
  5. Bem D. J. (2011). Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect. J. Personal. Soc. Psychol. 100, 407– 425. 10.1037/a0021524 21280961 · DOI · Paper page
  6. Benjamin D. J. Berger J. O. Johannesson M. Nosek B. A. Wagenmakers E. J. Berk R. (2018). Redefine Statistical Significance. Nat. Hum. Behav. 2, 6– 10. 10.1038/s41562-017-0189-z 30980045 · DOI · Paper page
  7. Borsboom D. (2008). Latent Variable Theory. Meas. Interdiscip. Res. Perspective 6, 25– 53. 10.1080/15366360802035497 · DOI
  8. Borsboom D. Mellenbergh G. J. van Heerden J. (2004). The Concept of Validity. Psychol. Rev. 111, 1061– 1071. 10.1037/0033-295X.111.4.1061 15482073 · DOI
  9. Callaway E. (2011). Report Finds Massive Fraud at Dutch Universities. Nature 479, 15. 10.1038/479015a 22051650 · DOI
  10. Carpinella C. M. Wyman A. B. Perez M. A. Stroessner S. J. (2017). “ The Robotic Social Attributes Scale (RoSAS),” in HRI '17: Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction (Vienna Austria: ACM/IEEE), 254– 262.
  11. Bartneck C. Belpaeme T. Eyssel F. Kanda T. Keijsers M. Šabanović S. (Editors) (2020). Human-robot Interaction: An Introduction (Cambridge, New York, NY, Port Melbourne: Cambridge University Press).
  12. Cronbach L. J. Meehl P. E. (1955). Construct Validity in Psychological Tests. Psychol. Bull. 52, 281– 302. 10.1037/h0040957 13245896 · DOI · Paper page
  13. Dolata U. (2013). The Transformative Capacity of New Technologies. London, United Kingdom: Routledge.
  14. Eagly A. H. Riger S. (2014). Feminism and Psychology: Critiques of Methods and Epistemology. Am. Psychol. 69, 685– 702. 10.1037/a0037372 25046701 · DOI
  15. Ebersole C. R. Atherton O. E. Belanger A. L. Skulborstad H. M. Allen J. M. Banks J. B. (2016). Many Labs 3: Evaluating Participant Pool Quality across the Academic Semester via Replication. J. Exp. Soc. Psychol. 67, 68– 82. 10.1016/j.jesp.2015.10.012 · DOI · Paper page
  16. Elson M. Huff M. Utz S. (2020). Metascience on Peer Review: Testing the Effects of a Study's Originality and Statistical Significance in a Field Experiment. Adv. Methods Practices Psychol. Sci. 3, 53– 65. 10.1177/2515245919895419 · DOI
  17. Esterwood C. Essenmacher K. Yang H. Zeng F. Robert L. P. (2021). A Meta-Analysis of Human Personality and Robot Acceptance in Human-Robot Interaction. CHI '21: Proc. 2021 CHI Conf. Hum. Factors Comput. Syst., 1– 18.
  18. Eyssel F. (2017). An Experimental Psychological Perspective on Social Robotics. Robotics Autonomous Syst. 87, 363– 371. 10.1016/j.robot.2016.08.029 · DOI
  19. Fischer K. (2021). Effect Confirmed, Patient Dead. J. Hum.-Robot Interact. 10, 1– 4. 10.1145/3439714 · DOI
  20. Flake J. K. Pek J. Hehman E. (2017). Construct Validation in Social and Personality Research. Soc. Psychol. Personal. Sci. 8, 370– 378. 10.1177/1948550617693063 · DOI
  21. Gergen K. J. (1973). Social Psychology as History. J. Personal. Soc. Psychol. 26, 309– 320. 10.1037/h0034436 · DOI · Paper page
  22. Gollob E. Mayer M. Braumann J. (2021). “Using Robotics and A.I. To Physically Explore a Space of Aesthetic Possibilities,” in. TEI '21: Proc. Fifteenth Int. Conf. Tangible, Embedded, Embodied Interaction, 1– 8.
  23. Greenwood J. D. (1982). On the Relation between Laboratory Experiments and Social Behaviour: Causal Explanation and Generalization. J. Theor. Soc. Behav. 12, 225– 250. 10.1111/j.1468-5914.1982.tb00449.x · DOI
  24. Hancock P. A. Kessler T. T. Kaplan A. D. Brill J. C. Szalma J. L. (2021). Evolving Trust in Robots: Specification through Sequential and Comparative Meta-Analyses. Hum. Factors 63, 1196– 1229. 10.1177/0018720820922080 32519902 · DOI
  25. Hanson N. R. (1958). Patterns of Discovery. Cambridge University Press.
  26. Hasse C. Trentemøller S. Sorenson J. (2018). “The Use of Ethnography to Identify and Address Ethical, Legal, and Societal (ELS) Issues,” in. HRI '18 Companion 2018 ACM/IEEE Int. Conf. Human-Robot Interaction, 393– 394.
  27. Hensel W. M. (2020). Double Trouble? the Communication Dimension of the Reproducibility Crisis in Experimental Psychology and Neuroscience. Euro Jnl Phil Sci. 10. 10.1007/s13194-020-00317-6 · DOI
  28. Hoffman G. Zhao X. (2021). A Primer for Conducting Experiments in Human-Robot Interaction. J. Hum.-Robot Interact. 10, 1– 31. 10.1145/3412374 · DOI
  29. Hughes T. P. (1987). “ The Evolution of Large Technological Systems,” in The Social Construction of Technological Systems: New Directions in the Sociology and History of Technology. Editors Bijker W. E. Hughes T. P. Pinch T. J. (Cambridge: MIT Press), 51– 82.:
  30. Hussey I. Hughes S. (2020). Hidden Invalidity Among 15 Commonly Used Measures in Social and Personality Psychology. Adv. Methods Practices Psychol. Sci. 3, 166– 184. 10.1177/2515245919882903 · DOI · Paper page
  31. Ioannidis J. P. A. Fanelli D. Dunne D. D. Goodman S. N. (2015). Meta-research: Evaluation and Improvement of Research Methods and Practices. Plos Biol. 13, e1002264. 10.1371/journal.pbio.1002264 26431313 PMC4592065 · DOI · Paper page
  32. Ioannidis J. P. A. Stanley T. D. Doucouliagos H. (2017). The Power of Bias in Economics Research. Econ. J. 127, 236– 265. 10.1111/ecoj.12461 · DOI · Paper page
  33. Ioannidis J. P. A. (2008). Why Most Discovered True Associations Are Inflated. Epidemiology (Cambridge, Mass.) 19, 640– 648. 10.1097/EDE.0b013e31818131e7 18633328 · DOI · Paper page
  34. Ioannidis J. P. A. (2005). Why Most Published Research Findings Are False. Plos Med. 2, e124. 10.1371/journal.pmed.0020124 16060722 PMC1182327 · DOI · Paper page
  35. Irfan B. Kennedy J. Lemaignan S. Papadopoulos F. Senft E. Belpaeme T. (2018). “Social Psychology and Human-Robot Interaction: An Uneasy Marriage,” in. HRI '18 Companion 2018 ACM/IEEE Int. Conf. Human-Robot Interaction, 13– 20.
  36. Klein R. A. Ratliff K. A. Vianello M. Adams R. B. Bahník Š. Bernstein M. J. (2014). Investigating Variation in Replicability. Soc. Psychol. 45, 142– 152. 10.1027/1864-9335/a000178 · DOI · Paper page
  37. Körber M. Radlmayr J. Bengler K. (2016). Bayesian Highest Density Intervals of Take-Over Times for Highly Automated Driving in Different Traffic Densities. Proc. Hum. Factors Ergon. Soc. Annu. Meet., 2009– 2013.
  38. Kuhn T. S. (1962). The Structure of Scientific Revolutions. Chicago, London: The University of Chicago Press. First Edition
  39. Ladyman J. (2019). “ The History of Philosophy of Science,”. Editors Becker K. Thomson I. D. (Cambridge: University Press), 189– 209. 10.1017/9781316779651.017 The Cambridge Hist. Philos. 1945–2015 · DOI
  40. Lakatos I. (1976). “ Falsification and the Methodology of Scientific Research Programmes,” in Can Theories Be Refuted? Essays on the Duhem-Quine Thesis. Editor Harding S. G. (Dordrecht: D. Reidel Publishing Company), 205– 259. 10.1007/978-94-010-1863-0_14 · DOI · Paper page
  41. Lakens D. Adolfi F. G. Albers C. J. Anvari F. Apps M. A. J. Argamon S. E. (2018). Justify Your Alpha. Nat. Hum. Behav. 2, 168– 171. 10.1038/s41562-018-0311-x · DOI
  42. Leichtmann B. Nitsch V. (2020b). Is the Social Desirability Effect in Human–Robot Interaction Overestimated? A Conceptual Replication Study Indicates Less Robust Effects. Int. J. Soc. Robotics 2, 1. 10.1007/s12369-020-00688-z · DOI
  43. Leichtmann B. Nitsch V. (2020a). How Much Distance Do Humans Keep toward Robots? Literature Review, Meta-Analysis, and Theoretical Considerations on Personal Space in Human-Robot Interaction. J. Environ. Psychol. 68, 101386. 10.1016/j.jenvp.2019.101386 · DOI · Paper page
  44. Lupetti M. L. Zaga C. Cila N. (2021). “Designerly Ways of Knowing in HRI,” in. HRI '21: Proc. 2021 ACM/IEEE Int. Conf. Human-Robot Interaction, 389– 398.
  45. Luria M. Hoggenmüller M. Lee W.-Y. Hespanhol L. Jung M. Forlizzi J. (2021). “ Research through Design Approaches in Human-Robot Interaction” in HRI '21 Companion: Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, 685– 687.
  46. Mara M. Appel M. Gnambs T. (2021). Human-like Robots and the Uncanny Valley: A Meta-Analysis of User Responses Based on the Godspeed Scales. Z. für Psychol. 10.1027/2151-2604/a000486 · DOI
  47. Mara M. Meyer K. Heiml M. Pichler H. Haring R. Krenn B. (2021a). “ CoBot Studio VR: A Virtual Reality Game Environment for Transdisciplinary Research on Interpretability and Trust in Human-Robot Collaboration,” in Int. Workshop on Virtual, Augmented, and Mixed-Reality for Human-Robot Interaction (VAM-HRI 2021).
  48. Mara M. Stein J.-P. Latoschik M. E. Lugrin B. Schreiner C. Hostettler R. (2021b). User Responses to a Humanoid Robot Observed in Real Life, Virtual Reality, 3D and 2D. Front. Psychol. 12, 633178. 10.3389/fpsyg.2021.633178 33935883 PMC8082015 · DOI
  49. Marvel J. A. Bagchi S. Zimmerman M. Aksu M. Antonishek B. Li X. (2021). “Novel and Emerging Test Methods and Metrics for Effective HRI,” in. HRI '21 Companion: Companion 2021 ACM/IEEE Int. Conf. Human-Robot Interaction, 730– 732.
  50. Marvel J. A. Bagchi S. Zimmerman M. Aksu M. Antonishek B. Wang Y. (2020). “Test Methods and Metrics for Effective HRI in Real-World Human-Robot Teams,” in. HRI '20: Companion 2020 ACM/IEEE Int. Conf. Human-Robot Interaction, 652– 653.
  51. Maxwell S. E. (2004). The Persistence of Underpowered Studies in Psychological Research: Causes, Consequences, and Remedies. Psychol. Methods 9, 147– 163. 10.1037/1082-989X.9.2.147 15137886 · DOI
  52. Meehl P. E. (1990a). Appraising and Amending Theories: The Strategy of Lakatosian Defense and Two Principles that Warrant it. Psychol. Inq. 1, 108– 141. 10.1207/s15327965pli010210.1207/s15327965pli0102_1 · DOI
  53. Meehl P. E. (1990b). Why Summaries of Research on Psychological Theories Are Often Uninterpretable. Psychol. Rep. 66, 195– 244. 10.2466/pr0.1990.66.1.195 · DOI
  54. Motyl M. Demos A. P. Carsel T. S. Hanson B. E. Melton Z. J. Mueller A. B. (2017). The State of Social and Personality Science: Rotten to the Core, Not So Bad, Getting Better, or Getting Worse? J. Personal. Soc. Psychol. 113, 34– 58. 10.1037/pspa0000084 28447837 · DOI
  55. Mou Y. Shi C. Shen T. Xu K. (2020). A Systematic Review of the Personality of Robot: Mapping its Conceptualization, Operationalization, Contextualization and Effects. Int. J. Human-Computer Interaction 36, 591– 605. 10.1080/10447318.2019.1663008 · DOI
  56. Munafò M. R. Nosek B. A. Bishop D. V. M. Button K. S. Chambers C. D. Du Percie Sert N. (2017). A Manifesto for Reproducible Science. Nat. Hum. Behav. 1, e124. 10.1038/s41562-016-0021 PMC7610724 33954258 · DOI · Paper page
  57. Muthukrishna M. Henrich J. (2019). A Problem in Theory. Nat. Hum. Behav. 3, 221– 229. 10.1038/s41562-018-0522-1 30953018 · DOI
  58. Nuijten M. B. (2021). “ Assessing and Improving Robustness of Psychological Research Findings in Four Steps” in Clinical Psychology and Questionable Research. Editors O'Donohue W. Masuda A. Lielienfeld S. (Springer). Psychol. questionable Res.
  59. Onnasch L. Roesler E. (2021). A Taxonomy to Structure and Analyze Human-Robot Interaction. Int. J. Soc. Robotics 13, 833– 849. 10.1007/s12369-020-00666-5 · DOI
  60. Orne M. T. (1962). On the Social Psychology of the Psychological experiment: With Particular Reference to Demand Characteristics and Their Implications. Am. Psychol. 17, 776– 783. 10.1037/h0043424 · DOI · Paper page
  61. Ötting S. K. Masjutin L. Steil J. J. Maier G. W. (2020). Let's Work Together: A Meta-Analysis on Robot Design Features that Enable Successful Human-Robot Interaction at Work. Hum. Factors, 18720820966433. 10.1177/0018720820966433 33176488 · DOI
  62. Reis H. T. (2019). “ How We Got Here from There: A Brief History of Social Psychology,” in Advanced Social Psychology: The State of the Science. Editors Finkel E. J. Baumeister R. F.. Second edition, 25– 60.
  63. Roberts A. C. Yeap Y. W. Seah H. S. Chan E. Soh C.-K. Christopoulos G. I. (2019). Assessing the Suitability of Virtual Reality for Psychological Testing. Psychol. Assess. 31, 318– 328. 10.1037/pas0000663 30802117 · DOI
  64. Roesler E. Manzey D. Onnasch L. (2021). A Meta-Analysis on the Effectiveness of Anthropomorphism in Human-Robot Interaction. Sci. robotics 6, eabj5425. 10.1126/scirobotics.abj5425 34516745 · DOI
  65. Romero F. (2019). Philosophy of Science and the Replicability Crisis. Philos. Compass 14. 10.1111/phc3.12633 · DOI
  66. Sabanovic S. Berry C. Bethel C. (2017). Introduction to the Special Issue on HRI Education. J. Human-Robot Interaction 6, 1. 10.5898/jhri.6.2.sabanovic · DOI
  67. Scheel A. M. Schijen M. R. M. J. Lakens D. (2021a). An Excess of Positive Results: Comparing the Standard Psychology Literature with Registered Reports. Adv. Methods Practices Psychol. Sci. 4, 251524592110074. 10.1177/25152459211007467 · DOI
  68. Schmidt S. (2009). Shall We Really Do it Again? the Powerful Concept of Replication Is Neglected in the Social Sciences. Rev. Gen. Psychol. 13, 90– 100. 10.1037/a0015108 · DOI · Paper page
  69. Schönbrodt F. D. Perugini M. (2013). At what Sample Size Do Correlations Stabilize? J. Res. Personal. 47, 609– 612. 10.1016/j.jrp.2013.05.009 · DOI · Paper page
  70. Seibt J. Vestergaard C. Damholdt M. F. (2021). The Complexity of Human Social Interactions Calls for Mixed Methods in HRI. J. Hum.-Robot Interact. 10, 1– 4. 10.1145/3439715 · DOI
  71. Simmons J. P. Nelson L. D. Simonsohn U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychol. Sci. 22, 1359– 1366. 10.1177/0956797611417632 22006061 · DOI · Paper page
  72. Stower R. Calvo-Barajas N. Castellano G. Kappas A. (2021). A Meta-Analysis on Children’s Trust in Social Robots. Int. J. Soc. Robotics. 10.1007/s12369-020-00736-8 · DOI
  73. Strait M. Lier F. Bernotat J. Wachsmuth S. Eyssel F. Goldstone R. (2020). “ A Three-Site Reproduction of the Joint Simon Effect with the NAO Robot,” in HRI '20: Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, 103– 111.
  74. Świątkowski W. Dompnier B. (2017). Replicability Crisis in Social Psychology: Looking at the Past to Find New Pathways for the Future. Int. Rev. Soc. Psychol. 30, 111– 124. 10.5334/irsp.66 · DOI
  75. Teo T. (2015). Critical Psychology: A Geography of Intellectual Engagement and Resistance. Am. Psychol. 70, 243– 254. 10.1037/a0038727 25621493 · DOI
  76. Ullman D. Aladia S. Malle B. F. (2021). Challenges and Opportunities for Replication Science in HRI. HRI '21: Proc. 2021 ACM/IEEE Int. Conf. Human-Robot Interaction, 110– 118.
  77. Veling L. McGinn C. (2021). Qualitative Research in HRI: A Review and Taxonomy. Int. J. Soc. Robotics. 10.1007/s12369-020-00723-z · DOI
  78. Wagenmakers E.-J. Wetzels R. Borsboom D. van der Maas H. L. J. (2011). Why Psychologists Must Change the Way They Analyze Their Data: the Case of Psi: Comment on Bem (2011). J. Personal. Soc. Psychol. 100, 426– 432. 10.1037/a0022790 21280965 · DOI · Paper page
  79. Weber J. (2005). Helpless Machines and True loving Care Givers: a Feminist Critique of Recent Trends in Human‐robot Interaction. J. Inf. Com Eth Soc. 3, 209– 218. 10.1108/14779960580000274 · DOI
  80. Wicherts J. M. (2011). Psychology Must Learn a Lesson from Fraud Case. Nature 480, 7. 10.1038/480007a 22129686 · DOI
  81. Wiggins B. J. Christopherson C. D. (2019). The Replication Crisis in Psychology: An Overview for Theoretical and Philosophical Psychology. J. Theor. Phil. Psychol. 39, 202– 217. 10.1037/teo0000137 · DOI
  82. Wigginton B. Lafrance M. N. (2019). Learning Critical Feminist Research: A Brief Introduction to Feminist Epistemologies and Methodologies. Feminism Psychol., 095935351986605. 10.1177/0959353519866058 · DOI
  83. Wijnen L. Bremner P. Lemaignan S. Giuliani M. (2020). “ Performing Human-Robot Interaction User Studies in Virtual Reality,” in 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 794.
  84. Wingen T. Berkessel J. B. Englich B. (2020). No Replication, No Trust? How Low Replicability Influences Trust in Psychology. Soc. Psychol. Personal. Sci. 11, 454– 463. 10.1177/1948550619877412 · DOI
  85. Winkle K. Melsión G. I. McMillan D. Leite I. (2021). “ Boosting Robot Credibility and Challenging Gender Norms in Responding to Abusive Behaviour,” in HRI '21 Companion: Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, 29– 37.
  86. Yuan F. Klavon E. Liu Z. Lopez R. P. Zhao X. (2021). A Systematic Review of Robotic Rehabilitation for Cognitive Training. Front. Robot. AI 8, 605715. 10.3389/frobt.2021.605715 34046433 PMC8144708 · DOI
  87. Zimmerman J. Forlizzi J. Evenson S. (2007). “ Research through Design as a Method for Interaction Design Research in HCI,” in CHI '07: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 493– 502.

Cited by

Sorted by citations

Authors
Leichtmann, B., Nitsch, V., & Mara, M.
Affiliations
  • Johannes Kepler University of Linz (AT)
  • RWTH Aachen University (DE)
Corresponding author
Leichtmann, Benedikt
ORCID
Venue
Frontiers in Robotics and AI
Publisher
Frontiers Media SA
Published
2022-03-11
Volume
9
Pages
838116-838116
ISSN
2296-9144 (ISSN-L)
DOI
10.3389/frobt.2022.838116
PMC
PMC8961736
OpenAlex
W4220808445
Cited by
66
References
20
Citations by year
2026: 17 · 2025: 19 · 2024: 20 · 2023: 9 · 2022: 1
FWCI
1.83
Type
journal article
Language
en
Open access
Licence
Locations
  • Frontiers in Robotics and AI (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • University Library Linz repository (Johannes Kepler Universitat Linz) (submittedVersion, cc-by)
  • RWTH Publications (RWTH Aachen) (submittedVersion, other-oa)
Topics
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords
human-robot interaction, replication crisis, replication study, user study, small sample size, meta-analysis, behavioral science

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review