The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Evaluating Fluency in Human–Robot Collaboration
Abstract
Collaborative fluency is the coordinated meshing of joint activities between members of a well-synchronized team. In recent years, researchers in human-robot collaboration have been developing robots to work alongside humans aiming not only at task efficiency, but also at human-robot fluency. As part of this effort, we have developed a number of metrics to evaluate the level of fluency in human-robot shared-location teamwork. While these metrics are being used in existing research, there has been no systematic discussion on how to measure fluency and how the commonly used metrics perform and compare. In this paper, we codify subjective and objective human-robot fluency metrics, provide an analytical model for four objective metrics, and assess their dynamics in a turn-taking framework. We also report on a user study linking objective and subjective fluency metrics and survey recent use of these metrics in the literature.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
Development and validation of the working alliance inventory
Horvath, A. O., & Greenberg, L. S. (1989) · Journal of Counseling Psychology · doi:10.1037//0022-0167.36.2.223 · Cited by 3,520
Cognitive Load Measurement as a Means to Advance Cognitive Load Theory
Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. M. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_8 · Cited by 2,527
MUIR, B. M., & MORAY, N. (1996) · Ergonomics · doi:10.1080/00140139608964474 · Cited by 993
Expressing thought: Improving robot readability with animation principles
Takayama, L., Dooley, D., & Ju, W. (2011) · Proceedings of the 6th international conference on Human-robot interaction · doi:10.1145/1957656.1957674 · Cited by 289
Decision-making authority, team efficiency and human worker satisfaction in mixed human–robot teams
Gombolay, M. C., Gutierrez, R. A., Clarke, S. G., Sturla, G. F., & Shah, J. A. (2015) · Autonomous Robots · doi:10.1007/s10514-015-9457-9 · Cited by 256
Effects of Robot Motion on Human-Robot Collaboration
Dragan, A. D., Bauman, S., Forlizzi, J., & Srinivasa, S. S. (2015) · Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2696454.2696473 · Cited by 247
Improved human-robot team performance using chaski, a human-inspired plan execution system
Shah, J., Wiken, J., Williams, B., & Breazeal, C. (2011) · Proceedings of the 6th international conference on Human-robot interaction · doi:10.1145/1957656.1957668 · Cited by 218
Collaboration in Human-Robot Teams
Hoffman, G., & Breazeal, C. (2004) · AIAA 1st Intelligent Systems Technical Conference · doi:10.2514/6.2004-6434 · Cited by 208
Self-citationHuman-robot mutual adaptation in collaborative tasks: Models and experiments
Nikolaidis, S., Hsu, D., & Srinivasa, S. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364917690593 · Cited by 199
Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks
Nikolaidis, S., Ramakrishnan, R., Gu, K., & Shah, J. (2015) · Cited by 192
Cost-Based Anticipatory Action Selection for Human–Robot Fluency
Hoffman, G., & Breazeal, C. (2007) · IEEE Transactions on Robotics · doi:10.1109/tro.2007.907483 · Cited by 177
Self-citationWhose job is it anyway? A study of human-robot interaction in a collaborative task
Hinds, P., Roberts, T., & Jones, H. (2004) · Human-Computer Interaction · doi:10.1207/s15327051hci1901&2_7 · Cited by 173
Gombolay, M., Bair, A., Huang, C., & Shah, J. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364916688255 · Cited by 172
A dancing robot for rhythmic social interaction
Michalowski, M. P., Sabanovic, S., & Kozima, H. (2007) · Proceedings of the ACM/IEEE international conference on Human-robot interaction · doi:10.1145/1228716.1228729 · Cited by 160
Using spatial and temporal contrast for fluent robot-human hand-overs
Çakmak, M., Srinivasa, S. S., Lee, M. K., Kiesler, S. B., & Forlizzi, J. (2011) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/1957656.1957823 · Cited by 143
Nikolaidis, S., & Shah, J. (2013) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2013.6483499 · Cited by 137
Adaptive Coordination Strategies for Human-Robot Handovers
Huang, C., Çakmak, M., & Mutlu, B. (2015) · doi:10.15607/rss.2015.xi.031 · Cited by 134
Computational Human-Robot Interaction
Thomaz, A. L., Hoffman, G., & Çakmak, M. (2016) · Foundations and Trends in Robotics · doi:10.1561/2300000049 · Cited by 133
Self-citationHow quickly should communication robots respond?
Shiwa, T., Kanda, T., Imai, M., Ishiguro, H., & Hagita, N. (2008) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/1349822.1349843 · Cited by 119
Nikolaidis, S., & Shah, J. (2013) · doi:10.5555/2447556.2447563 · Cited by 115
Initiative in robot assistance during collaborative task execution
Baraglia, J., Çakmak, M., Nagai, Y., Rao, R. P. N., & Asada, M. (2016) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451735 · Cited by 96
Unhelkar, V. V., Siu, H. C., & Shah, J. (2014) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/2559636.2559655 · Cited by 92
Flexible cooperation between human and robot by interpreting human intention from gaze information
Sakita, K., Ogawam, K., Murakami, S., Kawamura, K., & Ikeuchi, K. (2005) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2004.1389458 · Cited by 82
Effects of anticipatory perceptual simulation on practiced human-robot tasks
Hoffman, G., & Breazeal, C. (2009) · Autonomous Robots · doi:10.1007/s10514-009-9166-3 · Cited by 78
Self-citationTiming in Multimodal Turn-Taking Interactions: Control and Analysis Using Timed Petri Nets
Chao, C., & Thomaz, A. L. (2012) · Journal of Human-Robot Interaction · doi:10.5898/jhri.1.1.chao · Cited by 68
Rahman, S. M. M. (2017) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2017.7510760 · Cited by 58
Dynamic-Autonomy for Urban Search and Rescue
Bruemmer, D. J., Dudenhoeffer, D. D., & Marble, J. L. (2002) · Cited by 56
Robot-human interaction with an anthropomorphic percussionist
Weinberg, G., & Driscoll, S. (2006) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124957 · Cited by 54
Human-Centered Robotics and Interactive Haptic Simulation
Khatib, O., Brock, O., Chang, K.-S., Ruspini, D. C., Sentis, L., & Viji, S. (2004) · The International Journal of Robotics Research · doi:10.1177/0278364904041325 · Cited by 54
Task-model based human robot cooperation using vision
Kimura, H., Horiuchi, T., & Ikeuchi, K. (2003) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.1999.812762 · Cited by 42
Coordination of Human-Robot Teaming with Human Task Preferences.
Gombolay, M., Huang, C., & Shah, J. (2015) · Cited by 40
“Me and you together” movement impact in multi-user collaboration tasks
Faria, M., Silva, R., Alves‐Oliveira, P., Melo, F. S., & Paiva, A. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206109 · Cited by 17
Cited by
Larson, L., & DeChurch, L. A. (2020) · The Leadership Quarterly · doi:10.1016/j.leaqua.2019.101377 · Cited by 358
Coronado, E., Kiyokawa, T., Ricardez, G. A. G., Ramirez-Alpizar, I. G., Venture, G., & Yamanobe, N. (2022) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2022.04.007 · Cited by 276
Towards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations
Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3331846 · Cited by 278
van Zoelen, E. M., van den Bosch, K., & Neerincx, M. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.692811 · Cited by 31
Metadata
- Authors
- Hoffman, G.
- Venue
- IEEE Transactions on Human-Machine Systems
- Published
- 2019-06
- Volume
- 49
- Issue
- 3
- Pages
- 209-218
- ISSN
- 2168-2291 (ISSN-L)
- 2168-2305
- OpenAlex
- W2927819804
- Cited by
- 454
- References
- 33
- FWCI
- 58.35
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.