CoChat
Sign Up
PreprintCited by 8Reinforcement Learning in RoboticsRobot Manipulation and LearningSocial Robot Interaction and HRI

The DOI matches its DataCite record; its reference list is not available.

Adaptive Coordination in Social Embodied Rearrangement

Open access
Szot, Andrew, Jain, Unnat, Batra, Dhruv, Kira, Zsolt, Desai, Ruta, Rai, Akshara·2023·arXiv
DOI 10.48550/arxiv.2306.00087arXiv 2306.00087
Open this paper in CoChat — free accountOpen PDF

Abstract

We present the task of "Social Rearrangement", consisting of cooperative everyday tasks like setting up the dinner table, tidying a house or unpacking groceries in a simulated multi-agent environment. In Social Rearrangement, two robots coordinate to complete a long-horizon task, using onboard sensing and egocentric observations, and no privileged information about the environment. We study zero-shot coordination (ZSC) in this task, where an agent collaborates with a new partner, emulating a scenario where a robot collaborates with a new human partner. Prior ZSC approaches struggle to generalize in our complex and visually rich setting, and on further analysis, we find that they fail to generate diverse coordination behaviors at training time. To counter this, we propose Behavior Diversity Play (BDP), a novel ZSC approach that encourages diversity through a discriminability objective. Our results demonstrate that BDP learns adaptive agents that can tackle visual coordination, and zero-shot generalize to new partners in unseen environments, achieving 35% higher success and 32% higher efficiency compared to baselines.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2306.00087

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

Cited by 8

Research works printed but not linked: 64

  1. Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sünderhauf, N., Reid, I., Gould, S., and van den Hengel, A. Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In CVPR, 2018.
  2. Barrett, S., Stone, P., and Kraus, S. Empirical evaluation of ad hoc teamwork in the pursuit domain. In The 10th International Conference on Autonomous Agents and Multiagent Systems- Volume 2, pp. 567-574, 2011.
  3. Batra, D., Chang, A. X., Chernova, S., Davison, A. J., Deng, J., Koltun, V., Levine, S., Malik, J., Mordatch, I., Mottaghi, R., et al. Rearrangement: A challenge for embodied ai. arXiv preprint arXiv:2011.01975, 2020a. 2, 4 · arXiv
  4. Batra, D., Gokaslan, A., Kembhavi, A., Maksymets, O., Mottaghi, R., Savva, M., Toshev, A., and Wijmans, E. Objectnav revisited: On evaluation of embodied agents navigating to objects. arXiv preprint arXiv:2006.13171, 2020b. · arXiv
  5. Calli, B., Singh, A., Walsman, A., Srinivasa, S., Abbeel, P., and Dollar, A. M. The ycb object and model set: Towards common benchmarks for manipulation research. In 2015 international conference on advanced robotics (ICAR), pp. 510-517. IEEE, 2015.
  6. Carroll, M., Shah, R., Ho, M. K., Griffiths, T., Seshia, S., Abbeel, P., and Dragan, A. On the utility of learning about humans for human-ai coordination. Advances in neural information processing systems, 32, 2019. 1, 3, 4, 7
  7. Chaplot, D. S., Salakhutdinov, R., Gupta, A., and Gupta, S. Neural topological slam for visual navigation. CVPR, 2020.
  8. Charakorn, R., Manoonpong, P., and Dilokthanakul, N. Investigat- ing partner diversification methods in cooperative multi-agent deep reinforcement learning. In International Conference on Neural Information Processing, pp. 395-402. Springer, 2020.
  9. Chattopadhyay, P., Hoffman, J., Mottaghi, R., and Kembhavi, A. Robustnav: Towards benchmarking robustness in embodied navigation. ICCV, 2021.
  10. Chen, B., Song, S., Lipson, H., and Vondrick, C. Visual hide and seek. arXiv preprint arXiv:1910.07882, 2019. · arXiv
  11. Chen, C., Jain, U., Schissler, C., Gari, S. V. A., Al-Halah, Z., Ithapu, V. K., Robinson, P., and Grauman, K. Soundspaces: Audio-visual navigation in 3d environments. ECCV, 2020.
  12. Choudhury, R., Swamy, G., Hadfield-Menell, D., and Dragan, A. D. On the utility of model learning in hri. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 317-325. IEEE, 2019.
  13. Cully, A., Clune, J., Tarapore, D., and Mouret, J.-B. Robots that can adapt like animals. Nature, 521(7553):503-507, 2015.
  14. Derek, K. and Isola, P. Adaptable agent populations via a gen- erative model of policies. Advances in Neural Information Processing Systems, 34:3902-3913, 2021.
  15. Ehsani, K., Han, W., Herrasti, A., VanderBilt, E., Weihs, L., Kolve, E., Kembhavi, A., and Mottaghi, R. Manipulathor: A framework for visual object manipulation. In CVPR, 2021.
  16. Eysenbach, B., Gupta, A., Ibarz, J., and Levine, S. Diversity is all you need: Learning skills without a reward function. arXiv preprint arXiv:1802.06070, 2018. 3, 5 Fetch. Fetch. http://fetchrobotics.com/, 2020. 2, 3 · arXiv
  17. Gan, C., Schwartz, J., Alter, S., Mrowca, D., Schrimpf, M., Traer, J., De Freitas, J., Kubilius, J., Bhandwaldar, A., Haber, N., et al. Threedworld: A platform for interactive multi-modal physical simulation. NeurIPS Datasets and Benchmarks Track, 2021.
  18. Gu, J., Chaplot, D. S., Su, H., and Malik, J. Multi-skill mo- bile manipulation for object rearrangement. arXiv preprint arXiv:2209.02778, 2022. · arXiv
  19. He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. CVPR, 2016. 14
  20. Heinrich, J. and Silver, D. Deep reinforcement learning from self-play in imperfect-information games. arXiv preprint arXiv:1603.01121, 2016. 3, 6, 7, 14 · arXiv
  21. Heinrich, J., Lanctot, M., and Silver, D. Fictitious self-play in extensive-form games. In International conference on machine learning, pp. 805-813. PMLR, 2015.
  22. Hochreiter, S. and Schmidhuber, J. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.
  23. Hu, H., Lerer, A., Peysakhovich, A., and Foerster, J. "other-play" for zero-shot coordination. In International Conference on Machine Learning, pp. 4399-4410. PMLR, 2020.
  24. Hu, H., Lerer, A., Cui, B., Pineda, L., Brown, N., and Foerster, J. Off-belief learning. In International Conference on Machine Learning, pp. 4369-4379. PMLR, 2021.
  25. Jaderberg, M., Dalibard, V., Osindero, S., Czarnecki, W. M., Don- ahue, J., Razavi, A., Vinyals, O., Green, T., Dunning, I., Si- monyan, K., et al. Population based training of neural networks. arXiv preprint arXiv:1711.09846, 2017. 3, 14 · arXiv
  26. Jain, U., Weihs, L., Kolve, E., Rastegari, M., Lazebnik, S., Farhadi, A., Schwing, A. G., and Kembhavi, A. Two body problem: Collaborative visual task completion. In CVPR, 2019.
  27. Jain, U., Weihs, L., Kolve, E., Farhadi, A., Lazebnik, S., Kembhavi, A., and Schwing, A. G. A cordial sync: Going beyond marginal policies for multi-agent embodied tasks. In ECCV, 2020.
  28. Jain, U., Liu, I.-J., Lazebnik, S., Kembhavi, A., Weihs, L., and Schwing, A. G. Gridtopix: Training embodied agents with minimal supervision. In ICCV, 2021.
  29. Juliani, A., Berges, V.-P., Teng, E., Cohen, A., Harper, J., Elion, C., Goy, C., Gao, Y., Henry, H., Mattar, M., et al. Unity: A general platform for intelligent agents. arXiv preprint arXiv:1809.02627, 2018. · arXiv
  30. Kolve, E., Mottaghi, R., Han, W., VanderBilt, E., Weihs, L., Her- rasti, A., Gordon, D., Zhu, Y., Gupta, A., and Farhadi, A. AI2- THOR: an interactive 3d environment for visual AI. arXiv preprint arXiv:1712.05474, 2019. · arXiv
  31. Kurach, K., Raichuk, A., Stańczyk, P., Zajkac, M., Bachem, O., Espeholt, L., Riquelme, C., Vincent, D., Michalski, M., Bous- quet, O., et al. Google research football: A novel reinforcement learning environment. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 4501-4510, 2020.
  32. Lanctot, M., Zambaldi, V., Gruslys, A., Lazaridou, A., Tuyls, K., Pérolat, J., Silver, D., and Graepel, T. A unified game-theoretic approach to multiagent reinforcement learning. Advances in neural information processing systems, 30, 2017.
  33. Lupu, A., Cui, B., Hu, H., and Foerster, J. Trajectory diversity for zero-shot coordination. In International Conference on Machine Learning, pp. 7204-7213. PMLR, 2021. 3, 5, 6, 7, 12, 15
  34. Mahajan, A., Rashid, T., Samvelyan, M., and Whiteson, S. Maven: Multi-agent variational exploration. Advances in Neural Infor- mation Processing Systems, 32, 2019.
  35. McKee, K. R., Leibo, J. Z., Beattie, C., and Everett, R. Quantifying the effects of environment and population diversity in multi- agent reinforcement learning. Autonomous Agents and Multi- Agent Systems, 36(1):1-16, 2022. 3, 8
  36. Padmakumar, A., Thomason, J., Shrivastava, A., Lange, P., Narayan-Chen, A., Gella, S., Piramithu, R., Tur, G., and Hakkani-Tur, D. TEACh: Task-driven embodied agents that chat. arXiv, 2021. URL https://arxiv.org/abs/ 2110.00534.
  37. Patel, S., Wani, S., Jain, U., Schwing, A., Lazebnik, S., Savva, M., and Chang, A. Interpretation of emergent communication in heterogeneous collaborative embodied agents. ICCV, 2021.
  38. Premack, D. and Woodruff, G. Does the chimpanzee have a theory of mind? Behavioral and brain sciences, 1(4):515-526, 1978.
  39. Pugh, J. K., Soros, L. B., and Stanley, K. O. Quality diversity: A new frontier for evolutionary computation. Frontiers in Robotics and AI, pp. 40, 2016.
  40. Puig, X., Ra, K., Boben, M., Li, J., Wang, T., Fidler, S., and Torralba, A. Virtualhome: Simulating household activities via programs. In CVPR, 2018.
  41. Puig, X., Shu, T., Li, S., Wang, Z., Liao, Y.-H., Tenenbaum, J. B., Fidler, S., and Torralba, A. Watch-and-help: A challenge for social perception and human-ai collaboration. arXiv preprint arXiv:2010.09890, 2020. · arXiv
  42. Rahman, A., Fosong, E., Carlucho, I., and Albrecht, S. V. Towards robust ad hoc teamwork agents by creating diverse training teammates. arXiv preprint arXiv:2207.14138, 2022. · arXiv
  43. Roman, H. R., Bisk, Y., Thomason, J., Celikyilmaz, A., and Gao, J. Rmm: A recursive mental model for dialog navigation. In EMNLP Findings, 2020.
  44. Savva, M., Kadian, A., Maksymets, O., Zhao, Y., Wijmans, E., Jain, B., Straub, J., Liu, J., Koltun, V., Malik, J., Parikh, D., and Batra, D. Habitat: A Platform for Embodied AI Research. ICCV, 2019. 2, 4
  45. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. · arXiv
  46. Sclar, M., Neubig, G., and Bisk, Y. Symmetric machine theory of mind. In International Conference on Machine Learning, pp. 19450-19466. PMLR, 2022.
  47. Shridhar, M., Thomason, J., Gordon, D., Bisk, Y., Han, W., Mot- taghi, R., Zettlemoyer, L., and Fox, D. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In CVPR, 2020.
  48. Stone, P., Kaminka, G. A., Kraus, S., and Rosenschein, J. S. Ad hoc autonomous agent teams: Collaboration without pre- coordination. In Twenty-Fourth AAAI Conference on Artificial Intelligence, 2010.
  49. Strouse, D., McKee, K., Botvinick, M., Hughes, E., and Everett, R. Collaborating with humans without human data. Advances in Neural Information Processing Systems, 34:14502-14515, 2021. 1, 3, 4, 6, 7, 15
  50. Szot, A., Clegg, A., Undersander, E., Wijmans, E., Zhao, Y., Turner, J., Maestre, N., Mukadam, M., Chaplot, D., Maksymets, O., et al. Habitat 2.0: Training home assistants to rearrange their habitat. arXiv preprint arXiv:2106.14405, 2021. 2, 4, 6, 12, 13 · arXiv
  51. Team, O. E. L., Stooke, A., Mahajan, A., Barros, C., Deck, C., Bauer, J., Sygnowski, J., Trebacz, M., Jaderberg, M., Mathieu, M., et al. Open-ended learning leads to generally capable agents. arXiv preprint arXiv:2107.12808, 2021. · arXiv
  52. Thomason, J., Murray, M., Cakmak, M., and Zettlemoyer, L. Vision-and-dialog navigation. In CoRL, 2020.
  53. Wang, C., Pérez-D'Arpino, C., Xu, D., Fei-Fei, L., Liu, K., and Savarese, S. Co-gail: Learning diverse strategies for human- robot collaboration. In Conference on Robot Learning, pp. 1279-1290. PMLR, 2022.
  54. Wani, S., Patel, S., Jain, U., Chang, A., and Savva, M. Mul- tion: Benchmarking semantic map memory using multi-object navigation. NeurIPS, 2020.
  55. Weihs, L., Jain, U., Salvador, J., Lazebnik, S., Kembhavi, A., and Schwing, A. Bridging the imitation gap by adaptive insubordi- nation. arXiv preprint arXiv:2007.12173, 2020a. · arXiv
  56. Weihs, L., Salvador, J., Kotar, K., Jain, U., Zeng, K.-H., Mottaghi, R., and Kembhavi, A. Allenact: A framework for embodied ai research. arXiv preprint arXiv:2008.12760, 2020b. · arXiv
  57. Weihs, L., Deitke, M., Kembhavi, A., and Mottaghi, R. Visual room rearrangement. In CVPR, 2021a. 2
  58. Weihs, L., Kembhavi, A., Ehsani, K., Pratt, S. M., Han, W., Her- rasti, A., Kolve, E., Schwenk, D., Mottaghi, R., and Farhadi, A. Learning generalizable visual representations via interactive gameplay. In ICLR, 2021b.
  59. Wijmans, E., Kadian, A., Morcos, A., Lee, S., Essa, I., Parikh, D., Savva, M., and Batra, D. Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames. ICLR, 2019.
  60. Wu, S. A., Wang, R. E., Evans, J. A., Tenenbaum, J. B., Parkes, D. C., and Kleiman-Weiner, M. Too many cooks: Bayesian inference for coordinating multi-agent collaboration. Topics in Cognitive Science, 13(2):414-432, 2021.
  61. Xia, F., Zamir, A. R., He, Z., Sax, A., Malik, J., and Savarese, S. Gibson env: Real-world perception for embodied agents. CVPR, 2018.
  62. Xia, F., Shen, W. B., Li, C., Kasimbeg, P., Tchapmi, M., Toshev, A., Martín-Martín, R., and Savarese, S. Interactive gibson: A benchmark for interactive navigation in cluttered environments. arXiv preprint arXiv:1910.14442, 2019. · arXiv
  63. Xiang, F., Qin, Y., Mo, K., Xia, Y., Zhu, H., Liu, F., Liu, M., Jiang, H., Yuan, Y., Wang, H., Yi, L., Chang, A. X., Guibas, L. J., and Su, H. Sapien: A simulated part-based interactive environment. In CVPR, 2020.
  64. Zhao, R., Song, J., Haifeng, H., Gao, Y., Wu, Y., Sun, Z., and Wei, Y. Maximum entropy population based training for zero-shot human-ai coordination. arXiv preprint arXiv:2112.11701, 2021. 3 · arXiv

Metadata

Authors
Szot, A., Jain, U., Batra, D., Kira, Z., Desai, R., & Rai, A.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2306.00087
arXiv
2306.00087
OpenAlex
W4379251682
Cited by
8
References
0
Citations by year
2025: 4 · 2024: 3 · 2023: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
multi-agent coordination, human-robot collaboration, behavioral diversity, embodied robots, egocentric vision, long-horizon tasks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review