CoChat
Sign Up
Cited by 17Categorization, perception, and languageNatural Language Processing TechniquesMultimodal Machine Learning Applications

The DOI matches its Crossref record; the references it cites are not checked yet.

Online perceptual learning and natural language acquisition for autonomous robots

Open access
Alomari, Muhannad, Li, Fangjun, Hogg, David C., Cohn, Anthony G.·2022·Artificial Intelligence
DOI 10.1016/j.artint.2021.103637
Open this paper in CoChat — free accountOpen PDF

Abstract

In this work, the problem of bootstrapping knowledge in language and vision for autonomous robots is addressed through novel techniques in grammar induction and word grounding to the perceptual world. In particular, we demonstrate a system, called OLAV, which is able, for the first time, to (1) learn to form discrete concepts from sensory data; (2) ground language (n-grams) to these concepts; (3) induce a grammar for the language being used to describe the perceptual world; and moreover to do all this incrementally, without storing all previous data. The learning is achieved in a loosely-supervised manner from raw linguistic and visual data. Moreover, the learnt model is transparent, rather than a black-box model and is thus open to human inspection. The visual data is collected using three different robotic platforms deployed in real-world and simulated environments and equipped with different sensing modalities, while the linguistic data is collected using online crowdsourcing tools and volunteers. The analysis performed on these robots demonstrates the effectiveness of the framework in learning visual concepts, language groundings and grammatical structure in these three online settings.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.artint.2021.103637

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

The citation lists haven’t been fetched yet

References 99 · Cited by 17

Research works printed but not linked: 32

  1. Winograd, 1972, Understanding natural language, Cogn. Psychol., 3, 1, 10.1016/0010-0285(72)90002-3 · DOI
  2. Siskind, 1996, A computational study of cross-situational techniques for learning word-to-meaning mappings, Cognition, 61, 39, 10.1016/S0010-0277(96)00728-7 · DOI
  3. Needham, 2005, Protocols from perceptual observations, Artif. Intell., 167, 103, 10.1016/j.artint.2005.04.006 · DOI
  4. Yu, 2015, A compositional framework for grounding language inference, generation, and acquisition in video, J. Artif. Intell. Res., 52, 601, 10.1613/jair.4556 · DOI
  5. Tellex, 2020, Robots that use language, Annu. Rev. Control, Robot. Auton. Syst., 3, 25, 10.1146/annurev-control-101119-071628 · DOI
  6. Dukes, 2014, Semeval-2014 task 6: supervised semantic parsing of robotic spatial commands, SemEval, 2014, 45
  7. Chen, 2011, Learning to Interpret Natural Language Navigation Instructions from Observations, vol. 2, 1
  8. Nevens, 2020, From continuous observations to symbolic concepts: a discrimination-based strategy for grounded concept learning, Front. Robot. AI, 7, 84, 10.3389/frobt.2020.00084 · DOI
  9. Lauria, 2002, Mobile robot programming using natural language, Robot. Auton. Syst., 38, 171, 10.1016/S0921-8890(02)00166-5 · DOI
  10. Tellex, 2011, Approaching the symbol grounding problem with probabilistic graphical models, AI Mag., 32, 64
  11. Barrett, 2018, Driving under the influence (of language), IEEE Trans. Neural Netw. Learn. Syst., 29, 2668, 10.1109/TNNLS.2017.2693278 · DOI
  12. Roesler, 2019, Action learning and grounding in simulated human-robot interactions, Knowl. Eng. Rev., 34, e13, 10.1017/S0269888919000079 · DOI
  13. Thomason, 2017, Opportunistic active learning for grounding natural language descriptions, vol. 78, 67
  14. Steels, 2000, Aibo's first words: the social learning of language and meaning, Evol. Commun., 4, 3, 10.1075/eoc.4.1.03ste · DOI
  15. She, 2014, Back to the blocks world: learning new actions through situated human-robot dialogue, vol. 89, 89
  16. Thomason, 2020, Jointly improving parsing and perception for natural language commands through human-robot dialog, J. Artif. Intell. Res., 67, 327, 10.1613/jair.1.11485 · DOI
  17. Roy, 2002, Learning visually grounded words and syntax for a scene description task, Comput. Speech Lang., 16, 353, 10.1016/S0885-2308(02)00024-4 · DOI
  18. Mahalanobis, 1936, On the generalized distance in statistics, Proc. Natl. Inst. Sci. (Calcutta), 2, 49
  19. Sridhar, 2010, Discovering an Event Taxonomy from Video Using Qualitative Spatio-Temporal Graphs, vol. 215, 1103
  20. Allen, 1983, Maintaining knowledge about temporal intervals, Commun. ACM, 26, 832, 10.1145/182.358434 · DOI · Paper page
  21. Dubba, 2015, Learning relational event models from video, J. Artif. Intell. Res., 10.1613/jair.4395 · DOI
  22. Howorka, 1977, A characterization of distance-hereditary graphs, Q. J. Math., 28, 417, 10.1093/qmath/28.4.417 · DOI
  23. Jones, 1972, A statistical interpretation of term specificity and its application in retrieval, J. Doc., 28, 11, 10.1108/eb026526 · DOI
  24. Dominey, 2005, Developmental stages of perception and language acquisition in a perceptually grounded robot, Cogn. Syst. Res., 6, 243, 10.1016/j.cogsys.2004.11.005 · DOI
  25. Lari, 1990, The estimation of stochastic context-free grammars using the inside-outside algorithm, Comput. Speech Lang., 4, 35, 10.1016/0885-2308(90)90022-X · DOI
  26. Rosenberg, 2007, V-measure: a conditional entropy-based external cluster evaluation measure, vol. 7, 410
  27. Rabiner, 1989, A tutorial on hidden Markov models and selected applications in speech recognition, Proc. IEEE, 77, 257, 10.1109/5.18626 · DOI · Paper page
  28. Abney, 1996, Partial parsing via finite-state cascades, Nat. Lang. Eng., 2, 337, 10.1017/S1351324997001599 · DOI
  29. Younger, 1967, Recognition and parsing of context-free languages in time n3, Inf. Control, 10, 189, 10.1016/S0019-9958(67)80007-X · DOI · Paper page
  30. Danna, 2005, Exploring relaxation induced neighborhoods to improve MIP solutions, Math. Program., 102, 71, 10.1007/s10107-004-0518-7 · DOI
  31. Glover, 1990, Tabu search—part ii, ORSA J. Comput., 2, 4, 10.1287/ijoc.2.1.4 · DOI
  32. Bennett, 2016, Defining Relations: A General Incremental Approach with Spatial Temporal Case Studies, vol. 263, 23

Metadata

Authors
Alomari, M., Li, F., Hogg, D. C., & Cohn, A. G.
Affiliations
  • University of Leeds (GB)
  • Tongji University (CN)
Corresponding author
Cohn, Anthony G.
ORCID
Venue
Artificial Intelligence
Publisher
Elsevier BV
Published
2022-02
Volume
303
Pages
103637
ISSN
0004-3702 (ISSN-L)
DOI
10.1016/j.artint.2021.103637
OpenAlex
W2775703036
Cited by
17
References
99
Citations by year
2026: 3 · 2025: 5 · 2024: 6 · 2023: 1 · 2022: 1
FWCI
1.93
Type
journal article
Language
en
Open access
Licence
Locations
  • Artificial Intelligence (publishedVersion, cc-by)
  • White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
Funders
  • FP7-ICT (600623)
  • Alan Turing Institute
  • Horizon 2020 (825619)
  • Seventh Framework Programme
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Categorization, perception, and language
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
natural language acquisition, autonomous robots, grammatical inference, concept learning, n-gram language models, transparent models, visual concepts

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Online perceptual learning and natural language acquisition for autonomous robots. https://app.cochat.ai/papers/10.1016/j.artint.2021.103637
The paper
Alomari, M., Li, F., Hogg, D. C., & Cohn, A. G. (2022). Online perceptual learning and natural language acquisition for autonomous robots. Artificial Intelligence. https://doi.org/10.1016/j.artint.2021.103637