Challenges in Task Incremental Learning for Assistive Robotics
Open accessAbstract
Recent breakthroughs in computer vision areas, ranging from detection, segmentation, to classification, rely on the availability of large-scale representative training datasets. Yet, robotic vision poses new challenges towards applying visual algorithms developed from these datasets because the latter implicitly assume a fixed set of categories and time-invariant distribution of tasks. In practice, assistive robots should be able to operate in dynamic environments with everyday changes. The variations of four commonly observed factors, including illumination, occlusion, camera-object distance/angles and clutter, could make lifelong/continual learning in computer vision more challenging. Large-scale datasets previously made publicly available were relatively simple, and rarely include such real-world challenges in data collection. Benefited from the recent released OpenLORIS-Object dataset, which explicitly includes these real-world challenges in the lifelong object recognition task, we evaluate three most adopted regularization methods in lifelong/continual learning (Learning without Forgetting, Elastic Weights Consolidation, and Synaptic Intelligence). Their performances were compared with the naive and cumulative training modes as the lower bound and upper bound of performances, respectively. The experiments conducted on the dataset focused on task incremental learning, i.e., incremental difficulty based on the four environment of factors. However, all the three most reported lifelong/continual learning algorithms have failed with the increase in encountered batches across various metrics with indistinguishable performance comparing to the naive training mode. Our results highlight the current challenges in lifelong object recognition for assistive robots to operate in real-world dynamic scene.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Through testing on OpenLORIS-Object, Mozaffari et al. found that EWC, SI, and LwF have little effect on solving lifelong vision challenges for assistive robotics.
“Through testing based on the OpenLORIS-Object dataset, they found that the three most adopted regularization methods in lifelong learning (EWC, SI and LwF) have little effect on solving the lifelong vision challenges for assistive robotics.”
From An Appraisal of Incremental Learning Methods (2020) · cites this paper as [2]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755 · see citing works
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899 · see citing works
McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256 · see citing works
Li, Z., & Hoiem, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2773081 · Cited by 4,202 · see citing works
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770 · see citing works
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117 · see citing works
Three scenarios for continual learningPreprint
van de Ven, G. M., & Tolias, A. S. (2019) · Lirias · doi:10.48550/arxiv.1904.07734 · Cited by 714 · see citing works
Variational Continual Learning
Nguyen, C. V., Li, Y., Bui, T. D., & Turner, R. E. (2017) · arXiv · doi:10.48550/arxiv.1710.10628 · Cited by 491 · see citing works
Myoelectric forearm prostheses: State of the art from a user-centered perspective
Peerdeman, B., Boere, D., Witteveen, H. J. B., `tVeld, R. H. I., Hermens, H. J., Stramigioli, S., Rietman, H. S., Veltink, P. H., & Misra, S. (2011) · The Journal of Rehabilitation Research and Development · doi:10.1682/jrrd.2010.08.0161 · Cited by 467 · see citing works
Zhai, X., Jelfs, B., Chan, R. H. M., & Tin, C. (2017) · Frontiers in Neuroscience · doi:10.3389/fnins.2017.00379 · Cited by 361 · see citing works
Self-citation(2013) · Cited by 290 · see citing works
Are We Ready for Service Robots? The OpenLORIS-Scene Datasets for Lifelong SLAM
Shi, X., Li, D., Zhao, P., Tian, Q., Tian, Y., Long, Q., Zhu, C., Song, J., Qiao, F., Song, L., et al. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9196638 · Cited by 185 · see citing works
Self-citationDeep learning-based artificial vision for grasp classification in myoelectric hands
Ghazaei, G., Alameer, A., Degenaar, P. A., Morgan, G., & Nazarpour, K. (2017) · Journal of Neural Engineering · doi:10.1088/1741-2552/aa6802 · Cited by 178 · see citing works
Chen, Z., & Liu, B. (2018) · Synthesis Lectures on Artificial Intelligence and Machine Learning · doi:10.2200/s00737ed1v01y201610aim033 · Cited by 154 · see citing works
Sensor fusion and computer vision for context-aware control of a multi degree-of-freedom prosthesis
Marković, M., Dosen, S. S., Popović, D. B., et al. (2015) · Journal of Neural Engineering · doi:10.1088/1741-2560/12/6/066022 · Cited by 138 · see citing works
Don't forget, there is more than forgetting: new metrics for Continual LearningPreprint
Díaz-Rodríguez, N., Lomonaco, V., Filliat, D., & Maltoni, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.13166 · Cited by 102 · see citing works
Enhancing brain-machine interface (BMI) control of a hand exoskeleton using electrooculography (EOG)
Witkowski, M., Cortese, M., Cempini, M., Mellinger, J., Vitiello, N., & Soekadar, S. R. (2014) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/1743-0003-11-165 · Cited by 83 · see citing works
Are We Ready for Service Robots? The OpenLORIS-Scene Datasets for Lifelong SLAMPreprint
Shi, X., Li, D., Zhao, P., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.05603 · Cited by 19 · see citing works
Self-citationOpenLORIS-Object: A Dataset and Benchmark towards Lifelong Object Recognition
She, Q., Feng, F., Hao, X., Yang, Q., Lan, C., Lomonaco, V., Shi, X., Wang, Z., Guo, Y., Zhang, Y., et al. (2019) · Cited by 10 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 3
- H. Shin, J. K. Lee, J. Kim, and J. Kim, ''Continual learning with deep generative replay,'' in Proc. Adv. Neural Inf. Process. Syst., Long Beach, CA, USA, Dec. 2017, pp. 2990-2999.
- F. Zenke, B. Poole, and S. Ganguli, ''Continual learning through synap- tic intelligence,'' in Proc. 34th Int. Conf. Mach. Learn., vol. 70, 2017, pp. 3987-3995.
- S.-W. Lee, J.-H. Kim, J. Jun, J.-W. Ha, and B.-T. Zhang, ''Overcom- ing catastrophic forgetting by incremental moment matching,'' in Proc. Adv. Neural Inf. Process. Syst., Long Beach, CA, USA, Dec. 2017, pp. 4652-4662.
Cited by
Sorted by citations
Are We Ready for Service Robots? The OpenLORIS-Scene Datasets for Lifelong SLAM
Shi, X., Li, D., Zhao, P., Tian, Q., Tian, Y., Long, Q., Zhu, C., Song, J., Qiao, F., Song, L., et al. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9196638 · Cited by 185 · see citing works
Self-citationAn Appraisal of Incremental Learning Methods
Luo, Y., Yin, L., Bai, W., & Mao, K. (2020) · Entropy · doi:10.3390/e22111190 · Cited by 120 · see citing works
Replay in Deep Learning: Current Approaches and Missing Biological Elements
Hayes, T. L., Krishnan, G. P., Bazhenov, M., Siegelmann, H. T., Sejnowski, T. J., & Kanan, C. (2021) · Neural Computation · doi:10.1162/neco_a_01433 · Cited by 106 · see citing works
- Authors
- Feng, F., Chan, R. H. M., Shi, X., Zhang, Y., & She, Q.
- Affiliations
- City University of Hong Kong (HK)
- Intel (United States) (US)
- Venue
- IEEE Access
- Published
- 2020
- Volume
- 8
- Pages
- 3434-3441
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2991061683
- Cited by
- 39
- References
- 21
- Citations by year
- 2026: 5 · 2025: 5 · 2024: 5 · 2023: 2 · 2022: 6 · 2021: 7 · 2020: 9
- FWCI
- 2.54
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Research Grants Council, University Grants Committee (CityU11215618)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Keywords
- task-incremental learning, lifelong learning, continual learning, assistive robotics, robot vision, learning without forgetting, elastic weight consolidation, illumination variation, occlusion, clutter, camera angles
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
