The DOI matches its Crossref record; 33 of the 36 references it cites were found in a registry or an index.
Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems
Open accessAbstract
This letter was motivated by the problem of how to make robots fuse and transfer their experience so that they can effectively use prior knowledge and quickly adapt to new environments. To address the problem, we present a learning architecture for navigation in cloud robotic systems: Lifelong Federated Reinforcement Learning (LFRL). In the letter, we propose a knowledge fusion algorithm for upgrading a shared model deployed on the cloud. Then, effective transfer learning methods in LFRL are introduced. LFRL is consistent with human cognitive science and fits well in cloud robotic systems. Experiments show that LFRL greatly improves the efficiency of reinforcement learning for robot navigation. The cloud robotic system deployment also shows that LFRL is capable of fusing prior knowledge. In addition, we release a cloud robotic navigation-learning website to provide the service based on LFRL: www.shared-robotics.com.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Through communication rounds, the server can accumulate a significant knowledge of different robots to build a powerful learning model, and thus the imitation learning efficiency and accuracy of local robots can be improved in comparison with centralized learning approaches.”
From Federated Learning for Internet of Things: A Comprehensive Survey (2021) · cites this paper as [164]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Through communication rounds, the server can accumulate a significant knowledge of different robots to build a powerful learning model, and thus the imitation learning efficiency and accuracy of local robots can be improved in comparison with centralized learning approaches.”
From Federated Learning for Internet of Things: A Comprehensive Survey (2021) · cites this paper as [164]
In the citing paper
“FL can assist to improve the security and privacy of medical industry data analysis and prediction by incorporating a data protection system [25, 161, 162] .”
From Federated learning-based AI approaches in smart healthcare: Concepts, taxonomies, challenges and open issues (2023) · cites this paper as [161]
In the citing paper
“Hospital mortality prediction, duration of stay, or rather the rate of admission can also be examined using FL-based models for the proper management of some critical care units in the hospital while protecting the privacy of in-hospital data [161, 166].”
From Federated learning-based AI approaches in smart healthcare: Concepts, taxonomies, challenges and open issues (2023) · cites this paper as [161]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Target-driven visual navigation in indoor scenes using deep reinforcement learning
Zhu, Y., Mottaghi, R., Kolve, E., Lim, J. J., Gupta, A., Fei-Fei, L., & Farhadi, A. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989381 · Cited by 1,533
Rusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., & Hadsell, R. (2016) · arXiv · doi:10.48550/arxiv.1606.04671 · Cited by 1,307
Federated Learning for Mobile Keyboard Prediction
Hard, A., Rao, K., Mathews, R., Ramaswamy, S., Beaufays, F., Augenstein, S., Eichner, H., Kiddon, C., & Ramage, D. (2018) · arXiv · doi:10.48550/arxiv.1811.03604 · Cited by 1,148
Reinforcement Learning with Unsupervised Auxiliary TasksPreprint
Jaderberg, M., Mnih, V., Czarnecki, W. M., Schaul, T., Leibo, J. Z., Silver, D., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05397 · Cited by 1,117
In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning
Wang, X., Han, Y., Wang, C., Zhao, Q., Chen, X., & Chen, M. (2019) · IEEE Network · doi:10.1109/mnet.2019.1800286 · Cited by 1,030 · cited 1×
“Different from the traditional joint learning method where multiple edges are learning at the same time, LFRLA adopts the method of first training then mergingto reduce the dependence on the quality of communication[25][26].”
MethodFederated Learning of Deep Networks using Model AveragingPreprint
McMahan, H. B., Moore, E., Ramage, D., & y Arcas, B. A. (2016) · arXiv (Cornell University) · Cited by 1,005
Optimal and Autonomous Control Using Reinforcement Learning: A Survey
Kiumarsi, B., Vamvoudakis, K. G., Modares, H., & Lewis, F. L. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2773458 · Cited by 924
End-to-End Learning of Driving Models from Large-Scale Video Datasets
Xu, H., Gao, Y., Yu, F., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.376 · Cited by 884
A Survey of Research on Cloud Robotics and Automation
Kehoe, B., Patil, S., Abbeel, P., & Goldberg, K. (2015) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2014.2376492 · Cited by 862
Learning to Navigate in Complex EnvironmentsPreprint
Mirowski, P., Pascanu, R., Viola, F., Soyer, H., Ballard, A. J., Banino, A., Denil, M., Goroshin, R., Sifre, L., Kavukcuoglu, K., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03673 · Cited by 847
Tai, L., Paolo, G., & Liu, M. (2017) · IEEE/RJS International Conference on Intelligent RObots and Systems · doi:10.1109/iros.2017.8202134 · Cited by 843
Self-citationUniversal Value Function Approximators
Schaul, T., Horgan, D., Gregor, K., & Silver, D. (2015) · Cited by 701
Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning
Long, P., Fanl, T., Liao, X., Liu, W., Zhang, H., & Pan, J. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461113 · Cited by 558
Chen, Z., & Liu, B. (2018) · Synthesis lectures on artificial intelligence and machine learning · doi:10.1007/978-3-031-01581-6 · Cited by 551
Self-citationDistributed Federated Learning for Ultra-Reliable Low-Latency Vehicular Communications
Samarakoon, S., Bennis, M., Saad, W., & Debbah, M. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2956472 · Cited by 459
Deep reinforcement learning with successor features for navigation across similar environments
Zhang, J.-W., Springenberg, J., Boedecker, J., & Burgard, W. (2016) · IEEE/RJS International Conference on Intelligent RObots and Systems · doi:10.1109/iros.2017.8206049 · Cited by 331
A Deep Hierarchical Approach to Lifelong Learning in Minecraft
Tessler, C., Givony, S., Zahavy, T., Mankowitz, D. J., & Mannor, S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10744 · Cited by 300
Rusu, A. A., Colmenarejo, S. G., Gulcehre, C., Desjardins, G., Kirkpatrick, J., Pascanu, R., Mnih, V., Kavukcuoglu, K., & Hadsell, R. (2015) · arXiv · doi:10.48550/arxiv.1511.06295 · Cited by 270
Li, C. (2019) · Chapman & Hall/CRC eBooks · doi:10.1201/9781351006620-6 · Cited by 257
Deep reinforcement learning with smooth policy update: Application to robotic cloth manipulation
Tsurumine, Y., Cui, Y., Uchibe, E., & Matsubara, T. (2018) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2018.11.004 · Cited by 175
Sewak, M. (2019) · Springer eBooks · doi:10.1007/978-981-13-8285-7 · Cited by 175
Kamei, K., Nishio, S., Hagita, N., & Sato, M. (2012) · IEEE Network · doi:10.1109/mnet.2012.6201213 · Cited by 167
Lifelong Machine Learning, Second Edition
Chen, Z., & Liu, B. (2018) · Synthesis lectures on artificial intelligence and machine learning · doi:10.2200/s00832ed1v01y201802aim037 · Cited by 160
Self-citationOnline Multi-Task Learning for Policy Gradient Methods
Ammar, H. B., Eaton, E., Ruvolo, P., & Taylor, M. E. (2014) · Cited by 153
Neural SLAM: Learning to Explore with External MemoryPreprint
Zhang, J., Tai, L., Liu, M., Boedecker, J., & Burgard, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.09520 · Cited by 151
Self-citationDeep Reinforcement LearningPreprint
Li, Y. (2018) · Reinforcement Learning for Cyber-Physical Systems · doi:10.48550/arxiv.1810.06339 · Cited by 130
RoboEarth - A World Wide Web for Robots
Waibel, M., Beetz, M., Raffaello D’andrea, B., Janssen, R., Tenorth, M., Civera, J., Elfring, J., Gálvez‐López, D., Häußermann, K., Montiel, J. M. M., et al. (2011) · mediaTUM (Technical University of Munich) · doi:10.1109/mra.2011.941632 · Cited by 118
Nasr, M., Shokri, R., & Houmansadr, A. (2018) · arXiv (Cornell University) · Cited by 104
PAC-inspired Option Discovery in Lifelong Reinforcement Learning
Brunskill, E., & Li, L. (2014) · Cited by 79
Real-Time Multisensor Data Retrieval for Cloud Robotic Systems
Wang, L., Liu, M., & Meng, M. Q. ‐. (2015) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2015.2408634 · Cited by 64
Self-citationRavi, N., Mala, T., Srinivasan, M. K., & Sarukesi, K. (2013) · IEEE International Conference on Mobile Data Management (MDM) · doi:10.1109/mdm.2013.89 · Cited by 4
Flagged first, then by citations
3 of the 36 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- kuffner, 0, Cloud-enabled robots, Proc Int Conf Humanoid Robot
Cited by
Sorted by citations
Federated Learning for Internet of Things: A Comprehensive Survey
Nguyen, D. C., Ding, M., Pathirana, P. N., Seneviratne, A., Li, J., & Vincent Poor, H. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3075439 · Cited by 1,503
“The authors in [163], [164] investigate a federated imitation learning scheme for cloud robotics.”
BackgroundA survey on security and privacy of federated learning
Mothukuri, V., Parizi, R. M., Pouriyeh, S., Huang, Y., Dehghantanha, A., & Srivastava, G. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2020.10.007 · Cited by 1,480
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., & He, B. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124599 · Cited by 1,407
Federated learning on non-IID data: A survey
Zhu, H., Xu, J., Liu, S., & Jin, Y. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.098 · Cited by 1,062
Federated Learning for Internet of Things: A Comprehensive Survey
Nguyen, D., Ding, M., Pathirana, P. N., Seneviratne, A., Li, J., & Poor, H. (2021) · Deakin Research Online (Deakin University) · Cited by 889
Metadata
- Authors
- Liu, B., Wang, L., & Liu, M.
- Affiliations
- Shenzhen Institutes of Advanced Technology (CN)
- Hong Kong University of Science and Technology (HK)
- Venue
- IEEE Robotics and Automation Letters
- Published
- 2019-10
- Volume
- 4
- Issue
- 4
- Pages
- 4555-4562
- ISSN
- 2377-3766 (ISSN-L)
- 2377-3774
- OpenAlex
- W2911450448
- Cited by
- 242
- References
- 36
- Citations by year
- 2026: 5 · 2025: 29 · 2024: 38 · 2023: 40 · 2022: 33 · 2021: 50 · 2020: 27 · 2019: 20
- FWCI
- 15.00
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Robotics and Automation Letters (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Shenzhen Science and Technology Innovation Commission (JCYJ2017081853518789)
- Guangdong Science and Technology Plan Guangdong-Hong Kong Cooperation Innovation Platform (2018B050502009)
- National Natural Science Foundation of China (61603376, U1713211)
- Shenzhen Science, Technology and Innovation Commission (JCYJ20160428154842603)
- Basic Research Project of Shanghai Science and Technology Commission (16JC1401200)
- Chengzhong Xu from University of Macau
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Engineering › Control and Systems Engineering › Robotics and Automated Systems
- Keywords
- lifelong learning, federated reinforcement learning, robot navigation, cloud robotics, knowledge fusion, transfer learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
