Designing deep reinforcement learning systems for musculoskeletal modeling and locomotion analysis using wearable sensor feedback
Abstract
In this paper, a system for lower-limb amputees to collect and analyze their locomotion activities is developed. Wearable Gait Lab (WGL) system is a pair of smart insoles that can collect plantar pressure data and foot motion data of subjects. A reinforcement learning model is trained to imitate the walking pattern of the lower-limb amputee on a musculoskeletal model by introducing realistic velocity data into the training process. The plantar pressure data collected from our Wearable Gait Lab were used to recognize the locomotion modes of the amputees. Experiments showed that the outcome of the musculoskeletal model can be a reference for analyzing muscle activities of the amputee. The system also shows the promising and stable performance of recognizing locomotion modes in amputees' daily life. The accuracy of locomotion mode recognition reaches 98.02%. Monitoring muscle activities and locomotion modes of lower-limb amputees can help them prevent secondary impairments in rehabilitation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
[231] developed a reinforcement learning model for imitating the walking pattern of a lower-limb amputee on a musculoskeletal model.
“Ref. [231] developed a reinforcement learning model for imitating the walking pattern of a lower-limb amputee on a musculoskeletal model.”
From Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances (2022) · cites this paper as [231]
The system in [231] showed 98.02% locomotion mode recognition accuracy.
“The system showed 98.02% locomotion mode recognition accuracy.”
From Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances (2022) · cites this paper as [231]
Reinforcement learning is a relatively new area being explored for select areas in HAR, such as modelling muscle and arm movements.
“Reinforcement learning is a relatively new area that is being explored for select areas in HAR, such as modelling muscle and arm movements [230,231].”
From Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances (2022) · cites this paper as [231]
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
The Timed “Up & Go”: A Test of Basic Functional Mobility for Frail Elderly Persons
Podsiadlo, D., & Richardson, S. (1991) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.1991.tb01616.x · Cited by 14,418 · cited 1× · see citing works
“Two methods for mobility measures have been proved as valid and are widely used in amputee rehabilitation, including the ‘Timed Up and Go’ [3] and ‘Two-Minute Walk’ [4] test.”
MethodContinuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · see citing works
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · see citing works
A Brief Survey of Deep Reinforcement Learning
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · arXiv · doi:10.48550/arxiv.1708.05866 · Cited by 4,733 · see citing works
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733 · see citing works
End-to-End Training of Deep Visuomotor PoliciesPreprint
Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2015) · Journal of machine learning research · doi:10.48550/arxiv.1504.00702 · Cited by 4,011 · see citing works
Deterministic Policy Gradient Algorithms
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014) · HAL · Cited by 3,275 · see citing works
End-to-end training of deep visuomotor policies
Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2016) · Journal of Machine Learning Research · Cited by 2,192 · see citing works
Learning to select and generalize striking movements in robot table tennis
Muelling, K., Kober, J., Kroemer, O., & Peters, J. (2012) · AAAI Fall Symposium: Robots Learning Interactively from Human Teachers · doi:10.1177/0278364912472380 · Cited by 486 · see citing works
Berger, J. O., & Lehmann, E. L. (1984) · Journal of the American Statistical Association · doi:10.2307/2288729 · Cited by 185 · see citing works
Morgenroth, D. C., Segal, A. D., Zelik, K. E., Czerniecki, J. M., Klute, G. K., Adamczyk, P. G., Orendurff, M. S., Hahn, M. E., Collins, S. H., & Kuo, A. D. (2011) · Gait & Posture · doi:10.1016/j.gaitpost.2011.07.001 · Cited by 180 · see citing works
Reliability of the two-minute walk test in individuals with transtibial amputation
Brooks, D., Hunter, J. P., Parsons, J. A., Livsey, E., Quirt, J., & Devlin, M. (2002) · Archives of Physical Medicine and Rehabilitation · doi:10.1053/apmr.2002.34600 · Cited by 157 · see citing works
Liu, M., Wang, D., & Huang, H. (2015) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2015.2420539 · Cited by 145 · see citing works
Bring Gait Lab to Everyday Life: Gait Analysis in Terms of Activities of Daily Living
Chen, D., Cai, Y. M., Qian, X., Ansari, R., Xu, W., Chu, K., & Huang, M.-C. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2954387 · Cited by 85 · see citing works
Self-citationCustomizable Pressure Sensor Array: Design and Evaluation
Chen, D., Cai, Y. M., & Huang, M.-C. (2018) · IEEE Sensors Journal · doi:10.1109/jsen.2018.2832129 · Cited by 71 · see citing works
Self-citationPolicy Transfer using Reward Shaping
Brys, T., Harutyunyan, A., Taylor, M. E., & Nowé, A. (2015) · doi:10.5555/2772879.2772905 · Cited by 62 · see citing works
From Virtual Demonstration to Real-World Manipulation Using LSTM and MDN
Rahmatizadeh, R., Abolghasemi, P., Behal, A., & Bölöni, L. (2016) · AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12099 · Cited by 59 · see citing works
Smart Insole-Based Indoor Localization System for Internet of Things Applications
Chen, D., Cao, H., Chen, H., Zhu, Z., Qian, X., Xu, W., & Huang, M.-C. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2915791 · Cited by 51 · see citing works
Self-citationArtificial Intelligence for Prosthetics: Challenge Solutions
Kidziński, Ł., Ong, C. F., Mohanty, S. P., Hicks, J. L., Carroll, S. F., Zhou, B., Zeng, H., Wang, F., Lian, R., Tian, H., et al. (2019) · The Springer series on challenges in machine learning · doi:10.1007/978-3-030-29135-8_4 · Cited by 42 · see citing works
Real-time gait event detection for lower limb amputees using a single wearable sensor
Maqbool, H. F., Husman, M. A., Awad, M. I., Abouhossein, A., Mehryar, P., Iqbal, N., & Dehghani-Sanij, A. A. (2016) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2016.7591866 · Cited by 41 · see citing works
Xuegang, W., Wang, Q., Zheng, E., Wei, K., & Wang, L. (2012) · Advances in intelligent systems and computing · doi:10.1007/978-3-642-33932-5_26 · Cited by 39 · see citing works
Level walking and stair climbing gait in above-knee amputees
Bae, T. S., Choi, K., & Mun, M. (2009) · Journal of Medical Engineering & Technology · doi:10.1080/03091900701404043 · Cited by 38 · see citing works
Díaz-Paz, J. P., da Silva, R. L., Zhong, B., Huang, H., & Lobaton, E. J. (2018) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2018.8512614 · Cited by 30 · see citing works
Learning a Super Mario controller from examples of human play
Lee, G., Luo, M., Zambetta, F., & Li, X. (2014) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2014.6900246 · Cited by 22 · see citing works
Learning Manipulation Trajectories Using Recurrent Neural NetworksPreprint
Rahmatizadeh, R., Abolghasemi, P., & Bölöni, L. (2016) · arXiv.org · doi:10.48550/arxiv.1603.03833 · Cited by 14 · see citing works
Zheng, E., Wang, L., Luo, Y., Wei, K., & Wang, Q. (2013) · International Conference on Rehabilitation Robotics · doi:10.1109/icorr.2013.6650410 · Cited by 13 · see citing works
The Prevalence of Osteoarthritis of the Intact Hip and Knee Among Traumatic Leg Amputees
Swiontkowski, M. F. (2009) · Yearbook of Orthopedics · doi:10.1016/s0276-1092(09)79532-x · Cited by 9 · see citing works
Mathur, N., Glesk, I., Davidson, A., Paul, G., Banford, J., Irvine, J. M., & Buis, A. W. P. (2016) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2016.7527503 · Cited by 6 · see citing works
Wearable Gait Lab System providing quantitative statistical support for human balance tests
Cui, J., Chen, J., Qu, G., Starkman, J., Zeng, X., Madigan, E. A., Pekarek, M., Xu, W., & Huang, M.-C. (2017) · Smart Health · doi:10.1016/j.smhl.2017.05.001 · Cited by 6 · see citing works
Self-citationSwiontkowski, M. F. (2006) · Yearbook of Orthopedics · doi:10.1016/s0276-1092(08)70408-5 · Cited by 5 · see citing works
Explore Correlation Between Body Balance and Perception using mHealth Technology
Ali, R. Z., Zheng, J., Chen, J., & Huang, M.-C. (2019) · Wearable and Implantable Body Sensor Networks · doi:10.1109/bsn.2019.8771083 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
2 of the 34 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances
Zhang, S., Li, Y., Zhang, S., Shahabi, F., Xia, S., Deng, Y., & Alshurafa, N. (2022) · Sensors · doi:10.3390/s22041476 · Cited by 625 · see citing works
“[338] developed a reinforcement learning model for imitating the walking pattern of a lower-limb amputee on a musculoskeletal model.”
BackgroundApplication of artificial intelligence in wearable devices: Opportunities and challenges
Nahavandi, D., Alizadehsani, R., Khosravi, A., & Acharya, U. (2021) · Comput. Methods Programs Biomed. · doi:10.1016/j.cmpb.2021.106541 · Cited by 305 · see citing works
Intelligent Wearable Systems: Opportunities and Challenges in Health and Sports
Yang, L., Amin, O., & Shihada, B. (2024) · ACM Computing Surveys · doi:10.1145/3648469 · Cited by 145 · see citing works
Energy-aware human activity recognition for wearable devices: A comprehensive review
Contoli, C., Freschi, V., & Lattanzi, E. (2024) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2024.101976 · Cited by 43 · see citing works
Yang, J., & Yin, Y. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3019053 · Cited by 30 · see citing works
A systematic review of artificial neural network techniques for analysis of foot plantar pressure
Wang, C., Evans, K. A., Hartley, D., Morrison, S., Veidt, M., & Wang, G. (2024) · Journal of Applied Biomedicine · doi:10.1016/j.bbe.2024.01.005 · Cited by 29 · see citing works
A Review on Locomotion Mode Recognition and Prediction When Using Active Orthoses and Exoskeletons
Moreira, L., Figueiredo, J., Cerqueira, J., & Santos, C. P. (2022) · Sensors · doi:10.3390/s22197109 · Cited by 16 · see citing works
Machine Learning for Musculoskeletal Modeling of Upper Extremity
Sharma, R., Dasgupta, A., Cheng, R., Mishra, C., & Nagaraja, V. H. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3197461 · Cited by 13 · see citing works
Wearable-Gait-Analysis-Based Activity Recognition: A Review
Ansah, S., & Chen, D. (2022) · International Journal on Smart Sensing and Intelligent Systems · doi:10.2478/ijssis-2022-0021 · Cited by 12 · see citing works
Self-citationReinforcement Learning-Based Grasp Pattern Control of Upper Limb Prosthetics in an AI Platform
Nia, N. G., Nasab, A., & Kaplanoğlu, E. (2022) · International Informatics and Software Engineering Conference (IISEC) · doi:10.1109/iisec56263.2022.9998307 · Cited by 5 · see citing works
Integration of Machine Learning with Wearable Technologies
Nahavandi, D., Alizadehsani, R., & Khosravi, A. (2023) · Wiley eBooks · doi:10.1002/9781119863663.ch31 · Cited by 3 · see citing works
Application of machine learning methods in flexible and wearable sensing systems
Peng, P., Li, Y., Zhang, R., Li, W., & Wang, X. (2026) · Wearable electronics. · doi:10.1016/j.wees.2025.11.002 · Cited by 2 · see citing works
Real-Time Detection of Lower Limb Training Stability Function Based on Smart Wearable Sensors
Zhang, Y. (2022) · Journal of Sensors · doi:10.1155/2022/7503668 · Cited by 2 · see citing works
RetractedA Review on Neural Network based Gait Estimation Methods
Lee, J., Sivakumar, S., & Ning, L. W. (2022) · International Conference on Green Energy, Computing and Sustainable Technology (GECOST) · doi:10.1109/gecost55694.2022.10010660 · Cited by 2 · see citing works
Providing Informative Feedback in a Low-Cost Rehabilitation System Using Machine Learning
Rodrigues, P., Amorim, I., & Cunha, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-77738-7_8
AI in Wearable Imaging Devices
Adebayo, P. O., Aremu, I. B., & Babatunde, A. J. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-0903-3.ch006
Intelligent Prosthetics Motion Learning and Neuro-Muscular Feedback Integration
Pavithra, M. R., Chitra, V., Subhashini, V., Hemalatha, D., & Kiran, S. R. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8241-8.ch004
- Authors
- Zheng, J., Cao, H., Chen, D., Ansari, R., Chu, K.-C., & Huang, M.-C.
- Affiliations
- Case Western Reserve University (US)
- National Taipei University of Nursing and Health Science (TW)
- Venue
- IEEE Sensors Journal
- Published
- 2020-08-15
- Volume
- 20
- Issue
- 16
- Pages
- 9274-9282
- ISSN
- 1530-437X (ISSN-L)
- 1558-1748
- 2379-9153
- OpenAlex
- W3016045827
- Cited by
- 17
- References
- 34
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 4 · 2023: 1 · 2022: 7 · 2021: 1 · 2020: 1
- FWCI
- 0.79
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Sensors Journal (publishedVersion)
- Funders
- National Science Foundation (NSF-1664368)
- The Technology Validation and Start Fund (TVSF) from the Ohio Third Frontier
- Topics
- Physical Sciences › Engineering › Biomedical Engineering › Prosthetics and Rehabilitation Robotics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Engineering › Biomedical Engineering › Muscle activation and electromyography studies
- Keywords
- lower limb amputees, plantar pressure, gait analysis, locomotion mode recognition, musculoskeletal modeling, deep reinforcement learning, muscle activity analysis, gait rehabilitation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
