The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Stochastic policy gradient reinforcement learning on a simple 3D biped
Abstract
We present a learning system which is able to quickly and reliably acquire a robust feedback control policy for 3D dynamic walking from a blank-slate using only trials implemented on our physical robot. The robot begins walking within a minute and learning converges in approximately 20 minutes. This success can be attributed to the mechanics of our robot, which are modeled after a passive dynamic walker, and to a dramatic reduction in the dimensionality of the learning problem. We reduce the dimensionality by designing a robot with only 6 internal degrees of freedom and 4 actuators, by decomposing the control system in the frontal and sagittal planes, and by formulating the learning problem on the discrete return map dynamics. We apply a stochastic policy gradient algorithm to this reduced problem and decrease the variance of the update using a state-based estimate of the expected cost. This optimized learning system works quickly enough that the robot is able to continually adapt to the terrain as it walks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1999) · Cited by 5,147
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
McGeer, T. (1990) · The International Journal of Robotics Research · doi:10.1177/027836499000900206 · Cited by 3,359
A Three-Dimensional Passive-Dynamic Walking Robot with Two Legs and Knees
Collins, S. H., Wisse, M., & Ruina, A. (2001) · The International Journal of Robotics Research · doi:10.1177/02783640122067561 · Cited by 789
Policy gradient reinforcement learning for fast quadrupedal locomotion
Kohl, N., & Stone, P. (2004) · IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004 · doi:10.1109/robot.2004.1307456 · Cited by 564
Infinite-Horizon Policy-Gradient Estimation
Baxter, J., & Bartlett, P. L. (2001) · Cited by 525
Infinite-Horizon Policy-Gradient Estimation
Baxter, J., & Bartlett, P. L. (2001) · Journal of Artificial Intelligence Research · doi:10.1613/jair.806 · Cited by 343
Biped dynamic walking using reinforcement learning
Benbrahim, H., & Franklin, J. A. (1997) · Robotics and Autonomous Systems · doi:10.1016/s0921-8890(97)00043-2 · Cited by 198
Real-time neural network control of a biped walking robot
Miller, W. T. (1994) · IEEE Control Systems · doi:10.1109/37.257893 · Cited by 165
An Uncontrolled Walking Toy That Cannot Stand Still
Coleman, M. J., & Ruina, A. L. (1998) · Physical Review Letters · doi:10.1103/physrevlett.80.3658 · Cited by 158
Cited by
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160
High-Dimensional Continuous Control Using Generalized Advantage Estimation
Schulman, J., Moritz, P., Levine, S., Jordan, M., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1506.02438 · Cited by 2,103
Learning agile and dynamic motor skills for legged robots
Hwangbo, J., Lee, J., Dosovitskiy, A., Bellicoso, D., Tsounis, V., Koltun, V., & Hutter, M. (2019) · Science Robotics · doi:10.1126/scirobotics.aau5872 · Cited by 1,482
Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
Tan, J., Zhang, T., Coumans, E., Iscen, A., Bai, Y., Hafner, D., Bohez, S., & Vanhoucke, V. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.010 · Cited by 735
Self-citation
Metadata
- Authors
- Tedrake, R., Zhang, T. W., & Seung, H. S.
- Venue
- 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566)
- Publisher
- IEEE
- Published
- 2005-04-01
- Volume
- 3
- Pages
- 2849-2854
- OpenAlex
- W2109008048
- Cited by
- 264
- References
- 13
- FWCI
- 11.92
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Stochastic policy gradient reinforcement learning on a simple 3D biped. https://app.cochat.ai/papers/10.1109/iros.2004.1389841
- The paper
- Tedrake, R., Zhang, T. W., & Seung, H. S. (2005). Stochastic policy gradient reinforcement learning on a simple 3D biped. 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566). https://doi.org/10.1109/iros.2004.1389841
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
