LIBERO-RECOVER: Beyond Task Success Towards Failure Recovery in Robotic Manipulation Models
The DOI matches its DataCite record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Vision-Language-Action (VLA) or World Action (WAM) models have recently demonstrated remarkable performance in robotic manipulation. On LIBERO, SOTA method have achieved nearly 100\% success rates, seemingly suggesting that the models are ready for deployment in real world. However, near perfect performance on existing benchmarks can be misleading: success under ideal conditions does not imply real world robustness. Existing benchmarks primarily evaluate task completion from predefined initial states, while real world interactions inevitably involve failures such as failed grasps, collisions, and unintended object movements. A robot must therefore not only execute tasks successfully, but also recognize and recover from failures to continue the task. Yet this capability remains largely unmeasured, revealing a critical gap between benchmark performance and real world reliability. To address this gap, we introduce LIBERO-Recover Benchmark, a large scale benchmark for failure recovery in robotic manipulation. Built upon LIBERO, we collect real execution failures from SOTA embodied models and construct 1,000+ scenarios across four recovery levels: (1) Action Retry, (2) Action Adaptation, (3) Object State Recovery, and (4) Environmental Recovery. We evaluate four core capabilities: spatial understanding, object structure reasoning, interaction understanding, and topological reasoning. As the first large-scale benchmark for embodied failure recovery, LIBERO-Recover shifts evaluation from \emph{Can the robot succeed?''} to \emph{Can the robot recover after failure?''}, promoting robust and generalizable embodied agents. The project will be avaible in \textcolor{blue}{https://liulin815.github.io/LIBERO-Recovery/}.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
RLBench: The Robot Learning Benchmark & Learning Environment
James, S., Ma, Z., Arrojo, D. R., & Davison, A. J. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2974707 · Cited by 405
Mees, O., Hermann, L., Rosete-Beas, E., & Burgard, W. B. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3180108 · Cited by 189
LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
Liu, B., Zhu, Y., Gao, C., Feng, Y., Liu, Q., Zhu, Y., & Stone, P. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1939 · Cited by 135
FAST: Efficient Action Tokenization for Vision-Language-Action Models
Pertsch, K., Stachowicz, K., Ichter, B., Driess, D., Nair, S., Vuong, Q., Mees, O., Finn, C., & Levine, S. (2025) · arXiv · doi:10.48550/arxiv.2501.09747 · Cited by 73
Wan: Open and Advanced Large-Scale Video Generative Models
Wan, T., Wang, A., Ai, B., Wen, B., Mao, C., Xie, C.-W., Chen, D., Yu, F., Zhao, H., Yang, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.20314 · Cited by 14
$π_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
Intelligence, P., Black, K., Brown, N., Darpinian, J., Dhabalia, K., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., et al. (2025) · arXiv · doi:10.48550/arxiv.2504.16054 · Cited by 3
LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models
Fei, S., Wang, S., Shi, J., Dai, Z., Cai, J., Qian, P., Ji, L., He, X., Zhang, S., Fei, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2510.13626 · cited 1×
“However, these benchmarks still primarily evaluate performance under predefined perturbations or challenging conditions, leaving failure recognition, state recovery, and post-failure replanning largely unexplored.”
LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization
Zhou, X., Xu, Y., Tie, G., Chen, Y., Zhang, G., Chu, D., Zhou, P., & Sun, L. (2025) · arXiv · doi:10.48550/arxiv.2510.03827 · cited 1×
“(2026) considers variations in task and environmental conditions, and LIBERO-Safety Cui et al. (2026) focuses on safety-critical failures and undesired interactions.”
Cited by
RegenHarness: A Robot Agent Harness with Evidence-Gated Recursive Self-Improvement
Wang, K., Jie, H., Yan, Y., Heng, Z., & Li, Z. (2026) · arXiv · doi:10.48550/arxiv.2609.27612
Metadata
- Authors
- Liu, L., Bao, Z., Zhang, L., Song, Z., Yang, W., Zhuang, Y., Tao, S., Liu, W., Jia, C., & Lu, H.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- arXiv
- 2609.05178
- References
- 8
- Type
- Preprint
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.