The DOI matches its OpenAlex record; 53 of the 53 references it cites were found in a registry or an index.
Conservative Q-Learning for Offline Reinforcement Learning
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
A contextual-bandit approach to personalized news article recommendation
Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010) · Proceedings of the 19th international conference on World wide web · doi:10.1145/1772690.1772758 · Cited by 2,562
The Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3912 · Cited by 2,161
Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Jacot, A., Gabriel, F., & Hongler, C. (2018) · arXiv · doi:10.48550/arxiv.1806.07572 · Cited by 1,562
Tree-Based Batch Mode Reinforcement Learning
Ernst, D., Geurts, P., & Wehenkel, L. A. (2005) · doi:10.5555/1046920.1088690 · Cited by 854
Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations
Rajeswaran, A., Kumar, V., Gupta, A., Vezzani, G., Schulman, J., Todorov, E., & Levine, S. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.049 · Cited by 847
Self-citationOffline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
Levine, S., Kumar, A., Tucker, G., & Fu, J. (2020) · arXiv · doi:10.48550/arxiv.2005.01643 · Cited by 824
Self-citationNear-optimal Regret Bounds for Reinforcement Learning
Jaksch, T., Ortner, R., & Auer, P. L. (2010) · Cited by 784
QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
Kalashnikov, D., Irpan, A., Pastor, P., Ibarz, J., Herzog, A., Jang, E., Quillen, D., Holly, E., Kalakrishnan, M., Vanhoucke, V., et al. (2018) · arXiv · doi:10.48550/arxiv.1806.10293 · Cited by 697
Self-citationApproximately Optimal Approximate Reinforcement Learning
Kakade, S. M., & Langford, J. C. (2002) · Cited by 617
Iyengar, G. N. (2005) · Mathematics of Operations Research · doi:10.1287/moor.1040.0129 · Cited by 566
Exploration by Random Network Distillation
Burda, Y., Edwards, H., Storkey, A., & Klimov, O. (2018) · arXiv · doi:10.48550/arxiv.1810.12894 · Cited by 527
Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
Vecerik, M., Hester, T., Scholz, J., Wang, F., Pietquin, O., Piot, B., Heess, N., Rothörl, T., Lampe, T., & Riedmiller, M. (2017) · arXiv · doi:10.48550/arxiv.1707.08817 · Cited by 478
Eligibility Traces for Off-Policy Policy Evaluation
Precup, D., Sutton, R. S., & Singh, S. P. (2000) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 439
Addressing Function Approximation Error in Actor-Critic Methods
Fujimoto, S., van Hoof, H., & Meger, D. (2018) · UvA-DARE (University of Amsterdam) · Cited by 407
Reinforcement Learning with Deep Energy-Based Policies
Haarnoja, T., Tang, H., Abbeel, P., & Levine, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08165 · Cited by 392
Self-citation(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/1532443041827907 · Cited by 388
Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction
Kumar, A., Fu, J., Tucker, G., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.00949 · Cited by 344
Self-citationD4RL: Datasets for Deep Data-Driven Reinforcement Learning
Fu, J., Kumar, A., Nachum, O., Tucker, G., & Levine, S. (2020) · arXiv · doi:10.48550/arxiv.2004.07219 · Cited by 310
Self-citationDeep Q-learning from Demonstrations
Hester, T., Vecerík, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., Dulac-Arnold, G., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.03732 · Cited by 270
Off-Policy Deep Reinforcement Learning without Exploration
Fujimoto, S. (2024) · TIB Data Manager · doi:10.57702/vkcnqiqb · Cited by 258
Off-Policy Temporal Difference Learning with Function Approximation
Precup, D., Sutton, R. S., & Dasgupta, S. (2001) · Cited by 243
Behavior Regularized Offline Reinforcement Learning
Wu, Y., Tucker, G., & Nachum, O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.11361 · Cited by 231
Self-citationDistributional Reinforcement Learning with Quantile RegressionPreprint
Dabney, W., Rowland, M., Bellemare, M. G., & Munos, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.10044 · Cited by 222
An Optimistic Perspective on Offline Reinforcement LearningPreprint
Agarwal, R., Schuurmans, D., & Norouzi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.04543 · Cited by 222
AWAC: Accelerating Online Reinforcement Learning with Offline Datasets
Nair, A., Gupta, A., Dalal, M., & Levine, S. (2020) · arXiv · doi:10.48550/arxiv.2006.09359 · Cited by 175 · cited 1×
“Most of these methods require a separately estimated model to the behavior policy, πβ(a|s) [15, 30, 59, 27, 54, 55], and are thus limited by their ability to accurately estimate the unknown behavior policy [42], which might be especially complex in settings where the data is collected from multiple sources [34].”
BackgroundSelf-citationBreaking the Curse of Horizon: Infinite-Horizon Off-Policy EstimationPreprint
Liu, Q., Li, L., Tang, Z., & Zhou, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.12429 · Cited by 158
Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning
Peng, X. B., Kumar, A., Zhang, G., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00177 · Cited by 158
Self-citationRelay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning
Gupta, A., Kumar, V., Lynch, C., Levine, S., & Hausman, K. (2019) · arXiv · doi:10.48550/arxiv.1910.11956 · Cited by 153 · cited 1×
“Next, we evaluate CQL on the Franka kitchen domain [18] from D4RL [14].”
MethodSelf-citationDualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution CorrectionsPreprint
Nachum, O., Chow, Y., Dai, B., & Li, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.04733 · Cited by 148
Information-Theoretic Considerations in Batch Reinforcement LearningPreprint
Chen, J., & Jiang, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.00360 · Cited by 138
Error Propagation for Approximate Policy and Value Iteration
Farahmand, A., Munos, R., & Szepesvári, C. (2010) · HAL · Cited by 137
Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog
Jaques, N., Ghandeharioun, A., Shen, J. H., Ferguson, C., Lapedriza, A., Jones, N., Gu, S., & Picard, R. (2019) · arXiv · doi:10.48550/arxiv.1907.00456 · Cited by 126
High Confidence Policy Improvement
Thomas, P. S., Theocharous, G., & Ghavamzadeh, M. (2015) · International Conference on Machine Learning · Cited by 103
Variance-based Regularization with Convex Objectives
Namkoong, H., & Duchi, J. C. (2017) · neural information processing systems · Cited by 100
Towards Characterizing Divergence in Deep Q-LearningPreprint
Achiam, J., Knight, E., & Abbeel, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.08894 · Cited by 99
Safe Policy Improvement by Minimizing Robust Baseline RegretPreprint
Petrik, M., Chow, Y., & Ghavamzadeh, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.03842 · Cited by 91
Robustness in Markov Decision Problems with Uncertain Transition Matrices
Nilim, A., & Ghaoui, L. E. (2003) · Cited by 76
Scaling Up Robust MDPs using Function Approximation
Tamar, A., Mannor, S., & Xu, H. (2014) · International Conference on Machine Learning · Cited by 65
Off-Policy Deep Reinforcement Learning by Bootstrapping the Covariate Shift
Gelada, C., & Bellemare, M. G. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013647 · Cited by 65
Learning from Scarce ExperiencePreprint
Peshkin, L., & Shelton, C. R. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0204043 · Cited by 61
DisCor: Corrective Feedback in Reinforcement Learning via Distribution CorrectionPreprint
Kumar, A., Gupta, A., & Levine, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.07305 · Cited by 55
Self-citationApproximate Policy Iteration Schemes: A ComparisonPreprint
Scherrer, B. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.2878 · Cited by 52
Increasing the Action Gap: New Operators for Reinforcement Learning
Bellemare, M. G., Ostrovski, G., Guez, A., Thomas, P. S., & Munos, R. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10303 · Cited by 43
Consistent On-Line Off-Policy EvaluationPreprint
Hallak, A., & Mannor, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.07121 · Cited by 40
Q* Approximation Schemes for Batch Reinforcement Learning: A Theoretical ComparisonPreprint
Xie, T., & Jiang, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.03924 · Cited by 32
Variational Bayesian Reinforcement Learning with Regret BoundsPreprint
O’Donoghue, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.09647 · Cited by 27
Safe Policy Improvement with Baseline BootstrappingPreprint
Nadjahi, K., Laroche, R., & Combes, R. T. D. (2017) · arXiv (Cornell University) · Cited by 19
Learning Self-Correctable Policies and Value Functions from Demonstrations with Negative SamplingPreprint
Luo, Y., Xu, H., & Ma, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.05634 · Cited by 10
Safe Policy Improvement with an Estimated Baseline Policy
Simão, T. D., Laroche, R., & Combes, R. T. D. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/hlnw2204 · Cited by 9
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Self-citationOffline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
Levine, S., Kumar, A., Tucker, G., & Fu, J. (2020) · arXiv · doi:10.48550/arxiv.2005.01643 · Cited by 824
Self-citationSafe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning
Brunke, L., Greeff, M., Hall, A. W., Yuan, Z., Zhou, S., Panerati, J., & Schoellig, A. P. (2022) · Annual Review of Control, Robotics, and Autonomous Systems · doi:10.1146/annurev-control-042920-020211 · Cited by 769
Reinforcement learning algorithms: A brief survey
Shakya, A. K., Pillai, G., & Chakrabarty, S. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120495 · Cited by 656
Autonomous driving system: A comprehensive survey
Zhao, J., Zhao, W., Deng, B., Wang, Z., Zhang, F., Zheng, W., Cao, W., Nan, J., Lian, Y., & Burke, A. F. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122836 · Cited by 361
Metadata
- Authors
- Kumar, A., Zhou, A., Tucker, G., & Levine, S.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2020-06-08
- Volume
- 33
- Pages
- 1179-1191
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2006.04779
- OpenAlex
- W3033324992
- Cited by
- 543
- References
- 53
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.