CoChat
Sign Up
Cited by 51Autonomous Vehicle Technology and SafetyReinforcement Learning in RoboticsTraffic control and management

Modified DDPG car-following model with a real-world human driving experience with CARLA simulator

Open access
Li, Dianzhao, Okhrin, Ostap·2023·Transportation Research Part C: Emerging Technologies
DOI 10.1016/j.trc.2022.103987
Open this paper in CoChat — free account

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.trc.2022.103987

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Most RL-based driving studies have focused on safety constraints or reward shaping, without explicitly operationalizing ethical reasoning across multiple road users.

    “Still, most RL-based driving studies have focused on safety constraints or reward shaping, without explicitly operationalizing ethical reasoning across multiple road users.”

    From Ethics-Aware Safe Reinforcement Learning for Rare-Event Risk Control in Interactive Urban Driving (2026) · cites this paper as [13]

    The Introduction (p. 3) describes RL driving studies concerned with collision avoidance, efficiency, and car-following, and the Reward function section (p. 12) emphasizes safety, comfort, and efficiency. These passages support the observation that such objectives feature in RL driving work, but the paper does not establish that *most* studies focus on them or assess whether studies explicitly operationalize ethical reasoning across multiple road users. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 37

  1. Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L.D., Monfort, M., Muller, U., Zhang, J., et al., 2016. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316 . · arXiv
  2. Codevilla, F., Santana, E., L´ opez, A.M., Gaidon, A., 2019. Exploring the limitations of behavior cloning for autonomous driving, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 93299338.
  3. Dadashi, R., Rezaeifar, S., Vieillard, N., Hussenot, L., Pietquin, O., Geist, M., 2021. Offline reinforcement learning with pseudometric learning, in: International Conference on Machine Learning, PMLR. pp. 2307-2318.
  4. Dupuis, M., Strobl, M., Grezlikowski, H., 2010. Opendrive 2010 and beyondstatus and future of the de facto standard for the description of road networks, in: Proc. of the Driving Simulation Conference Europe, pp. 231242.
  5. Dworak, D., Ciepiela, F., Derbisz, J., Izzat, I., Komorkiewicz, M., W´ ojcik, M., 2019. Performance of lidar object detection deep learning architectures based on artificially generated point cloud data from carla simulator, in: 2019 24th International Conference on Methods and Models in Automation and Robotics (MMAR), IEEE. pp. 600-605.
  6. Fujimoto, S., Meger, D., Precup, D., 2019. Off-policy deep reinforcement learning without exploration, in: International Conference on Machine Learning, PMLR. pp. 2052-2062.
  7. G´ omez-Hu´ elamo, C., Del Egido, J., Bergasa, L.M., Barea, R., L´ opez-Guill´ en, E., Arango, F., Araluce, J., L´ opez, J., 2020. Train here, drive there: Simulating real-world use cases with fully-autonomous driving architecture in carla simulator, in: Workshop of Physical Agents, Springer. pp. 44-59.
  8. Gu, S., Holly, E., Lillicrap, T., Levine, S., 2017. Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates, in: 2017 IEEE international conference on robotics and automation (ICRA), IEEE. pp. 3389-3396.
  9. de Haan, P., Jayaraman, D., Levine, S., 2019. Causal confusion in imitation learning. Advances in Neural Information Processing Systems 32, 1169811709.
  10. Hanin, B., Rolnick, D., 2018. How to start training: The effect of initialization and architecture. arXiv preprint arXiv:1803.01719 . · arXiv
  11. Hart, F., Okhrin, O., Treiber, M., 2021. Formulation and validation of a car-following model based on deep reinforcement learning. arXiv preprint arXiv:2109.14268 . · arXiv
  12. He, Y., Zhao, N., Yin, H., 2017. Integrated networking, caching, and computing for connected vehicles: A deep reinforcement learning approach. IEEE Transactions on Vehicular Technology 67, 44-55.
  13. Hellmund, A.M., Wirges, S., Ta¸ s, ¨ O.S ¸., Bandera, C., Salscheider, N.O., 2016. Robot operating system: A modular software framework for automated driving, in: 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), IEEE. pp. 1564-1570.
  14. Hester, T., Vecerik, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., Osband, I., et al., 2018. Deep qlearning from demonstrations, in: Proceedings of the AAAI Conference on Artificial Intelligence, p. 3223-3230.
  15. Huang, Z., Wu, J., Lv, C., 2021. Efficient deep reinforcement learning with imitative expert priors for autonomous driving. arXiv preprint arXiv:2103.10690 . · arXiv
  16. Isele, D., Rahimi, R., Cosgun, A., Subramanian, K., Fujimura, K., 2018. Navigating occluded intersections with autonomous vehicles using deep reinforcement learning, in: 2018 IEEE International Conference on Robotics and Automation (ICRA), IEEE. pp. 2034-2039.
  17. Kalman, R.E., 1960. A new approach to linear filtering and prediction problems. Transaction of the ASME - Journal of Basic Engineering , 35-45.
  18. Kim, D., 1999. Normalization methods for input and output vectors in backpropagation neural networks. International journal of computer mathematics 71, 161-171.
  19. Levine, S., Kumar, A., Tucker, G., Fu, J., 2020. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643 . · arXiv
  20. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D., 2015. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 . · arXiv
  21. Liu, H., Huang, Z., Lv, C., 2021. Improved deep reinforcement learning with expert demonstrations for urban autonomous driving. arXiv preprint arXiv:2102.09243 . · arXiv
  22. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M., 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 . · arXiv
  23. Ngai, D.C.K., Yung, N.H.C., 2011. A multiple-goal reinforcement learning method for complex vehicle overtaking maneuvers. IEEE Transactions on Intelligent Transportation Systems 12, 509-522.
  24. Niranjan, D., VinayKarthik, B., et al., 2021. Deep learning based object detection model for autonomous driving research using carla simulator, in: 2021 2nd International Conference on Smart Electronics and Communication (ICOSEC), IEEE. pp. 1251-1258.
  25. Nosrati, M.S., Abolfathi, E.A., Elmahgiubi, M., Yadmellat, P., Luo, J., Zhang, Y., Yao, H., Zhang, H., Jamil, A., 2018. Towards practical hierarchical reinforcement learning for multi-lane autonomous driving, in: 2018 NIPS MLITS Workshop.
  26. Ornstein, L.S., Uhlenbeck, G.E., 1930. On the theory of the brownian motion. Physical review 36, 823.
  27. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y., et al., 2009. Ros: an open-source robot operating system, in: ICRA workshop on open source software, Kobe, Japan. p. 5.
  28. Ravi Kiran, B., Roldao, L., Irastorza, B., Verastegui, R., Suss, S., Yogamani, S., Talpaert, V., Lepoutre, A., Trehard, G., 2018. Real-time dynamic object detection for autonomous driving using prior 3d-maps, in: Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pp. 0-0.
  29. Sallab, A.E., Abdou, M., Perot, E., Yogamani, S., 2016. End-to-end deep reinforcement learning for lane keeping assist, in: Proceedings of MLITS, NIPS Workshop, pp. vol. 2, pp. 1-9.
  30. Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al., 2017. Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint arXiv:1712.01815 . · arXiv
  31. Thrun, S., Montemerlo, M., Dahlkamp, H., Stavens, D., Aron, A., Diebel, J., Fong, P., Gale, J., Halpenny, M., Hoffmann, G., et al., 2006. Stanley: The robot that won the darpa grand challenge. Journal of field Robotics 23, 661-692.
  32. Tran, L.A., Le, M.H., 2019. Robust u-net-based road lane markings detection for autonomous driving, in: 2019 International Conference on System Science and Engineering (ICSSE), IEEE. pp. 62-66.
  33. Treiber, M., Hennecke, A., Helbing, D., 2000. Congested traffic states in empirical observations and microscopic simulations. Physical review E 62, 1805.
  34. Treiber, M., Kesting, A., 2013. Traffic flow dynamics. Traffic Flow Dynamics: Data, Models and Simulation, Springer-Verlag Berlin Heidelberg , 9831000.
  35. Vecerik, M., Hester, T., Scholz, J., Wang, F., Pietquin, O., Piot, B., Heess, N., Roth¨ orl, T., Lampe, T., Riedmiller, M., 2017. Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards. arXiv preprint arXiv:1707.08817 . · arXiv
  36. Wang, P., Chan, C.Y., de La Fortelle, A., 2018. A reinforcement learning based approach for automated lane change maneuvers, in: 2018 IEEE Intelligent Vehicles Symposium (IV), IEEE. pp. 1379-1384.
  37. Wiedemann, R., 1974. Simulation des Straßenverkehrsflusses. Institut f¨ ur Verkehrswesen, University of Karlsruhe, Germany, Vol. 8 of Schriftenreihe des IfV.

Cited by

Sorted by citations

Authors
Li, D., & Okhrin, O.
Affiliations
Technische Universität Dresden (DE)
Corresponding author
Li, Dianzhao
ORCID
Venue
Transportation Research Part C: Emerging Technologies
Publisher
Elsevier BV
Published
2023-02
Volume
147
Pages
103987
ISSN
0968-090X (ISSN-L)
DOI
10.1016/j.trc.2022.103987
OpenAlex
W4312059601
Cited by
51
References
2
Citations by year
2026: 7 · 2025: 13 · 2024: 20 · 2023: 10 · 2022: 1
FWCI
2.93
Type
journal article
Language
en
Open access
Licence
Locations
  • Transportation Research Part C Emerging Technologies (publishedVersion)
  • SSRN Electronic Journal (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Engineering › Control and Systems Engineering › Traffic control and management
Keywords
vehicle following, deep deterministic policy gradient, autonomous driving, human-robot interaction, CARLA simulator, Robot Operating System, driving scenarios

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review