Vision-Language Models for Autonomous Driving: CLIP-Based Dynamic Scene Understanding
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Scene understanding is essential for enhancing driver safety, generating human-centric explanations for Automated Vehicle (AV) decisions, and leveraging Artificial Intelligence (AI) for retrospective driving video analysis. This study developed a dynamic scene retrieval system using Contrastive Language–Image Pretraining (CLIP) models, which can be optimized for real-time deployment on edge devices. The proposed system outperforms state-of-the-art in-context learning methods, including the zero-shot capabilities of GPT-4o, particularly in complex scenarios. By conducting frame-level analyses on the Honda Scenes Dataset, which contains a collection of about 80 h of annotated driving videos capturing diverse real-world road and weather conditions, our study highlights the robustness of CLIP models in learning visual concepts from natural language supervision. The results also showed that fine-tuning the CLIP models, such as ViT-L/14 (Vision Transformer) and ViT-B/32, significantly improved scene classification, achieving a top F1-score of 91.1%. These results demonstrate the ability of the system to deliver rapid and precise scene recognition, which can be used to meet the critical requirements of advanced driver assistance systems (ADASs). This study shows the potential of CLIP models to provide scalable and efficient frameworks for dynamic scene understanding and classification. Furthermore, this work lays the groundwork for advanced autonomous vehicle technologies by fostering a deeper understanding of driver behavior, road conditions, and safety-critical scenarios, marking a significant step toward smarter, safer, and more context-aware autonomous driving systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These capabilities are increasingly vital for agents operating in domains such as robotics [88], autonomous vehicles [89,90], and visual content moderation [91,92].”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [89]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Saito, T., & Rehmsmeier, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118432 · Cited by 5,309
Vision Language Models in Autonomous Driving: A Survey and Outlook
Zhou, X., Liu, M., Yurtsever, E., Žagar, B. L., Zimmer, W. M. X., Cao, H., & Knoll, A. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3402136 · Cited by 401
Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoning
Ramanishka, V., Chen, Y.-T., Misu, T., & Saenko, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00803 · Cited by 314
Analysis of the Performance Impact of Fine-Tuned Machine Learning Model for Phishing URL Detection
Samad, S. R. A., Balasubaramanian, S., Kaabi, A. S. R. A., Sharma, B., Chowdhury, S., Mehbodniya, A., Webber, J. L., & Bostani, A. (2023) · Electronics · doi:10.3390/electronics12071642 · Cited by 137
An Improved Deep Network-Based Scene Classification Method for Self-Driving Cars
Ni, J., Shen, K., Chen, Y., Cao, W., & Yang, S. X. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3146923 · Cited by 127
Ping, P., Qin, W., Xu, Y., Miyajima, C., & Takeda, K. (2019) · IEEE Access · doi:10.1109/access.2019.2920489 · Cited by 119
Tami, M. A., Ashqar, H. I., Elhenawy, M. M. Z., Glaser, S., & Rakotonirainy, A. (2024) · Vehicles · doi:10.3390/vehicles6030074 · Cited by 100
Self-citationA Survey on Deep Learning Based Approaches for Scene Understanding in Autonomous Driving
Guo, Z., Huang, Y., Hu, X., Wei, H., & Zhao, B. (2021) · Electronics · doi:10.3390/electronics10040471 · Cited by 85
Research works printed but not linked: 1
- Dosovitskiy, A. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
Cited by
AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103599 · Cited by 671
AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2025) · arXiv · doi:10.48550/arxiv.2505.10468 · Cited by 295
AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2025) · SuperIntelligence - Robotics - Safety & Alignment · doi:10.70777/si.v2i3.15161 · Cited by 186
Large Language Models (LLMs) as Traffic Control Systems at Urban Intersections: A New Paradigm
Masri, S., Ashqar, H. I., & Elhenawy, M. M. Z. (2025) · Vehicles · doi:10.3390/vehicles7010011 · Cited by 68
Self-citationHussain, K., Moreira, C., Pereira, J. M., Jardim, S., & Jorge, J. A. (2025) · Smart Cities · doi:10.3390/smartcities8030079 · Cited by 16
Metadata
- Authors
- Elhenawy, M., Ashqar, H. I., Rakotonirainy, A., Alhadidi, T. I., Jaber, A., & Tami, M. A.
- Affiliations
- Queensland University of Technology (AU)
- Arab American University (PS)
- Al-Ahliyya Amman University (JO)
- Budapest University of Technology and Economics (HU)
- Corresponding author
- Jaber, Ahmed
- ORCID
- Venue
- Electronics
- Publisher
- MDPI AG
- Published
- 2025-03-24
- Volume
- 14
- Issue
- 7
- Pages
- 1282
- ISSN
- 2079-9292 (ISSN-L)
- OpenAlex
- W4408781028
- Cited by
- 37
- References
- 29
- Citations by year
- 2026: 21 · 2025: 16
- FWCI
- 13.08
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Electronics (publishedVersion, cc-by)
- Funders
- Australian Government (DP220102598)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- autonomous driving, scene understanding, CLIP, scene classification, edge deployment, in-context learning, zero-shot learning, driver assistance systems, driver behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.