CoChat

Real-Time Vehicle Classification and Tracking Using a Transfer Learning-Improved Deep Learning Network

Open access

The DOI matches its Crossref record; 54 of the 56 references it cites were found in a registry or an index; 12 of the 29 sources its claims cite were checked for support.

Abstract

Accurate vehicle classification and tracking are increasingly important subjects for intelligent transport systems (ITSs) and for planning that utilizes precise location intelligence. Deep learning (DL) and computer vision are intelligent methods; however, accurate real-time classification and tracking come with problems. We tackle three prominent problems (P1, P2, and P3): the need for a large training dataset (P1), the domain-shift problem (P2), and coupling a real-time multi-vehicle tracking algorithm with DL (P3). To address P1, we created a training dataset of nearly 30,000 samples from existing cameras with seven classes of vehicles. To tackle P2, we trained and applied transfer learning-based fine-tuning on several state-of-the-art YOLO (You Only Look Once) networks. For P3, we propose a multi-vehicle tracking algorithm that obtains the per-lane count, classification, and speed of vehicles in real time. The experiments showed that accuracy doubled after fine-tuning (71% vs. up to 30%). Based on a comparison of four YOLO networks, coupling the YOLOv5-large network to our tracking algorithm provided a trade-off between overall accuracy (95% vs. up to 90%), loss (0.033 vs. up to 0.036), and model size (91.6 MB vs. up to 120.6 MB). The implications of these results are in spatial information management and sensing for intelligent transport planning.

Overview

CoChat Summary

## TL;DR

The paper presents a real-time vehicle classification and tracking system that combines fine-tuned YOLO detectors with a custom centroid-based multi-vehicle tracker for lane-level counts, classification, and speed estimation. Using a newly created dataset of nearly 30,000 samples across seven vehicle classes, the best model (YOLOv5l) reached 95% overall accuracy in the tracking/classification setup after transfer-learning fine-tuning.

## Key findings

  • The authors created a training dataset of nearly 30,000 samples from Thailand surveillance cameras covering 7 classes: car, bus, taxi, bike, pickup, truck, and trailer.
  • Fine-tuning on a smaller target dataset boosted mAP_50 for all four YOLO models: YOLOv3 from 0.30 to 0.71, YOLOv3t from 0.26 to 0.67, YOLOv5l from 0.27 to 0.70, and YOLOv5s from 0.25 to 0.68.
  • On the tracking/classification task, YOLOv5l performed best with precision 0.96, recall 0.95, and overall accuracy 0.95; the other models had OA values of 0.90 (YOLOv3), 0.90 (YOLOv3t), and 0.83 (YOLOv5s).
  • In the stream-quality experiment, YOLOv5l counted/classified 99.6% and 94% of vehicles on clear video streams, and the average accuracy across four videos was 94%.

## Methods

  • Two-stage training design: TR1 was a large source dataset built from 23 surveillance cameras over 3 days (6.3 TB of video, manually annotated), and TR2 was a smaller target dataset from the deployment cameras used for transfer-learning fine-tuning.
  • Compared four YOLO variants: YOLOv3, YOLOv3-tiny, YOLOv5-large, and YOLOv5-small, all trained with 640 × 640 inputs and evaluated with mAP_50, precision, recall, loss, and model size.
  • Added a centroid-based multi-vehicle tracking algorithm that uses detected bounding boxes and lane polygons to register vehicles, count per lane, estimate speed, and produce lane-based reports in real time.

## Limitations

  • The paper states that there is “more room for improvement” in detection, counting, and tracking, and that metrics were calculated on full image frames; limiting detection to a smaller ROI could further improve accuracy.
  • Performance dropped in adverse conditions, especially at night, due to black-and-white streaming, headlights, and sudden light changes; the paper notes nighttime classification was highly affected.
  • Evident from the text: the system depends on manually drawn lane polygons, camera placement, and specific road geometry, which may limit generality across sites.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/s22103813

Cites

54 of 54 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Multi-vehicle tracking algorithm for the lane-based count and speed detection of vehicles.
Figure 1 Multi-vehicle tracking algorithm for the lane-based count and speed detection of vehicles.CC BY · PubMed Central
Figure 2 The samples of car, bus, taxi, bike, pickup, truck, and trailer (Classes 0 to 6).
Figure 2 The samples of car, bus, taxi, bike, pickup, truck, and trailer (Classes 0 to 6).CC BY · PubMed Central
Figure 3 Network architecture of YOLOv3 (adapted from [53] and modified) with a backbone of DarkNet-53.
Figure 3 Network architecture of YOLOv3 (adapted from [53] and modified) with a backbone of DarkNet-53.CC BY · PubMed Central
Figure 4 Network architecture of YOLOv5 (adapted from [57] and modified) with a backbone of Focus and CSP and a neck of CSP and PANet.
Figure 4 Network architecture of YOLOv5 (adapted from [57] and modified) with a backbone of Focus and CSP and a neck of CSP and PANet.CC BY · PubMed Central
Figure 5 Network training results of YOLOv3, YOLOv3t, YOLOv5l, and YOLOv5s on the TR1 dataset. (a) mAP at IoU = 50; (b) classification loss.
Figure 5 Network training results of YOLOv3, YOLOv3t, YOLOv5l, and YOLOv5s on the TR1 dataset. (a) mAP at IoU = 50; (b) classification loss.CC BY · PubMed Central
Figure 6 Sample results produced by the trained YOLOv5l model with the test images of TR1. The classes are predicted from 0 to 6, for car, bus, taxi, bike, pickup, truck, and trailer, respectively.
Figure 6 Sample results produced by the trained YOLOv5l model with the test images of TR1. The classes are predicted from 0 to 6, for car, bus, taxi, bike, pickup, truck, and trailer, respectively.CC BY · PubMed Central
Figure 7 An experimental setup of four surveillance cameras for traffic monitoring. The lanes in the four streams are drawn as seen in their corresponding images (Cam 1, …, Cam 4).
Figure 7 An experimental setup of four surveillance cameras for traffic monitoring. The lanes in the four streams are drawn as seen in their corresponding images (Cam 1, …, Cam 4).CC BY · PubMed Central
Figure 8 Sample image frames of four videos used for the experimentation on different qualities of streams. (a) Vid 1 (noisy video). (b) Vid 2 (high light intensity at 3 p.m.). (c) Vid 3 (proper light intensity at 5 p.m.). (d) Vid 4 (nighttime at 3 a.m.).
Figure 8 Sample image frames of four videos used for the experimentation on different qualities of streams. (a) Vid 1 (noisy video). (b) Vid 2 (high light intensity at 3 p.m.). (c) Vid 3 (proper light intensity at 5 p.m.). (d) Vid 4 (nighttime at 3 a.m.).CC BY · PubMed Central

Verification

Claims the paper makes

8 Valid · 6 Not valid · 7 Not checked · 62 Own results and interpretations

32 more claims

Create a free account to see all 83 claims and to verify the 1 reference still open.

See all claims — free account
  • Probably not a substantial claim. 16
  • It may state two separate claims as one. 33
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Citations

Cites

Research works printed but not linked: 1

  1. Kouw, W.M., and Loog, M. (2018). An introduction to domain adaptation and transfer learning. arXiv.

Cited by

See all 86

Metadata

Authors
Neupane, B., Horanont, T., & Aryal, J.
Affiliations
  • Advanced Geospatial Technology Research Unit, Sirindhorn International Institute of Technology, 131 Moo 5, Tiwanon Road, Bangkadi, Mueang Pathum Thani 12000, Pathum Thani, Thailand
  • School of Information, Computer, and Communication Technology (ICT), Sirindhorn International Institute of Technology, 131 Moo 5, Tiwanon Road, Bangkadi, Mueang Pathum Thani 12000, Pathum Thani, Thailand
  • The University of Melbourne (AU)
Corresponding author
Horanont, Teerayut
Venue
Sensors
Publisher
MDPI AG
Published
2022-05-18
Volume
22
Issue
10
Pages
3813
ISSN
1424-8220 (ISSN-L)
OpenAlex
W4280527735
Cited by
85
References
56
Citations by year
2026: 8 · 2025: 19 · 2024: 28 · 2023: 23 · 2022: 7
FWCI
6.30
Type
journal article
Language
en
Open access
Locations
  • Sensors (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Minerva Access (University of Melbourne) (submittedVersion, cc-by)
Funders
  • Thammasat University Research Fund under the TU Research Scholar (TUFT 88/2564, TUFF19/2564, TUFF24/2565)
  • Thailand Science Research and Innovation Fundamental Fund (TUFT 88/2564, TUFF19/2564, TUFF24/2565)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
  • Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
Keywords
vehicle classification, multi-vehicle tracking, real-time object detection, transfer learning, domain shift, YOLOv5, fine-tuning, vehicle speed estimation, intelligent transportation systems, training dataset construction
MeSH terms
Deep Learning, Algorithms, Humans, Neural Networks, Computer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.