CoChat

MME-YOLO: Multi-Sensor Multi-Level Enhanced YOLO for Robust Vehicle Detection in Traffic Surveillance

Open access
Zhu, Jianxiao, Li, Xu, Jin, Peng, Xu, Qimin, Sun, Zhengliang, Song, Xiang · 2020 · Sensors

The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index; 10 of the 19 sources its claims cite were checked for support.

Abstract

As an effective means of solving collision problems caused by the limited perspective on board, the cooperative roadside system is gaining popularity. To improve the vehicle detection abilities in such online safety systems, in this paper, we propose a novel multi-sensor multi-level enhanced convolutional network model, called multi-sensor multi-level enhanced convolutional network architecture (MME-YOLO), with consideration of hybrid realistic scene of scales, illumination, and occlusion. MME-YOLO consists of two tightly coupled structures, i.e., the enhanced inference head and the LiDAR-Image composite module. More specifically, the enhanced inference head preliminarily equips the network with stronger inference abilities for redundant visual cues by attention-guided feature selection blocks and anchor-based/anchor-free ensemble head. Furthermore, the LiDAR-Image composite module cascades the multi-level feature maps from the LiDAR subnet to the image subnet, which strengthens the generalization of the detector in complex scenarios. Compared with YOLOv3, the enhanced inference head achieves a 5.83% and 4.88% mAP improvement on visual dataset LVSH and UA-DETRAC, respectively. Integrated with the composite module, the overall architecture gains 91.63% mAP in the collected Road-side Dataset. Experiments show that even under the abnormal lightings and the inconsistent scales at evening rush hours, the proposed MME-YOLO maintains reliable recognition accuracy and robust detection performance.

Overview

CoChat Summary

TL;DR

MME-YOLO is a multi-sensor vehicle detector that fuses LiDAR-derived depth/height/hue maps with image features at multiple levels, plus a cross-level attention and ensemble inference head. The paper reports improved vehicle-detection mAP over YOLOv3/YOLOv4 variants, especially for small vehicles and in adverse lighting/occlusion.

Key findings

  • On the visual enhancement tests, E-YOLO achieved 72.48% mAP on LSVH and 92.99% mAP on UA-DETRAC.
  • Compared with YOLOv3, the enhanced inference head improved mAP by 5.83% on LVSH and 4.88% on UA-DETRAC; the top-down attention structure alone improved overall detection by 2.47% on LSVH, with 5.36% improvement for small vehicles.
  • On the collected roadside dataset of nearly 6K frames, adding LiDAR multi-level features made MME-YOLO improve over YOLOv3 by 7.46% mAP, and over the best YOLOv4-L by 1.66%.
  • In the abstract, the full MME-YOLO system is reported to reach 91.63% mAP on the collected Road-side Dataset; the text also says the Car-category improvement grew from 0.80% to 7.37% when LiDAR was added.

Methods

  • Two-part architecture: an E-YOLO visual enhancement module and an MME-YOLO multi-sensor composite backbone, both built on YOLOv3.
  • Data/sample: evaluated on LSVH (14K images), UA-DETRAC (82K images), and a collected roadside dataset of about 6K frames from a HIKVISION DX-2XM6000 camera and Velodyne VLP-32 LiDAR.
  • LiDAR processing projects point clouds to the image plane, interpolates sparse depth/height maps, extracts hue from images, and stacks depth + height + hue as LiDAR-subnet inputs; analysis is mainly mAP comparison, plus ablations on attention blocks and inference heads.

Limitations

  • The paper states that MME-YOLO faces challenges from the effective distance of the LiDAR; it reports a minimum–maximum measurement distance of 93 m, beyond which cars are only occasionally captured.
  • The text makes evident that the LiDAR-subnet design is still limited by sparse point-cloud resolution and inconsistent roadside distributions (the paper explicitly discusses these as sensor difficulties); this is a stated problem context, not a separate experimental limitation.
  • The paper notes future work is needed for more weather conditions, more complex situations, time series, and other targets such as pedestrians and riders, indicating the current evaluation is limited in scope.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/s21010027

Cites

51 of 51 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 The differences between the traditional traffic management devices and the cooperative roadside system. While the traditional algorithms in traffic management devices concentrate on the occlusion scenarios, the ones in cooperative systems pay attention to the multi-scales and the robust detection ability towards the adverse environments.
Figure 1 The differences between the traditional traffic management devices and the cooperative roadside system. While the traditional algorithms in traffic management devices concentrate on the occlusion scenarios, the ones in cooperative systems pay attention to the multi-scales and the robust detection ability towards the adverse environments.CC BY · PubMed Central
Figure 2 The overall design of the multi-sensor multi-level enhanced convolutional network architecture (MME-YOLO) network. Our work creatively transmits the feature maps of the LiDAR subnet to the Image subnet in a recursive manner. Additionally, we develop cross-level attention blocks to strengthen the merged features. Then, the ensemble head is introduced for better generalization in real scenes.
Figure 2 The overall design of the multi-sensor multi-level enhanced convolutional network architecture (MME-YOLO) network. Our work creatively transmits the feature maps of the LiDAR subnet to the Image subnet in a recursive manner. Additionally, we develop cross-level attention blocks to strengthen the merged features. Then, the ensemble head is introduced for better generalization in real scenes.CC BY · PubMed Central
Figure 3 The attention guided feature selection blocks in cross-level attention module. (a) The top-down attention block; (b) The bottom-up attention block. Feature maps with the same size are multiplied or added in element-wise. These special designs of the attention blocks have greatly improved the performance of detectors, especially for small-scale vehicles. The capital T means the target feature maps and the capital S means the source feature maps. The symbol S0,S1,S2 refer to the feature map from the upper or lower layer, the attention mask, and the attention mask with the same size as the source feature map, respectively.
Figure 3 The attention guided feature selection blocks in cross-level attention module. (a) The top-down attention block; (b) The bottom-up attention block. Feature maps with the same size are multiplied or added in element-wise. These special designs of the attention blocks have greatly improved the performance of detectors, especially for small-scale vehicles. The capital T means the target feature maps and the capital S means the source feature maps. The symbol S0,S1,S2 refer to the feature map from the upper or lower layer, the attention mask, and the attention mask with the same size as the source feature map, respectively.CC BY · PubMed Central
Figure 4 Examples of projected points on KITTI and private Roadside Dataset. We randomly choose an example in both datasets and project the points into the image. The outcomes are listed as the following: (a) the projected image on KITTI. These points are covered nearly all cars except for the occluded ones or far-off ones. (b) The projected image on Roadside Dataset. The distribution of unbalanced points makes it difficult to identify multi-scale vehicles. For example, the bottom right black car receives fewer points than the farther white one. We remove the points on the road for better visualization.
Figure 4 Examples of projected points on KITTI and private Roadside Dataset. We randomly choose an example in both datasets and project the points into the image. The outcomes are listed as the following: (a) the projected image on KITTI. These points are covered nearly all cars except for the occluded ones or far-off ones. (b) The projected image on Roadside Dataset. The distribution of unbalanced points makes it difficult to identify multi-scale vehicles. For example, the bottom right black car receives fewer points than the farther white one. We remove the points on the road for better visualization.CC BY · PubMed Central
Figure 5 Two cases of the inputs of the Image subnet (the left column) and the LiDAR subnet (the right column). Case 1 shows a small-scale white car. Case 2 exhibits a light-blue bus and a medium-scale black car. From the visualizations of the LiDAR subnet input, the accurate distance and height value significantly contributes to the recognition of vehicles at semantic levels. However, the LiDAR-image also introduces inevitable noises into low-level cues, such as the edges. Therefore, it is acceptable that fusion from the semantic levels is more useful than fusion from the beginning level.
Figure 5 Two cases of the inputs of the Image subnet (the left column) and the LiDAR subnet (the right column). Case 1 shows a small-scale white car. Case 2 exhibits a light-blue bus and a medium-scale black car. From the visualizations of the LiDAR subnet input, the accurate distance and height value significantly contributes to the recognition of vehicles at semantic levels. However, the LiDAR-image also introduces inevitable noises into low-level cues, such as the edges. Therefore, it is acceptable that fusion from the semantic levels is more useful than fusion from the beginning level.CC BY · PubMed Central
Figure 6 Visualization of the multi-sensor multi-level composite module. The feature maps in the LiDAR subnet are added to the feature maps in the Image subnet in a recursive manner. This idea was originally published by CBNet [26]. We extend the composite methodology into the multi-sensor fusion field with a recursive multi-sensor multi-level composite structure.
Figure 6 Visualization of the multi-sensor multi-level composite module. The feature maps in the LiDAR subnet are added to the feature maps in the Image subnet in a recursive manner. This idea was originally published by CBNet [26]. We extend the composite methodology into the multi-sensor fusion field with a recursive multi-sensor multi-level composite structure.CC BY · PubMed Central
Figure 7 Detection Examples of YOLOv3 and E-YOLO (the visual enhanced YOLO) on LVSH (columns 1 and 2) and UA-DETRAC (columns 3 and 4). After plugged with the proposed attention structure and ensemble inference head, E-YOLO can easily detect the small-scale vehicles that are timely neglected by YOLOv3. The Chinese characters in the figures are the locations of the cameras, which are not used in our experiments. Best viewed on the screen.
Figure 7 Detection Examples of YOLOv3 and E-YOLO (the visual enhanced YOLO) on LVSH (columns 1 and 2) and UA-DETRAC (columns 3 and 4). After plugged with the proposed attention structure and ensemble inference head, E-YOLO can easily detect the small-scale vehicles that are timely neglected by YOLOv3. The Chinese characters in the figures are the locations of the cameras, which are not used in our experiments. Best viewed on the screen.CC BY · PubMed Central
Figure 8 The activation ranks of the objectiveness of two models in the small-scale vehicles. Pictures in the second column of each scene are the activation ranks of traditional YOLOv3, and the third column is the ones of E-YOLO. Objects are identified with different colors in different objectiveness ranks. The colors from light blue to orange-red are defined as the confidence from zero to one by indexing the confidence rank in HTML Color Table. Best viewed on screen.
Figure 8 The activation ranks of the objectiveness of two models in the small-scale vehicles. Pictures in the second column of each scene are the activation ranks of traditional YOLOv3, and the third column is the ones of E-YOLO. Objects are identified with different colors in different objectiveness ranks. The colors from light blue to orange-red are defined as the confidence from zero to one by indexing the confidence rank in HTML Color Table. Best viewed on screen.CC BY · PubMed Central
Figure 9 Qualitative detections of the E-YOLO and the YOLOv4-M on highway scenarios. The red bounding boxes are predicted by E-YOLO, the Blue bounding boxes are predicted by YOLOv4-M. Compared with the YOLOv4-M, which has a better detection toward the occlusion, the E-YOLO shows excellent performance on multi-scale recognition, especially for the cars. Best viewed on screen.
Figure 9 Qualitative detections of the E-YOLO and the YOLOv4-M on highway scenarios. The red bounding boxes are predicted by E-YOLO, the Blue bounding boxes are predicted by YOLOv4-M. Compared with the YOLOv4-M, which has a better detection toward the occlusion, the E-YOLO shows excellent performance on multi-scale recognition, especially for the cars. Best viewed on screen.CC BY · PubMed Central
Figure 10 Comparison between YOLOv4-L and MME-YOLO on the Road-side Dataset. Error histograms showed in the top-left corner. Minimum–maximum distances are listed in the bottom-left corner. Qualitative images are shown on the right. Light blue bounding boxes refer to the ground truth; red ones mean that wrong positions were predicted; orange ones mean that missed targets; green ones mean matched detections. We manually enlarged the areas for better visualization.
Figure 10 Comparison between YOLOv4-L and MME-YOLO on the Road-side Dataset. Error histograms showed in the top-left corner. Minimum–maximum distances are listed in the bottom-left corner. Qualitative images are shown on the right. Light blue bounding boxes refer to the ground truth; red ones mean that wrong positions were predicted; orange ones mean that missed targets; green ones mean matched detections. We manually enlarged the areas for better visualization.CC BY · PubMed Central

Verification

Claims the paper makes

7 Valid · 6 Not valid · 8 Not checked · 49 Own results and interpretations

36 more claims

Create a free account to see all 70 claims and to verify the 5 references still open.

See all claims — free account
  • Probably not a substantial claim. 18
  • It may state two separate claims as one. 11

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 51 references

Cited by

See all 51

Metadata

Authors
Zhu, J., Li, X., Jin, P., Xu, Q., Sun, Z., & Song, X.
Affiliations
  • Southeast University (CN)
  • Traffic Management Research Institute (CN)
  • Nanjing Xiaozhuang University (CN)
Corresponding author
Li, Xu
Venue
Sensors
Publisher
MDPI AG
Published
2020-12-23
Volume
21
Issue
1
Pages
27
ISSN
1424-8220 (ISSN-L)
OpenAlex
W3115103396
Cited by
51
References
51
Citations by year
2026: 5 · 2025: 12 · 2024: 12 · 2023: 9 · 2022: 10 · 2021: 3
FWCI
2.14
Type
journal article
Language
en
Open access
Locations
  • Sensors (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, other-oa)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Jiangsu Provincial Key Research and Development Program (BE2019106)
  • National Natural Science Foundation of China (61801227)
  • the Qing Lan Project of Jiangsu Province (QLGC2020)
  • the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (18KJB413007)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
Keywords
vehicle detection, traffic monitoring, multi-sensor fusion, YOLO, occlusion handling, illumination robustness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: MME-YOLO: Multi-Sensor Multi-Level Enhanced YOLO for Robust Vehicle Detection in Traffic Surveillance. https://app.cochat.ai/papers/10.3390/s21010027
The paper
Zhu, J., Li, X., Jin, P., Xu, Q., Sun, Z., & Song, X. (2020). MME-YOLO: Multi-Sensor Multi-Level Enhanced YOLO for Robust Vehicle Detection in Traffic Surveillance. Sensors. https://doi.org/10.3390/s21010027

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.