CoChat
Sign Up
Cited by 33Advanced Neural Network ApplicationsAutonomous Vehicle Technology and SafetyAdvanced Vision and Imaging

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

LET-3D-AP: Longitudinal Error Tolerant 3D Average Precision for Camera-Only 3D Detection

Open access
Hung, Wei-Chih, Casser, Vincent, Kretzschmar, Henrik, Hwang, Jyh-Jing, Anguelov, Dragomir·2024·2024 IEEE International Conference on Robotics and Automation (ICRA)
DOI 10.1109/icra57147.2024.10609986
Open this paper in CoChat — free accountOpen PDF

Abstract

The 3D Average Precision (3DAP) relies on the intersection over union between predictions and ground truth objects. However, camera-only detectors have limited depth accuracy, which may cause otherwise reasonable predictions that suffer from such longitudinal localization errors to be treated as false positives. We therefore propose variants of the 3DAP metric to be more permissive with respect to depth estimation errors. Specifically, our novel longitudinal error tolerant metrics, LET-3D-AP and LET-3D-APL, allow longitudinal localization errors of the prediction boxes up to a given tolerance. To evaluate the proposed metrics, we also construct a new test set for the Waymo Open Dataset, tailored to camera-only 3D detection methods. Surprisingly, we find that state-of-the-art camera-based detectors can outperform popular LiDAR-based detectors with our new metrics past at 10% depth error tolerance, suggesting that existing camera-based detectors already have the potential to surpass LiDAR-based detectors in downstream applications. We believe the proposed metrics and the new benchmark dataset will facilitate advances in the field of camera-only 3D detection by providing more informative signals that can better indicate the system-level performance.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/icra57147.2024.10609986

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • LET-AP/APL/APH are used for cuboid metrics, and F1, x-coordinate rMSE and accuracy for lane detection.

    “We use LET-AP/APL/APH for cuboid metrics (Hung et al., 2024), and F1, x-coordinate rMSE and accuracy for detection, regression and classification scores of lane detection.”

    From World Simulation with Video Foundation Models for Physical AI (2025) · cites this paper as Hung et al., 2024

    Section C (‘LET-AP v.s. AP’) compares LET-AP and LET-APL for 3D object detections, supporting their use as cuboid metrics. The Related Work section mentions APH as a Waymo Open Dataset metric, but our copy does not identify a LET-APH metric. The lane-detection scores occur after the citation and are not attributed to this source. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 44

  1. 10.1109/CVPR46437.2021.00567 · DOI
  2. 10.1109/CVPR42600.2020.01054 · DOI · Paper page
  3. 10.1109/CVPR.2019.01298 · DOI · Paper page
  4. 10.1109/CVPR.2019.01296 · DOI · Paper page
  5. 10.1109/ICCV.2019.00937 · DOI · Paper page
  6. 10.1109/CVPR.2018.00472 · DOI · Paper page
  7. 10.1109/ICRA.2019.8794195 · DOI · Paper page
  8. 10.1109/CVPR.2016.236 · DOI · Paper page
  9. 10.1109/ICCV.2019.00938 · DOI · Paper page
  10. 10.1109/CVPR.2017.597 · DOI · Paper page
  11. 10.1109/CVPR.2018.00249 · DOI · Paper page
  12. 10.1109/ICCV.2019.00208 · DOI · Paper page
  13. 10.1109/ICCVW54120.2021.00107 · DOI · Paper page
  14. Wang, Detr3d: 3d object detection from multi-view images via 3d-to-2d queries
  15. 10.1109/WACV51458.2022.00133 · DOI · Paper page
  16. 10.1007/978-3-031-19839-7_23 · DOI
  17. 10.1109/CVPR42600.2020.00252 · DOI · Paper page
  18. 10.1007/s11263-014-0733-5 · DOI · Paper page
  19. 10.1007/978-3-319-10602-1_48 · DOI · Paper page
  20. 10.1109/CVPR.2012.6248074 · DOI · Paper page
  21. 10.1007/978-3-030-58520-4_32 · DOI
  22. 10.1007/978-3-030-58542-6_46 · DOI
  23. 10.1109/CVPR42600.2020.01211 · DOI · Paper page
  24. 10.1007/978-3-030-58580-8_38 · DOI
  25. 10.1007/978-3-031-19812-0_31 · DOI
  26. 10.1109/CVPR52688.2022.00398 · DOI
  27. 10.1109/iccv51070.2023.00840 · DOI
  28. 10.1109/iccv51070.2023.00302 · DOI
  29. 10.1007/978-3-030-58601-0_19 · DOI
  30. 10.1109/CVPR42600.2020.01169 · DOI
  31. Qian, End-to-end pseudolidar for image-based 3d object detection, 5881
  32. 10.1109/CVPR42600.2020.01255 · DOI
  33. 10.1109/ICCV48922.2021.00314 · DOI
  34. 10.1007/978-3-031-20077-9_1 · DOI · Paper page
  35. Weng, Monocular 3d object detection with pseudolidar point cloud
  36. 10.1609/aaai.v37i2.25233 · DOI
  37. 10.1609/aaai.v37i1.25185 · DOI
  38. 10.1145/3503161.3547859 · DOI
  39. 10.1007/978-3-031-20077-9_23 · DOI
  40. 10.1007/978-3-031-19809-0_8 · DOI
  41. 10.1109/CVPR46437.2021.00469 · DOI
  42. Rezatofighi, Generalized intersection over union
  43. 10.3390/s19194093 · DOI
  44. 10.1007/978-3-031-20080-9_25 · DOI

Cited by

Sorted by citations

Metadata

Authors
Hung, W.-C., Casser, V., Kretzschmar, H., Hwang, J.-J., & Anguelov, D.
Affiliations
Waymo LLC
ORCID
Hwang, Jyh-Jing
Venue
2024 IEEE International Conference on Robotics and Automation (ICRA)
Publisher
IEEE
Published
2024-05-13
Pages
8272-8279
ISSN
1050-4729 (ISSN-L)
DOI
10.1109/icra57147.2024.10609986
OpenAlex
W4401414801
Cited by
33
References
5
Citations by year
2026: 6 · 2025: 7 · 2024: 7
FWCI
2.90
Type
conference paper
Language
en
Open access
Licence
Locations
  • Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
Keywords
evaluation metrics, error tolerance, Waymo Open Dataset, benchmark dataset

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review