Cited by 741Advanced Vision and ImagingAdvanced Neural Network ApplicationsRobotics and Sensor-Based Localization

The DOI matches its Crossref record; 57 of the 59 references it cites were found in a registry or an index; none of the 18 sources its claims cite is checked for support yet.

3D Packing for Self-Supervised Monocular Depth Estimation

Open access
Guizilini, Vitor, Ambrus, Rares, Pillai, Sudeep, Raventos, Allan, Gaidon, Adrien·2020·2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Abstract

Although cameras are ubiquitous, robotic platforms typically rely on active sensors like LiDAR for direct 3D perception. In this work, we propose a novel self-supervised monocular depth estimation method combining geometry with a new deep network, PackNet, learned only from unlabeled monocular videos. Our architecture leverages novel symmetrical packing and unpacking blocks to jointly learn to compress and decompress detail-preserving representations using 3D convolutions. Although self-supervised, our method outperforms other self, semi, and fully supervised methods on the KITTI benchmark. The 3D inductive bias in PackNet enables it to scale with input resolution and number of parameters without overfitting, generalizing better on out-of-domain data such as the NuScenes dataset. Furthermore, it does not require large-scale supervised pretraining on ImageNet and can run in real-time. Finally, we release DDAD (Dense Depth for Automated Driving), a new urban driving dataset with more challenging and accurate depth evaluation, thanks to longer-range and denser ground-truth depth generated from high-density LiDARs mounted on a fleet of self-driving cars operating world-wide.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

PackNet is a self-supervised monocular depth architecture using symmetrical 3D packing/unpacking blocks, trained on unlabeled monocular videos, optionally with velocity supervision for metric scale. It reports state-of-the-art KITTI results, better generalization on NuScenes, real-time inference, and releases the DDAD dataset for long-range dense depth evaluation.

Key findings

  • On DDAD, PackNet-SfM achieves Abs Rel 0.162, Sq Rel 3.917, RMSE 13.452, RMSE log 0.269, and δ<1.25 0.823, outperforming Monodepth2 (R18) at Abs Rel 0.381 and RMSE 21.277; the gap increases at longer ranges up to 200m.
  • On NuScenes generalization without fine-tuning, PackNet reaches Abs Rel 0.187, Sq Rel 1.852, RMSE 7.636, RMSE log 0.289, and δ<1.25 0.742, better than ResNet50 (Abs Rel 0.216, RMSE 8.477) and ResNet50‡ (Abs Rel 0.210, RMSE 8.111).
  • Ablation on KITTI at 640×192 shows PackNet with D=8 3D filters reaches Abs Rel 0.111, Sq Rel 0.785, RMSE 4.601, RMSE log 0.189, and δ<1.25 0.878, improving over no pack/unpack (Abs Rel 0.121, RMSE 4.831).
  • PackNet reaches about 128M parameters with 60ms inference on a Titan V100, improvable to <30ms with TensorRT; a packing/unpacking reconstruction test achieves loss 0.0079 versus 0.063 for max-pooling/bilinear upsampling.

Methods

  • Self-supervised monocular SfM jointly trains a depth network and pose network on unlabeled monocular videos using a photometric loss (SSIM + L1), minimum over source images, auto-masking, and edge-aware depth smoothness; an optional velocity supervision loss aligns pose-translation magnitude with measured velocity to resolve scale ambiguity.
  • PackNet is a symmetrical encoder-decoder with packing blocks (Space2Depth, 3D convolution expansion, 2D contraction) and unpacking blocks (2D convolution, 3D convolution, Depth2Space), plus skip connections and multiple inverse-depth outputs.
  • Training uses KITTI (39,810 train, 4,424 validation, 697 evaluation; improved ground truth 652 maps), DDAD (17,050 train and 4,150 evaluation frames, Luminar-H2, up to 200m), NuScenes (6,019 front-facing validation images), and CityScapes (88,250 training images) for pretraining; PyTorch on 8 Titan V100 GPUs, Adam, 100 epochs, batch size 4, depth LR 2e-4, pose LR 5e-4, α=0.85, λ1=0.001, λ2=0.05.

Limitations

  • The paper states monocular SfM is inherently scale-ambiguous; metric scale requires velocity supervision at training time, otherwise test-time LiDAR ground-truth scaling is needed.
  • The paper states DDAD experiments use only the front-facing camera, and that packing/unpacking do not ensure invertibility.
  • No explicit limitations section is provided; evident constraints include KITTI evaluation up to 80m and DDAD up to 200m, and pose results are deferred to supplementary material due to space.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/cvpr42600.2020.00256

Cites

57 of 57 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

27 Own result · 14 Not checked yet · 13 No basis given · 12 Own interpretation

Verify all 18 references — free account
  • It may state two separate claims as one. 17
  • The quote only partly states this claim. 1
  • Probably not a substantial claim. 7
21 of 21
  • The paper relies on this without support.

    Instantaneous velocity measurements are ubiquitous in most mobile systems today.

    “Since instantaneous velocity measurements are ubiquitous in most mobile systems today, we show that they can be directly incorporated in our self-supervised objective to learn a metrically accurate and scale-aware monocular depth estimator.”
  • The paper relies on this without support.

    The same pixel will not be occluded or out-of-bounds in all context images, and the association with minimal photometric loss is correct.

    “The intuition is that the same pixel will not be occluded or out-of-bounds in all context images, and that the association with minimal photometric loss should be the correct one.”
  • The paper relies on this without support.

    Depth maps made by projecting LiDAR pointclouds onto the camera frame are accurate enough to serve as ground truth for evaluating monocular depth estimation.

    “The depth maps generated from projecting these Luminar pointclouds onto the camera frame allow us to evaluate depth estimation methods in a much more challenging way, both in terms of denseness and longer ranges.”
    “At test-time, all monocular methods (M) scale estimated depths with median ground-truth LiDAR information.”
3 of 3

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 59 references

Research works printed but not linked: 2

  1. kingma, 0, Glow: Generative flow with invertible 1×1 convolutions, Advances in neural information processing systems
  2. graham, 2015, Fractional Max-Pooling

Cited by

Sorted by citations

See all 741

Metadata

Authors
Guizilini, V., Ambrus, R., Pillai, S., Raventos, A., & Gaidon, A.
Affiliations
Toyota Research Institute (US)
Venue
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2020-06
Pages
2482-2491
ISSN
1063-6919 (ISSN-L)
OpenAlex
W2991840778
Cited by
741
References
59
Citations by year
2026: 51 · 2025: 108 · 2024: 140 · 2023: 155 · 2022: 139 · 2021: 125 · 2020: 19
FWCI
33.78
Type
conference paper
Language
en
Open access
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
Keywords
monocular depth estimation, self-supervised learning, 3D convolution, out-of-domain generalization, real-time depth estimation, autonomous driving, KITTI dataset, nuScenes

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review