The DOI matches its Crossref record; 54 of the 72 references it cites were found in a registry or an index; 4 of the 21 sources its claims cite were checked for support.
OmniDet: Surround View Cameras Based Multi-Task Visual Perception Network for Autonomous Driving
Open accessAbstract
Surround View fisheye cameras are commonly deployed in automated driving for 360°near-field sensing around the vehicle. This work presents a multi-task visual perception network on unrectified fisheye images to enable the vehicle to sense its surrounding environment. It consists of six primary tasks necessary for an autonomous driving system: depth estimation, visual odometry, semantic segmentation, motion segmentation, object detection, and lens soiling detection. We demonstrate that the jointly trained model performs better than the respective single task versions. Our multi-task model has a shared encoder providing a significant computational advantage and has synergized decoders where tasks support each other. We propose a novel camera geometry based adaptation mechanism to encode the fisheye distortion model both at training and inference. This was crucial to enable training on the WoodScape dataset, comprised of data from different parts of the world collected by 12 different cameras mounted on three different cars with different intrinsics and viewpoints. Given that bounding boxes is not a good representation for distorted fisheye images, we also extend object detection to use a polygon with non-uniformly sampled vertices. We additionally evaluate our model on standard automotive datasets, namely KITTI and Cityscapes. We obtain the state-of-the-art results on KITTI for depth estimation and pose estimation tasks and competitive performance on the other tasks. We perform extensive ablation studies on various architecture choices and task weighting methodologies. A short video at https://youtu.be/xbSjZ5OfPes provides qualitative results.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
OmniDet is a multi-task visual perception network for unrectified surround-view fisheye cameras that jointly performs depth/distance estimation, visual odometry, semantic segmentation, motion segmentation, object detection, and lens soiling detection. It reports that joint training with a shared encoder and synergized decoders outperforms single-task baselines and achieves state-of-the-art KITTI depth and pose results among monocular methods.
Key findings
- The final WoodScape OmniDet (SAN10-patch) model reports RMSE 1.210, δ<1.25 0.929, semantic mIoU 78.9, motion mIoU 79.2, and detection mAP 74.1, improving over its own ablations without the proposed components.
- Multi-task learning outperformed single-task models on WoodScape, KITTI, and Cityscapes; for example, on WoodScape the multi-task setting reached distance RMSE 1.332, semantic mIoU 76.6, motion mIoU 75.3, and detection mAP 68.4, while single-task values included distance RMSE 1.681, semantic mIoU 72.5, motion mIoU 68.1, and detection mAP 63.5.
- VarNorm task weighting gave the best WoodScape task-weighting result: Sq.Rel 0.046, Abs.Rel 0.276, semantic mIoU 76.6, motion mIoU 75.3, and detection mAP 68.4, compared with equal weighting at Sq.Rel 0.058, Abs.Rel 0.302, semantic mIoU 70.3, motion mIoU 67.3, and detection mAP 64.6.
- On KITTI, OmniDet reports state-of-the-art monocular depth and pose results: depth Abs.Rel 0.092, Sq.Rel 0.657, RMSE 3.984 on the Eigen split, and pose errors of 0.009 ± 0.004 on sequence 09 and 0.008 ± 0.005 on sequence 10 with GT; with online refinement, depth improves to Abs.Rel 0.077, Sq.Rel 0.641, RMSE 3.859.
Methods
- The model uses a shared self-attention encoder with task-specific decoders for six tasks on unrectified fisheye images: distance/depth, pose/visual odometry, semantic segmentation, motion segmentation, object detection, and soiling segmentation.
- Key components include a camera geometry tensor to encode fisheye distortion and camera intrinsics, VarNorm variance-based task weighting, semantically guided distance and detection decoders via pixel-adaptive convolutions, dynamic-object masking from motion/semantic segmentation, and a 24-sided non-uniform polygon representation for fisheye object detection.
- Training and evaluation use WoodScape (10,000 images split 6:1:3, 12 cameras, three cars), KITTI, and Cityscapes; the model is trained in PyTorch for 20 epochs with batch size 24 on a Titan RTX, and inference is measured with TensorRT FP16 on NVIDIA Jetson AGX.
Limitations
- The paper states that scaling to more tasks still faces practical challenges: building a diverse and balanced dataset, corner case mining, stable training mechanisms, and designing an optimal map representation that combines all tasks while reducing post-processing.
- The paper states that the soiling dataset is independently built and cannot be trained jointly in a traditional manner; the shared encoder is frozen and only the soiling decoder is trained, and asynchronous backpropagation achieved the same accuracy.
- Evident from the text: polygon detection on raw fisheye is limited to only two classes, pedestrians and vehicles; motion labels for KITTI and Cityscapes are available only for the cars category; and additional proprietary data was used for pre-training and initialization on WoodScape.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
23 Own interpretation · 12 Own result · 9 No full text of the sources · 6 Sources not found · 6 No basis given · 3 Couldn’t confirm · 1 Partly supported · 1 Not checked yet · accuracy not yet measured
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The main goal of these losses is to prevent the training objective from getting stuck at multiple local minima for homogeneous areas as fisheye images have considerably larger homogeneous areas than their rectilinear counterpart.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > A. Self-Supervised Distance and Pose Estimation Networks
What the sources say
- Partly supportedcites [11] sourceExperimental — accuracy not yet measured
The Multi-scale Estimation section says multi-scale depth prediction and image reconstruction are used to keep training from getting stuck in local minima, and discusses ambiguity in large low-texture regions. It does not attribute that effect to feature-metric losses or compare homogeneous areas in fisheye and rectilinear images. Page numbers are unavailable in our copy. (LLM, conf=0.85)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Fisheye images have considerably larger homogeneous areas than their rectilinear counterpart.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Quantitative results from our experiments indicate that a multitask network with 6 tasks, 5 diverse tasks perform better than the single task models along with our proposed synergies explained in Section III-B.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > B. Single Task vs Multi-Task Learning
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When adding the camera geometry tensor to this setting, we observe a significant increase in accuracy since we train multiple cameras with different camera intrinsics and viewing angles.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This novel fusion technique in our OmniDet framework significantly improves the detection decoder's accuracy, which can be seen in Table I.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > III. NETWORK DETAILS OF MTL OMNIDET
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“we use semantically guided features to the detection decoder as described in Section III-B2, which yields significant gains in mAP”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We outperform the previous methods listed in Table VI”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > D. State-of-the-Art Comparison on KITTI
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We demonstrate the first real-time six-task model for surround-view fisheye camera perception.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > I. INTRODUCTION
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“We also trained soiling jointly using asynchronous backpropagation [22], but it achieved the same accuracy as using the frozen encoder.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Segmentation Tasks
What the sources say
- Not checked — no full text of the sourcecites [22] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We observe that uniform sampling cannot efficiently represent high curvature variations in the fisheye image object contours.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > B. Generalized Object Detection
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we evaluate our network on five tasks on KITTI and Cityscapes, achieving competitive results.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > V. CONCLUSION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The quote only partly states this claim. 2
- It may state two separate claims as one. 24
- Probably not a substantial claim. 13
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Using the online refinement method from [42], we obtain a significant improvement.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > D. State-of-the-Art Comparison on KITTI
What the sources say
- Could not verifycites [42] sourceExperimental — accuracy not yet measured
The Conclusion (p. 8) attributes improved performance to sparse convolution modules, while Boosting Single-Image Depth Prediction (pp. 14–15) discusses training with different depth maps. Neither passage addresses Casser et al.’s online refinement method or an improvement from using it; searches for “Casser” and “online refinement” found no such passage in our copy. (LLM, conf=0.76)
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
- Could not verify
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“In general, there is minimal work in the area of fisheye perception.Specifically, there is only our previous work on multi-task learning: FisheyeMultiNet [6] which discusses a more straight forward three task network.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > II. PERCEPTION TASKS AND LOSSES IN OUR MTL
What the sources say
- Could not verifycites [6] sourceExperimental — accuracy not yet measured
The Related Work section discusses single-view depth prediction and networks that jointly predict depth and segmentation, but does not address how much work exists on fisheye perception. Searches for “fisheye,” “FisheyeMultiNet,” and fisheye multi-task learning found no passage identifying FisheyeMultiNet or describing a three-task network. (LLM, conf=0.76)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Could not verify
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Shu et al. [11] proposed a convergent loss to have a relatively large convergence radius to enable gradient descent from a remote distance.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > A. Self-Supervised Distance and Pose Estimation Networks
What the sources say
- Could not verifycites [11] sourceExperimental — accuracy not yet measured
The Per-Pixel Minimum Reprojection Loss and Multi-scale Estimation sections describe Godard et al.’s depth-estimation losses, not a convergent loss proposed by Shu et al. Searches for “convergent loss,” “convergence radius,” and “Shu et al.” found no passage in our copy addressing the attributed claim. Page numbers were not available in the indexed passages. (LLM, conf=0.75)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Could not verify
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We employ the proposed VarNorm method for all the further experiments as it achieved the best results.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > D. Joint Optimization
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Table III, equal weighting is the worst, and our multitask network performs better than the single task networks by using any dynamic task weighting method presented above.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > D. Joint Optimization
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“24-sided Polygon 86.6 44.6 48”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We demonstrate that the jointly trained model performs better than the respective single task versions.”
- Also stated in: § I. INTRODUCTION · plain
“We showcase a six-task model on WoodScape and fivetask model on KITTI and Cityscapes performing better than the single task baselines.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > Abstract
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § I. INTRODUCTION · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We obtain the state-of-the-art results on KITTI for depth estimation and pose estimation tasks and competitive performance on the other tasks.”
- Also stated in: § I. INTRODUCTION · plain
“We obtain state-of-the-art results for depth and pose estimation tasks on KITTI among monocular methods.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
- Also stated in: § I. INTRODUCTION · plain
In the paper
“Fisheye cameras have a strong radial distortion that cannot be corrected without disadvantages, including reduced field-of-view and resampling distortion artifacts at the periphery [1].”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > I. INTRODUCTION
What the sources say
- Not checked — no full text of the sourcecites [1] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Unlike traffic lights and traffic signs, these categories are non-rigid in nature and quite diverse in appearance, making them suitable for polygon regression.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > 1) WoodScape:
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All our contributed features and the synergy between tasks help the OmniDet framework to achieve a good scene understanding with high accuracy in each task's predictions.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“We also experiment using cylindrical rectification (Cyl Rect.), which provides a good trade-off between loss of field-of-view and reducing distortion [29].”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
What the sources say
- Not checked — no full text of the sourcecites [29] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Learning these points helps to regress better polygon shapes, as these points at high curvature define the shape of the object contours.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > 1) WoodScape:
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As the weights are shared in the Siamese encoder, the previous frame's encoder can be saved and re-used instead of re-computing.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > III. NETWORK DETAILS OF MTL OMNIDET
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Motion is a dominant cue in automotive scenes, and it requires at least two frames or the use of dense optical flow [8].”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > II. PERCEPTION TASKS AND LOSSES IN OUR MTL
What the sources say
- Not checked — no full text of the sourcecites [8] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, this contribution possesses the potential to solve the infinite distance issue.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“Fisheye cameras are mounted low on a vehicle and are susceptible to lens soiling due to splashing of mud or water from the road.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > II. PERCEPTION TASKS AND LOSSES IN OUR MTL
What the sources say
- Not checked — no full text of the sourcecites [7] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As dynamic objects violate the static world assumption, information about their depth/distance is essential in autonomous driving; else, we would encounter the infinite depth issue during the inference stage and incur contamination to the reconstruction losses.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > III. NETWORK DETAILS OF MTL OMNIDET
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: Dynamic objects violate the static world assumption.
In the paper
“The proposed approach can, in principle, be applied to any fisheye projection model of choice explained in [1].”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > III. NETWORK DETAILS OF MTL OMNIDET
What the sources say
- Not checked — no full text of the sourcecites [1] source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, generic object detection using geometric cues like motion or depth for rare objects is typically used.They will also complement the detection of standard objects and provide more robustness.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > II. PERCEPTION TASKS AND LOSSES IN OUR MTL
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
- Rests on an assumption: Semantic tasks typically require a large annotated dataset covering various objects, and it is infeasible to practically cover every possible object.
In the paper
“Self-supervised methods have recently dominated depth estimation, which has also been demonstrated on fisheye images [9].”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > II. PERCEPTION TASKS AND LOSSES IN OUR MTL
What the sources say
- Not matchedcites [9]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
In the paper
“We build upon this work and generalize to arbitrary camera geometries, including fisheye cameras.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > III. NETWORK DETAILS OF MTL OMNIDET
What the sources say
- Not matchedcites [30]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The aforementioned training strategy makes the network camera-independent and better generalizes to images taken from a different camera.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Mao et al. [4] illustrated that multi-task learning improves adversarial robustness, which is critical for safety applications.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > I. INTRODUCTION
What the sources say
- Not checked — no full text of the sourcecites [4] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These losses help the gradient descent approaches to transit smoothly towards optimal solutions.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A single model for these four cameras instead of four individual models would also have several practical advantages such as (1) an improved efficiency on the embedded system requiring less memory and data rate to transmit, (2) an improved training by access to a larger dataset and regularization through different views, and (3) maintenance and certification of one model instead of four.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > III. NETWORK DETAILS OF MTL OMNIDET
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“This, in particular, breaks up the spatial invariance of the convolutions and allows the incorporation of location-specific semantic knowledge into the multi-level distance features.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > III. NETWORK DETAILS OF MTL OMNIDET
What the sources say
- Not matchedcites [32]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
In the paper
“The closest work is CAM-Convs [30], which uses camera-aware convolutions for pinhole cameras.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > III. NETWORK DETAILS OF MTL OMNIDET
What the sources say
- Not matchedcites [30]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our multi-task model has a shared encoder providing a significant computational advantage and has synergized decoders where tasks support each other.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“In the automotive multi-task setting, MultiNet [5] was one of the first to demonstrate a three task network on KITTI, and further works have primarily worked on a three task setting.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > I. INTRODUCTION
What the sources say
- Not matchedcites [5]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
None of the sources cited for this claim have been checked yet.
In the paper
“Multi-task learning (MTL) is an efficient design pattern commonly used where most of the computation is shared across all the tasks [2], [3].”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > I. INTRODUCTION
What the sources say
- Not checked — no full text of the sourcecites [2] source
- Not matchedcites [3]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Variance normalization re-scales the dispersion between the different task loss distributions based on the previous n epochs.A large dispersion leads to a lower task weight, whereas a small dispersion to a higher one.Its final effect tends to homogenize the learning speed of tasks.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > D. Joint Optimization
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper relies on this without support.
Appearance variations of objects are larger due to spatially variant distortion, particularly for close-by objects.
“Appearance variations of objects are larger due to the spatially variant distortion, particularly for close-by objects.”
The paper relies on this without support.
Fisheye perception is relatively less explored despite its prevalence.
“Thus fisheye perception is a challenging task, however it is relatively less explored despite its prevalence.”
The paper relies on this without support.
Learning features for multiple tasks can act as a regularizer, improving generalization.
“Furthermore, learning features for multiple tasks can act as a regularizer, improving generalization.”
The paper relies on this without support.
Semantic tasks typically require a large annotated dataset covering various objects, and it is infeasible to practically cover every possible object.
“The semantic tasks typically require a large annotated dataset covering various objects.It is infeasible to practically cover every possible object.”
The paper relies on this without support.
The visual odometry task is required to place the detected objects in a temporally consistent map.
“Finally, the visual odometry task is required to place the detected objects in a temporally consistent map.”
The paper relies on this without support.
Fisheye images have considerably larger homogeneous areas than their rectilinear counterpart.
“as fisheye images have considerably larger homogeneous areas than their rectilinear counterpart.”
The paper relies on this without support.
Although the underlying camera intrinsics model is the same for a particular family of vehicles, manufacturing variations require calibration of each camera instance.
“Although the underlying camera intrinsics model is the same for a particular family of vehicles, there are variations due to manufacturing processes, which require the calibration of each camera instance.”
The paper relies on this without support.
Calibration can vary after deployment due to high environmental temperature or aging.
“Even after deployment, calibration can vary due to high environmental temperature or due to aging.”
The paper relies on this without support.
Public datasets have a single camera instance for both the training and test dataset.
“This contrasts with public datasets, which have a single camera instance for both the training and test dataset.”
The authors state this as an assumption.
Dynamic objects violate the static world assumption.
“As dynamic objects violate the static world assumption, information about their depth/distance is essential in autonomous driving; else, we would encounter the infinite depth issue during the inference stage and incur contamination to the reconstruction losses.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > D. Joint Optimization
“As shown in Table III, equal weighting is the worst”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
“Single Task 0.060 0.304 72.5 94.8 68.1 94.1 63.5 Equal 0.058 0.302 70.3 92.7 67.3 93.3 64.6 DTP [25] 0.047 0.281 75.8 95.6 75.3 95.3 67.9 DWA [26] 0.054 0.293 75.4 95.2 74.7 95.1 67.5 Geometric [27] 0.061 0.297 74.2 94.1 73.2 94.3 66.7”
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · cited 1×
“It is a linear combination of the general robust pixel-wise loss term [12] and the Structural Similarity (SSIM) [13] as described in [14].”
MethodFocal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · cited 1×
“During training, the network predicts the posterior probability Y t which is optimized in a supervised fashion by Lovasz-Softmax [20] loss, and Focal [21] loss for handling class imbalance instead of the cross-entropy loss used in our previous work.”
MethodYOLOv3: An Incremental Improvement
Redmon, J., & Farhadi, A. (2018) · arXiv · doi:10.48550/arxiv.1804.02767 · Cited by 17,141 · cited 1×
“We adapted the YOLOv3 [19] decoder to output polygons and other representations listed above for a uniform comparison.”
MethodAre we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · cited 1×
“We systematically train and test all our single and multi-task models on Woodscape [29], a Surround View fisheye dataset with annotations for multiple tasks and on the pinhole camera datasets KITTI [35] and Cityscapes [36].”
MethodThe Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · cited 1×
“We systematically train and test all our single and multi-task models on Woodscape [29], a Surround View fisheye dataset with annotations for multiple tasks and on the pinhole camera datasets KITTI [35] and Cityscapes [36].”
MethodVision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913 · cited 1×
“To facilitate comparison to previous methods, we also train our distance estimation method in the classical depth estimation setting on the KITTI Eigen split [39] whose results are shown in Table V.”
MethodFocal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858826 · Cited by 9,556
Multiscale structural similarity for image quality assessment
Wang, Z., Simoncelli, E. P., & Bovik, A. C. (2004) · The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003 · doi:10.1109/acssc.2003.1292216 · Cited by 6,253
Douglas, D. H., & Peucker, T. K. (1973) · Cartographica The International Journal for Geographic Information and Geovisualization · doi:10.3138/fm57-6770-u75u-7727 · Cited by 4,252 · cited 1×
“Then we reduce the set of points using the algorithm in [18] to get the most representative simplified curves.”
MethodMulti-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
Cipolla, R., Gal, Y., & Kendall, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00781 · Cited by 3,867 · cited 1×
“We evaluate the uncertainty loss from Kendall [23], the gradient magnitude normalization GradNorm [24], the dynamic task prioritization DTP [25], the dynamic weight average DWA [26] and the geometric loss [27].”
MethodDepth Map Prediction from a Single Image using a Multi-Scale Deep Network
Eigen, D., Puhrsch, C., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1406.2283 · Cited by 3,442 · cited 1×
“We also evaluate using the Original [40] as well as Improved [41] ground truth depth maps.”
MethodUnsupervised Learning of Depth and Ego-Motion from Video
Zhou, T., Brown, M., Snavely, N., & Lowe, D. G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.700 · Cited by 2,910 · cited 1×
“In Table VI, we report the average trajectory error in meters of the pose estimation network by following the same protocols by Zhou [43] on the official KITTI odometry split (containing 11 sequences with ground-truth (GT) odometry acquired with the IMU/GPS measurements, which is used for…”
MethodEigen, D., & Fergus, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.304 · Cited by 2,862 · cited 1×
“[37] for self-supervised depth estimation with an input size of 640 × 192 px for all the tasks.”
MethodDigging Into Self-Supervised Monocular Depth Estimation
Godard, C., Aodha, O. M., Firman, M., & Brostow, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00393 · Cited by 2,561 · cited 1×
“The total loss comprised of a reconstruction matching term L r , regularization term L s , which enforces edge-aware smoothness inside the distance map ˆ D t as introduced in [10].”
BackgroundOn the Variance of the Adaptive Learning Rate and BeyondPreprint
Liu, L., Jiang, H., He, P., Chen, W., Liu, X., Gao, J., & Han, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.03265 · Cited by 1,837 · cited 1×
“We employ the Ranger (RAdam [33] + LookAhead [34]) optimizer to minimize the training objective function.”
MethodEnd-To-End Multi-Task Learning With Attention
Liu, S., Johns, E., & Davison, A. J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00197 · Cited by 1,299 · cited 1×
“We evaluate the uncertainty loss from Kendall [23], the gradient magnitude normalization GradNorm [24], the dynamic task prioritization DTP [25], the dynamic weight average DWA [26] and the geometric loss [27].”
MethodGeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose
Yin, Z., & Shi, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00212 · Cited by 1,280
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017) · arXiv · doi:10.48550/arxiv.1708.02002 · Cited by 1,227
Berman, M., Triki, A. R., & Blaschko, M. B. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00464 · Cited by 1,163 · cited 2×
“During training, the network predicts the posterior probability Yt which is optimized in a supervised fashion by Lovasz-Softmax [20] loss, and Focal [21] loss for handling class imbalance instead of the cross-entropy loss used in our previous work.”
MethodExploring Self-Attention for Image Recognition
Zhao, H., Jia, J., & Koltun, V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01009 · Cited by 1,002 · cited 2×
“3) Linking Self-Attention and Semantic features to 2D detection: To leverage the multi-task learning setup, at first, we extract the Self-Attention Network (SAN) [28] encoder features and feed it as an input signal to the Eq.”
MethodMulti-Task Learning for Dense Prediction Tasks: A Survey
Vandenhende, S., Georgoulis, S., Gansbeke, W. V., Proesmans, M., Dai, D., & Gool, L. V. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3054719 · Cited by 936 · cited 1×
“Multi-task learning (MTL) is an efficient design pattern commonly used where most of the computation is shared across all the tasks [2], [3].”
BackgroundUhrig, J., Schneider, N., Schneider, L., Franke, U., Brox, T., & Geiger, A. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00012 · Cited by 924 · cited 1×
“We also evaluate using the Original [40] as well as Improved [41] ground truth depth maps.”
MethodGradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask NetworksPreprint
Chen, Z., Badrinarayanan, V., Lee, C., & Rabinovich, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.02257 · Cited by 898
A General and Adaptive Robust Loss Function
Barron, J. T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00446 · Cited by 791 · cited 1×
“It is a linear combination of the general robust pixel-wise loss term [12] and the Structural Similarity (SSIM) [13] as described in [14].”
MethodAn Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution
Liu, R., Lehman, J., Molino, P., Such, F. P., Frank, E., Sergeev, A., & Yosinski, J. (2018) · arXiv · doi:10.48550/arxiv.1807.03247 · Cited by 784
3D Packing for Self-Supervised Monocular Depth Estimation
Guizilini, V., Ambrus, R., Pillai, S., Raventos, A., & Gaidon, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00256 · Cited by 741
On the detection of dominant points on digital curves
Teh, C.-H., & Chin, R. T. (1989) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.31447 · Cited by 730 · cited 1×
“We adopt the algorithm in [17] to detect the dominant points in a given curved shape, which best represents the object.”
MethodMultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving
Teichmann, M., Weber, M., Zollner, M., Cipolla, R., & Urtasun, R. (2018) · 2018 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2018.8500504 · Cited by 681 · cited 1×
“In the automotive multi-task setting, MultiNet [5] was one of the first to demonstrate a three task network on KITTI, and further works have primarily worked on a three task setting.”
MethodRanjan, A., Jampani, V., Balles, L., Kim, K., Sun, D., Wulff, J., & Black, M. J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01252 · Cited by 659
Kokkinos, I. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.579 · Cited by 657 · cited 1×
“We also trained soiling jointly using asynchronous backpropagation [22], but it achieved the same accuracy as using the frozen encoder.”
MethodLookahead Optimizer: k steps forward, 1 step backPreprint
Zhang, M. R., Lucas, J., Hinton, G., & Ba, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.08610 · Cited by 503 · cited 1×
“We employ the Ranger (RAdam [33] + LookAhead [34]) optimizer to minimize the training objective function.”
MethodDynamic Task Prioritization for Multitask Learning
Guo, M., Haque, A., Huang, D.-A., Yeung, S. Y., & Fei-Fei, L. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_17 · Cited by 393 · cited 1×
“We evaluate the uncertainty loss from Kendall [23], the gradient magnitude normalization GradNorm [24], the dynamic task prioritization DTP [25], the dynamic weight average DWA [26] and the geometric loss [27].”
MethodPixel-Adaptive Convolutional Neural Networks
Su, H., Jampani, V., Sun, D., Gallo, O., Learned-Miller, E., & Kautz, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01142 · Cited by 348 · cited 1×
“…work [14], to better incorporate the semantic knowledge extracted from the segmentation branch of the multi-task network into the distance estimation, we incorporate it using pixel-adaptive convolutions [32] (PAC) to distill the knowledge from the semantic features into the distance decoder.”
MethodWoodScape: A Multi-Task, Multi-Camera Fisheye Dataset for Autonomous Driving
Yogamani, S., Hughes, C., Horgan, J., Sistu, G., Chennupati, S., Uricar, M., Milz, S., Simon, M., Amende, K., Witt, C., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00940 · Cited by 334 · cited 3×
“In the Woodscape fisheye dataset [29], there are 12 different cameras with slight intrinsic variations to evaluate this effect.”
BackgroundMethodSelf-citationGradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
Zhao, C., Badrinarayanan, V., Lee, C., & Rabinovich, A. (2018) · International Conference on Machine Learning · Cited by 275
Chen, Y., Schmid, C., & Sminchisescu, C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00716 · Cited by 270
Feature-Metric Loss for Self-supervised Learning of Depth and Egomotion
Shu, C., Yu, K., Duan, Z., & Yang, K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58529-7_34 · Cited by 259 · cited 3×
“We incorporate feature-metric losses from [11] where L dis and L cvt are computed on I t ’s feature representation, where we learn the features using a self-attention autoencoder.”
BackgroundMethodInfluentialPoly-YOLO: higher speed, more precise detection and instance segmentation for YOLOv3
Hurtík, P., Molek, V., Hůla, J., Vajgl, M., Vlašánek, P., & Nejezchleba, T. (2022) · Neural Computing and Applications · doi:10.1007/s00521-021-05978-9 · Cited by 193 · cited 1×
“The object contour can be uniformly sampled in the range of 360 ° split into N equal polygon vertices, each represented by the radial distance r from the object’s centroid as used in PolyYOLO [16].”
MethodCAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth
Facil, J. M., Ummenhofer, B., Zhou, H., Montesano, L., Brox, T., & Civera, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01210 · Cited by 130 · cited 2×
“5) Normalized Coordinates ( nc ): Additionally, we add two channels of normalized coordinates [31], [30] whose values vary linearly between − 1 and 1 with respect to the image coordinates.”
MethodRashed, H., Mohamed, E., Sistu, G., Kumar, V. R., Eising, C., El-Sallab, A., & Yogamani, S. K. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00232 · Cited by 104 · cited 2×
“We briefly summarize the details here and refer to our expanded paper [15] for more details on generalized object detection.”
BackgroundSelf-citationMultiNet++: Multi-Stream Feature Aggregation and Geometric Loss Strategy for Multi-Task Learning
Chennupati, S., Sistu, G., Yogamani, S. K., & Rawashdeh, S. A. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00159 · Cited by 103 · cited 1×
“We evaluate the uncertainty loss from Kendall [23], the gradient magnitude normalization GradNorm [24], the dynamic task prioritization DTP [25], the dynamic weight average DWA [26] and the geometric loss [27].”
MethodSelf-citationMODNet: Motion and Appearance based Moving Object Detection Network for Autonomous Driving
Siam, M., Mahgoub, H., Zahran, M., Yogamani, S., Jagersand, M., & El-Sallab, A. (2018) · 2018 21st International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569744 · Cited by 95 · cited 2×
“Motion is a dominant cue in automotive scenes, and it requires at least two frames or the use of dense optical flow [8].”
BackgroundMethodSelf-citationKumar, V. R., Klingner, M., Yogamani, S. K., Milz, S., Fingscheidt, T., & Maeder, P. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00011 · Cited by 88 · cited 2×
“2) Semantically-Guided Distance and Detection Decoder: Following our previous work [14], to better incorporate the semantic knowledge extracted from the segmentation branch of the multi-task network into the distance estimation, we incorporate it using pixel-adaptive convolutions [32] (PAC) to…”
MethodSelf-citationKumar, V. R., Hiremath, S. A., Bach, M., Milz, S., Witt, C., Pinard, C., Yogamani, S. K., & Mäder, P. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197319 · Cited by 74
Self-citationSMSnet: Semantic motion segmentation using deep convolutional neural networks
Vertens, J., Valada, A., & Burgard, W. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202211 · Cited by 69 · cited 1×
“For the motion segmentation, we use the annotations provided by DeepMotion [38] for Cityscapes and KITTI MoSeg [8].”
MethodCasser, V., Pirk, S., Mahjourian, R., & Angelova, A. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018001 · Cited by 64 · cited 2×
“Using the online refinement method from [42], we obtain a significant improvement.”
MethodMultitask Learning Strengthens Adversarial Robustness
Mao, C., Gupta, A., Nitin, V., Ray, B., Song, S., Yang, J., & Vondrick, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58536-5_10 · Cited by 64
SoilingNet: Soiling Detection on Automotive Surround-View Cameras
Uricar, M., Krizek, P., Sistu, G., & Yogamani, S. (2019) · 2019 IEEE Intelligent Transportation Systems Conference (ITSC) · doi:10.1109/itsc.2019.8917178 · Cited by 55 · cited 2×
“Compared to our previous work SoilingNet [7], we moved from the tiled output to pixel-level segmentation.”
BackgroundMethodSelf-citationKumar, V. R., Yogamani, S. K., Bach, M., Witt, C., Milz, S., & Mäder, P. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340732 · Cited by 50 · cited 3×
“Fisheye cameras have a strong radial distortion that cannot be corrected without disadvantages, including reduced field-of-view and resampling distortion artifacts at the periphery [1].”
BackgroundMethodInfluentialSelf-citationNeurAll: Towards a Unified Visual Perception Model for Automated Driving
Sistu, G., Leang, I., Chennupati, S., Yogamani, S. K., Eising, C., Milz, S., & Rawashdeh, S. A. (2019) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2019.8917043 · Cited by 36 · cited 1×
“Multi-task learning (MTL) is an efficient design pattern commonly used where most of the computation is shared across all the tasks [2], [3].”
BackgroundSelf-citationPoly-YOLO: higher speed, more precise detection and instance segmentation for YOLOv3Preprint
Hurtík, P., Molek, V., Hůla, J., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.13243 · Cited by 25
Multitask Learning Strengthens Adversarial RobustnessPreprint
Mao, C., Gupta, A., Nitin, V., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.07236 · Cited by 12
CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View DepthPreprint
Fácil, J. M., Ummenhofer, B., Zhou, H., Montesano, L., Brox, T., & Civera, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02028 · Cited by 10 · cited 2×
“5) Normalized Coordinates ( nc ): Additionally, we add two channels of normalized coordinates [31], [30] whose values vary linearly between − 1 and 1 with respect to the image coordinates.”
MethodMaddu, P., Doherty, W., Sistu, G., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.11066 · Cited by 8 · cited 1×
“Specifically, there is only our previous work on multi-task learning: FisheyeMultiNet [6] which discusses a more straight forward three task network.”
BackgroundSelf-citation
Research works printed but not linked: 1
- geiger, 0, Proc Conf Comput Vis Pattern Recognit, 3354
Cited by
Sorted by citations
Understanding World or Predicting Future? A Comprehensive Survey of World Models
Ding, J., Zhang, Y., Shang, Y., Zhang, Y., Zong, Z., Feng, J., Yuan, Y., Su, H., Li, N., Sukiennik, N., et al. (2026) · ACM Computing Surveys · doi:10.1145/3746449 · Cited by 169
Review on Panoramic Imaging and Its Applications in Scene Understanding
Gao, S., Yang, K., Shi, H., Wang, K., & Bai, J. Y. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3216675 · Cited by 121
“other fisheye [324], [325], [326], [327], panoramic [328], [329], [330], and omnidirectional [331], [332], [333], [334]”
Surround-View Fisheye Camera Perception for Automated Driving: Overview, Survey & Challenges
Kumar, V. R., Eising, C., Witt, C., & Yogamani, S. K. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3235057 · Cited by 132
“Finally, a holistic real-time scene understanding for the near-field perception of the environment using cameras only was presented in OmniDet [123].”
MethodInfluentialSelf-citationNeuromorphic computing chip with spatiotemporal elasticity for multi-intelligent-tasking robots
Ma, S., Pei, J., Zhang, W., Wang, G., Feng, D., Yu, F., Song, C., Qu, H., Ma, C., Lü, M., et al. (2022) · Science Robotics · doi:10.1126/scirobotics.abk2948 · Cited by 70
“Moreover, the application of multiple models ( 11 – 13 ) and multi-tasking NNs ( 14 – 17 ) can further improve the ability and adaptability of robots.”
MethodSynWoodScape: Synthetic Surround-View Fisheye Camera Dataset for Autonomous Driving
Sekkat, A. R., Dupuis, Y., Kumar, V. R., Rashed, H., Yogamani, S. K., Vasseur, P., & Honeiné, P. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3188106 · Cited by 64
Self-citation
Metadata
- Authors
- Ravi Kumar, V., Yogamani, S., Rashed, H., Sitsu, G., Witt, C., Leang, I., Milz, S., & Mader, P.
- Affiliations
- Valeo (Ireland) (IE)
- Valeo (Germany) (DE)
- Valeo (France) (FR)
- Technische Universität Ilmenau (DE)
- Venue
- IEEE Robotics and Automation Letters
- Published
- 2021-04
- Volume
- 6
- Issue
- 2
- Pages
- 2830-2837
- ISSN
- 2377-3766 (ISSN-L)
- 2377-3774
- OpenAlex
- W3129414859
- Semantic Scholar
- 58fce566cc744fac4851353ee3e6132e8fbe20e5
- Cited by
- 121
- References
- 72
- Citations by year
- 2026: 15 · 2025: 25 · 2024: 19 · 2023: 23 · 2022: 26 · 2021: 12
- FWCI
- 7.05
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Robotics and Automation Letters (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- surround-view cameras, fisheye images, multi-task learning, autonomous driving, depth estimation, visual odometry, semantic segmentation, motion segmentation, object detection, shared encoder, KITTI, cityscape
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
