The DOI matches its Crossref record; 54 of the 72 references it cites were found in a registry or an index; 4 of the 21 sources its claims cite were checked for support.
OmniDet: Surround View Cameras Based Multi-Task Visual Perception Network for Autonomous Driving
Open accessAbstract
Surround View fisheye cameras are commonly deployed in automated driving for 360°near-field sensing around the vehicle. This work presents a multi-task visual perception network on unrectified fisheye images to enable the vehicle to sense its surrounding environment. It consists of six primary tasks necessary for an autonomous driving system: depth estimation, visual odometry, semantic segmentation, motion segmentation, object detection, and lens soiling detection. We demonstrate that the jointly trained model performs better than the respective single task versions. Our multi-task model has a shared encoder providing a significant computational advantage and has synergized decoders where tasks support each other. We propose a novel camera geometry based adaptation mechanism to encode the fisheye distortion model both at training and inference. This was crucial to enable training on the WoodScape dataset, comprised of data from different parts of the world collected by 12 different cameras mounted on three different cars with different intrinsics and viewpoints. Given that bounding boxes is not a good representation for distorted fisheye images, we also extend object detection to use a polygon with non-uniformly sampled vertices. We additionally evaluate our model on standard automotive datasets, namely KITTI and Cityscapes. We obtain the state-of-the-art results on KITTI for depth estimation and pose estimation tasks and competitive performance on the other tasks. We perform extensive ablation studies on various architecture choices and task weighting methodologies. A short video at https://youtu.be/xbSjZ5OfPes provides qualitative results.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
OmniDet is a multi-task visual perception network for unrectified surround-view fisheye cameras that jointly performs depth/distance estimation, visual odometry, semantic segmentation, motion segmentation, object detection, and lens soiling detection. It reports that joint training with a shared encoder and synergized decoders outperforms single-task baselines and achieves state-of-the-art KITTI depth and pose results among monocular methods.
Key findings
- The final WoodScape OmniDet (SAN10-patch) model reports RMSE 1.210, δ<1.25 0.929, semantic mIoU 78.9, motion mIoU 79.2, and detection mAP 74.1, improving over its own ablations without the proposed components.
- Multi-task learning outperformed single-task models on WoodScape, KITTI, and Cityscapes; for example, on WoodScape the multi-task setting reached distance RMSE 1.332, semantic mIoU 76.6, motion mIoU 75.3, and detection mAP 68.4, while single-task values included distance RMSE 1.681, semantic mIoU 72.5, motion mIoU 68.1, and detection mAP 63.5.
- VarNorm task weighting gave the best WoodScape task-weighting result: Sq.Rel 0.046, Abs.Rel 0.276, semantic mIoU 76.6, motion mIoU 75.3, and detection mAP 68.4, compared with equal weighting at Sq.Rel 0.058, Abs.Rel 0.302, semantic mIoU 70.3, motion mIoU 67.3, and detection mAP 64.6.
- On KITTI, OmniDet reports state-of-the-art monocular depth and pose results: depth Abs.Rel 0.092, Sq.Rel 0.657, RMSE 3.984 on the Eigen split, and pose errors of 0.009 ± 0.004 on sequence 09 and 0.008 ± 0.005 on sequence 10 with GT; with online refinement, depth improves to Abs.Rel 0.077, Sq.Rel 0.641, RMSE 3.859.
Methods
- The model uses a shared self-attention encoder with task-specific decoders for six tasks on unrectified fisheye images: distance/depth, pose/visual odometry, semantic segmentation, motion segmentation, object detection, and soiling segmentation.
- Key components include a camera geometry tensor to encode fisheye distortion and camera intrinsics, VarNorm variance-based task weighting, semantically guided distance and detection decoders via pixel-adaptive convolutions, dynamic-object masking from motion/semantic segmentation, and a 24-sided non-uniform polygon representation for fisheye object detection.
- Training and evaluation use WoodScape (10,000 images split 6:1:3, 12 cameras, three cars), KITTI, and Cityscapes; the model is trained in PyTorch for 20 epochs with batch size 24 on a Titan RTX, and inference is measured with TensorRT FP16 on NVIDIA Jetson AGX.
Limitations
- The paper states that scaling to more tasks still faces practical challenges: building a diverse and balanced dataset, corner case mining, stable training mechanisms, and designing an optimal map representation that combines all tasks while reducing post-processing.
- The paper states that the soiling dataset is independently built and cannot be trained jointly in a traditional manner; the shared encoder is frozen and only the soiling decoder is trained, and asynchronous backpropagation achieved the same accuracy.
- Evident from the text: polygon detection on raw fisheye is limited to only two classes, pedestrians and vehicles; motion labels for KITTI and Cityscapes are available only for the cars category; and additional proprietary data was used for pre-training and initialization on WoodScape.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
23 Own interpretation · 12 Own result · 9 No full text of the sources · 6 Sources not found · 6 No basis given · 3 Couldn’t confirm · 1 Partly supported · 1 Not checked yet · accuracy not yet measured
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The main goal of these losses is to prevent the training objective from getting stuck at multiple local minima for homogeneous areas as fisheye images have considerably larger homogeneous areas than their rectilinear counterpart.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > A. Self-Supervised Distance and Pose Estimation Networks
What the sources say
- Partly supportedcites [11] sourceExperimental — accuracy not yet measured
The Multi-scale Estimation section says multi-scale depth prediction and image reconstruction are used to keep training from getting stuck in local minima, and discusses ambiguity in large low-texture regions. It does not attribute that effect to feature-metric losses or compare homogeneous areas in fisheye and rectilinear images. Page numbers are unavailable in our copy. (LLM, conf=0.85)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Fisheye images have considerably larger homogeneous areas than their rectilinear counterpart.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Quantitative results from our experiments indicate that a multitask network with 6 tasks, 5 diverse tasks perform better than the single task models along with our proposed synergies explained in Section III-B.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > B. Single Task vs Multi-Task Learning
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When adding the camera geometry tensor to this setting, we observe a significant increase in accuracy since we train multiple cameras with different camera intrinsics and viewing angles.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This novel fusion technique in our OmniDet framework significantly improves the detection decoder's accuracy, which can be seen in Table I.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > III. NETWORK DETAILS OF MTL OMNIDET
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“we use semantically guided features to the detection decoder as described in Section III-B2, which yields significant gains in mAP”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We outperform the previous methods listed in Table VI”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > D. State-of-the-Art Comparison on KITTI
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We demonstrate the first real-time six-task model for surround-view fisheye camera perception.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > I. INTRODUCTION
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“We also trained soiling jointly using asynchronous backpropagation [22], but it achieved the same accuracy as using the frozen encoder.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Segmentation Tasks
What the sources say
- Not checked — no full text of the sourcecites [22] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We observe that uniform sampling cannot efficiently represent high curvature variations in the fisheye image object contours.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > B. Generalized Object Detection
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we evaluate our network on five tasks on KITTI and Cityscapes, achieving competitive results.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > V. CONCLUSION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The quote only partly states this claim. 2
- It may state two separate claims as one. 24
- Probably not a substantial claim. 13
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Using the online refinement method from [42], we obtain a significant improvement.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > D. State-of-the-Art Comparison on KITTI
What the sources say
- Could not verifycites [42] sourceExperimental — accuracy not yet measured
The Conclusion (p. 8) attributes improved performance to sparse convolution modules, while Boosting Single-Image Depth Prediction (pp. 14–15) discusses training with different depth maps. Neither passage addresses Casser et al.’s online refinement method or an improvement from using it; searches for “Casser” and “online refinement” found no such passage in our copy. (LLM, conf=0.76)
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
- Could not verify
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“In general, there is minimal work in the area of fisheye perception.Specifically, there is only our previous work on multi-task learning: FisheyeMultiNet [6] which discusses a more straight forward three task network.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > II. PERCEPTION TASKS AND LOSSES IN OUR MTL
What the sources say
- Could not verifycites [6] sourceExperimental — accuracy not yet measured
The Related Work section discusses single-view depth prediction and networks that jointly predict depth and segmentation, but does not address how much work exists on fisheye perception. Searches for “fisheye,” “FisheyeMultiNet,” and fisheye multi-task learning found no passage identifying FisheyeMultiNet or describing a three-task network. (LLM, conf=0.76)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Could not verify
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Shu et al. [11] proposed a convergent loss to have a relatively large convergence radius to enable gradient descent from a remote distance.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > A. Self-Supervised Distance and Pose Estimation Networks
What the sources say
- Could not verifycites [11] sourceExperimental — accuracy not yet measured
The Per-Pixel Minimum Reprojection Loss and Multi-scale Estimation sections describe Godard et al.’s depth-estimation losses, not a convergent loss proposed by Shu et al. Searches for “convergent loss,” “convergence radius,” and “Shu et al.” found no passage in our copy addressing the attributed claim. Page numbers were not available in the indexed passages. (LLM, conf=0.75)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Could not verify
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We employ the proposed VarNorm method for all the further experiments as it achieved the best results.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > D. Joint Optimization
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Table III, equal weighting is the worst, and our multitask network performs better than the single task networks by using any dynamic task weighting method presented above.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > D. Joint Optimization
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“24-sided Polygon 86.6 44.6 48”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We demonstrate that the jointly trained model performs better than the respective single task versions.”
- Also stated in: § I. INTRODUCTION · plain
“We showcase a six-task model on WoodScape and fivetask model on KITTI and Cityscapes performing better than the single task baselines.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > Abstract
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § I. INTRODUCTION · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We obtain the state-of-the-art results on KITTI for depth estimation and pose estimation tasks and competitive performance on the other tasks.”
- Also stated in: § I. INTRODUCTION · plain
“We obtain state-of-the-art results for depth and pose estimation tasks on KITTI among monocular methods.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
- Also stated in: § I. INTRODUCTION · plain
In the paper
“Fisheye cameras have a strong radial distortion that cannot be corrected without disadvantages, including reduced field-of-view and resampling distortion artifacts at the periphery [1].”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > I. INTRODUCTION
What the sources say
- Not checked — no full text of the sourcecites [1] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Unlike traffic lights and traffic signs, these categories are non-rigid in nature and quite diverse in appearance, making them suitable for polygon regression.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > 1) WoodScape:
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All our contributed features and the synergy between tasks help the OmniDet framework to achieve a good scene understanding with high accuracy in each task's predictions.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“We also experiment using cylindrical rectification (Cyl Rect.), which provides a good trade-off between loss of field-of-view and reducing distortion [29].”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
What the sources say
- Not checked — no full text of the sourcecites [29] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Learning these points helps to regress better polygon shapes, as these points at high curvature define the shape of the object contours.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > 1) WoodScape:
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As the weights are shared in the Siamese encoder, the previous frame's encoder can be saved and re-used instead of re-computing.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > III. NETWORK DETAILS OF MTL OMNIDET
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Motion is a dominant cue in automotive scenes, and it requires at least two frames or the use of dense optical flow [8].”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > II. PERCEPTION TASKS AND LOSSES IN OUR MTL
What the sources say
- Not checked — no full text of the sourcecites [8] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, this contribution possesses the potential to solve the infinite distance issue.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“Fisheye cameras are mounted low on a vehicle and are susceptible to lens soiling due to splashing of mud or water from the road.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > II. PERCEPTION TASKS AND LOSSES IN OUR MTL
What the sources say
- Not checked — no full text of the sourcecites [7] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As dynamic objects violate the static world assumption, information about their depth/distance is essential in autonomous driving; else, we would encounter the infinite depth issue during the inference stage and incur contamination to the reconstruction losses.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > III. NETWORK DETAILS OF MTL OMNIDET
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: Dynamic objects violate the static world assumption.
In the paper
“The proposed approach can, in principle, be applied to any fisheye projection model of choice explained in [1].”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > III. NETWORK DETAILS OF MTL OMNIDET
What the sources say
- Not checked — no full text of the sourcecites [1] source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, generic object detection using geometric cues like motion or depth for rare objects is typically used.They will also complement the detection of standard objects and provide more robustness.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > II. PERCEPTION TASKS AND LOSSES IN OUR MTL
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
- Rests on an assumption: Semantic tasks typically require a large annotated dataset covering various objects, and it is infeasible to practically cover every possible object.
In the paper
“Self-supervised methods have recently dominated depth estimation, which has also been demonstrated on fisheye images [9].”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > II. PERCEPTION TASKS AND LOSSES IN OUR MTL
What the sources say
- Not matchedcites [9]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
In the paper
“We build upon this work and generalize to arbitrary camera geometries, including fisheye cameras.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > III. NETWORK DETAILS OF MTL OMNIDET
What the sources say
- Not matchedcites [30]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The aforementioned training strategy makes the network camera-independent and better generalizes to images taken from a different camera.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Mao et al. [4] illustrated that multi-task learning improves adversarial robustness, which is critical for safety applications.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > I. INTRODUCTION
What the sources say
- Not checked — no full text of the sourcecites [4] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These losses help the gradient descent approaches to transit smoothly towards optimal solutions.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A single model for these four cameras instead of four individual models would also have several practical advantages such as (1) an improved efficiency on the embedded system requiring less memory and data rate to transmit, (2) an improved training by access to a larger dataset and regularization through different views, and (3) maintenance and certification of one model instead of four.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > III. NETWORK DETAILS OF MTL OMNIDET
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“This, in particular, breaks up the spatial invariance of the convolutions and allows the incorporation of location-specific semantic knowledge into the multi-level distance features.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > III. NETWORK DETAILS OF MTL OMNIDET
What the sources say
- Not matchedcites [32]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
In the paper
“The closest work is CAM-Convs [30], which uses camera-aware convolutions for pinhole cameras.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > III. NETWORK DETAILS OF MTL OMNIDET
What the sources say
- Not matchedcites [30]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our multi-task model has a shared encoder providing a significant computational advantage and has synergized decoders where tasks support each other.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“In the automotive multi-task setting, MultiNet [5] was one of the first to demonstrate a three task network on KITTI, and further works have primarily worked on a three task setting.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > I. INTRODUCTION
What the sources say
- Not matchedcites [5]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
None of the sources cited for this claim have been checked yet.
In the paper
“Multi-task learning (MTL) is an efficient design pattern commonly used where most of the computation is shared across all the tasks [2], [3].”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > I. INTRODUCTION
What the sources say
- Not checked — no full text of the sourcecites [2] source
- Not matchedcites [3]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Variance normalization re-scales the dispersion between the different task loss distributions based on the previous n epochs.A large dispersion leads to a lower task weight, whereas a small dispersion to a higher one.Its final effect tends to homogenize the learning speed of tasks.”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > D. Joint Optimization
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper relies on this without support.
Appearance variations of objects are larger due to spatially variant distortion, particularly for close-by objects.
“Appearance variations of objects are larger due to the spatially variant distortion, particularly for close-by objects.”
The paper relies on this without support.
Fisheye perception is relatively less explored despite its prevalence.
“Thus fisheye perception is a challenging task, however it is relatively less explored despite its prevalence.”
The paper relies on this without support.
Learning features for multiple tasks can act as a regularizer, improving generalization.
“Furthermore, learning features for multiple tasks can act as a regularizer, improving generalization.”
The paper relies on this without support.
Semantic tasks typically require a large annotated dataset covering various objects, and it is infeasible to practically cover every possible object.
“The semantic tasks typically require a large annotated dataset covering various objects.It is infeasible to practically cover every possible object.”
The paper relies on this without support.
The visual odometry task is required to place the detected objects in a temporally consistent map.
“Finally, the visual odometry task is required to place the detected objects in a temporally consistent map.”
The paper relies on this without support.
Fisheye images have considerably larger homogeneous areas than their rectilinear counterpart.
“as fisheye images have considerably larger homogeneous areas than their rectilinear counterpart.”
The paper relies on this without support.
Although the underlying camera intrinsics model is the same for a particular family of vehicles, manufacturing variations require calibration of each camera instance.
“Although the underlying camera intrinsics model is the same for a particular family of vehicles, there are variations due to manufacturing processes, which require the calibration of each camera instance.”
The paper relies on this without support.
Calibration can vary after deployment due to high environmental temperature or aging.
“Even after deployment, calibration can vary due to high environmental temperature or due to aging.”
The paper relies on this without support.
Public datasets have a single camera instance for both the training and test dataset.
“This contrasts with public datasets, which have a single camera instance for both the training and test dataset.”
The authors state this as an assumption.
Dynamic objects violate the static world assumption.
“As dynamic objects violate the static world assumption, information about their depth/distance is essential in autonomous driving; else, we would encounter the infinite depth issue during the inference stage and incur contamination to the reconstruction losses.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > D. Joint Optimization
“As shown in Table III, equal weighting is the worst”
§ OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving > C. Ablation Study of our Contributions
“Single Task 0.060 0.304 72.5 94.8 68.1 94.1 63.5 Equal 0.058 0.302 70.3 92.7 67.3 93.3 64.6 DTP [25] 0.047 0.281 75.8 95.6 75.3 95.3 67.9 DWA [26] 0.054 0.293 75.4 95.2 74.7 95.1 67.5 Geometric [27] 0.061 0.297 74.2 94.1 73.2 94.3 66.7”
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · cited 1×
“It is a linear combination of the general robust pixel-wise loss term [12] and the Structural Similarity (SSIM) [13] as described in [14].”
MethodFocal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · cited 1×
“During training, the network predicts the posterior probability Y t which is optimized in a supervised fashion by Lovasz-Softmax [20] loss, and Focal [21] loss for handling class imbalance instead of the cross-entropy loss used in our previous work.”
MethodYOLOv3: An Incremental Improvement
Redmon, J., & Farhadi, A. (2018) · arXiv · doi:10.48550/arxiv.1804.02767 · Cited by 17,141 · cited 1×
“We adapted the YOLOv3 [19] decoder to output polygons and other representations listed above for a uniform comparison.”
MethodAre we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · cited 1×
“We systematically train and test all our single and multi-task models on Woodscape [29], a Surround View fisheye dataset with annotations for multiple tasks and on the pinhole camera datasets KITTI [35] and Cityscapes [36].”
MethodThe Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · cited 1×
“We systematically train and test all our single and multi-task models on Woodscape [29], a Surround View fisheye dataset with annotations for multiple tasks and on the pinhole camera datasets KITTI [35] and Cityscapes [36].”
MethodVision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913 · cited 1×
“To facilitate comparison to previous methods, we also train our distance estimation method in the classical depth estimation setting on the KITTI Eigen split [39] whose results are shown in Table V.”
MethodFocal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858826 · Cited by 9,556
Multiscale structural similarity for image quality assessment
Wang, Z., Simoncelli, E. P., & Bovik, A. C. (2004) · The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003 · doi:10.1109/acssc.2003.1292216 · Cited by 6,253
Douglas, D. H., & Peucker, T. K. (1973) · Cartographica The International Journal for Geographic Information and Geovisualization · doi:10.3138/fm57-6770-u75u-7727 · Cited by 4,252 · cited 1×
“Then we reduce the set of points using the algorithm in [18] to get the most representative simplified curves.”
MethodMulti-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
Cipolla, R., Gal, Y., & Kendall, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00781 · Cited by 3,867 · cited 1×
“We evaluate the uncertainty loss from Kendall [23], the gradient magnitude normalization GradNorm [24], the dynamic task prioritization DTP [25], the dynamic weight average DWA [26] and the geometric loss [27].”
Method
Research works printed but not linked: 1
- geiger, 0, Proc Conf Comput Vis Pattern Recognit, 3354
Cited by
Sorted by citations
Items 101–118 of 118
Weakly Supervised Monocular Fisheye Camera Distance Estimation with Segmentation Constraints
Zhang, Z., & Yang, X. (2025) · Electronics · doi:10.3390/electronics14173429
Vision Transformer-Based Monocular Depth Estimation for Fisheye Cameras
Zhang, Z., Jiang, Y., & Jin, L. (2025) · Chinese Control Conference (CCC) · doi:10.23919/ccc64809.2025.11179747
Feature-aligned Fisheye Object Detection Network for Autonomous Driving
Cao, H., Sun, D., Song, R., et al. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246605
“Building upon this dataset, FisheyeYOLO [17] and OmniDet [18] have been proposed for fisheye object detection and multi-task perception, respectively.”
InfluentialChen, K., Ma, J., He, H., Ma, C., Liu, Z., & Xiong, S. (2026) · IEEE Sensors Journal · doi:10.1109/jsen.2026.3652194
“Cameras capture high-resolution RGB imagery enabling rich semantic interpretation essential for object classification and scene understanding [3], yet suffer performance degradation under extreme lighting conditions or adverse weather, particularly with distant small objects due to low pixel…”
Research on Enhanced Autonomous Emergency Braking System for Intelligent Connected Vehicles
Liu, J. (2025) · IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC) · doi:10.1109/imcec66174.2025.11331696
“Autonomous researchers are advancing AEBS by fusing multi-sensor data[1–3], leveraging machine learning for predictive analytics[4, 5], enhancing sensor hardware capabilities[6, 7], and collaborating on regulatory standards to improve system robustness and reduce costs.”
Liu, W., Wang, W., & Meng, J. (2025) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc60802.2025.11423179
“OmniDet [9] presented a unified multi-task perception framework operating on raw fisheye images, enabling near-field surround environment perception such as depth estimation, object detection and segmentation.”
Segmentation of road scene omnidirectional images
Sekkat, A. R. (2022) · theses.fr (ABES) · doi:10.70675/6c5add14zaebez4287z800dzab48102c8f8b
Altamirano-Flores, Y. V., Toro-Guajardo, S., Gónzalez, I., et al. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-16992-1_8
Sparse4D: Sparse-Based End-to-End Multi-Sensor Temporal Perception
Lin, X., Pei, Z., Li, K., Lin, T., Huang, L., Yu, C., See, J., Su, Z., & Yang, C. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3688545
“LiDAR and vision-based models each have distinct strengths and weaknesses in the realm of autonomous driving [33], [34], [35],[36],[37],[38].”
Network-agnostic distortion-robust projections for wide-angle image understanding
Athwale, A., Ahmad, O., & Lalonde, J.-F. (2026) · IEEE Workshop/Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00065
“In the context of depth estimation for wide-angle images, some methods [40–42] leverage self-supervised learning combined with a multi-task learning strategy.”
Barakat, B., Islam, M. W., RASMY, M., & Hemayed, E. E. (2026) · Signal Image and Video Processing · doi:10.1007/s11760-026-05341-2
“Inter-task loss balancing uses the uncertainty weighting strategy of [24].”
Yang, F., Duan, X., Li, F., & Zhang, L. (2026) · Sensors · doi:10.3390/s26113533
“In parallel, datasets such as WoodScape incorporate soiling into the multi-task perception setting of autonomous driving, extending evaluation from standard forward-view images to fisheye surround-view scenarios that are more relevant to production systems [12,13].”
A dual-layer visual servoing scheme oriented to FOV constraints and trajectory smoothness
Ye, L., Hao, T. A. N. G., Jin, J., et al. (2026) · Hainan Daxue xuebao. Ziran kexue ban · doi:10.65658/j.hndk.2026033001
Yang, T., Qian, Y., Wang, C., & Yang, M. Z. (2026) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2026.3704370
Fisheye-aware multi-camera framework for automotive near-field pedestrian localization
Barakat, B., Islam, M., RASMY, M., et al. (2026) · Neural Computing and Applications · doi:10.1007/s00521-026-12199-5
“Mathematical notation is summarized in Table 1 .”
Reinforcement Learning Based Advanced Driver Assistance System for Unmarked Road Scenarios
Chandra, S., Nandi, A., Anand, V., et al. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-19521-0_5
Ge, X., Ren, Y., Jiang, H., et al. (2026) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2026.3733853
“Kumar et al. [23] developed a multi-task visual perception network based on fisheye imagery, enabling the joint learning of six core autonomous driving tasks within a unified model.”
SDFA-Net: Urban Manhole Cover Detection via Fisheye Image and LiDAR Point Cloud Fusion
Lan, Q., Hu, S., Li, J., Huang, Y., Li, Z., & Fan, Q. (2026) · Remote Sensing · doi:10.3390/rs18183206 · Cited by 1
Metadata
- Authors
- Ravi Kumar, V., Yogamani, S., Rashed, H., Sitsu, G., Witt, C., Leang, I., Milz, S., & Mader, P.
- Affiliations
- Valeo (Ireland) (IE)
- Valeo (Germany) (DE)
- Valeo (France) (FR)
- Technische Universität Ilmenau (DE)
- Venue
- IEEE Robotics and Automation Letters
- Published
- 2021-04
- Volume
- 6
- Issue
- 2
- Pages
- 2830-2837
- ISSN
- 2377-3766 (ISSN-L)
- 2377-3774
- OpenAlex
- W3129414859
- Semantic Scholar
- 58fce566cc744fac4851353ee3e6132e8fbe20e5
- Cited by
- 121
- References
- 72
- Citations by year
- 2026: 15 · 2025: 25 · 2024: 19 · 2023: 23 · 2022: 26 · 2021: 12
- FWCI
- 7.05
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Robotics and Automation Letters (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- surround-view cameras, fisheye images, multi-task learning, autonomous driving, depth estimation, visual odometry, semantic segmentation, motion segmentation, object detection, shared encoder, KITTI, cityscape
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
