The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index; 15 of the 35 sources its claims cite were checked for support.
Unified Perceptual Parsing for Scene Understanding
Open accessAbstract
Humans recognize the visual world at multiple levels: we effortlessly categorize scenes and detect objects inside, while also identifying the textures and surfaces of the objects along with their different compositional parts. In this paper, we study a new task called Unified Perceptual Parsing, which requires the machine vision systems to recognize as many visual concepts as possible from a given image. A multi-task framework called UPerNet and a training strategy are developed to learn from heterogeneous image annotations. We benchmark our framework on Unified Perceptual Parsing and show that it is able to effectively segment a wide range of concepts from images. The trained networks are further applied to discover visual knowledge in natural scenes (Models are available at https://github.com/CSAILVision/unifiedparsing).
Overview
CoChat Summary
TL;DR
The paper introduces Unified Perceptual Parsing (UPP), a task requiring a model to recognize scene, object, part, material, and texture concepts from an image, and proposes UPerNet, a multi-task FPN-based network trained on heterogeneous annotations. UPerNet jointly parses these levels on Broden+ and is used to extract compositional visual knowledge, with competitive ADE20K segmentation and modest texture accuracy.
Key findings
- On ADE20K object segmentation, UPerNet’s FPN+PPM+Fusion (/4) reaches 41.22 mIoU, 79.98 pixel accuracy, and 60.60 overall, close to PSPNet’s 41.68/80.04/60.86 while using 38.7 vs. 61.1 training hours; FPN (/16) reaches 34.46/76.04 in 18.1 hours.
- On Broden+ UPP, the object baseline yields 24.72 mIoU/78.03 P.A. and the material baseline 52.78/84.32; joint object+part training yields 23.92/77.48 on object and 30.21/48.30 on part, and adding scene prediction gives 71.35% scene top-1 accuracy with negligible downgrades.
- Joint material+object+part+scene training yields 54.19/84.45 on material, 23.36/77.09 on object, and 28.75/46.92 on part; after fine-tuning, texture classification reaches 35.10 top-1 accuracy.
- The trained model discovers relations such as toilet made of ceramic (65%) and plastic (35%), microwave made of glass (55%) and metal (45%), coffee table top made of wood (69%) and glass (31%), bed headboard made of wood (77%) and fabric (23%), and tv monitor screen made of glass (100%).
Methods
- UPP is evaluated on Broden+, a standardized heterogeneous dataset combining ADE20K, Pascal-Context, Pascal-Part, OpenSurfaces, and DTD; it contains 57,095 images (51,617 train / 5,478 validation) with 365 scene, 335 object, 77 object-with-part, 152 part, 26 material, and 47 texture classes. Metrics are top-1 accuracy for scene/texture, mIoU and pixel accuracy for object/material, and mIoU-bg and pixel accuracy for part.
- UPerNet uses a ResNet backbone with FPN and a Pyramid Pooling Module; scene head is on P5 after PPM, object/part heads on fused FPN features, material head on P2, and texture head on C2 with blocked gradients and fine-tuning after other tasks. Training randomly samples a data source per iteration and updates only the related path, using synchronized SGD across 8 GPUs with 2 images per GPU.
- ADE20K is used for segmentation comparisons; Broden+ is used for UPP; visual knowledge is extracted using the Places-365 validation set of 36,500 images from 365 scenes.
Limitations
- The paper states texture performance is only 35.10 top-1 and that fine-tuning on texture labels alone is not optimal, though necessary to avoid harmful fusion of natural and synthetic data sources; it hopes future work better uses image-level annotations for pixel-level predictions.
- The paper states object and part parsing suffer slight performance degradation due to heterogeneity when jointly trained with material, object, part, and scene, while material improves.
- Evident from the text: there is no single dataset annotated with all visual levels; Broden+ is standardized from heterogeneous sources, with scene and texture labels at image level and material only partially annotated, which constrains pixel-level texture/material learning and evaluation.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
12 Valid · 7 Not valid · 7 Not checked · 49 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“we find that samples from different classes are unbalanced.”
What the sources say
- Supportedcites [26] Bau, D., Zhou, B., Khosla, A., Oliva, A., Torralba, A.: Network dissection: quantifying interpretability of deep visual representations. In: Proceedings of CVPR (2017) source
Section 2.1 (p. 2), Table 1 reports markedly different average numbers of image samples per label class across Broden's categories, from 38 for scenes to 59,250 for colors. This supports the statement that samples are unbalanced across classes, although the table gives category averages rather than counts for every individual class. (LLM, conf=0.88)
- Not checked — no full text of the sourcecites [29] Zhou, B., Bau, D., Oliva, A., Torralba, A.: Interpreting deep visual representations via network dissection. In: IEEE Trans. Pattern Anal. Mach. Intell. (2018) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Broden unifies several densely labeled image datasets, namely ADE20K [2], Pascal-Context [27], Pascal-Part [28], OpenSurfaces [6], and the Describable Textures Dataset (DTD) [4].”
What the sources say
- Not checked — no full text of the sourcecites [2] Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., Torralba, A.: Scene parsing through ADE20K dataset. In: Proceedings of CVPR (2017) source
- Not checked — no full text of the sourcecites [27] Mottaghi, R., et al.: The role of context for object detection and semantic segmentation in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014) source
- Supportedcites [28] Chen, X., Mottaghi, R., Liu, X., Fidler, S., Urtasun, R., Yuille, A.: Detect what you can: detecting and representing objects using holistic models and body parts. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014) source
The citation is attached to Pascal-Part, not to the entire list of datasets or the statement about Broden. The Experimental Evaluation (p. 6) says the authors augmented PASCAL VOC 2010 with detailed body-part masks and bounding boxes; the Conclusion (p. 7) identifies annotations for six animal categories. This supports citing the paper for the Pascal-Part dataset. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Various image datasets are constructed only for specific task, such as ADE20K for scene parsing [2], the Describe Texture Dataset (DTD) for texture recognition [4], and OpenSurfaces for material and surface recognition [6].”
What the sources say
- Not checked — no full text of the sourcecites [2] Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., Torralba, A.: Scene parsing through ADE20K dataset. In: Proceedings of CVPR (2017) source
- Supportedcites [4] Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., Vedaldi, A.: Describing textures in the wild. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3606–3613. IEEE (2014) source
The Abstract identifies this work as introducing the Describable Textures Dataset (DTD) for recognizing texture attributes in images. The Introduction likewise describes its texture images and recognition objective, supporting the DTD example to which this citation is attached. (LLM, conf=0.91)
- Not checked — no full text of the sourcecites [6] Bell, S., Upchurch, P., Snavely, N., Bala, K.: OpenSurfaces: a richly annotated catalog of surface appearance. ACM Trans. Graph. (TOG) 32(4), 111 (2013) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“since it is originally collected to discover the alignment between visual concepts and hidden units of Convolutional Neural Networks (CNNs) for network interpretability [26,29]”
What the sources say
- Supportedcites [26] Bau, D., Zhou, B., Khosla, A., Oliva, A., Torralba, A.: Network dissection: quantifying interpretability of deep visual representations. In: Proceedings of CVPR (2017) source
Section 2.1 (p. 2) says the authors assembled Broden to ascertain alignment between visual concepts and CNN units. Section 2 (p. 2) explains that this alignment is measured to assess CNN interpretability. Broden unified existing labeled datasets rather than collecting all its images anew. (LLM, conf=0.94)
- Not checked — no full text of the sourcecites [29] Zhou, B., Bau, D., Oliva, A., Torralba, A.: Interpreting deep visual representations via network dissection. In: IEEE Trans. Pattern Anal. Mach. Intell. (2018) source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“First, there is no single image dataset annotated with all levels of visual information.”
What the sources say
- Not checked — no full text of the sourcecites [2] Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., Torralba, A.: Scene parsing through ADE20K dataset. In: Proceedings of CVPR (2017) source
- Could not verifycites [4] Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., Vedaldi, A.: Describing textures in the wild. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3606–3613. IEEE (2014) sourceExperimental — accuracy not yet measured
The Introduction describes a dataset of texture images annotated with 47 texture attributes, and the Related work section discusses other texture and material datasets. Neither passage says that no image dataset is annotated at all levels of visual information. Searches for that assertion and for datasets annotated at multiple visual levels found no supporting passage in our copy; page numbers were not provided. (LLM, conf=0.72)
- Not checked — no full text of the sourcecites [6] Bell, S., Upchurch, P., Snavely, N., Bala, K.: OpenSurfaces: a richly annotated catalog of surface appearance. ACM Trans. Graph. (TOG) 32(4), 111 (2013) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
34 more claims · 1 with their cited sources checked
Create a free account to see all 75 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 23
- Probably not a substantial claim. 21
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“We used an UPerNet decoder [23] and ran hyperparameter tuning with TerraTorch-iterate for all models (ten trials, tuning weight decay and learning rate). The training with the best validation hyperparameters was repeated ten times with different seeds, and the results are reported on the test set.”
From Prithvi-EO-2.0: A Versatile Multitemporal Foundation Model for Earth Observation Applications (2026) · cites this paper as [23]
The cited source says what the claim attributes to it.
In the citing paper
“For all models, we used an UPerNet [23] decoder and tuned hyperparameters with TerraTorch-iterate (ten trials, tuning weight decay and learning rate). Then, we repeated the training with the best validation hyperparameters ten times with different seeds and reported results on the test set.”
From Prithvi-EO-2.0: A Versatile Multitemporal Foundation Model for Earth Observation Applications (2026) · cites this paper as [23]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 5×
“-sofais made of fabric. classification [1] and scene recognition [2]; objects and stuff are parsed and segmented precisely at pixel-level [3,2]; Texture and material perception and recognition have been studied in [4] and [5].”
BackgroundMethodInfluentialSelf-citationVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 2×
“Due to a lack of sufficient training samples, segmentation networks are usually initialized from networks pre-trained for image classification [37,7,8].”
BackgroundMethodImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2×
“The layers in the backbone network are initialized with weights pre-trained on ImageNet [37].”
BackgroundMethodGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · cited 1×
“To generate pixel-wise semantic predictions for a given image, image classification networks [7,8,9,1] are extended to generate semantic segmentation masks.”
MethodFully convolutional networks for semantic segmentationPreprint
Long, J., Shelhamer, E., & Darrell, T. (2014) · Computer Vision and Pattern Recognition · doi:10.48550/arxiv.1605.06211 · Cited by 43,735
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · cited 3×
“The most prevalent framework designed for this pixel-level classification task is the Fully Convolutional Network (FCN) [11], which replaces fully-connected layers in classification networks with convolutional layers.”
BackgroundMethodInfluentialFeature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · cited 3×
“[31] with a Pyramid Pooling Module (PPM) [16] appended on the last layer of the back-bone network before feeding it into the top-down branch in FPN. Top-right: We use features at various semantic levels.”
MethodBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 1×
“Unified Perceptual Parsing for Scene Understanding 9 FPN, have batch normalization [34] with 512-channel output.”
MethodChen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392 · cited 1×
“Same as [36], we use the “poly” learning rate policy where the learning rate at current iteration equals the initial learning rate multiplying 1 − iter max iter power .”
Method
Cited by
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01167 · Cited by 8,494
“ADE20K validation results using UperNet [85].”
MethodMasked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489
Deep High-Resolution Representation Learning for Visual Recognition
Wang, J., Sun, K., Cheng, T., Jiang, B., Deng, C., Zhao, Y., Liu, D., Mu, Y., Tan, M., Wang, X., et al. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2983686 · Cited by 4,894
“…skip connections [189] or exchanging information between them [49]; studying the details of the up-sample process [147]; combining multi-scale pyramid representations [22], [154]; stacking multiple DeconvNets/U-Nets/Hourglass [44], [149] with dense connections [135]. conv. unit . upsample Fig.”
BackgroundImage segmentation using deep learning: A survey
Minaee, S., Boykov, Y. Y., Porikli, F., Plaza, A. J., Kehtarnavaz, N., & Terzopoulos, D. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3059968 · Cited by 3,796
“Network (UperNet) [136], segmentation by re-training and self-training [137], densely connected neural architecture search [138], hierarchical multiscale attention [139], Efficient RGB-D Semantic Segmentation (ESA-Net) [140], Iterative Pyramid Contexts [141], and Learning Dynamic Routing for Semantic Segmentation [142…”
Method
Metadata
- Authors
- Xiao, T., Liu, Y., Zhou, B., Jiang, Y., & Sun, J.
- Affiliations
- Peking University (CN)
- MIT Computer Science and Artificial Intelligence Laboratory (US)
- ByteDance (China) (CN)
- Megvii (China) (CN)
- Corresponding author
- Xiao, Tete
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer International Publishing
- Published
- 2018
- Pages
- 432-448
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W2884822772
- Cited by
- 2,201
- References
- 48
- Citations by year
- 2026: 266 · 2025: 509 · 2024: 527 · 2023: 427 · 2022: 200 · 2021: 135 · 2020: 86 · 2019: 45 · 2018: 5
- FWCI
- 20.42
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Lecture notes in computer science (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- scene understanding, UPerNet, multi-task learning, object detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Unified Perceptual Parsing for Scene Understanding. https://app.cochat.ai/papers/10.1007/978-3-030-01228-1_26
- The paper
- Xiao, T., Liu, Y., Zhou, B., Jiang, Y., & Sun, J. (2018). Unified Perceptual Parsing for Scene Understanding. Lecture Notes in Computer Science. https://doi.org/10.1007/978-3-030-01228-1_26
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.