Activity-centric scene synthesis for functional 3D scene modeling
The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
We present a novel method to generate 3D scenes that allow the same activities as real environments captured through noisy and incomplete 3D scans. As robust object detection and instance retrieval from low-quality depth data is challenging, our algorithm aims to model semantically-correct rather than geometrically-accurate object arrangements. Our core contribution is a new scene synthesis technique which, conditioned on a coarse geometric scene representation, models functionally similar scenes using prior knowledge learned from a scene database. The key insight underlying our scene synthesis approach is that many real-world environments are structured to facilitate specific human activities, such as sleeping or eating. We represent scene functionalities through virtual agents that associate object arrangements with the activities for which they are typically used. When modeling a scene, we first identify the activities supported by a scanned environment. We then determine semantically-plausible arrangements of virtual objects -- retrieved from a shape database -- constrained by the observed scene geometry. For a given 3D scan, our algorithm produces a variety of synthesized scenes which support the activities of the captured real environments. In a perceptual evaluation study, we demonstrate that our results are judged to be visually appealing and functionally comparable to manually designed scenes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Context based object categorization: A critical survey
Galleguillos, C., & Belongie, S. (2010) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2010.02.004 · Cited by 351
Shape2Pose: human-centric shape analysis
Kim, V. G., Chaudhuri, S., Guibas, L., & Funkhouser, T. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601117 · Cited by 112
Cited by
Resolving 3D Human Pose Ambiguities With 3D Scene Constraints
Hassan, M., Choutas, V., Tzionas, D., & Black, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00237 · Cited by 274
Graph2Plan: learning floorplan generation from layout graphs
Hu, R., Huang, Z., Tang, Y., Van Kaick, O., Zhang, H., & Huang, H. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392391 · Cited by 266
Language-driven synthesis of 3D scenes from scene databases
Ma, R., Patil, A. G., Fisher, M., Li, M., Pirk, S., Hua, B.-S., Yeung, S.-K., Tong, X., Guibas, L., & Zhang, H. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275035 · Cited by 100
Self-citationProcedural Generation of Narrative Worlds
Balint, J. T., & Bidarra, R. (2023) · IEEE Transactions on Games · doi:10.1109/tg.2022.3216582 · Cited by 11
Zhao, Y., Zhao, Z., Li, J., Dong, S., & Gao, S. (2024) · 2024 International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00132 · Cited by 10
“Conventional scene synthesis methods [3, 5, 12, 13, 16, 29, 56] rely on predefined rules as constraints to guide the optimization process to achieve a reasonable synthesis result, but the de-sign of rules highly depends on the knowledge from experts, which are not scalable.”
Method
Metadata
- Authors
- Fisher, M., Savva, M., Li, Y., Hanrahan, P., & Nießner, M.
- Venue
- ACM Transactions on Graphics
- Published
- 2015-11-02
- Volume
- 34
- Issue
- 6
- Pages
- 1-13
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W2060268527
- Cited by
- 122
- References
- 3
- FWCI
- 10.05
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.