SmartEye: Assisting Instant Photo Taking via Integrating User Preference with Deep View Proposal Network
The DOI matches its Crossref record; 63 of the 64 references it cites were found in a registry or an index.
Abstract
Instant photo taking and sharing has become one of the most popular forms of social networking. However, taking high-quality photos is difficult as it requires knowledge and skill in photography that most non-expert users lack. In this paper we present SmartEye, a novel mobile system to help users take photos with good compositions in-situ. The back-end of SmartEye integrates the View Proposal Network (VPN), a deep learning based model that outputs composition suggestions in real time, and a novel, interactively updated module (P-Module) that adjusts the VPN outputs to account for personalized composition preferences. We also design a novel interface with functions at the front-end to enable real-time and informative interactions for photo taking. We conduct two user studies to investigate SmartEye qualitatively and quantitatively. Results show that SmartEye effectively models and predicts personalized composition preferences, provides instant high-quality compositions in-situ, and outperforms the non-personalized systems significantly.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, in real-world decision-making, users' decision-making models can change as users interact with AI services and encounter more task instances [64], so a static model is not enough.”
From Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making (2023) · cites this paper as [64]
Citations
Cites
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 2×
“The VPN [52] is a real-time Convolutional Neural Network based on anchor boxes (pre-defined bounding boxes), inspired by recent successful object detection frameworks [29, 44] in computer vision.”
Method(2016) · Cited by 31,590
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1993.341010 · Cited by 14,265
A model of saliency-based visual attention for rapid scene analysis
Itti, L., Koch, C., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,271 · cited 1×
“Researchers have proposed various methods for auto-composition [11, 20, 26, 30, 33, 45, 57].”
Method2013 IEEE International Conference on Computer Vision
(2013) · doi:10.1109/iccv19984.2013 · Cited by 4,183
R-FCN: Object Detection via Region-based Fully Convolutional Networks
Dai, J., Li, Y., He, K., & Sun, J. Z. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.06409 · Cited by 3,149 · cited 1×
“Two recent works [15, 51] adopt fast object detection frameworks [12, 44] for image thumb-nailing and image cropping for efficient inference.”
MethodGlobal contrast based salient region detection
Cheng, M.-M., Zhang, G.-X., Mitra, N. J., Huang, X., & Hu, S.-M. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995344 · Cited by 2,976 · cited 2×
“The components of the 32D feature can be classified into the following categories: 1) geometry based (10D): the geometrical center (coordinates of the center-point), size in pixels, relative size to the original photo, aspect ratio and offset to left/right/top/bottom boundaries of the original photo; 2) saliency based…”
BackgroundMethodGlobal Contrast Based Salient Region Detection
Cheng, M.-M., Mitra, N. J., Huang, X., Torr, P. H. S., & Hu, S.-M. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2345401 · Cited by 2,463
Google news personalization: scalable online collaborative filtering
Das, A. S., Datar, M., Garg, A., & Rajaram, S. (2007) · Proceedings of the 16th international conference on World Wide Web · doi:10.1145/1242572.1242610 · Cited by 1,511
Convolutional Neural Networks for No-Reference Image Quality Assessment
Kang, L., Ye, P., Li, Y., & Doermann, D. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.224 · Cited by 1,270 · cited 1×
“Many deep learning methods have been proposed to find views with good compositions for various applications, such as image cropping [8, 9, 17, 21, 26, 30, 31, 54, 55], image thumbnail generation [15] and view recommendation [7, 10, 48].”
Background
Research works printed but not linked: 1
- 10.5555/944919.944965 · doi:10.5555/944919.944965
Cited by
Ma, S., Lei, Y., Wang, X., Zheng, C., Shi, C., Yin, M., & Ma, X. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581058 · Cited by 143
Self-citationA Review of Recent Deep Learning Approaches in Human-Centered Machine Learning
Kaluarachchi, T., Reis, A., & Nanayakkara, S. (2021) · Sensors · doi:10.3390/s21072514 · Cited by 124
“Considering the works that focused on the user side, some researchers catered to general end-users or consumers [83,101,200,210], while others on specific end-users.”
BackgroundMa, S., Wang, X., Lei, Y., Shi, C., Yin, M., & Ma, X. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642671 · Cited by 54
“The field of AI-assisted decision-making is gaining significant attention within HCI communities [6, 47, 49, 70, 72, 78, 79, 85, 87, 89, 93].”
Self-citationCharting the Future of AI in Project-Based Learning: A Co-Design Exploration with Students
Zheng, C., Yuan, K., Guo, B., Hadi Mogavi, R., Peng, Z., Ma, S., & Ma, X. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642807 · Cited by 70
Self-citationCharting the Future of AI in Project-Based Learning: A Co-Design Exploration with Students
Zheng, C., Yuan, K., Guo, B., Mogavi, R. H., Peng, Z., Ma, S., & Ma, X. (2024) · arXiv · doi:10.48550/arxiv.2401.14915 · Cited by 70
Self-citation
Metadata
- Authors
- Ma, S., Wei, Z., Tian, F., Fan, X., Zhang, J., Shen, X., Lin, Z., Huang, J., Měch, R., Samaras, D., & Wang, H.
- Affiliations
- Chinese Academy of Sciences (CN)
- Stony Brook University (US)
- Adobe Systems (United States) (US)
- ByteDance AI Lab, Menlo Park, CA, USA
- ORCID
- Venue
- Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2019-05-02
- Pages
- 1-12
- OpenAlex
- W2941860851
- Cited by
- 24
- References
- 64
- Citations by year
- 2026: 3 · 2025: 2 · 2024: 5 · 2023: 5 · 2022: 3 · 2021: 4 · 2020: 2
- FWCI
- 0.73
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Keywords
- photographic composition, real-time image analysis, user interface design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: SmartEye: Assisting Instant Photo Taking via Integrating User Preference with Deep View Proposal Network. https://app.cochat.ai/papers/10.1145/3290605.3300701
- The paper
- Ma, S., Wei, Z., Tian, F., Fan, X., Zhang, J., Shen, X., Lin, Z., Huang, J., Měch, R., Samaras, D., & Wang, H. (2019). SmartEye: Assisting Instant Photo Taking via Integrating User Preference with Deep View Proposal Network. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290605.3300701
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.