A New Deep Learning-Based Food Recognition System for Dietary Assessment on An Edge Computing Service Infrastructure
The DOI matches its Crossref record; 53 of the 56 references it cites were found in a registry or an index.
Abstract
Literature has indicated that accurate dietary assessment is very important for assessing the effectiveness of weight loss interventions. However, most of the existing dietary assessment methods rely on memory. With the help of pervasive mobile devices and rich cloud services, it is now possible to develop new computer-aided food recognition system for accurate dietary assessment. However, enabling this future Internet of Things-based dietary assessment imposes several fundamental challenges on algorithm development and system design. In this paper, we set to address these issues from the following two aspects: (1) to develop novel deep learning-based visual food recognition algorithms to achieve the best-in-class recognition accuracy; (2) to design a food recognition system employing edge computing-based service computing paradigm to overcome some inherent problems of traditional mobile cloud computing paradigm, such as unacceptable system latency and low battery life of mobile devices. We have conducted extensive experiments with real-world data. Our results have shown that the proposed system achieved three objectives: (1) outperforming existing work in terms of food recognition accuracy; (2) reducing response time that is equivalent to the minimum of the existing approaches; and (3) lowering energy consumption which is close to the minimum of the state-of-the-art.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“At the end level, video capture devices, such as smartphones and surveillance cameras are responsible for video capture, media data compression [102], image preprocessing, and image segmentation [103].”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [103]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
(2014) · Cited by 33,419
Rapid object detection using a boosted cascade of simple features
Viola, P., & Jones, M. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990517 · Cited by 18,300
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 4×
“Some excellent examples include VGG net [38], ZFNet[39], GoogLeNet[40], Residual Network [41]).”
BackgroundInfluentialReceptive fields, binocular interaction and functional architecture in the cat’s visual cortex
Hubel, D. H., & Wiesel, T. N. (1962) · The Journal of Physiology · doi:10.1113/jphysiol.1962.sp006837 · Cited by 13,680 · cited 1×
“CNNs are biologicallyinspired [34] (animal visual cortex) variants of Multilayer”
BackgroundSpeeded-Up Robust Features (SURF)
Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L. (2008) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2007.09.014 · Cited by 13,557 · cited 1×
“The first one, entitled as C-System, employs different types of computer vision algorithms using hand engineered features (e.g., SIFT [19], SURF [54], HOG [55], Cas-cade [56]) running at the mobile device for food image recognition, without relying on any algorithms running in the server.”
BackgroundCaffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Edge computing: Vision and challenges
Shi, W., Cao, J., Zhang, Q., Li, Y., & Xu, L. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2579198 · Cited by 8,733
Prevalence of Childhood and Adult Obesity in the United States, 2011-2012
Ogden, C. L., Carroll, M. D., Kit, B. K., & Flegal, K. M. (2014) · JAMA · doi:10.1001/jama.2014.732 · Cited by 8,072
Learning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
The case for vm-based cloudlets in mobile computing
Satyanarayanan, M., Bahl, P., Caceres, R., & Davies, N. (2009) · IEEE Pervasive Computing · doi:10.1109/mprv.2009.82 · Cited by 3,660
Learning openCV: computer vision with the openCV library
Bradski, G. R., & Kaehler, A. (2008) · CERN Document Server (European Organization for Nuclear Research) · Cited by 2,468
ROSENTHALER, L. (2007) · The Focal Encyclopedia of Photography · doi:10.1016/b978-0-240-80740-9.50068-4 · Cited by 2,312
Food-101 – Mining Discriminative Components with Random Forests
Bossard, L., Guillaumin, M., & Van Gool, L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10599-4_29 · Cited by 2,072
Long-term weight loss maintenance
Wing, R. R., & Phelan, S. (2005) · The American Journal of Clinical Nutrition · doi:10.1093/ajcn/82.1.222s · Cited by 1,941 · cited 1×
“Estimating dietary intake accuratelywith a high-quality food journal is crucial for managing weight loss [27].”
BackgroundMXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
Chen, T., Li, M., Li, Y., Lin, M., Wang, N., Wang, M., Xiao, T., Xu, B., Zhang, C., & Zhang, Z. (2015) · arXiv · doi:10.48550/arxiv.1512.01274 · Cited by 1,819
Convolutional neural networks at constrained time cost
He, K., & Sun, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299173 · Cited by 1,558 · cited 1×
“layers, we resort to the lessons learned in recent paper [50],”
MethodDevelopment, validation and utilisation of food-frequency questionnaires – a review
Cade, J., Thompson, R., Burley, V., & Warm, D. (2002) · Public Health Nutrition · doi:10.1079/phn2001318 · Cited by 1,423
A Survey of Fog Computing: Concepts, Applications and Issues
Yi, S., Li, C., & Li, Q. (2015) · Proceedings of the 2015 Workshop on Mobile Big Data · doi:10.1145/2757384.2757397 · Cited by 1,339
Shi, W., & Dustdar, S. (2016) · Computer · doi:10.1109/mc.2016.145 · Cited by 1,241
Quantized Convolutional Neural Networks for Mobile Devices
Wu, J., Leng, C., Wang, Y., Hu, Q., & Cheng, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.521 · Cited by 1,212
Beaton, G., Milner, J., Corey, P., McGuire, V, Cousins, M., Stewart, E., de Ramos, M., Hewitt, D., Grambsch, P., Kassim, N., et al. (1979) · The American Journal of Clinical Nutrition · doi:10.1093/ajcn/32.12.2546 · Cited by 986
(2014) · arXiv (Cornell University) · Cited by 942
Edge-centric Computing: Vision and Challenges
Garcia Lopez, P., Montresor, A., Epema, D., Datta, A., Higashino, T., Iamnitchi, A., Barcellos, M., Felber, P., & Riviere, E. (2015) · ACM SIGCOMM Computer Communication Review · doi:10.1145/2831347.2831354 · Cited by 934 · cited 2×
“The last (but not the least) related research area is edge computing service infrastructure [22], [23], [24], [25], [26].”
BackgroundFog Computing: Platform and Applications
Yi, S., Hao, Z., Qin, Z., & Li, Q. (2015) · 2015 Third IEEE Workshop on Hot Topics in Web Systems and Technologies (HotWeb) · doi:10.1109/hotweb.2015.22 · Cited by 829 · cited 1×
“Edge computing is a non-trivial extension of cloud computing from the core network to the edge network [20], [21], [22], [23], [24], [25], [26].”
BackgroundIm2Calories: Towards an Automated Mobile Vision Food Diary
Myers, A., Johnston, N., Rathod, V., Korattikara, A., Gorban, A., Silberman, N., Guadarrama, S., Papandreou, G., Huang, J., & Murphy, K. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.146 · Cited by 485
The Use of Mobile Devices in Aiding Dietary Assessment and Evaluation
Zhu, F., Bosch, M., Woo, I., Kim, S., Boushey, C. J., Ebert, D. S., & Delp, E. J. (2010) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2010.2051471 · Cited by 355 · cited 3×
“To address this issue, visual-based food recognition algorithms and systems have been proposed [6], [7], [8], [9], [10], [11].”
BackgroundDeepFood: Deep Learning-Based Food Image Recognition for Computer-Aided Dietary Assessment
Liu, C., Cao, Y., Luo, Y., Chen, G., Vokkarane, V., & Ma, Y. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-39601-9_4 · Cited by 289
Self-citationFood recognition using statistics of pairwise local features
Yang, S., Chen, M., Pomerleau, D., & Sukthankar, R. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539907 · Cited by 263
Measuring food intake with digital photography
Martin, C. K., Nicklas, T., Gunturk, B., Correa, J. B., Allen, H. R., & Champagne, C. (2014) · Journal of Human Nutrition and Dietetics · doi:10.1111/jhn.12014 · Cited by 261
Food image recognition with deep convolutional features
Kawano, Y., & Yanai, K. (2014) · Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication · doi:10.1145/2638728.2641339 · Cited by 209
FAST: A fog computing assisted distributed analytics system to monitor fall for stroke mitigation
Cao, Y., Chen, S., Hou, P., & Brown, D. (2015) · 2015 IEEE International Conference on Networking, Architecture and Storage (NAS) · doi:10.1109/nas.2015.7255196 · Cited by 193
Self-citationFoodCam: A real-time food recognition system on a smartphone
Kawano, Y., & Yanai, K. (2015) · Multimedia Tools and Applications · doi:10.1007/s11042-014-2000-8 · Cited by 191
Smart Items, Fog and Cloud Computing as Enablers of Servitization in Healthcare
Stantchev, V., Barnawi, A., Ghulam, S., SCHUBERT, J., & Tamm, G. (2015) · DOAJ (DOAJ: Directory of Open Access Journals) · Cited by 180
Daugherty, B. L., Schap, T. E., Ettienne-Gittens, R., Zhu, F. M., Bosch, M., Delp, E. J., Ebert, D. S., Kerr, D. A., & Boushey, C. J. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1967 · Cited by 176 · cited 3×
“To address this issue, visual-based food recognition algorithms and systems have been proposed [6], [7], [8], [9], [10], [11].”
BackgroundCloudlets: at the Leading Edge of Mobile-Cloud Convergence
Satyanarayanan, M., Chen, Z., Ha, K., Hu, W., Richter, W., & Pillai, P. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257757 · Cited by 165 · cited 2×
“The last (but not the least) related research area is edge computing service infrastructure [22], [23], [24], [25], [26].”
BackgroundCao, Y., Liu, C., Liu, B., Brunette, M. J., Zhang, N., Sun, T., Zhang, P., Peinado, J., Garavito, E. S., Garcia, L. L., et al. (2016) · 2016 IEEE First International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE) · doi:10.1109/chase.2016.18 · Cited by 104
Self-citationThe Definitive Guide to Django
Holovaty, A., & Kaplan-Moss, J. (2009) · Apress eBooks · doi:10.1007/978-1-4302-1937-8 · Cited by 80
Image-based food volume estimation
Xu, C., He, Y., Khanna, N., Parra, A., Boushey, C. J., & Delp, E. J. (2013) · doi:10.1145/2506023.2506037 · Cited by 73
An Overview of the State of the Art of Automated Capture of Dietary Intake Information
Steele, R. (2015) · Critical Reviews in Food Science and Nutrition · doi:10.1080/10408398.2013.765828 · Cited by 54
Measuring Objective Quality of Colonoscopy
Oh, J., Hwang, S., Cao, Y., Tavanapong, W., Liu, D., Wong, J., & de Groen, P. C. (2008) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2008.2006035 · Cited by 51
Self-citationSatyanarayanan, M. (2013) · International ACM SIGSOFT Conference on Quality of Software Architectures (QoSA) · doi:10.1145/2465478.2465494 · Cited by 43
Research works printed but not linked: 4
- goodfellow, 2016, Deep Learning
- 10.1109/CVPR.2005.177 · DOI · Paper page
- matsuda, 0, Multiple-food recognition considering co-occurrence employing manifold ranking, Proc 21st Int Conf Pattern Recognit, 2017
- 10.1109/5.726791 · DOI · Paper page
Cited by
Sarker, I. H. (2021) · SN Computer Science · doi:10.1007/s42979-021-00815-1 · Cited by 2,678
Learning IoT in Edge: Deep Learning for the Internet of Things with Edge Computing
Li, H., Ota, K., & Dong, M. (2018) · IEEE Network · doi:10.1109/mnet.2018.1700202 · Cited by 1,566
“Liu et al. [15] proposed the first work that introduces deep learning into the edge computing environment.”
BackgroundDeep Learning With Edge Computing: A Review
Chen, J., & Ran, X. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2921977 · Cited by 1,540
“[92] built a food recognition system with two preprocessing steps: first, they discard blurry images, and second, they crop the image so that it only contains the objects of interest.”
BackgroundConvergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
Deep Learning for IoT Big Data and Streaming Analytics: A Survey
Mohammadi, M., Al-Fuqaha, A., Sorour, S., & Guizani, M. (2018) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2844341 · Cited by 1,471
Metadata
- Authors
- Liu, C., Cao, Y., Luo, Y., Chen, G., Vokkarane, V., Yunsheng, M., Chen, S., & Hou, P.
- Affiliations
- University of Massachusetts Lowell (US)
- George Mason University (US)
- Venue
- IEEE Transactions on Services Computing
- Published
- 2018-03-01
- Volume
- 11
- Issue
- 2
- Pages
- 249-261
- ISSN
- 1939-1374 (ISSN-L)
- OpenAlex
- W2585425668
- Cited by
- 333
- References
- 56
- Citations by year
- 2026: 8 · 2025: 22 · 2024: 35 · 2023: 46 · 2022: 51 · 2021: 59 · 2020: 58 · 2019: 32 · 2018: 19
- FWCI
- 20.40
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Services Computing (publishedVersion)
- The Journal of the American Medical Association (JAMA) Network (American Medical Association) (submittedVersion)
- Funders
- US National Science Foundation (NSF) of the United States (1547428, 1541434, 1524462, 1440737, 1229213)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Health Sciences › Health Professions › Health Information Management › Innovation in Digital Healthcare Systems
- Keywords
- food recognition, dietary assessment, deep learning, edge computing, mobile cloud computing, Internet of Things, weight loss interventions, system latency, energy consumption, mobile devices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.