The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Towards Emotionally Aware AI Smart Classroom: Current Issues and Directions for Engineering and Education
Open accessAbstract
Future smart classrooms that we envision will significantly enhance learning experience and seamless communication among students and teachers using real-time sensing and machine intelligence. Existing developments in engineering have brought the state-of-the-art to an inflection point, where they can be utilized as components of a smart classroom. In this paper, we propose a smart classroom system that consists of these components. Our proposed system is capable of making real-time suggestions to an in-class presenter to improve the quality and memorability of their presentation by allowing the presenter to make real-time adjustments/corrections to their non-verbal behavior, such as hand gestures, facial expressions, and body language. We base our suggested system components on existing research in affect sensing, deep learning-based emotion recognition, and real-time mobile-cloud computing. We provide a comprehensive study of these technologies and determine the computational requirements of a system that incorporates these technologies. Based on these requirements, we provide a feasibility study of the system. Although the state-of-the-art research in most of the components we propose in our system are advanced enough to realize the system, the main challenge lies in: 1) the integration of these technologies into a holistic system design; 2) their algorithmic adaptation to allow real-time execution; and 3) quantification of valid educational variables for use in algorithms. In this paper, we discuss current issues and provide future directions in engineering and education disciplines to deploy the proposed system.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
An argument for basic emotions
Ekman, P. (1992) · Cognition and Emotion · doi:10.1080/02699939208411068 · Cited by 9,646 · cited 1×
“This modeling is based upon ‘‘basic’’ emotions that are universally recognizable, proposed by Ekman [94].”
MethodLearning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Public-Key Cryptosystems Based on Composite Degree Residuosity Classes
Paillier, P. (2007) · Lecture Notes in Computer Science · doi:10.1007/3-540-48910-x_16 · Cited by 7,250
The Cognitive Structure of Emotions
Ortony, A., Clore, G. L., & Collins, A. (1988) · doi:10.1017/cbo9780511571299 · Cited by 6,208
Pekrun, R. (2006) · Educational Psychology Review · doi:10.1007/s10648-006-9029-9 · Cited by 5,566
Greedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Validity of the single processor approach to achieving large scale computing capabilities
Amdahl, G. M. (1967) · Proceedings of the April 18-20, 1967, spring joint computer conference on - AFIPS '67 (Spring) · doi:10.1145/1465482.1465560 · Cited by 4,339
Nine Ways to Reduce Cognitive Load in Multimedia Learning
Mayer, R. E., & Moreno, R. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_6 · Cited by 4,207
Cognitive load theory, learning difficulty, and instructional design
Sweller, J. (1994) · Learning and Instruction · doi:10.1016/0959-4752(94)90003-5 · Cited by 3,801 · cited 1×
“, task difficulty, time available), contribute to differing levels of cognitive load for each individual while performing a task [167].”
BackgroundZimmerman, B. J. (2008) · American Educational Research Journal · doi:10.3102/0002831207312909 · Cited by 3,735
Opensmile: the munich versatile and fast open-source audio feature extractor
Eyben, F., Wöllmer, M., & Schuller, B. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874246 · Cited by 2,642 · cited 3×
“This methodology is suitable for the computations in our system [16]–[20], because the added memory requirement is an acceptable tradeoff and the intensive offline computations can be performed at a school datacenter at night, which is when the computational resources are nearly idle.”
BackgroundMethodLIBET, B., GLEASON, C. A., WRIGHT, E. W., & PEARL, D. K. (1983) · Brain · doi:10.1093/brain/106.3.623 · Cited by 2,498
MAUI: making smartphones last longer with code offload
Cuervo, E., Balasubramanian, A., Cho, D.-K., Wolman, A., Saroiu, S., Chandra, R., & Bahl, P. (2010) · Proceedings of the 8th international conference on Mobile systems, applications, and services · doi:10.1145/1814433.1814441 · Cited by 2,281
Cognitive Load Theory and Complex Learning: Recent Developments and Future Directions
van Merriënboer, J. J. G., & Sweller, J. (2005) · Educational Psychology Review · doi:10.1007/s10648-005-3951-0 · Cited by 2,206
Task-evoked pupillary responses, processing load, and the structure of processing resources.
Beatty, J. (1982) · Psychological Bulletin · doi:10.1037/0033-2909.91.2.276 · Cited by 1,959
Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
Calvo, R. A., & D'Mello, S. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.1 · Cited by 1,893 · cited 1×
“Surveys that study this topic in detail can be found in [92], [97], [98], and [105]–[109].”
Background(2011) · doi:10.1145/1966445.1966473 · Cited by 1,870
Pupil Diameter and Load on Memory
Kahneman, D., & Beatty, J. (1966) · Science · doi:10.1126/science.154.3756.1583 · Cited by 1,617
Amdahl's Law in the Multicore Era
Hill, M. D., & Marty, M. R. (2008) · Computer · doi:10.1109/mc.2008.209 · Cited by 1,262
Advanced Encryption Standard (AES)
Heron, S. (2009) · Network Security · doi:10.1016/s1353-4858(10)70006-4 · Cited by 1,216
Development of an instrument for measuring different types of cognitive load
Leppink, J., Paas, F., Van der Vleuten, C. P. M., Van Gog, T., & Van Merriënboer, J. J. G. (2013) · Behavior Research Methods · doi:10.3758/s13428-013-0334-1 · Cited by 1,066 · cited 1×
“Different methods have been proposed in the literature for measuring cognitive load, such as various rating scales [76] and data from pupillary response [77].”
MethodPaas, F., Renkl, A., & Sweller, J. (2004) · Instructional Science · doi:10.1023/b:truc.0000021806.17516.d0 · Cited by 1,054
Toward detecting emotions in spoken dialogs
Lee, C. M., & Narayanan, S. S. (2005) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2004.838534 · Cited by 988
Pupil size variation as an indication of affective processing
Partala, T., & Surakka, V. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00017-x · Cited by 836 · cited 1×
“Pupil dilation is associated with hedonic valence and emotional arousal as well [84]–[86], where higher arousal or intense emotions (positive or negative) lead to larger mean pupil diameter than neutral emotional state [86].”
BackgroundCloud-Vision: Real-time face recognition using a mobile-cloudlet-cloud acceleration architecture
Soyata, T., Muraleedharan, R., Funai, C., Kwon, M., & Heinzelman, W. (2012) · 2012 IEEE Symposium on Computers and Communications (ISCC) · doi:10.1109/iscc.2012.6249269 · Cited by 420
Self-citationDeep learning for robust feature generation in audiovisual emotion recognition
Kim, Y., Lee, H., & Provost, E. M. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638346 · Cited by 401
Self-citationSparse Autoencoder-Based Feature Transfer Learning for Speech Emotion Recognition
Deng, J., Zhang, Z., Marchi, E., & Schuller, B. (2013) · 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction · doi:10.1109/acii.2013.90 · Cited by 366
Grandjean, D., Sander, D., & Scherer, K. R. (2008) · Consciousness and Cognition · doi:10.1016/j.concog.2008.03.019 · Cited by 302 · cited 1×
“In this section, we discuss categorical and dimensional [92], [93] affective labeling approaches that can quantify emotion of given data.”
BackgroundMemory load and the cognitive pupillary response in aging
Van Gerven, P. W. M., Paas, F., Van Merriënboer, J. J. G., & Schmidt, H. G. (2004) · Psychophysiology · doi:10.1111/j.1469-8986.2003.00148.x · Cited by 292 · cited 1×
“Because pupils dilate while cognitive processing and brain activation increases [77], [79]–[83], it can be used to compute the cognitive load ([C] vector).”
BackgroundDeep neural networks for acoustic emotion recognition: Raising the benchmarks
Stuhlsatz, A., Meyer, C., Eyben, F., Zielke, T., Meier, G., & Schuller, B. (2011) · 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2011.5947651 · Cited by 288 · cited 1×
“[125] also used a Generalized Discriminant Analysis (GerDA)-based deep learning models for speech emotion recognition.”
MethodKlingner, J., Tversky, B., & Hanrahan, P. (2011) · Psychophysiology · doi:10.1111/j.1469-8986.2010.01069.x · Cited by 255
Machine Analysis of Facial Expressions
Pantic, M., & Stewart, M. (2007) · Face Recognition · doi:10.5772/4847 · Cited by 248
The INTERSPEECH 2012 speaker trait challenge
Schuller, B., Steidl, S., Batliner, A., Nöth, E., Vinciarelli, A., Burkhardt, F., Son, R. V., Weninger, F., Eyben, F., Bocklet, T., et al. (2012) · Interspeech 2012 · doi:10.21437/interspeech.2012-86 · Cited by 211
Emotion Recognition Based on Joint Visual and Audio Cues
Sebe, N., Cohen, I., Gevers, T., & Huang, T. S. (2006) · 18th International Conference on Pattern Recognition (ICPR'06) · doi:10.1109/icpr.2006.489 · Cited by 202
Hovardas, T., Tsivitanidou, O. E., & Zacharia, Z. C. (2014) · Computers & Education · doi:10.1016/j.compedu.2013.09.019 · Cited by 176
Large-Scale Distributed Dedicated- and Non-Dedicated Smart City Sensing Systems
Habibzadeh, H., Qin, Z., Soyata, T., & Kantarci, B. (2017) · IEEE Sensors Journal · doi:10.1109/jsen.2017.2725638 · Cited by 114
Self-citationHabibzadeh, H., Boggio-Dandry, A., Qin, Z., Soyata, T., Kantarci, B., & Mouftah, H. T. (2018) · IEEE Communications Magazine · doi:10.1109/mcom.2018.1700304 · Cited by 64
Self-citation
Cited by
Artificial Intelligence in Education: A Review
Chen, L., Chen, P., & Lin, Z. (2020) · IEEE Access · doi:10.1109/access.2020.2988510 · Cited by 4,133
“Ideally, using personalized learning, students choose what they’re interested in, and instructors adjust teaching course and method to the students’ interests [43].”
BackgroundA review on artificial intelligence in education
Huang, J., Saleh, S., & Liu, Y. (2021) · Academic Journal of Interdisciplinary Studies · doi:10.36941/ajis-2021-0077 · Cited by 439
Adoption of artificial intelligence in smart cities: A comprehensive review
Herath, H. M. K. K. M. B., & Mittal, M. (2022) · International Journal of Information Management Data Insights · doi:10.1016/j.jjimei.2022.100076 · Cited by 354
Habibzadeh, H., Nussbaum, B. H., Anjomshoa, F., Kantarci, B., & Soyata, T. (2019) · Sustainable Cities and Society · doi:10.1016/j.scs.2019.101660 · Cited by 303
Self-citationShaik, T., Tao, X., Li, Y., Dann, C., McDonald, J., Redmond, P., & Galligan, L. (2022) · IEEE Access · doi:10.1109/access.2022.3177752 · Cited by 286
Metadata
- Authors
- Kim, Y., Soyata, T., & Behnagh, R. F.
- Venue
- IEEE Access
- Published
- 2018
- Volume
- 6
- Pages
- 5308-5331
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2784169908
- Cited by
- 213
- References
- 38
- FWCI
- 10.04
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.