The DOI matches its Crossref record; 173 of the 173 references it cites were found in a registry or an index.
Behavioral Signal Processing: Deriving Human Behavioral Informatics From Speech and Language
Open accessAbstract
The expression and experience of human behavior are complex and multimodal and characterized by individual and contextual heterogeneity and variability. Speech and spoken language communication cues offer an important means for measuring and modeling human behavior. Observational research and practice across a variety of domains from commerce to healthcare rely on speech- and language-based informatics for crucial assessment and diagnostic information and for planning and tracking response to an intervention. In this paper, we describe some of the opportunities as well as emerging methodologies and applications of human behavioral signal processing (BSP) technology and algorithms for quantitatively understanding and modeling typical, atypical, and distressed human behavior with a specific focus on speech- and language-based communicative, affective, and social behavior. We describe the three important BSP components of acquiring behavioral data in an ecologically valid manner across laboratory to real-world settings, extracting and analyzing behavioral cues from measured data, and developing models offering predictive and decision-making support. We highlight both the foundational speech and language processing building blocks as well as the novel processing and modeling opportunities. Using examples drawn from specific real-world applications ranging from literacy assessment and autism diagnostics to psychotherapy for addiction and marital well being, we illustrate behavioral informatics applications of these signal processing techniques that contribute to quantifying higher level, often subjectively described, human behavior in a domain-sensitive fashion.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A simplest systematics for the organization of turn-taking for conversation
Sacks, H., Schegloff, E. A., & Jefferson, G. (1974) · Language · doi:10.2307/412243 · Cited by 11,350
Libsvm : A library for support vector machines
Ferrari, V. (2008) · Medical Entomology and Zoology · Cited by 10,123
The Content Analysis Guidebook
Neuendorf, K. A. (2017) · doi:10.4135/9781071802878 · Cited by 9,286 · cited 1×
“As Neuendorf [18] states ''without the establishment of reliability, content analysis measures are useless.''”
BackgroundWeighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733
Motivational interviewing: preparing people for change.
Miller, W. R., & Rollnick, S. (2002) · Guilford Press eBooks · Cited by 7,770
Lord, C., Risi, S., Lambrecht, L., Cook, E. H., Leventhal, B. L., DiLavore, P. C., Pickles, A., & Rutter, M. (2000) · Journal of Autism and Developmental Disorders · doi:10.1023/a:1005592401947 · Cited by 7,520
Ecological momentary assessment
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.3.022806.091415 · Cited by 6,454
Content Analysis for the Social Sciences and Humanities.
Parker, E. B., & Holsti, O. R. (1970) · American Sociological Review · doi:10.2307/2093233 · Cited by 4,564
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
Self-citationPerception and the Representative Design of Psychological Experiments
Brunswik, E. (1956) · doi:10.1525/9780520350519 · Cited by 2,974
Dawson, G., Rogers, S., Munson, J., Smith, M., Winter, J., Greenson, J., Donaldson, A., & Varley, J. (2010) · Pediatrics · doi:10.1542/peds.2009-0958 · Cited by 2,835 · cited 1×
“and communication skills in autistic children [188].”
Opensmile: the munich versatile and fast open-source audio feature extractor
Eyben, F., Wöllmer, M., & Schuller, B. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874246 · Cited by 2,642
Emotion recognition in human-computer interaction
Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., & Taylor, J. G. (2001) · IEEE Signal Processing Magazine · doi:10.1109/79.911197 · Cited by 2,603
Vocal communication of emotion: A review of research paradigms
Scherer, K. (2003) · Speech Communication · doi:10.1016/s0167-6393(02)00084-5 · Cited by 2,044
Reliability of Content Analysis: The Case of Nominal Scale Coding
Scott, W. A. (1955) · Public Opinion Quarterly · doi:10.1086/266577 · Cited by 1,853
(1999) · Cited by 1,435
The Autism Diagnostic Observation Schedule: Revised Algorithms for Improved Diagnostic Validity
Gotham, K., Risi, S., Pickles, A., & Lord, C. (2007) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-006-0280-1 · Cited by 1,161
Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech
Stolcke, A., Ries, K., Coccaro, N., Shriberg, E., Bates, R., Jurafsky, D., Taylor, P., Martin, R., Ess-Dykema, C. V., & Meteer, M. (2000) · Computational Linguistics · doi:10.1162/089120100561737 · Cited by 1,108
Toward detecting emotions in spoken dialogs
Lee, C. M., & Narayanan, S. S. (2005) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2004.838534 · Cited by 988
Self-citationSocial signal processing: Survey of an emerging domain
Vinciarelli, A., Pantic, M., & Bourlard, H. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.11.007 · Cited by 842
Analysis of emotion recognition using facial expressions, speech and multimodal information
Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C. M., Kazemzadeh, A., Lee, S., Neumann, U., & Narayanan, S. (2004) · Proceedings of the 6th international conference on Multimodal interfaces · doi:10.1145/1027933.1027968 · Cited by 840 · cited 1×
“Research has also shown how spoken language information can be combined with visual information for enhancing emotion recognition [103]–[106].”
BackgroundSelf-citationOn phonetic convergence during conversational interaction
Pardo, J. S. (2006) · The Journal of the Acoustical Society of America · doi:10.1121/1.2178720 · Cited by 796
MCMC-based particle filtering for tracking a variable number of interacting targets
Khan, Z., Balch, T., & Dellaert, F. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.223 · Cited by 771
(2017) · Cambridge University Press eBooks · doi:10.1017/9781316676202 · Cited by 763
Schuller, B., Batliner, A., Steidl, S., & Seppi, D. (2011) · Speech Communication · doi:10.1016/j.specom.2011.01.011 · Cited by 758
Emotions and speech: some acoustical correlates.
Williams, C. E., & Stevens, K. N. (1972) · The Journal of the Acoustical Society of America · doi:10.1121/1.1913238 · Cited by 735 · cited 1×
“Features for emotion detectionVRole of F0 and other speech features: The role of F0 in the encoding and perception of emotion information in speech has been widely studied [112] –[114].”
BackgroundClient commitment language during motivational interviewing predicts drug use outcomes.
Amrhein, P. C., Miller, W. R., Yahne, C. E., Palmer, M., & Fulcher, L. (2003) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.71.5.862 · Cited by 729 · cited 1×
“…behavioral therapy, dialog-act-based features offer a means to verify desirable therapy characteristics such as reflection by the therapist [98]; more details are offered in Section III-D. e) Interaction modeling: A key aspect in understanding social and communicative behavior lies in…”
BackgroundToward a histology of social behavior: Judgmental accuracy from thin slices of the behavioral stream
Ambady, N., Bernieri, F. J., & Richeson, J. A. (2000) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(00)80006-4 · Cited by 718
Speaker Diarization: A Review of Recent Research
Anguera Miro, X., Bozonnet, S., Evans, N., Fredouille, C., Friedland, G., & Vinyals, O. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2125954 · Cited by 669
Shriberg, L. D., Paul, R., McSweeny, J. L., Klin, A., Cohen, D. J., & Volkmar, F. R. (2001) · Journal of Speech, Language, and Hearing Research · doi:10.1044/1092-4388(2001/087 · Cited by 601
Embodiment in conversational interfaces: Rea
Cassell, J., Bickmore, T., Billinghurst, M., Campbell, L., Chang, K., Vilhjálmsson, H., & Yan, H. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303150 · Cited by 566
THE DICTIONARY OF AFFECT IN LANGUAGE
WHISSELL, C. M. (1989) · The Measurement of Emotions · doi:10.1016/b978-0-12-558704-4.50011-6 · Cited by 536
Rapid Biologically-Inspired Scene Classification Using Features Shared with Visual Attention
Siagian, C., & Itti, L. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.40 · Cited by 526
Estimating the Reliability, Systematic Error and Random Error of Interval Data
Krippendorff, K. (1970) · Educational and Psychological Measurement · doi:10.1177/001316447003000105 · Cited by 491
Ickes, W. (1993) · Journal of Personality · doi:10.1111/j.1467-6494.1993.tb00783.x · Cited by 459
The Reliability, Validity, and Utility of Self-Assessment
Ross, J. A. (2020) · doi:10.7275/9wph-vv65 · Cited by 453
Nicolaou, M. A., Gunes, H., & Pantic, M. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.9 · Cited by 450
The role of intonation in emotional expressions
Bänziger, T., & Scherer, K. R. (2005) · Speech Communication · doi:10.1016/j.specom.2005.02.016 · Cited by 449
Expressive Writing, Emotional Upheavals, and Health
Pennebaker, J. W., & Chung, C. K. (2006) · Foundations of Health Psychology · doi:10.1093/oso/9780195139594.003.0011 · Cited by 449
Language Style Matching as a Predictor of Social Dynamics in Small Groups
Gonzales, A. L., Hancock, J. T., & Pennebaker, J. W. (2010) · Communication Research · doi:10.1177/0093650209351468 · Cited by 436
Prosody-based automatic segmentation of speech into sentences and topics
Shriberg, E., Stolcke, A., Hakkani-Tür, D., & Tür, G. (2000) · Speech Communication · doi:10.1016/s0167-6393(00)00028-5 · Cited by 424
Toward an Affect-Sensitive AutoTutor
D'Mello, S., Picard, R. W., & Graesser, A. (2007) · IEEE Intelligent Systems · doi:10.1109/mis.2007.79 · Cited by 386
Christensen, A., Atkins, D. C., Berns, S., Wheeler, J., Baucom, D. H., & Simpson, L. E. (2004) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.72.2.176 · Cited by 380
Emotion recognition using a hierarchical binary decision tree approach
Lee, C.-C., Mower, E., Busso, C., Lee, S., & Narayanan, S. (2011) · Speech Communication · doi:10.1016/j.specom.2011.06.004 · Cited by 355
Self-citationEffects of Visual and Verbal Interaction on Unintentional Interpersonal Coordination.
Richardson, M. J., Marsh, K. L., & Schmidt, R. C. (2005) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.31.1.62 · Cited by 347
Oller, D. K., Niyogi, P., Gray, S., Richards, J. A., Gilkerson, J., Xu, D., Yapanel, U., & Warren, S. F. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1003882107 · Cited by 344
Wöllmer, M., Eyben, F., Reiter, S., Schuller, B., Cox, C., Douglas-Cowie, E., & Cowie, R. (2008) · Interspeech 2008 · doi:10.21437/interspeech.2008-192 · Cited by 328
Juslin, P. N., & Scherer, K. R. (2005) · The New Handbook of Methods in Nonverbal Behavior Research · doi:10.1093/oso/9780198529613.003.0003 · Cited by 318
Making Space for Voice: Technologies to Support Children’s Fantasy and Storytelling
Cassell, J., & Ryokai, K. (2001) · Personal and Ubiquitous Computing · doi:10.1007/pl00000018 · Cited by 297
Receptive and Expressive Prosodic Ability in Children With High-Functioning Autism
Peppé, S., McCann, J., Gibbon, F., O’Hare, A., & Rutherford, M. (2007) · Journal of Speech, Language, and Hearing Research · doi:10.1044/1092-4388(2007/071 · Cited by 296
Application of the wavelet transform for pitch detection of speech signals
Kadambe, S., & Boudreaux-Bartels, G. F. (1992) · IEEE Transactions on Information Theory · doi:10.1109/18.119752 · Cited by 296
Primitives-based evaluation and estimation of emotions in speech
Grimm, M., Kroschel, K., Mower, E., & Narayanan, S. (2007) · Speech Communication · doi:10.1016/j.specom.2007.01.010 · Cited by 291
Self-citationParalinguistics in speech and language—State-of-the-art and the challenge
Schuller, B., Steidl, S., Batliner, A., Burkhardt, F., Devillers, L., Müller, C., & Narayanan, S. (2013) · Computer Speech & Language · doi:10.1016/j.csl.2012.02.005 · Cited by 284
Self-citationAnalysis of Emotionally Salient Aspects of Fundamental Frequency for Emotion Detection
Busso, C., Lee, S., & Narayanan, S. (2009) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2008.2009578 · Cited by 278
Self-citationEMOTION IN SPEECH: RECOGNITION AND APPLICATION TO CALL CENTERS
Petrushin, V. A. (1999) · Cited by 276
Automatic labeling of prosodic patterns
Wightman, C. W., & Ostendorf, M. (1994) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.326607 · Cited by 275
Ladd, D. R., Silverman, K. E. A., Tolkmitt, F., Bergmann, G., & Scherer, K. R. (1985) · The Journal of the Acoustical Society of America · doi:10.1121/1.392466 · Cited by 260
The social psychophysiology of marriage.
Gottman, J. M., & LEVENSON, R. W. (1988) · Cited by 258
An overview of spoken language technology for education
Eskenazi, M. (2009) · Speech Communication · doi:10.1016/j.specom.2009.04.005 · Cited by 256
The Nature of Conversational Involvement and Nonverbal Encoding Patterns
COKER, D. A., & BURGOON, J. (1987) · Human Communication Research · doi:10.1111/j.1468-2958.1987.tb00115.x · Cited by 247
Discrimination of speech from nonspeech based on multiscale spectro-temporal Modulations
Mesgarani, N., Slaney, M., & Shamma, S. A. (2006) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/tsa.2005.858055 · Cited by 224
Weighted autocorrelation for pitch extraction of noisy speech
Shimamura, T., & Kobayashi, H. (2001) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.952490 · Cited by 221
A Framework for Automatic Human Emotion Classification Using Emotion Profiles
Mower, E., Mataric, M. J., & Narayanan, S. (2011) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2010.2076804 · Cited by 215
Self-citationThe ICSI RT07s Speaker Diarization System
Wooters, C., & Huijbregts, M. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-68585-2_47 · Cited by 201
Schuller, B., Batliner, A., Seppi, D., Steidl, S., Vogt, T., Wagner, J., Devillers, L., Vidrascu, L., Amir, N., Kessous, L., et al. (2007) · Interspeech 2007 · doi:10.21437/interspeech.2007-612 · Cited by 191
Fear-type emotion recognition for future audio-based surveillance systems
Clavel, C., Vasilescu, I., Devillers, L., Richard, G., & Ehrette, T. (2008) · Speech Communication · doi:10.1016/j.specom.2008.03.012 · Cited by 180
Predicting student emotions in computer-human tutoring dialogues
Litman, D. J., & Forbes-Riley, K. (2004) · Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics - ACL '04 · doi:10.3115/1218955.1219000 · Cited by 179
Manual for the Motivational Interviewing Skill Code (MISC)
Amrhein, P. C., Miller, W. R., Moyers, T. B., & Ernst, D. (2008) · Cited by 176
High frequency word entrainment in spoken dialogue
Nenkova, A., Gravano, A., & Hirschberg, J. (2008) · Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies Short Papers - HLT '08 · doi:10.3115/1557690.1557737 · Cited by 175
An acoustic study of emotions expressed in speech
Yildirim, S., Bulut, M., Lee, C. M., Kazemzadeh, A., Deng, Z., Lee, S., Narayanan, S., & Busso, C. (2004) · Interspeech 2004 · doi:10.21437/interspeech.2004-242 · Cited by 169
Self-citationCombining acoustic and language information for emotion recognition
Lee, C. M., Narayanan, S. S., & Pieraccini, R. (2002) · 7th International Conference on Spoken Language Processing (ICSLP 2002) · doi:10.21437/icslp.2002-296 · Cited by 165
Self-citationWollmer, M., Schuller, B., Eyben, F., & Rigoll, G. (2010) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2010.2057200 · Cited by 165
Pause and gap length in face-to-face interaction
Edlund, J., Heldner, M., & Hirschberg, J. (2009) · Interspeech 2009 · doi:10.21437/interspeech.2009-710 · Cited by 159
Waldinger, R. J., Schulz, M. S., Hauser, S. T., Allen, J. P., & Crowell, J. A. (2004) · Journal of Family Psychology · doi:10.1037/0893-3200.18.1.58 · Cited by 158
Automatic Prosodic Event Detection Using Acoustic, Lexical, and Syntactic Evidence
Ananthakrishnan, S., & Narayanan, S. S. (2007) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2007.907570 · Cited by 157
Self-citationWhere am I? Scene Recognition for Mobile Robots using Audio Features
Chu, S., Narayanan, S. S., Kuo, C. ‐. J., & Matarić, M. J. (2006) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2006.262661 · Cited by 157
Self-citationFletcher, R., Dobson, K., Goodwin, M. S., Eydgahi, H., Wilder-Smith, O., Fernholz, D., Kuboyama, Y., Hedman, E. B., Poh, M., & Picard, R. W. (2010) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2009.2038692 · Cited by 155
Context-Sensitive Learning for Enhanced Audiovisual Emotion Classification
Metallinou, A., Wöllmer, M., Katsamanis, A., Eyben, F., Schuller, B. W., & Narayanan, S. S. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.40 · Cited by 153
Self-citationInterrelation Between Speech and Facial Gestures in Emotional Utterances: A Single Subject Study
Busso, C., & Narayanan, S. S. (2007) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2007.905145 · Cited by 152
Self-citationWu, D., Courtney, C. G., Lance, B. J., Narayanan, S. S., Dawson, M. E., Oie, K. S., & Parsons, T. D. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.12 · Cited by 151
Self-citationRecognition of negative emotions from the speech signal
Lee, C. M., Narayanan, S. S., & Pieraccini, R. (2005) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2001.1034632 · Cited by 151
Self-citationAutomatic pitch contour stylization using a model of tonal perception
d’Alessandro, C. R., & Mertens, P. (1995) · Computer Speech & Language · doi:10.1006/csla.1995.0013 · Cited by 150
Creating conversational interfaces for children
Narayanan, S. S., & Potamianos, A. (2002) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.985544 · Cited by 144
Self-citationThe emerging evidence base for motivational interviewing: a meta-analytic and qualitative inquiry
Burke, B. L., Dunn, C. W., Atkins, D. C., & Phelps, J. S. (2004) · Journal of Cognitive Psychotherapy · doi:10.1891/jcop.18.4.309.64002 · Cited by 143
Automatic detection of sentence boundaries and disfluencies based on recognized words
Stolcke, A., Shriberg, E. E., Bates, R. A., Mari Ostendorf, E., Hakkani, D. Z., Plauché, M., Tür, G., & Lu, Y. (1998) · International Conference on Spoken Language Processing (ICSLP) · doi:10.21437/icslp.1998-486 · Cited by 139
Evidence-based approaches to assessing couple distress.
Snyder, D. K., Heyman, R. E., & Haynes, S. N. (2005) · Psychological Assessment · doi:10.1037/1040-3590.17.3.288 · Cited by 135 · cited 3×
“Snyder et al. [30] provide indepth discussion on evidence-based approaches, especially relevant to distressed couples but which generalizes at a certain extent to other BSP domains.”
BackgroundInfluentialUsing Actor Portrayals to Systematically Study Multimodal Emotion Expression: The GEMEP Corpus
Bänziger, T., & Scherer, K. R. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74889-2_42 · Cited by 135
Forbes-Riley, K., & Litman, D. J. (2011) · Speech Communication · doi:10.1016/j.specom.2011.02.006 · Cited by 133
Interpreting ambiguous emotional expressions
Mower, E., Metallinou, A., Lee, C.-C., Kazemzadeh, A., Busso, C., Lee, S., & Narayanan, S. S. (2009) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2009.5349500 · Cited by 132
Self-citationCombining multiple estimators of speaking rate
Morgan, N., & Fosler‐Lussier, E. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1998.675368 · Cited by 128
Multimodal Physical Activity Recognition by Fusing Temporal and Cepstral Information
Li, M., Rozgić, V., Thatte, G., Lee, S., Emken, A., Annavaram, M., Mitra, U., Spruijt‐Metz, D., & Narayanan, S. S. (2010) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2010.2053217 · Cited by 125
Self-citationThe nuts and bolts of behavioral observation of marital and family interaction
Margolin, G., Oliver, P. H., Gordis, E. B., O'Hearn, H. G., Medina, A. M., Ghosh, C. M., & Morland, L. (1998) · Clinical Child and Family Psychology Review · doi:10.1023/a:1022608117322 · Cited by 123
Measuring changes in emotion during psychotherapy: Conceptual and methodological issues.
Sloan, D. M., & Kring, A. M. (2007) · Clinical Psychology Science and Practice · doi:10.1111/j.1468-2850.2007.00092.x · Cited by 123
Automatic detection of discourse structure for speech recognition and understanding
Jurafsky, D., Bates, R., Coccaro, N., Martin, R., Meteer, M. W., Ries, K., Shriberg, E. E., Stolcke, A., Taylor, P., & Ess-Dykema, C. V. (2002) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.1997.658992 · Cited by 122
Black, M. P., Katsamanis, A., Baucom, B., Lee, C.-C., Lammert, A. C., Christensen, A., Georgiou, P., & Narayanan, S. S. (2011) · Speech Communication · doi:10.1016/j.specom.2011.12.003 · Cited by 112
Self-citationLee, C.-C., Black, M., Katsamanis, A., Lammert, A. C., Baucom, B., Christensen, A., Georgiou, P., & Narayanan, S. S. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-287 · Cited by 110
Self-citationDetecting emotional state of a child in a conversational computer game
Yıldırım, S., Narayanan, S. S., & Potamianos, A. (2010) · Computer Speech & Language · doi:10.1016/j.csl.2009.12.004 · Cited by 100
Self-citationRobust Speech Rate Estimation for Spontaneous Speech
Wang, D., & Narayanan, S. S. (2007) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2007.905178 · Cited by 99
Self-citationFifty years of progress in speech and speaker recognition
Furui, S. (2004) · The Journal of the Acoustical Society of America · doi:10.1121/1.4784967 · Cited by 97
The ICSI Meeting Project: Resources and Research
Janin, A. L., Ang, J., Bhagat, S., Dhillon, R., Edwards, J. A., Marcias-Guarasa, J., Morgan, N., Peskin, B., Shriberg, E. E., Stolcke, A., et al. (2004) · Cited by 95
Using Context to Improve Emotion Detection in Spoken Dialog Systems
Liscombe, J., Riccardi, G., & Hakkani-Tür, D. Z. (2005) · Columbia Academic Commons (Columbia University) · doi:10.7916/d81j9k7f · Cited by 93
Sridhar, V. K. R., Bangalore, S., & Narayanan, S. S. (2008) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2008.917071 · Cited by 90
Self-citationMetallinou, A., Lee, S., & Narayanan, S. (2010) · 2010 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2010.5494890 · Cited by 89
Self-citationLee, C.-C., Katsamanis, A., Black, M., Baucom, B. R., Christensen, A., Georgiou, P. G., & Narayanan, S. S. (2014) · Computer Speech and Language · doi:10.1016/j.csl.2012.06.006 · Cited by 87
Self-citationBaucom, B., Atkins, D. C., Rowe, L. S., & Christensen, A. (2009) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0014405 · Cited by 85
Prominence Detection Using Auditory Attention Cues and Task-Dependent High Level Information
Kalinli, O., & Narayanan, S. S. (2009) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2009.2014795 · Cited by 84
Self-citationUsing neutral speech models for emotional speech analysis
Busso, C., Lee, S., & Narayanan, S. S. (2007) · Interspeech (USB) · doi:10.21437/interspeech.2007-605 · Cited by 82
Self-citationKNOWME: a case study in wireless body area sensor network design
Mitra, U., Emken, B. A., Lee, S., Li, M., Rozgić, V., Thatte, G., Vathsangam, H., Zois, D., Annavaram, M., Narayanan, S. S., et al. (2012) · IEEE Communications Magazine · doi:10.1109/mcom.2012.6194391 · Cited by 82
Self-citation“You” and “I” need to talk about “us”: Linguistic patterns in marital interactions
Baucom, K. J. W., Atkins, D. C., Sevier, M., Eldridge, K. A., & Christensen, A. (2010) · Personal Relationships · doi:10.1111/j.1475-6811.2010.01251.x · Cited by 81
Detecting friendly, flirtatious, awkward, and assertive speech in speed-dates
Ranganath, R., Jurafsky, D., & McFarland, D. A. (2012) · Computer Speech & Language · doi:10.1016/j.csl.2012.01.005 · Cited by 80
Spotting agreement and disagreement: A survey of nonverbal audiovisual cues and tools
Bousmalis, K., Méhu, M., & Pantić, M. (2009) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2009.5349477 · Cited by 79
An Acoustic Measure for Word Prominence in Spontaneous Speech
Wang, D., & Narayanan, S. S. (2007) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2006.881703 · Cited by 74
Self-citationModeling mutual influence of interlocutor emotion states in dyadic spoken interactions
Lee, C.-C., Busso, C., Lee, S., & Narayanan, S. S. (2009) · Interspeech (USB) · doi:10.21437/interspeech.2009-480 · Cited by 74
Self-citationRachel: Design of an emotionally targeted interactive agent for children with autism
Mower, E., Black, M. P., Flores, E., Williams, M. E., & Narayanan, S. S. (2011) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2011.6011990 · Cited by 72
Self-citationMultiple person and speaker activity tracking with a particle filter
Checka, N., Wilson, K., Siracusa, M. R., & Darrell, T. (2004) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2004.1327252 · Cited by 72
Using context to improve emotion detection in spoken dialog systems
Liscombe, J., Riccardi, G., & Hakkani-Tür, D. Z. (2005) · Interspeech (USB) · doi:10.21437/interspeech.2005-583 · Cited by 71
(2002) · doi:10.1109/icme.2002.1035887 · Cited by 69
Entrainment in Speech Preceding Backchannels
Levitan, R., Gravano, A., & Hirschberg, J. (2011) · doi:10.7916/d89z9dcs · Cited by 65
Brute-forcing hierarchical functionals for paralinguistics: A waste of feature space?
Schuller, B. W., Wimmer, M., Mösenlechner, L., Kern, C., Arsić, D., & Rigoll, G. (2008) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2008.4518656 · Cited by 60
Gubbins, C. A., Perosa, L. M., & Bartle‐Haring, S. (2010) · Contemporary Family Therapy · doi:10.1007/s10591-010-9132-4 · Cited by 59
Bone, D., Black, M. P., Lee, C.-C., et al. (2012) · Interspeech (USB) · doi:10.21437/interspeech.2012-307 · Cited by 58
Combining lexical, syntactic and prosodic cues for improved online dialog act tagging
Sridhar, V. K. R., Bangalore, S., & Narayanan, S. S. (2008) · Computer Speech & Language · doi:10.1016/j.csl.2008.12.001 · Cited by 57
Self-citationGeorgiou, P., Black, M. P., Lammert, A. C., Baucom, B., & Narayanan, S. S. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24600-5_12 · Cited by 55
Self-citationEthical Issues in Couple and Family Research.
Margolin, G., Chien, D., Duman, S., Fauchier, A., Gordis, E. B., Oliver, P. H., Ramos, M. C., & Vickerman, K. A. (2005) · Journal of Family Psychology · doi:10.1037/0893-3200.19.1.157 · Cited by 54
Analyzing the Language of Therapist Empathy in Motivational Interview based Psychotherapy.
Xiao, B., Can, D., Georgiou, P., Atkins, D. C., & Narayanan, S. S. (2012) · PubMed · Cited by 52
Self-citationVisual emotion recognition using compact facial representations and viseme information
Metallinou, A., Busso, C., Lee, S., & Narayanan, S. S. (2010) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2010.5494893 · Cited by 52
Self-citationAutomatic classification of married couples' behavior using audio features
Black, M., Katsamanis, A., Lee, C.-C., Lammert, A. C., Baucom, B., Christensen, A., Georgiou, P., & Narayanan, S. S. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-574 · Cited by 50
Self-citationTBALL data collection: the making of a young children's speech corpus
Kazemzadeh, A., You, H., Iseli, M. R., Jones, B., Cui, X., Heritage, M., Price, P. J., Anderson, E., Narayanan, S. S., & Alwan, A. A. (2005) · Interspeech (USB) · doi:10.21437/interspeech.2005-462 · Cited by 50
Self-citationGottman, J. M., Guralnick, M. J., Wilson, B. J., Swanson, C., & Murray, J. D. (1997) · Development and Psychopathology · doi:10.1017/s0954579497002113 · Cited by 50
Automatic Prediction of Children's Reading Ability for High-Level Literacy Assessment
Black, M. P., Tepperman, J., & Narayanan, S. S. (2010) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2076389 · Cited by 50
Self-citationPoliteness and frustration language in child-machine interactions
Arunachalam, S., Gould, D., Andersen, E. S., Byrd, D., & Narayanan, S. S. (2001) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.2001-626 · Cited by 49
Self-citationKimura, M., & Daibo, I. (2006) · Journal of Nonverbal Behavior · doi:10.1007/s10919-006-0011-5 · Cited by 49
Ananthakrishnan, S., & Narayanan, S. B. (2006) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2005.1415102 · Cited by 49
Self-citationAttraction and Social Coordination: Mutual Entrainment of Vocal Activity Rhythms
McGarva, A. R., & Warner, R. M. (2003) · Journal of Psycholinguistic Research · doi:10.1023/a:1023547703110 · Cited by 48
Cacioppo, J. T., Berntson, G. G., Sheridan, J. F., & McClintock, M. K. (2000) · Psychological Bulletin · doi:10.1037//0033-2909.126.6.829 · Cited by 47
Recognizing Uncertainty in Speech
Pon-Barry, H., & Shieber, S. M. (2010) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/2011/251753 · Cited by 46
Kernel models for affective lexicon creation
Malandrakis, N., Potamianos, A., Iosif, E., & Narayanan, S. S. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-745 · Cited by 43
Self-citationTracking changes in continuous emotion states using body language and prosodic cues
Metallinou, A., Katsamanis, A., Wang, Y., & Narayanan, S. S. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5946939 · Cited by 42
Self-citationTheoretical Perspectives on Affect and Deep Learning
Graesser, A. C., & D’Mello, S. K. (2011) · Springer eBooks · doi:10.1007/978-1-4419-9625-1_2 · Cited by 41
Audhkhasi, K., & Narayanan, S. S. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.139 · Cited by 39
Self-citationHassan: A Virtual Human for Tactical Questioning
Traum, D. R., Roque, A., Leuski, A., Georgiou, P., Gerten, J., Martinovski, B., Narayanan, S. S., Robinson, S. M., & Vaswani, A. (2007) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2007.sigdial-1.15 · Cited by 38
Self-citationAudio-visual speaker tracking with importance particle filters
Gática-Pérez, D., Lathoud, G., McCowan, I. A., et al. (2004) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2003.1247172 · Cited by 37
Can, D., Georgiou, P., Atkins, D. C., & Narayanan, S. S. (2012) · Interspeech (USB) · doi:10.21437/interspeech.2012-134 · Cited by 36
Self-citationLee, C.-C., Katsamanis, A., Black, M. P., Baucom, B., Georgiou, P., & Narayanan, S. S. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-776 · Cited by 35
Self-citationDetecting Politeness and frustration state of a child in a conversational computer game
Yıldırım, S., Lee, C. M., Lee, S., Potamianos, A., & Narayanan, S. S. (2005) · Interspeech (USB) · doi:10.21437/interspeech.2005-700 · Cited by 33
Self-citationClassification of Atypical Language in Autism
Prud’hommeaux, E. T., Brian Roark, A., Black, L. M., & van Santen, J. P. H. (2011) · Cited by 33
Multimodal Sensing for Pediatric Obesity Applications
Annavaram, M., Medvidovic, N., Mitra, U., Narayanan, S. B., Sukhatme, G., Meng, Z., Qiu, S., Kumar, R., Thatte, G., Spruijt‐Metz, D., et al. (2008) · Cited by 33
Self-citationInvestigating the role of phoneme-level modifications in emotional speech resynthesis
Bulut, M., Busso, C., Yıldırım, S., Kazemzadeh, A., Lee, C. M., Lee, S., & Narayanan, S. S. (2005) · Interspeech (USB) · doi:10.21437/interspeech.2005-378 · Cited by 32
Self-citationA Framework for Eliciting Emotional Speech: Capitalizing on the Actor’s Process
Hirschberg, J., & Enos, F. (2006) · HAL AMU · doi:10.7916/d88s4zcg · Cited by 31
A hierarchical static-dynamic framework for emotion classification
Mower, E., & Narayanan, S. S. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5946960 · Cited by 28
Self-citationPredicting interruptions in dyadic spoken interactions
Lee, C.-C., & Narayanan, S. S. (2010) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2010.5494991 · Cited by 27
Self-citationAcoustic and prosodic correlates of social behavior
Gravano, A., Levitan, R., Willson, L., Stefan Benus, A., Hirschberg, J., & Nenkova, A. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-25 · Cited by 27
Aggressive Cognitions of Violent Versus Nonviolent Spouses
Clements, K., & Holtzworth-Munroe, A. (2008) · Cognitive Therapy and Research · doi:10.1007/s10608-007-9139-9 · Cited by 26
A Generative Student Model for Scoring Word Reading Skills
Tepperman, J., Lee, S., Narayanan, S. S., & Alwan, A. A. (2010) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2047812 · Cited by 24
Self-citationThe USC CARE corpus: child-psychologist interactions of children with autism spectrum disorders
Black, M. P., Bone, D., Williams, M. E., Gorrindo, P., Levitt, P. R., & Narayanan, S. S. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-314 · Cited by 24
Self-citationBlack, M. P., Georgiou, P., Katsamanis, A., Baucom, B., & Narayanan, S. S. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-23 · Cited by 23
Self-citationRecognition of Physical Activities in Overweight Hispanic Youth Using KNOWME Networks
Emken, B. A., Li, M., Thatte, G., Lee, S., Annavaram, M., Mitra, U., Narayanan, S. S., & Spruijt‐Metz, D. (2012) · Journal of Physical Activity and Health · doi:10.1123/jpah.9.3.432 · Cited by 21
Self-citationThatte, G., Li, M., Lee, S., Emken, A., Narayanan, S. S., Mitra, U., Spruijt‐Metz, D., & Annavaram, M. (2012) · ACM Transactions on Embedded Computing Systems · doi:10.1145/2331147.2331158 · Cited by 20
Self-citationMultiple Instance Learning for Classification of Human Behavior Observations
Katsamanis, A., Gibson, J., Black, M. P., & Narayanan, S. S. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24600-5_18 · Cited by 19
Self-citationLee, C.-C., Katsamanis, A., Black, M. P., Baucom, B., Georgiou, P., & Narayanan, S. S. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24571-8_4 · Cited by 19
Self-citationEmotion Twenty Questions: Toward a Crowd-Sourced Theory of Emotions
Kazemzadeh, A., Lee, S., Georgiou, P., & Narayanan, S. S. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24571-8_1 · Cited by 19
Self-citationGibson, J., Katsamanis, A., Black, M. P., & Narayanan, S. S. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-470 · Cited by 15
Self-citationAnalyzing the nature of ECA interactions in children with autism
Mower, E., Lee, C.-C., Gibson, J., Chaspari, T., Williams, M. E., & Narayanan, S. S. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-748 · Cited by 14
Self-citationCognitive state classification in a spoken tutorial dialogue system
Zhang, T., Hasegawa‐Johnson, M., & Levinson, S. E. (2005) · Speech Communication · doi:10.1016/j.specom.2005.09.006 · Cited by 13
An Empirical Analysis of User Uncertainty in Problem-Solving Child-Machine Interactions
Black, M., Chang, J. N., & Narayanan, S. S. (2008) · Cited by 11
Self-citationAudhkhasi, K., & Narayanan, S. S. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-647 · Cited by 10
Self-citationA cluster-profile representation of emotion using agglomerative hierarchical clustering
Mower, E., Han, K. J., Lee, S., & Narayanan, S. S. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-288 · Cited by 10
Self-citationChaspari, T., Lee, C.-C., & Narayanan, S. S. (2012) · Interspeech (USB) · doi:10.21437/interspeech.2012-316 · Cited by 10
Self-citationVirtual Museum Guides demonstration
Swartout, W., Traum, D. R., Artstein, R., Noren, D., Debevec, P. E., Bronnenkant, K., Williams, J., Leuski, A., Narayanan, S. S., Piepol, D., et al. (2010) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2010.5700842 · Cited by 9
Self-citationDoohan, E.-A. M. (2007) · International Journal of Listening · doi:10.1080/10904010709336844 · Cited by 7
Using measures of vocal entrainment to inform outcome-related behaviors in marital conflicts
Lee, C.-C., Katsamanis, A., Baucom, B., Georgiou, P., & Narayanan, S. S. (2012) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · Cited by 4
Self-citation
Cited by
A review of speaker diarization: Recent advances with deep learning
Park, T. J., Kanda, N., Dimitriadis, D., Han, K. J., Watanabe, S., & Narayanan, S. (2022) · Computer Speech & Language · doi:10.1016/j.csl.2021.101317 · Cited by 354
Self-citationLab And/Or Field? Measuring Personality Processes and Their Social Consequences
Wrzus, C., & Mehl, M. R. (2015) · European Journal of Personality · doi:10.1002/per.1986 · Cited by 200
A Taxonomy of Social Errors in Human-Robot Interaction
Tian, L., & Oviatt, S. (2021) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3439720 · Cited by 133
Tanana, M. J., Soma, C. S., Srikumar, V., Atkins, D. C., & Imel, Z. E. (2019) · Journal of Medical Internet Research · doi:10.2196/12529 · Cited by 100
Long Dialogue Emotion Detection Based on Commonsense Knowledge Graph Guidance
Nie, W., Bao, Y., Zhao, Y., & Liu, A. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3267295 · Cited by 96
Metadata
- Authors
- Narayanan, S., & Georgiou, P. G.
- Venue
- Proceedings of the IEEE
- Published
- 2013-05
- Volume
- 101
- Issue
- 5
- Pages
- 1203-1233
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W2014399678
- Cited by
- 294
- References
- 173
- FWCI
- 37.62
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.