The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Deception Detection using Real-life Trial Data
Abstract
Hearings of witnesses and defendants play a crucial role when reaching court trial decisions. Given the high-stake nature of trial outcomes, implementing accurate and effective computational methods to evaluate the honesty of court testimonies can offer valuable support during the decision making process. In this paper, we address the identification of deception in real-life trial data. We introduce a novel dataset consisting of videos collected from public court trials. We explore the use of verbal and non-verbal modalities to build a multimodal deception detection system that aims to discriminate between truthful and deceptive statements provided by defendants and witnesses. We achieve classification accuracies in the range of 60-75% when using a model that extracts and fuses features from the linguistic and gesture modalities. In addition, we present a human deception detection study where we evaluate the human capability of detecting deception in trial hearings. The results show that our system outperforms the human capability of identifying deceit.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
DePaulo, B. M., Lindsay, J. J., Malone, B. E., Muhlenbruck, L., Charlton, K., & Cooper, H. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.1.74 · Cited by 2,479 · cited 1×
“We specifically focus on the annotation of facial displays and hand movements, as they have been previously found to correlate with deceptive behavior [9].”
BackgroundLinguistic Inquiry and Word Count (LIWC): Pronounced “Luke,” . . . and Other Useful Facts
Chung, C. K., & Pennebaker, J. W. (2012) · Applied Natural Language Processing · doi:10.4018/978-1-60960-741-8.ch012 · Cited by 2,151
Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage
Ekman, P. (1985) · Cited by 1,716
Lying Words: Predicting Deception from Linguistic Styles
Newman, M. L., Pennebaker, J. W., Berry, D. S., & Richards, J. M. (2003) · Personality and Social Psychology Bulletin · doi:10.1177/0146167203029005010 · Cited by 1,439
Detecting Lies and Deceit: The Psychology of Lying and the Implications for Professional Practice
Vrij, A. (2000) · Medical Entomology and Zoology · Cited by 625
Darwin, Deception, and Facial Expression
EKMAN, P. (2003) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1280.010 · Cited by 624
Syntactic Stylometry for Deception Detection
Feng, S., Banerjee, R., & Choi, Y. (2012) · Cited by 499
Towards a General Rule for Identifying Deceptive Opinion Spam
Li, J., Ott, M., Cardie, C., & Hovy, E. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1147 · Cited by 362
Tian, Y.-L., Kanade, T., & Cohn, J. F. (2005) · Handbook of Face Recognition · doi:10.1007/0-387-27257-7_12 · Cited by 347
The lie detector: explorations in the automatic recognition of deceptive language
Mihalcea, R., & Strapparava, C. (2009) · Proceedings of the ACL-IJCNLP 2009 Conference Short Papers on - ACL-IJCNLP '09 · doi:10.3115/1667583.1667679 · Cited by 335
Self-citationMaxwell, K. T., Oberlander, J., & Croft, W. B. (2013) · The Association for Computational Linguistics · Cited by 294
Seeing through the face of deception
Pavlidis, I., Eberhardt, N. L., & Levine, J. A. (2002) · Nature · doi:10.1038/415035a · Cited by 288
The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena
Allwood, J., Cerrato, L., Jokinen, K., Navarretta, C., & Paggio, P. (2007) · Language Resources and Evaluation · doi:10.1007/s10579-007-9061-5 · Cited by 260
A new theoretical perspective on deception detection: On the psychology of instrumental mind-reading
Granhag, P. A., & Hartwig, M. (2008) · Psychology, Crime & Law · doi:10.1080/10683160701645181 · Cited by 248
Who can best catch a liar?: A meta-analysis of individual differences in detecting deception.
Aamodt, M. G., & Custer, H. (2006) · Cited by 238
Kanade, T. (2005) · Lecture Notes in Computer Science · doi:10.1007/11564386_1 · Cited by 223
Dating deception: Gender, online dating, and exaggerated self-presentation
Guadagno, R. E., Okdie, B. M., & Kruse, S. A. (2012) · Computers in Human Behavior · doi:10.1016/j.chb.2011.11.010 · Cited by 184
Distinguishing deceptive from non-deceptive speech
Hirschberg, J., Stefan Benus, A., Brenier, J., Enos, F., Friedman, S., Gilman, S., Girand, C., Graciarena, M., Kathol, A., Michaelis, L., et al. (2005) · Interspeech (USB) · doi:10.21437/interspeech.2005-580 · Cited by 160
Ekman, P. (2003) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1280.002 · Cited by 132
Deception Detection through Automatic, Unobtrusive Analysis of Nonverbal Behavior
Meservy, T. O., Jensen, M. L., Kruse, J., Twitchell, D. P., Tsechpenakis, G., Burgoon, J. K., Metaxas, D., & Nunamaker, J. F. (2005) · IEEE Intelligent Systems · doi:10.1109/mis.2005.85 · Cited by 129
Distinguishing Deceptive from Non-Deceptive Speech
Hirschberg, J., Beňuš, Š., Brenier, J., Enos, F., Friedman, S., Gilman, S., Girand, C., Graciarena, M., Kathol, A., Michaelis, L. A., et al. (2005) · doi:10.7916/d8697c06 · Cited by 106
Automatic deception detection in Italian court cases
Fornaciari, T., & Poesio, M. (2013) · Artificial Intelligence and Law · doi:10.1007/s10506-013-9140-4 · Cited by 103
Reading between the lines: linguistic cues to deception in online dating profiles
Toma, C. L., & Hancock, J. T. (2010) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/1718918.1718921 · Cited by 98
Blob Analysis of the Head and Hands: A Method for Deception Detection
Lu, S., Tsechpenakis, G., Metaxas, D., Jensen, M. L., & Kruse, J. (2005) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2005.122 · Cited by 86
The design and development of a Lie Detection System using facial micro-expressions
Owayjan, M. J., Kashour, A., Haddad, N. A., Fadel, M., & Souki, G. A. (2012) · International Conference on Advances in Computational Tools for Engineering Applications · doi:10.1109/ictea.2012.6462897 · Cited by 84
Using Deep Linguistic Features for Finding Deceptive Opinion Spam
Xu, Q., & Zhao, H. (2012) · Cited by 82
Explanations for the Perpetration of and Reactions to Deception in a Virtual Community
Joinson, A., & Dietz-Uhler, B. L. (2002) · Social Science Computer Review · doi:10.1177/08939302020003005 · Cited by 76
THE IMPACT OF DECEPTION AND SUSPICION ON DIFFERENT HAND MOVEMENTS
Caso, L., Maricchiolo, F., Bonaiuto, M., Vrij, A., & Mann, S. A. (2006) · Journal of Nonverbal Behavior · doi:10.1007/s10919-005-0001-z · Cited by 73
Deception detection using a multimodal approach
Abouelenien, M., Pérez‐Rosas, V., Mihalcea, R. F., & Burzo, M. G. (2014) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2663204.2663229 · Cited by 67
Self-citationWarrants and deception in computer mediated communication
Warkentin, D., Woodworth, M., Hancock, J. T., & Cormier, N. S. (2010) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/1718918.1718922 · Cited by 67
Detecting Concealment of Intent in Transportation Screening: A Proof of Concept
Burgoon, J. K., Twitchell, D. P., Jensen, M. L., Meservy, T. O., Adkins, M., Kruse, J., Deokar, A. V., Tsechpenakis, G., Lu, S., Metaxas, D., et al. (2009) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2008.2011700 · Cited by 53
Are you Awerewolf? Detecting deceptive roles and outcomes in a conversational role-playing game
Chittaranjan, G. T., & Hung, H. (2010) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2010.5494961 · Cited by 49
Seeing through Deception: A Computational Approach to Deceit Detection in Written Communication
Almela, Á., Valencia‐García, R., & Cantos-Gómez, P. (2012) · Hispana · Cited by 46
A Multimodal Dataset for Deception Detection
Pérez‐Rosas, V., Mihalcea, R. F., Narvaez, A., & Burzo, M. G. (2014) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3go39mfxnfzo · Cited by 46
Self-citationAutomatic, Multimodal Evaluation of Human Interaction
Jensen, M. L., Meservy, T. O., Burgoon, J. K., & Nunamaker, J. F. (2009) · Group Decision and Negotiation · doi:10.1007/s10726-009-9171-0 · Cited by 45
Exonerations in the United States, 1989-2012: Report by the National Registry of Exonerations
Gross, S. R., & Shaffer, M. (2012) · bepress Legal Repository · Cited by 41
Coding Hand Gestures: A Reliable Taxonomy and a Multi-media Support
Maricchiolo, F., Gnisci, A., & Bonaiuto, M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34584-5_36 · Cited by 40
Automatic detection of deception in child-produced speech using syntactic complexity features
Yancheva, M., & Rudzicz, F. (2013) · Cited by 34
Nonverbal indicators of deception: How iconic gestures reveal thoughts that cannot be suppressed
Cohen, D., Beattie, G. W., & Shovelton, H. (2010) · Semiotica · doi:10.1515/semi.2010.055 · Cited by 27
Establishing a foundation for automated human credibility screening
Nunamaker, J. F., Burgoon, J. K., Twyman, N. W., Proudfoot, J. G., Schuetzler, R. M., & Giboney, J. S. (2012) · IEEE International Conference on Intelligence and Security Informatics (ISI) · doi:10.1109/isi.2012.6284309 · Cited by 25
Um … they were wearing …: The effect of deception on specific hand gestures
Hillman, J., Vrij, A., & Mann, S. A. (2011) · Legal and Criminological Psychology · doi:10.1111/j.2044-8333.2011.02014.x · Cited by 24
Control and resistance in the psychology of lying
Derksen, M. (2012) · Theory & Psychology · doi:10.1177/0959354311427487 · Cited by 24
HMM-Based Deception Recognition from Visual Cues
Tsechpenakis, G., Metaxas, D., Adkins, M., Kruse, J., Burgoon, J. K., Jensen, M. L., Meservy, T. O., Twitchell, D. P., Deokar, A. V., & Nunamaker, J. F. (2005) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2005.1521550 · Cited by 23
Cited by
Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms
Ben, X., Ren, Y., Zhang, J., Wang, S.-J., Kpalma, K., Meng, W., & Liu, Y.-J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3067464 · Cited by 306
Kenton, Z., Everitt, T., Weidinger, L., Gabriel, I., Mikulik, V., & Irving, G. (2021) · arXiv · doi:10.48550/arxiv.2103.14659 · Cited by 45
Masking domain-specific information for cross-domain deception detection
Sánchez-Junquera, J., Villaseñor-Pineda, L., Montes-y-Gómez, M., Rosso, P., & Stamatatos, E. (2020) · Pattern Recognition Letters · doi:10.1016/j.patrec.2020.04.020 · Cited by 24
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
Liang, P. P., Lyu, Y., Fan, X., Wu, Z., Cheng, Y., Wu, J., Chen, L., Wu, P., Lee, M. A., Zhu, Y., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.07502 · Cited by 85
Metadata
- Authors
- Pérez-Rosas, V., Abouelenien, M., Mihalcea, R., & Burzo, M.
- Venue
- Proceedings of the 2015 ACM on International Conference on Multimodal Interaction
- Publisher
- ACM
- Published
- 2015-11-09
- Pages
- 59-66
- OpenAlex
- W2163833659
- Cited by
- 269
- References
- 43
- FWCI
- 17.60
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.