The MAHNOB Mimicry Database: A database of naturalistic human interactions
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
We present an audiovisual dataset for investigation of mimicry behaviour.We report baseline performances from per-session mimicry classification experiments.Performance is session-dependent, due to variability in subject expressiveness.Current mimicry classification methods need more development for spontaneous data. People mimic verbal and nonverbal expressions and behaviour of their counterparts in various social interactions. Research in psychology and social sciences has shown that mimicry has the power to influence social judgment and various social behaviours, including negotiation and debating, courtship, empathy and helping behaviour. Hence, automatic recognition of mimicry behaviour would be a valuable tool in various domains, and especially in negotiation skills enhancement and medical help provision training. In this work, we present the MAHNOB mimicry database, a set of fully synchronised, multi-sensory, audiovisual recordings of naturalistic dyadic interactions, suitable for investigation of mimicry and negotiation behaviour. The database contains 11?h of recordings, split over 54 sessions of dyadic interactions between 12 confederates and their 48 counterparts, being engaged either in a socio-political discussion or negotiating a tenancy agreement. To provide a benchmark for efforts in machine understanding of mimicry behaviour, we report a number of baseline experiments based on visual data only. Specifically, we consider face and head movements, and report on binary classification of video sequences into mimicry and non-mimicry categories based on the following widely-used methodologies: two similarity-based methods (cross correlation and time warping), and a state-of-the-art temporal classifier (Long Short Term Memory Recurrent Neural Network). The best reported results are session-dependent, and affected by the sparsity of positive examples in the data. This suggests that there is much room for improvement upon the reported baseline experiments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
The chameleon effect: the perception–behavior link and social interaction
Chartrand, T. L., & Bargh, J. A. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.76.6.893 · Cited by 3,674
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
Toward a mechanistic psychology of dialogue.
Pickering, M. J., & Garrod, S. (2004) · Behavioral and Brain Sciences · doi:10.1017/s0140525x04000056 · Cited by 2,678
Lakin, J. L., Jefferis, V. E., Cheng, C. M., & Chartrand, T. L. (2003) · Journal of Nonverbal Behavior · doi:10.1023/a:1025389814290 · Cited by 960 · cited 2×
“Research in psychology and social sciences has shown that presence or absence of motor mimicry behaviour can serve as a (positive or negative) indicator of co-operativeness [16], social judgment [13], presence of autism spectrum disorder [20], and even traumatic brain injury [18].”
BackgroundMcKeown, G., Valstar, M., Cowie, R., Pantic, M., & Schroder, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.20 · Cited by 665
Emotional Mimicry as Social Regulation
Hess, U., & Fischer, A. (2013) · Personality and Social Psychology Review · doi:10.1177/1088868312472607 · Cited by 596
Interpersonal synchrony: a survey of evaluation methods across disciplines.
Delaherche, E., Chetouani, M., Mahdhaoui, A., Saint-Georges, C., Viaux, S., & Cohen, D. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2012.12 · Cited by 474
Bridging the Gap between Social Animal and Unsocial Machine: A Survey of Social Signal Processing
Vinciarelli, A., Pantic, M., Heylen, D., Pelachaud, C., Poggi, I., D'Errico, F., & Schroeder, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.27 · Cited by 381
Cited by
A review on the long short-term memory model
Van Houdt, G., Mosquera, C., & Nápoles, G. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09838-1 · Cited by 1,991
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
Bagher Zadeh, A., Liang, P. P., Poria, S., Cambria, E., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1208 · Cited by 1,433
Sewa db: A rich database for audio-visual emotion and sentiment research in the wild
Kossaifi, J., Walecki, R., Panagakis, Y., Shen, J., Schmitt, M., Ringeval, F., Han, J., Pandit, V., Toisoul, A., Schuller, B., et al. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2944808 · Cited by 243
AFEW-VA database for valence and arousal estimation in-the-wild
Kossaifi, J., Tzimiropoulos, G., Todorovic, S., & Pantic, M. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.02.001 · Cited by 236
Deep Multimodal Emotion Recognition on Human Speech: A Review
Koromilas, P., & Giannakopoulos, T. (2021) · Applied Sciences · doi:10.3390/app11177962 · Cited by 58
“SEMAINE [79] 06:30:00 20 {A, V} English Greek Hebrew AFEW [80] 02:28:03 330 {A, V} English AFEW-VA [81] 00:40:00 240 {A, V} English Mimicry [82] 11:00:00 48 {A, V} English”
Metadata
- Authors
- Bilakhia, S., Petridis, S., Nijholt, A., & Pantic, M.
- Venue
- Pattern Recognition Letters
- Publisher
- Elsevier BV
- Published
- 2015-11
- Volume
- 66
- Pages
- 52-61
- ISSN
- 0167-8655 (ISSN-L)
- OpenAlex
- W2027624334
- Cited by
- 57
- References
- 11
- FWCI
- 8.95
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The MAHNOB Mimicry Database: A database of naturalistic human interactions. https://app.cochat.ai/papers/10.1016/j.patrec.2015.03.005
- The paper
- Bilakhia, S., Petridis, S., Nijholt, A., & Pantic, M. (2015). The MAHNOB Mimicry Database: A database of naturalistic human interactions. Pattern Recognition Letters. https://doi.org/10.1016/j.patrec.2015.03.005
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.