The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index; none of the 5 sources its claims cite is checked for support yet.
Multimodal Emotion Recognition Using Deep Neural Networks
Overview
CoChat Summary
TL;DR
The paper introduces Bimodal-LSTM and Bimodal Deep Denoising AutoEncoder (BDDAE) models for multimodal emotion recognition using EEG plus eye movement or peripheral physiological signals. Bimodal-LSTM reports 93.97% mean accuracy on SEED and 83.23%/83.82% mean accuracy for arousal/valence on DEAP.
Key findings
- On SEED, Bimodal-LSTM achieved the best accuracy at 93.97%, about 2 percentage points higher than state-of-the-art approaches, with the smallest standard deviation of 7.03%; comparison models were Feature Fusion 83.80%, BDDAE 88.98%, Liu et al. 91.01%, and Yang et al. 91.51%.
- On DEAP, Bimodal-LSTM obtained mean accuracies of 83.23% for arousal and 83.82% for valence; the paper states this is state-of-the-art on both tasks, though Table 2 lists Liu et al. at 85.2% for valence.
- BDDAE was also evaluated on SEED, reaching 88.98% mean accuracy, below Bimodal-LSTM but above Feature Fusion and below Liu et al./Yang et al.
- The paper reports that Bimodal-LSTM uses both temporal and frequency-domain information and outperforms methods that treat time steps as independent samples.
Methods
- Two models were proposed: BDDAE, a denoising autoencoder with separate EEG/other-modality encoders, a combined encoder, and a classifier network using L2-regularized hinge loss (linear SVM); and Bimodal-LSTM, with two LSTM encoders for EEG and other modalities, dropout, and an L2-regularized hinge-loss classification layer, trained with Adam and then an SVM whose weights are copied back.
- SEED data: EEG and eye movement from 15 subjects, but experiments used the same data as prior work, containing 27 experiments from 9 subjects; EEG features were differential entropy in five bands (310 dimensions), eye movement features were 41 dimensions, and sequences of length 60 were used. DEAP data: EEG and peripheral physiological signals from 32 participants; EEG features were differential entropy in four bands (128 dimensions), peripheral features were 48 dimensions, sequences of length 5 were used, and 10-fold cross validation was applied with threshold 5 for binary arousal/valence classification.
- For SEED, 200 sets of hyper-parameters were randomly selected per experiment; for DEAP, 15 sets were randomly selected and tuned with 10-fold cross validation. Performance was compared against baseline and state-of-the-art methods using mean accuracy and standard deviation.
Limitations
- The paper does not state a limitations section.
- Evident from the text: SEED evaluation uses a subset of 27 experiments from 9 of the 15 available subjects, so results do not cover the full dataset.
- Evident from the text: DEAP arousal and valence are binarized using a threshold of 5, and hyper-parameters are randomly selected within ranges, which may limit comparability and reproducibility.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 0 Not valid · 3 Not checked · 7 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our experimental results indicate that the Bimodal-LSTM model outperforms other state-of-the-art methods with a mean accuracy of 93.97%.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and the smallest standard deviation (7.03%)”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“with the mean accuracies of 83.23% and 83.83%, respectively.”
- Also stated in: § 6 Conclusion · plain
“mean accuracies of 83.23% and 83.83% for arousal and valence classification tasks”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 6 Conclusion · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“with the mean accuracies of 83.23% and 83.83%, respectively.”
- Also stated in: § 6 Conclusion · plain
“mean accuracies of 83.23% and 83.83% for arousal and valence classification tasks”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 6 Conclusion · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The Bimodal-LSTM model is also examined on DEAP dataset with EEG and peripheral physiological signals, and it achieves the state-of-the-art results with a mean accuracy of 83.53%.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
3 more claims
Create a free account to see all 10 claims and every source verdict.
- Probably not a substantial claim. 2
- It may state two separate claims as one. 1
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 5 Experiment Results
“the mean accuracies of 83.23% and 83.83%, respectively.”
§ 4 https://www.tensorflow.org/ > 91.01 91.51
“Valence (%) 65.29/7.93 85.2/- 83.04/- 83.82/5.01”
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“LSTM neural network, which is a RNN using LSTM blocks, can prevent the vanishing (and exploding) gradient problem [1] and has the ability to learn information from long sequences.”
BackgroundExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319
(1993) · Cited by 1,173
Emotional state classification from EEG data using machine learning approach
Wang, X.-W., Nie, D., & Lu, B.-L. (2014) · Neurocomputing · doi:10.1016/j.neucom.2013.06.046 · Cited by 755 · cited 1×
“Most of the existing methods treated features at each time step as independent samples, and ignored the temporal dependency property of emotions [11].”
MethodSelf-citationDeep Learning using Linear Support Vector Machines
Tang, Y. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1306.0239 · Cited by 708
Recognition of emotions using multimodal physiological signals and an ensemble deep learning model
Yin, Z., Zhao, M., Wang, Y., Yang, J., & Zhang, J. (2017) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2016.12.005 · Cited by 416
The microsoft 2016 conversational speech recognition system
Xiong, W., Droppo, J., Huang, X., Seide, F., Seltzer, M., Stolcke, A., Yu, D., & Zweig, G. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953159 · Cited by 326
Cited by
EmotionMeter: A Multimodal Framework for Recognizing Human Emotions
Zheng, W.-L., Liu, W., Lu, Y., Lu, B.-L., & Cichocki, A. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2797176 · Cited by 1,276
Self-citationLiu, W., Qiu, J.-L., Zheng, W.-L., & Lu, B.-L. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3071170 · Cited by 452
Self-citationAn Efficient LSTM Network for Emotion Recognition From Multichannel EEG Signals
Du, X., Ma, C., Zhang, G., Li, J., Lai, Y.-K., Zhao, G., Deng, X., Liu, Y.-J., & Wang, H. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3013711 · Cited by 357
Bota, P. J., Wang, C., Fred, A. L. N., & Placido Da Silva, H. (2019) · IEEE Access · doi:10.1109/access.2019.2944001 · Cited by 322
Emotion Recognition from Multi-Channel EEG through Parallel Convolutional Recurrent Neural Network
Yang, Y., Wu, Q., Qiu, M., Wang, Y., & Chen, X. (2018) · 2018 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489331 · Cited by 325
Metadata
- Authors
- Tang, H., Liu, W., Zheng, W.-L., & Lu, B.-L.
- Affiliations
- Shanghai Jiao Tong University (CN)
- Corresponding author
- Lu, Bao-Liang
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer International Publishing
- Published
- 2017
- Pages
- 811-819
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W2766825571
- Semantic Scholar
- db1cbdab69f887f8a12551a376a80dbe7df4add4
- Cited by
- 194
- References
- 16
- Citations by year
- 2026: 13 · 2025: 20 · 2024: 40 · 2023: 32 · 2022: 37 · 2021: 18 · 2020: 17 · 2019: 10 · 2018: 7
- FWCI
- 3.52
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Lecture notes in computer science (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Keywords
- multimodal emotion recognition, deep denoising autoencoder, EEG features, eye movement features, peripheral physiological signals, temporal information, SEED dataset, DEAP dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Multimodal Emotion Recognition Using Deep Neural Networks. https://app.cochat.ai/papers/10.1007/978-3-319-70093-9_86
- The paper
- Tang, H., Liu, W., Zheng, W.-L., & Lu, B.-L. (2017). Multimodal Emotion Recognition Using Deep Neural Networks. Lecture Notes in Computer Science. https://doi.org/10.1007/978-3-319-70093-9_86
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.