The DOI matches its Crossref record; 31 of the 34 references it cites were found in a registry or an index.
Multimodal integration-a statistical view
Abstract
We present a statistical approach to developing multimodal recognition systems and, in particular, to integrating the posterior probabilities of parallel input signals involved in the multimodal system. We first identify the primary factors that influence multimodal recognition performance by evaluating the multimodal recognition probabilities. We then develop two techniques, an estimate approach and a learning approach, which are designed to optimize accurate recognition during the multimodal integration process. We evaluate these methods using Quickset, a speech/gesture multimodal system, and report evaluation results based on an empirical corpus collected with Quickset. From an architectural perspective, the integration technique presented offers enhanced robustness. It also is premised on more realistic assumptions than previous multimodal systems using semantic fusion. From a methodological standpoint, the evaluation techniques that we describe provide a valuable tool for evaluating multimodal systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For instance, Masked Language Modelling (MLM) in the text domain has been applied to audio and image, e.g., Masked Acoustic Modelling [180], [200], Masked Image Region Prediction [190]”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [200]
Citations
Cites
Pattern classification and scene analysis
Grenander, U., Duda, R., & Hart, P. (1974) · A Wiley-Interscience publication · doi:10.2307/1573081 · Cited by 17,399
(1973) · CERN Document Server (European Organization for Nuclear Research) · Cited by 12,895
Kittler, J., Hatef, M., Duin, R. P. W., & Matas, J. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.667881 · Cited by 5,394
Bolt, R. A. (1980) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/800250.807503 · Cited by 1,493
Bridle, J. S. (1990) · Springer eBooks · doi:10.1007/978-3-642-76153-9_28 · Cited by 1,278
Bolt, R. A. (1980) · ACM SIGGRAPH Computer Graphics · doi:10.1145/965105.807503 · Cited by 921
The logic of typed feature structures
Carpenter, B. (1992) · Cited by 600
Cohen, P. R., Johnston, M. O., McGee, D. R., Oviatt, S. L., Pittman, J., Smith, I., Chen, L., & Clow, J. (1997) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/266180.266328 · Cited by 466
Self-citationData fusion in robotics and machine intelligence
Abidi, M. A., & Gonzalez, R. C. (1992) · Academic Press Professional, Inc. eBooks · Cited by 444
The Logic of Typed Feature Structures
Carpenter, R. L. (1992) · Cambridge University Press eBooks · doi:10.1017/cbo9780511530098 · Cited by 410
Research works printed but not linked: 3
- poddar, 1998, toward natural gesture/speech hci: a case study of weather narration, Proc 1998 Workshop on Perceptual User Interfaces (PUI 98), 1
- wu, 0, From members to teams to committee— A robust approach to gestural and multimodal recognition
- young, 1996, Large vocabulary continuous speech recognition A review
Cited by
Sorted by citations
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072
Schuller, B. W., Batliner, A. M., Steidl, S., & Seppi, D. (2011) · Speech Communication · doi:10.1016/j.specom.2011.01.011 · Cited by 765
Automatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning
Ong, S. C. W., & Ranganath, S. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.112 · Cited by 562
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527
Multimodal interaction: A review
Turk, M. (2013) · Pattern Recognition Letters · doi:10.1016/j.patrec.2013.07.003 · Cited by 554
Metadata
- Authors
- Wu, L., Oviatt, S. L., & Cohen, P. R.
- Venue
- IEEE Transactions on Multimedia
- Published
- 1999
- Volume
- 1
- Issue
- 4
- Pages
- 334-341
- ISSN
- 1520-9210 (ISSN-L)
- OpenAlex
- W2151194828
- Cited by
- 190
- References
- 34
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 2 · 2023: 5 · 2022: 4 · 2021: 1 · 2020: 5 · 2019: 5 · 2018: 4 · 2017: 7 · 2016: 3 · 2015: 3 · 2014: 5 · 2013: 8 · 2012: 8
- FWCI
- 9.94
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Multimedia (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Keywords
- multimodal recognition, multimodal integration, robustness, semantic fusion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
