A deep neural network for audio-visual person recognition
Abstract
This paper presents applications of special types of deep neural networks (DNNs) for audio-visual biometrics. A common example is the DBN-DNN that uses the generative weights of deep belief networks (DBNs) to initialize the feature detecting layers of deterministic feed forward DNNs. In this paper, we propose the DBM-DNN that uses the generative weights of deep Boltzmann machines (DBMs) for initialization of DNNs. Then, a softmax layer is added on top and the DNNs are trained discriminatively. Our experimental results show that lower error rates can be achieved using the DBM-DNN compared to the support vector machine (SVM), linear regression-based classifier (LRC) and the DBN-DNN. Experiments were carried out on two publicly available audio-visual datasets: the VidTIMIT and MOBIO.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The structure of deep multimodal Boltzmann machines is similar to that of DBM, but it can admit more than one modality.
“In [48], the structure of deep multimodal Boltzmann machines (DMBM) [71] is similar to that of DBM, but it can admit more than one modality.”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [48]
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928 · see citing works
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · see citing works
Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions
Tan, X., & Triggs, B. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2042645 · Cited by 2,872 · see citing works
(2011) · Cited by 2,752 · see citing works
Salakhutdinov, R., & Hinton, G. E. (2009) · Cited by 1,890 · see citing works
An Efficient Learning Procedure for Deep Boltzmann Machines
Salakhutdinov, R., & Hinton, G. E. (2012) · Neural Computation · doi:10.1162/neco_a_00311 · Cited by 477 · see citing works
Multi-Region Probabilistic Histograms for Robust and Scalable Identity Inference
Sanderson, C., & Lovell, B. C. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-01793-3_21 · Cited by 303 · see citing works
Bi-Modal Person Recognition on a Mobile Phone: Using Mobile Phone Data
McCool, C., Marcel, S., Hadid, A., Pietikäinen, M., Matějka, P., Černocký, J., Poh, N., Kittler, J., Larcher, A., Lévy, C., et al. (2012) · IEEE International Conference on Multimedia and Expo workshops · doi:10.1109/icmew.2012.116 · Cited by 215 · see citing works
MSR Identity Toolbox v1.0: A MATLAB Toolbox for Speaker Recognition Research
Sadjadi, S. O., Slaney, M., & Heck, L. P. (2013) · Cited by 195 · see citing works
Deep belief networks for i-vector based speaker recognition
Ghahabi, O., & Hernando, J. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6853888 · Cited by 90 · see citing works
Enhanced Gradient and Adaptive Learning Rate for Training Restricted Boltzmann Machines
Cho, K., Raiko, T., & Ihler, A. (2011) · Cited by 78 · see citing works
Preliminary investigation of Boltzmann machine classifiers for speaker recognition.
Stafylakis, T., Kenny, P. J., Senoussaoui, M. E. A., et al. (2012) · Espace ÉTS (ETS) · Cited by 49 · see citing works
First attempt of boltzmann machines for speaker verification.
Senoussaoui, M. E. A., Dehak, N., Kenny, P. J., et al. (2012) · Espace ÉTS (ETS) · Cited by 41 · see citing works
Speaker recognition by means of Deep Belief Networks
Vasilakakis, V., Cumani, S., & Laface, P. (2013) · PORTO Publications Open Repository TOrino (Politecnico di Torino) · Cited by 40 · see citing works
Bi-modal biometric authentication on mobile phones in challenging conditions
Khoury, E., Shafey, L. E., McCool, C., et al. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2013.10.001 · Cited by 38 · see citing works
A confidence-based late fusion framework for audio-visual biometric identification
Alam, M. R., Bennamoun, M., Togneri, R. B., et al. (2014) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.10.006 · Cited by 27 · see citing works
Self-citationA Two-Stage Pretraining Algorithm for Deep Boltzmann Machines
Cho, K., Raiko, T., Ilin, A., & Karhunen, J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40728-4_14 · Cited by 20 · see citing works
Linear Regression-based Classifier for audio visual person identification
Alam, M. R., Togneri, R. B., Sohel, F. A., et al. (2013) · International Conference on Communications, Signal Processing, and their Applications (ICCSPA) · doi:10.1109/iccspa.2013.6487281 · Cited by 12 · see citing works
Self-citationCO-LDA: A Semi-supervised Approach to Audio-Visual Person Recognition
Zhao, X., Evans, N., & Dugelay, J. (2012) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2012.14 · Cited by 8 · see citing works
Unsupervised multi-view dimensionality reduction with application to audio-visual speaker retrieval
Zhao, X., Evans, N., & Dugelay, J. (2013) · IEEE International Workshop on Information Forensics and Security · doi:10.1109/wifs.2013.6707786 · Cited by 6 · see citing works
Multi-view semi-supervised discriminant analysis: A new approach to audio-visual person recognition
Zhao, X.-R., Evans, N., & Dugelay, J. (2012) · Graduate School and Research Center in Digital Science (EURECOM) · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Bayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A. (2022) · The Visual Computer · doi:10.1007/s00371-021-02166-7 · Cited by 456 · see citing works
- Authors
- Alam, M. R., Bennamoun, M., Togneri, R., & Sohel, F.
- Affiliations
- The University of Western Australia (AU)
- Venue
- 2015 IEEE 7th International Conference on Biometrics Theory, Applications and Systems (BTAS)
- Publisher
- IEEE
- Published
- 2015-09
- Pages
- 1-6
- OpenAlex
- W2197258718
- Cited by
- 8
- References
- 21
- Citations by year
- 2021: 2 · 2019: 2 · 2017: 1 · 2016: 3
- FWCI
- 0.75
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS) (publishedVersion)
- Murdoch University Research Portal (submittedVersion)
- UWA Profiles and Research Repository (UWA) (submittedVersion)
- Funders
- Australian Research Council
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Biometric Identification and Security
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- deep neural network, deep belief network, deep Boltzmann machine, weight initialization, discriminative training, softmax layer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
