The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Video-Based Sign Language Recognition Without Temporal Segmentation
Open accessAbstract
Millions of hearing impaired people around the world routinely use some variants of sign languages to communicate, thus the automatic translation of a sign language is meaningful and important. Currently, there are two sub-problems in Sign Language Recognition (SLR), i.e., isolated SLR that recognizes word by word and continuous SLR that translates entire sentences. Existing continuous SLR methods typically utilize isolated SLRs as building blocks, with an extra layer of preprocessing (temporal segmentation) and another layer of post-processing (sentence synthesis). Unfortunately, temporal segmentation itself is non-trivial and inevitably propagates errors into subsequent steps. Worse still, isolated SLR methods typically require strenuous labeling of each word separately in a sentence, severely limiting the amount of attainable training data. To address these challenges, we propose a novel continuous sign recognition framework, the Hierarchical Attention Network with Latent Space (LS-HAN), which eliminates the preprocessing of temporal segmentation. The proposed LS-HAN consists of three components: a two-stream Convolutional Neural Network (CNN) for video feature representation generation, a Latent Space (LS) for semantic gap bridging, and a Hierarchical Attention Network (HAN) for latent space based recognition. Experiments are carried out on two large scale datasets. Experimental results demonstrate the effectiveness of the proposed framework.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1174 · Cited by 4,712
Real-Time Compressive Tracking
Zhang, K., Zhang, L., & Yang, M.-H. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33712-3_62 · Cited by 1,427
Sequence to Sequence -- Video to Text
Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R., Darrell, T., & Saenko, K. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.515 · Cited by 1,352
Real-time American sign language recognition using desk and wearable computer based video
Starner, T., Weaver, J., & Pentland, A. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.735811 · Cited by 1,335
Translating Videos to Natural Language Using Deep Recurrent Neural Networks
Venugopalan, S., Xu, H., Donahue, J., Rohrbach, M., Mooney, R., & Saenko, K. (2015) · Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.3115/v1/n15-1173 · Cited by 959
Jointly Modeling Embedding and Translation to Bridge Video and Language
Pan, Y., Mei, T., Yao, T., Li, H., & Rui, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.497 · Cited by 563
Self-citationKoller, O., Forster, J., & Ney, H. (2015) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2015.09.013 · Cited by 533
A Hierarchical Neural Autoencoder for Paragraphs and Documents
Li, J., Luong, T., & Jurafsky, D. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1107 · Cited by 490
Latent-Dynamic Discriminative Models for Continuous Gesture Recognition
Morency, L.-P., Quattoni, A., & Darrell, T. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383299 · Cited by 374
Cui, R., Liu, H., & Zhang, C. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.175 · Cited by 338
Sign Language Recognition using 3D convolutional neural networks
Huang, J., Zhou, W., Li, H., & Li, W. (2015) · 2015 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2015.7177428 · Cited by 317
Self-citationKoller, O., Ney, H., & Bowden, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.412 · Cited by 278
Re-Sign: Re-Aligned End-to-End Sequence Modelling with Deep Recurrent CNN-HMMs
Koller, O., Zargaran, S., & Ney, H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.364 · Cited by 266
A Real Time System For Dynamic Hand Gesture Recognition With A Depth Sensor
Kurakin, A., Liu, Z., & Zhang, Z. (2016) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.52481 · Cited by 254
A Real-Time Hand Posture Recognition System Using Deep Neural Networks
Tang, A., Lü, K., Wang, Y., Huang, J., & Li, H. (2015) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2735952 · Cited by 165
Self-citationChinese sign language recognition with adaptive HMM
Zhang, J., Zhou, W., Xie, C., Pu, J., & Li, H. (2016) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2016.7552950 · Cited by 165
Self-citationLarge-Vocabulary Continuous Sign Language Recognition Based on Transition-Movement Models
Fang, G., Gao, W., & Zhao, D. (2006) · IEEE Transactions on Systems Man and Cybernetics - Part A Systems and Humans · doi:10.1109/tsmca.2006.886347 · Cited by 136
Sign language recognition with long short-term memory
Liu, T., Zhou, W., & Li, H. (2016) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2016.7532884 · Cited by 135
Self-citationConvolutional Neural Network-Based Robot Navigation Using Uncalibrated Spherical Images
Ran, L., Zhang, Y., Zhang, Q., & Yang, T. (2017) · Sensors · doi:10.3390/s17061341 · Cited by 118
Self-citationEffective Active Skeleton Representation for Low Latency Human Action Recognition
Cai, X., Zhou, W., Wu, L., Luo, J., & Li, H. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2505089 · Cited by 92
Self-citationDiscriminative Exemplar Coding for Sign Language Recognition With Kinect
Sun, C., Zhang, T., Bao, B., Xu, C., & Mei, T. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2265337 · Cited by 89
Bidirectional Long-Short Term Memory for Video Description
Bin, Y., Yang, Y., Shen, F., Xu, X., & Shen, H. T. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2967258 · Cited by 87
A Hyperspectral Image Classification Framework with Spatial Pixel Pair Features
Ran, L., Zhang, Y., Wei, W., & Zhang, Q. (2017) · Sensors · doi:10.3390/s17102421 · Cited by 83
Self-citationContinuous sign language recognition using level building based on fast hidden Markov model
Yang, W., Tao, J., & Ye, Z. (2016) · Pattern Recognition Letters · doi:10.1016/j.patrec.2016.03.030 · Cited by 73
Li, K., Zhou, Z., & Lee, C. (2016) · ACM Transactions on Accessible Computing · doi:10.1145/2850421 · Cited by 63
Online Early-Late Fusion Based on Adaptive HMM for Sign Language Recognition
Guo, D., Zhou, W., Li, H., & Wang, M. (2017) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3152121 · Cited by 61
Self-citationA Threshold-based HMM-DTW Approach for Continuous Sign Language Recognition
Zhang, J., Zhou, W., & Li, H. (2014) · International Conference on Internet Multimedia Computing and Service · doi:10.1145/2632856.2632931 · Cited by 58
Self-citationSign Language Recognition with Multi-modal Features
Pu, J., Zhou, W., & Li, H. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-48896-7_25 · Cited by 34
Self-citationMulti-View Visual Recognition of Imperfect Testing Data
Zhang, Q., & Hua, G. (2015) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2733373.2806224 · Cited by 26
Self-citationCan Visual Recognition Benefit from Auxiliary Information in Training?
Zhang, Q., Hua, G., Liu, W., Liu, Z., & Zhang, Z. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16865-4_5 · Cited by 26
Self-citationAuxiliary Training Information Assisted Visual Recognition
Zhang, Q., Hua, G., Liu, W., Liu, Z., & Zhang, Z. (2015) · IPSJ Transactions on Computer Vision and Applications · doi:10.2197/ipsjtcva.7.138 · Cited by 20
Self-citation
Cited by
Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective
Bragg, D., Koller, O., Bellard, M., Berke, L., Boudreault, P., Braffort, A., Caselli, N., Huenerfauth, M., Kacorri, H., Verhoef, T., et al. (2019) · Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3308561.3353774 · Cited by 422
Attention, please! A survey of neural attention models in deep learning
de Santana Correia, A., & Colombini, E. L. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10148-x · Cited by 298
Skeleton Aware Multi-modal Sign Language Recognition
Jiang, S., Sun, B., Wang, L., Bai, Y., Li, K., & Fu, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00380 · Cited by 290
Improving Sign Language Translation with Monolingual Data by Sign Back-Translation
Zhou, H., Zhou, W., Qi, W., Pu, J., & Li, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00137 · Cited by 282
Self-citationIterative Alignment Network for Continuous Sign Language Recognition
Pu, J., Zhou, W., & Li, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00429 · Cited by 275
Self-citation
Metadata
- Authors
- Huang, J., Zhou, W., Zhang, Q., Li, H., & Li, W.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2018-04-26
- Volume
- 32
- Issue
- 1
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2964253156
- Cited by
- 451
- References
- 35
- FWCI
- 36.08
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.