Probability of error of some adaptive pattern-recognition machines
The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
A simple taught pattern-recognition machine for detecting an unknown, fixed, randomly occurring pattern is derived using a Bayes' approach, and its probability of error is analyzed. It is shown that with probability one, the machine converges to the optimal detector (a matched filter) for the unknown pattern, that the asymptotic decision function statistics are Gaussian, and that for an important class of problems, the central-limit theorem can be invoked to calculate the approximate probability of error at any stage of convergence. An untaught adaptive pattern-recognition machine may be made from the taught machine by using its own output instead of a teacher, and the asymptotic probability of error of this device is derived. It is shown that it does not converge to a matched filter for the unknown pattern, but that in any practical case it performs almost as well. Finally, the results of an experimental simulation of both machines are presented as curves of the relative frequency of error vs. time, and are compared with the values calculated by theory.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mathematical Methods of Statistics.
Geary, R. C., & Cramer, H. (1947) · The Economic Journal · doi:10.2307/2226151 · Cited by 5,927
Bharucha-Reid, A. T., Gnedenko, B. V., & Seckler, B. D. (1963) · American Mathematical Monthly · doi:10.2307/2312289 · Cited by 146
Learning to recognize patterns in a random environment
Abramson, N., & Braverman, D. (1962) · IEEE Transactions on Information Theory · doi:10.1109/tit.1962.1057775 · Cited by 88
Cited by
(2006) · The MIT Press eBooks · doi:10.7551/mitpress/9780262033589.001.0001 · Cited by 4,280
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
Sohn, K., Berthelot, D., Li, C.-L., Zhang, Z., Carlini, N., Cubuk, E. D., Kurakin, A., Zhang, H., & Raffel, C. (2020) · arXiv · doi:10.48550/arxiv.2001.07685 · Cited by 2,892
“This idea was introduced decades ago [27, 40].”
BackgroundInfluentialSelf-Training With Noisy Student Improves ImageNet Classification
Xie, Q., Luong, M.-T., Hovy, E., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01070 · Cited by 2,393
Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision
Chen, X., Yuan, Y., Zeng, G., & Wang, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00264 · Cited by 1,209
A Survey on Deep Semi-Supervised Learning
Yang, X., Song, Z., King, I., & Xu, Z. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3220219 · Cited by 923
Review of Image Classification Algorithms Based on Convolutional Neural Networks
Chen, L., Li, S., Bai, Q., Yang, J., Jiang, S., & Miao, Y. (2021) · Remote Sensing · doi:10.3390/rs13224712 · Cited by 789
Snorkel: rapid training data creation with weak supervision
Ratner, A., Bach, S. H., Ehrenberg, H., Fries, J., Wu, S., & Ré, C. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3157794.3157797 · Cited by 777
Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation
Kolesnikov, A., & Lampert, C. H. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_42 · Cited by 719
It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners
Schick, T., & Schütze, H. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.185 · Cited by 747
SCAN: Learning to Classify Images Without Labels
Van Gansbeke, W., Vandenhende, S., Georgoulis, S., Proesmans, M., & Van Gool, L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58607-2_16 · Cited by 550
Pham, H., Dai, Z., Xie, Q., & Le, Q. V. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01139 · Cited by 509
Deep Co-Training for Semi-Supervised Image Recognition
Qiao, S., Shen, W., Zhang, Z., Wang, B., & Yuille, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_9 · Cited by 507
FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling
Zhang, B., Wang, Y., Hou, W., Wu, H., Wang, J., Okumura, M., & Shinozaki, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08263 · Cited by 466
Data Distillation: Towards Omni-Supervised Learning
Radosavovic, I., Dollar, P., Girshick, R., Gkioxari, G., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00433 · Cited by 421
Rethinking Pre-training and Self-training
Zoph, B., Ghiasi, G., Lin, T.-Y., Cui, Y., Liu, H., Cubuk, E. D., & Le, Q. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06882 · Cited by 369
Semi-Supervised Sequence Modeling with Cross-View Training
Clark, K., Luong, M.-T., Manning, C. D., & Le, Q. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1217 · Cited by 366
SemiBoost: Boosting for Semi-Supervised Learning
Mallapragada, P. K., Jin, R., Jain, A. K., & Liu, Y. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.235 · Cited by 340
Wang, C., Riviere, M., Lee, A., Wu, A., Talnikar, C., Haziza, D., Williamson, M., Pino, J., & Dupoux, E. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.80 · Cited by 329
A Simple Semi-Supervised Learning Framework for Object Detection
Sohn, K., Zhang, Z., Li, C., Zhang, H., Lee, C., & Pfister, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.04757 · Cited by 306
CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning
Wei, C., Sohn, K., Mellina, C., Yuille, A., & Yang, F. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01071 · Cited by 298
State of the art in pattern recognition
Nagy, G. (1968) · Proceedings of the IEEE · doi:10.1109/proc.1968.6414 · Cited by 293
WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition
Zhu, Z., Huang, G., Deng, J., Ye, Y., Huang, J., Chen, X., Zhu, J., Yang, T., Lu, J., Du, D., et al. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01035 · Cited by 290
Patterns in pattern recognition: 1968-1974
Kanal, L. (1974) · IEEE Transactions on Information Theory · doi:10.1109/tit.1974.1055306 · Cited by 284
Chung, Y.-A., Zhang, Y., Han, W., Chiu, C.-C., Qin, J., Pang, R., & Wu, Y. (2021) · 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) · doi:10.1109/asru51503.2021.9688253 · Cited by 293
Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning
Cascante-Bonilla, P., Tan, F., Qi, Y., & Ordonez, V. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16852 · Cited by 281
Alonso, I., Sabater, A., Ferstl, D., Montesano, L., & Murillo, A. C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00811 · Cited by 264
MultiMAE: Multi-modal Multi-task Masked Autoencoders
Bachmann, R., Mizrahi, D., Atanov, A., & Zamir, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_20 · Cited by 268
Improved Noisy Student Training for Automatic Speech Recognition
Park, D. S., Zhang, Y., Jia, Y., Han, W., Chiu, C.-C., Li, B., Wu, Y., & Le, Q. V. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-1470 · Cited by 218
Self-Training for End-to-End Speech Recognition
Kahn, J., Lee, A., & Hannun, A. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054295 · Cited by 196
“One such approach, self-training [2], uses noisy labels generated from a model trained on a much smaller labelled data set.”
BackgroundWeakly- and Semi-supervised Panoptic Segmentation
Li, Q., Arnab, A., & Torr, P. H. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_7 · Cited by 171
Naive-Student: Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation
Chen, L.-C., Lopes, R. G., Cheng, B., Collins, M. D., Cubuk, E. D., Zoph, B., Adam, H., & Shlens, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58545-7_40 · Cited by 165
Active and Semisupervised Learning for the Classification of Remote Sensing Images
Persello, C., & Bruzzone, L. (2014) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2014.2305805 · Cited by 165
“16 and 17 report the distribution of unlabeled and semi-labeled samples at different iterations of the PS 3 VM learning process for experiments 2 and 4, respectively.”
BackgroundZhang, Y., Park, D. S., Han, W., Qin, J., Gulati, A., Shor, J., Jansen, A., Xu, Y., Huang, Y., Wang, S., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3182537 · Cited by 161
“learning [38]–[40] for ASR in the context of deep-learning.”
BackgroundDetecting Hallucinated Content in Conditional Neural Sequence Generation
Zhou, C., Neubig, G., Gu, J., Diab, M., Guzmán, F., Zettlemoyer, L., & Ghazvininejad, M. (2021) · Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 · doi:10.18653/v1/2021.findings-acl.120 · Cited by 122
Denoising Pretraining for Semantic Segmentation
Brempong, E. A., Kornblith, S., Chen, T., Parmar, N., Minderer, M., & Norouzi, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00462 · Cited by 120
“Self-training (self-learning or pseudo-labeling) is one of the oldest SSL algorithms [1, 30, 78, 97].”
BackgroundSnorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale
Bach, S. H., Rodriguez, D., Liu, Y., Luo, C., Shao, H., Xia, C., Sen, S., Ratner, A., Hancock, B., Alborzi, H., et al. (2019) · Proceedings of the 2019 International Conference on Management of Data · doi:10.1145/3299869.3314036 · Cited by 118
Multi-Task Self-Training for Learning General Representations
Ghiasi, G., Zoph, B., Cubuk, E. D., Le, Q. V., & Lin, T.-Y. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00873 · Cited by 84
“Self-training: Self-training is a popular technique to incorporate unlabeled data into supervised learning [57, 45, 42, 31].”
Wang, Y., Tang, P., Zhou, Y., Shen, W., Fishman, E. K., & Yuille, A. L. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3060066 · Cited by 34
“, [19] relied on self-training strategies [20].”
Pre-training in Medical Data: A Survey
Qiu, Y., Lin, F., Chen, W., & Xu, M. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1382-8 · Cited by 30
“ing new problems through the influence of experience and knowledge in pre-training on the target tasks[25, 96].”
BackgroundSemi-Supervised Few-Shot Class-Incremental Learning
Cui, Y., Xiong, W., Tavakolian, M., & Liu, L. (2021) · 2021 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip42928.2021.9506346 · Cited by 16
ProteinGLUE multi-task benchmark suite for self-supervised protein modeling
Capel, H., Weiler, R., Dijkstra, M., Vleugels, R., Bloem, P., & Feenstra, K. A. (2022) · Scientific Reports · doi:10.1038/s41598-022-19608-4 · Cited by 11
ProteinGLUE: A multi-task benchmark suite for self-supervised protein modeling
Capel, H., Weiler, R., Dijkstra, M., Vleugels, R., Bloem, P., & Feenstra, K. A. (2021) · doi:10.1101/2021.12.13.472460 · Cited by 11
Synthetic continued pretraining
Yang, Z., Band, N., Li, S., Candès, E., & Hashimoto, T. (2024) · arXiv · doi:10.48550/arxiv.2409.07431 · Cited by 2
Metadata
- Authors
- Scudder, H.
- Venue
- IEEE Transactions on Information Theory
- Published
- 1965-07
- Volume
- 11
- Issue
- 3
- Pages
- 363-371
- ISSN
- 0018-9448 (ISSN-L)
- 1557-9654
- OpenAlex
- W2111316763
- Cited by
- 518
- References
- 3
- FWCI
- 6.49
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.