The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Query by committee
Abstract
We propose an algorithm called query by commitee, in which a committee of students is trained on the same data set. The next query is chosen according to the principle of maximal disagreement. The algorithm is studied for two toy models: the high-low game and perceptron learning of another perceptron. As the number of queries goes to infinity, the committee algorithm yields asymptotically finite information gain. This leads to generalization error that decreases exponentially with the number of examples. This in marked contrast to learning from randomly chosen inputs, for which the information gain approaches zero and the generalization error decreases with a relatively slow inverse power law. We suggest that asymptotically finite information gain may be an important characteristic of good query algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Fedorov, V. V., Studden, W. J., & Klimko, E. M. (1972) · Cited by 2,092
Theory of Optimal Experiments.
Herzberg, A. M., Wynn, H. P., Fedorov, V. V., Studden, W. J., & Klimko, E. M. (1972) · Biometrika · doi:10.2307/2334826 · Cited by 1,897
Statistical mechanics of learning from examples
Seung, H. S., Sompolinsky, H., & Tishby, N. (1992) · Physical Review A · doi:10.1103/physreva.45.6056 · Cited by 532
Self-citationThree unfinished works on the optimal storage capacity of networks
Gardner, E., & Derrida, B. (1989) · Journal of Physics A: Mathematical and General · doi:10.1088/0305-4470/22/12/004 · Cited by 188
Neural net algorithms that learn in polynomial time from examples and queries
Baum, E. B. (1991) · IEEE Transactions on Neural Networks · doi:10.1109/72.80287 · Cited by 168
Consistent inference of probabilities in layered networks: predictions and generalizations
Tishby, Levin, & Solla (1989) · International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1989.118274 · Cited by 161
Generalization performance of Bayes optimal classification algorithm for learning a perceptron
Opper, M., & Haussler, D. H. (1991) · Physical Review Letters · doi:10.1103/physrevlett.66.2677 · Cited by 152
Self-citationImproving a Network Generalization Ability by Selecting Examples
Kinzel, W., & Ruján, P. (1990) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/13/5/016 · Cited by 113
Selecting examples for perceptrons
Watkin, T. L. H., & Rau, A. (1992) · Journal of Physics A Mathematical and General · doi:10.1088/0305-4470/25/1/016 · Cited by 32
Cited by
(2009) · Minds at UW (University of Wisconsin) · Cited by 4,791
Selection of relevant features and examples in machine learning
Blum, A. L., & Langley, P. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00063-5 · Cited by 3,298
Machine learning and the physical sciences
Carleo, G., Cirac, I., Cranmer, K., Daudet, L., Schuld, M., Tishby, N., Vogt-Maranto, L., & Zdeborová, L. (2019) · Reviews of Modern Physics · doi:10.1103/revmodphys.91.045002 · Cited by 2,417
A sequential algorithm for training text classifiers
Lewis, D., & Gale, W. A. (1994) · arXiv (Cornell University) · doi:10.5555/188490.188495 · Cited by 1,731
Unke, O. T., Chmiela, S., Sauceda, H. E., Gastegger, M., Poltavsky, I., Schütt, K. T., Tkatchenko, A., & Müller, K.-R. (2021) · Chemical Reviews · doi:10.1021/acs.chemrev.0c01111 · Cited by 1,647
Improving generalization with active learning
Cohn, D., Atlas, L., & Ladner, R. (1994) · Machine Learning · doi:10.1007/bf00993277 · Cited by 1,591
Support vector machine active learning for image retrieval
Tong, S., & Chang, E. (2001) · Proceedings of the ninth ACM international conference on Multimedia · doi:10.1145/500141.500159 · Cited by 1,328
A Sequential Algorithm for Training Text Classifiers
Lewis, D. D., & Gale, W. A. (1994) · SIGIR ’94 · doi:10.1007/978-1-4471-2099-5_1 · Cited by 1,208
Tri-training: exploiting unlabeled data using three classifiers
Zhou, Z.-H., & Li, M. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.186 · Cited by 1,171
A Survey of Deep Active Learning
Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Gupta, B. B., Chen, X., & Wang, X. (2022) · ACM Computing Surveys · doi:10.1145/3472291 · Cited by 1,169
Heterogeneous Uncertainty Sampling for Supervised Learning
Lewis, D. D., & Catlett, J. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50026-x · Cited by 1,150
Selective Sampling Using the Query by Committee Algorithm
Freund, Y., Seung, H. S., Shamir, E., & Tishby, N. (1997) · Machine Learning · doi:10.1023/a:1007330508534 · Cited by 1,097
Self-citationAn analysis of active learning strategies for sequence labeling tasks
Settles, B., & Craven, M. (2008) · Proceedings of the Conference on Empirical Methods in Natural Language Processing - EMNLP '08 · doi:10.3115/1613715.1613855 · Cited by 972
A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective
Roh, Y., Heo, G., & Whang, S. E. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2946162 · Cited by 968
A Survey on Multi-view Learning
Xu, C., Tao, D., & Xu, C. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1304.5634 · Cited by 942
Four Generations of High-Dimensional Neural Network Potentials
Behler, J. (2021) · Chemical Reviews · doi:10.1021/acs.chemrev.0c00868 · Cited by 873
Automated reverse engineering of nonlinear dynamical systems
Bongard, J., & Lipson, H. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0609476104 · Cited by 792
Formal theory of creativity, fun, and intrinsic motivation (1990–2010)
Schmidhuber, J. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2056368 · Cited by 747
Learning Loss for Active Learning
Yoo, D., & Kweon, I. S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00018 · Cited by 740
Interactive deduplication using active learning
Sarawagi, S., & Bhamidipaty, A. (2002) · Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/775047.775087 · Cited by 692
Data-Driven Evolutionary Optimization: An Overview and Case Studies
Jin, Y., Wang, H., Chugh, T., Guo, D., & Miettinen, K. (2019) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2018.2869001 · Cited by 686
Smith, J. S., Nebgen, B. T., Zubatyuk, R., Lubbers, N., Devereux, C., Barros, K., Tretiak, S., Isayev, O., & Roitberg, A. E. (2019) · Nature Communications · doi:10.1038/s41467-019-10827-4 · Cited by 664
A Survey on Malware Detection Using Data Mining Techniques
Ye, Y., Li, T., Adjeroh, D., & Iyengar, S. S. (2018) · ACM Computing Surveys · doi:10.1145/3073559 · Cited by 623
Settles, B. (2012) · Synthesis Lectures on Artificial Intelligence and Machine Learning · doi:10.2200/s00429ed1v01y201207aim018 · Cited by 605
A Survey of Active Learning Algorithms for Supervised Remote Sensing Image Classification
Tuia, D., Volpi, M., Copa, L., Kanevski, M., & Munoz-Mari, J. (2011) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2011.2139193 · Cited by 558
Bayesian Active Learning for Classification and Preference Learning
Houlsby, N., Huszár, F., Ghahramani, Z., & Lengyel, M. (2011) · arXiv · doi:10.48550/arxiv.1112.5745 · Cited by 507
“Perhaps the most ubiquitous is active learning for SVMs [Tong and Koller, 2001, Seung et al., 1992], where the volume of Version Space (VS) is used as a proxy for the posterior entropy.”
BackgroundMethodActive Learning Methods for Remote Sensing Image Classification
Tuia, D., Ratle, F., Pacifici, F., Kanevski, M. F., & Emery, W. J. (2009) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2008.2010404 · Cited by 502
Committee-Based Sampling For Training Probabilistic Classifiers
Dagan, I., & Engelson, S. P. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50027-x · Cited by 499
A machine learning approach to sentiment analysis in multilingual Web texts
Boiy, E., & Moens, M.-F. (2009) · Information Retrieval · doi:10.1007/s10791-008-9070-z · Cited by 471
iCoseg: Interactive co-segmentation with intelligent scribble guidance
Batra, D., Kowdle, A., Parikh, D., Luo, J., & Chen, T. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540080 · Cited by 468
Learning from Labeled and Unlabeled Data
(2017) · Encyclopedia of Machine Learning and Data Mining · doi:10.1007/978-1-4899-7687-1_100253 · Cited by 462
Machine Learning Paradigms for Speech Recognition: An Overview
Deng, L., & Li, X. (2013) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2013.2244083 · Cited by 456
Liu, H., Ong, Y.-S., & Cai, J. (2018) · Structural and Multidisciplinary Optimization · doi:10.1007/s00158-017-1739-8 · Cited by 444
Active Learning by Querying Informative and Representative Examples
Huang, S.-J., Jin, R., & Zhou, Z.-H. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2307881 · Cited by 440
Hyperspectral Image Classification—Traditional to Deep Models: A Survey for Future Prospects
Ahmad, M., Shabbir, S., Roy, S. K., Hong, D., Wu, X., Yao, J., Khan, A. M., Mazzara, M., Distefano, S., & Chanussot, J. (2022) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2021.3133021 · Cited by 438
An instance level analysis of data complexity
Smith, M. R., Martinez, T., & Giraud-Carrier, C. (2014) · Machine Learning · doi:10.1007/s10994-013-5422-z · Cited by 436
Epitope Landscape in Breast and Colorectal Cancer
Segal, N. H., Parsons, D. W., Peggs, K. S., Velculescu, V., Kinzler, K. W., Vogelstein, B., & Allison, J. P. (2008) · Cancer Research · doi:10.1158/0008-5472.can-07-3095 · Cited by 416
Batch mode active learning and its application to medical image classification
Hoi, S. C. H., Jin, R., Zhu, J., & Lyu, M. R. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143897 · Cited by 421
Wang, H., Jin, Y., & Doherty, J. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2710978 · Cited by 415
Semi-supervised learning by disagreement
Zhou, Z.-H., & Li, M. (2010) · Knowledge and Information Systems · doi:10.1007/s10115-009-0209-z · Cited by 410
Machine Learning for Electronically Excited States of Molecules
Westermayr, J., & Marquetand, P. (2021) · Chemical Reviews · doi:10.1021/acs.chemrev.0c00749 · Cited by 398
Improve Computer-Aided Diagnosis With Machine Learning Techniques Using Undiagnosed Samples
Li, M., & Zhou, Z.-H. (2007) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/tsmca.2007.904745 · Cited by 377
A survey on instance selection for active learning
Fu, Y., Zhu, X., & Li, B. (2013) · Knowledge and Information Systems · doi:10.1007/s10115-012-0507-8 · Cited by 364
Boosting and Other Ensemble Methods
Drucker, H., Cortes, C., Jackel, L. D., LeCun, Y., & Vapnik, V. (1994) · Neural Computation · doi:10.1162/neco.1994.6.6.1289 · Cited by 357
Active learning for logistic regression: an evaluation
Schein, A. I., & Ungar, L. H. (2007) · Machine Learning · doi:10.1007/s10994-007-5019-5 · Cited by 346
Buus, S., Lauemøller, S. L., Worning, P., Kesmir, C., Frimurer, T., Corbet, S., Fomsgaard, A., Hilden, J., Holm, A., & Brunak, S. (2003) · Tissue Antigens · doi:10.1034/j.1399-0039.2003.00112.x · Cited by 329
Outlier detection by active learning
Abe, N., Zadrozny, B., & Langford, J. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150459 · Cited by 322
Diverse ensembles for active learning
Melville, P., & Mooney, R. J. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015385 · Cited by 319
Batch-Mode Active-Learning Methods for the Interactive Classification of Remote Sensing Images
Demir, B., Persello, C., & Bruzzone, L. (2011) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2010.2072929 · Cited by 317
Human-centric artificial intelligence architecture for industry 5.0 applications
Rožanec, J. M., Novalija, I., Zajec, P., Kenda, K., Tavakoli Ghinani, H., Suh, S., Bian, S., Veliou, E., Papamartzivanos, D., Giannetsos, T., et al. (2023) · International Journal of Production Research · doi:10.1080/00207543.2022.2138611 · Cited by 314
Generative Models as an Emerging Paradigm in the Chemical Sciences
Anstine, D. M., & Isayev, O. (2023) · Journal of the American Chemical Society · doi:10.1021/jacs.2c13467 · Cited by 313
Learning object identification rules for information integration
Tejada, S., Knoblock, C. A., & Minton, S. (2001) · Information Systems · doi:10.1016/s0306-4379(01)00042-4 · Cited by 310
Neural Network Potentials: A Concise Overview of Methods
Kocer, E., Ko, T. W., & Behler, J. (2022) · Annual Review of Physical Chemistry · doi:10.1146/annurev-physchem-082720-034254 · Cited by 306
Bottom-up Coarse-Graining: Principles and Perspectives
Jin, J., Pak, A. J., Durumeric, A. E. P., Loose, T. D., & Voth, G. A. (2022) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.2c00643 · Cited by 305
Paul, S., Weiskopf, D., Angelo, M. A., Sidney, J., Peters, B., & Sette, A. (2013) · The Journal of Immunology · doi:10.4049/jimmunol.1302101 · Cited by 298
Machine Learning in Materials Science: Recent Progress and Emerging Applications
Mueller, T., Kusne, A. G., & Ramprasad, R. (2016) · Reviews in Computational Chemistry · doi:10.1002/9781119148739.ch4 · Cited by 295
State-of-the-Art and Comparative Review of Adaptive Sampling Methods for Kriging
Fuhg, J. N., Fau, A., & Nackenhorst, U. (2021) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-020-09474-6 · Cited by 295
Self-Directed Learning: A Cognitive and Computational Perspective
Gureckis, T. M., & Markant, D. B. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612454304 · Cited by 292
An Active Learning Approach to Hyperspectral Data Classification
Rajan, S., Ghosh, J., & Crawford, M. M. (2008) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2007.910220 · Cited by 290
Active Deep Learning for Classification of Hyperspectral Images
Liu, P., Zhang, H., & Eom, K. B. (2017) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2016.2598859 · Cited by 290
Bayesian Inference and Optimal Design for the Sparse Linear Model
Seeger, M. (2008) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · doi:10.5555/1390681.1390707 · Cited by 277
Agent-based model calibration using machine learning surrogates
Lamperti, F., Roventini, A., & Sani, A. (2018) · Journal of Economic Dynamics and Control · doi:10.1016/j.jedc.2018.03.011 · Cited by 261
Large-scale text categorization by batch mode active learning
Hoi, S. C. H., Jin, R., & Lyu, M. R. (2006) · Proceedings of the 15th international conference on World Wide Web · doi:10.1145/1135777.1135870 · Cited by 253
On Issues of Instance Selection
Liu, H., & Motoda, H. (2002) · Data Mining and Knowledge Discovery · doi:10.1023/a:1014056429969 · Cited by 191
A stopping criterion for active learning
Vlachos, A. (2008) · Computer Speech & Language · doi:10.1016/j.csl.2007.12.001 · Cited by 153
Xie, J., Sun, L., & Zhao, Y. F. (2025) · Engineering · doi:10.1016/j.eng.2024.04.024 · Cited by 112
Explainable Active Learning (XAL): Toward AI Explanations as Interfaces for Machine Teachers
Ghai, B., Liao, Q. V., Zhang, Y., Bellamy, R., & Mueller, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3432934 · Cited by 106
Query-by-committee improvement with diversity and density in batch active learning
Kee, S., del Castillo, E., & Runger, G. (2018) · Information Sciences · doi:10.1016/j.ins.2018.05.014 · Cited by 98
(2012) · Cited by 98
Human Activity Recognition Based on Dynamic Active Learning
Bi, H., Perello-Nieto, M., Santos-Rodriguez, R., & Flach, P. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3013403 · Cited by 81
A multifaceted vision of the human-AI collaboration: a comprehensive review
Puerta-Beldarrain, M., Gómez-Carmona, O., Sánchez-Corcuera, R., Casado-Mansilla, D., López-de-Ipiña, D., & Chen, L. (2025) · IEEE Access · doi:10.1109/access.2025.3536095 · Cited by 73
Applying active learning to assertion classification of concepts in clinical text
Chen, Y., Mani, S., & Xu, H. (2012) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2011.11.003 · Cited by 57
Active learning via query synthesis and nearest neighbour search
Wang, L., Hu, X., Yuan, B., & Lu, J. (2015) · Neurocomputing · doi:10.1016/j.neucom.2014.06.042 · Cited by 59
“The most commonly used strategies are uncertainty sampling [10] [27] [28] [11] and query by committee [29] [30] [13].”
MethodActive and semi-supervised learning for object detection with imperfect data
Rhee, P. K., Erdenee, E., Kyun, S. D., Ahmed, M. U., & Jin, S. (2017) · Cognitive Systems Research · doi:10.1016/j.cogsys.2017.05.006 · Cited by 55
Intelligent labeling based on fisher information for medical image segmentation using deep learning
Sourati, J., Gholipour, A., Dy, J. G., Tomas-Fernandez, X., Kurugol, S., & Warfield, S. K. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2907805 · Cited by 50
(2011) · The Electronic Journal of e-Learning · Cited by 24
SEAL: Self-supervised embodied active learning using exploration and 3D consistency
Chaplot, D. S., Dalal, M., Gupta, S., Malik, J., & Salakhutdinov, R. (2021) · arXiv · doi:10.48550/arxiv.2112.01001 · Cited by 22
“Researchers have explored the use of prediction uncertainty [45, 16], coverage [42] and meta-learning [28, 56] to learn such data selection functions.”
BackgroundActive learning with missing values considering imputation uncertainty
Han, J., & Kang, S. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107079 · Cited by 21
Active learning of neural network potentials for rare events
(2024) · doi:10.1039/d3dd00216k · Cited by 20
Towards Fine-grained Sampling for Active Learning in Object Detection
Vikas Desai, S., & Balasubramanian, V. N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00470 · Cited by 14
Active learning for e-rulemaking: public comment categorization
(2008) · Scholarship @ Cornell Law (Cornell University) · doi:10.5555/1367832.1367873 · Cited by 13
Using Chao’s Estimator as a Stopping Criterion for Technology-Assisted Review
Bron, M., van der Heijden, P. G. M., Feelders, A., & Siebes, A. (2025) · ACM Transactions on Information Systems · doi:10.1145/3724116 · Cited by 7
OLALA: Object-Level Active Learning for Efficient Document Layout Annotation
Shen, Z., Zhao, J., Dell, M., Yu, Y., & Li, W. (2020) · arXiv · doi:10.48550/arxiv.2010.01762 · Cited by 6
Cache & Distil: Optimising API Calls to Large Language Models
Ramírez, G., Lindemann, M., Birch, A., & Titov, I. (2023) · arXiv · doi:10.48550/arxiv.2310.13561 · Cited by 4
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244302760185243
Metadata
- Authors
- Seung, H. S., Opper, M., & Sompolinsky, H.
- Venue
- Proceedings of the fifth annual workshop on Computational learning theory
- Publisher
- ACM
- Published
- 1992-07
- Pages
- 287-294
- OpenAlex
- W2080021732
- Cited by
- 1,651
- References
- 9
- FWCI
- 7.57
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.