The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Never-ending learning
Open accessAbstract
Whereas people learn many different types of knowledge from diverse experiences over many years, and become better learners over time, most current machine learning systems are much more narrow, learning just a single function or data model based on statistical analysis of a single data set. We suggest that people learn better than computers precisely because of this difference, and we suggest a key direction for machine learning research is to develop software architectures that enable intelligent agents to also learn many types of knowledge, continuously over many years, and to become better learners over time. In this paper we define more precisely this never-ending learning paradigm for machine learning, and we present one case study: the Never-Ending Language Learner (NELL), which achieves a number of the desired properties of a never-ending learner. NELL has been learning to read the Web 24hrs/day since January 2010, and so far has acquired a knowledge base with 120mn diverse, confidence-weighted beliefs (e.g., servedWith(tea,biscuits) ), while learning thousands of interrelated functions that continually improve its reading competence over time. NELL has also learned to reason over its knowledge base to infer new beliefs it has not yet read from those it has, and NELL is inventing new relational predicates to extend the ontology it uses to represent beliefs. We describe the design of NELL, experimental results illustrating its behavior, and discuss both its successes and shortcomings as a case study in never-ending learning. NELL can be tracked online at http://rtw.ml.cmu.edu, and followed on Twitter at @CMUNELL.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 1×
“Work on curriculum learning (Bengio et al. 2009) explores potential synergies across sets or sequences of learning tasks.”
BackgroundText classification from labeled and unlabeled documents using EM
Nigam, K., Mccallum, A. K., Thrun, S., & Mitchell, T. (2000) · Machine Learning · doi:10.1023/a:1007692713085 · Cited by 2,736
Self-citationToward an architecture for never-ending language learning
Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka, E., & Mitchell, T. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7519 · Cited by 2,000
Self-citationIncorporating Vector Space Similarity in Random Walk Inference over Knowledge Bases
Gardner, M., Talukdar, P., Krishnamurthy, J., & Mitchell, T. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1044 · Cited by 189
Self-citation
Research works printed but not linked: 36
- 10.1007/11503415_8 · DOI
- 10.1561/2200000006 · DOI
- 10.1145/279943.279962 · DOI
- 10.5555/3023476.3023483 · doi:10.5555/3023476.3023483
- 10.5555/2898607.2898816 · doi:10.5555/2898607.2898816
- 10.1145/1718487.1718501 · DOI · Paper page
- 10.5555/3086758 · doi:10.5555/3086758
- 10.1109/ICCV.2013.178 · DOI · Paper page
- 10.5555/295240.295725 · doi:10.5555/295240.295725
- 10.1145/1458082.1458165 · DOI
- 10.1109/ISDA.2014.7066260 · DOI
- 10.1145/988672.988687 · DOI
- 10.5555/2283396.2283398 · doi:10.5555/2283396.2283398
- 10.5555/2002472.2002545 · doi:10.5555/2002472.2002545
- 10.1016/0004-3702(87)90050-6 · DOI · Paper page
- 10.1145/122344.122365 · DOI
- 10.5555/2145432.2145494 · doi:10.5555/2145432.2145494
- 10.1016/S0004-3702(83)80005-8 · DOI
- Maaten , L.v.d. , Hinton , G. Visualizing data using t- SNE. J. Machine Learning Res. 9 , Nov (2008): 2579 -- 2605 . Maaten, L.v.d., Hinton, G. Visualizing data using t-SNE. J. Machine Learning Res. 9, Nov (2008):2579--2605.
- Mitchell , T.M. , Allen , J. , Chalasani , P. , Cheng , J. , Etzioni , O. , Ringuette , M.N. , Schlimmer , J.C. THEO: A framework for self-improving systems. Arch. for Intel. ( 1991 ), 323--356. Mitchell, T.M., Allen, J., Chalasani, P., Cheng, J., Etzioni, O., Ringuette, M.N., Schlimmer, J.C. THEO: A framework for self-improving systems. Arch. for Intel. (1991), 323--356.
- 10.5555/2886521.2886641 · doi:10.5555/2886521.2886641
- 10.5555/2145432.2145586 · doi:10.5555/2145432.2145586
- Muggleton , S. , Buntine , W. Machine invention of first-order predicates by inverting resolution. Inductivelogic programming ( 1992 ), 261--280. Muggleton, S., Buntine, W. Machine invention of first-order predicates by inverting resolution. Inductivelogic programming (1992), 261--280.
- 10.1007/978-3-642-34654-5_24 · DOI
- 10.5555/3020751.3020822 · doi:10.5555/3020751.3020822
- 10.5555/3045390.3045540 · doi:10.5555/3045390.3045540
- Platanios , E.A. , Poon , H. , Mitchell , T.M. , Horvitz , E. Estimating Accuracy from Unlabeled Data: A Probabilistic Logic Approach ( 2017 ). preprint, https://arxiv.org/abs/1705.07086. Platanios, E.A., Poon, H., Mitchell, T.M., Horvitz, E. Estimating Accuracy from Unlabeled Data: A Probabilistic Logic Approach (2017). preprint, https://arxiv.org/abs/1705.07086. · arXiv
- 10.1007/978-3-642-41335-3_34 · DOI · Paper page
- 10.5555/2891460.2891622 · doi:10.5555/2891460.2891622
- 10.1145/1242572.1242667 · DOI · Paper page
- 10.1016/0921-8890(95)00004-Y · DOI
- 10.5555/296635 · doi:10.5555/296635
- 10.5555/1642194.1642209 · doi:10.5555/1642194.1642209
- 10.1109/ICDM.2007.104 · DOI
- Wieting , J. , Bansal , M. , Gimpel , K. , Livescu , K. Towards universal paraphrastic sentence embeddings . In Proceedings of the International Conference on Learning Representations (ICLR) ( 2015 ). Wieting, J., Bansal, M., Gimpel, K., Livescu, K. Towards universal paraphrastic sentence embeddings. In Proceedings of the International Conference on Learning Representations (ICLR) (2015).
- Yang , B. , Mitchell , T. Leveraging knowledge bases in lstms for improving machine reading. ACL ( 2017 ). Yang, B., Mitchell, T. Leveraging knowledge bases in lstms for improving machine reading. ACL (2017).
Cited by
Unifying Large Language Models and Knowledge Graphs: A Roadmap
Pan, S., Luo, L., Wang, Y., Chen, C., Wang, J., & Wu, X. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3352100 · Cited by 1,376
Adversarial NLI: A New Benchmark for Natural Language Understanding
Nie, Y., Williams, A., Dinan, E., Bansal, M., Weston, J., & Kiela, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.441 · Cited by 656
HyTE: Hyperplane-based Temporally aware Knowledge Graph Embedding
Dasgupta, S. S., Ray, S. N., & Talukdar, P. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1225 · Cited by 406
Self-citationA survey on data‐efficient algorithms in big data era
Adadi, A. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00419-9 · Cited by 331
Dynabench: Rethinking Benchmarking in NLP
Kiela, D., Bartolo, M., Nie, Y., Kaushik, D., Geiger, A., Wu, Z., Vidgen, B., Prasad, G., Singh, A., Ringshia, P., et al. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.324 · Cited by 254
Voice in Human–Agent Interaction: A Survey
Seaborn, K., Miyake, N. P., Pennefather, P., & Otake-Matsuura, M. (2022) · ACM Computing Surveys · doi:10.1145/3386867 · Cited by 249
Introduction: What is a knowledge graph?
Fensel, D., Şimşek, U., Angele, K., Huaman, E., Kärle, E., Panasiuk, O., Toma, I., Umbrich, J., & Wahler, A. (2020) · Knowledge Graphs · doi:10.1007/978-3-030-37439-6_1 · Cited by 244
Biological underpinnings for lifelong learning machines
Kudithipudi, D., Aguilar-Simon, M., Babb, J., Bazhenov, M., Blackiston, D., Bongard, J., Brna, A. P., Chakravarthi Raja, S., Cheney, N., Clune, J., et al. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00452-0 · Cited by 236
Embedding Uncertain Knowledge Graphs
Chen, X., Chen, M., Shi, W., Sun, Y., & Zaniolo, C. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013363 · Cited by 132
Self-citationFutures of artificial intelligence through technology readiness levels
Martínez-Plumed, F., Gómez, E., & Hernández-Orallo, J. (2021) · Telematics and Informatics · doi:10.1016/j.tele.2020.101525 · Cited by 130
Rethinking search: making domain experts out of dilettantes
Metzler, D., Tay, Y., Bahri, D., & Najork, M. (2021) · ACM SIGIR Forum · doi:10.1145/3476415.3476428 · Cited by 112
Knowledge Graph Quality Management: a Comprehensive Survey
Xue, B., & Zou, L. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3150080 · Cited by 108
BoxE: A Box Embedding Model for Knowledge Base Completion
Abboud, R., Ceylan, İ. İ., Lukasiewicz, T., & Salvatori, T. (2020) · arXiv · doi:10.48550/arxiv.2007.06267 · Cited by 74
Huang, T., Chen, P., & Li, R. (2022) · Proceedings of the ACM Web Conference 2022 · doi:10.1145/3485447.3511984 · Cited by 61
Artificial cognition vs. artificial intelligence for next-generation autonomous robotic agents
Sandini, G., Sciutti, A., & Morasso, P. (2024) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2024.1349408 · Cited by 40
GEO-Bench: Toward Foundation Models for Earth Monitoring
Lacoste, A., Lehmann, N., Rodriguez, P., Sherwin, E., Kerner, H., Lütjens, B., Irvin, J., Dao, D., Alemohammad, H., Drouin, A., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2223 · Cited by 40
Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions
Roller, S., Boureau, Y.-L., Weston, J., Bordes, A., Dinan, E., Fan, A., Gunning, D., Ju, D., Li, M., Poff, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.12442 · Cited by 38
A Definition of Continual Reinforcement Learning
Abel, D., Barreto, A., Roy, B. V., Precup, D., Hasselt, H. V., & Singh, S. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2192 · Cited by 36
General intelligence requires rethinking exploration
Jiang, M., Rocktäschel, T., & Grefenstette, E. (2022) · arXiv · doi:10.48550/arxiv.2211.07819 · Cited by 17
AutoML Strategy Based on Grammatical Evolution: A Case Study about Knowledge Discovery from Text
Estevez-Velarde, S., Gutiérrez, Y., Montoyo, A., & Almeida-Cruz, Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1428 · Cited by 15
TeCre: A Novel Temporal Conflict Resolution Method Based on Temporal Knowledge Graph Embedding
Ma, J., Zhou, C., Chen, Y., Wang, Y., Hu, G., & Qiao, Y. (2023) · Information · doi:10.3390/info14030155 · Cited by 7
Credible Intervals for Knowledge Graph Accuracy Estimation
Marchesin, S., & Silvello, G. (2025) · Proceedings of the ACM on Management of Data · doi:10.1145/3725279 · Cited by 7
ConceptFormer: Towards Efficient Use of Knowledge-Graph Embeddings in Large Language Models
Barmettler, J., Bernstein, A., & Rossetto, L. (2025) · arXiv · doi:10.48550/arxiv.2504.07624 · Cited by 3
Self-Improving LLM Agents at Test-Time
Acikgoz, E. C., Qian, C., Ji, H., Hakkani-Tür, D., & Tur, G. (2025) · arXiv
“However, this learning paradigm can remain narrow and inefficient compared to actual human learning (Mitchell et al., 2018).”
Toward Enactive Artificial Intelligence
Rafiee, B., & Sutton, R. (2026) · arXiv · doi:10.48550/arxiv.2605.24238
Metadata
- Authors
- Mitchell, T., Cohen, W., Hruschka, E., Talukdar, P., Yang, B., Betteridge, J., Carlson, A., Dalvi, B., Gardner, M., Kisiel, B., Krishnamurthy, J., Lao, N., Mazaitis, K., Mohamed, T., Nakashole, N., Platanios, E., Ritter, A., Samadi, M., Settles, B., Wang, R., Wijaya, D., Gupta, A., Chen, X., Saparov, A., Greaves, M., & Welling, J.
- Venue
- Communications of the ACM
- Published
- 2018-04-24
- Volume
- 61
- Issue
- 5
- Pages
- 103-115
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W277886906
- Cited by
- 507
- References
- 6
- FWCI
- 33.10
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Funders
- National Science Foundation (IIS-1065251)
- Defense Advanced Research Projects Agency (FA8750-13-2-0005)
- Air Force Office of Scientific Research (FA9550-17-1-0218)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
