The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Machine learning for medical diagnosis: history, state of the art and perspective
Abstract
The paper provides an overview of the development of intelligent data analysis in medicine from a machine learning perspective: a historical view, a state-of-the-art view, and a view on some future trends in this subfield of applied artificial intelligence. The paper is not intended to provide a comprehensive overview but rather describes some subareas and directions which from my personal point of view seem to be important for applying machine learning in medical diagnosis. In the historical overview, I emphasize the naive Bayesian classifier, neural networks and decision trees. I present a comparison of some state-of-the-art systems, representatives from each branch of machine learning, when applied to several medical diagnostic tasks. The future trends are illustrated by two case studies. The first describes a recently developed method for dealing with reliability of decisions of classifiers, which seems to be promising for intelligent data analysis in medicine. The second describes an approach to using machine learning in order to verify some unexplained phenomena from complementary medicine, which is not (yet) approved by the orthodox medical community but could in the future play an important role in overall medical diagnosis and treatment.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Human-AI teams under-perform the AI alone in many tasks.
“However, many studies have shown that human-AI teams under-perform the AI alone in tasks [3, 4, 6, 14, 18, 20, 21, 30].”
From A Decision Theoretic Framework for Measuring AI Reliance (2024) · cites this paper as [20]
Citations
Cites
Quinlan, J. R. (1986) · Machine Learning · doi:10.1007/bf00116251 · Cited by 21,439
Neural networks and physical systems with emergent collective computational abilities.
Hopfield, J. J. (1982) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.79.8.2554 · Cited by 19,270
Classification And Regression Trees
Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (2017) · doi:10.1201/9781315139470 · Cited by 11,417
Hopfield, J. J. (1984) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.81.10.3088 · Cited by 6,918
Analysis of probability as an aid in the clinical diagnosis of coronary artery disease.
Diamond, G. A., & Forrester, J. S. (1979) · New England Journal of Medicine · doi:10.1056/nejm197906143002402 · Cited by 2,664
Flagged first, then by citations
Research works printed but not linked: 66
- Shavlik JW, Dietterich TG, editors. Readings in machine learning. Los Altos, CA: Morgan Kaufmann, 1990.
- Mitchell T. Machine learning. New York: McGraw-Hill, 1997.
- Michalski RS, Bratko I, Kubat M, editors. Machine learning, data mining and knowledge discovery: methods and applications. New York: Wiley, 1998.
- Hunt E, Martin J, Stone P. Experiments in induction. New York: Academic Press, 1966.
- Nilsson N. Learning machines. New York: McGraw-Hill, 1965.
- Kononenko I, Bratko I, Kukar M. Application of machine learning to medical diagnosis. In: Michalski RS, Bratko I, Kubat M, editors. Machine learning, data mining and knowledge discovery: methods and applications. New York: Wiley, 1998.
- Pompe U, Kononenko I. Probabilistic first-order classification. In: Lavrač N, Džeroski S, editors. Proceedings of the 7th International Workshop ILP-97 on Inductive Logic Programming. Berlin: Springer, 1997, pp. 235–42. · DOI
- Spiegelhalter, 1993, Bayesian analysis in expert systems, Stat. Sci., 8, 219, 10.1214/ss/1177010888 · DOI
- Good IJ. The estimation of probabilities — an essay on modern Bayesian methods. Cambridge: MIT Press, 1964.
- Kononenko I, Bratko I, Roškar E. Experiments in automatic learning of medical diagnostic rules. In: Proceedings of the International School for the Synthesis of Expert’s Knowledge Workshop, 1984 Aug; Bled, Slovenia.
- Cestnik B, Kononenko I, Bratko I. ASSISTANT 86: a knowledge elicitation tool for sophisticated users. In: Bratko I, Lavrac N, editors. Progress in machine learning. Wilmslow: Sigma Press, 1987.
- Kononenko I. Interpretation of neural networks decisions. In: Proceedings of the IASTED International Conference on Expert Systems and Applications, 1989 Jun 26–29; Zurich, pp. 224–7 (also Proceedings of the ISSEK Workshop, 1989 Sep; Udine).
- Kononenko, 1993, Inductive and Bayesian learning in medical diagnosis, Appl. Artif. Intell., 7, 317, 10.1080/08839519308949993 · DOI
- Cestnik B. Estimating probabilities: a crucial task in machine learning. In: Proceedings of the European Conference on Artificial Intelligence, 1990 Aug; Stockholm, pp. 147–9.
- Kononenko I. Semi-naive Bayesian classifier. In: Kodratoff Y, editor. Proceedings of the European Working Session on Learning-91, 1991 Mar 4–6; Porto. Berlin: Springer, pp. 206–19. · DOI
- Kononenko, 1992, Naive Bayesian classifier and continuous attributes, Informatica, 16, 1
- Langley P. Induction of recursive Bayesian classifiers. In: Proceedings of the European Conference on Machine Learning, 1993 Apr; Vienna. · DOI
- Pazzani M. Searching for dependencies in Bayesian classifiers. Artificial intelligence and statistics IV. Lecture notes in statistics. New York: Springer, 1997. · DOI
- Minsky M. In: Papert S, editor. Perceptrons. Cambridge, MA: MIT Press, 1969.
- Rumelhart DE, Hinton GE, Williams RJ. Learning internal representations by error propagation. In: Rumelhart DE, McClelland JL, editors. Parallel distributed processing, vol. 1: Foundations. Cambridge: MIT Press, 1986.
- Kononenko, 1989, Bayesian neural networks, Biol. Cybernetics J., 61, 361, 10.1007/BF00200801 · DOI
- Quinlan JR. Discovering rules from large collections of examples. In: Michie D, editor. Expert systems in the microelectronic age. Edinburgh: Edinburgh University Press, 1979.
- Michalski, 1980, Learning by being told and learning from examples: an experimental comparison of the two methods of knowledge acquisition in the context of developing an expert system for soybean disease diagnosis, Int. J. Policy Anal. Inform. Syst., 4, 125
- Bratko, 1980, An experiment in automatic learning of diagnostic rules, Informatica (Ljubljana), 4, 18
- Elomaa T, Holsti N. An experimental comparison of inducing decision trees and decision lists in noisy domains. In: Proceedings of the 4th European Working Session on Learning, 1989 Dec 4–6; Montpeiller, pp. 59–69.
- Horn, 1985, An expert system for the interpretation of thyroid assays in a clinical laboratory, Aust. Comput. J., 17, 7
- Hojker S, Kononenko I, Jauk A, Fidler V, Porenta M. Expert system’s development in the management of thyroid diseases. In: Proceedings of the European Congress for Nuclear Medicine, 1988 Sep; Milano.
- Quinlan R, Compton P, Horn KA, Lazarus L. Inductive knowledge acquisition: a case study. In: Quinlan JR, editor. Applications of expert systems. Reading, MA: Addison-Wesley, 1987 (also Proceedings of the 2nd Australian Conference on Applications of Expert Systems, 1986 May 14–16; Sydney).
- Kononenko I, Jauk A, Janc T. Induction of reliable decision rules. In: Proceedings of the International School for the Synthesis of Expert’s Knowledge Workshop, 1988 Sep 10–13; Udine, Italy.
- Karalič, 1991, Significance level based classification with multiple trees, Informatica (Ljubljana), 15, 54
- Kern J, Deželič G, Težak-Benčič M, Durrigl T. Medical decision making using inductive learning program [in Croatian]. In: Proceedings of the 1st Congress on Yugoslav Medical Informatics, 1990 Dec 6–8; Beograd, pp. 221–8.
- Baim, 1988, A method for attribute selection in inductive learning systems, IEEE Trans. PAMI, 10, 888, 10.1109/34.9110 · DOI
- Chan KCC, Wong AKC. Automatic construction of expert systems from data: a statistical approach. In: Proceedings of the IJCAI Workshop on Knowledge Discovery in Databases, 1989 Aug; Detroit, MI, pp. 37–48.
- Bratko I, Mozetič I, Lavrač N. KARDIO: a study in deep and qualitative knowledge for expert systems. Cambridge, MA: MIT Press, 1989.
- Clark P, Boswell R. Rule induction with CN2: some recent improvements. In: Proceedings of the European Working Session on Learning-91, 1991 Mar 4–6; Porto, Portugal, pp. 151–63. · DOI
- Catlett J. On changing continuous attributes into ordered discrete attributes. In: Proceedings of the European Working Session on Learning-91, 1991 Mar 4–6; Porto, Portugal, pp. 164–78. · DOI
- Muggleton S. Inductive acquisition of expert knowledge. Reading, MA: Addison-Wesley, 1990.
- Nunez M. Decision tree induction using domain knowledge. In: Wielinga B, et al., editors. Current trends in knowledge acquisition. Amsterdam: ISO Press, 1990.
- Kira K, Rendell L. A practical approach to feature selection. In: Sleeman D, Edwards P, editors. Proceedings of the International Conference on Machine Learning, 1992 Jul; Aberdeen. Los Altos, CA: Morgan Kaufmann, 1992, pp. 249–56. · DOI
- Kira K, Rendell L. The feature selection problem: traditional methods and new algorithm. In: Proceedings of the AAAI’92, 1992 Jul; San Jose, CA.
- Robnik-Šikonja M, Kononenko I. An adaptation of Relief for attribute estimation in regression. In: Proceedings of the International Conference on Machine Learning ICML-97, 1997 Jul; Nashville, pp. 296–304.
- Kukar, 1996, Machine learning in prognostics of the femoral neck fracture recovery, Artif. Intell. Med., 8, 431, 10.1016/S0933-3657(96)00351-X · DOI
- Kononenko I. Estimating attributes: analysis and extensions of RELIEF. In: De Raedt L, Bergadano F, editors. Proceedings of the European Conference on Machine Learning, 1994 Apr; Catania. Berlin: Springer, 1994, pp. 171–82. · DOI
- Ragavan H, Rendell L. Lookahead feature construction for learning hard concepts. In: Proceedings of the 10th International Conference on Machine Learning, 1993 Jun; Amherst, MA. Los Altos, CA: Morgan Kaufmann, 1993, pp. 252–9. · DOI
- Ragavan H, Rendell L, Shaw M, Tessmer A. Learning complex real-world concepts through feature construction. Technical report UIUC-BI-AI-93-03, The Beckman Institute, University of Illinois, 1993. · DOI
- Weigand S, Huberman A, Rumelhart DE. Predicting the future: a connectionist approach. Int. J. Neural Syst. 1990;1(3):101–117. · DOI
- Craven MW, Shavlik JW. Learning symbolic rules using artificial neural networks. In: Proceedings of the 10th International Conference on Machine Learning; Amherst, MA. Los Altos, CA: Morgan Kaufmann, 1993, pp. 73–80. · DOI
- Quinlan JR. C4.5: programs for machine learning. San Mateo, CA: Morgan Kaufmann, 1993.
- Pirnat V, Kononenko I, Janc T, Bratko I. Medical estimation of automatically induced decision rules. In: Proceedings of the of 2nd European Conference on Artificial Intelligence in Medicine, 1989 Aug 29–31; City University, London, pp. 24–36. · DOI
- Kononenko, 1991, Information based evaluation criterion for classifier’s performance, Machine Learning, 6, 67, 10.1007/BF00153760 · DOI
- Kukar, 1999, Analysing and improving the diagnosis of ishaemic heart disease with machine learning, Artif. Intell. Med., 16, 25, 10.1016/S0933-3657(98)00063-3 · DOI
- Kukar M. Estimating the reliability of classifications and cost-sensitive combining of different machine learning methods. Ph.D. Thesis [in Slovene], Faculty of Computer and Information Science, University of Ljubljana, Ljubljana, Slovenia, 2001.
- Pollock, 1983, Computer-assisted interpretation of noninvasive tests for diagnosis of coronary artery disease, Cardiovasc. Rev. Rep., 4, 367
- Kukar M, Grošelj C. Machine learning in stepwise diagnostic process. In: Proceedings of the Joint European Conference on Artificial Intelligence in Medicine and Medical Decision Making, 1999; Aalborg, Denmark, pp. 315–25. · DOI
- Korotkov K, Kouznetsov A. The theory of morfogenetic synergization of biological objects and the phantom leaf effect. In: Proceedings of the 3rd International Conference for Medical and Applied Bio-Electrography, 1997 Apr; Helsinki, Finland, pp. 55–7.
- Krashenuk A, Krashenuk S, Korotkov K, Buzian N, Lesiovskaya E, Bogaeva N. GDV analysis of hirudotherapy effect to rats. In: Proceedings of the International Scientific Conference Kirlionics, White Nights 98, 1998 Jun; Federal Technical University SPIFMO, St. Petersburg, Russia.
- Voeikov VL. Living blood outside an organism. In: Proceedings of the International Scientific Conference Kirlionics, White Nights 98, 1998 Jun; Federal Technical University SPIFMO, St. Petersburg, Russia.
- Gurvits, 1998, A new concept of the early diagnosis of cancer, Consciousness Phys. Reality, 1, 84
- Lee SD. The application of Kirlian photography in dentistry. In: Proceedings of the International Scientific Conference Kirlionics, White Nights 98, 1998 Jun; Federal Technical University SPIFMO, St. Petersburg, Russia.
- Hein M. Bio-synergetix: a new paradigm in subtle energy health care. In: Proceedings of the International Conference on Biology and Cognitive Sciences, 1999 Oct 12–14; Ljubljana, Slovenia, pp. 72–9.
- Kononenko I, Zrimec T, Prihavec B, Bevk M, Stanojević S. Machine learning and GDV images: diagnosis and therapy verification. In: Proceedings of the International Conference on Biology and Cognitive Sciences, 1999 Oct; Ljubljana, Slovenia, pp. 84–7.
- Kononenko I, Zrimec T, Sadikov A, Mele K, Milharčič T. Machine learning and GDV images: current research and results. In: Proceedings of the International Conference on Biology and Cognitive Science, 1999 Oct; Ljubljana, Slovenia, pp. 80–3.
- Bevk M, Kononenko I, Zrimec T. Relation between energetic diagnoses and GDV images. In: Proceedings of the New Science of Consciousness: 3rd International Conference on Cognitive Science, 2000 Oct; Ljubljana, Slovenia, pp. 54–7.
- Kononenko I, Zrimec T, Sadikov A, Skočaj D. GDV images: current research and results. In: Proceedings of the New Science of Consciousness: 3rd International Conference on Cognitive Science, 2000 Oct; Ljubljana, Slovenia, pp. 65–8.
- Trampuž, 2000, Experiental and biophysical effects of the art of living programme, Int. J. Psychol., 35, 12
- Kononenko I, Bosnič Z, Žgajnar B. The influence of mobile telephones on human bioelectromagnetic field. In: Proceedings of the New Science of Consciousness: 3rd International Conference on Cognitive Science, 2000 Oct; Ljubljana, Slovenia, pp. 69–72.
Cited by
Sorted by citations
Machine Learning Applications in Cancer Prognosis and Prediction
Kourou, K., Exarchos, T. P., Exarchos, K. P., Karamouzis, M. V., & Fotiadis, D. I. (2015) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2014.11.005 · Cited by 3,338
Christodoulou, E., Ma, J., Collins, G. S., Steyerberg, E. W., Verbakel, J. Y., & Van Calster, B. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.02.004 · Cited by 2,119
Machine learning for medical imaging
Erickson, B. J., Korfiatis, P., Akkus, Z., & Kline, T. L. (2017) · RadioGraphics · doi:10.1148/rg.2017160130 · Cited by 1,672
Machine learning in materials science
Wei, J., Chu, X., Sun, X., Xu, K., Deng, H., Chen, J., Wei, Z., & Lei, M. (2019) · InfoMat · doi:10.1002/inf2.12028 · Cited by 1,069
Predictive data mining in clinical medicine: Current issues and guidelines
BELLAZZI, R., & ZUPAN, B. (2008) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2006.11.006 · Cited by 823
The use of machine learning algorithms in recommender systems: a systematic review
Portugal, I., Alencar, P., & Cowan, D. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.12.020 · Cited by 743
“Today, ML algorithms are used in several areas besides computer science, including business [6], advertising [16] and medicine [32].”
MethodManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728
Gibon, Y., Blaesing, O. E., Hannemann, J., Carillo, P., Höhne, M., Hendriks, J. H. M., Palacios, N., Cross, J., Selbig, J., & Stitt, M. (2004) · The Plant Cell · doi:10.1105/tpc.104.025973 · Cited by 552
Machine learning in medical applications: a review of state-of-the-art methods
Shehab, M., Abualigah, L., Shambour, Q., Abu-Hashem, M. A., Shambour, M. K. Y., Alsalibi, A. I., & Gandomi, A. H. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105458 · Cited by 542
Leveraging uncertainty information from deep neural networks for disease detection
Leibig, C., Allken, V., Ayhan, M. S., Berens, P., & Wahl, S. (2017) · Scientific Reports · doi:10.1038/s41598-017-17876-z · Cited by 510
Physician Confidence in Artificial Intelligence: An Online Mobile Survey
Oh, S., Kim, J. H., Choi, S.-W., Lee, H. J., Hong, J., & Kwon, S. H. (2019) · Journal of Medical Internet Research · doi:10.2196/12422 · Cited by 387
Predicting suicidal behavior from longitudinal electronic health records.
Barak-Corren, Y., Castro, V. M., Javitt, S., Hoffnagle, A. G., Dai, Y., Perlis, R. H., Nock, M. K., Smoller, J. W., & Reis, B. Y. (2017) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2016.16010077 · Cited by 339
Smart health: Big data enabled health paradigm within smart cities
Pramanik, M. I., Lau, R. Y. K., Demirkan, H., & Azad, M. A. K. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.06.027 · Cited by 341
Machine learning for renewable energy materials
Gu, G. H., Noh, J., Kim, I., & Jung, Y. (2019) · Journal of Materials Chemistry A · doi:10.1039/c9ta02356a · Cited by 311
A systematic survey of computer-aided diagnosis in medicine: Past and present developments
Yanase, J., & Triantaphyllou, E. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.112821 · Cited by 303
Human-centered XAI: Developing design patterns for explanations of clinical decision support systems
Schoonderwoerd, T. A. J., Jorritsma, W., Neerincx, M. A., & van den Bosch, K. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102684 · Cited by 210
Privacy-Preserving Patient Similarity Learning in a Federated Environment: Development and Analysis
Lee, J., Sun, J., Wang, F., Wang, S., Jun, C.-H., & Jiang, X. (2018) · JMIR Medical Informatics · doi:10.2196/medinform.7744 · Cited by 196
An analytical method for diseases prediction using machine learning techniques
Nilashi, M., Ibrahim, O. B., Ahmadi, H., & Shahmoradi, L. (2017) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2017.06.011 · Cited by 190
Homomorphic Encryption for Machine Learning in Medicine and Bioinformatics
Wood, A., Najarian, K., & Kahrobaei, D. (2021) · ACM Computing Surveys · doi:10.1145/3394658 · Cited by 179
Federated Tensor Factorization for Computational Phenotyping
Kim, Y., Sun, J., Yu, H., & Jiang, X. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098118 · Cited by 128
How Machine Learning is Changing e-Government
Alexopoulos, C., Lachana, Z., Androutsopoulou, A., Diamantopoulou, V., Charalabidis, Y., & Loutsaris, M. A. (2019) · Proceedings of the 12th International Conference on Theory and Practice of Electronic Governance · doi:10.1145/3326365.3326412 · Cited by 88
Passerat-Palmbach, J., Farnan, T., McCoy, M., Harris, J. D., Manion, S. T., Flannery, H. L., & Gleim, B. (2020) · 2020 IEEE International Conference on Blockchain (Blockchain) · doi:10.1109/blockchain50366.2020.00080 · Cited by 84
Pocket diagnosis: Secure federated learning against poisoning attack in the cloud
Ma, Z., Ma, J., Miao, Y., Liu, X., Choo, K.-K. R., & Deng, R. H. (2022) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2021.3090771 · Cited by 50
Research on digital technology use in cardiology: bibliometric analysis
Yeung, A. W. K., Kulnik, S. T., Parvanov, E. D., Fassl, A., Eibensteiner, F., Völkl-Kernstock, S., Kletecka-Pulker, M., Crutzen, R., Gutenberg, J., Höppchen, I., et al. (2022) · Journal of Medical Internet Research · doi:10.2196/36086 · Cited by 44
Zhao, C., Jiang, J., Guan, Y., Guo, X., & He, B. (2018) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2018.03.005 · Cited by 41
Cilla, M., Pérez‐Rey, I., Martínez, M. A., Peña, E., & Martínez, J. (2018) · International Journal for Numerical Methods in Biomedical Engineering · doi:10.1002/cnm.3121 · Cited by 38
FPT: A Fixed-Point Accelerator for Torus Fully Homomorphic Encryption
Van Beirendonck, M., D'Anvers, J.-P., Turan, F., & Verbauwhede, I. (2023) · Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3576915.3623159 · Cited by 38
Machine Learning in Control Systems: An Overview of the State of the Art
Moe, S., Rustad, A. M., & Hanssen, K. G. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-04191-5_23 · Cited by 37
A Decision Theoretic Framework for Measuring AI Reliance
Guo, Z., Wu, Y., Hartline, J. D., & Hullman, J. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658901 · Cited by 27
Eisbach, S., Mai, O., & Hertel, G. (2024) · New Ideas in Psychology · doi:10.1016/j.newideapsych.2024.101077 · Cited by 12
Fang, J., Peng, Y., Zhang, X., Wang, Y., Yi, X., Zhang, G., Xu, Y., Wu, B., Liu, S., Li, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.07407 · Cited by 6
Metadata
- Authors
- Kononenko, I.
- Venue
- Artificial Intelligence in Medicine
- Publisher
- Elsevier BV
- Published
- 2001-08
- Volume
- 23
- Issue
- 1
- Pages
- 89-109
- ISSN
- 0933-3657
- Cited by
- 1,306
- References
- 5
- Language
- en
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review