CoChat
Sign Up
Cited by 18Ethics and Social Impacts of AIAcademic integrity and plagiarismMisinformation and Its Impacts

The reanimation of pseudoscience in machine learning and its ethical repercussions

Open access
Andrews, Mel, Smart, Andrew, Birhane, Abeba·2024·Patterns
DOI 10.1016/j.patter.2024.101027PMC PMC11573791
Open this paper in CoChat — free account

Abstract

The present perspective outlines how epistemically baseless and ethically pernicious paradigms are recycled back into the scientific literature via machine learning (ML) and explores connections between these two dimensions of failure. We hold up the renewed emergence of physiognomic methods, facilitated by ML, as a case study in the harmful repercussions of ML-laundered junk science. A summary and analysis of several such studies is delivered, with attention to the means by which unsound research lends itself to social harms. We explore some of the many factors contributing to poor practice in applied ML. In conclusion, we offer resources for research best practices to developers and practitioners.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.patter.2024.101027

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The epistemic failings are inextricably linked with the ethical failings we observe in AI today.

    “As the prevalence of pseudoscientific formulations in AI is increasingly targeted by critical scholars (e.g., Andrews et al. 2024), it becomes clearer how the epistemic failings are inextricably linked with the ethical failings we observe in AI today.”

    From Abundant intelligences: placing AI within Indigenous knowledge frameworks (2025) · cites this paper as Andrews et al. 2024

  • There are deep-seated connections between white supremacy and the quest of artificial general intelligence (AGI).

    “One thing to address here in Legg and Hutter's definition is the deep-seated connections between white supremacy and the quest of artificial general intelligence (AGI).”

    From Abundant intelligences: placing AI within Indigenous knowledge frameworks (2025) · cites this paper as Andrews et al. 2024

    The Introduction discusses the revival of physiognomic research through machine learning, and “A history of misbegotten science” links physiognomy and eugenics to racial hierarchies. Neither passage addresses the pursuit of artificial general intelligence or connects that pursuit to white supremacy. Searches for AGI, general intelligence, and Legg and Hutter found no passage making the attributed point. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 70 · Cited by 18

Research works printed but not linked: 56

  1. [1]Narayanan A. Kapoor S. AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference 2024 Princeton University Press
  2. [2]Sloane M. Moss E. Chowdhury R. A Silicon Valley love triangle: Hiring algorithms, pseudo-science, and the quest for auditability Patterns 3 2022 100425 PMC8848005 35199067
  3. [5]Alam M.S. Rashid M.M. Roy R. Faizabadi A.R. Gupta K.D. Ahsan M.M. Empirical study of autism spectrum disorder diagnosis using facial images by improved transfer learning approach Bioengineering 9 2022 710 36421111 PMC9687350
  4. [6]Chandraprabha K. Shwetha A. Kavitha M. Sumathi R. Real time-employee emotion detection system (RtEED) using machine learning 2021 Third International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (ICICV) 2021 759 763
  5. [8]Kabir M.M. Safir F.B. Shahen S. Maua J. Binte Awlad I.A. Mridha M.F. Human abnormality classification using combined CNN-RNN approach 2020 IEEE 17th International Conference on Smart Communities: Improving Quality of Life Using ICT, IoT and AI (HONET) 2020 204 208 10.1109/HONET50430.2020.9322814 · DOI
  6. [9]Kachur A. Osin E. Davydov D. Shutilov K. Novokshonov A. Assessing the Big Five personality traits using real-life static facial images Sci. Rep. 10 2020 8487 PMC7244587 32444847
  7. [10]Kosinski M. Khambatta P. Wang Y. Facial recognition technology and human raters can predict political orientation from images of expressionless faces even when controlling for demographics and self-presentation Am. Psychol. 2024 10.1037/amp0001295 38512164 · DOI
  8. [11]Mindoro J.N. Pilueta N.U. Austria Y.D. Lolong Lacatan L. Dellosa R.M. Capturing students’ attention through visible behavior: A prediction utilizing YOLOv3 approach 2020 11th IEEE Control and System Graduate Research Colloquium (ICSGRC) 2020 328 333 10.1109/ICSGRC49013.2020.9232659 · DOI
  9. [12]Parde C.J. Hu Y. Castillo C. Sankaranarayanan S. O’Toole A.J. Social trait information in deep convolutional neural networks trained for face identification Cognit. Sci. 43 2019 e12729 PMC8735867 31204800
  10. [13]Peterson J.C. Uddenberg S. Griffiths T.L. Todorov A. Suchow J.W. Deep models of superficial face judgments Proc. Natl. Acad. Sci. USA 119 2022 e2115228119 PMC9169911 35446619
  11. [14]Mujeeb Rahman M. Subashini M.M. Identification of autism in children using static facial features and deep neural networks Brain Sci. 12 2022 94 35053837 PMC8773918
  12. [17]Tsuchiya K. Hatano R. Nishiyama H. Detecting deception using machine learning with facial expressions and pulse rate Artif. Life Robot. 28 2023 509 519 PMC10141812 37360281
  13. [18]Verma H. Lotia S. Singh A. Convolutional neural network based criminal detection 2020 IEEE Region 10 Conference (TENCON) 2020 1124 1129
  14. [19]Vrskova R. Hudec R. Kamencay P. Sykora P. A new approach for abnormal human activities recognition based on ConvLSTM architecture Sensors 22 2022 2946 35458929 PMC9028541
  15. [20]Wang Y. Kosinski M. Deep neural networks are more accurate than humans at detecting sexual orientation from facial images Preprint at PsyArXiv 2017 10.31234/osf.io/hv28a Version 4 29389215 · DOI
  16. [26]y Arcas B.A. Mitchell M. Todorov A. Physiognomy in the age of AI Browne J. Feminist AI: Critical Perspectives on Algorithms, Data, and Intelligent Machines 2023 Oxford Academic 208 236
  17. [27]McQuillan D. Resisting AI: An Anti-fascist Approach to Artificial Intelligence 2022 Policy Press 10.1332/policypress/9781529213492.001.0001 · DOI
  18. [29]Andrews M. The Immortal Science of ML: Machine Learning & the Theory-Free Ideal Preprint at ResearchGate 2023 10.13140/RG.2.2.28311.75685 · DOI
  19. [31]Kaltheuner F. Fake AI 2021 Meatspace Press 10.58704/kcha-1h20 · DOI
  20. [33]Wu X. Zhang X. Automated inference on criminality using face images Preprint at arXiv 2016 10.48550/arXiv.1611.04135 · DOI
  21. [34]Porter R. McCall B. The Cambridge History of Science, Volume 4: Eighteenth-Century Science Bowen A.C. Rihll T.E. Aestimatio: Critical Reviews in the History of Science 2007 Gorgias Press 198 200
  22. [35]Jones A. Taub L. The Cambridge History of Science, Volume 1: Ancient Science 2018 Cambridge University Press
  23. [36]Evans E.C. The study of physiognomy in the second century A.D. Transactions and proceedings of the American Philological Association 1941 The Johns Hopkins University Press 96 108
  24. [37]Fara P. Marginalized practices The Cambridge History of Science, Volume 4: Eighteenth-Century Science 4 2003 485 507
  25. [42]Rutherford A. A cautionary history of eugenics Science 373 2021 1419 34554795
  26. [43]Amy J.-J. Rowlands S. Legalised non-consensual sterilisation–eugenics put into practice before 1945, and the aftermath. Part 2: Europe Eur. J. Contracept. Reprod. Health Care 23 2018 194 200 29671357
  27. [44]Wolstenholme G.E.W. Man and His Future 2019 John Wiley & Sons
  28. [45]Ziegler J. Philosophers and physicians on the scientific validity of Latin physiognomy, 1200-1500 Early Sci. Med. 12 2007 285 312 18175464
  29. [46]Galen On Temperaments. On Non-Uniform Distemperment. The Soul’s Traits Depend on Bodily Temperament, Loeb Classical Library 2020 Harvard University Press
  30. [49]Mohri M. Rostamizadeh A. Talwalkar A. Foundations of Machine Learning 2018 MIT Press
  31. [50]Schultz M.G. Betancourt C. Gong B. Kleinert F. Langguth M. Leufen L.H. Mozaffari A. Stadtler S. Can deep learning beat numerical weather prediction? Philos. Trans. A Math. Phys. Eng. Sci. 379 2021 20200097 PMC7898133 33583266
  32. [51]Douglas H. Science, Policy, and the Value-Free Ideal 2019 University of Pittsburgh Press
  33. [52]Elliott K.C. A Tapestry of Values: An Introduction to Values in Science 2017 Oxford University Press
  34. [53]Longino H.E. Science as Social Knowledge: Values and Objectivity in Scientific Inquiry 1990 Princeton University Press
  35. [56]Popper K. Bartley W.W. III Realism and the Aim of Science: From the Postscript to the Logic of Scientific Discovery 2013 Routledge
  36. [58]Wolpert D.H. Macready W.G. No free lunch theorems for optimization IEEE Trans. Evol. Comput. 1 1997 67 82
  37. [61]Gitelman L. Raw Data Is an Oxymoron 2013 MIT Press
  38. [62]Leonelli S. What distinguishes data from models? Eur. J. Philos. Sci. 9 2019 22 30873249 PMC6383958
  39. [63]Bogen J. Empiricism and After 2016 Oxford University Press
  40. [65]Lewontin R.C. The analysis of variance and the analysis of causes Int. J. Epidemiol. 35 2006 520 525 16645033
  41. [67]Pence C.H. “Describing our whole experience”: The statistical philosophies of W. F. R. Weldon and Karl Pearson Stud. Hist. Philos. Biol. Biomed. Sci. 42 2011 475 485 22035721
  42. [68]Weldon W.F.R. Attempt to measure the death-rate due to the selective destruction of Carcinus moenas with respect to a particular dimension Proc. Roy. Soc. Lond. 57 1895 360 379
  43. [69]Salmon W.C. Scientific Explanation and the Causal Structure of the World 1984 Princeton University Press
  44. [70]Potochnik A. Idealization and the Aims of Science 2017 University of Chicago Press
  45. [71]Woodward J. Making Things Happen: A Theory of Causal Explanation 2005 Oxford University Press
  46. [72]Ukai T. Iso H. Yamagishi K. Saito I. Kokubo Y. Yatsuya H. Muraki I. Eshak E.S. Sawada N. Tsugane S. Habitual tub bathing and risks of incident coronary heart disease and stroke Heart 106 2020 732 737 32209614
  47. [74]Pellicano E. Psychological models of autism: An overview Roth I. Rezaie P. Researching the Autism Spectrum: Contemporary Perspectives 2011 Cambridge University Press 219 265
  48. [75]LaCroix T. Autism and the pseudoscience of mind Preprint at PhilSci-Archive 2023 https://philsci-archive.pitt.edu/22817/
  49. [76]Wu X. Zhang X. Responses to critiques on machine learning of criminality perceptions Preprint at arXiv 2016 10.48550/arXiv.1611.04135 · DOI
  50. [77]Wang Y. Wu H. Sun Z.S. The biological basis of sexual orientation: How hormonal, genetic, and environmental factors influence to whome we are sexually attracted Front. Neuroendocrinol. 55 2019 100798 10.1016/j.yfrne.2019.100798 31593707 · DOI
  51. [78]Lipton Z.C. Steinhardt J. Troubling trends in machine learning scholarship Preprint at arXiv 2018 10.48550/arXiv.1807.03341 · DOI
  52. [81]Bailey J.M. Pillard R.C. A genetic study of male sexual orientation Arch. Gen. Psychiatr. 48 1991 1089 1096 1845227
  53. [88]Jacobs A.Z. Measurement as governance in and for responsible AI Preprint at arXiv 2021 10.48550/arXiv.2109.05658 · DOI
  54. [89]Kapoor S. Cantrell E. Peng K. Pham T.H. Bail C.A. Gundersen O.E. Hofman J. Hullman J. Lones M.A. Malik M.M. Reforms: Consensus-based recommendations for machine-learning-based science Sci. Adv. 10 2024 eadk3452 PMC11092361 38691601
  55. [90]Guest O. What makes a good theory, and how do we make a theory good? Comput. Brain Behav. 2024 10.1007/s42113-023-00193-2 PMC13298656 42459536 · DOI
  56. [91]Harvey E. Sandhaus H. Jacobs A.Z. Moss E. Sloane M. The cadaver in the machine: The social practices of measurement and validation in motion capture technology Preprint at arXiv 2024 10.48550/arXiv.2401.10877 · DOI

Authors
Andrews, M., Smart, A., & Birhane, A.
Affiliations
  • University of Cincinnati (US)
  • Google (United States) (US)
  • Mozilla Foundation (US)
Corresponding author
Smart, Andrew
ORCID
Venue
Patterns
Publisher
Elsevier BV
Published
2024-09
Volume
5
Issue
9
Pages
101027
ISSN
2666-3899 (ISSN-L)
DOI
10.1016/j.patter.2024.101027
PMC
PMC11573791
OpenAlex
W4401215755
Cited by
18
References
70
Citations by year
2026: 9 · 2025: 8 · 2024: 1
FWCI
1.37
Type
journal article
Language
en
Open access
Licence
Locations
  • Patterns (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
Colorado Mesa University
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Safety Research › Academic integrity and plagiarism
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
Keywords
pseudoscience, junk science, machine learning ethics, physiognomy, facial analysis, facial expression recognition, epistemic validity, research integrity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review