Alchemy in the behavioral sciences
Abstract
Access to powerful new computers has encouraged routine use of highly complex analytic techniques, often in the absence of any theory, hypotheses, or model to guide the researcher's expectations of results. The author examines the potential of such techniques for generating spurious results, and urges that in exploratory work the outcome be subjected to a more rigorous criterion than the usual tests of statistical significance.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Einhorn concluded that the modern researcher cannot rely on the computer to turn data into meaningful and valuable scientific information.
“Einhorn (1972) concluded, 'Just as the alchemists were not successful in turning base metal into gold, the modern researcher cannot rely on the 'computer' to turn his data into meaningful and valuable scientific information.'”
From Illusions in Regression Analysis (2011) · cites this paper as Einhorn (1972)
Cites
Still being checked.
Cited by
Sorted by citations
An exploratory technique for investigating large quantities of categorical data
Kass, G. V. (1980) · Applied Statistics · doi:10.2307/2986296 · Cited by 2,880 · see citing works
Hair, J. F., Sarstedt, M., Pieper, T. M., & Ringle, C. M. (2012) · Long Range Planning · doi:10.1016/j.lrp.2012.09.008 · Cited by 2,454 · see citing works
Fifty Years of Classification and Regression Trees
Loh, W. (2014) · International Statistical Review · doi:10.1111/insr.12016 · Cited by 665 · see citing works
A method of choosing multiway partitions for classification and decision trees
Biggs, D., de Ville, B., & Suen, E. (1991) · Journal of Applied Statistics · doi:10.1080/02664769100000005 · Cited by 574 · see citing works
Psychographics: A Critical Review
Wells, W. D. (1975) · Journal of Marketing Research · doi:10.1177/002224377501200210 · Cited by 529 · see citing works
CART (Classification and Regression Trees)
(2004) · The SAGE Encyclopedia of Social Science Research Methods · doi:10.4135/9781412950589.n88 · Cited by 430 · see citing works
Illusions in regression analysis
Armstrong, J. S. (2012) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2012.02.001 · Cited by 327 · see citing works
Nelson, M. W., & Tan, H. (2005) · Auditing A Journal of Practice & Theory · doi:10.2308/aud.2005.24.supplement.41 · Cited by 305 · see citing works
Multiple Comparisons in Induction Algorithms
Jensen, D. D., & Cohen, P. R. (2000) · Machine Learning · doi:10.1023/a:1007631014630 · Cited by 241 · see citing works
The use of classification and regression trees in clinical epidemiology
Marshall, R. J. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00344-9 · Cited by 213 · see citing works
Landers, R. N., Brusso, R. C., Cavanaugh, K. J., & Collmus, A. B. (2016) · Psychological Methods · doi:10.1037/met0000081 · Cited by 195 · see citing works
CART: Classification and Regression Trees
Steinberg, D. (2009) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420089653-17 · Cited by 166 · see citing works
DISTINGUISHING FACTS AND ARTIFACTS IN GROUP‐BASED MODELING*
Skarðhamar, T. (2010) · Criminology · doi:10.1111/j.1745-9125.2010.00185.x · Cited by 141 · see citing works
Nelson, M., & Tan, H. (2005) · Auditing A Journal of Practice & Theory · doi:10.2308/aud.2005.24.s-1.41 · Cited by 130 · see citing works
Wright, J. D., & Wright, S. R. (1976) · American Sociological Review · doi:10.2307/2094258 · Cited by 122 · see citing works
Brighton, H., & Gigerenzer, G. (2015) · Journal of Business Research · doi:10.1016/j.jbusres.2015.01.061 · Cited by 110 · see citing works
Nelson, M. W., & Tan, H. (2005) · SSRN Electronic Journal · doi:10.2139/ssrn.761706 · Cited by 88 · see citing works
Steinbach, M. S., & Tan, P.-N. (2009) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420089653-15 · Cited by 87 · see citing works
A survey of human judgement and quantitative forecasting methods
Zellner, M., Abbas, A. E., Budescu, D., & Galstyan, A. (2021) · Royal Society Open Science · doi:10.1098/rsos.201187 · Cited by 63 · see citing works
Illusions in Regression Analysis
Armstrong, J. S. (2011) · doi:10.2139/ssrn.1969740 · Cited by 73 · see citing works
Xue, H., Yang, Q., & Chen, S. (2009) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420089653-10 · Cited by 75 · see citing works
A Psychographic Classification of Vacation Life Styles
Perreault, W. D., Darden, D. K., & Darden, W. R. (1977) · Journal of Leisure Research · doi:10.1080/00222216.1977.11970330 · Cited by 61 · see citing works
Prognostic Factors in Mammary Carcinoma
Wallgren, A., Silfverswärd, C., & EKLUND, G. A. (1976) · Acta Radiologica Therapy Physics Biology · doi:10.3109/02841867609132703 · Cited by 58 · see citing works
Zinkhan, G. M., & Pereira, A. (1994) · International Journal of Research in Marketing · doi:10.1016/0167-8116(94)90001-9 · Cited by 52 · see citing works
CHAID and Earlier Supervised Tree Methods
Ritschard, G. (2013) · Routledge eBooks · doi:10.4324/9780203403020-11 · Cited by 45 · see citing works
Forecasting methods and principles: Evidence-based checklists
Armstrong, J. S., & Green, K. C. (2018) · Journal of Global Scholars of Marketing Science · doi:10.1080/21639159.2018.1441735 · Cited by 44 · see citing works
Data mining with genetic algorithms on binary trees
Sörensen, K., & Janssens, G. K. (2003) · European Journal of Operational Research · doi:10.1016/s0377-2217(02)00824-x · Cited by 40 · see citing works
Partitioning methods for classification and decision making in medicine
Marshall, R. J. (1986) · Statistics in Medicine · doi:10.1002/sim.4780050516 · Cited by 38 · see citing works
Predictive Modeling of Multivariable and Multivariate Data
Gillo, M. W., & Shelly, M. W. (1974) · Journal of the American Statistical Association · doi:10.1080/01621459.1974.10480182 · Cited by 37 · see citing works
Market Segmentation with Multivariate Aid
MacLachlan, D. L., & Johansson, J. K. (1981) · Journal of Marketing · doi:10.1177/002224298104500108 · Cited by 34 · see citing works
Trotman, K. T. (2005) · Auditing A Journal of Practice & Theory · doi:10.2308/aud.2005.24.s-1.73 · Cited by 34 · see citing works
Blind Analysis as a Correction for Confirmatory Bias in Physics and in Psychology
MacCoun, R. J., & Perlmutter, S. (2017) · doi:10.1002/9781119095910.ch15 · Cited by 30 · see citing works
Applications of the General Linear Model to Survey Data
Berk, R. A. (1983) · Elsevier eBooks · doi:10.1016/b978-0-12-598226-9.50019-7 · Cited by 22 · see citing works
Attitudes towards deafness: hearing students at a hearing and deaf college.
Emerton, R. G., & Rothman, G. (1978) · PubMed · Cited by 20 · see citing works
Research Methodology in Communication: Review and Commentary
Cappella, J. N. (1977) · Annals of the International Communication Association · doi:10.1080/23808985.1977.11923669 · Cited by 20 · see citing works
The Vagaries and Vulgarities of "Scientific" Jury Selection
Berk, R. A., Hennessy, M., & Swan, J. (1977) · Evaluation Review · doi:10.1177/0193841x7700100106 · Cited by 19 · see citing works
McKenzie, D. P., & Low, L. H. (1992) · Computers in Human Behavior · doi:10.1016/0747-5632(92)90001-u · Cited by 17 · see citing works
Data Mining at the Interface of Computer Science and Statistics
Smyth, P. (2001) · Massive computing · doi:10.1007/978-1-4615-1733-7_3 · Cited by 14 · see citing works
Discretionary Methodological Decisions in Applied Research
Berk, R. A. (1977) · Sociological Methods & Research · doi:10.1177/004912417700500303 · Cited by 12 · see citing works
Applying statistical knowledge to database analysis and knowledge base construction
de Ville, B. (2002) · Conference on Artificial Intelligence for Applications · doi:10.1109/caia.1990.89168 · Cited by 10 · see citing works
An Assessment of the Contribution of Log Linear Models to Marketing Research
Blattberg, R. C., & Dolan, R. J. (1981) · Journal of Marketing · doi:10.1177/002224298104500209 · Cited by 10 · see citing works
Cross-Validatory Selection of Binary Variables in Differential Diagnosis
Mabbett, A. J., Stone, M., & Washbrook, J. (1980) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2986306 · Cited by 9 · see citing works
Toward a New Payment System for Inpatient Rehabilitation
Saitto, C., Marino, C., Fusco, D., Arcà, M., & Perucci, C. A. (2005) · Medical Care · doi:10.1097/01.mlr.0000173567.80190.32 · Cited by 8 · see citing works
Fildes, R. A., & Petropoulos, F. (2015) · Journal of Business Research · doi:10.1016/j.jbusres.2015.01.059 · Cited by 6 · see citing works
Gini’s mean difference offers a response to Leamer’s critique
Yitzhaki, S. (2015) · METRON · doi:10.1007/s40300-014-0057-9 · Cited by 6 · see citing works
Betting After the Race Is Over. The Perils of Post Hoc Hypothesizing
Payne, J. L., & Dyer, J. (1975) · American Journal of Political Science · doi:10.2307/2110545 · Cited by 5 · see citing works
Machine Learning and Forecasting: A Review
Potgieter, P. H. (2020) · Springer eBooks · doi:10.1007/978-3-030-40601-1_8 · Cited by 4 · see citing works
Benefit- und Lifestyle-Segmentierung
Balderjahn, I., & Scholderer, J. (2007) · Gabler Verlag eBooks · doi:10.1007/978-3-8349-9517-9_12 · Cited by 4 · see citing works
Ignorance, Arrogance, and Social Psychology: A Response to Helmreich
Thorngate, W. (1976) · Personality and Social Psychology Bulletin · doi:10.1177/014616727600200211 · Cited by 4 · see citing works
Segmented Linear Regression Trees
Zheng, X., & Chen, S. X. (2025) · Acta Mathematica Sinica English Series · doi:10.1007/s10114-025-3349-5 · Cited by 2 · see citing works
Fiendishly Difficult Questions: Possible Limits and Aesthetic Pleasures of Simulation
Thorngate, W. (2011) · Simulation & Gaming · doi:10.1177/1046878110396005 · Cited by 2 · see citing works
Predictive Models in Human Development Planning
Mishra, M., & Chatterjee, S. (2020) · Springer eBooks · doi:10.1007/978-981-15-4083-7_8 · Cited by 1 · see citing works
Analysis of Company Growth Data Using Genetic Algorithms on Binary Trees
Janssens, G. K., Sösrensen, K., Limère, A., & Vanhoof, K. (2005) · Lecture notes in computer science · doi:10.1007/11430919_29 · Cited by 1 · see citing works
Knowledge evaluation: statistical evaluations
Jensen, D. D. (2002) · Oxford University Press eBooks · Cited by 1 · see citing works
INSTABILITY IN A TREE APPROACH TO REGRESSION
Kim, S. (1992) · ETS Research Report Series · doi:10.1002/j.2333-8504.1992.tb01431.x · Cited by 1 · see citing works
Fienberg, S. E. (1975) · Social Indicators Research · doi:10.1007/bf00300474 · Cited by 1 · see citing works
Hamerle, A. (1983) · University of Regensburg Publication Server (University of Regensburg) · doi:10.5283/epub.27231
EMPLOYING FINANCIAL ACCOUNTING STANDARDS FOR ENTITIES WITHOUT PUBLIC ACCOUNTABILITY FOR AUDITING
Akie Rusaktiva., R., & Nugraha, A. P. (2013) · EL MUHASABA Jurnal Akuntansi (e-Journal) · doi:10.18860/em.v3i2.2339
Reliability of the Aid Analysis in Samples from Fixed Populations
Kuusela, V., & Kronström, M. (1982) · Lecture notes in medical informatics · doi:10.1007/978-3-642-93201-4_113
The Use of Data in Developing Theory: Scientifically Acceptable or Alchemy?
Hunt, K. A. (2015) · Developments in marketing science: proceedings of the Academy of Marketing Science · doi:10.1007/978-3-319-17046-6_91
Shaking the trees: Abilities and Capabilities of Regression and Decision Trees for Political Science
Waldhauser, C., & Hochreiter, R. (2017) · ITM Web of Conferences · doi:10.1051/itmconf/20171400009
Machine learning approaches for astrophysics and cosmology
Morice-Atkinson, X. (2019)
Visualization with Various Tree-based Regression Results in Health Information Data
Cho, H. S., & Lee, E. K. (2020) · Journal of Health Informatics and Statistics · doi:10.21032/jhis.2020.45.1.16
Gini's Mean Difference Offers a Response to Leamer's CritiquePreprint
Yitzhaki, S. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2410750
PREDICTIVE ANALYTICS FOR OPTIMIZING PRODUCT DEMAND AND SALES
(2024) · International Research Journal of Modernization in Engineering Technology and Science · doi:10.56726/irjmets60467
- Authors
- Einhorn, H. J.
- Affiliations
- University of Chicago (US)
- Corresponding author
- Einhorn, Hillel J.
- Venue
- Public Opinion Quarterly
- Publisher
- Oxford University Press (OUP)
- Published
- 1972
- Volume
- 36
- Issue
- 3
- Pages
- 367
- ISSN
- 0033-362X (ISSN-L)
- OpenAlex
- W2011577343
- Cited by
- 65
- References
- 0
- Citations by year
- 2025: 1 · 2024: 1 · 2021: 1 · 2020: 3 · 2019: 1 · 2018: 1 · 2017: 2 · 2016: 1 · 2015: 4 · 2014: 2 · 2013: 2 · 2012: 2
- FWCI
- 2.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Public Opinion Quarterly (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- exploratory analysis, spurious results, statistical significance testing, hypothesis testing, behavioral research methodology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review