Selecting the “Best” Regression Equation

Draper, Norman R.; Smith, Harry

doi:10.1002/9781118625590.ch15

Cited by 61Statistical Methods and InferenceStatistical Methods and ApplicationsAdvanced Statistical Methods and Models

Selecting the “Best” Regression Equation

Draper, Norman R., Smith, Harry·1998·Wiley Series in Probability and Statistics

DOI 10.1002/9781118625590.ch15

Open this paper in CoChat — free account

Abstract

This chapter talks about some specific statistical procedures for selecting variables in regression. It discusses: all possible regressions using three criteria: R2, s2, and the Mallows Cp\, and the best subset regression using R2, R2 (adjusted), and Cp\. The chapter also focuses on stepwise regression, backward elimination, and some variations on previous methods. It explains the uses of the R2 statistic, residual mean square and Mallows Cp Statistic. The stepwise regression procedure starts off by choosing an equation containing the single best X variable and then attempts to build up with subsequent additions of X”s one at a time as long as these additions are worthwhile. The Hald data is used to illustrate the workings of the stepwise procedure. The backward elimination method is also an economical procedure. It tries to examine only the “best” regressions containing a certain number of variables.

Integrity signals

Registry

Crossref · 10.1002/9781118625590.ch15

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Stepwise backwards selection was applied to preserve statistical power given the high number of candidate variables relative to sample size.
“Given the high number of candidate variables relative to the sample size, we apply stepwise backwards selection procedure to preserve statistical power [36].”
From Students under lockdown: Comparisons of students’ social networks and mental health before and during the COVID-19 crisis in Switzerland (2020) · cites this paper as [36]

Sorted by citations

Students under lockdown: Comparisons of students’ social networks and mental health before and during the COVID-19 crisis in Switzerland
Elmer, T., Mepham, K., & Stadtfeld, C. (2020) · PLOS ONE · doi:10.1371/journal.pone.0236337 · Cited by 1,421 · see citing works
“Given the high number of candidate variables relative to the sample size, we apply stepwise backwards selection procedure to preserve statistical power [36].”
Method
Students under lockdown: Comparisons of students’ social networks and mental health before and during the COVID-19 crisis in SwitzerlandPreprint
Elmer, T., Mepham, K., & Stadtfeld, C. (2020) · doi:10.31234/osf.io/ua6tq · Cited by 292 · see citing works
Effects of Small Ischemic Lesions in the Primary Motor Cortex on Neurophysiological Organization in Ventral Premotor Cortex
Dancause, N., Barbay, S., Frost, S. B., Zoubina, E. V., Plautz, E. J., Mahnken, J. D., & Nudo, R. J. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00792.2006 · Cited by 108 · see citing works
Heat and water stressed field-grown soybean: A multivariate study on the relationship between physiological-biochemical traits and yield
Ergo, V. V., Lascano, R., Vega, C. R. C., Parola, R., & Carrera, C. S. (2017) · Environmental and Experimental Botany · doi:10.1016/j.envexpbot.2017.12.023 · Cited by 98 · see citing works
Detecting creeping thistle in sugar beet fields using vegetation indices
Kazmi, W., García-Ruíz, F., Nielsen, J., et al. (2015) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2015.01.008 · Cited by 86 · see citing works
“A model was created by adding and removing features one by one given the enter and exit tolerances of 0.1 as proposed by Draper and Smith (1998).”
MethodInfluential
Characterizing the Performance of Interstate Flexible Pavements Using Artificial Neural Networks and Random Parameters Regression
Yamany, M. S., Saeed, T. U., Volovski, M., et al. (2020) · Journal of Infrastructure Systems · doi:10.1061/(asce)is.1943-555x.0000542 · Cited by 83 · see citing works
The key kinematic determinants of undulatory underwater swimming at maximal velocity
Connaboy, C., Naemi, R., Brown, S., Psycharakis, S. G., McCabe, C. B., Coleman, S. G. S., & Sanders, R. H. (2015) · Journal of Sports Sciences · doi:10.1080/02640414.2015.1088162 · Cited by 36 · see citing works
“Backward elimination (BE) ANCOVA models were utilised to ascertain which individual kinematic variable(s) provided the best predictive models for each of the three DVs for all participants (Draper & Smith, 1998).”
Method
A History of Regression and Related Model-Fitting in the Earth Sciences (1636?-2000)
Howarth, R. J. (2001) · Natural Resources Research · doi:10.1023/a:1013928826796 · Cited by 35 · see citing works
Data-driven farm-wide fatigue estimation on jacket-foundation OWTs for multiple SHM setups
Santos, F. D. N., Noppe, N., Weijtjens, W., & Devriendt, C. (2022) · Wind energy science · doi:10.5194/wes-7-299-2022 · Cited by 31 · see citing works
Sugarcane yields prediction at the row level using a novel cross-validation approach to multi-year multispectral images
Akbarian, S., Xu, C., Wang, W., et al. (2022) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2022.107024 · Cited by 31 · see citing works
A genesis potential index for W estern N orth P acific tropical cyclones by using oceanic parameters
Zhang, M., Zhou, L., Chen, D., et al. (2016) · Journal of Geophysical Research Oceans · doi:10.1002/2016jc011851 · Cited by 31 · see citing works
Comparison of signalment and computed tomography findings in French Bulldogs, Pugs, and English Bulldogs with and without clinical signs associated with thoracic hemivertebra
Decker, S. D., Packer, R. M. A., Cappello, R., et al. (2019) · Journal of Veterinary Internal Medicine · doi:10.1111/jvim.15556 · Cited by 30 · see citing works
“05 cutoff).(28) Odd ratios (OR) of significant variables were inspected to determine if clinical signs were more or less likely to occur in association with each variable.”
Background
Modeling the Aerobic Growth and Decline of Staphylococcus aureus as Affected by pH and Potassium Sorbate Concentration
Giannuzzi, L., Contreras, E. M., & Zaritzky, N. E. (1999) · Journal of Food Protection · doi:10.4315/0362-028x-62.4.356 · Cited by 29 · see citing works
Energy performance analysis of continuous processes using surrogate models
Beisheim, B., Rahimi‐Adli, K., Krämer, S., et al. (2019) · Energy · doi:10.1016/j.energy.2019.05.176 · Cited by 29 · see citing works
Relation Between Inpatient and Postdischarge Services and Outcomes 1 Year Postinjury in People With Traumatic Spinal Cord Injury
Backus, D. A., Gassaway, J. A., Smout, R. J., Hsieh, C. J., Heinemann, A. W., DeJong, G., & Horn, S. D. (2013) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2013.01.012 · Cited by 20 · see citing works
An impact of economic slowdown on health. New evidence from 21 European countries
Prędkiewicz, P., Bem, A., Siedlecki, R., Kowalska, M., & Robakowska, M. (2022) · BMC Public Health · doi:10.1186/s12889-022-13740-6 · Cited by 18 · see citing works
Diagnostics-Driven Nonstationary Emulators Using Kernel Mixtures
Volodina, V., & Williamson, D. (2020) · SIAM/ASA Journal on Uncertainty Quantification · doi:10.1137/19m124438x · Cited by 19 · see citing works
Pepper morphological traits related with resistance to Phytophthora capsici
Egea‐Gilabert, C., Bilotti, G., Requena, M. E., et al. (2008) · Biologia Plantarum · doi:10.1007/s10535-008-0019-2 · Cited by 17 · see citing works
Deep-Manager: a versatile tool for optimal feature selection in live-cell imaging analysis
Mencattini, A., D’Orazio, M., Casti, P., Comes, M. C., Giuseppe, D. D., Antonelli, G., Filippi, J., Corsi, F., Ghibelli, L., Veith, I., et al. (2023) · Communications Biology · doi:10.1038/s42003-023-04585-9 · Cited by 15 · see citing works
“● METHOD 6: stepwise linear regression ﬁ t for feature selection 14 .”
Method
Microcirculation assessment using an individualized model for diffuse reflectance spectroscopy and conventional laser Doppler flowmetry
Tomas Strömberg, A., Jonasson, H., Fredriksson, I., Nyström, F. H., & Larsson, M. (2014) · Journal of Biomedical Optics · doi:10.1117/1.jbo.19.5.057002 · Cited by 15 · see citing works
“This test comparing residual spectra for different models is described elsewhere.(21)”
Method
Multivariate Mixed Linear Model Analysis of Longitudinal Data: An Information-Rich Statistical Technique for Analyzing Plant Disease Resistance
Veturi, Y., Kump, K. L., Walsh, E., et al. (2012) · Phytopathology · doi:10.1094/phyto-10-11-0268 · Cited by 11 · see citing works
Regression
Carè, A., & Camporeale, E. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-811788-0.00004-4 · Cited by 9 · see citing works
Elemental diet therapy plays a significant role in preventing surgical recurrence of Crohn’s disease in the era of biologics
Shinozaki, M., Yokoyama, T., Saigusa, N., Satō, H., Yazawa, K., Tsurita, G., Kurokawa, T., Hata, K., & Yokoyama, Y. (2020) · Surgery Today · doi:10.1007/s00595-020-02112-5 · Cited by 8 · see citing works
Arm impairment and walking speed explain real-life activity of the affected arm and leg after stroke
Andersson, S., Danielsson, A., Ohlsson, F., Wipenmyr, J., & Murphy, M. A. (2021) · Journal of Rehabilitation Medicine · doi:10.2340/16501977-2838 · Cited by 8 · see citing works
Proteins Associated with Thermally Induced Gelation of Turkey Breast Meat
Updike, M. S., Zerby, H. N., Utrata, K. L., et al. (2006) · Journal of Food Science · doi:10.1111/j.1750-3841.2006.00184.x · Cited by 8 · see citing works
Saybolt color prediction for condensates and light crude oils
Leam, J. J., Khor, C. S., & Dass, S. C. (2020) · Journal of Petroleum Exploration and Production Technology · doi:10.1007/s13202-020-01031-y · Cited by 7 · see citing works
Predicting CO2 Solubility in Diverse Ionic Liquids: A Data-Driven Approach Using Machine Learning Algorithms
Bastami, Z., Sobati, M. A., & Amereh, M. (2025) · Energy & Fuels · doi:10.1021/acs.energyfuels.5c01345 · Cited by 6 · see citing works
Data‐driven multi‐objective optimization via grid compatible simplex technique and desirability approach for challenging high throughput chromatography applications
Konstantinidis, S., Welsh, J. P., Titchener-Hooker, N. J., et al. (2018) · Biotechnology Progress · doi:10.1002/btpr.2673 · Cited by 6 · see citing works
Estimation of Work of Breathing from Respiratory Muscle Activity In Spontaneous Ventilation: A Pilot Study
Muñoz, I. C., Hernández, A. M., & Mañanas, M. Á. (2019) · Applied Sciences · doi:10.3390/app9102007 · Cited by 5 · see citing works
Comparison of Different Regularized and Shrinkage Regression Methods to Predict Daily Tropospheric Ozone Concentration in the Grand Casablanca Area
Oufdou, H., Bellanger, L., Bergam, A., Ghaziri, A. E., Khomsi, K., & Qannari, E. M. (2018) · Advances in Pure Mathematics · doi:10.4236/apm.2018.810049 · Cited by 5 · see citing works
Optimal data-driven policies for disease screening under noisy biomarker measurement
Sadeghzadeh, S., Bish, E. K., & Bish, D. R. (2019) · IISE Transactions · doi:10.1080/24725854.2019.1630867 · Cited by 5 · see citing works
“Finally, we choose the variable set with the best performance (i.e., the lowest RMSE) (Draper and Smith, 1998).”
BackgroundMethodInfluential
Brain–computer interface performance analysis of monozygotic twins with discordant hand dominance: A case study
Carino-Escobar, R. I., Galicia-Alvarado, M. A., Marrufo, O., Carrillo‐Mora, P., & Cantillo-Negrete, J. (2020) · Laterality Asymmetries of Body Brain and Cognition · doi:10.1080/1357650x.2019.1710525 · Cited by 4 · see citing works
“Associations between twins’ BCI performance with visual and somatosensory feedback, with KVIQ results, subcortical structures’ volumes, and ERD/ ERS were assessed using a stepwise multiple linear regression analysis (Draper & Smith, 1998).”
Method
Bilingualism Predicts Affective Theory of Mind in Autistic Adults
Cummings, K. K., Greene, R. K., Cernasov, P., Kan, D. D. D., Parish‐Morris, J., Dichter, G. S., & Kinard, J. L. (2024) · Journal of Speech Language and Hearing Research · doi:10.1044/2024_jslhr-23-00431 · Cited by 4 · see citing works
DoE-Assisted Development of a Novel Glycosaminoglycan-Based Injectable Formulation for Viscosupplementation
Cicognani, M., Rossi, S., Vecchi, G., et al. (2020) · Pharmaceutics · doi:10.3390/pharmaceutics12070681 · Cited by 4 · see citing works
Predicting the onset of hypertension for workers: does including work characteristics improve risk predictive accuracy?
Chantarat, T., McGovern, P. M., Enns, E. A., et al. (2022) · Journal of Human Hypertension · doi:10.1038/s41371-022-00666-0 · Cited by 3 · see citing works
Multi-Space Seasonal Precipitation Prediction Model Applied to the Source Region of the Yangtze River, China
Du, Y., Berndtsson, R., An, D., et al. (2019) · Water · doi:10.3390/w11122440 · Cited by 3 · see citing works
“The least-squares method was employed to calibrate the parameters and a stepwise method [34] was used to further select qualified predictors from all potential predictors.”
Method
HONEYCOMBING AND DEFORMATION OF SIX WOOD SPECIES AND THEIR RELATIONSHIP WITH SEVERAL PHYSICAL
Yuniarti, K., Basri, E., & Abdurachman, A. (2017) · Jurnal Penelitian Hasil Hutan · doi:10.20886/jphh.2017.35.2.115-122 · Cited by 2 · see citing works
“For regression analysis purpose, the procedure used is “all possible regression” considering that the number of predictor/independent variables is only three (Draper & Smith, 1966).”
MethodInfluential
Predictive Models from Statistically Nonconforming Databases
Fritz, W. P., Igusa, T., & Jones, N. P. (2009) · Journal of Structural Engineering · doi:10.1061/(asce)0733-9445(2009)135:5(567 · Cited by 2 · see citing works
Identification of Technological Effectiveness for Mechanized Harvesting of Red Currant Cultivars Based on Bush Morphology and Mechanical Fruit ParametersPreprint
Panfilova, O., Kahramanoğlu, İ., Ondrašek, G., Ryago, N., Tsoy, M., Seguel, A., Meier, S., Okatan, V., Usanmaz, S., & Helvacı, M. (2023) · Preprints.org · doi:10.20944/preprints202302.0087.v1 · Cited by 2 · see citing works
Çoklu bağlantı sorununda Ridge, Liu ve LASSO tahmin edicilerinin kullanımı: yumurta tavukçuluğunda bir uygulaması
Yener, A., Sayım, A. A., & Akçay, A. (2025) · Veteriner Hekimler Derneği Dergisi · doi:10.33188/vetheder.1554543 · Cited by 1 · see citing works
Time to Treatment Initiation for the 30 Most Prevalent Cancer Types: Trends and Predictors of Change
Dafflisio, G. J., Wang, M., Wang, X., Spratt, D. E., Zuhour, R. J., Khorana, A. A., Bilimoria, K. Y., & Zaorsky, N. G. (2025) · JCO Oncology Practice · doi:10.1200/op.23.00614 · Cited by 3 · see citing works
Echocardiography can accurately estimate pulmonary artery wedge pressure without left atrial volume information—diagnostic and prognostic performance
Lindow, T., Manouras, A., Strange, G. A., et al. (2025) · European Heart Journal - Imaging Methods and Practice · doi:10.1093/ehjimp/qyaf082 · Cited by 1 · see citing works
A Stepwise Approach of Factors Affecting Agricultural Production in Nigeria
Edem, U. A. (2020) · Science Journal of Business and Management · doi:10.11648/j.sjbm.20200803.17 · Cited by 1 · see citing works
“While tree crops production are determined by natural factors, internal migrations, agricultural innovation diffusion and existence of marketing board [17].”
Background
Backward Stepwise Modelling for Unemployment Rate in Malaysia
Mansor, M. M., Mostapar, N. F., Hasan, N., et al. (2023) · IEEE International Conference on Computing (ICOCO) · doi:10.1109/icoco59262.2023.10397764 · Cited by 1 · see citing works
“Backward stepwise regression is also an economic methodology [42].”
Method
Interferometric, absolute sensor of angular micro-displacement
Dobosz, M., & Jankowski, M. C. (2021) · Precision Engineering · doi:10.1016/j.precisioneng.2021.04.018 · Cited by 1 · see citing works
Effect of extraction conditions on the extraction yield, swelling capacity, emulsion capacity and emulsion stability of psyllium husk seed mucilage extracted from Plantago ovatePreprint
Assi, A. G., Kalkan, E., & Maskan, M. (2023) · Research Square · doi:10.21203/rs.3.rs-3610684/v1 · Cited by 1 · see citing works
Ventral Premotor Cortex Motor Cortex on Neurophysiological Organization in Effects of Small Ischemic Lesions in the Primary
Dancause, N., Barbay, S., Frost, S. B., et al. (2015)
The Relationship of Psychotherapist Demographic Variables to Discipline Board Misconduct ContentPreprint
Ström, E. L., & Dykeman, C. (2020) · doi:10.31234/osf.io/wt5au
Comment on wes-2021-76
Santos, F. D. N., Noppe, N., Weijtjens, W., et al. (2021) · doi:10.5194/wes-2021-76-rc2
Reply on AC1
Santos, F. D. N. (2021) · doi:10.5194/wes-2021-76-ac4
The Efficient Measurement of Resource Utilisation in Clinical Trials in Critical Care
Delaney, A. P. (2014) · The Sydney eScholarship Repository (The University of Sydney)
Estimation of main carcass components by using bootstrapping regression method
., S. S., & Görgülü, M. (2003) · Journal of Animal and Feed Sciences · doi:10.22358/jafs/67766/2003
Reply on RC3
Santos, F. D. N. (2021) · doi:10.5194/wes-2021-76-ac3
Reply on RC2
Santos, F. D. N. (2021) · doi:10.5194/wes-2021-76-ac2
Reply on RC1
Santos, F. D. N. (2021) · doi:10.5194/wes-2021-76-ac1
Comment on wes-2021-76
Santos, F. D. N., Noppe, N., Weijtjens, W., et al. (2021) · doi:10.5194/wes-2021-76-rc1
Comment on wes-2021-76
d N Santos, F., Noppe, N., Weijtjens, W., et al. (2021) · doi:10.5194/wes-2021-76-rc3
Role of amorphous phase, temperature fluctuations, and vacuum, on the performance of alkali-activated lunar regolith simulants
Driouich, A., Goudarzi, A., Santoro, F., Dalconi, M. C., Dau, M., Bettanini, C., Pilehvar, S., & Valentini, L. (2026) · Materials and Structures · doi:10.1617/s11527-026-03243-w

Authors: Draper, N. R., & Smith, H.
Venue: Wiley Series in Probability and Statistics
Publisher: Wiley
Published: 1998-04-09
Pages: 327-368
ISSN: 1940-6347 (ISSN-L)
DOI: 10.1002/9781118625590.ch15
OpenAlex: W1555441379
Cited by: 61
References: 0
Citations by year: 2026: 1 · 2025: 6 · 2024: 1 · 2023: 4 · 2022: 4 · 2021: 10 · 2020: 9 · 2019: 5 · 2018: 3 · 2017: 2 · 2016: 1 · 2015: 3 · 2014: 2 · 2013: 1 · 2012: 1
Type: other
Language: en
Open access: closed
Licence: doi.wiley.com/10.1002/tdm_license_1.1 (tdm)
Locations: Wiley series in probability and statistics (publishedVersion)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
Keywords: feature selection, regression analysis, best subset regression, adjusted R-squared, stepwise regression, backward elimination

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Selecting the “Best” Regression Equation

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cited by

Related

Record

About these checks