Cited by 490Advanced Causal Inference TechniquesMeta-analysis and systematic reviewsStatistical Methods and Bayesian Inference

The DOI matches its Crossref record; 54 of the 55 references it cites were found in a registry or an index.

The importance of covariate selection in controlling for selection bias in observational studies.

Steiner, Peter M., Cook, Thomas D., Shadish, William R., Clark, M. H.·2010·Psychological Methods

DOI 10.1037/a0018719

Open this paper in CoChat — free account

Abstract

The assumption of strongly ignorable treatment assignment is required for eliminating selection bias in observational studies. To meet this assumption, researchers often rely on a strategy of selecting covariates that they think will control for selection bias. Theory indicates that the most important covariates are those highly correlated with both the real selection process and the potential outcomes. However, when planning a study, it is rarely possible to identify such covariates with certainty. In this article, we report on an extensive reanalysis of a within-study comparison that contrasts a randomized experiment and a quasi-experiment. Various covariate sets were used to adjust for initial group differences in the quasi-experiment that was characterized by self-selection into treatment. The adjusted effect sizes were then compared with the experimental ones to identify which individual covariates, and which conceptually grouped sets of covariates, were responsible for the high degree of bias reduction achieved in the adjusted quasi-experiment. Such results provide strong clues about preferred strategies for identifying the covariates most likely to reduce bias when planning a study and when the true selection process is not known.

Integrity signals

Registry

Crossref · 10.1037/a0018719

Cites

54 of 54 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Untitled work
(2002) · Cited by 13,211
Self-citation
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Missing data: our view of the state of the art
Schafer, J. L., & Graham, J. W. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.2.147 · Cited by 11,638
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436
The development of markers for the Big-Five factor structure
Goldberg, L. R. (1992) · Psychological Assessment · doi:10.1037/1040-3590.4.1.26 · Cited by 5,530
Statistics and Causal Inference
Holland, P. W. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478354 · Cited by 5,293
Reducing Bias in Observational Studies Using Subclassification on the Propensity Score
Rosenbaum, P. R., & Rubin, D. B. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478078 · Cited by 3,082
Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs
Dehejia, R. H., & Wahba, S. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10473858 · Cited by 2,395
Using Propensity Scores to Help Design Observational Studies: Application to the Tobacco Litigation
Rubin, D. B. (2001) · Health Services and Outcomes Research Methodology · doi:10.1023/a:1020363010465 · Cited by 2,228 · cited 1×
“After PS-adjustment, standardized mean differences d should be close to zero, and variance ratios v should be close to one (Rubin, 2001).”
Background

Flagged first, then by citations

Show all 55 references

1 of the 55 references couldn’t be matched to a known paper.

Sorted by citations

Endogenous Selection Bias: The Problem of Conditioning on a Collider Variable
Elwert, F., & Winship, C. (2014) · Annual Review of Sociology · doi:10.1146/annurev-soc-071913-043455 · Cited by 1,168
Predictors of early growth in academic achievement: the head-toes-knees-shoulders task
McClelland, M. M., Cameron, C. E., Duncan, R., Bowles, R. P., Acock, A. C., Miao, A., & Pratt, M. E. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00599 · Cited by 597
Revising the JBI quantitative critical appraisal tools to improve their applicability: an overview of methods and the development process
Barker, T. H., Stone, J. C., Sears, K., Klugar, M., Leonardi-Bee, J., Tufanaru, C., Aromataris, E., & Munn, Z. (2023) · JBI Evidence Synthesis · doi:10.11124/jbies-22-00125 · Cited by 471
What Would It Take to Change an Inference? Using Rubin’s Causal Model to Interpret the Robustness of Causal Inferences
Frank, K. A., Maroulis, S. J., Duong, M. Q., & Kelcey, B. M. (2013) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373713493129 · Cited by 406
Using the Propensity Score Method to Estimate Causal Effects
Li, M. (2012) · Organizational Research Methods · doi:10.1177/1094428112447816 · Cited by 382
Meta-analysis of randomised controlled trials testing behavioural interventions to promote household action on climate change
Nisa, C. F., Bélanger, J. J., Schumpe, B. M., & Faller, D. G. (2019) · Nature Communications · doi:10.1038/s41467-019-12457-2 · Cited by 349
Spanking, corporal punishment and negative long-term outcomes: A meta-analytic review of longitudinal studies
Ferguson, C. J. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2012.11.002 · Cited by 295
The impact of entrepreneurship education in high school on long-term entrepreneurial performance
Elert, N., Andersson, F. W., & Wennberg, K. (2015) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2014.12.020 · Cited by 290
Qualitative Research Methods
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.010 · Cited by 279
Research Design in Clinical Psychology
Kazdin, A. E. (2021) · doi:10.1017/9781108993647 · Cited by 240
The Effect of Study Abroad on Intercultural Competence Among Undergraduate College Students
Salisbury, M. H., An, B. P., & Pascarella, E. T. (2013) · NASPA Journal · doi:10.1515/jsarp-2013-0001 · Cited by 250
Water, sanitation and hygiene interventions to combat childhood diarrhoea in developing countries
(3ie), I. I. F. I. E., Waddington, H. S., Snilstveit, B., (3ie), I. I. F. I. E., White, H., (3ie), I. I. F. I. E., & Fewtrell, L. (2012) · doi:10.23846/sr0017 · Cited by 229
The Renaissance of Field Experimentation in Evaluating Interventions
Shadish, W. R., & Cook, T. D. (2008) · Annual Review of Psychology · doi:10.1146/annurev.psych.60.110707.163544 · Cited by 208
Self-citation
Propensity Score Analysis
West, S. G., Cham, H., & Thoemmes, F. J. (2015) · Wiley eBooks · doi:10.1002/9781118625392.wbecp505 · Cited by 185
Sustainable consumption and third-party certification labels: Consumers’ perceptions and reactions
Brach, S., Walsh, G., & Shaw, D. (2017) · European Management Journal · doi:10.1016/j.emj.2017.03.005 · Cited by 203
Theory and Practice in Nonprobability Surveys
Mercer, A. W., Kreuter, F., Keeter, S., & Stuart, E. A. (2016) · Public Opinion Quarterly · doi:10.1093/poq/nfw060 · Cited by 183
Employee adaptive performance and job satisfaction during organizational crisis: the role of self-leadership
Marques‐Quinteiro, P., Vargas, R., Eifler, N., & Curral, L. A. (2018) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2018.1551882 · Cited by 167
Improving classroom quality with the ruler approach to social and emotional learning: Proximal and distal outcomes
Hagelskamp, C., Brackett, M. A., Rivers, S. E., & Salovey, P. (2013) · American Journal of Community Psychology · doi:10.1007/s10464-013-9570-x · Cited by 160
Implications of Small Samples for Generalization: Adjustments and Rules of Thumb
Tipton, E., Hallberg, K., Hedges, L. V., & Chan, W. (2016) · Evaluation Review · doi:10.1177/0193841x16655665 · Cited by 178
On the Importance of Reliable Covariate Measurement in Selection Bias Adjustments Using Propensity Scores
Steiner, P. M., Cook, T. D., & Shadish, W. R. (2011) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998610375835 · Cited by 152
Self-citation
Evaluation of occupational health interventions using a randomized controlled trial: challenges and alternative research designs
Schelvis, R. M. C., Hengel, K. M. O., Alex Burdorf, A., Blatter, B. M., Strijk, J. E., & van der Beek, A. J. (2015) · Scandinavian Journal of Work Environment & Health · doi:10.5271/sjweh.3505 · Cited by 132
Examining moderation analyses in propensity score methods: Application to depression and substance use.
Green, K. M., & Stuart, E. A. (2014) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0036515 · Cited by 132
Chronic absence, eighth-grade achievement, and high school attainment in the Chicago Longitudinal Study.
Smerillo, N. E., Reynolds, A. J., Temple, J. A., & Ou, S.-R. (2018) · Journal of School Psychology · doi:10.1016/j.jsp.2017.11.001 · Cited by 125
Employment, late-life work, retirement, and well-being in Europe and the United States
Nikolova, M., & Graham, C. L. (2014) · IZA Journal of European Labor Studies · doi:10.1186/2193-9012-3-5 · Cited by 126
Is It the Music or Is It Selection Bias? A Nationwide Analysis of Music and Non-Music Students’ SAT Scores
Elpus, K. (2013) · Journal of Research in Music Education · doi:10.1177/0022429413485601 · Cited by 124
Generalist versus specialist CEOs and acquisitions: Two‐sided matching and the impact of CEO characteristics on firm outcomes
Chen, G., Huang, S. Z., Meyer‐Doyle, P., & Mindruta, D. (2020) · Strategic Management Journal · doi:10.1002/smj.3258 · Cited by 117
Case matching and the reduction of selection bias in quasi-experiments: The relative importance of pretest measures of outcome, of unreliable measurement, and of mode of data analysis.
Cook, T. D., & Steiner, P. M. (2010) · Psychological Methods · doi:10.1037/a0018536 · Cited by 117
Self-citation
Drawing Causal Inferences Using Propensity Scores: A Practical Guide for Community Psychologists
Lanza, S. T., Moore, J. E., & Butera, N. M. (2013) · American Journal of Community Psychology · doi:10.1007/s10464-013-9604-4 · Cited by 123
A Primer on Propensity Score Analysis
Shadish, W. R., & Steiner, P. M. (2010) · Newborn and Infant Nursing Reviews · doi:10.1053/j.nainr.2009.12.010 · Cited by 121
Self-citation
Establishing Equivalence: Methodological Progress in Group-matching Design and Analysis
Kover, S. T., & Atwood, A. K. (2013) · American Journal on Intellectual and Developmental Disabilities · doi:10.1352/1944-7558-118.1.3 · Cited by 118
Propensity scores as a basis for equating groups: Basic principles and application in clinical treatment outcome research.
West, S. G., Cham, H., Thoemmes, F. J., Renneberg, B., Schulze, J., & Weiler, M. (2014) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0036387 · Cited by 118
The Effect of High School Shootings on Schools and Student Performance
Beland, L., & Kim, D. (2015) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373715590683 · Cited by 106
Promoting cooperation in innovation ecosystems: evidence from European traditional manufacturing SMEs
Radičić, D., Pugh, G., & Douglas, D. E. (2018) · Small Business Economics · doi:10.1007/s11187-018-0088-3 · Cited by 104
Directional Ordering of Self-Concept, School Grades, and Standardized Tests Over Five Years: New Tripartite Models Juxtaposing Within- and Between-Person Perspectives
Marsh, H. W., Pekrun, R. H., & Lüdtke, O. (2022) · Educational Psychology Review · doi:10.1007/s10648-022-09662-9 · Cited by 108
Assessing mediation using marginal structural models in the presence of confounding and moderation.
Coffman, D. L., & Zhong, W. (2012) · Psychological Methods · doi:10.1037/a0029311 · Cited by 103
A history of meta‐regression: Technical, conceptual, and practical developments between 1974 and 2018
Tipton, E., Pustejovsky, J. E., & Ahmadi, H. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1338 · Cited by 112
One-year outcomes of diabetes self-management training among medicare beneficiaries newly diagnosed with diabetes
Strawbridge, L. M., Lloyd, J. T., Meadow, A., Riley, G. F., & Howell, B. L. (2017) · Medical Care · doi:10.1097/mlr.0000000000000653 · Cited by 96
Openness to Experience as a Predictor and Outcome of Upward Job Changes into Managerial and Professional Positions
Nieß, C., & Zacher, H. (2015) · PLoS ONE · doi:10.1371/journal.pone.0131115 · Cited by 104
Teacher qualification matters: The association between cumulative teacher qualification and students’ educational attainment
Lee, S. W., & Lee, E. A. (2020) · International Journal of Educational Development · doi:10.1016/j.ijedudev.2020.102218 · Cited by 105
Matching and Propensity Scores
Steiner, P. M., & Cook, D. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199934874.013.0013 · Cited by 96
Self-citation
The Impact of First-Year Seminars on College Students’ Life-long Learning Orientations
Padgett, R. D., Keup, J. R., & Pascarella, E. T. (2013) · NASPA Journal · doi:10.1515/jsarp-2013-0011 · Cited by 87
The SAGE International Handbook of Educational Evaluation
Ryan, K. E., & Cousins, J. (2009) · Sage eBooks · doi:10.4135/9781452226606 · Cited by 87
The effects of constraints and mastery on mental and physical health: Conceptual and methodological considerations.
Infurna, F. J., & Mayer, A. (2015) · Psychology and Aging · doi:10.1037/a0039050 · Cited by 90
The First Partnership Experience and Personality Development: A Propensity Score Matching Study in Young Adulthood
Wagner, J., Becker, M., Lüdtke, O., & Trautwein, U. (2015) · Social Psychological and Personality Science · doi:10.1177/1948550614566092 · Cited by 85
Collegial Climate and Novice Teachers’ Intent to Remain Teaching
Pogodzinski, B., Youngs, P. A., & Frank, K. A. (2013) · American Journal of Education · doi:10.1086/673123 · Cited by 85
Propensity score analysis: promise, reality and irrational exuberance
Shadish, W. R. (2012) · Journal of Experimental Criminology · doi:10.1007/s11292-012-9166-8 · Cited by 84
Self-citation
Student Involvement in Ethnic Student Organizations: Examining Civic Outcomes 6 Years After Graduation
Bowman, N. A., Park, J. J., & Denson, N. (2014) · Research in Higher Education · doi:10.1007/s11162-014-9353-8 · Cited by 83
How Bias Reduction Is Affected by Covariate Choice, Unreliability, and Mode of Data Analysis: Results From Two Types of Within-Study Comparisons
Cook, T. D., Steiner, P. M., & Pohl, S. (2009) · Multivariate Behavioral Research · doi:10.1080/00273170903333673 · Cited by 81
Self-citation
PROTOCOL: Interventions for improving learning outcomes and access to education in low‐ and middle‐income countries: a systematic review
Snilstveit, B., Gallagher, E., Phillips, D., Vojtkova, M., Eyers, J., Skaldiou, D., Stevenson, J. L., Bhavsar, A., & Davies, P. (2017) · Campbell Systematic Reviews · doi:10.1002/cl2.176 · Cited by 52
An examination of interactive effects of employees’ warmth and competence and service failure types on customer’s service recovery cooperation intention
Huang, Y., Meng, Z., Gürsoy, D., & Shi, S. (2020) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-01-2020-0028 · Cited by 83
The Millennium Villages Project: a retrospective, observational, endline evaluation
Mitchell, S., Gelman, A., Ross, R., Chen, J., Bari, S., Huynh, U. K., Harris, M. W., Sachs, S. E., Stuart, E. A., Feller, A., et al. (2018) · The Lancet Global Health · doi:10.1016/s2214-109x(18)30065-2 · Cited by 80
A Brief Guide to Decisions at Each Step of the Propensity Score Matching Process
Harris, H. D., & Horst, S. J. (2020) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · doi:10.7275/yq7r-4820 · Cited by 82
Theory and Analysis of Total, Direct, and Indirect Causal Effects
Mayer, A., Thoemmes, F. J., Rose, N., Steyer, R., & West, S. G. (2014) · Multivariate Behavioral Research · doi:10.1080/00273171.2014.931797 · Cited by 78
In transit: The well-being of migrants from transition and post-transition countries
Nikolova, M., & Graham, C. L. (2015) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2015.02.003 · Cited by 83
Effects of Post-Session Administration of Methylene Blue on Fear Extinction and Contextual Memory in Adults With Claustrophobia
Telch, M. J., Bruchey, A. K., Rosenfield, D., Cobb, A. R., Smits, J. A. J., Pahl, S. A., & Gonzalez-Lima, F. (2014) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2014.13101407 · Cited by 77
Future Directions in the Study and Treatment of Parent–Child Separation
Humphreys, K. L. (2018) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2018.1534209 · Cited by 81
Single-Level and Multilevel Mediation Analysis
Tofighi, D., & Thoemmes, F. J. (2013) · The Journal of Early Adolescence · doi:10.1177/0272431613511331 · Cited by 76
Interactivity in online chat: Conversational cues and visual cues in the service recovery process
Huang, Y., Gürsoy, D., Zhang, M., Nunkoo, R., & Shi, S. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2021.102360 · Cited by 73
Effect of early grade retention on school completion: A prospective study.
Hughes, J. N., West, S. G., Kim, H., & Bauer, S. S. (2017) · Journal of Educational Psychology · doi:10.1037/edu0000243 · Cited by 70
Validez Estructurada para una investigación cuasi-experimental de calidad: se cumplen 50 años de la presentación en sociedad de los diseños cuasi-experimentales
Fernández-García, P., Vallejo-Seco, G., Livácic-Rojas, P., & Tuero-Herrero, E. (2014) · Anales de Psicología · doi:10.6018/analesps.30.2.166911 · Cited by 51
Vineland-II adaptive behavior profile of children with attention-deficit/hyperactivity disorder or specific learning disorders
Balboni, G., Incognito, O., Belacchi, C., Bonichini, S., & Cubelli, R. (2017) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2016.12.003 · Cited by 69
Improving Causal Inference: Recommendations for Covariate Selection and Balance in Propensity Score Methods
Kainz, K., Greifer, N., Givens, A., Swietek, K., Lombardi, B. M., Zietz, S., & Kohn, J. L. (2017) · Journal of the Society for Social Work and Research · doi:10.1086/691464 · Cited by 71
The impact of technological and non-technological innovations on export intensity in SMEs
Radičić, D., & Djalilov, K. (2019) · Journal of Small Business and Enterprise Development · doi:10.1108/jsbed-08-2018-0259 · Cited by 63
STI and DUI innovation modes in micro-, small-, medium- and large-sized firms: distinctive patterns across Europe and the U.S.
Parrilli, M. D., & Radičić, D. (2020) · European Planning Studies · doi:10.1080/09654313.2020.1754343 · Cited by 62
Unbiased Causal Inference From an Observational Study: Results of a Within-Study Comparison
Pohl, S., Steiner, P. M., Eisermann, J., Soellner, R., & Cook, T. D. (2009) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373709343964 · Cited by 64
Self-citation
Control variables and causal inference: a question of balance
York, R. (2018) · International Journal of Social Research Methodology · doi:10.1080/13645579.2018.1468730 · Cited by 62
What Works for Whom? The Effects of Gender Responsive Programming on Girls and Boys in Secure Detention
Day, J. C., Zahn, M. A., & Tichavsky, L. P. (2014) · Journal of Research in Crime and Delinquency · doi:10.1177/0022427814538033 · Cited by 67
PROPENSITY SCORE ANALYSIS AND ASSESSMENT OF PROPENSITY SCORE APPROACHES USING SAS ® PROCEDURES
Lanehart, R. E., de Gil, P. R., Kim, E., Bellara, A. P., & Lee, R. S. (2012) · Cited by 60
Precautionary Behavior and Depression in Older Adults during the COVID-19 Pandemic: An Online Cross-Sectional Study in Hubei, China
Liang, W., Duan, Y., Shang, B., Hu, C., Baker, J. S., Lin, Z., He, J., & Wang, Y. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18041853 · Cited by 60
Exploring the benefits of respite services to family caregivers: methodological issues and current findings
Zarit, S. H., Bangerter, L. R., Liu, Y., & Rovine, M. J. (2016) · Aging & Mental Health · doi:10.1080/13607863.2015.1128881 · Cited by 67
Assessing Medium-term Impacts of Conservation Interventions on Local Livelihoods in Northern Cambodia
Beauchamp, E., Clements, T., & Milner‐Gulland, E. J. (2017) · World Development · doi:10.1016/j.worlddev.2017.08.008 · Cited by 61
The Effects of Homeownership on Children's Outcomes: Real Effects or Self‐Selection?
Holupka, S., & Newman, S. J. (2012) · Real Estate Economics · doi:10.1111/j.1540-6229.2012.00330.x · Cited by 62
Does It Matter Which Parent is Absent? Labor Migration, Parenting, and Adolescent Development in China
Xu, Y., Xu, D., Simpkins, S. D., & Warschauer, M. (2019) · Journal of Child and Family Studies · doi:10.1007/s10826-019-01382-z · Cited by 58
Collective Property Leads to Household Investments: Lessons From Land Titling in Afro-Colombian Communities
Peña, X., Vélez, M. A., Cárdenas, J.-C., Perdomo, N., & Matajira, C. (2016) · World Development · doi:10.1016/j.worlddev.2017.03.025 · Cited by 56
Relationships between adolescent sexual outcomes and exposure to sex in media: Robustness to propensity-based analysis.
Collins, R. L., Martino, S. C., Elliott, M. N., & Miu, A. (2011) · Developmental Psychology · doi:10.1037/a0022563 · Cited by 59
Quasi-experimental designs for causal inference: an overview
Cham, H., Lee, H., & Migunov, I. (2024) · Asia Pacific Education Review · doi:10.1007/s12564-024-09981-2 · Cited by 55
The Effect of the WIC Program on the Health of Newborns
Foster, E. M., Jiang, M., & Gibson‐Davis, C. M. (2010) · Health Services Research · doi:10.1111/j.1475-6773.2010.01115.x · Cited by 56
Cost-effectiveness of integrated care in frail elderly using the ICECAP-O and EQ-5D: does choice of instrument matter?
Makai, P., Looman, W., Adang, E., Melis, R. J. F., Stolk, E. A., & Fabbricotti, I. N. (2014) · The European Journal of Health Economics · doi:10.1007/s10198-014-0583-7 · Cited by 59
Them That’s Got: How Tie Formation in Partnership Networks Gives High Schools Differential Access to Social Capital
Bridwell-Mitchell, E. N. (2017) · American Educational Research Journal · doi:10.3102/0002831217717815 · Cited by 53
Estimating the causal effect of randomization versus treatment preference in a doubly randomized preference trial.
Marcus, S. M., Stuart, E. A., Wang, P., Shadish, W. R., & Steiner, P. M. (2012) · Psychological Methods · doi:10.1037/a0028031 · Cited by 53
Self-citation
Empirical Performance of Covariates in Education Observational Studies
Wong, V. C., Valentine, J. C., & Miller-Bains, K. (2016) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2016.1164781 · Cited by 54
Adult outcomes of sustained high-quality early child care and education: Do they vary by family income?
Bustamante, A. S., Dearing, E., Zachrisson, H. D., & Vandell, D. L. (2021) · Child Development · doi:10.1111/cdev.13696 · Cited by 53
ANCOVA versus Change Score for the Analysis of Two-Wave Data
Lüdtke, O., & Robitzsch, A. (2023) · The Journal of Experimental Education · doi:10.1080/00220973.2023.2246187 · Cited by 51
The transmission of values from math teachers to their ninth-grade students: Different mechanisms for different value dimensions?
Parrisius, C., Gaspard, H., Trautwein, U., & Nagengast, B. (2020) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2020.101891 · Cited by 58
Randomized Controlled Studies and Alternative Designs in Outcome Studies
Shadish, W. R. (2011) · Research on Social Work Practice · doi:10.1177/1049731511403324 · Cited by 50
Self-citation
Do diversity courses improve college student outcomes? A meta-analysis.
Denson, N., Bowman, N. A., Ovenden, G., Culver, K. C., & Holmes, J. M. (2020) · Journal of Diversity in Higher Education · doi:10.1037/dhe0000189 · Cited by 50
Youth Participatory Action Research in the High School Curriculum: Education Outcomes for Student Participants in a District-Wide Initiative
Voight, A., & Velez, V. (2018) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2018.1431345 · Cited by 49
WHAT IS A “GOOD” SOCIAL NETWORK FOR POLICY IMPLEMENTATION? THE FLOW OF KNOW-HOW FOR ORGANIZATIONAL CHANGE
Frank, K. A., Penuel, W. R., & Krause, A. E. (2015) · Journal of Policy Analysis and Management · doi:10.1002/pam.21817 · Cited by 49
Research on the Academic Benefits of the Advanced Placement Program
Warne, R. T. (2017) · SAGE Open · doi:10.1177/2158244016682996 · Cited by 49
Breastfeeding and the Child Cognitive Outcomes: A Propensity Score Matching Approach
Jiang, M., Foster, E. M., & Gibson‐Davis, C. M. (2010) · Maternal and Child Health Journal · doi:10.1007/s10995-010-0677-5 · Cited by 48
Comparing Propensity Score Methods in Balancing Covariates and Recovering Impact in Small Sample Educational Program Evaluations
Stone, C. A. (2020) · doi:10.7275/qkqa-9k50 · Cited by 51
Promotive and Corrosive Factors in African American Students’ Math Beliefs and Achievement
Diemer, M. A., Marchand, A. D., McKellar, S. E., & Malanchuk, O. (2016) · Journal of Youth and Adolescence · doi:10.1007/s10964-016-0439-9 · Cited by 47
Effect of retention in elementary grades on dropping out of school early
Hughes, J. N., Cao, Q., West, S. G., Allee-Smith, P. J., & Cerda, C. A. (2017) · Journal of School Psychology · doi:10.1016/j.jsp.2017.06.003 · Cited by 47
Residential Mobility and Delinquency Revisited: Causation or Selection?
Porter, L. C., & Vogel, M. (2013) · Journal of Quantitative Criminology · doi:10.1007/s10940-013-9200-7 · Cited by 49
A Two-Step Bayesian Approach for Propensity Score Analysis: Simulations and Case Study
Kaplan, D. E., & Chen, J. (2012) · Psychometrika · doi:10.1007/s11336-012-9262-8 · Cited by 46
Quantifying the robustness of causal inferences: Sensitivity analysis for pragmatic social science
Frank, K. A., Lin, Q., Xu, R., Maroulis, S. J., & Mueller, A. S. (2022) · Social Science Research · doi:10.1016/j.ssresearch.2022.102815 · Cited by 45
Is early ability grouping good for high-achieving students’ psychosocial development? Effects of the transition into academically selective schools.
Becker, M., Neumann, M., Tetzner, J., Böse, S., Knoppick, H., Maaz, K., Baumert, J., & Lehmann, R. (2014) · Journal of Educational Psychology · doi:10.1037/a0035425 · Cited by 50
Estimating Causal Effects in Observational Studies Using Electronic Health Data: Challenges and (some) Solutions
Stuart, E. A., DuGoff, E. H., Abrams, M. T., Salkever, D. S., & Steinwachs, D. M. (2013) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.13063/2327-9214.1038 · Cited by 48
Administrative Climate and Novices' Intent to Remain Teaching
Pogodzinski, B., Youngs, P. A., Frank, K. A., & Belman, D. L. (2012) · The Elementary School Journal · doi:10.1086/667725 · Cited by 45
Constructing “Experts” Among Peers: Educational Infrastructure, Test Data, and Teachers’ Interactions About Teaching
Spillane, J. P., Shirrell, M., & Adhikari, S. (2018) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373718785764 · Cited by 45

Authors: Steiner, P. M., Cook, T. D., Shadish, W. R., & Clark, M. H.
Affiliations: Northwestern University (US)
ORCID: Steiner, Peter M.
Cook, Thomas D.
Clark, M. H.
Venue: Psychological Methods
Publisher: American Psychological Association (APA)
Published: 2010
Volume: 15
Issue: 3
Pages: 250-267
ISSN: 1082-989X (ISSN-L)
1939-1463
DOI: 10.1037/a0018719
OpenAlex: W2168206775
Cited by: 490
References: 55
Citations by year: 2026: 23 · 2025: 28 · 2024: 34 · 2023: 26 · 2022: 26 · 2021: 53 · 2020: 32 · 2019: 20 · 2018: 37 · 2017: 36 · 2016: 38 · 2015: 28 · 2014: 31 · 2013: 32 · 2012: 19
FWCI: 18.95
Type: journal article
Language: en
Open access: closed
Locations: Psychological Methods (publishedVersion)
PubMed (publishedVersion)
Funders: US Department of Education (R305U070003)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Keywords: covariate selection, selection bias, observational studies, quasi-experiments, self-selection, randomized experiments, bias mitigation
MeSH terms: Data Interpretation, Statistical, Epidemiologic Methods, Humans, Mathematics, Random Allocation, Models, Statistical, Selection Bias, Causality, Patient Selection, Behavioral Research, Propensity Score

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

The importance of covariate selection in controlling for selection bias in observational studies.

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks