The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Identification Problems in the Social Sciences
Open accessAbstract
Methodological research in the social sciences aims to learn what conclusions can and cannot be drawn given empirically relevant combinations of assumptions and data. Methodologists have long found it useful to separate inferential problems into statistical and identification components. Studies of identification seek to characterize the conclusions that could be drawn if the researcher had available a sample of unlimited size. Studies of statistical inference seek to characterize the generally weaker conclusions that can be drawn given a sample of positive but finite size. Statistical and identification problems limit in distinct ways the conclusions that may be drawn in empirical research. Statistical problems are most severe when the available sample is small. Identification problems are most severe when the researcher knows little about the population under study and the sampling process yields only weak data on the population. This paper synthesizes some of my recent research and thinking on identification problems in the social sciences. Four problems are discussed: extrapolation of regressions, the selection problem, identification of endogenous social effects from outcome data, and identification of subjective phenomena. These problems arise regularly in social science research and are the source of many substantive disputes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1975) · PhilPapers (PhilPapers Foundation) · Cited by 36,737
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701
The Statistical Analysis of Failure Time Data.
Aitkin, M., Kalbfleisch, J. D., & Prentice, R. L. (1981) · Biometrics · doi:10.2307/2530581 · Cited by 9,877
Estimation of Relationships for Limited Dependent Variables
Tobin, J. (1958) · Econometrica · doi:10.2307/1907382 · Cited by 6,996
EQUALITY OF EDUCATIONAL OPPORTUNITY
Coleman, J. S. (1968) · Equity & Excellence in Education · doi:10.1080/0020486680060504 · Cited by 5,308
(1976) · Annals of Economic and Social Measurement · Cited by 5,035
Dynamic models of segregation†
Schelling, T. C. (1971) · The Journal of Mathematical Sociology · doi:10.1080/0022250x.1971.9989794 · Cited by 4,915
(1986) · Journal of Economic Literature · Cited by 2,705
Applied nonparametric regression
(1991) · Choice Reviews Online · doi:10.5860/choice.29-0364 · Cited by 2,580
Marsden, P. V. (1990) · Annual Review of Sociology · doi:10.1146/annurev.so.16.080190.002251 · Cited by 2,176
Cited by
Recent Developments in the Econometrics of Program Evaluation
Imbens, G. W., & Wooldridge, J. M. (2009) · Journal of Economic Literature · doi:10.1257/jel.47.1.5 · Cited by 4,926
Causal Inference without Balance Checking: Coarsened Exact Matching
Iacus, S. M., King, G., & Porro, G. (2012) · Political Analysis · doi:10.1093/pan/mpr013 · Cited by 3,563
Large sample properties of matching estimators for average treatment effects
Abadie, A., & Imbens, G. W. (2006) · Econometrica · doi:10.1111/j.1468-0262.2006.00655.x · Cited by 2,908
Diez Roux, A. V., & Mair, C. (2010) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2009.05333.x · Cited by 2,774
Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs
Dehejia, R. H., & Wahba, S. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10473858 · Cited by 2,395
Learning about a New Technology: Pineapple in Ghana
Conley, T. G., & Udry, C. R. (2010) · American Economic Review · doi:10.1257/aer.100.1.35 · Cited by 2,180
Statistical Methods for Comparing Regression Coefficients Between Models
Clogg, C. C., Petkova, E., & Haritou, A. (1995) · American Journal of Sociology · doi:10.1086/230638 · Cited by 2,072
Economic Analysis of Social Interactions
Manski, C. F. (2000) · Journal of Economic Perspectives · doi:10.1257/jep.14.3.115 · Cited by 2,064
Self-citationDiscrete Choice with Social Interactions
Brock, W. A., & Durlauf, S. N. (2001) · The Review of Economic Studies · doi:10.1111/1467-937x.00168 · Cited by 1,664
Lucas, S. R. (2001) · American Journal of Sociology · doi:10.1086/321300 · Cited by 1,656
Smith, K. P., & Christakis, N. A. (2008) · Annual Review of Sociology · doi:10.1146/annurev.soc.34.040507.134601 · Cited by 1,638
The role of information and financial reporting in corporate governance and debt contracting
Armstrong, C. S., Guay, W. R., & Weber, J. P. (2010) · Journal of Accounting and Economics · doi:10.1016/j.jacceco.2010.10.001 · Cited by 1,494
Standards of Evidence: Criteria for Efficacy, Effectiveness and Dissemination
Flay, B. R., Biglan, A., Boruch, R. F., Castro, F. G., Gottfredson, D., Kellam, S., Mościcki, E. K., Schinke, S., Valentine, J. C., & Ji, P. (2005) · Prevention Science · doi:10.1007/s11121-005-5553-y · Cited by 1,338
Analyzing developmental trajectories of distinct but related behaviors: A group-based method.
Nagin, D. S., & Tremblay, R. E. (2001) · Psychological Methods · doi:10.1037//1082-989x.6.1.18 · Cited by 1,280
Identification and Inference in Nonlinear Difference-in-Differences Models
Athey, S., & Imbens, G. W. (2006) · Econometrica · doi:10.1111/j.1468-0262.2006.00668.x · Cited by 1,084
The Endogeneity between Language and Earnings: International Analyses
Chiswick, B. R., & Miller, P. W. (1995) · Journal of Labor Economics · doi:10.1086/298374 · Cited by 931
THE ESTIMATION OF CAUSAL EFFECTS FROM OBSERVATIONAL DATA
Winship, C., & Morgan, S. L. (1999) · Annual Review of Sociology · doi:10.1146/annurev.soc.25.1.659 · Cited by 890
Duflo, E., & Saez, E. (2002) · National Bureau of Economic Research · doi:10.3386/w8885 · Cited by 810
Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs
Dehejia, R., & Wahba, S. (1998) · National Bureau of Economic Research · doi:10.3386/w6586 · Cited by 805
Aral, S., & Walker, D. (2011) · Management Science · doi:10.1287/mnsc.1110.1421 · Cited by 786
Participation and investment decisions in a retirement plan: the influence of colleagues’ choices
Duflo, E., & Saez, E. (2002) · Journal of Public Economics · doi:10.1016/s0047-2727(01)00098-6 · Cited by 782
The (mis)estimation of neighborhood effects: causal inference for a practicable social epidemiology
Oakes, J. M. (2004) · Social Science & Medicine · doi:10.1016/j.socscimed.2003.08.004 · Cited by 760
Social Capital and Finding a Job: Do Contacts Matter?
Mouw, T. (2003) · American Sociological Review · doi:10.1177/000312240306800604 · Cited by 712
Neighbors Matter: Causal Community Effects and Stock Market Participation
BROWN, J. R., IVKOVIĆ, Z., SMITH, P. A., & WEISBENNER, S. (2008) · The Journal of Finance · doi:10.1111/j.1540-6261.2008.01364.x · Cited by 701
Micro Data, Heterogeneity, and the Evaluation of Public Policy: Nobel Lecture
Heckman, J. J. (2001) · Journal of Political Economy · doi:10.1086/322086 · Cited by 675
Caspi, A., Wright, B. R. E., Moffitt, T. E., & Silva, P. A. (1998) · American Sociological Review · doi:10.2307/2657557 · Cited by 649
Balbo, N., Billari, F. C., & Mills, M. (2013) · European Journal of Population / Revue européenne de Démographie · doi:10.1007/s10680-012-9277-y · Cited by 623
Sensitivity analysis for average treatment effects
Becker, S. O., & Caliendo, M. (2007) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0700700104 · Cited by 620
Brand, J. E., & Xie, Y. (2010) · American Sociological Review · doi:10.1177/0003122410363567 · Cited by 587
The Dangers of Extreme Counterfactuals
King, G., & Zeng, L. (2006) · Political Analysis · doi:10.1093/pan/mpj004 · Cited by 540
Brock, W., & Durlauf, S. (2000) · National Bureau of Economic Research · doi:10.3386/t0258 · Cited by 528
Neighbourhood Effects Research: New Perspectives
(2012) · Springer eBooks · doi:10.1007/978-94-007-2309-2 · Cited by 526
Does Indiscriminate Violence Incite Insurgent Attacks?: Evidence from Chechnya
Lyall, J. (2009) · Journal of Conflict Resolution · doi:10.1177/0022002708330881 · Cited by 499
Marginal Structural Models to Estimate the Joint Causal Effect of Nonrandomized Treatments
Hernán, M. A., Brumback, B., & Robins, J. M. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753168154 · Cited by 493
Applied MANOVA and Discriminant Analysis
Huberty, C. J., & Olejnik, S. (2006) · Wiley Series in Probability and Statistics · doi:10.1002/047178947x · Cited by 488
Identification and Estimation of Triangular Simultaneous Equations Models Without Additivity
(2009) · Econometrica · doi:10.3982/ecta7108 · Cited by 488
Covariance Adjustment in Randomized Experiments and Observational Studies
Rosenbaum, P. R. (2002) · Statistical Science · doi:10.1214/ss/1042727942 · Cited by 477
Sobel, M. E. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000636 · Cited by 468
Nonparametric Analysis of Randomized Experiments with Missing Covariate and Outcome Data
Horowitz, J. L., & Manski, C. F. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473902 · Cited by 427
Self-citationDuflo, E., & Saez, E. (2002) · SSRN Electronic Journal · doi:10.2139/ssrn.315659 · Cited by 425
Experimental Designs for Identifying Causal Mechanisms
Imai, K., Tingley, D., & Yamamoto, T. (2013) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2012.01032.x · Cited by 408
Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy
Heckman, J. J. (2010) · Journal of Economic Literature · doi:10.1257/jel.48.2.356 · Cited by 410
The econometrics of DSGE models
Fernández-Villaverde, J. (2010) · SERIEs · doi:10.1007/s13209-009-0014-7 · Cited by 402
Causal Inference in Sociological Research
Gangl, M. (2010) · Annual Review of Sociology · doi:10.1146/annurev.soc.012809.102702 · Cited by 401
Interacting agents, spatial externalities and the evolution of residential land use patterns
Irwin, E. G., & Bockstael, N. E. (2002) · Journal of Economic Geography · doi:10.1093/jeg/2.1.31 · Cited by 397
Matching Estimators of Causal Effects: Prospects and Pitfalls in Theory and Practice
Morgan, S. L., & Harding, D. J. (2006) · Sociological Methods & Research · doi:10.1177/0049124106289164 · Cited by 392
Marginal Mean Models for Dynamic Regimes
Murphy, S. A., van der Laan, M. J., Robins, J. M., & Conduct Problems Prevention Research Group (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382327 · Cited by 393
Tie Strength, Embeddedness, and Social Influence: A Large-Scale Networked Experiment
Aral, S., & Walker, D. (2014) · Management Science · doi:10.1287/mnsc.2014.1936 · Cited by 389
Estimating neighborhood health effects: the challenges of causal inference in a complex world
Diez Roux, A. V. (2004) · Social Science & Medicine · doi:10.1016/s0277-9536(03)00414-3 · Cited by 386
On the Complexities of Complex Economic Dynamics
Rosser, J. B. (1999) · Journal of Economic Perspectives · doi:10.1257/jep.13.4.169 · Cited by 377
Manski, C. F. (1997) · Econometrica · doi:10.2307/2171738 · Cited by 376
Self-citationHeckman, J. J., & Vytlacil, E. J. (2007) · Handbook of Econometrics · doi:10.1016/s1573-4412(07)06071-0 · Cited by 369
Muller, E., & Peres, R. (2019) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2018.05.003 · Cited by 367
Mannering, F., Bhat, C. R., Shankar, V., & Abdel-Aty, M. (2020) · Analytic Methods in Accident Research · doi:10.1016/j.amar.2020.100113 · Cited by 367
Bapna, R., & Umyarov, A. (2015) · Management Science · doi:10.1287/mnsc.2014.2081 · Cited by 350
The Long View of Crime: A Synthesis of Longitudinal Research
(2008) · Springer eBooks · doi:10.1007/978-0-387-71165-2 · Cited by 353
Orphans and schooling in Africa: A longitudinal analysis
Evans, D. K., & Miguel, E. (2007) · Demography · doi:10.1353/dem.2007.0002 · Cited by 351
Causal inference with observational data
Nichols, A. (2007) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0800700403 · Cited by 339
Why Most People Disapprove of Me: Experience Sampling in Impression Formation.
Denrell, J. (2005) · Psychological Review · doi:10.1037/0033-295x.112.4.951 · Cited by 337
Estimating Heterogeneous Treatment Effects with Observational Data
Xie, Y., Brand, J. E., & Jann, B. (2012) · Sociological Methodology · doi:10.1177/0081175012452652 · Cited by 333
Exercise contagion in a global social network
Aral, S., & Nicolaides, C. (2017) · Nature Communications · doi:10.1038/ncomms14753 · Cited by 329
Impact of traffic oscillations on freeway crash occurrences
Zheng, Z., Ahn, S., & Monsere, C. M. (2010) · Accident Analysis & Prevention · doi:10.1016/j.aap.2009.10.009 · Cited by 325
Powell, L. M., Tauras, J. A., & Ross, H. (2005) · Journal of Health Economics · doi:10.1016/j.jhealeco.2005.02.002 · Cited by 324
Behrman, J. R., Kohler, H.-P., & Watkins, S. C. (2002) · Demography · doi:10.1353/dem.2002.0033 · Cited by 321
Scott-Clayton, J. (2011) · Journal of Human Resources · doi:10.1353/jhr.2011.0013 · Cited by 314
Administrative Records Mask Racially Biased Policing
KNOX, D., LOWE, W., & MUMMOLO, J. (2020) · American Political Science Review · doi:10.1017/s0003055420000039 · Cited by 314
Psychological Pressure in Competitive Environments: Evidence from a Randomized Natural Experiment
Apesteguia, J., & Palacios-Huerta, I. (2010) · American Economic Review · doi:10.1257/aer.100.5.2548 · Cited by 305
Tools for Intuition about Sample Selection Bias and Its Correction
Stolzenberg, R. M., & Relles, D. A. (1997) · American Sociological Review · doi:10.2307/2657318 · Cited by 301
Chapter 50 Neighborhood effects
Durlauf, S. N. (2004) · Handbook of Regional and Urban Economics · doi:10.1016/s1574-0080(04)80007-5 · Cited by 289
Do Payments for Environmental Services Affect Forest Cover? A Farm-Level Evaluation from Costa Rica
Arriagada, R. A., Ferraro, P. J., Sills, E. O., Pattanayak, S. K., & Cordero-Sancho, S. (2012) · Land Economics · doi:10.3368/le.88.2.382 · Cited by 287
Identity and entrepreneurship: do school peers shape entrepreneurial intentions?
Falck, O., Heblich, S., & Luedemann, E. (2012) · Small Business Economics · doi:10.1007/s11187-010-9292-5 · Cited by 283
Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)
Imbens, G. W. (2010) · Journal of Economic Literature · doi:10.1257/jel.48.2.399 · Cited by 278
Benkard, C. L., & Bajari, P. (2004) · National Bureau of Economic Research · doi:10.3386/w10278 · Cited by 276
The impact of the National School Lunch Program on child health: A nonparametric bounds analysis
Gundersen, C., Kreider, B., & Pepper, J. (2012) · Journal of Econometrics · doi:10.1016/j.jeconom.2011.06.007 · Cited by 269
Bounding the effects of food insecurity on children’s health outcomes
Gundersen, C., & Kreider, B. (2009) · Journal of Health Economics · doi:10.1016/j.jhealeco.2009.06.012 · Cited by 269
Narrative Explanation: An Alternative to Variable-Centered Explanation?
Abell, P. (2004) · Annual Review of Sociology · doi:10.1146/annurev.soc.29.010202.100113 · Cited by 264
Life satisfaction and the economic and social characteristics of neighbourhoods
Shields, M. A., Wheatley Price, S., & Wooden, M. (2009) · Journal of Population Economics · doi:10.1007/s00148-007-0146-7 · Cited by 264
Demographic Patterns of Cumulative Arrest Prevalence by Ages 18 and 23
Brame, R., Bushway, S. D., Paternoster, R., & Turner, M. G. (2014) · Crime & Delinquency · doi:10.1177/0011128713514801 · Cited by 262
Land use externalities, open space preservation, and urban sprawl
Irwin, E. G., & Bockstael, N. E. (2004) · Regional Science and Urban Economics · doi:10.1016/j.regsciurbeco.2004.03.002 · Cited by 257
When Can History Be Our Guide? The Pitfalls of Counterfactual Inference
KING, G., & ZENG, L. (2007) · International Studies Quarterly · doi:10.1111/j.1468-2478.2007.00445.x · Cited by 256
Cumulative prevalence of arrest from ages 8 to 23 in a national sample
Brame, R., Turner, M. G., Paternoster, R., & Bushway, S. D. (2012) · Pediatrics · doi:10.1542/peds.2010-3710 · Cited by 236
Messer, L. C., Oakes, J. M., & Mason, S. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwp435 · Cited by 161
Statistics and causal inference: A review
Pearl, J. (2003) · Test · doi:10.1007/bf02595718 · Cited by 144
The cultural evolution of fertility decline
Colleran, H. (2016) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2015.0152 · Cited by 124
Oakes, J. M. (2006) · International Journal of Epidemiology · doi:10.1093/ije/dyl054 · Cited by 115
Braver, S. L., & Smith, M. C. (1996) · Evaluation and Program Planning · doi:10.1016/s0149-7189(96)00029-8 · Cited by 37
QCA is of questionable value for policy research
Tanner, S. (2014) · Policy and Society · doi:10.1016/j.polsoc.2014.08.003 · Cited by 16
Estimating Treatment Effects: Matching Quantification to the Question
Loughran, T. A., & Mulvey, E. P. (2010) · Handbook of Quantitative Criminology · doi:10.1007/978-0-387-77650-7_9 · Cited by 16
Metadata
- Authors
- Manski, C. F.
- Venue
- Sociological Methodology
- Publisher
- JSTOR
- Published
- 1993
- Volume
- 23
- Pages
- 1
- ISSN
- 0081-1750 (ISSN-L)
- OpenAlex
- W2020682813
- Cited by
- 1,130
- References
- 51
- FWCI
- 6.95
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.