The DOI matches its Crossref record; 145 of the 145 references it cites were found in a registry or an index.
Instruments, Randomization, and Learning about Development
Abstract
There is currently much debate about the effectiveness of foreign aid and about what kind of projects can engender economic development. There is skepticism about the ability of econometric analysis to resolve these issues or of development agencies to learn from their own experience. In response, there is increasing use in development economics of randomized controlled trials (RCTs) to accumulate credible knowledge of what works, without overreliance on questionable theory or statistical methods. When RCTs are not possible, the proponents of these methods advocate quasi-randomization through instrumental variable (IV) techniques or natural experiments. I argue that many of these applications are unlikely to recover quantities that are useful for policy or understanding: two key issues are the misunderstanding of exogeneity and the handling of heterogeneity. I illustrate from the literature on aid and growth. Actual randomization faces similar problems as does quasi-randomization, notwithstanding rhetoric to the contrary. I argue that experiments have no special ability to produce more credible knowledge than other methods, and that actual experiments are frequently subject to practical problems that undermine any claims to statistical or epistemic superiority. I illustrate using prominent experiments in development and elsewhere. As with IV methods, RCT-based evaluation of projects, without guidance from an understanding of underlying mechanisms, is unlikely to lead to scientific progress in the understanding of economic development. I welcome recent trends in development experimentation away from the evaluation of projects and toward the evaluation of theoretical mechanisms. (JEL C21, F35, O19)
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2001) · RePEc: Research Papers in Economics · Cited by 27,577
A Contribution to the Empirics of Economic Growth
Mankiw, N. G., Romer, D., & Weil, D. N. (1992) · The Quarterly Journal of Economics · doi:10.2307/2118477 · Cited by 15,119
The colonial origins of comparative development: An empirical investigation
Acemoglu, D., Johnson, S., & Robinson, J. A. (2001) · American Economic Review · doi:10.1257/aer.91.5.1369 · Cited by 8,517
The Colonial Origins of Comparative Development: An Empirical Investigation: Reply
Acemoglu, D., Johnson, S., & Robinson, J. A. (2012) · American Economic Review · doi:10.1257/aer.102.6.3077 · Cited by 6,047
Moffitt, R. A. (2009) · American Economic Review · doi:10.1257/aer.99.2.660 · Cited by 4,694
Regression discontinuity designs in economics
Lee, D. S., & Lemieux, T. (2010) · Journal of Economic Literature · doi:10.1257/jel.48.2.281 · Cited by 4,534
Identification and estimation of local average treatment effects
Imbens, G. W., & Angrist, J. D. (1994) · Econometrica · doi:10.2307/2951620 · Cited by 4,101
Burnside, C., & Dollar, D. (2000) · American Economic Review · doi:10.1257/aer.90.4.847 · Cited by 3,728
Manipulation of the Running Variable in the Regression Discontinuity Design: A Density Test,
McCrary, J. (2008) · Journal of Econometrics · doi:10.1016/j.jeconom.2007.05.005 · Cited by 3,600
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563
Cited by
Understanding and misunderstanding randomized controlled trials
Deaton, A., & Cartwright, N. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2017.12.005 · Cited by 1,841
“Please cite this article as: Deaton, A., Social Science & Medicine (2017), https://doi.”
Self-citationIMAI, K., KEELE, L., TINGLEY, D., & YAMAMOTO, T. (2011) · American Political Science Review · doi:10.1017/s0003055411000414 · Cited by 1,537
Explaining Fixed Effects: Random Effects Modeling of Time-Series Cross-Sectional and Panel Data
Bell, A., & Jones, K. (2015) · Political Science Research and Methods · doi:10.1017/psrm.2014.7 · Cited by 1,515
The State of Applied Econometrics: Causality and Policy Evaluation
Athey, S., & Imbens, G. W. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.3 · Cited by 1,397
Social and Economic Returns to College Education in the United States
Hout, M. (2012) · Annual Review of Sociology · doi:10.1146/annurev.soc.012809.102503 · Cited by 943
“Apparently, the IV estimates contained more than just a correction for ability bias (Deaton 2010).”
BackgroundEvidence for Health Decision Making — Beyond Randomized, Controlled Trials
Frieden, T. R. (2017) · New England Journal of Medicine · doi:10.1056/nejmra1614394 · Cited by 770
Assessing the Gold Standard — Lessons from the History of RCTs
Bothwell, L. E., Greene, J. A., Podolsky, S. H., & Jones, D. S. (2016) · New England Journal of Medicine · doi:10.1056/nejmms1604593 · Cited by 664
Agnostic notes on regression adjustments to experimental data: Reexamining Freedman’s critique
Lin, W. (2013) · The Annals of Applied Statistics · doi:10.1214/12-aoas583 · Cited by 641
Blunt Instruments: Avoiding Common Pitfalls in Identifying the Causes of Economic Growth
Bazzi, S., & Clemens, M. A. (2013) · American Economic Journal: Macroeconomics · doi:10.1257/mac.5.2.152 · Cited by 640
The Missionary Roots of Liberal Democracy
WOODBERRY, R. D. (2012) · American Political Science Review · doi:10.1017/s0003055412000093 · Cited by 596
Craig, P., Katikireddi, S. V., Leyland, A., & Popham, F. (2017) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031816-044327 · Cited by 570
Tax Compliance and Enforcement
Slemrod, J. (2019) · Journal of Economic Literature · doi:10.1257/jel.20181437 · Cited by 563
The Econometrics of Randomized Experiments
(2017) · Handbook of Economic Field Experiments · doi:10.1016/bs.hefe.2016.10.003 · Cited by 562
KALLA, J. L., & BROOCKMAN, D. E. (2018) · American Political Science Review · doi:10.1017/s0003055417000363 · Cited by 454
Toward More “Evidence‐Informed” Policy Making?
Head, B. W. (2016) · Public Administration Review · doi:10.1111/puar.12475 · Cited by 453
Econometric Methods for Program Evaluation
Abadie, A., & Cattaneo, M. D. (2018) · Annual Review of Economics · doi:10.1146/annurev-economics-080217-053402 · Cited by 431
McEwan, P. J. (2015) · Review of Educational Research · doi:10.3102/0034654314553127 · Cited by 413
Leaving theory behind: Why simplistic hypothesis testing is bad for International Relations
Mearsheimer, J. J., & Walt, S. M. (2013) · European Journal of International Relations · doi:10.1177/1354066113494320 · Cited by 412
Going to a Better School: Effects and Behavioral Responses
Pop-Eleches, C., & Urquiola, M. (2013) · American Economic Review · doi:10.1257/aer.103.4.1289 · Cited by 411
Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars
Mayo, D. G. (2018) · doi:10.1017/9781107286184 · Cited by 376
Broadening the range of designs and methods for impact evaluations
Stern, E., Stame, N., Mayne, J., Forss, K., Davies, R., & Befani, B. (2012) · doi:10.22163/fteval.2012.100 · Cited by 369
“2.21 The idea that there is a single superior method of producing evidence has also been widely challenged (Ravallion 2009, Deaton 2010).”
InfluentialThe Impact of Microcredit on the Poor in Bangladesh: Revisiting the Evidence
Roodman, D., & Morduch, J. (2014) · The Journal of Development Studies · doi:10.1080/00220388.2013.858122 · Cited by 371
Kabeer, N. (2016) · Feminist Economics · doi:10.1080/13545701.2015.1090009 · Cited by 366
What Is Your Estimand? Defining the Target Quantity Connects Statistical Evidence to Theory
Lundberg, I., Johnson, R., & Stewart, B. M. (2021) · American Sociological Review · doi:10.1177/00031224211004187 · Cited by 364
From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application
Banerjee, A., Banerji, R., Berry, J., Duflo, E., Kannan, H., Mukerji, S., Shotland, M., & Walton, M. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.4.73 · Cited by 344
Berge, L. I. O., Bjorvatn, K., & Tungodden, B. (2015) · Management Science · doi:10.1287/mnsc.2014.1933 · Cited by 343
Evaluation of biodiversity policy instruments: what works and what doesn't?
Miteva, D. A., Pattanayak, S. K., & Ferraro, P. J. (2012) · Oxford Review of Economic Policy · doi:10.1093/oxrep/grs009 · Cited by 338
Shift-Share Instruments and the Impact of Immigration
Jaeger, D., Ruist, J., & Stuhler, J. (2018) · National Bureau of Economic Research · doi:10.3386/w24285 · Cited by 333
The Power and Pitfalls of Experiments in Development Economics: Some Non‐random Reflections
Barrett, C. B., & Carter, M. R. (2010) · Applied Economic Perspectives and Policy · doi:10.1093/aepp/ppq023 · Cited by 308
The causal effects of education on health outcomes in the UK Biobank
Davies, N. M., Dickson, M., Davey Smith, G., van den Berg, G. J., & Windmeijer, F. (2018) · Nature Human Behaviour · doi:10.1038/s41562-017-0279-y · Cited by 302
Promises and Perils of Pre-Analysis Plans
Olken, B. A. (2015) · Journal of Economic Perspectives · doi:10.1257/jep.29.3.61 · Cited by 299
Lu, G., Ding, X. (., Peng, D. X., & Hao‐Chun Chuang, H. (2018) · Journal of Operations Management · doi:10.1016/j.jom.2018.10.001 · Cited by 297
Agent-Based Models in Empirical Social Research
Bruch, E., & Atwell, J. (2015) · Sociological Methods & Research · doi:10.1177/0049124113506405 · Cited by 296
Swedberg, R. (2020) · The Production of Knowledge · doi:10.1017/9781108762519.002 · Cited by 293
Mr. Rossi, Mr. Hu and politics. The role of immigration in shaping natives' voting behavior
Barone, G., D'Ignazio, A., de Blasio, G., & Naticchioni, P. (2016) · Journal of Public Economics · doi:10.1016/j.jpubeco.2016.03.002 · Cited by 290
Sampling‐Based versus Design‐Based Uncertainty in Regression Analysis
Abadie, A., Athey, S., Imbens, G. W., & Wooldridge, J. M. (2020) · Econometrica · doi:10.3982/ecta12675 · Cited by 285
Constantino, S. M., Sparkman, G., Kraft-Todd, G. T., Bicchieri, C., Centola, D., Shell-Duncan, B., Vogt, S., & Weber, E. U. (2022) · Psychological Science in the Public Interest · doi:10.1177/15291006221105279 · Cited by 285
Agricultural innovation and inclusive value-chain development: a review
Devaux, A., Torero, M., Donovan, J., & Horton, D. (2018) · Journal of Agribusiness in Developing and Emerging Economies · doi:10.1108/jadee-06-2017-0065 · Cited by 284
Advances in Measuring the Environmental and Social Impacts of Environmental Programs
Ferraro, P. J., & Hanauer, M. M. (2014) · Annual Review of Environment and Resources · doi:10.1146/annurev-environ-101813-013230 · Cited by 276
Financial incentives in health: New evidence from India's Janani Suraksha Yojana
Powell-Jackson, T., Mazumdar, S., & Mills, A. (2015) · Journal of Health Economics · doi:10.1016/j.jhealeco.2015.07.001 · Cited by 270
Worms at Work: Long-run Impacts of a Child Health Investment*
Baird, S., Hicks, J. H., Kremer, M., & Miguel, E. (2016) · The Quarterly Journal of Economics · doi:10.1093/qje/qjw022 · Cited by 269
Endogeneity in survey research
Sande, J. B., & Ghosh, M. (2018) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2018.01.005 · Cited by 264
Conn, K. M. (2017) · Review of Educational Research · doi:10.3102/0034654317712025 · Cited by 263
General Equilibrium Effects of Cash Transfers: Experimental Evidence From Kenya
Egger, D., Haushofer, J., Miguel, E., Niehaus, P., & Walker, M. (2022) · Econometrica · doi:10.3982/ecta17945 · Cited by 223
Muralidharan, K., & Niehaus, P. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.4.103 · Cited by 194
Labrecque, J., & Swanson, S. A. (2018) · Current Epidemiology Reports · doi:10.1007/s40471-018-0152-1 · Cited by 190
Can Tax Breaks Beat Geography? Lessons from the French Enterprise Zone Experience
Briant, A., Lafourcade, M., & Schmutz, B. (2015) · American Economic Journal: Economic Policy · doi:10.1257/pol.20120137 · Cited by 148
Bellon, M. R., Ntandou-Bouzitou, G. D., & Caracciolo, F. (2016) · PLoS ONE · doi:10.1371/journal.pone.0162535 · Cited by 146
“It should be stressed however, that while the variables we use as instruments are predetermined and we have argued for their exogeneity, there are neither statistical tests nor narrative justification to entirely insure that they are uncorrelated with all unobservable determinants of the final outcome, as good instrum…”
BackgroundCausal Empiricism in Quantitative Research
Samii, C. (2016) · The Journal of Politics · doi:10.1086/686690 · Cited by 127
Analytical sociology and computational social science
Keuschnigg, M., Lovsjö, N., & Hedström, P. (2018) · Journal of Computational Social Science · doi:10.1007/s42001-017-0006-5 · Cited by 90
Terrorism and the macroeconomy: Evidence from Pakistan
Mehmood, S. (2014) · Defence and Peace Economics · doi:10.1080/10242694.2013.793529 · Cited by 68
Assessing ‘what works’ in international development: meta-analysis for sophisticated dummies
Duvendack, M., Hombrados, J. G., Palmer-Jones, R., & Waddington, H. (2012) · Journal of Development Effectiveness · doi:10.1080/19439342.2012.710642 · Cited by 52
BUENO DE MESQUITA, E., & TYSON, S. A. (2020) · American Political Science Review · doi:10.1017/s0003055419000911 · Cited by 31
Five Facts about MPCs: Evidence from a Randomized Experiment
Boehm, J., Fize, E., & Jaravel, X. (2025) · American Economic Review · doi:10.1257/aer.20240138 · Cited by 30
Sustaining latrine use: Peers, policies, and sanitation behaviors
Pakhtigian, E. L., Dickinson, K. L., Orgill-Meyer, J., & Pattanayak, S. K. (2022) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2022.05.024 · Cited by 25
Addressing cross-national generalizability in educational impact evaluation
Hanushek, E. A. (2021) · International Journal of Educational Development · doi:10.1016/j.ijedudev.2020.102318 · Cited by 20
Metadata
- Authors
- Deaton, A.
- Venue
- Journal of Economic Literature
- Publisher
- American Economic Association
- Published
- 2010-06-01
- Volume
- 48
- Issue
- 2
- Pages
- 424-455
- ISSN
- 0022-0515 (ISSN-L)
- OpenAlex
- W2103130810
- Cited by
- 1,286
- References
- 145
- FWCI
- 59.25
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.