Accounting for No-Shows in Experimental Evaluation Designs
Abstract
This article examines how to estimate the effect of a program in the presence of no- shows—persons who are assigned to the program but do not participate. The article briefly discusses the methodological problems involved, describes two current experimental evaluations that are subject to these problems, presents several estimators that overcome these problems, outlines the conditions necessary for these estimators to be feasible, and describes two extensions of the analysis that illustrate a potentially broad range of further applications.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
CACE modeling is one analytic method for incomplete adoption.
“One analytical approach to accounting for incomplete adoption is to use Complier Average Causal Effect (commonly known as CACE) modeling (3, 9, 60).”
From An Overview of Research and Evaluation Designs for Dissemination and Implementation (2017) · cites this paper as [9]
Cites
Attrition Bias in Experimental and Panel Data: The Gary Income Maintenance Experiment
Hausman, J. A., & Wise, D. A. (1979) · Econometrica · doi:10.2307/1914193 · Cited by 505 · see citing works
Campbell, D. T. (1971) · Urban Affairs Quarterly · doi:10.1177/107808747100700202 · Cited by 47 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Metcalf, C.E., 1979, A Research Design for the Employment Opportumty Pilot Projects
- Smith, D.A., 1983, The Downriver Community Conference Economic Readjustment Activity Program: Impact Findings from the First Phase of Operations
Cited by
Sorted by citations
Recent Developments in the Econometrics of Program Evaluation
Imbens, G. W., & Wooldridge, J. M. (2009) · Journal of Economic Literature · doi:10.1257/jel.47.1.5 · Cited by 4,926 · see citing works
The Economics and Econometrics of Active Labor Market Programs
Heckman, J. J., Lalonde, R. J., & Smith, J. A. (1999) · Handbook of labour economics · doi:10.1016/s1573-4463(99)03012-6 · Cited by 2,992 · see citing works
Angrist, J. D., & Krueger, A. B. (2001) · Journal of Economic Perspectives · doi:10.1257/jep.15.4.69 · Cited by 2,355 · see citing works
(2009)
Policies to Foster Human Capital
Heckman, J. J. (1999) · Research in Economics · doi:10.1006/reec.1999.0225 · Cited by 1,824 · see citing works
“Source: Bloom et al. (1993). Enrollee estimates obtained using the procedure in Bloom (1984). †At any time during the follow-up period.”
MethodTraining, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects
Lee, D. S. (2009) · The Review of Economic Studies · doi:10.1111/j.1467-937x.2009.00536.x · Cited by 1,416 · see citing works
Moving to Opportunity in Boston: Early Results of a Randomized Mobility Experiment
Katz, L. F., Kling, J. R., & Liebman, J. B. (2001) · The Quarterly Journal of Economics · doi:10.1162/00335530151144113 · Cited by 1,115 · see citing works
Neighborhoods, obesity, and diabetes—a randomized social experiment.
Ludwig, J., Sanbonmatsu, L., Gennetian, L., Adam, E., Duncan, G. J., Katz, L. F., Kessler, R. C., Kling, J. R., Lindau, S. T., Whitaker, R. C., et al. (2011) · New England Journal of Medicine · doi:10.1056/nejmsa1103216 · Cited by 964 · see citing works
Moving to Opportunity: an Experimental Study of Neighborhood Effects on Mental Health
Leventhal, T., & Brooks-Gunn, J. (2003) · American Journal of Public Health · doi:10.2105/ajph.93.9.1576 · Cited by 776 · see citing works
“To supplement these regression analyses, which in all likelihood represent an underestimation of the program’s effects because of the relatively low take-up rate, treatment-ontreated (TOT) effects were estimated with 2-stage least squares regression or instrumental variable analysis.(15,16) The first model used random…”
MethodCausal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · Thammasat University Digital Collections · Cited by 838 · see citing works
Love, J. M., Kisker, E. E., Ross, C., Raikes, H., Constantine, J., Boller, K., Brooks-Gunn, J., Chazan-Cohen, R., Tarullo, L. B., Brady-Smith, C., et al. (2005) · Developmental Psychology · doi:10.1037/0012-1649.41.6.885 · Cited by 800 · see citing works
“To estimate impacts for participants only, we divided the impacts for eligible applicants (all who had applied for Early Head Start enrollment and were randomized) by the site’s program group participation rate (Angrist, Imbens, & Rubin, 1996; Bloom, 1984).”
BackgroundNeighborhood Effects on the Long-Term Well-Being of Low-Income Adults
Ludwig, J., Duncan, G. J., Gennetian, L. A., Katz, L. F., Kessler, R. C., Kling, J. R., & Sanbonmatsu, L. (2012) · Science · doi:10.1126/science.1224648 · Cited by 747 · see citing works
Little, R. J., & Rubin, D. B. (2000) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.21.1.121 · Cited by 721 · see citing works
“Our derivation of this estimator makes explicit assumptions that are hidden in other formulations (5).”
BackgroundAbadie, A., Angrist, J. D., & Imbens, G. W. (2002) · Econometrica · doi:10.1111/1468-0262.00270 · Cited by 728 · see citing works
Urban Poverty and Juvenile Crime: Evidence from a Randomized Housing-Mobility Experiment
Ludwig, J., Duncan, G. J., & Hirschfield, P. (2001) · The Quarterly Journal of Economics · doi:10.1162/00335530151144122 · Cited by 661 · see citing works
“…are similar across groups.16 Table I also presents background information for the MTO teens “at risk” for criminal involvement during the postprogram period, which we initially de ne as the 336 teens who are at least eleven but less than sixteen years of age at the time of random assignment.”
BackgroundMethodKling, J. R., Ludwig, J., & Katz, L. F. (2005) · The Quarterly Journal of Economics · doi:10.1162/0033553053327470 · Cited by 724 · see citing works
“The TOT impact can be calculated as the ITT effect divided by the difference in treatment take-up rates [Bloom 1984].”
MethodThe Parties in Our Heads: Misperceptions about Party Composition and Their Consequences
Ahler, D. J., & Sood, G. (2018) · The Journal of Politics · doi:10.1086/697253 · Cited by 586 · see citing works
The Econometrics of Randomized Experiments
Athey, S. C., & Imbens, G. W. (2017) · Handbook of economic field experiments · doi:10.1016/bs.hefe.2016.10.003 · Cited by 561 · see citing works
“t deals directly with the non-compliance is to use instrumental variables methods and related methods based on principal stratification (Frangakis and Rubin, 2002; Barnard, Du, Hill, and Rubin, 1998). Bloom (1984), Zelen (1979, 1990), Baker and Lindeman (1994), Cuzick, Edwards, and Segnan (1997), contain early and inde…”
BackgroundMethodBayesian inference for causal effects in randomized experiments with noncompliance
Imbens, G. W., & Rubin, D. B. (1997) · The Annals of Statistics · doi:10.1214/aos/1034276631 · Cited by 382 · see citing works
“…the discussion in Angrist, Imbens and Rubin (1996)], monotonicity is especially plausible in the context of a randomized trial of a drug whose access is restricted to units assigned to take it, implying Di 0 = 0 for all i, as in the application in Section 6 and in Zelen (1979) and Bloom (1984).”
BackgroundRecent Developments in the Econometrics of Program Evaluation
Imbens, G. W., & Wooldridge, J. M. (2008) · National Bureau of Economic Research · doi:10.3386/w14251 · Cited by 557 · see citing works
Evaluating Anti-Poverty Programs
Ravallion, M. D. (2005) · World Bank, Washington, DC eBooks · doi:10.1596/1813-9450-3625 · Cited by 519 · see citing works
An Overview of Research and Evaluation Designs for Dissemination and Implementation
Brown, C. H., Curran, G., Palinkas, L. A., Aarons, G. A., Wells, K. B., Jones, L., Collins, L. M., Duan, N., Mittman, B. S., Wallace, A., et al. (2017) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031816-044215 · Cited by 509 · see citing works
“One analytical approach to accounting for incomplete adoption is to use Complier Average Causal Effect (commonly known as CACE) modeling (3, 9, 60).”
MethodThinking, fast and slow? Some field experiments to reduce crime and dropout in Chicago
Heller, S. B., Shah, A. K., Guryan, J., Ludwig, J., Mullainathan, S., & Pollack, H. A. (2017) · The Quarterly Journal of Economics · doi:10.1093/qje/qjw033 · Cited by 493 · see citing works
“…in the program (defined as having attended 1 session) for those who participate, which we estimate using two-stage least squares with random assignment (Zis) as an instrumental variable (IV) for participation (Pist), as in equations (2) and (3) (Bloom 1984; Angrist, Imbens, and Rubin 1996).”
Dynamics of performance measurement systems
Bititci, U., Turner, U., & Begemann, C. (2000) · International Journal of Operations & Production Management · doi:10.1108/01443570010321676 · Cited by 459 · see citing works
Does Job Corps Work? Impact Findings from the National Job Corps Study
Schochet, P. Z., Burghardt, J., & McConnell, S. (2008) · American Economic Review · doi:10.1257/aer.98.5.1864 · Cited by 447 · see citing works
Nonparametric Analysis of Randomized Experiments with Missing Covariate and Outcome Data
Horowitz, J. L., & Manski, C. F. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473902 · Cited by 429 · see citing works
“The identifiability of treatment effects within the subpopulation of persons who comply with treatment has been also studied (Bloom 1984; Angrist, Imbens, and Rubin 1996).”
BackgroundThe impact of a carbon tax on manufacturing: Evidence from microdata
Martin, R., de Preux, L. B., & Wagner, U. J. (2014) · Journal of Public Economics · doi:10.1016/j.jpubeco.2014.04.016 · Cited by 397 · see citing works
Neighborhoods and Academic Achievement: Results from the Moving to Opportunity Experiment
Sanbonmatsu, L., Kling, J. R., Duncan, G. J., & Jeanne Brooks-gunn, B. (2006) · National Bureau of Economic Research · doi:10.3386/w11909 · Cited by 375 · see citing works
Heckman, J. J., & Vytlacil, E. (2007) · Handbook of econometrics · doi:10.1016/s1573-4412(07)06071-0 · Cited by 365 · see citing works
“Imbens and Newey (2002) consider identification and estimation of the average structural function in a nonparametric model using the control variate approach, building on the work of Blundell and Powell (2004) and Altonji and Matzkin (2005).”
BackgroundMethodInfluentialTreatment Effect Heterogeneity in Theory and Practice
Angrist, J. D. (2004) · The Economic Journal · doi:10.1111/j.0013-0133.2003.00195.x · Cited by 365 · see citing works
“This is incompatible with the Normal latent-index model since it requires c0 1⁄4 )¥, but E(Di|Zi 1⁄4 0) 1⁄4 0 is an important special case in practice, most commonly in randomised trials with partial compliance only in the treated group (Bloom, 1984; Angrist and Imbens, 1991).”
InfluentialEmsley, R., Dunn, G., & White, I. R. (2010) · Statistical Methods in Medical Research · doi:10.1177/0962280209105014 · Cited by 356 · see citing works
Pelletier, K. R. (2001) · American Journal of Health Promotion · doi:10.4278/0890-1171-16.2.107 · Cited by 341 · see citing works
Jacob, B. (2003) · American Economic Review · doi:10.1257/000282804322970788 · Cited by 387 · see citing works
What Can We Learn about Neighborhood Effects from the Moving to Opportunity Experiment?
Ludwig, J., Liebman, J. B., Kling, J. R., Duncan, G. J., Katz, L. F., Kessler, R. C., & Sanbonmatsu, L. (2008) · American Journal of Sociology · doi:10.1086/588741 · Cited by 321 · see citing works
Opiates for the Matches: Matching Methods for Causal Inference
Sekhon, J. S. (2009) · Annual Review of Political Science · doi:10.1146/annurev.polisci.11.060606.135444 · Cited by 316 · see citing works
Moving to Opportunity: Interim Impacts Evaluation
Orr, L. L., Feins, J. D., Jacob, R. T., Beecroft, E. A., Sanbonmatsu, L., Katz, L. F., Liebman, J. B., & Kling, J. R. (2003) · Cited by 305 · see citing works
Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects
Lee, D. S. (2005) · National Bureau of Economic Research · doi:10.3386/w11721 · Cited by 298 · see citing works
Moving From the Lab to the Field: The Role of Fidelity and Achieved Relative Intervention Strength
Hulleman, C. S., & Cordray, D. S. (2009) · Journal of Research on Educational Effectiveness · doi:10.1080/19345740802539325 · Cited by 291 · see citing works
“…approach is often utilized to estimate the effects of an intervention for only those treatment participants who actually received the intervention, that is, complier average causal effect estimation (e.g., Angrist, Imbens, & Rubin, 1996; Bloom, 1984; Jo, Asparouhov, Muthen, Ialongo, & Brown, 2008).”
BackgroundRecent developments in the econometrics of program evaluation
Wooldridge, J. M., & Imbens, G. W. (2008) · doi:10.1920/wp.cem.2008.2408 · Cited by 323 · see citing works
Self-Fulfilling Misperceptions of Public Polarization
Ahler, D. J. (2014) · Journal of Politics · doi:10.1017/s0022381614000085 · Cited by 168 · see citing works
Calcagno, J. C., & Long, B. T. (2008) · National Bureau of Economic Research · doi:10.3386/w14194 · Cited by 274 · see citing works
Impact of the JOBS intervention on unemployed workers varying in risk for depression
Vinokur, A. D., Price, R., & Schul, Y. (1995) · American Journal of Community Psychology · doi:10.1007/bf02506922 · Cited by 261 · see citing works
“This 8-item measure had a Cronbach alpha coefficient of .83. An internal control orientation measure was based on 10 items from the original Rotter I-E scale that were demonstrated by Gurin et al. (1978) to best capture a personal, rather than ideological, orientation.”
MethodBloom, H. S., Orr, L. L., Bell, S. H., Cave, G., Doolittle, F., Lin, W. T., & Bos, J. M. (1997) · The Journal of Human Resources · doi:10.2307/146183 · Cited by 256 · see citing works
Self-citationHotz, V. J., Mullin, C. H., & Sanders, S. G. (1997) · The Review of Economic Studies · doi:10.2307/2971732 · Cited by 255 · see citing works
Contraception as Development? New Evidence from Family Planning in Colombia
Miller, G. (2009) · The Economic Journal · doi:10.1111/j.1468-0297.2009.02306.x · Cited by 245 · see citing works
Regression–Discontinuity Analysis: A Survey of Recent Developments in Economics
van der Klaauw, W. (2008) · Labour · doi:10.1111/j.1467-9914.2008.00419.x · Cited by 242 · see citing works
“Moreover, although the sharp design represents a special case of selection on observables (Heckman and Robb, 1985), common solutions to such selection problems, such as propensity score-matching methods, are not applicable here as the design violates the strong ignorability condition of Rosenbaum and Rubin (1983), whi…”
BackgroundMethodInfluentialCareer Academies: Impacts on Students' Engagement and Performance in High School.
Kemple, J. J., & Snipes, J. C. (2000) · Cited by 234 · see citing works
Little, R. J. A., & Yau, L. H. Y. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.2.147 · Cited by 234 · see citing works
“which is the instrumental variable estimator of the complier-average causal effect derived by Bloom (1984). An equivalent expression is:”
BackgroundMethodInfluentialA Randomized Study of Neighborhood Effects on Low-Income Children's Educational Outcomes.
Leventhal, T., & Jeanne Brooks-gunn, B. (2004) · Developmental Psychology · doi:10.1037/0012-1649.40.4.488 · Cited by 226 · see citing works
A Review of Health-Related Outcomes of Multi-Component Worksite Health Promotion Programs
Heaney, C. A., & Goetzel, R. Z. (1997) · American Journal of Health Promotion · doi:10.4278/0890-1171-11.4.290 · Cited by 219 · see citing works
Instrumental variables methods in experimental criminological research: what, why and how
Angrist, J. D. (2006) · Journal of Experimental Criminology · doi:10.1007/s11292-005-5126-x · Cited by 209 · see citing works
“6 The fact that a randomized trial with one-sided non-compliance can be used to estimate the effect of treatment on the treated was first noted by Bloom (1984) .”
Summer jobs reduce violence among disadvantaged youth
Heller, S. B. (2014) · Science · doi:10.1126/science.1257809 · Cited by 206 · see citing works
A Practical Guide to Regression Discontinuity
Jacob, R. T., Zhu, P., Somers, M., & Bloom, H. (2012) · MDRC · Cited by 204 · see citing works
Self-citationIdentification of Causal Parameters in Randomized Studies With Mediating Variables
Sobel, M. E. (2008) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998607307239 · Cited by 202 · see citing works
“In general, the effect of treatment assignment Z on an outcome Y should not be interpreted as the effect of treatment itself unless all subjects comply with their assignments (Bloom, 1984).”
BackgroundDunn, G., Maracy, M. R., & Tomenson, B. (2005) · Statistical Methods in Medical Research · doi:10.1191/0962280205sm403oa · Cited by 203 · see citing works
Adaptive Designs for Randomized Trials in Public Health
Brown, C. H., Ten Have, T. R., Jo, B., Dagne, G., Wyman, P. A., Muthén, B., & Gibbons, R. D. (2009) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.031308.100223 · Cited by 184 · see citing works
Heaney, C. A., Price, R., & Rafferty, J. A. (1995) · Journal of Organizational Behavior · doi:10.1002/job.4030160405 · Cited by 179 · see citing works
Kling, J. R., Liebman, J. B., Katz, L. F., & Sanbonmatsu, L. (2004) · SSRN Electronic Journal · doi:10.2139/ssrn.588942 · Cited by 173 · see citing works
The Impact of CETA Programs on Earnings: A Review of the Literature
Barnow, B. S. (1987) · The Journal of Human Resources · doi:10.2307/145901 · Cited by 176 · see citing works
Manski, C. F. (2000) · Journal of Econometrics · doi:10.1016/s0304-4076(99)00045-7 · Cited by 172 · see citing works
“The term has since been adopted by Einhorn and Hogarth (1986), Camerer and Weber (1992), and other decision theorists.”
BackgroundInfluentialA Critical Survey of Empirical Methods for Evaluating Active Labor Market Policies
Smith, J. A. (2000) · RePEc: Research Papers in Economics · Cited by 166 · see citing works
Experimental Analysis of Neighborhood Effects on YouthPreprint
Kling, J. R., & Liebman, J. B. (2004) · SSRN Electronic Journal · doi:10.2139/ssrn.600596 · Cited by 160 · see citing works
“9 Inflation by the proportion in the treatment group who actually received the treatment was introduced in the program evaluation context by Bloom (1984); see Heckman, LaLonde, and Smith (1999) for a comprehensive discussion of alternative parameters of interest in the evaluation of social programs.”
BackgroundPromisingInterventions, O’Connell, M. E., Boat, T. F., & Warner, K. E. (2009) · Cited by 159 · see citing works
Battistin, E., & Rettore, E. (2007) · Journal of Econometrics · doi:10.1016/j.jeconom.2007.05.006 · Cited by 153 · see citing works
Incentives and Services for College Achievement: Evidence from a Randomized Trial
Oreopoulos, P., Lang, D. W., & Angrist, J. D. (2009) · RePEc: Research Papers in Economics · doi:10.1257/app.1.1.136 · Cited by 406 · see citing works
Revisiting field experimentation: Field notes for the future.
Shadish, W. R. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.1.3 · Cited by 154 · see citing works
Accounting for Dropouts in Evaluations of Social Programs
Heckman, J. J., Smith, J. A., & Taber, C. R. (1998) · The Review of Economics and Statistics · doi:10.1162/003465398557203 · Cited by 150 · see citing works
Statistical power in randomized intervention studies with noncompliance.
Jo, B. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.2.178 · Cited by 150 · see citing works
“Bloom (1984) introduced the idea of estimating CACE on the basis of an instrumental-variable (IV) approach, in which treatment effect estimates were adjusted by considering the noncompliance rate.”
The Benefits and Costs of Head Start
Ludwig, J., & Phillips, D. (2007) · National Bureau of Economic Research · doi:10.3386/w12973 · Cited by 149 · see citing works
Statistical Models for Causation
Freedman, D. A. (2006) · Evaluation Review · doi:10.1177/0193841x06293771 · Cited by 143 · see citing works
“This sort of estimator is discussed by, among others, Bloom (1984), Smith, Kulik, Stromsdorfer (1984), Sommer and Zeger (1991), Imbens and Angrist (1994), Angrist, Imbens, and Rubin (1996). Also see Heckman and Robb (1985).”
BackgroundEstimation of Intervention Effects with Noncompliance: Alternative Model Specifications
Jo, B. (2002) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986027004385 · Cited by 141 · see citing works
Trenholm, C. A., Devaney, B., Fortson, K., Clark, M. A., Quay, L., & Wheeler, J. (2008) · Journal of Policy Analysis and Management · doi:10.1002/pam.20324 · Cited by 138 · see citing works
Reducing Poverty through Preschool Interventions
Duncan, G. J., Ludwig, J., & Magnuson, K. (2007) · The Future of Children · doi:10.1353/foc.2007.0015 · Cited by 137 · see citing works
Dunn, G., Maracy, M. R., Dowrick, C. F., Ayuso‐Mateos, J. L., Dalgard, O. S., Hl, P., Lehtinen, V., Casey, P., Wilkinson, C., Vázquez‐Barquero, J. L., et al. (2003) · The British Journal of Psychiatry · doi:10.1192/bjp.183.4.323 · Cited by 135 · see citing works
“If there are no missing outcome data, equation (3) simplifies to the come data, equation (3) simplifies to the estimator first described by Bloom (1984). estimator first described by Bloom (1984).”
BackgroundMethodResultSchool Vouchers and Student Outcomes: Experimental Evidence from W ashington, DC
Wolf, P. J., Kisida, B., Gutmann, B., Puma, M., Eissa, N., & Rizzo, L. (2013) · Journal of Policy Analysis and Management · doi:10.1002/pam.21691 · Cited by 134 · see citing works
“To estimate the magnitude of the impact of actually using a scholarship, if offered one, we employed Bloom adjustments (Bloom, 1984).”
MethodBorman, G. D., & Dowling, N. M. (2006) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737028001025 · Cited by 135 · see citing works
Cook, P., Dodge, K., Farkas, G., Fryer, R., Guryan, J., Ludwig, J., Mayer, S., Pollack, H., & Steinberg, L. (2014) · National Bureau of Economic Research · doi:10.3386/w19862 · Cited by 130 · see citing works
Kling, J. R., Ludwig, J., & Katz, L. F. (2005) · Quarterly Journal of Economics · doi:10.1162/qjec.2005.120.1.87 · Cited by 311 · see citing works
Abadie, A., Angrist, J. D., & Imbens, G. W. (1999) · SSRN Electronic Journal · doi:10.2139/ssrn.195733 · Cited by 132 · see citing works
“Although our results also apply in more general settings with partial compliance in both treatment and control groups, one-sided noncompliance is of special importance in social experiments, and the JTPA example is not particularly unusual in this regard (see, e.g., Bloom (1984) ).3”
ResultMaking Young Voters: The Impact of Preregistration on Youth Turnout
Holbein, J. B., & Hillygus, D. S. (2015) · American Journal of Political Science · doi:10.1111/ajps.12177 · Cited by 131 · see citing works
The Core Analytics of Randomized Experiments for Social Research
Bloom, H. S. (2012) · SAGE Publications Ltd eBooks · doi:10.4135/9781446212165.n9 · Cited by 129 · see citing works
Self-citationBlack, M. M., Bentley, M. E., Papas, M. A., Oberlander, S. E., Teti, L. O., McNary, S. W., Le, K., & O'Connell, M. A. (2006) · PEDIATRICS · doi:10.1542/peds.2005-2318 · Cited by 128 · see citing works
The Long-Term Effects of Universal Preschool in Boston
Gray-Lobe, G., Pathak, P. A., & Walters, C. (2022) · The Quarterly Journal of Economics · doi:10.1093/qje/qjac036 · Cited by 128 · see citing works
Yau, L. H. Y., & Little, R. J. A. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753381887 · Cited by 127 · see citing works
“An IV estimator in the longitudinal case is obtained by tting a multivariate version of Bloom’s model (1984) using SAS Proc Mixed (SAS 1992), with outcomes yi and regression parameters now as vectors with three entries.”
BackgroundMethodInfluentialEvidence From Maternity Leave Expansions of the Impact of Maternal Care on Early Child Development
Baker, M., & Milligan, K. S. (2008) · National Bureau of Economic Research · doi:10.3386/w13826 · Cited by 127 · see citing works
Moving to Opportunity in Boston: Early Results of a Randomized Mobility ExperimentPreprint
Katz, L. F., Kling, J. R., & Liebman, J. B. (2000) · SSRN Electronic Journal · doi:10.2139/ssrn.234791 · Cited by 144 · see citing works
Estimating the Cream Skimming Effect of School Choice
Altonji, J. G., Huang, C., & Taber, C. R. (2010) · Journal of Political Economy · doi:10.1086/679497 · Cited by 124 · see citing works
Using Cluster Random Assignment to Measure Program Impacts
Bloom, H. S., Bos, J. M., & Lee, S.-W. (1999) · Evaluation Review · doi:10.1177/0193841x9902300405 · Cited by 121 · see citing works
Self-citation<i>Long‐Term Effects of Head Start on Low‐Income Children</i>
Ludwig, J., & Phillips, D. A. (2008) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1425.005 · Cited by 119 · see citing works
The Demand for Effective Charter Schools
Walters, C. (2018) · Journal of Political Economy · doi:10.1086/699980 · Cited by 120 · see citing works
“In a model with multiple charter schools the estimand for the most commonly used IV estimator does not equal the TOT for the charter sector because IV captures a different weighted average across schools.”
InfluentialPaving Streets for the Poor: Experimental Analysis of Infrastructure Effects
Gonzalez-Navarro, M., & Quintana‐Domeque, C. (2016) · The Review of Economics and Statistics · doi:10.1162/rest_a_00553 · Cited by 118 · see citing works
“C1: One-sided non-compliance (Bloom, 1984; Angrist, Imbens, and Rubin, 1996): only units assigned to receive street pavement (Z = 1) can potentially end up being unpaved (D = 0); all units assigned to the control (Z = 0) do comply with not being finally paved (D = 0).”
BackgroundA Cautionary Tale: Using Propensity Scores to Estimate the Effect of Food Stamps on Food Insecurity
Gibson‐Davis, C. M., & Foster, E. M. (2006) · Social Service Review · doi:10.1086/499416 · Cited by 117 · see citing works
Vinokur, A. D., Price, R., & Caplan, R. D. (1991) · American Journal of Community Psychology · doi:10.1007/bf00937991 · Cited by 115 · see citing works
“Using data collected in a field experiment of preventive intervention for unemployed persons (Caplan et al., 1989), this paper applied a procedure suggested by Bloom (1984) to provide estimates for the effects of the intervention on actual participants.”
MethodInfluentialDesign and Analysis of Experiments
Mattei, A., Mealli, F., & Nodehi, A. (2021) · Springer eBooks · doi:10.1007/978-3-319-57365-6_40-1 · Cited by 118 · see citing works
Inference on endogenously censored regression models using conditional moment inequalities
Khan, S. A., & Tamer, E. (2009) · Journal of Econometrics · doi:10.1016/j.jeconom.2009.01.006 · Cited by 114 · see citing works
“Given the random assignment of the type of treatment, the treatment indicator (TREAT) is a natural choice (see, e.g. Bloom (1984)) of an instrumental variable as it is correlated with LOS.”
BackgroundMethodDemocracy, Redistribution, and Political Participation: Evidence From Sweden 1919-1938
Hinnerich, B. T., & Pettersson‐Lidbom, P. (2014) · Econometrica · doi:10.3982/ecta9607 · Cited by 113 · see citing works
“Thus, our RD approach is a fuzzy design but since we only have a one-sided compliance problem, i.e., the treatment rule is binding for those above the cut-off point, the estimated treatment effect corresponds to the treatment-on-the-treated effect (Bloom 1984, Battistin and Rettore 2008).”
MethodUmansky, I. M. (2016) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373716664802 · Cited by 112 · see citing works
“Below, I describe how I account for imperfect compliance through use of the Wald estimator (Angrist, Imbens, & Rubin, 1996; Bloom, 1984).”
MethodCareer Academies: Impacts on Labor Market Outcomes and Educational Attainment.
Kemple, J. J. (2004) · MDRC · Cited by 112 · see citing works
Estimating Intervention Effects of Prevention Programs: Accounting for Noncompliance
Stuart, E. A., Perry, D. F., Le, H., & Ialongo, N. S. (2008) · Prevention Science · doi:10.1007/s11121-008-0104-y · Cited by 112 · see citing works
“In this paper we describe these methods, known broadly as “instrumental variable” (IV) or CACE approaches (Angrist et al. 1996; Bloom 1984; Little and Rubin 2000).”
MethodLittle, R. J., Long, Q., & Lin, X. (2009) · Biometrics · doi:10.1111/j.1541-0420.2008.01066.x · Cited by 111 · see citing works
“compliance) divided by the proportion of compliers in the treatment group ( Bloom 1984;”
Background
- Authors
- Bloom, H. S.
- Affiliations
- Harvard University (US)
- Corresponding author
- Bloom, Howard S.
- Venue
- Evaluation Review
- Publisher
- SAGE Publications
- Published
- 1984-04
- Volume
- 8
- Issue
- 2
- Pages
- 225-246
- ISSN
- 0145-4692 (ISSN-L)
- 0193-841X
- 1552-3926
- OpenAlex
- W2046165968
- Cited by
- 532
- References
- 2
- Citations by year
- 2026: 5 · 2025: 4 · 2024: 15 · 2023: 9 · 2022: 18 · 2021: 13 · 2020: 15 · 2019: 14 · 2018: 14 · 2017: 17 · 2016: 22 · 2015: 22 · 2014: 29 · 2013: 19 · 2012: 38
- FWCI
- 1.35
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Evaluation Review (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- missed appointments, program participation, program evaluation, experimental design, estimation bias, non-compliance, treatment effect estimation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
