Making The Most Out Of Programme Evaluations and Social Experiments: Accounting For Heterogeneity in Programme Impacts

Abstract

The conventional approach to social programme evaluation focuses on estimating mean impacts of programmes. Yet many interesting questions regarding the political economy of programmes, the distribution of programme benefits and the option values conferred on programme participants require knowledge of the distribution of impacts, or features of it. This paper presents evidence that heterogeneity in response to programmes is empirically important and that classical probability inequalities are not very informative in producing estimates or bounds on the distribution of programme impacts. We explore two methods for supplementing the information in these inequalities based on assumptions about participant decision-making processes and about the strength in dependence between outcomes in the participation and non-participation states. Dependence is produced as a consequence of rational choice by participants. We test for stochastic rationality among programme participants and present and implement methods for estimating the option values of social programmes.