Dealing with randomisation bias in a social experiment: The case of ERA

Arbeitspapier

One of the most powerful critiques of the use of randomised experiments in the social sciences is the possibility that individuals might react to the randomisation itself, thereby rendering the causal inference from the experiment irrelevant for policy purposes. In this paper we set out a theoretical framework for the systematic consideration of "randomisation bias", and provide what is to our knowledge the first empirical evidence on this form of bias in an actual social experiment, the UK Employment Retention and Advancement (ERA) study. Specifically, we empirically test the extent to which random assignment has affected the process of participation in the ERA study. We further propose a non-experimental way of assessing the extent to which the treatment effects stemming from the experimental sample are representative of the impacts that would have been experienced by the population who would have been exposed to the program in routine mode. We consider both the case of administrative outcome measures available for the entire relevant sample and of survey-based outcome measures. For the case of survey outcomes we extend our estimators to also account for selective non-response based on observed characteristics. Both for the case of administrative and survey data we further extend our proposed estimators to deal with the nonlinear case of binary outcomes.

Object Details

• Language: Englisch

References and Relationships

• Bibliographic citation: Series: IFS Working Papers ; No. W14/10

Classification and Topics

• Classification: Wirtschaft ; Single Equation Models; Single Variables: Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions ; Demographic Economics: Public Policy ; Wages, Compensation, and Labor Costs: Public Policy
• Subject: social experiments ; randomisation bias ; sample selection ; treatment effects ; matching methods ; reweighting estimators

Contributors, Places and Time

• Event: Geistige Schöpfung
• (who): Sianesi, Barbara
• Event: Veröffentlichung
• (who): Institute for Fiscal Studies (IFS)
• (where): London
• (when): 2014

Further information

• DOI: doi:10.1920/wp.ifs.2014.1410
• Handle: http://hdl.handle.net/10419/101337
• Last update: 10.03.2025, 11:41 AM CET