Amanda Kowalski (University of Michigan)

Title

Counting Defiers in Health Care: A Design-Based Model of an Experiment Can Reveal Evidence Against Monotonicity

Abstract

We show that a design-based model of an experiment with a binary intervention and outcome can reveal empirical evidence against a ``monotonicity'' assumption that the intervention affects all subjects in weakly the same direction. A canonical sampling-based model cannot, but we show that other sampling-based models can. Using statistical decision theory, we propose a maximum likelihood decision rule that does not assume monotonicity and provide conditions for its optimality. Under these conditions, we calculate the exact performance of our rule in small samples and show that the gains relative to a rule that assumes monotonicity grow with the sample size. In a real experiment in health care, we use visualizations of potential outcomes to illustrate evidence against monotonicity, which we quantify with a likelihood ratio. Despite a large and statistically significant average effect, our rule reveals positive counts of compilers affected in one direction and defiers affected in the other.