Molly Offer-Westort presents "Adaptive experimental designs for policy learning and evaluation, with applications to Facebook Messenger studies"

When running multi-arm trials, experimenters may wish to both learn and evaluate data-driven policies; for example, learning which version of treatment is most effective and evaluating the effect of that treatment in comparison to a control condition. Response adaptive algorithms, which dynamically update treatment assignment mechanisms based on observed response, facilitate experimental designs where the most data is collected about the most effective interventions, and can improve policy learning over conventional randomized trials. I discuss design decisions when running adaptive experiments, and considerations for inference when using adaptively collected data. I review applications to Facebook Messenger studies using different adaptive algorithms.