A randomization-based perspective on analysis of variance: a test statistic robust to treatment effect heterogeneity

Summary 

Fisher randomization tests for Neyman's null hypothesis of no average treatment effect are considered in a finite-population setting associated with completely randomized experiments involving more than two treatments. The consequences of using the $F$ statistic to conduct such a test are examined, and we argue that under treatment effect heterogeneity, use of the $F$ statistic in the Fisher randomization test can severely inflate the Type I error under Neyman's null hypothesis. We propose to use an alternative test statistic, derive its asymptotic distributions under Fisher's and Neyman's null hypotheses, and demonstrate its advantages through simulations.

http://ift.tt/2BTOLVp