Month: November 2014

You don’t need RCTs to know what works

In recent times a great deal has been made of the importance of using RCTs in education in order to build a body of evidence about which practices are effective. This is very laudable, but as others have pointed out, there are many implementation issues with RCTs. My concern here is not so much with the implementation, as with their usefulness – or possible lack thereof. While the statistical power of a randomised controlled trial potentially provides greater confidence in the outcome, interpretation of the data needs to be approached cautiously. Take, for example, a trial using a randomised control group. The experimental group receives a reading intervention and the other does not. The mean of the intervention group improves more than the control group. How much does this tell us? Who gained the most benefit? Can we be confident that the intervention alone was responsible for the raised average? Did most students move a little, or did a few move a lot. Overall, it can be argued, on an average basis the intervention is more effective than no intervention. But …