Experimenting with Measurement Error: Techniques with Applications to the Caltech Cohort Study

Measurement error is ubiquitous in experimental work. It leads to imperfect statistical controls, attenuated estimated effects of elicited behaviors, and biased correlations between characteristics. We develop simple statistical techniques for dealing with experimental measurement error. These techniques are applied to data from the Caltech Cohort Study, which conducts repeated incentivized surveys of the Caltech student body. We illustrate the impact of measurement error by replicating three classic experiments, and showing that results change substantially when measurement error is taken into account. Collectively, these results show that failing to properly account for measurement error may cause a field-wide bias: it may lead scholars to identify "new" effects and phenomena that are actually similar to those previously documented.