Accounting for Nonresponse in Election Polls: Total Margin of Error

The potential impact of nonresponse on election polls is well known and frequently acknowledged. Yet measurement and reporting of polling error has focused solely on sampling error, represented by the margin of error of a poll. Survey statisticians have long recommended measurement of the total survey error of a sample estimate by its mean square error (MSE), which jointly measures sampling and non-sampling errors. Extending the conventional language of polling, we think it reasonable to use the square root of maximum MSE to measure the total margin of error (TME). This paper demonstrates how to measure the potential impact of nonresponse using the concept of TME, which we suggest should be a standard feature in the reporting of election poll results. We first show how to jointly measure statistical imprecision and response bias when a pollster lacks any knowledge of the candidate preferences of non-responders. We then extend the analysis to settings where the pollster has partial knowledge that bounds the preferences of non-responders. In each setting, we derive the poll estimate that minimizes TME—a midpoint estimate—and compare it to a conventional poll estimate. We identify conditions under which the two estimates coincide, noting that the TME exceeds the margin of sampling error whenever the pollster has less than complete knowledge of the nature of nonresponse.