A Cautionary Tale in Comparative Effectiveness Research: Pitfalls and Perils of Observational Data Analysis

Administrative data such as insurance claims offer a potentially powerful data source to examine the relative benefits and costs of competing drug treatments. Motivated by a 2011 FDA warning about possible side effects of angiotensin-II receptor blockers (ARBs), we analyze the benefits and risks of ARBs compared to other classes of hypertension drugs using Medicare Parts A, B, and D data between 2006 and 2009. We study treatment adherence and crossover as well as non-random treatment assignment in detail and illustrate how different approaches to handling these issues impact our results. We find little evidence that ARBs increase cancer rates and weak evidence that they increase stroke rates, contrary to evidence from randomized controlled trials. Falsification tests raise doubts that any associations are causal. Overall, our results suggest comprehensive, robust analyses are needed in using observational data for comparative effectiveness analysis.