Do Medicare Report Cards Tell Consumers Anything They Don't Already Know?
The report-card effect on choice of HMO plan is entirely attributable to beneficiaries' responses to enrollee satisfaction scores (based on such factors as large parking lots and nice waiting rooms). Other reported quality measures, such as the mammography rate, did not affect enrollment.
Consumer report cards are not a new phenomenon - many organizations evaluate the quality of products and services and publish such information for consumers. Governments too have invested substantial resources in developing and disseminating quality report cards in a variety of settings, ranging from achievement of public school students to restaurant cleanliness to airline on-time performance. Perhaps the most important laboratory for these government-reporting initiatives is the health insurance market, through which nearly 15 percent of GDP flows.
In Do Report Cards Tell Consumers Anything They Don't Already Know? The Case of Medicare HMOs (NBER Working Paper No. 11420), authors Leemore Dafny and David Dranove quantify the effect of the largest public report-card experiment to date, the release of Health Maintenance Organization (HMO) report cards in 1999 and 2000 to 40 million Medicare enrollees, on the subsequent health plan choices of enrollees. They compare the magnitude of the learning induced by those report cards to that of ongoing, market-based learning by consumers.
The Balanced Budget Act of 1997 required all managed care plans participating in the Medicare program to gather and disclose quality data. The plans must report a set of standardized performance measures, collectively called The Health Plan Employer Data and Information Set. Beginning in 1998, this data was supplemented by an independent annual survey of Medicare beneficiaries. Respondents are asked a series of questions designed to assess their satisfaction with various aspects of their healthcare, including the communication skills of their physicians and the ease of obtaining care. These measures are used to construct an annual report card for every Medicare HMO. Report cards for those HMOs operating in a beneficiary's market area were mailed to all beneficiaries in November 1999 and in November 2000 in Medicare & You handbooks -- report cards were not included in subsequent handbooks, which now refer interested seniors to a website and a toll-free helpline.
The authors conclude that during the study period, 1994-2002, the report cards and market-based learning played roughly equal roles in shifting Medicare HMO enrollees to higher-quality health plans. Over that period, Medicare enrollees were switching into higher-quality plans independent of the government report cards issued in 1999 and 2000. Market-based learning attenuated over time, and was strongest in markets in which U.S. News & World Report provided report cards, and in which migration and prior HMO experience were relatively low. These findings suggest that market learning is facilitated by the private release of report cards, "word of mouth" within communities, and personal experience with HMOs.
These results are consistent with a recent survey of Medicare beneficiaries, which found that the majority who sought managed-care information used non-governmental sources. The evidence on market learning implies that prior estimates of the effects of report cards are likely biased upward, as these studies generally attribute all behavioral changes following a report card's release to the report card itself.
However, after controlling for market-based learning, Dafny and Dranove still find a significant response to the Medicare report cards. Over the entire 8-year study period, the report-card-induced enrollment changes are as large as the changes associated with market learning. But, the report-card effect is entirely attributable to beneficiaries' responses to enrollee satisfaction scores. Other reported quality measures, such as the mammography rate, did not affect enrollment.
Given that public report cards are often justified on the grounds that individuals' subjective opinions are not good measures of true quality, the authors find it surprising that satisfaction scores were included in the report cards. Also, it is potentially disconcerting that consumers would ignore an alternative, objective measure of quality that also was provided. The authors report that enrollee satisfaction is uncorrelated with the mammography rate, for example, and with other measures that are believed to reflect best practices in disease screening and prevention. Instead, enrollee satisfaction can be related to features that are not instrumental to producing better health, such as large parking lots and nice waiting rooms. The strong response of enrollees to average satisfaction ratings also creates an incentive for plans to maximize these ratings by directing resources toward "average" enrollees and away from outliers with catastrophic or expensive chronic conditions, precisely the individuals for w hom insurance is most valuable.
The authors further find that the report cards encouraged a substantial amount of switching among enrollees already in Medicare HMOs, but only drew a small fraction of enrollees in traditional Medicare into Medicare HMOs. This result is consistent with prior research in the private sector (using Preferred Provider Organizations as the outside option), and suggests that quality report cards alone will not be sufficient to convince Medicare enrollees to abandon traditional Medicare for the Medicare HMO program (currently known as Medicare Advantage).
-- Les Picker
