UPDATED: Health care statistics, undermining health care debate?

Peter Orszag
Image via Wikipedia

UPDATE: I had originally written this post based mainly off of the NY Times piece linked to below, coupled with my own knowledge of health care policy, but it seems that this article has sparked a vigorous debate in the blogosphere regarding just how accurate it's claims about the Dartmouth Health Atlas are.  I have edited parts of my post below to take into account new information, but for more, see Kevin Drum at Mother Jones, David Leonhardt at the NY Times, or the Dartmouth Health Atlas authors' response.  Needless to say, the situation is far more complex than the Times writers acknowledge, involving statistical analysis and the extrapolation of data.  More info below my blog posting.

____________________________________________________________

There's a fascinating article out today in the New York Times about a Dartmouth study on health care spending and the resultant health outcomes across hospitals, states, and regions of the U.S.  The study was used by the Obama Administration and Congressional Democrats to make the case for the efficacy of cutting health care spending while actually improving patients' health outcomes.  The article paints a picture of the Administration's efforts to sell their plan via the study:

Mr. Orszag even displayed maps produced by Dartmouth researchers that appeared to show where the waste in the system could be found. Beige meant hospitals and regions that offered good, efficient care; chocolate meant bad and inefficient.

The maps made reform seem relatively easy to many in Congress, some of whom demanded the administration simply trim the money Medicare pays to hospitals and doctors in the brown zones. The administration promised to seriously consider doing just that. (emphasis added)

Mr. Orszag is Peter Orszag, Budget Director of the Office of Management and Budget at the White House, and one of the primary cheerleaders for health care reform within the Administration.  Color-coded maps are like catnip to legislators eager to provide pseudo-intellectual cover for their legislative aims, as they can simply point to the maps to make their overly simplistic arguments to their constituents.  The idea that politicians would ever think that health care reform is easy is truly scary.  We really need more economists and statisticians in Congress.

But while the research compiled in the Dartmouth Atlas of Health Care has been widely interpreted as showing the country’s best and worst care, the Dartmouth researchers themselves acknowledged in interviews that in fact it mainly shows the varying costs of care in the government’s Medicare program. Measures of the quality of care are not part of the formula. (emphasis added)

Did you catch that?  The Dartmouth study solely looks at the cost of care, not the quality of care, yet somehow or another the link between lower spending and better care was made by policymakers in Washington.  How did that happen?

In addition to their hospital rankings, the Dartmouth researchers have also done separate studies of how Medicare spending affects patient care regionally. A 2003 study found that patients who lived in places most expensive for the Medicare program received no better care than those who lived in cheaper areas.

Because some regions spent nearly a third more than other regions without any apparent benefit, the Dartmouth team concluded that at least one dollar in three was wasted by Medicare. When applied generally to the nation’s health care system, that meant about $700 billion could be saved.

But as it began publicly discussing its research, the Dartmouth team often extrapolated beyond this basic finding. Not only do high-spending regions fail to provide better care, the Dartmouth team began to argue, but those regions actually offer worse care. 

While I haven't seen the exact data the Dartmouth team examined in making these claims, the argument that one-third of Medicare spending is wasteful seems like a rather large extrapolation to make.  The blunt instrument of health care spending statistics does not lend itself to nuanced analyses of the quality of doctors or nurses at a hospital or of their clinical judgments in deciding what treatments to order for patients.

Last year, New Yorker writer (and practicing doctor) Atul Gawande wrote a major article about McAllen, Texas, the city with the highest per-capita Medicare spending in the country after Miami, a city with a much higher cost-of-living and more elderly population.  Gawande describes a meeting he has with a group of McAllen doctors, one of whom responds to Gawande's inquiry about the high spending thusly:

“Come on,” the general surgeon finally said. “We all know these arguments are bullshit. There is overutilization here, pure and simple.” Doctors, he said, were racking up charges with extra tests, services, and procedures. 

The surgeon came to McAllen in the mid-nineties, and since then, he said, “the way to practice medicine has changed completely. Before, it was about how to do a good job. Now it is about ‘How much will you benefit?’ ”

Naturally, when the incentives for doctors are to over-prescribe treatments so that they are compensated, they will tend to run more tests than necessary, and to focus on moving their patients into the more expensive treatment programs, rather than lower-cost preventive medicine treatments, which may be more effective.  So when the Dartmouth researchers make the arguments that higher cost equates to worse outcomes, that may indeed be true, as more health care does not always mean better health care, but it cannot be the case universally across the country.  If a doctor correctly diagnoses a serious ailment, and prescribes a necessarily expensive treatment regimen for it, then simply looking at the spending figures will not be able to take into account the correct diagnosis of that ailment.

In interviews, Dr. Fisher and Mr. Skinner acknowledged that there was no proven link between greater spending and worse health outcomes. And Dr. Fisher acknowledged the apparent inconsistency between his statements in interviews with The New York Times and those made elsewhere, saying that he was sometimes less careful in discussing his team’s research than he should be.

In any case, the more-is-worse message has resonated with insurers, whose foundations now help to finance the Dartmouth Atlas. Dartmouth researchers also created a company, Health Dialog, to consult for insurers and others on Dartmouth’s findings. Valued at nearly $800 million, the company was sold to a British insurer in 2007 and still helps to finance the Dartmouth work.

It appears that, surprise, if as a researcher you make arguments that resonate with insurance companies, they will take a financial interest in your research.

Ultimately, the problem here is twofold: 1) high per capita Medicare spending in an area may signify the overutilization of care by doctors, but to make the conclusion that higher spending is always wasteful goes too far; 2) researchers and academics who create studies that fit neatly into the narrative that the powers-that-be seek to promulgate (in this case the Obama Administration and the health insurers) will have their conclusions amplified, despite arguments to the contrary.  This resulted in the further muddying of the already opaque health care reform discourse in which nothing is quite as cut-and-dry as we might wish.  While I still believe the reform bill was a good step towards a healthier America, it's unfortunate that the system has been gamed by so many parties.

UPDATE, cont...Further muddying of the waters:

3. Some research has shown that even the most glaring cost
differences noted in the Dartmouth research are associated with
improved outcomes for those conditions where something close to random
assignment of patients to alternative patient regimes occurs. The guy
who is doing this work is Joseph Doyle at MIT. If the Times article
had described some of his work, readers would have learned something.
For example, you can be pretty sure that people who suffer coronaries
while on vacation do not choose in which county to have the heart
attack. Doyle studied such patients. He found that they did better
in Florida counties that spent most, the very counties that the
Dartmouth folks have held up for their poor average outcomes. He also
found that preemies who were a couple of grams under the threshold at
which they are described as in need of intensive care did materially
better than did preemies a couple of grams over that threshold and,
hence, treated as normal births. The weight difference was too small
to matter medically and went in the wrong direction; the intensity of
care mattered.

Doyle's reasearch and the finding on the inverse relationship between
Medicare spending and private spending, together with the fact that we
can't predict worth crap who will and who will not benefits from many
procedures means not that the Dartmouth people are wrong, but that any
savings will be very hard to achieve without doing more harm than good
and will be very slow in coming. The Administration, and Orszag in
particular, were wrong in neglecting the warning to forecasters of
Scottish economist, Alec Cairncross, 'Give a number. Or a date.
Never both.'