Today, I stumbled onto a new blog,“Deeper Tweets–Sometimes 140 Isn't Enough.” There, “Medskep” has just reprinted a comment that I made on The Health Care Blog a year ago, turning it into a stand-alone post. At the time, I was responding to an essay by Dr. Nortin Hadler professor of medicine and microbiology/immunology at the University of North Carolina at Chapel Hill, and Dr. Robert McNutt, a professor of Medicine at Rush Medical College in Chicago titled “The Evidentiary Basis for a Clearly Meaningful Benefit.”  (Hadler is the author of Worried Sick: A Prescription for Health in an Overtreated America, The Last Well Person, and Stabbed in the Back).

As I re-read my comment, I decided that I would like to share it with HealthBeat readers. But since it is only a response, first let me offer some excerpts from their provocative post :

“We entered the 21st century awash in “evidence” and determined to anchor the practice of medicine [in that evidence]. There is the sense of triumph; in one generation we had displaced the dominance of theory, conviction and hubris at the bedside. The task now is to make certain that evidence serves no agenda other than the best interests of the patient.

“Evidence-based medicine is the conscientious and judicious use of current best evidence from clinical care research in the management of individual patients.”

“But, what does 'judicious' mean? What does 'current best' mean? If the evidence is tenuous, should it hold sway because it is currently the best we have? Or should we consider the result ‘negative’ pending a more robust demonstration of benefit? Ambiguity is intolerable when defining evidence because of the propensity of people to decide to do something rather than nothing. Can we and our patients make 'informed' medical decisions on thin evidentiary ice? How thin? Does tenuous evidence mean that no one is benefited or that the occasional individual may be benefited or that many may be benefited but only a very little bit? . . .

“The following are some of the major guideposts for a conversation between a patient and a physician when considering whether the results of any positive efficacy trial are relevant to the patient’s need to know, ‘Will this benefit me?’

“Small effects. The smaller the effect, the less reliable its measurement. Not only are there limitations consequent to the design of the trial, but there are limits consequent to biological variability. These are called randomization errors. When one does an RCT one attempts to randomize all relevant variables and compensate mathematically for any variables that are unequally represented. But often there are relevant variables that are not measured and others not yet quantified (such as heritability). . . That’s why a small effect may represent randomization error and not the benefit of the intervention. For trials with hard outcomes, such as death, heart attack, or stroke, our interest perks with absolute differences of 2% or greater. If we have to treat 50 or more people to benefit one, we are unimpressed, unconvinced and dissuasive in our discussions with our patients.

“The lottery mentality. But the patient says that if the intervention is reasonably benign and indemnified and benefits only 1 in 100, or 1 in 1000 treated, ‘What do I have to lose?’ To which we respond, ‘A RCT is not a lottery. You are essentially as likely to benefit without the intervention as with it. It’s like winning a lottery without buying a ticket.’

“Risk factors. So many amongst us rise up and go to sleep aware that our future is marked by some number: PSA, cholesterol, blood pressure, HbA1c, BMI, and the like. These have become surrogate markers for the disease they portend. The desire to expunge the harbinger is inescapably human and scientifically seductive. We have learned better, but it is a lesson that is hard learned. Never again should anyone be screened for anything unless the test is accurate, the disease is important, and something can be done about it.

“Secondary analysis. For the few trials that test for small, actual clinical outcomes the studies are large, prolonged and costly. There is much at stake and much pressure to eke a small effect out of the morass of data, so much pressure that industry trials are generally more likely to do so than government sponsored trials on the same agents. When all else fails, the trial is declared “negative’ but the data seldom left fallow. It is poked and prodded to look for associations that were not quite those for which the trial was designed. No one should ever assume this exercise can generate anything more than a hypothesis. But many such hypotheses have a life of their own (coronary artery bypass grafting for multi-vessel disease is a classic).

“Composite outcomes. We would not leap to purchase an automobile if we were told it was better than the competition in terms of price, reliability, safety or comfort. We’d like to know how much better and by which of the criteria. Yet trials abound touting as a positive outcome a reduction in all heart attacks, fatal heart attacks, congestive heart failure, or the ‘need’ for a procedure. . .

“So, the randomized trial may be decision activating only if the absolute benefit is large and absolute harm is low. Such a result provides evidence for action in patients with characteristics similar to the subjects in the trial. More importantly, this result becomes the prerequisite ‘gold standard’ against which effectiveness can be measured in comparing outcomes in practice in the community, i.e. comparative effectiveness research. Without such, one may be seeking the lesser of two evils. But observational data from large administrative data sets is far more likely to sink in the murk of unquantifiable, unmeasured variables than to answer the question as to whether particular practices in particular populations advantages patients at least as much as the subjects in the ‘positive’ RCT. Our science, hence, must be reformed to involve more people in studies to learn how specific attributes alter an individual’s benefits and harms. . .We are talking about moving our studies closer to the public at large with enough variation in clinical situations to find who and who does not do better or worse.

“It is time for a ‘state of the union’ about the standards for medical evidence. Without a uniform standard of how to obtain evidence for individuals, we will not be able to justly adjudicate the inevitable constraints on the allocations of medical services  . . .

“Buying health care services is presently poorly planned due to our inability to study what works better than something else for individual decision makers. We can expand insurance coverage, but more importantly we should be talking about what we should cover, and why. We presently do not have standards for measures of value and we do not have enough studies that inform the individual decision maker. We do not need comparative effectiveness studies until we decide what those studies should be aimed at finding. We do not have a uniform definition of evidence, and hence, we are left advising our patients about what is valuable to buy without knowing if anyone really is getting better, or if any individual would want what we are selling in the first place.

“But no matter how well we respond to these contingencies, we will be faced with uncertainty. Who better to hold our hand when we make decisions about our health based on imperfect data that our trusted physician. Why else do we need physicians going forward?”

         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can read Hadler and McNutt’s entire post by following this link. Below, my response, originally published as a comment on The Health Care Blog:

                            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

“I am reminded of Pascal’s remark: ‘All of man’s troubles stem from his inability to sit quietly in a room alone . . .’

“Our medical schools teach doctors: ‘Don’t just sit there, Do something.’ But often, patients would be better off if a doctor just sat there–listened, observed and talked, honestly and humbly, about the uncertainties of medicine. . . I don’t think those ambiguities will ever be vanquished. After all, we have only the human mind with which to study the human mind/body. To further complicate things, each of those bodies/minds is unique. At the very best we’re looking at a stand-off.

“We want to believe that doing something will help. As Hadler puts it: “it is human nature to hope that there is a demonstrable difference in outcome between those treated with an active intervention and the referent group.’ But this can lead us to exaggerate a small effect.

“Meanwhile, ‘So many RCTs are powered for small effects.’ Whoever is running the trial wants a positive outcome. If you aim for a small effect, you are more likely to get a positive result. The profit motive plays a major role here: There is much at stake and ‘much pressure to eke a small effect out of the morass of data, so much pressure that industry trials are generally more likely to do so than government sponsored trials on the same agents.’”

“Karen [another reader] asks: ‘How can you deny a patient their long shot . . ?’

“If that long shot came with no risk (and no cost) I might agree. But the fact is that every treatment and every test carries some risk. So on one side of the equation you have the long-shot possibility of a benefit; on the other side, the likelihood of side effects or unintended consequences (for example, the test leads to unnecessary treatment.)

“Moreover, in a world of finite resources it makes no sense to squander health care dollars on ‘interventions that cannot be shown to yield a meaningful benefit in any particular population.’ Those dollars could be spent on a child’s education or in some other way where we can be pretty certain of a benefit.

“My favorite line in the post: ‘Never again should anyone be screened for anything unless the test is accurate, the disease is important, and something can be done about it.’ Amen.”