"The Syndrome of Useless Information" Revisited

At a time when the press seems to be starting to "get it" regarding health IT myths, and expecting significant pushback, I thought it useful to repost a link to a post I authored in Jan. 2010 entitled "The Syndrome of Useless Information."

The post is here:  http://hcrenewal.blogspot.com/2010/01/syndrome-of-useless-information.html

The IT culture of late stinks to high heaven of that phenomenon.

-- SS

WSJ, Koppel and Soumerai: A Major Glitch for Digitized Health-Care Records - Cost Savings Illusory

Two EHR experts from Ivy-league institutions (which are typically hyperenthusiast-oriented) doubt cost savings from EHR technology.

Wall Street Journal
Opinion
September 17, 2012

A Major Glitch for Digitized Health-Care Records

Savings promised by the government and vendors of information technology are little more than hype.

By STEPHEN SOUMERAI And ROSS KOPPEL

In two years, hundreds of thousands of American physicians and thousands of hospitals that fail to buy and install costly health-care information technologies—such as digital records for prescriptions and patient histories—will face penalties through reduced Medicare and Medicaid payments. At the same time, the government expects to pay out tens of billions of dollars in subsidies and incentives to providers who install these technology programs.

Which, of course, means printing more money for luxuries we cannot afford in the midst of the worst economic downturn since 1929...

The mandate, part of the 2009 stimulus legislation [I think it would be more accurate to say "sneaked into the 2009 stimulus legislation for IT industry benefit" - ed.], was a major goal of health-care information technology lobbyists and their allies in Congress and the White House. The lobbyists promised that these technologies would make medical administration more efficient and lower medical costs by up to $100 billion annually. 

By up to $100 billion annually?   (Perhaps these lobbyists need lie detector tests - and/or drug tests to see what they may have been smoking?)  I note the lobbying was covered in the Washington Post in "The Machinery Behind Health-Care Reform: How an Industry Lobby Scored a Swift, Unexpected Victory by Channeling Billions to Electronic Records" by Robert O'Harrow Jr., May 16, 2009.

Many doctors and health-care administrators are wary of such claims—a wariness based on their own experience. An extensive new study indicates that the caution is justified: The savings turn out to be chimerical.

Some still have empirical observational skills and common sense, fortunately.  In my early days of health IT involvement two decades ago as an NIH-sponsored Medical Informatics post doctoral fellow, I believed there was some cost savings possible, but certainly not in the tens or hundreds of billions per year.  The technology was not solving problems that could lead to such savings (see for instance my Dec. 2010 post "Is Healthcare IT a Solution to the Wrong Problem?").  My experiences since my fellowship have only confirmed my own early opinions.

Since 2009, almost a third of health providers, a group that ranges from small private practices to huge hospitals—have installed at least some "health IT" technology. It wasn't cheap. For a major hospital, a full suite of technology products can cost $150 million to $200 million. Implementation—linking and integrating systems, training, data entry and the like—can raise the total bill to $1 billion.

That's countless billions that could have been spent on actual healthcare delivery itself, especially for underserved populations and the poor.  Instead, that money was funnelled to the IT industry for an experimental, unregulated, unvetted technology to be applied on live patients (and without patient consent, I add).

But the software—sold by hundreds of health IT firms—is generally clunky, frustrating, user-unfriendly and inefficient. For instance, a doctor looking for a patient's current medications might have to click and scroll through many different screens to find that essential information. Depending on where and when information on a patient's prescriptions were entered, the complete list of medications may only be found across five different screens.

Examples of that from actual systems (sketched to avoid IP problems) is at my ten part series on mission-hostile health IT at this blog, shortened link http://www.tinyurl.com/hostileuserexper.

Now, a comprehensive evaluation of the scientific literature has confirmed what many researchers suspected: The savings claimed by government agencies and vendors of health IT are little more than hype.

To conduct the study, faculty at McMaster University in Hamilton, Ontario, and its programs for assessment of technology in health—and other research centers, including in the U.S.—sifted through almost 36,000 studies of health IT. The studies included information about highly valued computerized alerts—when drugs are prescribed, for instance—to prevent drug interactions and dosage errors. From among those studies the researchers identified 31 that specifically examined the outcomes in light of the technology's cost-savings claims.

With a few isolated exceptions, the preponderance of evidence shows that the systems had not improved health or saved money.

One must understand this in the context of what these systems really are, what they do, and the problems they can actually address as in my post on 'solution to the wrong problem' mentioned supra.

The authors of "The Economics of Health Information Technology in Medication Management: A Systematic Review of Economic Evaluations" found no evidence from four to five decades of studies that health IT reduces overall health costs. Three studies examined in that McMaster review incorporated the gold standard of evidence: large randomized, controlled trials. They provide the best measure of the effects of health IT systems on total medical costs.

I point out that randomized controlled trials (RCT's) are notably rare in health IT studies.  Most "confirmatory" studies on health IT benefits are less powerful retrospective/observational studies in specialized settings, with many biases, poor science, etc.  In that sense, such studies can be considered non-generalizable and anecdotal at best (more on the "anecdotes" issue is below).

Of course, the pundits and Ddulites ("Luddite" with first four letters reversed; meaning ardent hyperenthusiasts) will likely say "it will all be fixed in version 2.0" despite decades of opportunity to get health IT "right" and demonstrate generalized improved outcomes and cost savings:

... At "Health IT: Ddulites and Irrational Exuberance" and related posts (query link) I've described the phenomenon of the: 'Hyper-enthusiastic technophile who either deliberately ignores or is blinded to technology's downsides, ethical issues, and repeated local and mass failures.'

From Regenstrief Institute, one of the premier and pioneering centers for Medical Informatics research and teaching:

A study from Regenstrief, a leading health IT research center associated with the Indiana University School of Medicine, found that there were no savings, and another from the same center found a significant increase in costs of $2,200 per doctor per year. The third study measured a small and statistically questionable savings of $22 per patient each year.

That's probably not taking into account increased medical malpractice costs as exemplified here and here and by other settlements I know of but cannot mention due to my consulting role in those cases.

A primer on health IT risks, containing a number of key articles and links, is downloadable at http://www.ischool.drexel.edu/faculty/ssilverstein/HIT_issues_Primer.zip. 

In short, the most rigorous studies to date contradict the widely broadcast claims that the national investment in health IT—some $1 trillion will be spent, by our estimate—will pay off in reducing medical costs. Those studies that do claim savings rarely include the full cost of installation, training and maintenance—a large chunk of that trillion dollars—for the nation's nearly 6,000 hospitals and more than 600,000 physicians.

$1 trillion would pay for a tremendous amount of actual healthcare delivery, I note.

These system lifecycle costs, known as "total cost of ownership", are due to upgrades and necessary changes as medicine itself, including the science, business and legal components change over time.   The costs are generally underestimated ahead of time.  A good source on these issues is the article "Pessimism, Computer Failure, and Information Systems Development in the Public Sector"  (Public Administration Review 67;5:917-929, Sept/Oct. 2007).  This is a cautionary article on IT that recommends much more critical attitudes towards major IT initiatives in all sectors. 

But by the time these health-care providers find out that the promised cost savings are an illusion, it will be too late. Having spent hundreds of millions on the technology, they won't be able to afford to throw it out like a defective toaster.

There, I disagree.  We may have to "throw it out" as the disruptions to good medical care by bad health IT (BHIT) increase, unless the industry reforms itself very rapidly.  As I state in the introduction to my teaching website "Contemporary Issues in Medical Informatics: Good IT, Bad IT, and Common Examples of Healthcare Information Technology Difficulties" (started in late 1998 due, in fact, to my observations of inappropriate leadership and poor quality of health IT):

The site takes a view consistent with medicine’s core values and with patients’ rights that health IT, as a medical apparatus that increasingly forms an enterprise clinical resource management and clinician workflow control mechanism, remains experimental and should be subject to the same ethical considerations, validation processes and regulatory oversight to which almost all other medical technologies and pharmaceuticals are subjected. The site is pro-health IT, but only when the health IT is “good IT” - as opposed to “bad IT."

Good Health IT ("GHIT") is defined as IT that provides a good user experience, enhances cognitive function, puts essential information as effortlessly as possible into the physician’s hands, keeps eHealth information secure, protects patient privacy and facilitates better practice of medicine and better outcomes.

Bad Health IT ("BHIT") is defined as IT that is ill-suited to purpose, hard to use, unreliable, loses data or provides incorrect data, causes cognitive overload, slows rather than facilitates users, lacks appropriate alerts, creates the need for hypervigilance (i.e., towards avoiding IT-related mishaps) that increases stress, is lacking in security, compromises patient privacy or otherwise demonstrates suboptimal design and/or implementation.

This site challenges medically and ethically controversial views that health IT merits special accommodations in terms of freedom from scientific rigor and evidence-based practices, freedom from regulation, and freedom from accountability. 

I  hold little hope of these issues being seriously addressed - let alone solved - any time in the near future due to:

1)  the irrational exuberance about this technology, cf: Twitter feeds on hashtags #EHR, #EMR, #HCIT and similar;
2)  the industry-promoted myths and fabrications over decades;
3)  the FDA-admitted impediments to information diffusion about defects and harms (link);
4)  IOM-admitted impediments to same (link);and
5)  the latter's industry-favoring, Milquetoast 2012 recommendations on monitoring, remediation and regulation.

These are indicative of an industry that has neatly avoided any meaningful regulation and is, in plain terms, out of control.

Why are we pushing ahead to digitize even more of the health-care system, when the technology record so far is so disappointing? So strong is the belief in health IT that skeptics and their science are not always welcome. Studies published several years ago in the Journal of the American Medical Association and the Annals of Internal Medicine reported that health IT systems evaluated by their own developers were far more likely to be judged "successful" than those assessed by independent evaluators.

"Not welcome" is an understatement. 

ONC itself is guilty of bad science, as in its grossly deficient March 2011 Health Affairs paper on health IT benefits. (See my Oct. 2011 post "ONC: The Benefits Of Health Information Technology: A Review Of The Recent Literature Shows Predominantly Positive Results".  Also see my Aug. 2012 post "ONC and Misdirection Regarding Mass Healthcare IT Failure".)

So bad is the "Health IT critique not welcome" phenomemon that, for example, Medical Informatics leaders such as Bill Hersh of Oregon Health Sciences University (OHSU) openly attack those who are candid on health IT risks (and make fools of themselves via the open attacks), such as the attack I wrote of at my Sept. 2010 post "The Dangers of Critical Thinking in A Politicized, Irrational Culture".

Also common in the "Health IT Critique Not Welcome Here" club, and probably deliberately so, is mistaking risk management activities for research observations and calling the former, even when reported by credible witnesses, "anecdotal" while calling at-best weak HIT-positive studies "definitive science."  See "From a Senior Clinician Down Under: Anecdotes and Medicine, We are Actually Talking About Two Different Things" for the definitive rebuttal of that type of patient-endangering intellectual blindness or dishonesty.

Government agencies like the Office of the National Coordinator of Healthcare Information Technology (an agency of the Department of Health and Human Services) serve as health IT industry boosters. ONC routinely touts stories of the technology's alleged benefits.

But almost never the opposite, which is equally if not more important.  Further, they tout "certification" as a means of HIT safety evaluation and assurance, or do not deny that misconception, despite statements by the very certification bodies or ATCB's they appoint to the contrary.  Yet they need to come clean on this. (I've seen the claim of 'certification equals safety' in actual legal briefs in defense of health IT.)  See my Feb. 2012 post "Hospitals and Doctors Use Health IT at Their Own Risk - Even if Certified."

ONC also seems to turn a blind eye to industry-ghostwriting abuses in the public comments period of Notices of Proposed Rulemaking for "meaningful use" (itself an Orwellian term) rules.  See for instance my Sept. 2012 posts "Health IT Vendor EPIC Caught Red-Handed: Ghostwriting And Using Customers as Stealth Lobbyists - Did ONC Ignore This?" and "Was EPIC successful in watering down the Meaningful Use Stage 2 Final Rule?"

We fully share the hope that health IT will achieve the promised cost and quality benefits. As applied researchers and evaluators, we actively work to realize both goals. But this will require an accurate appraisal of the technology's successes and failures, not a mixture of cheerleading and financial pressure by government agencies based on unsubstantiated promises.

I share the same sentiments.  As I've often written, the Achilles heel of the health IT sector is its ignoring the very science and evidence-based approaches of the field - medicine - that health IT is supposed to somehow "revolutionize" and "transform."

It is, quite frankly, the utmost in hypocrisy for an industry that demands evidence-based medicine - and touts its expensive products as leading to that change - to itself not practice what I've called "evidence-based computing."

Mr (sic) Soumerai is professor of population medicine at Harvard Medical School and Harvard Pilgrim Health Care Institute. Mr. Koppel is a professor of sociology and medicine at the University of Pennsylvania and principal investigator of its Study of Hospital Workplace Culture.

Dr. Soumerai also co-authored "Don't Repeat the UK's Electronic Health Records Failure" with Anthony Avery, Professor of Primary Care at the University of Nottingham Medical School, UK, Huffington Post, Dec. 5, 2010.

I also expect to hear an "Ivy tower academics - what do they know?" refrain from the industry.  However, this will not hold water, as the Ivory tower inhabitants, as I previously stated, are almost uniformly of the Ddulite-hyperenthusist persuasion and touted when they speak or write in uncritical favor of health IT.

-- SS

Addendum:

Penn Wharton professors had a similar opinion on cost-savings in 2009:

Information Technology: Not a Cure for the High Cost of Health Care: Knowledge@Wharton, Wharton School of Business, University of Pennsylvania, June 10, 2009. (PDF version available at this link)

As I observed in my June 20, 2009 post "Improving Patient Safety In The EU: HIT Should Be Classified As Medical Devices. And, Can We Drop the "Massive Cost Savings" Fable?", in a WSJ article that same day entitled "The Myth of Prevention", Abraham Verghese, Professor and Senior Associate Chair for the Theory and Practice of Medicine at Stanford echoed the Wharton professors' doubts about the cost savings and ultimate value of electronic medical records:

... I have similar problems with the way President Obama hopes to pay for the huge and costly health reform package he has in mind that will cover all Americans; he is counting on the “savings” that will come as a result of investing in preventive care and investing in the electronic medical record among other things. It’s a dangerous and probably an incorrect projection.

I also had related observations published in a Letter to the Editor in the WSJ in February of that year:

Digitizing Medical Records May Help, but It's Complex, Feb. 18, 2009

You observe that the true political goal is socialized medicine facilitated by health care information technology. You note that the public is being deceived, as the rules behind this takeover were stealthily inserted in the stimulus bill.

I have a different view on who is deceiving whom. In fact, it is the government that has been deceived by the HIT industry and its pundits. Stated directly, the administration is deluded about the true difficulty of making large-scale health IT work. The beneficiaries will largely be the IT industry and IT management consultants.

For £12.7 billion the U.K., which already has socialized medicine, still does not have a working national HIT system, but instead has a major IT quagmire, some of it caused by U.S. HIT vendors.

HIT (with a few exceptions) is largely a disaster. I'm far more concerned about a mega-expensive IT misadventure than an IT-empowered takeover of medicine.

The stimulus bill, to its credit, recognizes the need for research on improving HIT. However this is a tool to facilitate clinical care, not a cybernetic miracle to revolutionize medicine. The government has bought the IT magic bullet exuberance hook, line and sinker.

I can only hope patients get something worthwhile for the $20 billion.

Scot Silverstein, M.D.
Faculty
Biomedical Informatics
Drexel University Institute for Healthcare Informatics
Philadelphia

Next time you hear from some pundit about enormous "cost savings" from clinical IT, remember this post.

Let's implement health IT - and do it well, i.e., good health IT (GHIT) instead of bad health IT (BHIT)- to facilitate clinicians, not to work financial miracles or "revolutionize" anything.

A slide from my Aug. 1, 2012 presentation to the Health Informatics Society of Australia at HIC2012.  Click to enlarge.

-- SS

Another Health IT Mythbuster: Doctors order more X-rays, not fewer, with computer access

This piece is in the Washington Post about a just-released piece in Health Affairs (at this link) by Danny McCormick, David H. Bor, Stephanie Woolhandler and David U. Himmelstein of Harvard Medical School.

(Woolhandler and Himmelstein have written on such topics before, such as in the Nov. 2009 article “Hospital Computing and the Costs and Quality of Care: A National Study”, Amer J Med 123:1; 40-46.)

In the abstract of the new article, the researchers note:

Giving Office-Based Physicians Electronic Access To Patients’ Prior Imaging And Lab Results Did Not Deter Ordering Of Tests

Health Aff March 2012 vol. 31 no. 3 488-496

Abstract

Policy-based incentives for health care providers to adopt health information technology are predicated on the assumption that, among other things, electronic access to patient test results and medical records will reduce diagnostic testing and save money.

To test the generalizability of findings that support this assumption, we analyzed the records of 28,741 patient visits to a nationally representative sample of 1,187 office-based physicians in 2008. Physicians’ access to computerized imaging results (sometimes, but not necessarily, through an electronic health record) was associated with a 40–70 percent greater likelihood of an imaging test being ordered. The electronic availability of lab test results was also associated with ordering of additional blood tests.
The availability of an electronic health record in itself had no apparent impact on ordering; the electronic access to test results appears to have been the key. These findings raise the possibility that, as currently implemented, electronic access does not decrease test ordering in the office setting and may even increase it, possibly because of system features that are enticements to ordering. We conclude that use of these health information technologies, whatever their other benefits, remains unproven as an effective cost-control strategy with respect to reducing the ordering of unnecessary tests.

The WaPo offered these observations in their article:

Doctors order more X-rays, not fewer, with computer access

By Lena H. Sun, Published: March 5

In the debate over the high cost of health care, federal policymakers have always claimed that one way to cut costs is for doctors to use electronic medical records and other information technology. Doing so, they say, avoids duplication and saves money.
But new research suggests that may not be the case.

Doctors who have easy computer access to results of X-rays, CT scans and MRIs are 40 to 70 percent more likely to order those kinds of tests than doctors without electronic access, according to a study to be published in the March issue of the journal Health Affairs.

This is not surprising to me. Click a button, order an test, click a button, get the results. What could be easier?
What the pundits failed (and continually fail) to take into account is the social context of health IT. It is not used on a robotic assembly line. There are incentives, both proper and perverse (e.g., "defensive medicine", profits, patient demands) that figure into how any IT will "play out" in healthcare.
Making the blanket statement that "IT in healthcare will save money" without solid, sustained evidence (as opposed to the anecdotal evidence that now exists), while simultaneously ignoring contrary evidence is simply promoting an industry-created meme, a.k.a. wishful thinking.

“On average, this is comparing doctors who had electronic medical records and those who didn’t,” said lead author Danny McCormick, a physician and assistant professor of medicine at Harvard Medical School.
Researchers say the findings challenge a key premise of the nation’s multibillion-dollar effort to promote the widespread adoption of health information technology.
“This should give pause to those making the argument,” McCormick said. Instead of saving money, that effort could drive costs higher, he said.
For the study, researchers at the Cambridge Health Alliance, a health system in Cambridge, Mass., and the City University of New York analyzed data from a 2008 federal government survey. The data included information collected from 28,741 patient visits to 1,187 office-based doctors. The information included the type of doctor, their office computerization and the tests ordered at each visit. About half of the doctors’ offices surveyed had computerized access to results of X-rays and other imaging tests.
Researchers found that doctors who did not have computerized access ordered imaging tests in 12.9 percent of visits, while doctors with electronic access ordered imaging in 18 percent of visits, a 40 percent greater likelihood. Doctors with computerized access were even more likely — about 70 percent more likely — to order advanced imaging tests, such as PET scans, which experts said are most commonly used to detect cancer, heart problems, brain disorders and other central nervous system disorders.
The study found the results hold true even after taking into account other factors, such as patient demographics, doctor specialty and physician self-referral.

Unfortunately, nothing up to and including HIT-related patient deaths will give pause to health IT pundits pushing national rollout. They have too much skin in the game in the health IT bubble, psychologically (link) and financially.

The pundits need to be opposed by medical and legal professionals in an organized, systematic fashion until such time as this industry follows the National Academies' direct recommendations on getting health IT "right."

... Researchers were not able to determine why physicians ordered the imaging tests, or whether in those cases, physicians had looked at patients’ prior chest X-rays. Nor were they able to assess whether the increased imaging helped or harmed patients.

That needs to be followed up upon, obviously. Will it? Probably not.

At some flagship hospitals with customized health information technology systems, for example, doctors get specific feedback about diagnostic tests that can result in ordering fewer tests.

Note the word "can." (After decades there is no rigorous, unopposed proof, yet we are embarking on a national rollout with penalties for non-adopters?) This rationalization is both the "it will be better in version 2.0" and wishful thinking combined.

Michael Furukawa, a health economist in the Office of the National Coordinator for Health Information Technology, the administration’s health IT czar, said the researchers’ focus was not deep enough to support the study’s conclusions.
“The data are sound, the methods are appropriate, but the focus is limited,” he said. “They only looked at one piece of health IT.”

"But the focus is limited?"

So what?

That was all they intended to look at, and the result was concerning. Moving the goalposts is not a valid refutation of research.

On the other hand, coming from a political office that clearly does not understand how to conduct qualitative research and creates political promotion pieces masquerading as "research", such a statement is not surprising. See "ONC: "The Benefits Of Health Information Technology: A Review Of The Recent Literature Shows Predominantly Positive Results" at this link, where essential research methodologies were thrown under the bus for publication in Health Affairs.

At least the deviations from rigorous research methodologies were admitted:

“... Our findings must be qualified by two important limitations: the question of publication bias [e.g., bias in evidence selection - ed.], and the fact that we implicitly gave equal weight to all studies regardless of study design or sample size.”

Unfortunately, the media, politicians, financial decisionmakers and others are likely not to really comprehend, in-depth, the full significance of that sentence.

I can only imagine the reaction to such an excuse made by pharma for the mainstreaming of a risk-prone drug.

I personally take with the greatest of skepticism anything coming from ONC, an increasingly politicized government office in my observation.

You should, too, in my opinion.

-- SS