The Syndrome of Useless Information

I occasionally elevate comments and replies to the level of full posts if I feel they better illustrate and clarify significant points I raise.

In my Jan. 9, 2010 post "Does the CEO of Google Use Google? - And: Platform, Platform, Who's Got The Platform?" I lamented the intrusion of ill-informed, reductionistic, "database platform"-centric views of non-clinical IT personnel into healthcare.

I received the following feedback from an anonymous commenter:

Anonymous wrote:

Or then again, there's this:

"Non-physicians may reach correct diagnoses by using Google: a pilot study."

http://www.ncbi.nlm.nih.gov/pubmed/19130327

"Researchers found that almost six-in-10 difficult cases can be solved by using the world wide web as a diagnostic aid."

http://www.dailymail.co.uk/news/article-415562/Doctors-using-Google-diagnose-illnesses.html

I think this post [i.e., my HC Renewal post about Google's CEO and "platforms" - ed.] misses the larger point that Gawande and Schmidt were addressing (and to some extent agree on) - the "art" of the practice of medicine must be transformed to the science of the practice of medicine. And today, science and IT are necessarily intertwined in practical terms.

The future will no doubt involve ever more sophisticated and useful clinical DSS. Historically speaking, invoking the complexity argument is fraught with issue as technological advances provide better and better ways to represent and manage it.

January 16, 2010 10:03:00 AM EST

Anonymous apparently meant these article references to exemplify the coming Age of Cybernetic Miracles in medicine.

My response to this anonymous feedback was placed in the comment thread of the "Does the CEO use Google" post and covered a number of bases.

I've reproduced my response, and added additional explanatory notes not in my original response in bold red italics below:

MedInformaticsMD wrote:

Re: Anonymous Jan 16 @ 10:03 AM-

The article [the first, at nlm- ed.] you cite reports the following:

Non-physicians may reach correct diagnoses by using Google: a pilot study.

OBJECTIVE: We endeavoured to determine whether individuals who are not physicians are likely to arrive at correct diagnoses [note the stated objective carefully -- "are likely to arrive at correct diagnoses" - ed.] by using Internet resources.

METHODS: In this prospective study four non-physicians used Google to search for diagnoses. They reviewed the 26 diagnostic cases presented in the case records of the New England Journal of Medicine during 2005; they were blind to the correct diagnoses. The main measurement was the percentage of correct diagnoses arrived at by non-physicians by using Google. The diagnostic success of the four non-physicians was compared to that of four young physicians.

RESULTS: The average diagnostic success of non-physicians was 22.1% (95% confidence interval [CI] 4.5-39.7%). There was no statistically significant difference between the non-physicians regarding this outcome (p = 0.11). They took 8.9 +/- 6.7 (mean +/- standard deviation) minutes for case record reading and 17.4 +/- 7.9 minutes for Google searching per case. Non-physicians performed worse than physicians (50.9% [95% CI 37.4-64.5%]) in regard to diagnostic success (p <0.001).

CONCLUSION: Non-physicians, at least those who have similar characteristics to the participants in the present study, may occasionally reach correct diagnoses by performing a brief web-based search. Doctors should realise that patients may assume a more active role in their health decision-making process and take this development into consideration in physician-patient interaction.

--------------------------

This article suffers from what can be termed "the syndrome of useless information."

[It is, in fact, a failure or negative finding with regard to its stated objective - ed.]

Let's see:

The average diagnostic success of non-physicians was 22.1%

They omit the opposite semantic: that the google-armed nonphysicians got more than three quarters of the diagnoses WRONG. Will you trust your grandmother to them?

Non-physicians performed worse than [young, a.k.a. inexperienced] physicians (50.9% [95% CI 37.4-64.5%]) in regard to diagnostic success (p <0.001).

Young, inexperienced physicians (trainees or residents?) arriving at 51% of the diagnoses correctly using ONLY google (not, for example, Harrison's, and other standard medical texts that would not be highly usable to most non-physicians lacking background to fully understand it) is not unexpected.

What is surprising is that the article omitted an essential control group: *experienced* physicians. It's not that they're hard to find.

The article concludes:

Non-physicians, at least those who have similar characteristics to the participants in the present study, may occasionally reach correct diagnoses by performing a brief web-based search.

Occasionally reach correct diagnoses? How about "most often reach incorrect diagnoses?" Cf. a broken clock is still occasionally correct.

[I repeat, the article is, in fact, a failure or negative finding with regard to its stated objective of "determining whether individuals who are not physicians are likely to arrive at correct diagnoses." They are in fact unlikely to get it right, with only a 22% hit rate, although the authors appear to have de-emphasized that fact. This will only make physicians' work harder as they "take this development into consideration", i.e., deal with patients armed with search engine-gleaned misdiagnoses - ed.]

I do not view this article as revealing anything of practical value other than perhaps the dangers of allowing non-physicians armed with search engines to think they can perform medical diagnosis to any meaningful extent.

Anonymous wrote: the "art" of the practice of medicine must be transformed to the science of the practice of medicine. And today, science and IT are necessarily intertwined in practical terms.

Reductionist views of non-medical IT personnel spouting off about 'database platforms' will not advance the science or the art of medicine.

Also, yes, IT is a tool of science; however, it and its designers and implementers facilitate science; the enablers of science are: scientists, using their insight, creativity, ingenuity, experience and expertise.

Concerning the second article [at dailymail.co.uk - ed.], I note:

"But they [the authors] stress the efficiency of the search and the usefulness of the retrieved information depend on the searchers' knowledge base." [The 'searchers' in this case were apparently - experienced physicians - ed.]

In other words, search engines can facilitate experts. That is not exactly new knowledge.

The 'art' of medicine, i.e., judgment, likewise is both indispensible, and irreproducible via "database platforms." Perhaps one day with advances in cognitive computing we will get to that point, but at present we can't even do as well as a cat. Note the statement from the IBM P.I. that "there are no computers that can even remotely approach the remarkable feats the mind perform."

Until we get there, I think it's not unreasonable to hold off on non-clinicians touting reductionist information retrieval-centric views.

Anonymous wrote: Historically speaking, invoking the complexity argument is fraught with issue as technological advances provide better and better ways to represent and manage it.

You must not have read my original post. It is advances in information science (i.e., in informatics) that will provide those advances, not advances in [information] technology. IT is a tool; information science is an activity of the mind.

Your statements clearly demonstrate a conflation of information technology and information science.

Computers facilitate information science research, but they are certainly not its sine qua non.

I should also add that the second article referenced by "Anonymous" refers to "obscure conditions such as Cushing's syndrome." I'm not exactly sure to whom Cushing's disease is an obscure disease. A non-medical IT person, perhaps?

Now, back to reality. Computers serve as aids to clinicians, when the IT is "done well." However, there is no substitute -- except in sci fi -- for expertise.

Medical expertise can only come from ~4 years of premed and then 4+ years of hard medical study in a wide variety of preclinical and clinical sciences leading to the M.D. degree, 3-4 years of postgraduate residency, and often 2+ additional years of postdoctoral fellowship training beyond residency for specialists (it should be noted, unfortunately, that there are no additional medical degrees beyond M.D., although residency and fellowship training often makes pursuing a Ph.D. in most fields seem like a cakewalk). Add to all that years of additional clinical experience in actual practice.

I am quite disturbed that this study and experience has become so casually regarded outside of medicine, for example in the IT sector. I think it reflects poorly on the IT culture specifically, and on our current culture as a whole.

Medical judgment is borne of that study and experience, not of healthcare computer dabbling typical of the business IT/MIS world in hospitals, and the all-too-often technical bachelor's degrees held by leaders in that domain. Google CEO Schmidt at least is a true computer scientist with a doctorate in CS; that does not, however, qualify him to comment on medical and medical informatics-related issues as in my prior post.

Stated frankly: he and many others with similar views are venturing far outside their competencies and as a result are talking nonsensically.

Incidentally, I believe it's time to move on from continually responding to the non-medical, IT-personnel proffered platformorrhea on how IT databases, decision support, artificial intelligence etc. will transform medicine from an art to a science, revolutionize medicine, and similar positions trafficked for at least several decades now.

They are largely manifestations of the Syndrome of Inappropriate Overconfidence in Computing (SICC syndrome), a term I coined in the 1990's and described is some detail here in a post entitled "A 21st Century Plague? The Syndrome of Inappropriate Over-Confidence in Computing." At that post I stated that:

I, for one, would welcome a cessation of claims that IT will "revolutionize" any field that depends primarily on cognition, such as biomedicine, and a return to more temperate attitudes instead of the almost bellicose grandiosity about HIT we see today. That is to say, that HIT - with proper contributions from the aforementioned specialties [e.g., social science, social informatics, biomedical informatics, HCI, etc.] will facilitate better health care, not "revolutionize" it.

In the future, similar comments as those from "Anonymous" will simply get referred back to the post you are now reading and the bolded one on SICC above.

Finally, I will not hold my breath for Emergency Medical Hologram Mark I, or R2D2 and C3PO to appear anytime soon.

-- SS

Why Are Health Care Organizations' Ethics Codes News?

This just in from the Orlando Sentinel,

The head of Central Florida's largest blood bank told Florida senators Wednesday that she has completely overhauled the operation — from the boardroom to the bloodmobile.

'Believe me, I get it,' an apologetic Anne Chinoda told members of a committee investigating the business ways of Florida's Blood Centers and other blood banks across the state.

Chinoda said FBC now prohibits board members from doing business with the agency, has instituted nine-year term limits for board members and has put together a group to review executive compensation. She also unveiled a blood-donor bill of rights she promised to publicize.

Sen. Don Gaetz, chairman of the health regulation committee that asked Chinoda to appear, said he was pleased by her testimony. 'What we saw today was an admission of inappropriate practices in the past and a promise they will be fixed,' he said.

These new ethical standards were a response to a past controversy,

Gaetz's committee has been investigating FBC, which generates $100 million in revenues annually, and the statewide industry since the summer.

The inquiry was triggered in part by a series of stories in the Orlando Sentinel that showed FBC board members have sold millions of dollars in goods and services to the agency each year; that board members had no term limits; and that Chinoda was compensated nearly $600,000 annually. Her salary was not discussed at the hearing.

Like others in the industry, FBC gets its blood for free from donors, tests and repackages it. FBC then sells the blood to 70 hospitals and medical facilities in 21 counties in Central and South Florida.

How often do you hear pleas from local blood banks about urgent shortages?  How often do you donate blood, or feel bad because you did not or could not donate?  Many people who selflessly donate blood were likely unpleasantly surprised to find out that blood banks sell (at a high price) the blood they donated, and used the proceeds to pay their executives generous salaries, and pay vendors run by their board members.

Here was another example of a previously revered local, not-for-profit health care institution run apparently for the personal benefit of its top insider leaders.

Why was it news when the organization in question adopted a code of conduct that limited self-dealing by board members, limited board members to only(?) nine-year terms, and put in place some sort or organized process to set the CEO's compensation?  Do those not seem like they ought to be standard, and not  newsworthy practices?

Similarly, an article in the Newark Star-Ledger back in November recounted how  Hackensack University Medical Center in New Jersey, whose board members had been accused of self-dealing (see this post), and to which a New Jersey state legislator had been convicted of selling his services (see this post), had instituted "tightened ethics rules," including a provision that "board members will no longer be allowed to do business with the hospital."  Again, why should a rule like this be news?  One of the duties of board members of all not-for-profit organizations is the duty of loyalty, "a standard of faithfulness; a board member must give undivided allegiance when making decisions affecting the organization."  Overpaying crony executives and authorizing business dealings with one's own business seem to obviously violate this duty.

We have noted how the business culture of health care organizations has increasingly come to resemble the larger business culture in an era of laissez-faire capitalism and the rise of new oligarchs and robber barons.  So it is telling that it was also news when UBS, a large international bank enacted a code of ethics for its employees that required them to actually comply with "foreign tax reporting rules," (per Bloomberg.)

One would have once thought that health care organizations, given their responsibilities for human life and health, and their formerly sterling reputations, would have clear, comprehensive codes of ethics that are actually enforced.   But it is news when a health care organization develops such a code (and it still may be news when such a code is enforced.) 

This post summarized some current thinking on organizational ethics policies.  In conclusion, I asked readers to think about whether their own health care organization had anything resembling such a policy.  I suspect few could identify such policies. 

Health care professionals need to inquire why health care organizations, including drug, device, biotechnology and health care information technology companies,  health care insurers, to hospitals, academic medical centers, and medical schools, etc, almost never have real organizational ethics policies.  Of course, the suspicion is that lack of such policies makes it easier for insiders to direct the organization for their personal benefit.  At least if we could make such policies the norm, we could remind organizational leaders that they are supposed to be upholding the mission, not lining their pockets.