Hospitals and Doctors Use Health IT at Their Own Risk - Even if "Certified"

Due to my observations of confusion about health IT certification [1], and due to vague or incomplete seller language that could be misinterpreted by buyers (perhaps by design), I recently asked several ONC-ATCBs (HHS's Office of the National Coordinator for Health IT-Authorized Testing and Certification Bodies) the following.

I sent this question via email to their "questions" email addresses:

"Is EHR certification by an ATCB a certification of EHR safety, effectiveness, and a legal indemnification, i.e., certifying freedom from liability for EHR use of clinical users or organizations? Or does it signify less than that?"

One ONC-ATCB provided the following in response to my request for information.

From: Trivedi, Amit V (ICSA Labs)
Sent: Thursday, February 16, 2012 11:22 AM
To: Scot Silverstein
Subject: RE: Form submission from: Contact Us

Hello Scot,

Thanks for your email. Certification by an ATCB signifies that the product or system tested has the capabilities to meet specific criteria published by NIST and approved by the Office of the National Coordinator. In this case the criteria are designed to support providers and hospitals achieve "Meaningful Use." A subset of the criteria deal with the security and patient privacy capabilities of the system.

Here is a list of the specific criteria involved in our testing:
http://healthcare.nist.gov/use_testing/effective_requirements.html

In a nutshell, ONC-ATCB Certification deals with testing the capabilities of a system, some of them relate to patient safety, privacy and security functions (audit logging, encryption, emergency access, etc.).

What was suggested in the email below (freedom from liability for users of the system, etc.) would be out of scope for ONC-ATCB testing based on the given criteria. [I.e., certification criteria - ed.] I hope that helps to answer your question.

Thanks,

Amit

Amit Trivedi
Program Manager - Healthcare
ICSA Labs, an Independent Division of Verizon Business

Mar. 5, 2012 Addendum:  I also received a response from another ONC-ATCB, the Drummond Group:

From: Joani Hughes (Drummond Group)
Sent: Monday, March 05, 2012 1:06 PM
To: Scot Silverstein
Subject: RE: EHR certification question

Per our testing team:

It is less than that. It does not address indemnification although a certification could be used as a conditional part of some other form of indemnification function, such as a waiver or TOA, but that is ultimately out of the scope of the certification itself. Certification in this sense is an assurance that the EHR functions in way that could enable an eligible provider or eligible hospital to meet the CMS requirements of Meaningful Use Stage 1. Or to restate it more directly, CMS is expecting eligible providers or eligible hospitals to use their EHR in “meaningful way” quantified by various quantitative measure metrics and eligible providers or eligible hospitals can only be assured they can do this if they obtain a certified EHR technology.

Please let me know if you have any questions.

Thank you,
Joani.

Joani Hughes
Client Services Coordinator
Drummond Group Inc.

These are direct and clear statements.

My question was certainly answered. ONC cerification is not a safety validation, such as in a document from NASA on aerospace software safety certification, "Certification Processes for Safety-Critical and Mission-Critical Aerospace Software" (PDF) which specifies at pg. 6-7:

In order to meet most regulatory guidelines, developers must build a safety case as a means of documenting the safety justification of a system. The safety case is a record of all safety activities associated with a system throughout its life. Items contained in a safety case include the following:

• Description of the system/software
• Evidence of competence of personnel involved in development of safety-critical software and any
safety activity
• Specification of safety requirements
• Results of hazard and risk analysis
• Details of risk reduction techniques employed
• Results of design analysis showing that the system design meets all required safety targets
• Verification and validation strategy
• Results of all verification and validation activities
• Records of safety reviews
• Records of any incidents which occur throughout the life of the system
• Records of all changes to the system and justification of its continued safety

Health IT testing conspicuously lacks attention to most of the aerospace software safety points above. I note that there appears to be no reasonable excuse for such omissions.

IOM has recently studied the issue of HIT safety. IOM states in a Nov. 2011 report that HIT safety and safety testing is unsatisfactory, and has recommended HHS study it as well. IOM recommends HHS annually re-evaluate whether regulation is needed to improve safety, although IOM favors industry self-policing [2].

Thus, buyers and users of even "ONC certified" health IT are not indemnified from liability due to medical errors or problems caused by the health IT.

Sellers who exaggerate the value of certification or imply its meaning is akin to FDA device approval, likewise, could be faulted for making false representations about their products.

It would appear the sellers could potentially be sued for doing so by purchasers/users who themselves get into legal hot water due to EHR defects or other problems.

-- SS

Note:

[1] I believe confusion about EHR "certification" is in part due to the term itself. I raised objections to this term when it was first proposed based on my experience in pharma, suggesting what I felt was the more accurate expression "features qualification" instead.

[2] "Health IT and Patient Safety: Building Safer Systems for Better Care", Institute of Medicine of the National Academies, Nov. 2011, http://www.iom.edu/Reports/2011/Health-IT-and-Patient-Safety-Building-Safer-Systems-for-Better-Care.aspx

Logical Fallacies in Defense of Conflicts of Interest Employed by a Leader of Academic Medicine

We have repeatedly discussed the adverse effects of conflicts of interest on health care.  Recently, I argued that the most pernicious are conflicts of interest created as an incentive for trusted health care leaders, usually respected health care professionals or academics, to promote the vested interests of those who pay them, in the guise of the leaders' professional roles.  In this capacity, the leaders are often dubbed "key opinion leaders" by those who employ them, but may be regarded as mere "salesmen" by the corporate personnel who recruit them. (See posts here and here)  These relationships may be hidden, often behind confidentiality agreements, unless revealed by litigation.  Documents revealed by discovery in legal actions showed how companies planned other organized stealth marketing efforts for drugs that included activities by KOLs (e.g., see post here about marketing of Lexapro, and here about Neurontin).

However, defenses of conflicts of interest continue to appear regularly.  The latest example, which incorporates an important twist, appeared in yesterday's Wall Street Journal.  It was in the form of a "Boss Talk"  interview with a leader of a major health care organization (whose identity we will discuss later.)

The title of the interview indicated that the interviewee was not "worried about industry ties" of academia or of health care professionals.  Conflicts of interest were clearly its main focus.  For example, it began,

Many universities are wringing their hands over the increasing coziness of medical schools and their corporate partners.

Then it stated that the interviewee:

has no such qualms.

The defense of this lack of qualms was heavily based on logical fallacies.

Biased Sample or Hasty Generalization

The main reason for this lack of concern appeared to be complete disregard of the more serious kinds of conflict of interest briefly described above. The interviewer asked:

What are the trickiest conflicts of interest to navigate?

The answer was:

Surgery is particularly challenging. Let's say you're a physician and you come up with a new hip replacement. You invented it so you're going to make a lot of money. But if you're going to do my surgery, I want you to put in the device you think is best. How do you separate that, when the person is the inventor and the great technician?

This may be tricky, but is arguably not the most tricky kind of conflict of interest.

The trickier example of the key opinion leader hired as a salesman was not raised by the interviewer or the interviewee. Such a case was graphically revealed by testimony of the aborted TMAP trial which implicated the former state mental health director as hired by Johnson and Johnson subsidiary Janssen to "promote ­Risperdal as a safe and effective medication." (See this post.)

Industry spokespeople and key opinion leaders themselves tout KOLs as clinical, educational, and/or scientific experts chosen for their expertise to advance medicine, science and public health.  There have been  documented instances (e.g., see posts here and here) in which defectors from marketing departments of commercial health care corporations described KOLs as salespeople who could be more influential hidden within their professional or academic cloaks.  Even some physicians paid to be speakers on behalf of pharmaceutical corporations have acknowledged their role as salespeople in fancy dress (see post here).  There are cases of documents revealed by discovery in legal actions that show how companies planned organized stealth marketing efforts for drugs that included activities by KOLs (e.g., see post here about marketing of Lexapro, and here about Neurontin).

Perhaps the interviewee was unaware of these issues.  Whether due to ignorance or deliberate avoidance, failing to consider the full spectrum of conflicts of interest, and specifically ignoring those with the greatest likelihood of adverse effects, appears to be a logical fallacy in this instance.  If deliberate, it appears to be reasoning from an (intentionally) biased sample, if not, it appears to be a hasty generalization.

The interview also included a few other choice logical fallacies that went unchallenged by the interviewer.

False Dilemma

The interviewer asked:

What do you tell professors who won't work with drug or biotech companies?

The response was:

I think that's a huge mistake. If you're a professor now, and you want to get your discovery to society, you either need to start a company or work with a company to commercialize a product.

Of course, in the "good old days," academic researchers got their "discoveries to society" simply by publishing them. Developing and marketing products based on their discoveries, while worthwhile undertakings in their own rights, were not considered part of the academic mission. Professors could still do this, if their goal was not to get rich. Yet the Bayh-Dole act allowed academic institutions to make money from their professors' discoveries, and the rush to commercialize the university has been on ever since. So while professors and academic institutions who are motivated mainly by money might not consider just putting the knowledge they discover in the public domain, that course remains possible, just not so lucrative.

The assertion that the only way to get a "discovery to society" is to start or work with a company is simply false, and using this assertion in an argument appears to be an example of a false dilemma.

Straw Man

The interviewee worked another argument into the same paragraph:

When professors have told me they won't work with companies anymore because they feel they'll have this scarlet letter, I think: 'Wouldn't that be sad if all the best scientists and clinicians won't work with companies because the public has said they're evil?'

I doubt that any of even the most vociferous critics of the conflicts of interest that now befog health care have claimed that those involved are evil, much less that they are have successfully convinced the whole public at large that anyone who "works with companies" is evil. Implying that this would be the result of criticism of conflicts of interest does not appear to be supported by evidence, and is probably flat wrong. Asserting it here appears to be an example of the straw man fallacy.

Summary

So the Wall Street Journal has added to our collection of defenses of conflicts of interest that seem mainly to be based on logical fallacies. 

We have noted that logical fallacies are increasingly deployed to defend the status quo in health care, and particularly to defend the interests of those who are profiting the most from the current dysfunctional system.  We have noted that several defenses of the conflicts of interest generated by financial relationships between physicians and medical academics on one hand and commercial health care firms on the other, were based on logical fallacies.  (See examples here, here,  here, and here.)  I have yet to see a coherent, logical, fact-based argument that the benefits for patients' and the public's health of physicians and medical academics working part-time as consultants, advisers, speakers, and directors of health care corporations outweigh the obvious risks of biasing medical decision making, education and research in favor of vested interests.

In 2011, I noted, "I have also yet to see an argument in favor of conflicts of interest made by anyone who does not have such conflicts."At least, however, up to that point I had not noted any such arguments made by people who had much power to enforce their views, as opposed to the ability to just express them.  The interview discussed above, however, was a person who has such power.

The interview was with Dr Susan Desmond-Hellmann, the relatively new Chancellor of the University of California- San Francisco, who was just named one of the "25 most influential people in biopharma."  She has recently been advocating a change of direction towards commercialization for her campus, one of the most prestigious health care oriented universities/ academic medical centers in the country.  She previously justified this redirection again using logical fallacies (look here). 

She also appears to yet another advocate for conflicts of interest who has her own conflicts.  As we noted here, Dr Desmond-Hellmann had little academic experience before she became Chancellor, but had worked her way up in the corporate pharmaceutical world, leaving her position as President for drug development at Genentech after it was taken over by Roche. 

As we noted previously, even after that, Dr Desmond-Hellmann apparently has not completely left the corporate world.
- A web-site for a speakers' bureau in which she apparently still participates lists her as a current "Advisor of Genentech since April 2009."
- In 2010, Dr Desmond-Hellmann joined the board of directors of Procter and Gamble, a company which makes many health related products, although it sold its global pharmaceutical business. Last year, the company made an agreement with Teva to market over-the counter medications (see this Reuters article). Note that she got this position despite apparently not having any prior personal investment in P&G stock. However, per the company's 2011 proxy statement, she appears to be in line to collect over $250,000 a year in compensation for this position.

It does not seem impossible that these ongoing commercial interests may influence how she acts in her role as Chancellor.  Yet while it may be unsurprising, it is very disappointing that conflicts of interest are now being uncritically and illogically publicly defended by people in positions to exert so much influence on health care.

The noted cognitive psychologists George Loewenstein, Sunita Sah, and Daylian Cain just asserted in JAMA [Loewenstein G, Sah S, Cain DM. The unintended consequences of conflict of interest disclosure. JAMA 2012; 307: 669-670. Link here.]

Conflicts of interest, including fee-for-service arrangements, are at the heart of the astronomical increases in health care costs in the United States, and transparency is not substitute for more substantive reform.

True health care reform requires such substantive reform of the financial arrangements among corporations that sell health care services or products and health care professionals and others who make decisions about patients' or the public's health. To decide how to accomplish such reform, we need a better discussion informed by logic and evidence, sans logical fallacies. Those who lead health care ought to be able to participate in this discussion under these conditions.