EHRevent.org CEO Edward Fotsch MD: The Real Challenge with EHRs is -- User Error?

Additional detailed answers to the questions I raised here and here about a new site EHRevent.org, for reporting of healthcare IT-related medical errors, can now be found at a HIStalk interview entitled "HIStalk Interviews Edward Fotsch MD, CEO, PDR Network (EHR Event)" at this link.

It is an interesting interview. I certainly find the recognition of need for an EHR/clinical IT problems reporting service a major cultural advancement in healthcare.

It's still unclear to me how -- and why -- this organization originated with little to no public knowledge and involvement, especially considering the organization types mentioned below that participated, and how it will function in interactions with myriad healthcare IT stakeholders.

Here's an explanation by Dr. Fotsch:

... We work with a not-for-profit board called the iHealth Alliance. They Alliance is made up of medical society executives, professional liability carriers, and liaison representatives from the FDA. They govern some of the networks that we run, and in exchange for that, help us recruit physicians. Professional liability carriers, for example, promote our services that send drug alerts to doctors because that’s good and protective from a liability standpoint.

In the course of our conversations with them roughly a year ago, when we were talking about adding some drug safety information into electronic health records, we came across the fact that there were concerns from the liability carriers that there was no central place for reporting adverse EHR events or near misses or potential problems or issues with electronic health records.

[Translation: the carriers saw their losses potentially increasing as a result of litigation arising from EHR-related lawsuits, and decided to do something proactive- ed.]

They were interested in creating a single place where they could promote to their insured physicians that they could report adverse EHR events. Then it turned out that medical societies had similar concerns.

[That must have been one of the best-kept secrets on Earth considering the promotion EHR's have received as a miracle-working technology, and the lack of expression of concerns from those societies - ed.]

Rather than have each of them create a system, the Alliance took on a role of orchestrating all of the interests, including some interest from the FDA and ONC in creating an electronic health record problem reporting system. That’s how it came into play.

Our role in it, in addition to having a seat on the iHealth Alliance board, was really in network operations — in running the servers, if you will, which didn’t seem like a very complicated task. Since business partners we rely on for our core business were interested in it, it was easy to say yes. It frankly turned out to be somewhat more complicated than we originally thought [I predict they haven't seen anything yet; wait until they get knee deep into real world EHR issues - ed.], but now it’s up and available.

While I find the recognition of need for an EHR/clinical IT reporting service a major advancement, I am nonetheless troubled by certain statements made by Dr. Fotsch. They seem at odds with the theoretic and empirical findings of medical informatics, social informatics, human-computer interaction and other fields relevant for health IT evaluation, and/or seem to demonstrate biases about HIT. My comments are in red italics:

Fotsch:

… Probably what we’re seeing more often than not, the real challenge with EHRs like any technology, turns out to be some form of user error.

[What about contributory or causative designer error? – ed.]

“I didn’t know it would do that"

[Why did the user not know? Lack of training, poor manuals, or overly complex information systems lacking informative messages and consistency of control-action relationships, as an example? -ed]

... or “I didn’t know that it pre-populated that"

[Why did it pre-populate? Was that inappropriate for the clinical context, such as in this example?]

... or “I didn’t know I shouldn’t cut and paste"

[Then why did the software designers enable cut and paste, without some informative message on overuse, such as length of text cut and pasted?– ed.]

... or “I wasn’t paying attention to this"

[Perhaps due to distractions from mission hostile user interfaces? -ed]

... or maybe the user interface was a little confusing

[What is "a little confusing?" (Is that like "A little pregnant?) And why was it confusing? User intellectual inadequacy, or software design issues leading to cognitive overload? - ed.]

Actual software errors appear to be the exception rather than the rule as it relates to EHR events.

["Actual software errors" are defined as, what, exactly--? Loss of database relational integrity as a result of a programming error, as apparently recently happened at Trinity Health, a large Catholic hospital chain as reported in HIStalk? Memory leaks from poor code? Buffer overflows? What?]

That’s at least as I understand it.

[Understand it from whom? Hopefully not from me or my extensive website on the issues - ed.]

In summary, a "blame the user" attitude seems apparent. There appears to be little acknowledgment of the concept of IT "errorgenicity" - the capacity of a badly designed or poorly implemented information system to facilitate error, and of the systemic nature of errors in complex organizations to which ill-done IT can contribute.

These are concepts understood long ago in mission critical settings, as in this mid 1980's piece from the Air Force cited in my previously-linked eight part series on mission hostile health IT:

From "GUIDELINES FOR DESIGNING USER INTERFACE SOFTWARE"
ESD-TR-86-278
August 1986
Sidney L. Smith and Jane N. Mosier
The MITRE Corporation
Prepared for Deputy Commander for Development Plans and Support Systems, Electronic Systems Division, AFSC, United States Air Force, Hanscom Air Force Base, Massachusetts.

... SIGNIFICANCE OF THE USER INTERFACE

The design of user interface software is not only expensive and time-consuming, but it is also critical for effective system performance. To be sure, users can sometimes compensate for poor design with extra effort. Probably no single user interface design flaw, in itself, will cause system failure. But there is a limit to how well users can adapt to a poorly designed interface. As one deficiency is added to another, the cumulative negative effects may eventually result in system failure, poor performance, and/or user complaints.

Outright system failure can be seen in systems that are underused, where use is optional, or are abandoned entirely. There may be retention of (or reversion to) manual data handling procedures, with little use of automated capabilities. When a system fails in this way, the result is disrupted operation, wasted time, effort and money, and failure to achieve the potential benefits of automated information handling.

In a constrained environment, such as that of many military and commercial information systems, users may have little choice but to make do with whatever interface design is provided. There the symptoms of poor user interface design may appear in degraded performance. Frequent and/or serious errors in data handling may result from confusing user interface design [in medicine, this often translates to reduced safety and reduced care quality - ed.] Tedious user procedures may slow data processing, resulting in longer queues at the checkout counter, the teller's window, the visa office, the truck dock, [the hospital floor or doctor's office - ed.] or any other workplace where the potential benefits of computer support are outweighed by an unintended increase in human effort.

In situations where degradation in system performance is not so easily measured, symptoms of poor user interface design may appear as user complaints. The system may be described as hard to learn, or clumsy, tiring and slow to use [often heard in medicine, but too often blamed on "physician resistance" - ed.] The users' view of a system is conditioned chiefly by experience with its interface. If the user interface is unsatisfactory, the users' view of the system will be negative regardless of any niceties of internal computer processing.

I am not entirely happy when the CEO of an organization taking on the responsibility of being a central focus for EHR error reporting makes statements that are consistent with unfamiliarity with important HIT-relevant domains, as well as a possible pro-IT, anti-user biases.

For that reason as well as the other questions raised at my prior posts (such as the onerous legal contract and apparent lack of ability of the public to easily view the actual report texts themselves), I cannot recommend use of their site for EHR problems reporting.

I recommend the continued use of the FDA facilities until such time as a compelling argument exists to do otherwise.

-- SS

Addendum 11/28/10:

This passage ends the main essay at my site "Contemporary Issues in Medical Informatics: Common Examples of Healthcare Information Technology Difficulties" and is quite relevant here:

... An article worth reviewing is "Human error: models and management", James Reason (a fitting name!), BMJ 2000;320:768-770 (18 March), http://www.bmj.com/cgi/content/full/320/7237/768:

Summary points:

• Two approaches to the problem of human fallibility exist: the person and the system approaches
  

• The person approach focuses on the errors of individuals, blaming them for forgetfulness, inattention, or moral weakness

• The system approach concentrates on the conditions under which individuals work and tries to build defenses to avert errors or mitigate their effects

• High reliability organizations---which have less than their fair share of accidents---recognize that human variability is a force to harness in averting errors, but they work hard to focus that variability and are constantly preoccupied with the possibility of failure.

-- SS

Some answers about new site "EHRevent.org" for health IT and drug adverse event reporting - and a note on incendiaries

Some answers to the questions I raised here and here about a new site EHRevent.org, for reporting of healthcare IT and drug problems, can be found in a blog post at the site of Occam Practice Management at this link: http://www.occampm.com/blog/general/ehr-event-reporting/.

Its author, Michelle R. Wood, had noted this HC Renewal post. She researched some of the questions and wrote up her findings.

It is well worth a read.

I do have a small bone to pick with her post at Occam. She wrote:

"While HC Renewal occasionally borders on the incendiary side of things, Dr Silverstein posed some valid questions about a website that seem to have caught everyone by surprise..."

I maintain that the true incendiaries are fired by those we write about, those whose pronouncements and acts are "threats to health care's core values, especially those stemming from concentration and abuse of power."

Those 'incendiary' pronouncements and acts can indeed maim and kill (for example, as a relative of mine is now experiencing thanks to a commercial EMR 'mishap').

I may be more accurate to say we don't restrict ourselves to the confines of 'political correctness', that is, stunted discourse conventions that generally favor maintenance of the status quo.

As I wrote on that issue last year here in my series on mission hostile healthcare IT:

... Some have complained I am being "politically incorrect." At a time when our banks, major industries, investments, lifestyle and retirements have been seriously eroded by a combination of secrecy, incompetence, and criminal behavior on an unprecedented scale, I think such people need to get their priorities in order.

In his mantra "Critical thinking always, or your patient's dead", cardiothoracic surgeon Victor P. Satinsky, mentioned in earlier posts as my earliest medical mentor, did not include "but be polite about it" as part of the lesson.

On those pesky EMR curmudegons ... (click to enlarge)

-- SS

Addendum 11/17/10:

At the above Occam link Ms. Wood published my brief comment on this issue, and a thoughtful response. See the comment thread of her EHRevent essay.

-- SS

EHRevent: survey amateurism, bias, or something else?

At my post EHRevent.org: Web Site to Collect EHR Safety Reports, I wrote of my questions about a new organization, EHRevent.com, that seems to supercede or compete with the FDA's MAUDE and Medwatch medical device and medication adverse events reporting and analysis services.

Reviewing the EHRevent report form on this day (archived here, PDF), I note the following multiple choice question on page 7 (emphasis mine):

Notwithstanding the event you are reporting, has the adoption and use of an EHR by your practice added to patient safety, improved care or improved documentation? Select one option.

o Yes, definitely

o Likely

o Not sure

o No impact

A bias and/or survey amateurism is clearly evident in this question. And perhaps something more?

Missing is this option:

o None of the above; EHR adoption did not "add to"; rather, it subtracted, as worsening occurred

A fundamental rule of surveys is that the choices should not artificially limit the survey taker's ability to provide critical or relevant information.

As Ross Koppel, PhD, a sociologist studying healthcare IT at the University of Pennsylvania stated at the Feb. 25, 2010 HHS Certification/Adoption Workgroup Meeting on Health IT Safety (minutes of that meeting are archived at this link, PDF):

... Like everyone else, I want HIT to increase patient safety, care efficiency, treatment quality, savings, and drug ordering guidance. I want HIT to provide coherent structures for test results and other data, and I wanted to provide better visualization of complex clinical data.

Unlike many of my colleagues here who are HIT scholars and advocates, however, because of my training perhaps, I‘ve studied the surveys that have been used to guide and explain the current HIT strategy.

These surveys explored why doctors and hospitals have not embraced HIT‘s benefits. The findings pointed to the cost of HIT, to the physicians‘ resistance. They called them technophobic, hide bound, and perhaps most gruesome, too old. It also talked about overwhelmed hospital IT staff and other user pathologies or user inadequacies.

Now the years of research on HIT suggested that those answers could not be complete. They weren‘t right. The reasons the surveys found this, I‘ve been investigated, was I looked at the questions that were asked of the doctors and hospitals. And the only answer options dealt with the problems of hospitals and doctors. In other words, they found only the questions that they asked about physician difficulties, doctor difficulties. They didn‘t look for any other options, and they didn‘t even give other option answers to talk about the following issues.

So they could have asked, does HIT slow or speed your clinical work? Are EHR data presented in helpful ways, or do they generate unnecessary cognitive burdens because, for example, the data that should be contiguous are in five separate screens, where you‘re scrolling across vast wastelands of rows and columns looking for the needed information. How many information displays are understandable or are disarticulated, confusing, or missing key data? Does HIT distract from your patient care or improve it? And the last one I‘ll ask, although I have about 50 others, how responsive are HIT vendors to acknowledging and repairing defects? How quickly are these defects repaired?

The absence of the relevant questions divert us from understanding the actual HIT needs of clinicians and patients. Now I‘m certain that the people who asked these questions, designed these surveys, were not intentionally deceptive. Well, I‘m certain most were not intentionally deceptive. But the restricted options reflects a series of assumptions or … that says HIT is intrinsically beneficial. Anything that encourages HIT is good for patient safety. Anything that discourages it or retards it is bad by definition.

The creators of the EHRevent survey apparently were not present, or not listening, during Dr. Koppel's presentation.

I have not yet reviewed in depth the other survey questions for similar issues, but this one stood out like a sore thumb.



I cannot imagine an objective, competent scientist committing such an error.

-- SS

EHRevent.org: Web Site to Collect EHR Safety Reports

At "Cart before the horse, again" I observed the irrationality of creating "meaningful use" rules for health IT before usability issues (i.e., poor usability) had been robustly addressed, and the further irrationality of the IOM studying HIT safety after HIT became slated for national rollout in the next few years.

The horse seems to be starting to catch up to the cart (or is it the other way around?):

HDM Breaking News, November 15, 2010

The iHealth Alliance, a coalition of industry stakeholders, has launched an electronic health record safety reporting Web site, called EHRevent.org.

The site is designed to be a national system where providers can report safety issues related to the use of EHRs. Reported events will be confidential but used as the basis to generate other reports that medical societies, malpractice insurers and government agencies can use "to help educate providers on the potential challenges that EHR systems may bring," according to the alliance. The Food and Drug Administration will use data collected on the site to assist in evaluating safety issues that may arise during the forthcoming widespread implementation of EHRs.

Initial supporters of EHRevent.org include the American Medical Association and some state medical societies, American Cancer Society, National Patient Safety Foundation, American Pharmacists Association, CentrEast Regional Extension Center in Texas, and EHR vendor eMDs Inc.The PDR Network, which distributes drug labeling information, FDA-issued product safety alerts and other services such as the Physicians' Desk Reference, will operate the network. Participating insurers, medical societies, EHR vendors and other entities will have links to EHRevent.org on their own Web sites. A similar site to report adverse medication events, RxEvent.org, will launch in 30 to 60 days. [Is the FDA outsourcing MedWatch? - ed.]

Federal agency supporters include the Agency for Healthcare Research and Quality, and the Food and Drug Administration. Malpractice carrier supporters include COPIC Insurance Company and The Doctors Company.

From the homepage at http://www.ehrevent.org/:

The EHR Safety Event Reporting Service is a service of PDR Secure™, a Patient Safety Organization (“PSO”).

I have several concerns with this development:

• The site page on Privacy and Security states that "PDR Secure™, a certified Patient Safety Organization, receives oversight and governance from the iHealth Alliance, a not-for-profit organization whose mission is to protect the interests of patients and providers, as healthcare increasingly moves online." However, who's paying for the EHR Safety Event Reporting Service and the salaries of the people involved?
  

• How is it that these watchdog organizations seem to spring out of nowhere, with no opportunity for public comment or involvement before they "hatch?"

• Did Congress or other elected representatives have a role in this development?
  

• Will competitors also "hatch?"
  

• Why doesn't the FDA take on this role instead of simply "using data collected on the site to assist in evaluating safety issues that may arise during the forthcoming widespread implementation of EHRs?" They have the infrastructure and experience (e.g., the Medwatch and MAUDE reporting facilities).

• Will actual reports, de-identified as the the reporter, be available to the public in a searchable database as they are in, say, FDA's MAUDE database? (See posts on MAUDE here and here.) From the EHRevent site:
  

The Privacy and Security page states that "PDR Secure™ ... will collect, analyze and report back to healthcare providers and others on trends, recent new developments, and other information designed to reduce the risk of harm in the delivery of health care." The FAQ page states "PDR Secure will ... analyze and make available reports on the type of events, frequency of events, and other statistics as well as recommendations and best practices for using EHR systems. "

Will the actual report text itself be unavailable to the public?

• Re: "A similar site to report adverse medication events, RxEvent.org, will launch in 30 to 60 days." Is FDA abandoning its role in collecting drug AE reports?

Further, from the site's own language:

• What, exactly, will be done with the information? They state: "Reporting EHR events can help improve EHRs, support patient safety, reduce professional liability and help liability carriers and others properly educate physicians on safe use of EHRs." How?

• Is the reporting process really protected? Again, from the site itself:

How broad is the legal protection in this PSO?

Patient Safety Work Product - the information you provide in the reporting form, once it becomes part of the PSO environment - is entitled to greater protection from disclosure in legal matters as part of a new statutory and regulatory framework. PDR Secure™, as a certified and listed Patient Safety Organization under Federal law/regulations, has been designed to afford a reporter with the protections under this law. However, it is important to know that PSOs are relatively new entities that have not been tested in the courts of every jurisdiction. Each person or entity who submits information to PDR Secure™ should make their own independent evaluation as to the risks involved in submitting information and what particular information to submit.

Can I submit a report without identifying myself?

You need to provide your identity, which will become part of the PSO database. You can select whether your identity can be shared or is to be kept confidential under the regulations governing the PSO, and PDR Secure’s policies and procedures.

• Will the "contract" one has to agree with, a 16-page document studded with legalese (download here, .doc format), inhibit voluntary EMR-related problem reporting?
  

Even with those questions in mind, I think this may be a net positive development. It would at the very least seem to lay to rest the deterministic notion (and/or marketing message) that this technology is harmless and entirely beneficent.

Will it be an effective way to help reform the health IT industry? Time will tell.

-- SS

Addendum 11/16/10:
See my initial concerns with the EMRevent report form at my followup post here.

Addendum 11/17/10:
Some answers to these questions can be found in a blog post at the site of Occam Practice Management at this link: http://www.occampm.com/blog/general/ehr-event-reporting/, whose author noted this HC Renewal post. My comments are at a post here.