Monday, October 10, 2011

Is Usability of The #EHR the most important factor hindering adoption

As I mentioned last Friday, I'm reviewing the NIST draft on the Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records (NISTIR 7804).  I just finished my first read of the 108 page document (49 pages of text with about 59 pages of appendices).  I have a number of observations on the report, but I'll start with my first in this post, and take up others in subsequent ones.

The document provides about 3 pages of overview for the need for usability testing.  It starts off:

Experts have identified shortcomings in the usability of current EHR systems as a key barrier to adoption and meaningful use of these systems.  
To support this statement, the report quotes the HIMSS 2009: Defining and Testing EMR Usability: Principles and Proposed Methods of EMR Usability Evaluation and Rating report (written by EHR and clinical experts), and the PCAST report (written by IT experts but not EHR experts).

The HIMSS report states:

We submit that usability is one of the major factors—possibly the most important factor—hindering widespread adoption of EMRs.  
But neither the HIMSS report on usability nor the PCAST report were studies designed to test this conclusion in any way.  There are a number of reports and one study cited in the HIMSS report that apparently support this conclusion. As a test, I started digging into them:
  1. Smelcer JB, Miller‐Jacobs H, Kantrovich L. Usability of electronic medical records. J Usability Studies. 2009;4:70‐8
    The authors cite anecdotal data based on experience with EHR implementations in Indian Health Services and Mayo.  Anecdotes are good sources to come up with hypotheses, but shouldn't be considered as studies that test them.
  2. Zheng K, Padman R, Johnson MP, Diamond HS. An interface‐driven analysis of user interactions with an electronic health records system. JAMIA. 2009;16:228‐237.
    This is a study demonstrating a method to evaluate and improve a home-grown EHR system.  It does focus on usability and a single EHR, but doesn't supply anything more than another anecdote, along with an interesting computational technique to apply to human factors engineering. 
  3. Rosenbloom ST, Miller RA, Johnson KB, Elkin PL, Brown SH. A model for evaluating interface terminologies. JAMIA, 2008;15:65‐76.
    This is a study of clinical interface terminologies, like SNOMED CT, UMLS or Medcin, and is not really applicable to EHR projects, but rather the terminologies they might use.
  4. Kushniruk A, Borycki E, Anderson JG, Anderson MM. (2008). Combining two forms of simulation to predict the potential impact of interface design on technology‐induced error in healthcare. Proceedings of the 2008 Spring simulation multiconference (pp. 497‐504). San Diego, CA: The Society for Computer Simulation, International
    I'm unable to access the full text, but the abstract describes this as a study using simulation of user behavior and mathematical simulation of user error. 
  5. Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. JAMIA. 2005;12:505‐516.
    This study seems applicable.  It is a meta-analysis of several other time studies on EHRs and CPOE.  But is about use of time, rather than usability.  The two are clearly related, but I would hesitate to call them equivalent.
Of the five reports cited, only the last one, written in 2004, on time efficiency, seems to come even close to being able to support the assertion that "Usability is the most important factor hindering widespread adoption of EHRs".  Interestingly enough, the first report above states that 98% of physicians surveyed in the Netherlands and 89% of those in the UK use EHR systems.  Another theory that could be tested based on hypothesis consistent with that data is that EHR systems outside the US are more usable than those inside the US (Or perhaps it may be that the healthcare systems in those countries are more user-friendly).

After my review of the cited reports, I'm not finding any smoking guns that confirm the key assertion that .  So, there is a gaping hole in the evidence for usability as the most important factor preventing EHR adoption.  I'll note that NIST was never asked to question that assumption.  They were asked to come up with a process to improve usability.  To operate at the same level as the NIST report authors, one must must simply accept that Usability is a deterrent as being true.

I've been around IT long enough though to believe that usability is indeed an issue.  Before I target usability as "the problem", I might do a root cause analysis.  And the root cause that I suspect I would find is one in which the software being developed does not meet the user's needs and/or requirements.  Which leads me to a truism that I could apply anywhere:
I submit that a failure to be connected to a user's needs is one of the major factors—possibly the most important factor—causing any IT implementation project to fail.  
I can pretty much agree with that, so I think I'll simply let the assertion on usability pass.  Poor usability reflects a disconnectedness from the user needs, and that is certainly enough to cause an EHR implementation to fail.


2 comments:

  1. Anecdotally, the overwhelming complaint I hear from physicians on EHRs are concerns on usability. No real data to back up the claim, but I think it is certainly a huge factor...

    I like the Motorcycle Guy Theorem :-) Usability = Connectedness to user needs

    ReplyDelete
  2. See our whitepaper "9 Usability Mistakes Your Team Is Probably Making (And How To Fix Them)" http://tinyurl.com/7n7rksg

    I've long argued usability was a much greater barrier to widespread adoption than most Health IT organizations were thinking. Now that we're making some degree of progress on interoperability, HIEs, etc. it's nice to see the industry putting more attention on this important issue.

    Our understanding via the HIMSS usability task force is that this RFC they've put out will be the basis for upcoming meaningful use stage 2 criteria around usability.

    ReplyDelete