The document provides about 3 pages of overview for the need for usability testing. It starts off:
Experts have identified shortcomings in the usability of current EHR systems as a key barrier to adoption and meaningful use of these systems.To support this statement, the report quotes the HIMSS 2009: Defining and Testing EMR Usability: Principles and Proposed Methods of EMR Usability Evaluation and Rating report (written by EHR and clinical experts), and the PCAST report (written by IT experts but not EHR experts).
The HIMSS report states:
We submit that usability is one of the major factors—possibly the most important factor—hindering widespread adoption of EMRs.But neither the HIMSS report on usability nor the PCAST report were studies designed to test this conclusion in any way. There are a number of reports and one study cited in the HIMSS report that apparently support this conclusion. As a test, I started digging into them:
- Smelcer JB, Miller‐Jacobs H, Kantrovich L. Usability of electronic medical records. J Usability Studies. 2009;4:70‐8
The authors cite anecdotal data based on experience with EHR implementations in Indian Health Services and Mayo. Anecdotes are good sources to come up with hypotheses, but shouldn't be considered as studies that test them.
- Zheng K, Padman R, Johnson MP, Diamond HS. An interface‐driven analysis of user interactions with an electronic health records system. JAMIA. 2009;16:228‐237.
This is a study demonstrating a method to evaluate and improve a home-grown EHR system. It does focus on usability and a single EHR, but doesn't supply anything more than another anecdote, along with an interesting computational technique to apply to human factors engineering.
- Rosenbloom ST, Miller RA, Johnson KB, Elkin PL, Brown SH. A model for evaluating interface terminologies. JAMIA, 2008;15:65‐76.
This is a study of clinical interface terminologies, like SNOMED CT, UMLS or Medcin, and is not really applicable to EHR projects, but rather the terminologies they might use.
- Kushniruk A, Borycki E, Anderson JG, Anderson MM. (2008). Combining two forms of simulation to predict the potential impact of interface design on technology‐induced error in healthcare. Proceedings of the 2008 Spring simulation multiconference (pp. 497‐504). San Diego, CA: The Society for Computer Simulation, International
I'm unable to access the full text, but the abstract describes this as a study using simulation of user behavior and mathematical simulation of user error.
- Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. JAMIA. 2005;12:505‐516.
This study seems applicable. It is a meta-analysis of several other time studies on EHRs and CPOE. But is about use of time, rather than usability. The two are clearly related, but I would hesitate to call them equivalent.
After my review of the cited reports, I'm not finding any smoking guns that confirm the key assertion that . So, there is a gaping hole in the evidence for usability as the most important factor preventing EHR adoption. I'll note that NIST was never asked to question that assumption. They were asked to come up with a process to improve usability. To operate at the same level as the NIST report authors, one must must simply accept that Usability is a deterrent as being true.
I submit that a failure to be connected to a user's needs is one of the major factors—possibly the most important factor—causing any IT implementation project to fail.I can pretty much agree with that, so I think I'll simply let the assertion on usability pass. Poor usability reflects a disconnectedness from the user needs, and that is certainly enough to cause an EHR implementation to fail.