Measuring Interoperability

One of the challenges facing ONC for more than the past decade has been measuring interoperability (yes, ONC has been around for  that long).  One of the responsibilities of the National Coordinator in the original order creating the office back in 2004 and continuing on to this day is to "include measurable outcome goals".

These are some things that one might consider measuring with regard to Interoperability:

• Reductions in costs of care attributable to presence of absence of Interoperability
• Application support for Interoperability Standards and APIs
• Transmissions of Interoperable Information
• Speed of Uptake of Interoperable Solutions

However, not all of these are "Outcomes".  They represent three different kinds of measures, and then something special.  Cost reduction is clearly an outcome. Application support is a capability measurement. Transmissions of information is a process measurement.  Speed of uptake I'll talk more about at the end of the post.

There really is no definitive or easy way to put a $ figure on the ROI of interoperability in healthcare without spending a great deal of time.  It takes a well designed study which can show before and after effects of an interoperability based intervention.  And it isn't clear how much of the $ saving could be attributed to technology, vs. other process changes involved that use that technology.  Yet, among the above, that's in fact the only outcome measure listed.

Application support for standards and APIs is a capability measure.  We can definitively show that applications have more interoperability capabilities than they have.  But we don't have good evidence linking that capability to outcomes.

Transmissions of interoperable information is something where we are actually measuring a process. For example, in ePrescribing, we are measuring the number of prescriptions sent electronically.  We also have some good studies linking that process to improved outcomes, but it's not a direct measure of outcomes.

Finally, the speed of uptake.  This is an interesting measure.  It shows something about capability, in that it demonstrates availability of a particular capability.  But is also demonstrates an outcome, one related to ease of use.  If we look at the ease of uptake and flexibility of HL7 Version 2, HL7 CDA, HL7 Version 3, IHE XD*, and HL7 FHIR, we can get a pretty good understanding of the complexity of the various standards.  

As we embark next week in discussions about fitness for purpose, this is what I would consider to be an "outcome measure" for success in standards selection.  Ease of uptake directly relates to products with interoperable capabilities in products, interoperable processes that can be delivered upon, and real cost reductions in implementation of Health IT solutions.

As we look at "Fitness for Purpose", I think we need to consider "adoptability" as one of the key metrics to consider.  That means that the standard needs to be readily available, easily understood and implemented.  It's a tall order for a standard to meet, and hard to tell how to get there, but I can say this: You'll know it when you see it, and it doesn't happen by accident or intention alone, but more like a bit of both.

   Keith