Essential Elements of Information

Fishbone Diagram

I spend a lot time learning new stuff, and I like to share.  Most recently, I've spent a lot of time learning about Essential Elements of Information (EEIs), or as I like to call them, measures of situation awareness.

EEIs, or measures of situation awareness work the same way as quality control measures work on a process.  You follow the process, and measure those items that are critical to quality.  Finding the things that are critical to quality means looking at the various possible failure modes, and root causes behind those failures.

Let's follow the pathway, shall we, for COVID-19:

We look at the disease process in a single patient, and I'll start with a complaint, rather than earlier.  Patient complains of X (e.g., fever, dry cough, inability to smell/taste, et cetera).  From there, they are seen by a provider who collects subjective data (symptoms), objective data (findings), performs diagnostics, and makes recommendations for treatment (e.g., quarantining, rest, medication) or higher levels of care (admission to a hospital or ICU), more treatment (e.g., intubation), changing treatment (e.g., extubation), changing levels of care (discharge), and follow up (rehabilitation), monitoring long term changes in health (e.g., after effects, chronic conditions).

That's not the only pathway, there are others of interest.  There may be preventative medications or treatments (e.g., immunization).

In each of these cases, there are potential reasons why the course of action cannot be execute (a failure).  Root cause analysis can trace this back (ICU beds not available, medication not available, diagnostic testing not available or delayed).  

As new quality issues arise, each one gets its own root-cause-analysis, and new measures can be developed to identify onset, risk of those causes.

We (software engineers*, in fact engineers in general) do this all the time in critical software systems, and often, it's just a thought experiment, we don't need to see the event to predict that it might occur, and prepare for its eventuality.

Almost all of what has happened with COVID-19 with respect to situation awareness has either been readily predicable, OR has had a very early signal (a quality issue) that needed further analysis.  If one Canary dies in the coal mine, you don't wait for the second to start figuring out what to do.  The same should be true as we see quality issues arise during pandemic.

Let's talk about some of the reasons behind some of the Use Cases SANER proposed for measures, and why:

SANER started in the last week of March, by April 4 we had already understood the need for these measures:

1. PPE ... before there was a measure for PPE, there was a CDC spreadsheet to determine utilization of PPE, there were media report about mask availability, people couldn't buy disinfectants.
2. What happens when providers get sick? Or provider demand exceeds local supply?
3. Hearing about limited testing supplies in NYC (sample kits, not tests).
4. Stratification by age and gender: Many state dashboards were already showing this.
   
   and by April 10, 
5. Ethnic populations are getting harder hit we hear from a provider in Boston.  Social determinants need to be tracked.
   
   in June we added one more that we knew about but hadn't written down yet
6. Non-acute settings (e.g., Rehabilitation and long term care) need attention.  We actually knew this back in late February.
   
   In the Future we can already tell that:
7. There will be more diagnostic tests to account for.
8. As we learn about treatments (e.g., Medications), we'll need measures on use and supplies
9. As immunizations become available, again we'll need measures on use and supplies, but also measures on ambulatory provider capacity to deliver same.

Over time we saw various responses to shortages that meant different things also needed to be tracked:

1. Do you have a process to reuse usually disposable materials (e.g., masks)?
2. What is the rate of growth in cases/consumption/other quantifiable thing presently being experienced by your institution. Different stages of the pandemic have different growth characteristics, e.g., exponential, linear, steady state at different times and regions.

* And if you aren't doing this in your software on a routine basis, you are programming, not engineering.

Picking COVID19 Value Sets for SANER

It's always fun when it comes time to choose standards because there are so many to choose from.  Here's a list off COVID-19 value sets from VSAC:

 

Fifty three different value sets, with overlapping codes and purposes. Which ones would you choose from?  How would you decide?  The SANER project needs to make some decisions to illustrate how to create a measure.

I have the same problem in software development when selecting 3rd party components.  Here's the criteria I've used for the past 20 years:

1. How well does the component meet your need?
2. What's the quality of the component?
3. How well is the component maintained?
4. How likely is it that they will continue to maintain it?
5. How transparent are the maintainers development processes to you?
6. How well used is the component by the rest of the industry?
7. How good are the licensing terms?

These same criteria can be applied to value set selection.

For the VSAC Value Sets, there's basically 6 maintainers (in alphabetical order):

• CareEvolution
• Clinical Architecture
• Council for State and Territorial Epidemiologists (CSTE)
• Logica Health (formerly Health Services Platform Consortium or HSPC).
• MITRE Corporation acting on behalf of the COVID-19 Healthcare Coalition
• ONC

Let's do the evaluation:

1. How well does the component meet our need?  
   About equally well.
   
   
2. What's the quality of the component?
   Mostly the same.
   
   
3. How well is the component maintained?
   The first two maintainers are private firms contributing value sets to VSAC for public use.  They very likely have a good maintenance process.
   
   The last two are government contractors or agencies who aren't NORMALLY in the Value Set maintenance business, and will likely turn these over to others for longer term maintenance.  Thee MITRE work is being done in collaboration with the COVID-19 Healthcare Coalition and has a high quality process. 
   
   The ONC work relies on early work by others, and so, while authoritative, is probably not going to be something that we want to use (not a ding on ONC, just the reality, they did what needed to be done to get the ball rolling, then stepped aside once others took it on).
   
   The middle two are organizations focused on the development of Value Sets, and CSTE is very focused on Epidemiology. They have high quality maintenance processes.
   
   
4. How likely is it that they will continue to maintain it?
   For the proprietary solutions, I expect eventually to see them make way for an official maintainer of a value set for the same purpose.  The same is true for ONC and MITRE.  The COVID-19 Healthcare Coalition is formed for a very specific purpose, and hopefully will be short-lived (e.g., two years) as organizations go.  I expect that Logica and CSTE will have an ongoing and long-term commitment to their works.
   
   
5. How transparent are the maintainers development processes to you?
   Mostly transparent across the board, but ... I don't have an easy way to engage in the processes by the proprietary vendors.  Logica has a membership model and function that doesn't add value for my needs, though others find it useful.  MITRE's process is very transparent, and ONC's not so much.
   
   
6. How well used is the component?
   I cannot really answer this question today, but I can make some predictions for the future:
   CDC is very likely to rely on CSTE as they have done in the past.  The Logica work is going to see uptake by Logica members.  The MITRE work has seen uptake by members of the coalition it is working with.  ONC's work was early, and incorporated into other works, so also used, but more like merged with as components go.
   
   
7. How good are the licensing terms:
   For users of a value set, all of these are generally freely available for use.  For IG publishers who want to "make a copy of them", the terms are (or in the future could be) somewhat limiting from the proprietary vendors and Logica.  I'd love to simply be able to reference them from VSAC, but frankly the FHIR publication interface is miserable for developers, and the access to VSAC publication pages also has other challenges of it's own.  I've inquired about ways to address this, but that's likely going to have to wait on some real funding to NLM.

In short, my priority list goes like this:

1. If available from CSTE, use it.
2. If available from the COVID-19 Healthcare Coalition (MITRE), use it.
3. Don't bother with ONC, others have them covered better.
4. Look too Logica to fill gaps.
5. Skip the proprietary value sets.

Your mileage may vary.