Picking COVID19 Value Sets for SANER

It's always fun when it comes time to choose standards because there are so many to choose from.  Here's a list off COVID-19 value sets from VSAC:

 

Fifty three different value sets, with overlapping codes and purposes. Which ones would you choose from?  How would you decide?  The SANER project needs to make some decisions to illustrate how to create a measure.

I have the same problem in software development when selecting 3rd party components.  Here's the criteria I've used for the past 20 years:

1. How well does the component meet your need?
2. What's the quality of the component?
3. How well is the component maintained?
4. How likely is it that they will continue to maintain it?
5. How transparent are the maintainers development processes to you?
6. How well used is the component by the rest of the industry?
7. How good are the licensing terms?

These same criteria can be applied to value set selection.

For the VSAC Value Sets, there's basically 6 maintainers (in alphabetical order):

• CareEvolution
• Clinical Architecture
• Council for State and Territorial Epidemiologists (CSTE)
• Logica Health (formerly Health Services Platform Consortium or HSPC).
• MITRE Corporation acting on behalf of the COVID-19 Healthcare Coalition
• ONC

Let's do the evaluation:

1. How well does the component meet our need?  
   About equally well.
   
   
2. What's the quality of the component?
   Mostly the same.
   
   
3. How well is the component maintained?
   The first two maintainers are private firms contributing value sets to VSAC for public use.  They very likely have a good maintenance process.
   
   The last two are government contractors or agencies who aren't NORMALLY in the Value Set maintenance business, and will likely turn these over to others for longer term maintenance.  Thee MITRE work is being done in collaboration with the COVID-19 Healthcare Coalition and has a high quality process. 
   
   The ONC work relies on early work by others, and so, while authoritative, is probably not going to be something that we want to use (not a ding on ONC, just the reality, they did what needed to be done to get the ball rolling, then stepped aside once others took it on).
   
   The middle two are organizations focused on the development of Value Sets, and CSTE is very focused on Epidemiology. They have high quality maintenance processes.
   
   
4. How likely is it that they will continue to maintain it?
   For the proprietary solutions, I expect eventually to see them make way for an official maintainer of a value set for the same purpose.  The same is true for ONC and MITRE.  The COVID-19 Healthcare Coalition is formed for a very specific purpose, and hopefully will be short-lived (e.g., two years) as organizations go.  I expect that Logica and CSTE will have an ongoing and long-term commitment to their works.
   
   
5. How transparent are the maintainers development processes to you?
   Mostly transparent across the board, but ... I don't have an easy way to engage in the processes by the proprietary vendors.  Logica has a membership model and function that doesn't add value for my needs, though others find it useful.  MITRE's process is very transparent, and ONC's not so much.
   
   
6. How well used is the component?
   I cannot really answer this question today, but I can make some predictions for the future:
   CDC is very likely to rely on CSTE as they have done in the past.  The Logica work is going to see uptake by Logica members.  The MITRE work has seen uptake by members of the coalition it is working with.  ONC's work was early, and incorporated into other works, so also used, but more like merged with as components go.
   
   
7. How good are the licensing terms:
   For users of a value set, all of these are generally freely available for use.  For IG publishers who want to "make a copy of them", the terms are (or in the future could be) somewhat limiting from the proprietary vendors and Logica.  I'd love to simply be able to reference them from VSAC, but frankly the FHIR publication interface is miserable for developers, and the access to VSAC publication pages also has other challenges of it's own.  I've inquired about ways to address this, but that's likely going to have to wait on some real funding to NLM.

In short, my priority list goes like this:

1. If available from CSTE, use it.
2. If available from the COVID-19 Healthcare Coalition (MITRE), use it.
3. Don't bother with ONC, others have them covered better.
4. Look too Logica to fill gaps.
5. Skip the proprietary value sets.

Your mileage may vary.