Evaluating Open Source Software for HealthIT

This post is about learning how to fish.  I don't plan on providing dinner ;-)

There are a number of different open source software (OSS) products available that support IHE and HL7 profiles.  Some of that is used by widely known hospitals, integrated delivery networks, or health IT vendors.

How can you tell which OSS components are worth looking into?

1. Who is working on it?
This is a big tell.  There are several IT vendors which strongly support OSS initiatives and basically staff open source projects with small teams.  The affiliation of the OSS project committers will often tell you what organizations are backing the project.  If you can also find evidence that they use the OSS in their own offerings, then you have an even better signal.

2. Recency of Updates?
When was the project last updated?  How frequently do they ship new releases?  A project that ships new releases 2-3 times a year for the past six years is much more viable than one who has only two releases in the same time period.

3. How big is the team?
Are you dealing with OSS developed by one person, or is this a team effort?  Don't get me wrong.  Really good OSS can be developed in a one person effort, but not everyone is a James Clark (a well known contributor to OSS in the SGML world).

4.  What does the defect tracking system tell you?
If there isn't a defect tracking system, walk away.  If defects have been reported but nothing has ever been done about them, walk away.  If there are a bunch of defects reported, that's almost surely a good sign.  You can find out at least two things from defect reports: Who uses the OSS product, and how responsive the team is about the defects.  Fixes may not happen overnight, but if defects are responded to quickly, that is a good sign.  Response need not be a problem resolution.  Some problems can take a long time to fix.  But at least the initial assignment and analysis should be fairly quick.

6.  How many other OSS projects rely upon it?
In my own list of Open Source XDS implementations, the IHE Profiles project under Open Health Tools really stands out because there are about a half dozen other OSS projects I've run across using one or more of its components.  Similarly on the CDA side, there are four different projects that I can think of that are making use of MDHT.

7.  Finally, has it done any public testing or certification?
Testing and certification are fairly extensive undertakings.  Few OSS projects are well enough resourced enough to undertake such efforts.  Those that do are worth checking out.  For example, OHT attended an IHE Connectathon in 2010 and tested a dozen profiles.

With a little bit of digging, you can readily identify the best open source projects to use, and figure out which ones are keepers, and which should be thrown back to grow some more.