I'm in Portland Oregon for most of this week for a workout with one of our development teams. Joining me are my boss and one of our other Standards Geeks. I've long known the importance of face to face contact in developing relationships, and the benefits of my being out here for just one day are already apparent. For one, I've learned that my boss is a better pool player than I am, but not by much ;-)
Our discussions today were far ranging, but one of them centered around validating CDA constructs. There's a great tool that's been developed by NIST called the CDA Guideline Validator. This tool is based on collaborative work from NIST, Alschuler Associates, LLC, Integrating the Healthcare Enterprise (IHE) and the CCHIT Health IT Collaboration Effort "LAIKA" project. While it is a great tool, there are still some process issues that need to be worked out for it to be of greater service to the healthcare IT industry. Don't get me wrong, I love these tools, and point people to them several times a month, but I'd like to see a little bit more.
One of the issues that I run into is that we have to revalidate this tool every time it gets updated to use as part of our testing processes. Here are a few suggestions I'd like to make that I think will improve the use of this tool as part of certification testing and vendor implementation.
1. I'd like to see a design document that describes the overall design of the validation tools. The schematron source code for the validator is great, and frankly, I know a good bit about how that was built, but others need access to that information as well. This need not be a long document, it could be as short as 3 - 5 pages.
2. I'd like to see some explanations about how to read the output documented somewhere.
3. I'd really like to see a validation plan that shows how the rules implemented by the tool are tested, and a validation suite that tests the rules (both positively and negatively).
4. Having Andrew's and Mary's e-mail contact information available is terrific, but a little hard for people to find when they need to report problems. Also, not having an active bug list that people can view and track makes keeping up with the issues a bit of a struggle. I'd like to see a real bug tracking system installed and accessible from the main page.
5. Coordinating feedback from bugs to the organizations responsible for interpretation of the various templates is also difficult (I'm involved in three of them, and I hear from Andrew regularly with all three hats on).
6. I'd love to see a way to directly link each of the errors reported to the appropriate place in the document from which rule is derived. For HITSP and HL7 this is fairly straightforward (it's just a link to the appropriate constrain ID in the document), but for IHE is a little bit more difficult. The IHE profiles need to be a little bit more formal about their constraints in the profiles, now that it (we) have removed the Schematrons themselves from the technical framework (note to self, add as a discussion item at the PCC face to face in two weeks).
7. We need a little bit more formal governance model for how to deal with interpretation of the standards and implementation guides that this tool is supporting. Somewhere along the way I'd like to see a way to verify that the various tests do in fact meet the requirements specified by each of the various guides, and a process for resolving issues that require input from the appropriate authorities.
8. I think this tool could be taken further, and I'd like to see it run as an open source project(*) that we could all participate in. I think structuring it that way would provide more capacity for improvement.
We've already got a terrific bunch of players that has developed this tool, let's put them all on the same team.
* LAIKA is already an open source project, but only supports C32. I'd like to see the entire set of Schematron validation tools be part of a separate open source project. Oh, and it'd be really nice if they supported the IHE SVS profile for value set validation (ITI changed the profile during public comment last year to support an HTTP GET retrieval of the value set in XML that is suited for that purpose for that very reason). And furthermore..., oh just start the thing and I'll chime in.