CCHIT needs to follow test procedures for Certification

This showed up in my e-mail, but blogger marked it as spam, so I'm reposting it here.

Amit Trivedi has left a new comment on your post "CCHIT needs to follow test procedures for Certific...":

Keith – just want to set the facts straight. Our CCHIT Certified® testing requirements and the ONC-ATCB testing requirements are separate and distinct. We make it very clear to everyone that CCHIT uses the NIST test tools in its ONC-authorized certification program. We do *not* use the Laika tool for ONC-ATCB testing. We use the following tool as specified in the NIST test procedures for CCD/C32 validation: http://xreg2.nist.gov/cda-validation/mu.html

Your blog is well-read and we appreciate the work you do in the health IT standards community. We hope you will feel free to contact CCHIT directly to verify misstated claims such as this one.

Now for mea-culpas.  Yes, I should have checked in with CCHIT on this, and for failing to do that I apologize. 

On the flip side, CCHIT shouldn't be setting different requirements for Comprehensive Certification that CONFLICT with ONC-ATCB Certification, which is what appears to have done.  If it were me, I'd pick one tool and stick with it, but they could still use the LAIKA tool -- just update it to integrate with the RIGHT set of rules from NIST.

---

See comments below for updates. This may result from confusion between CCHIT Comprehensive certification (for which CCHIT can use their own procedures), and ONC-ATCB certification. In that case, comprehensive certification under CCHIT would seem to require something different from ONC-ATCB certification, which would certainly be unfortunate and undesirable, but not against the regulations.

---

Due to a technical problem, blogger lost the original content of this post.  I have reconstructed it below:

---

A recent post to the structured documents and CCD lists brings up the issue that the LAIKA tool isn't coming up with the same results as the NIST tools.  The LAIKA tool is wrong, and the NIST tool is right (click on the link above for details).

Under the applicable regulations [emphasis mine]:
§170.423 Principles of proper conduct for ONC-ATCBs.
...
(e) Use test tools and test procedures approved by the National Coordinator for the purposes of assessing Complete EHRs and/or EHR Modules compliance with the certification criteria adopted by the Secretary;

To my knowledge, only the NIST test procedures have been adopted by the secretary.  Those procedures  (pdf) reference the NIST test tool, not the LAIKA one.


Reading further in the Certification rule,

§170.560 Good standing as an ONC-ACB.
(a) Adhering to the Principles of Proper Conduct for ONC-ACBs;e.g., use procedures adopted by ONC...

So, going further, what's your recourse if affected?  A section down describes what ONC can do on recieving evidence of non-complaince.

§170.565 Revocation of authorized certification body status.
...
(b) Type-2 violations. The National Coordinator may revoke an ONC-ACB’s status for failing to timely or adequately correct a Type-2 violation. Type-2 violations comprise noncompliance with §170.560.

(1) Noncompliance notification. If the National Coordinator obtains reliable evidence that an ONC-ACB may no longer be in compliance with §170.560, the National Coordinator will issue a noncompliance notification with reasons for the notification to the ONC-ACB requesting that the ONC-ACB respond to the alleged violation and correct the violation, if applicable.

So, if you can show to ONC that CCHIT requires a testing procedure that is not the one accepted by ONC, you can almost certainly get CCHIT to fix it if ONC agrees that the procedure is in violation.

About that preliminary CCHIT Certification

Under the changes to the rules finalized for Meaningful Use recently, were some clarifications about certification and testing processes.  Did you previously certify your EHR product through CCHIT?  Even if it passed, you will probably have retest the features again.  Why?  Because under the final rule, only certification testing methods approved by the National Coordinator for Healthcare IT will be acceptable for meaningful use certification.

Right now, so far as I know, only NIST is developing testing scripts and methods that ONC is expected to approve.  CCHIT will either have to recertify using the NIST scripts and methods, or get the testing methods it used in its preliminary certification approved by the National Coordinator.  I would NOT place any bets on that being terribly successful, given the desire for a level playing field for ATCBs (but I've been wrong before).

I'm planning on having to retest everything, and so should you.  As to whether that was a useful expenditure, I'm not sure I care to comment.

As always: The opinions represented in this blog are my own, and not that of my employer or the respective standards organizations that I work with.