CCHIT needs to follow test procedures for Certification

This showed up in my e-mail, but blogger marked it as spam, so I'm reposting it here.

Amit Trivedi has left a new comment on your post "CCHIT needs to follow test procedures for Certific...":

Keith – just want to set the facts straight. Our CCHIT Certified® testing requirements and the ONC-ATCB testing requirements are separate and distinct. We make it very clear to everyone that CCHIT uses the NIST test tools in its ONC-authorized certification program. We do *not* use the Laika tool for ONC-ATCB testing. We use the following tool as specified in the NIST test procedures for CCD/C32 validation: http://xreg2.nist.gov/cda-validation/mu.html

Your blog is well-read and we appreciate the work you do in the health IT standards community. We hope you will feel free to contact CCHIT directly to verify misstated claims such as this one.

Now for mea-culpas.  Yes, I should have checked in with CCHIT on this, and for failing to do that I apologize. 

On the flip side, CCHIT shouldn't be setting different requirements for Comprehensive Certification that CONFLICT with ONC-ATCB Certification, which is what appears to have done.  If it were me, I'd pick one tool and stick with it, but they could still use the LAIKA tool -- just update it to integrate with the RIGHT set of rules from NIST.

---

See comments below for updates. This may result from confusion between CCHIT Comprehensive certification (for which CCHIT can use their own procedures), and ONC-ATCB certification. In that case, comprehensive certification under CCHIT would seem to require something different from ONC-ATCB certification, which would certainly be unfortunate and undesirable, but not against the regulations.

---

Due to a technical problem, blogger lost the original content of this post.  I have reconstructed it below:

---

A recent post to the structured documents and CCD lists brings up the issue that the LAIKA tool isn't coming up with the same results as the NIST tools.  The LAIKA tool is wrong, and the NIST tool is right (click on the link above for details).

Under the applicable regulations [emphasis mine]:
§170.423 Principles of proper conduct for ONC-ATCBs.
...
(e) Use test tools and test procedures approved by the National Coordinator for the purposes of assessing Complete EHRs and/or EHR Modules compliance with the certification criteria adopted by the Secretary;

To my knowledge, only the NIST test procedures have been adopted by the secretary.  Those procedures  (pdf) reference the NIST test tool, not the LAIKA one.


Reading further in the Certification rule,

§170.560 Good standing as an ONC-ACB.
(a) Adhering to the Principles of Proper Conduct for ONC-ACBs;e.g., use procedures adopted by ONC...

So, going further, what's your recourse if affected?  A section down describes what ONC can do on recieving evidence of non-complaince.

§170.565 Revocation of authorized certification body status.
...
(b) Type-2 violations. The National Coordinator may revoke an ONC-ACB’s status for failing to timely or adequately correct a Type-2 violation. Type-2 violations comprise noncompliance with §170.560.

(1) Noncompliance notification. If the National Coordinator obtains reliable evidence that an ONC-ACB may no longer be in compliance with §170.560, the National Coordinator will issue a noncompliance notification with reasons for the notification to the ONC-ACB requesting that the ONC-ACB respond to the alleged violation and correct the violation, if applicable.

So, if you can show to ONC that CCHIT requires a testing procedure that is not the one accepted by ONC, you can almost certainly get CCHIT to fix it if ONC agrees that the procedure is in violation.

AHRQ HealthIT Update -- NIST Partners with ONC and AHRQ to Deliver Guidance on EHR Usability

Crossed my desk this afternoon ...

---

NIST Partners with ONC and AHRQ to Deliver Guidance on EHR Usability

Two new publications from the National Institute of Standards and Technology (NIST) intended to help developers of software and computer systems improve the ease of use of electronic health records (EHRs) are now available as part of a federal effort led by HHS’s Office of The National Coordinator for Health Information Technology (ONC). Health care providers are encouraged to adopt and use EHRs that can bring about broad quality improvements and cost savings in the health care system. Efforts to improve the usability of EHRs are widely recognized as key to achieving widespread adoption and meaningful use of these systems. For questions about these reports, please contact Ben Stein (bstein@nist.gov; 301-975-3097). Select to access each report:

1.     NIST Guide to the Processes Approach for Improving the Usability of Electronic Health Records (NISTIR 7741) (PDF file; PDF help)

2.     Customized Common Industry Format Template for Electronic Health Record Usability Testing (NISTIR 7742) (PDF file; PDF help).

In addition, a recent AHRQ report identified gaps in the processes and practices used by EHR vendors to ensure the usability of their products. One key finding from the report highlighted the lack of standard approaches and formats for testing and reporting usability of EHR products across the industry. Select to access the report (PDF file; PDF help).  

My Review of the PCAST Report on HealthIT

See also The Language of HealthIT which goes into more details about what we already have, John Halamka's thoughts and those of Joyce Sensmieir from HIMSS.

Keith

---

The PCAST report on Health IT was released today.  The first question you might have is who is PCAST and how are they different from the HIT Standards Committee, Policy Committee and other FACAs related to healthcare (e.g., NCVHS). From the report:

The President’s Council of Advisors on Science and Technology (PCAST) is an advisory group of the nation’s leading scientists and engineers, appointed by the President to augment the science and technology advice available to him from inside the White House and from cabinet departments and other Federal agencies. PCAST is consulted about and often makes policy recommendations concerning the full range of issues where understandings from the domains of science, technology, and innovation bear potentially on the policy choices before the President.

So, basically, these are advisors to the President, rather than advisors to the executive branch of government (FACAs). They provide an interesting cross-check to other advisory commmittees.

One of the interesting points made by this report is the focus on ROBUST information exchange rather than simple exchange.  This is along the lines that John Moehrke has pointed out earlier this year, dealing with deeper issues and requirements for information access.  Simple exchange is just one part of the problem, and too much focus on it can (and has) detract from resolution of the complete problem.  As PCAST says:

Though the [Meaningful Use] rule expresses an intent to require more robust exchange of health information among providers at later stages of meaningful use, its initial requirements that EHR systems communicate with each other are very modest. This creates a danger that EHR adoption during early stages of meaningful use may exacerbate the problem of incompatible legacy systems. What is needed is a simultaneous focus on the capability for universal data exchange, able to unleash the power of the competitive market, to produce increasingly better and less expensive systems, and to create the "network effect" that spurs further adoption.  (Page 3)

I was amused by the attribution of CDA to ONC rather than HL7, but impressed with PCAST's understanding of its capabilities and limitations (page 4).

Creating the required capabilities is technically feasible, as demonstrated by technology frameworks with demonstrated success in other sectors of the economy. The best way to manage and store data for advanced data-analytical techniques is to break data down into the smallest individual pieces that make sense to exchange or aggregate. These individual pieces are called "tagged data elements," because each unit of data is accompanied by a mandatory "metadata tag" that describes the attributes, provenance, and required security protections of the data. Universal exchange languages for metadata-tagged data, called "extensible markup languages" are widely and successfully used. Indeed, ONC’s clinical document architecture standard (CDA) is such a markup language, and is an important step in the right direction...

 Later they point out on Page 6:

 ... ONC’s clinical document architecture standard (CDA) is an important step in the right direction, but needs more focus on data transmission, on innate privacy features, and on the enabling requirements of a more robust marketplace in new and innovative health IT products.

Later on Page 72 (underlining is mine):

However, the thrust of CDA seems largely that it be an extensible wrapper that can hold a variety of structured reports or documents, each with vocabulary-controlled metadata. While this shares many features with the universal exchange language that we envisage, it lacks many others. In particular, it perpetuates the record-centric notion that data elements should "live" inside documents (albeit metadata tagged). We think that a universal exchange language must facilitate the exchange of metadata tagged elements at a more atomic and disaggregated level, so that their varied assembly into documents or reports can itself be a robust, entrepreneurial marketplace of applications. In a similar vein, we view the semantics of metadata tags as an arena in which new players can participate (by "publishing"), not as one limited to a vocabulary controlled by the government.

Which is really an acknowledgement that Phase 1 of meaningful use doesn't really address data transmission at all (and isn't part of the CDA standard either). Also missing is a way to address one of the biggest issues in exchange which is how to address Privacy Policy at a National level.Wow, someone actually read the standard and understood its intent.  NOTE: There is a GOOD reason to have a record centric notion of clinical data, but that's another blog post (probably tomorrow's).  If they'd gone one step further, they'd have understood that the CDA is based on the HL7 RIM, and the RIM deals with the metadata tagged elements at that more atomic and disaggregated level.  One of the things that IHE has done has shown how CDA R2 and the HL7 V3 Patient Care Messages (also based on the RIM) can work together, one to produce complete documents, and the other to get at the detailed data. 

Chapter 4 of the report goes on to reinvent HL7 Version 3 ... if only they knew.  FYI: They estimate the cost to create the necessary standards to be around $20 to $40 million.  My guess is that is about right based on what IHE and HL7 have spent actually doing it already over the last decade.  Moving forward, maybe we need to spend 3-12  ($M) more them easier to use and implement, but please, don't send us all back to the drawing board AGAIN.

Chapter 5 on Privacy and Security I will leave to John Moehrke to provide detailed comment on.  I will point out that they recognize that perfect is the enemy of the good, but then go on to describe a nearly impossible to implement, perfect security architecure.  When a policy advisor starts talking about technical approaches, they've probably overstepped their bounds, and in this case, their expertise in real-world implementations (same applies to DEAS reinvention of V3).

In summary, it's a good report, and worth reading.  The council has some good ideas, but isn't aware of some of what has already been done.  They should stay away from designing architectures and focus on policy requirements.  Tomorrow's post will address some of their recommendations on a universal language and how ONC and CMS could move forward on that taking advantage of existing standards.

CDC informational webinar on syndromic surveillance guidance Tues 12/14

The International Society for Disease Surveillance (ISDS), with the support of the CDC BioSense Program, is developing guidance for the Meaningful Use objective that relates to syndromic surveillance. In close collaboration with the CDC, ISDS is using a consensus-driven process and engaging key stakeholders to develop the recommendations that will serve as a resource for electronic health record technology vendors, healthcare professionals and hospitals, and public health stakeholders supporting public health syndromic surveillance practice.  The final guidelines will be provided to the Office of the National Coordinator of Health Information Technology (ONC) in January 2011. ONC has indicated that this information will be used in the rulemaking process for Meaningful Use Stages 2 and 3.

Now through 12/17/2010, the ISDS is making the interim guidance available for review and readers are welcome to comment at the following link:www.syndromic.org/projects/meaningful-use

An informational webinar will take place on 12/14/2010 from 1:30-3 PM(EST). The purpose and content of the recommendation will be discussed in detail with participants. This opportunity is intended to assist you in providing constructive comments to ISDS before the 12/17 deadline.

Please SAVE THE DATE and join the webinar here:

https://cdc-informatics.dimdim.com/communications/

Public Health Surveillance Program Office POC

Sam Groseclose 404-498-0127                      

Taha Kass-Hout  404-498-2014

Webinar POC:  Rick Jones 770-841-2890

Rick Jones
Associate Director for Communications
Centers for Disease Control and Prevention
Office of Surveillance, Epidemiology & Laboratory Services
Public Health Informatics & Technology Program Office

PublicHealth Surveilance Recommendations for MeaningfulUse Published for Comment

At the beginning of December ISDS published its recommendations for Disease Surveillance for Meaningful Use.  I've commented on this work in this blog before.  It's now out for a second round of public comment. 

ISDS, while reporting that they have convened a workgroup of Meaningful Use stakeholders, definately did not include the stakeholders who have to implement the requirements described in their deliberations on the creation of this specification. This is pretty important in standards development.  You need to have the producers as well as the consumers present in the discussion, or you wind up with an unbalanced approach.

What has changed for the better though is that they report that the implementation guide work will be done in HL7, and they've simply reported the required data elements, rather than how they are to be built into an HL7 messaqge. I'm very happy to see this change.  It means that the standards development work will be done in an experiencd SDO which has balanced participation in the the development process.


I have a couple of comments on the document that they published:

1.  On reporting, they recommend reporting from Hospitals and Urgent Care centers every 24 hours, and indicate that there is a belief that ambulatory reporting would be beneficial, but there is a lack of experience in this space.  I would have to agree on the experience space.  I have a hypothesis that ambulatory reporting would not only be beneficial, but also, significantly better than hospital and urgent care reporting based one what I've learned from others.  I suspect that disease activitity in ambulatory care settings spikes ahead of urgent care centers and hospitals, because that is where patients with health coverage will go when they first start having symptoms of disease, whereas patients without will seek urgent or emergency care only after symptoms become severe.  This could amount to a several day lead on existing surveillance efforts.
So, it might be cheap and easy to get large volumes of data from hospitals, but the signal will be smaller and more delayed.  It might be more difficult or expensive to get data from ambulatory providers, but the signal will likely be stronger and sooner (but also different due to change in symptom severity).

From a policy perspective:  I would recommend research and funding into studies on the effectiveness of disease surveillance from the ambulatory perspective.  I would NOT postpone deployment of basic surveillance into urgent and hospital care settings, but would postpone advanced deployment into those settings until it has been determined whether that setting or the ambulatory setting would be the better place to spend money.

2.  On data elements:
Facility name and address data is noted as being RE, required to be sent when known.  I cannot imagine a hospital or urgent care center that does not have that data available.  It should be required.

Unique Patient Identifier and Medical Record Number are two different kinds of patient identifier.  No need to separate these.

Age is required but Gender is RE.  I don't get it.  If you can always figure out the age, then gender should be as easy.  Many of the reasons why you wouldn't be able to ascertain age would also impact gender (patient not concious).  I would assume that these two data elements would have the same level of requirement, and that it should likely be RE (because you may not be able to determine age).

For age, units should also be allowed to be decades, because after a certain age, that's all that HIPAA allows you to indicate.

For patient class, I would limit the value set to Emergency, Outpatient or Inpatient.  Other distinctions, such as obstetric, recurring, or pre-admit are variations on Outpatient as far as I can tell, and not of great utility in surveillance.  While these are values used in the HL7 standard, they don't seem as applicable to the surveillance task.

On Chief Complaint/Reason for Visit, I understand the desire for free text content, but would separate the requirements for free text and coded data and make both RE (send if you have it).  Having both fields for surveillance may be valuable, as you can make associatations between coded values and free text content based on entered data that could be useful later.

I'm curious while tempurature and pulse oximetry are RE, but other vital signs are not included.  I'm not a public health specialist, I can imagine that blood pressure might not correspond to a public health problem, but I could imagine that pulse and respiration might.  What's the significance of the signal conveyed by these data elements, and what is the significance criteria by which a data elements are chosen to be included in this data set.  Hmm, there's some interesting math to do on this set... it strikes me that there's at least a good research paper in evaluating the predictive value of various data elements in the recommended set to public health surveillance.

Harmonious Health IT

I'd like to welcome David Tao, MS, D.Sc., and author of the newly created Harmonious Health IT blog to the club of Health IT Bloggers.  David has long been a contributor of very good feedback on this blog.  I've known David for almost as long as I've been involved in HL7.  He was my first choice in HITSP for someone to handle the difficult taks of representing the Care Management and Health Records committee to the HITSP Internal Review Board (later called the Internal Review Team).

David is usually quiet and very soft-spoken, but he has penetrating insight to interoperability problems in healthcare IT.  I'll be listening to what he has to say, and so should you.

Thanks for joining us David, welcome to my blog roll, and please note: I sing Bass.

   Keith

HL7 Wants Your Input

At the September Working Group Meeting, Bob Dolin, the current HL7 Board Chair reviewed briefly the HL7 Strategic Initiatives.  That document has now been published for comment by the public at the HL7 Ballot Site.

Their is no fee to provide feedback on this document through the HL7 ballot process (or any other for comment only document for tha matter) .  You are simply asked to register.

The document appears below.  Your input to HL7 will be used to update the 2012 Strategic Initiatives.

I have several comments of my own on this document.

1. Strategic initiatives should be few, not many, so as to provide a point of focus.  Six strategic initiatives feels like too many. Three of the strategic initiatives:
   
   
   • Streamline the HL7 standards development process
   • Facilitate HL7 standards adoption and implementation
   • Define an overarching and internally consistent interoperability framework
   
   Look like they could be combined into one initiative whose goal is to Build better standards that are more responsive to customer needs.  They have overlapping (and sometimes conflicting) requirements that mean they should be addressed together.  This, FYI, was essentially my platform when I ran for the HL7 Board.
2. An initiative is something that you do to reach a goal, an action to take, not an outcome.  Initiative #1:  Lead the development of global technical and functional health informatics standards is more descriptive of an outcome, rather than an initiative.  As a strategic initiative, it also fails the test of obviousness.  This one is a no-brainer.  Of course HL7 wants to be a leader in this role, that is not really a strategic initiative, but rather it's purpose to begin with..  We are after all the global authority on standards for interoperability for heath information technology according to the HL7 home page.  I think the key issue here is that HL7 wants to be more appreciated as a recognized leader in the development of standards, and the initiative should be more targeted towards that objective.
   
   The success critieria for this initiative are a mixed bag.  I think the first criteria for success: Industry responsiveness has been demonstrated through the timely development of key standards belongs with the group I identified above. 
   
   In that same vein: Working Group Meetings have been effective is a bit weak without discussing how WGM effectiveness is being measured.  The adjectives "healthy" and "appropriate" are not measureable as described without some discussion of this. There is an initiative to ensure that working groups are healthy as measured by certain criteria, but without explaining what "healthy" means, this is a bit nebulous.  One of my measurements of a healthy working group is the number of people standing for cochair elections.  Many working groups (Structured Documents included) have only one person standing for election to a leadership position.  That doesn't seem to be indicitative of a healthy working group. 
   
   I'm also not certain how ensuring WGMs are effective ties into the lead development efforts goal.  I'd be more satisfied with success critieria that talked about increasing membership, participation in working groups and working group meetings (which is a clear measure of recognition of the importance of HL7).
3. The last strategic initiative: Align HL7's business and revenue models to be responsive to national bodies while supporting global standards development is interesting.  It speaks to HL7's need to remain an active body with suffucient funding, and it addresses a complaint that its current model of governance needs to be addressed to ensure that more international participation is encouraged.  The IP issue is tricky.  Under the current model, International Affiliates get to use HL7 IP as part of their association with HL7, and return some revenue from membership and sales of HL7 IP to HL7.  However, the membership and sales fees for International Affiliates are not uniform.  The challenge is that HL7 may need to use that IP in a new revenue model in a way that would trade some of the access provision to HL7 IP for more influence to the affiliates.  I'm still not sure how to address this one.

So, as I have it, the quick summary:  Reduce the document from six initiatives to three.  Make the first initiative stronger, less like a mission statement.  Combine adoption and customer responsiveness initiatives.  Keep working on the business model.

I think, as a point of focus, these are the areas that we (HL7) still need to be working on.  When we have demonstrated more success in these areas, it will be time to address new areas.

I will be happy to incorporate any comments I receive here into my input to HL7, but be advised that it will also be filtered by my understanding and goals.  If you want to make your own unfiltered comments to HL7, the best way to do that is register to comment on this document through the HL7 ballot site.