Tuesday, January 11, 2011

Standards Conformance Terminology across IHE, HITSP and HL7

One of the discussions the HL7 Structured Documents Workgroup held in Sydney was with the HL7 Conformance committee.  The topic of discussion was use of conformance terminology, and was applicable to the HL7/IHE/ONC consolidation project.  The results of the discussion were recorded on the HL7 Wiki.  The first part of what was written is pretty much dead on.  The R2 = Should in IHE requires some clarification.

If you haven't already, you MUST become familiar with RFC 2119.  This is the standard for standards makers on the use of the terms MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL.

The HL7 Publishing Facilitators Guide uses a subset of these terms: SHALL, SHALL NOT, SHOULD, SHOULD NOT, and MAY, and adds NEED NOT as the alternative for MAY stated in the "optional negative sense".  The subset is based on discussions of terms that needed to be used according to HL7 interpretation of ANSI rules.

IHE and HITSP use the terms SHALL, SHALL NOT, SHOULD, SHOULD NOT, and MAY most often and in the way that they are defined in RFC 2119, but make no direct reference to these terms.
HL7 uses the terms mandatory and required in HL7 Version 3 modeling, and in conformance statements.  Mandatory means that a data element in a message or document SHALL be present and SHALL not be empty (or contain an exceptional value or nullFlavor [all of which are essentially the same statement]).   Required means the a data element in a message or document SHALL be present, but MAY be empty (or contain an exceptional value or nullFlavor).  In tabular form, these are often shortened to M and R.  Data items in an HL7 model are always at a level no more granular than the HL7 data type or class contained in the model.  So, HL7 conformance statements using Mandatory/Required don't talk directly about the XML produced, but only do so through implication and application of the XML ITS.  Structured Documents CDA implementation guides have transitioned to talking about the XML being produced, and have conformance statements on the XML element content and sometimes the XML attributes.  I find this preferable from an implementer perspective because it addresses what needs to appear "on the wire", and that is where interoperability is most dramatically apparent.  The shift from HL7 class and class attribute dotted notation (e.g., ClinicalDocument.code) to XPath expressions (/ClinicalDocument/code) was subtle but powerful shift that made the guides  readily understood by XML aware implementers.

IHE also uses the terms Required (R), Required if Known (R2), Conditional (C), and Optional (O) in its specifications.  Required if Known is a special sort of conditional that means "if you have the data, you SHALL send it."  The use of term R2 comes from the DICOM standard which has a "Type 2 Required Element".  From Section 7.4.3 of Part 5 of DICOM standard:
IODs and SOP Classes define Type 2 Data Elements that shall be included and are mandatory Data Elements. However, it is permissible that if a Value for a Type 2 element is unknown it can be encoded with zero Value Length and no Value.
The Type 2 Required element in DICOM is most like Required in HL7 Version 3 modeling.  BTW:  DICOM uses Mandatory in much the same way that HL7 V3 does.

When applied to templates created by IHE PCC, R is identical to the use of SHALL requirement [an item using template X SHALL be present], and R2 most closely interpreted as SHOULD [an item using template X SHOULD be present], but the approximation is not exact.  Realistically, it means that if you capture the data, you must send it, but there is no requirement that you must capture the data (see HL7 V2 discussion of RE below).  If the section is not present, IHE does not require you to send an empty length section, instead you simply omit the data.  From a testing/conformance perspective, what I'd like to see for all R2 constraints on document sections is that Document Creators must either demonstrate the ability to create the content, or state clearly that their product does not capture the data and therefore cannot send the information.  Note that IHE PCC has no requirements on document consumers with respect to what they do with the data, so there is no specific requirement that a document section be dealt with by the receiver in the IHE profiles.  As Kevin Coonan pointed out to me in a side channel discussion on todays Consolidation call, the IHE PCC R2 is a dynamic, rather than static conformance criteria.

HL7 Version 2 has a set of conformance abbreviations it uses on fields described in a conformance profile. In addition to Required (R), Conditional (C) and Optional (O), it has Required but may be Empty (RE), and Conditional but may be Empty (CE).  RE in HL7 V2 and R2 in an IHE profile have nearly the same meaning, but HL7's definition for RE is stronger.  It indicates that an application MUST have the ability to send something in a field using RE in a conformance profile, while IHE PCC's use of R2 is a little weaker.

In profiling HL7, IHE sometimes uses R+ as well to indicate things that IHE has made Required which HL7 said were optional, just to make our lives a little easier.

I hope that clears things up about the current state of affairs.

Frankly, I prefer SHALL, SHOULD and MAY, because all of these shortcuts make it a lot harder to understand.  Later profiles in IHE PCC created after the great R2 debate of a couple of years ago (initiated by HITSP and spawned through IHE and HL7).  It is apparent after discussions with HL7 conformance that the discussion is not yet over, and that IHE will need to review its use of R2 in existing final text PCC profiles (XDS-MS, EDR, and XPHR) to ensure that we clearly capture the intent of the committee as it stood at the time the profile was created, and appropriately clarify its terminology. The Consolidation project will capture the intent as we move forward on those final text IHE profiles which become part of the work.

I'd also enjoy seeing a reference to RFC 2119 be used in IHE, and HL7 (although that is a losing battle, as the current terms are hard fought based on HL7 perceptions of ANSI rules).

In almost all of this discussion of conformance constraints, I have been talking about sender / creator responsibilities, and not about the receiver responsibilities to do something with the data that is sent to it.  This is mostly because many of us can imagine some very innovative ways for a receiver to do something with just one tiny bit of the message that we don't want to prevent by requiring the receiver to do anything more than accept a valid message.



  1. Note that R2 is a very specific reaction to vendors/applications/services that would have the value, but pass it in a private-data-element. Thus the data would be passed but in a way that was NOT helpful. So it is a signal to senders that if they don't have it, they don't have to figure out how to get it. This was critical allowance for some UI challenged embedded medical-device.

    So, R2 is a way of saying that a sender SHALL NOT -NOT- send it in the defined way if they have it. Double negative in this case is a positive.

    The receiver should not expect it. So the receiver has to handle it as if it was totally optional.

    But a sender/publisher SHOULD include it while at the same time saying they SHALL NOT send any other way and SHALL NOT -not- send it if they have it.

    Note: My understanding of IHE regarding not having a value is that they expect the underlying standard to be clear if the value is sent as a null flavor or simply not included. Usually IHE doesn't need to add clarity to this factor, it is usually clear in the underlying standard.