Conformance of an Application vs. of an Instance

One of the challenges with conformance is distinguishing between conformance of an instance, and conformance of an application.  All too often I hear that "this CCD document isn't valid" because it doesn't pass the NIST CCD Validator for Meaningful Use, when in fact, the instance is valid, because the data doesn't exist for that patient at the time the document was generated.  A perfect example is labs.  You have to be able to generate lab results when they are present, but not every patient has labs done during every visit.  The key to conformance for the instance is to generate the lab section correctly to indicate that none were drawn.

There have been some very long, drawn out discussions over the use of the terms SHOULD and R2 (Required if Known) in HL7 and IHE specifications that in part, result from this difference.
One of the ways that we could be more clear in creating interoperability specifications is to specify what an application must be able to capture as well as what it must be able to transmit.  In this way, we could separate application conformance from instance conformance.  For HL7 CDA, we've been a bit sloppy with this.

What required if known means is that an application SHALL be able to report a value when it is known, but it was never really clear whether it being known was a requirement of the application. We have often shortened the phrase "Required if Known" to "SHOULD" because it is simpler to say, even though SHOULD is somewhat different in meaning.  I have to take some of the blame for this; I was translating "Required if Known" to non-technical folks, and did so badly back in 2005, and it stuck.

HL7 uses the words Mandatory and Required to deal with this topic to some degree.  Mandatory means that A) the application must be able to capture the information (it can never be unknown) and B) must transmit the data.  Required means that the application A) must be able to capture the data (but it could be unknown), and B) must transmit it.

For required data elements, HL7 conformance says that:

... conforming applications must demonstrate their ability to provide and communicate not null values. Receiving applications must demonstrate their ability to receive and process (eg. Store, display to users) not null values for required elements.

DICOM breaks these down into Type 1, Type 2, and Type 3 attributes.  Type 1 is equivalent to HL7's Mandatory (in fact, DICOM uses the word Mandatory in the definition of Type 1).  Type 2 is equivalent to HL7's required, and adds this note:

The intent of Type 2 Data Elements is to allow a zero length to be conveyed when the operator or application does not know its value or has a specific reason for not specifying its value. It is the intent that the device should support these Data Elements.

In DICOM, Type 1 and Type 2 also have "Conditional" variants, known as Type 1C and Type 2C.  These are like Type 1 and Type 2, but only when the applicable condition is true.

In both cases, HL7 and DICOM provide a mechanism to say what an application must be able to do, and what an instance must look like.  However, we've gotten confused over time about what this means.  Separating required application behavior from what must appear in an instance could help clarify this.