On Backwards Compatibility

HL7 has a great deal of experience with defining rules for backwards compatibility (which is not to say that those rules always worked).  They also have a whole document devoted to substantive change (for non-members, simply download HL7 Version 3 Normative Edition and read Substantive Change under background documents).  IHE and DICOM have similar rules about substantive change, which is part of what versioning and backwards compatibility are supposed to address.

The problem breaks down into two parts.  We'll start with two systems, N (for New) and O for (Old) which are using some version of some standard.  This is what we need for backwards compatibility that will support "Asynchronous Bilateral Cutover".

New Sender (N) sending to Old Receiver (O)

In this case, System O has certain expectations, and is already in the field.  So whatever had to occur in the message sent to system O should still be possible to send in the new version of the standard (if it isn't possible, then System N and System O don't have a common set of capabilities).  Moreover, if it had to be sent under the old version of the standard, it should also be required in the new version.  If not, then the two systems cannot readily communicate.

Old Sender (O) sending to New Receiver (N)

In this case, System N also has certain expectations, but before the new standard is fully baked, is NOT already in the field.  When receiving information from System O, system N should be able to deal with the fact that it is NOT going to get stuff that is new or changed in the new version of the standard.  Usually System N is a new version of System O, and could work under the "old rules".  So, keep working that way. One of the things that often happens in making changes to the standard is that previously optional stuff now has to be required (constraint tightening).  What System N needs to do in these cases is figure out what a safe and reasonable default behavior would be when it gets a communication from System O where those tightened constraints are not yet used.

Moving On

Tightening of constraints should be OK behavior in moving from version O to version N of a standard.  We know that loosening constraints is often a problem as we move forward.  However, the expectation of the receiver should be that old constraints are still OK for some period of time, in order to allow others to catch up asynchronously.  Determining this set-point is an interesting policy challenge.  At the very least, you should "grandfather" older systems for at least as long as they have to "upgrade" to the new version of the standard, and perhaps a little longer to deal with laggards (as we've seen has been necessary for 5010 and ICD-10 cutovers).

At some point, you have to drop support for the old stuff.  A good time to do that might be when it is time to move to version M (for moving on) of the standard.  What we want to do to support Asynchronous Bilateral Cutover is think about changes so that required behavior is invoked more slowly, and that things that were not previously required start off first as being desired, or preferred behavior (which is required in new systems, and allowed in old systems).

Exceptions

There will always be cases where an exception needs to be made.  That needs to be carefully thought out, and in new versions of standards, Exceptions to the backwards compatibility rules should be clearly identified when the standards are balloted.  We shouldn't have to figure our where those are just by inspecting the new rules.  Some features need to be retired.  Some capabilities cannot be changed in a way that won't impact an older system.  The key is to clearly identify those changes so that we can get everyone to agree that they are good (which is not the same as agreeing that they aren't bad).

With the new Templates formalism, we should be able to identify these issues with C-CDA going forward.  I would very much like to see HL7 sponsor a project to ensure that C-CDA is made available in the new format once it finishes balloting.