Thursday and Friday at CAT15

Thursday is when the rubber meets the road.  By Thursday night, you've passed all the easy stuff, but if you are like the people sitting around me, you are trying to make sure that you can get that one last test started.  Because if you don't start it by Thursday to get it into the queue, you'll be dropping that profile (it's a Connectathon policy so that monitors aren't swamped with profile tests that won't be worth checking on Friday).

A lot of new code gets written on Thursday as the key priorities have all been hit, and people are reaching for their stretch goals.

One of the things that is new at Connectathon as of 2015 is the number of people who have finished the hard stuff  already.  Many vendors have already finished their XDS, XCA and ATNA tests.  CDA viewing is pretty much done.  One profile (Family Planning) had all vendors finished with it by Wednesday, including a couple of late adds from the Connectathon floor.

There are two factors I think that have to do with this.  One is that the testing infrastructure is improving year over year, which makes a lot of the testing go quicker.  Gazelle has improved to the point that there have only been a few capacity problems that we experienced over the week.  There is still room for improvement here, and also automating some of the verification processes, but compared to where we were ten years ago, this is monumental change.  The network is no longer spelled with two o's, and even though we had a bobble with a switch at our table, it was readily resolved.

The second change is the vendor community interest in interoperability.  I see a lot of newbies here this year, which would have led me to expect more people struggling.  But there's also a lot of people here who know the ropes who have helped them along.  And the volume of information that is available about how to implement the profiles, and the public domain implementations (many of which have been tested here over several years) also make it a lot easier.

Some folks still tell me that Connectathon won't scale, but in fact it has.  The model has also been adopted by a number of regional projects for preparing for real world implementations (we call these projectathons).  It's like the old concern about the cable network being used for Internet access. Detractors said the cable infrastructure wouldn't scale either, but it certainly appears to have done the job.  Connectathons will continue to be an important interoperability event, simply because there's nothing else like it where you can get so much done in so little time.

What takes weeks in the field, we do in hours here.

     Keith

Tuesday at CAT15

Tuesday's activities at Connectathon is where real interoperability starts to happen.  Monday is about figuring out your network connections, getting set up, and starting to execute on your no-peer tests. On Tuesday you really start working with peer systems.

A couple of things could happen today which would impact your work.

You might discover that a peer you have been working with abandoned a test you had started with them to work on another problem.  This is a persistent problem at Connectathon, because much of the work is interrupt driven, and your stack (or their stack) can often overflow.  It's a good idea to keep a list of things you haven't finished, and check up on the peers you are working with to ensure that you and they are still on track to finish the test.  Never expect someone else to complete a test without checking on it.  It's a mutual responsibility.

The only way to fail (or succeed) fast, is to get the work checked out fast.  Don't forget to mark any tests you've completed as being ready for validation.  I can't tell you how many times I've heard someone groan because they discovered hours or even days after finishing a test that it didn't get validated because nobody marked it as being ready for that step.  The validation queues get longer as the week goes on, so finish that step as soon as you can.  Keep working after the monitors go home if you need to, but get that in the queue today if at all possible.

Next, you may very well discover a blocking problem.  This usually happens when you fail your first or second verification attempt on a profile.  You need to quickly analyze those failed verifications. I've seen teams which learned on Thursday or Friday that a test that they had completed failed validation, and so there is a mad scramble to fix whatever the problem in time to get the missing test validated.  So remember to keep track of the tests you've done and make sure they are getting validated.  If you wait until Thursday, it could be too late.

When fixing a blocking problem, you will often stop work on tests that won't be valid.  Abort them so you don't have to wade through the clutter to find the results you really need.  And tell your test partners why you are doing that.  And for those tests you simply put on hold, make sure to go back and complete them.

You may discover today that something needs to be rebuilt (and pray that it is today rather than tomorrow). If you don't have the code and compiler locally, make sure that your teams back home have a way to get you an update quickly. Don't count on the Connectathon network to enable you to access a 300Mb download quickly and easily.  Consider what is happening on this network - with hundreds of engineers testing messages. Sometimes a screen refresh to an external website can take several minutes, even for a 140 byte tweet.  FedEx might just be faster...

Late this afternoon, you will probably discover your first ATNA bug in your new HTTP or other stack. Check the ATNA FAQ for some of the best advice gathered by myself and others over the past decade.  You should already have Wire Shark installed.

Monday at CAT15

Yesterday at the IHE Connectathon I got pulled in to help a developer who had gotten pulled in at the last minute to replace someone else who couldn't make it.  Being unfamiliar with the tools or the process, I showed him how to make a plan to succeed.

The first thing you do for your product is prioritize your work.  Since  profile actors embody the chunks of functionality being tested, they become what you prioritize.  Some profile actors are critical because they are going into shipping product, or could be of use at several customer sites, or will be demonstrated this year at the HIMSS Interoperability showcase. These have to get tested.

Some profile actors are nice to have.  These might be used in future releases, or could be important to a single customer, or you are thinking about demonstrating this profile actor at HIMSS, but haven't committed to that yet.  The nice to have stuff should get tested, but not if it prevents the critical things from getting done.

Finally, there are your stretch goals, or what I call gravy testing.  These are the profile actors that, got added to your plate last week by your product manager to support some hare-brained scheme you just heard about, or some attractive showcase leader walked up to you and asked if you support ___, and you said anything less convincing and firm than no, and as a result found yourself signed up to test a profile you just heard about five minutes ago.  The important thing about these is to remember NOT to let somebody else's priorities become yours simply because you haven't assigned any of your own.

Once you've assigned priorities for actors, now you can prioritize tests.  If you are doing anything requiring Concistent Time (CT), or Audit Trail and Node Authentication (ATNA), do your CT test first.  Then you should look for tests you could have prepared for before you got here (e.g., generating a CDA document to upload for verification).  The monitors expect these first, and it's always good to be nice to the monitors.  Ideally, this is on a stick, and you just upload your samples (you can even do this sometimes before you even get your system being tested unboxed).

Do your tests basically in this order:  No Peer Tests first, Peer to Peer tests second, workflow tests (supporting multiple peers) last.  Don't worry if you take some out of order.

Things not to do: Do not do options testing on a profile actor before nice to have tests are done unless the option is truly critical.  Options fit into the gravy.  The only exception to this is when multiple options are provided in a profile, and at least one must be supported.  Fine, pick your one and call that critical (if the profile actor is critical) or nice to have.

Also in the gravy category are tests with your own products.  These might also fit into the nice have category for you, but the reality is, you shouldn't need IHE to set up a Connectathon for you to test with your own products.  Do it if you can, but only after you are safely passed the critical and nice to have stuff.

After you've made your plan, you have one more job to do.  Go through the tests which are critical and find out who your potential partners can be.  Get their table numbers.  Now, go walk the floor (after you've completed your CT test and uploaded your samples), and introduce yourself to them. Let them know you'll be working together this week.

Ok, so that's Monday (a day late).  I'll try to get Tuesday done just a bit earlier.  What can I say, I had a critical issue to address yesterday, my blog is just gravy.