Those of you who regularly follow me know where I am this week... The IHE North American Connectathon. I think that connectathon is one of the coolest events the healthcare industry does, and its because of the collegial atmosphere and get it done attitude that permeates the whole event. I'll be live-posting on Connectathon more this week, but I wanted to start off the day with an interesting and related exploration.
A couple of days ago, Doug Fridsma (the keynote speaker at the Connectathon conference on Tuesday) commented to me that this is something that should also be available virtually and year-round (he attended the HL7 Working Group meeting which I was also at last week).
| Help Needed
I'm gathering all the web commentary on the PCAST report I can find for a submission to the ONC RFI. If you have links that may not be what I have already seen, tweet me or send me an e-mail with it. |
I've been thinking about this topic a little bit, and wondering what a virtual connectathon would look like. There are a few challenges to overcome to make the connectathon a virtual event, and the first of these is the intensity of the atmosphere. This year's North American Connectathon includes more than
150 systems (up 25% over last year), which will execute over the course of one week tens of thousands of tests which will be reviewed by a
team of more than 50 people. According to Bill Majurski, about 70% of the registrants will be using
XDS in some form.
I recommend that product teams send at least 2 people to connectathon per product being tested, which gives a fixed cost of about two weeks of effort. You can do it with one (I've been there and done that), but that's a mountain of effort to put on one persons plate. Teams also put in about 1-3 weeks of effort pretesting each profile (not including development time). Most companies test more than one profile, and can often take advantage of overlaps in product requirements to reduce the aggregated time. Even so, it's still a large time committment. Spreading that effort out in a virtual event over the course of a year reduces the intensity.
But the intensity is one of the reasons why connectathons are so valuable. Participants have a week to succeed. To do so, they
MUST work with their partners, sometimes deep into the night. There is no other choice, and this necessity makes for
partnerships not heard of in the real-world. How, in a virtual event, can we ensure this kind of participation? Outside this room, these people are often stiff competitors, but inside they are your testing partners. You work face to face with your partners to succeed at this event, sometimes shouting down the row, skyping or talking with cell-phones, while making code changes live. A success is often celebrated with a beer or dinner later with these same people.
So a virtual connectathon has to have another purpose, and another way to ensure success, than the event I know and love. It's pretty hard to share a beer virtually, and without the deadline, impossible to get that kind of coordinated effort among competitors. Some other possibilities come to mind. One is to have shorter, more frequent regional events. We've done that for several years. IHE members attended a number of different events and demonstrations, some of which included connectathon like testing including the
VITL Summit, the eHealth Connecticut Demonstration (2008), and PHIN's
annual conference. While it adds value, it's not what I think Doug is asking for.
So what would a virtual, year-round testing event look like? Who are the stakeholders? What are the benefits? What are the costs? Who would pay for them?
Stakeholders and Benefits
- Governments
- Readily available testing of healthcare IT that meets regional requirements.
- HIE Organizations
- An opportunity to test their systems with new products as they become available.
- Healthcare Providers
- Products that have been tested more recently and frequently than annually. The ability to test homegrown solutions and integrations.
- Healthcare IT Vendors
- The ability to test more frequently than annually, the ability to test at much lower cost and intensity of effort, the ability to test with partners that you didn't have the opportunity to test with at connectathon.
Costs
There are a couple of things that go into the cost equation. You probably need a virtual private network to set up a "Connectathon" like network environment. You need someone to manage this, assign access, et cetera. You also need monitors / test proctors and someone to manage them. Connectathon itself requires a team of 3-4 people to manage all that, plus a team of about 50 monitors. You could probably get by with a team of 1-2 to manage the virtual environment and manage test results to start off with, but you still need a larger team to proctor the tests.
The IHE monitoring team is made up of volunteers. The concentration of time at Connectathon is what makes it possible for many volunteers to participate. Some of the monitors are friends of friends of IHE who got dragged in once and keep coming back, others are IHE committee chairs or participants, and a few others are contractors working on large projects (e.g., HITSP) who come as part of their work. Most come to Connectathon because they like the atmosphere, support the work being done, and they also get free travel to Chicago in January. Some are giving a week of their lives for nothing more than the experience on the Connectathon floor, using up vacation time to boot. Many are rather skilled IT people, some with very specialized skills.
Something else has to be done to provide value for monitors other than the free travel because there is no free travel once you go virtual, and you've also radically changed the atmosphere. Because of the diversity of experience, you'd probably need to pay several skilled individuals to do the job, and you'd need to invest in their skills as well. My guess is that to do as much work as the 50 volunteers do in one week over the course of a year, you'd need to hire 3-6 part-time contractors to do the same work, and you'd also have to spend some time training them, which might also include participation in new IHE work.
I'd ballpark that a virtual connectathon would have a budget of anywhere from a half-million to a million dollars depending upon how you got the monitors.
Paying for It
So, just to make it simple, let's say that the cost was a cool million dollars. Cheap at twice the price. How could that be funded? At $4000 per system you'd need to get 250 systems to participate in the virtual connectathon. That's more systems than participate in the annual connectathon, and an enrollment likely not to be reached for some time if ever. I could see maybe 100 systems after two years if the right incentives were present to participate, and in the first year, 25 - 50 systems if you were lucky.
We could maybe cut the networking costs and staffing requirements by getting the state HIE's to supply resources, equipment, et cetera to the effort. Maybe the RECs which already have developed some requirements of their own could support testing efforts by supplying monitoring and management skillsets in exchange for having a testing environment and audience they could use for their own purposes. Educational institutions could support testing as well, exchanging student assistance with testing for the training that the virutal connectathon experience provided and the educational credits that the institution offered. Hmm, those two weren't even in my original list of stakeholders.
We still lose the intenstity of the live event, but that could be supplied (in smaller doses) by other special purpose time driven events. State HIE X could sponsor an event designed to test conformance of a variety of systems to an Immunization message. REC Y could sponsor an event designed to test conformance of a variety of systems to their own region's initiatives. Sponsorship of an event might have its own costs born by the sponsor, with possible additional fees assessed to event participants, but those fees would have to be reasonable.
I could certainly see the advantage to vendors to be able to have a system connected year round where these sorts of tests could be routinely performed. It could eliminate a lot of duplicated effort around the country, and could enable many activities not possible in current Connectathons. Some of those same resources could be used to support other opportunities in education and innovation later, but I wouldn't want to get too defocused in the first year or two.
It's an idea worth persuing further. If I were on the board IHE USA, I'd be thinking long and hard about this one. We could make this idea work, and it could shortly become self-sustaining. It wouldn't be the same as Connectathon, but nothing virtual ever would be. If you happen to be at Connectathon this week and have read this post, I'd be interested in getting your feedback.
