This crossed my desk this morning via POLITICO:
GAO: PEOPLE AREN'T LOOKING AT THEIR ONLINE DATA: The Government Accountability Office took aim at the accessible data requirement in meaningful use in a report released Wednesday. The report, requested by the Reboot group (which include Sens. Lamar Alexander and John Thune), argues that while the vast majority of hospitals and eligible professionals make data available for patient and caregiver consumption, the percentage actually following through isn't high — perhaps 15 to 30 percent, depending on the care setting and data analyzed.
Now, why that's the case is the familiar debate — is it a lack of interest from patients, or perhaps technical difficulties? A GAO analysis suggests the former is definitely at play. The office analyzed the top 10 most popular EHRs and found patient participation rates ranging from 10 to 48 percent.
Ultimately, however, GAO hits ONC for its lack of performance measures for its initiatives — whether, for example, using Blue Button increases patient uptake of data. HHS and ONC concurred with the recommendation.
GAO: PEOPLE AREN'T LOOKING AT THEIR ONLINE DATA: The Government Accountability Office took aim at the accessible data requirement in meaningful use in a report released Wednesday. The report, requested by the Reboot group (which include Sens. Lamar Alexander and John Thune), argues that while the vast majority of hospitals and eligible professionals make data available for patient and caregiver consumption, the percentage actually following through isn't high — perhaps 15 to 30 percent, depending on the care setting and data analyzed.
Now, why that's the case is the familiar debate — is it a lack of interest from patients, or perhaps technical difficulties? A GAO analysis suggests the former is definitely at play. The office analyzed the top 10 most popular EHRs and found patient participation rates ranging from 10 to 48 percent.
Ultimately, however, GAO hits ONC for its lack of performance measures for its initiatives — whether, for example, using Blue Button increases patient uptake of data. HHS and ONC concurred with the recommendation.
Here's my take on this.
TLDR; It isn't as bad as the GAO makes it out to be. The report is based on nearly two year old data, and based on it and prior data, we seem to be within reach of a major milestone: 50% of all patients having accessed their data.
Remember early fax machines? e-mail? These are technology diffusion challenges which took a number of years (even decades) to get over. We've finally reached a stage where nearly 90% of all providers are capable of offering patients their data in hospital or ambulatory settings, and now people are really getting to use this stuff.
What is the GAO report telling us? First of all, it is telling us about events in 2015, not 2016 as you might expect. It takes the government a while to get their act together acting on the data that they gather, and the accompanying study also took some time to set up and execute. This is in fact why many opponents of the MU deadlines said they were coming at us too fast, because we couldn't even analyze data from prior years to figure out how to course correct before it was time to update the regulations. We are hopefully past that stage now.
Secondly, we need to look at this in a slightly different context. Remember I said this was a technology diffusion challenge. If you are a long time reader of this blog, you might recall an article I wrote about the curve you might expect for technology adoption. It's a logistic growth curve.
The GAO numbers are saying we are around 30% for ambulatory use, and 15% for hospital use of data access by patients in 2015. Where are we now? It's hard to project forward from one data point, because fitting the logistic curve requires estimating two parameters, a time scale, and the inflection point. The inflection point is at 50%, and is where the rate of adoption reaches its maximum value.
To make something useful out of this data, you have to go back to some similar ONC reports on patient utilization. You can find the data in ONC Data Brief 30, which includes information from ONC Data Brief 20. The challenge here is that the GAO report doesn't quite report the same thing, so you have to adjust a bit. I know from a colleague of mine from early IHE years that some patients get most of their healthcare in the hospital setting (i.e., the ER), while others get their care principally from Ambulatory providers, and others have used both. That means that some patients have been given access through a hospital, and others through ambulatory providers, and the number of patients who have been given access to their health data is overall, probably greater than the sum of the parts, but we know these are largely overlapping sets. So, if I simply take the ambulatory data from the GAO report, and compare the number offered access and the number who used it, to similar figures from the previous ONC briefs, I can start to get somewhere. Here's the basic data.
Year | Offered | Accessed | Total |
2013 | 28% | 46% | 12.9% |
2014 | 38% | 55% | 20.9% |
2015 | 87% | 30% | 26.1% |
The number offered access is different in each year, so I have to normalize the results, which I do by multiplying the % of patients offered access to the % offered access who did access data, to get the total % of patients accessing records. That's the number we really care about anyway.
Now I have three points, which is JUST barely enough to estimate the parameters of the logistic growth curve. How to fit the data? Well, this paper was good enough to tell me. Basically, you compute an error function, which in the paper was least squares (a common enough curve fitting function), over the range of possible parameter values. So I did, and generated a couple of surfaces which show me where I might find the parameters that give the best fit. Can I tell you how good the fit is? Not really. I'm working with three data points, and estimating two parameters. There's only one degree of freedom remaining. This is about as back of the napkin hack as it gets.
Let's look at some pictures to see what we can see here:
First up, we need to look at the error surface to find the minimum.
Error Surface
We can pretty well see it's somewhere in the lower blue region, but we have to get to a much finer level of detail to find it perfectly. Sometimes tables are better than pictures. The minimum is somewhere in the middle of this table. If I expanded data points in the table even further, you'd see they are getting larger all around the area we are focused on.
0.36 | 0.38 | 0.40 | 0.42 | 0.44 | 0.46 | |
Jan-2017 | 1.03% | 0.70% | 0.44% | 0.26% | 0.14% | 0.07% |
Apr-2017 | 0.54% | 0.30% | 0.15% | 0.06% | 0.04% | 0.07% |
Jul-2017 | 0.23% | 0.09% | 0.04% | 0.05% | 0.12% | 0.25% |
Oct-2017 | 0.07% | 0.04% | 0.08% | 0.19% | 0.36% | 0.58% |
Jan-2018 | 0.05% | 0.12% | 0.26% | 0.47% | 0.72% | 1.02% |
The minimum error for this fit occurs somewhere in 2017, which is also the 50% saturation point.
There's still one more picture to show. This projects us forward along the boundaries of the fitted range. As you can clearly see, the projections show we are nearly at the 50% point. That's a major milestone, and something to crow about. It also tells me that unless there's another bump to push things ahead faster, we won't get to 90% access until sometime between 2021 and 2024, five to seven years from now. We have just such a bump (API support) coming ahead in 2018.
It isn't all as dark and gloomy as the GAO report suggests, but it might have been if that report was telling us where we were now, instead of where we were two years ago.
This is rough and ready calculation. I'm using data that was gathered through different means, and which may not be comparable. I don't have enough points to make any statements about the reliability of the projects.
It's still good enough for me. It shows that things aren't as bad as the GAO report might have suggested. ONC and HHS really need to PLAN ahead for this kind of reporting, so that we can create the dashboards needed to produce this sort of information as it becomes available, instead of 2 years after the fact.
Data reporting for 2016 is just around the corner. Two numbers: The % of patients offered access, and the % of patients who use it if it is offered will be enough to tell me how good my projection is for now. If those two numbers multiplied together come anywhere between 30 and 40%, I'll feel very comfortable with this projection.
Hello Keith,
ReplyDeleteI read this report a couple of weeks ago myself and was perturbed by the results and the inability of HHS and others to accurately to measure the impact of patient portals on the general public. It was discouraging. Thank you for this insightful analysis of their numbers. I'm banking that your projections are accurate and look forward to seeing the number from last year.