Friday, April 3, 2020

What is 6000 * 5 * 4 * 7 * 52 / 60 / 24?

To damn much if you ask me.

6000 hospitals.
x    5 minutes / form reported
x    4 forms reported / day
x    7 days / week
x   52 weeks / year
--------------------------------------------------
= 30333.333 hospital minutes

How fast could a computer report that if we but had the right infrastructure?  Surely 0.333 minutes would be enough.

I do NOT want to hear someone's brilliant idea about how to use the web to solve the COVID surveillance problem one more time if all they can do is implement <FORM>.  Sure, it's FORM on top of a Bootstrap/React UI with Node.js and NOSQL backend, but the front end input is still a human at a keyboard, the same as it was when Tim Berners Lee invented HTML in the mid-90's.

I do NOT want to hear about one more front-line medical profession typing in stuff into a form.

FIND ANOTHER WAY.

Use a camera, a sensor + arduino, an ultrasonic tag, or spare printer parts.  Do whatever it takes to enable people in a crisis to operate at the top of their license, not the bottom.

If "all you need is a phone", understand that in that phone is a supercomputer that is 100,000 times more powerful than necessary to land on the moon.  Use it the damn thing.

   Keith




Tuesday, March 31, 2020

Classifying Strata for Beds and Ventilators for The SANER Project

I'm presently working on terminology to help classify what is being counted in various measures for Beds, Ventilators and other equipment for The SANER Project.

As I look at the classifications that others are using:

Physical Capacity: Total number of beds (or other things being counted)

This breaks down in two different ways, by licensure and staffing.
Licensure:
Licensed Capacity: Number licensed (interesting but not important in emergency cases)
Surge Capacity: Number of additional that can be added in overflow situations.

Staffing:
Staffed Capacity: What has staff to support treatment.
Unstaffed Capacity: What does not have staff to support treatment.

Type of location:
A Bed is located in a part of a hospital (or similar facility) and is intended to support:
Inpatient care - Beds meant for patients with acute disease, but not needing emergency treatment.  Inpatient care includes patients admitted for "Observation".  They aren't well enough to return home, but they aren't sick enough to require a higher degree of attention.  The distinction between Inpatient and Observation is generally a billing distinction, not one that truly addresses other characteristics of location.

Acute care: A subtype of inpatient care that provides for treatment of patients with care, but not intensive.
Intensive care: A subtype of inpatient care, providing a higher level of care, treatment and staffing than acute care.
Critical care: Some institutions have a level of care between Acute and Intensive which has higher staffing levels and treatment needs that normal acute, but lower than "Intensive".  Cardiac care units and critical care units fit into this category.  For the current situation, we think that CCU (whether cardiac or critical care) should fit into the Intensive care category when counting.

Burn Units: A burn unit is a specialization of an ICU that supports those needing treatment for burns (heat or chemical).  These have additional equipment needed to treat patients who have significant loss of skin due to burns (e.g., cooling baths, higher temperature controls, and additional treatment resources).  Such units might be used to treat patients who need ICU for which a normal ICU is not available, but this is not ideal use of resources.  There's some question about whether these should be counted the same as ICU beds or differently from ICU beds.  For the current crisis, this question may not be able to be answered.  There will be crises in the future where this distinction is critically important.

Emergency care - Emergency beds are those meant to treat patients who have urgent or emergent
care needs which must be addressed before admission (or discharge to home or another location for treatment.

Post-Acute care - Other facilities also provide spaces for treatment of non-acute disease, which I would describe as care needed to support rehabilitation and recovery, or long-term care.

Outpatient care - Outpatient care beds include those meant for patients who are recovering from a procedure, or in other similar situations.  See rooms below.

Other hospital facility space includes:
Operating rooms - facilities for performing surgical procedures.
Procedure rooms - facilities for performing other procedures (usually diagnostic).
Recovery rooms - facilities for treating patients recovering from surgery or other procedures that do not need post-procedure acute care.

These spaces may be reconfigured in emergency to support other uses.

Beds may be designated to treat patients within a certain age group:
Neonatal care beds (Nursery, NICU) are designed to support newborns and infants.  They cannot be readily used to treat older children or adults, simply due to size limitations.

Pediatric care beds are designed to address the needs of children.  They might be used to treat adults in cases of emergency.

Adult care beds are designated to address the needs of adults.  They can be used to treat children in cases of emergency.

Many reasons for the distinctions between adult and pediatric beds are to address the different needs of these two populations when staffing and resourcing and planning the facility, rather than physical capacity restrictions (unlike the situation for neonatal care).

In Use/Available/Unavailable
Beds and other assets are either available or in-use or otherwise not available (broken, missing, on loan, et cetera).  When being counted according to current guidelines, nobody seems to have addressed the "not available yet not-in-use" state.

Supplies (test kits, masks) are either on-hand (available for use), consumed (used for their intended purpose), disposed of (e.g., due to contamination, transfer to another location, or otherwise).  For test kits, we need to distinguish between "test kits" which do the actual diagnostic test, and "specimen collection kits" which are used to collect and safely transport the specimen from the collection location to the testing facility.  A shortage of either can cause problems.  We also need to clarify the type of specimen used (blood vs. nasal or other swab) b/c sending a blood sample to a site that can only test via swab isn't going to help.

For equipment, we see discussion around ventilators and ventilator slots.  We should be counting both in some way if both are in use.  Adding slots to a ventilator (treating two or four patients with one ventilator) is being done today.  It's a modification of a medical device that should only be done under specialized scruitiny and only in emergencies, but it is being done in New York at this stage.

There should be some clarity around how these are counted, because if people start thinking about how to redeploy ventilators to support patients, and one facility is counting slots and another counting ventilators, that's a problem.  I'd suggest to "count ventilators" when talking about equipment, and count slots when talking about patients.

There's also the distinction between invasive ventilator (those requiring a tracheal tube), an non-invasive ventilation (CPAP and BiPAP), and also between those that are designed for long-term use and solutions that are coming out of the maker/innovator community which are specifically designed as "surge capacity devices."



Sunday, March 29, 2020

Printing a FHIR IG

panicking leslie nielsen GIF For folks who are used to working in Microsoft Word or with PDF documents, trying to review a FHIR Implementation Guide on the web can be a little bit daunting for the first time.  A colleague of mine tediously copied all the pages for a guide that we had produced for the Mobile Health workgroup so that others could do corrections in line.

It's really not that hard to do, just a tedious way to spend 15 minutes to an hour (depending on page count).

I've turned other highly linked web formats into single page HTML specs in the past ... it really isn't all that hard to do.  You need a scraping tool, HTML Tidy (jtidy won't cut it any more, it just hasn't gotten the love), and a very little bit of XSLT.  If the specification already has a table of contents page, it's even easier because then you don't have to worry about how to order the material.

Just about every FHIR IG now produces a ZIP file with the entire specification.  So, download the zip file, unzip it to a folder, and run the this XSLT against it (after you clean up the html).  I threw this together in like 10 minutes between five other things I'm trying to get done, but it worked.

If you are using a spec I worked on, there's a "download this specification" somewhere on the home page.  Otherwise, just replace index.html with full-ig.zip if the person who wrote the spec is using the FHIR IG Builder or older tools.  That will get you the full data.

  1. for %f in (*.html) do c:\util\tidy -utf8 -asxhtml -n -m %f
  2. java -cp c:\saxon9h2\saxon9he.jar net.sf.saxon.Transform -s:toc.html -xsl:gen_single.xsl  -o:single.html
  3. single.html

After you've launched the output, you can:

  1. Print to PDF
  2. Copy and paste everything on the single page to a Word document and distribute it.
  3. Turn on your printer and spend a ton of money printing something out.
There are a lot of places this could go.  I would love to see this little gist turned into something that would turn a FHIR IG not just into a PDF, but rather into an e-book.  It's not that difficult to do.  But, maybe I'll get back to it when the crazyness settles.

Meanwhile, stay sane, stay safe, and stay home.

    Keith

Playing with Control Charts for COVID19

I've been playing with Excel and control charts over the last week to get an understanding of where things are headed.  I get the human difficulties with understanding exponential growth, so I wanted to look at the data in a different way.  The italic text is my commentary on the blog.  The normal faced text is what I've been tweeting, the bold text is when.

3/20
If it looks to you like the exponent on infection growth rate is increasing, you are probably right. I just looked at the 5-day LOGEST values (estimate the exponential growth based on last 5 days activity), and the rate has risen 4 out of the last 5 days. Testing just started...

So, this isn't scary to me YET. What it means is not that the real exponential growth rate of infection is increasing, but rather that the rate of our knowledge of exponential rate is increasing. But more testing is still needed to get the numbers to settle down ...

There's gonna be lots of numbers for the epidemiologists and hyper-mathy folks to study RE impact of testing volumes (see ) on estimates of real growth rate when this is over. I don't recall signing up for that clinical trial though.

I suspect there they may even be some new hyper-mathy stuff that addresses statistical process controls applied to exponential growth curves. I kinda faked a control chart with my 5-day moving estimator for the growth rate. There should be some useful signal there ...

I know enough math to know that something should work, maybe even already exists (but maybe and even possibly not yet). I also have enough math to know it's not going to be me who actually proves it.

I've seen enough in that one graph to tell me what I already knew, the real rate is worse than a lot of people thought, or the current graphs show. The good news is that we are seeing a correction now. The bad news may be that it will take a longer time than we want to adjust.

But I'm still not panicking yet.
And neither should you.

3/27 (early morning) 
I just updated my spreadsheet. The good news is that the infection rate is declining, and has been for eight days in a row. However, it's also stabilized, which means it might be getting ready to inflect the other way.


If you look at the graph above, you see eight points in a row declining.  That's enough to trigger an alert in a statistical process control environment (actually 6 or 7 depending on whose set of controls you use).

3/27 (later in the day)
OK, my chart gets better. I can compute New Cases / Total Cases, which should be a constant for an exponential function. In fact 1/(1-New/Total Cases) recovers the base of it. Here's the new control chart on the base value, I find it more helpful.



This above graph was inspired by this great video on plotting exponential data.  It is worth including in this post, just so you can see what I'm doing is valid.  I still like the orientation of my approach better because you don't have the overwhelming sense associated with a graph plotted on logarithmic axes gives you.  It's just a little number bouncing around telling us how fast cases are growing, without really making it apparent how fast an exponential works.  Since I'm looking at that growth everyday, and listening to my family's response to it, I get the impact of it.

3/28
Today's update is promising. The downward trend in growth shows we are making progress. Note that it took two weeks to get farther than 1 standard deviation in the right direction. Maybe by tomorrow we'll be at 2.



Several additional notes here:
  1. Remember that set of 8 control points going down?  How come I don't have that once I switched to this format?  That's because I'm no longer reporting the 5 day average, just the current days rate.  See that dip on 3/21?  It gets included in the 5 day average in the previous format, which hides the current day's signal.  I kept the 5 day deviation because it tells me something about how the rate is settling in, or not.  So, now we have 6 on the same side of the average line, that's another kind of signal in statistical process controls (or 7 or 8, again, rules vary).
  2. Statistical process controls are for managing stable processes.  When the process is changed, you have to adjust the controls your are using.  I'm thinking that I should be marking the graphs above on a weekly basis with the per week mean and standard deviation lines so we can see HOW the process is changing over time.
  3. There's no API to work with the data I'm using, but I there's a way to scrape it from the page I'm recording numbers from.  I haven't bothered because it's licensed data.  What I've done above falls (as best I can tell) under fair use.  Scraping their page dynamically isn't what I'd call fair use.  If they do publish an API for this data set, well then I might take a crack at doing something with this little gem of a gist.
  4. There IS a source of publicly available data  from John's Hopkins that does work via APIs.  The API is simply: https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_daily_reports/{date}.csv





Saturday, March 28, 2020

Technical Environment for The SANER Project

This started out as a page describing the technical environment for the SANER Project, but I'm busy reintegrating some of my development tool set back into the build, and so have not been able to push a new build.


Resource utilization data can be provided from a number of sources:

  • Systems Having Clinical Data
    •   Electronic Health Record (EHR) systems
    •   Emergency Department systems
    •   Labor and Delivery systems
    •   ICU and/or Nursing Central Monitoring systems (or stations)
    •   Laboratory Information Systems (LIS)
    •   Clinical Data Repository (CDR)
  • Systems not having Clinical Data
    •   + Bed Management (a.k.a., Housekeeping) systems
    •   + Asset Management systems
    •   + Surgery/Operating Room Scheduling systems
    •   + Staff Scheduling systems
    •   + Inventory Control systems

Systems Having Clinical Data

Systems in this category have access to some or all of the health records for a patient,
and so can often provide information indicating COVID-19 positive or suspected patients, as
well as associated problems, patient demographics, and patient acuity (severity of
illness) data.

These systems are often used to place or discontinue orders that involve medical
equipment (ventilators, viral tests), and so may be used to determine in use (in
the case of ventilators) or consumed (in the case of test kits) equipment or
supplies.  Some orders might also indicate use of special equipment (e.g., isolation
rooms).

EHR Systems

Comprehensive Hospital EHR solutions may include the capabilities of the other systems
listed above, or may be integrated with other systems but not have direct access
to all data available to the other systems. Even when those capabilities are available
in the comprehensive EHR solution, other solutions may still be chosen by the facility
for a variety of reasons (features, cost, legacy, et cetera).

An EHR may have access to beds in use (because it has access to the active patient
census), but may not be able to report status of beds as known by the bed management
or housekeeping system (e.g., beds available for use, beds needing cleaning, beds
taken out of or added into service, et cetera).

An EHR may also be able to make an educated guess about number of ventilators in use
based on the number of orders for ventilation on the patients it knows about, and the
current status of the order.

Emergency Department Systems

Emergency department systems are simply specialized EHR systems that facilitate patient
care in an emergency room setting.  They may also support or be integrated with central
monitoring solutions enabling ED staff to monitor the status of patients on monitoring
equipment.

Labor and Delivery Systems

Labor and Delivery systems are another form of specialized EHR system that facilitate
treatment of mothers about to give birth in the hospital.  They generally support the
ongoing monitoring of the pregnant mother, and integrate with specialized equipment
used to support newborn delivery (e.g., fetal heart rate monitors, infusion pumps used
for anesthesia, et cetera), as well as routine charting while a mother is still in labor
but not yet ready to deliver.

ICU/Central Monitoring Systems

These systems bring real-time data from the EHR together with a variety of monitoring
and treatment equipment, often to provide clinical decision support for patients
needing intensive ongoing treatment and monitoring.  As a result, these systems have
awareness of the use of medical equipment, patient acuity, disease progression, as well
as the in-use status of ICU beds, and perhaps the total bed capacity of an ICU (but
not necessarily the availability).

Laboratory Information Systems (LIS)

These systems are used to track and control incoming laboratory orders, to manage
laboratory automation equipment, and to manage outgoing reports on orders.  They have
some access to patient clinical and demographic data, usually enough to facilitate the
interpretation of the laboratory test, but may not have access to more data.  Some
data available in an LIS might be used to assess patient acuity, but the EHR would
be a better source of this assessment.

An LIS may also be connected to external public health reporting systems to support
biosurveillance efforts (tracking of disease in populations).  Just the placement of
certain kinds of laboratory orders may be used as a trigger to initiate alerting to
public health (e.g., highly contagious disease such as Ebola or Zika, or a condition
which may indicate a high risk situation in the community such as food poisoning).

Past biosurveillance efforts have not generally considered the impact of disease (such
as COVID) on available beds, but the impact of COVID on hospital bed capacity has now
made this a significant consideration.  Some organizations do use data from internal
laboratory information systems to track the prevalence, type and locations associated
healthcare acquired infections (HAI) (e.g., due to antibiotic resistant strains of
bacteria), in order to provide appropriate treatment and infection mitigation precautions.

Clinical Data Repository (CDR)

Hospitals (especially those affiliated with academic medical centers) utilized CDRs
for long term storage of clinical data to support analysis, research, measurement and
quality improvement efforts.

CDRs may have information about the long term impacts of disease, treatment procedures
and other factors on hospital operations might be used to aid research, but do not
generally have real-time data that could support utilization reporting.  Some of the
data in a CDR might show impacts of high utilization on hospital operations, which could
aid in identifying and addressing long term monitoring efforts.

Systems Not Having Clinical Data

Systems in this category do not generally have access to health records for a patient,
but may have information about the status of hospital equipment and supplies.

Bed Management (a.k.a Housekeeping) systems

Bed Management systems are designed specifically to keep track of the status of beds
as it impacts the operations of the housekeeping in a facility.  After a patient is
discharged, the bed and room in which they resided need to be cleaned, special precautions
may be needed when rooms have been contaminated (exposed to blood, or infectious organism),
et cetera. These systems may also have operational data about ongoing bed turnaround
time (e.g., from unoccupied to available for use) which can also impact availability.

In smaller hospitals, the classic method of bed management is a bed board, which can
be as simple as a whiteboard with a table drawn on it, with room numbers, more complex
systems might use a magnetic board with pretty colored magnets. Modern bed boards get
really fancy, with ward layouts and color codes, and all sorts of bells and whistles
and reports and graphs.

Asset Management systems

Asset management systems usually involve solutions that enable a facility to manage
equipment inventory, tags that can be attached to equipment for tracking, and sensors
that can detect nearby tags deployed in the hospital environment.  Sensors typically
need to be connected in some way to the hospital network, and the tags need to be able
to operate in a radio-frequency and sound and barrier rich environment.  This is combined
with mapping software which can plot the location of a device in a facility (in 3
dimensions).

A typical small hospital might have 10 ICU beds.  With an average ventilator utilization
ranging somewhere between 15-45% (see https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3840149/),
a facility could reasonably get by with a half dozen or so ventilators for the ICU.
Under normal circumstances, such a small number would be reasonably managed with manual
processes, and for that reason, many facilities may not use asset management systems
to track ventilator locations.

Surgery/Operating Room Scheduling systems

In certain crises, the availability of an open OR for a patient needing immediate emergency
surgery would be a critical piece of facility resource utilization data.  Operating
rooms use a variety of complex, expensive medical equipment and resources.  Hospitals
that want to optimize use of these spaces and equipment will use information systems designed to
ensure greater utilization of available capacity.  These systems will be aware of the
availability of operating room schedules and equipment needs.

Staff Scheduling systems

Staff scheduling in a hospital is an ongoing effort of looking at the current patient
load, forecast patient load, existing staff schedules and available surge staffing resources.
Again, in smaller facilities, much of the essential management might be done through
human effort, and on a whiteboard. In larger facilities, software might be used to address
optimization of schedules, improving staff utilization.

Inventory Control systems

Inventory control systems are used to track expendable supplies, and manage replenishment
and distribution across the facility.  These systems may have information about available
supplies such as surgical and N95 masks and personal protective equipment (PPE) that is
frequently replaced.  But these systems don't always automate the counting process that is
often needed to track current inventory levels for this kind of equipment.

Friday, March 27, 2020

What will The SANER Project Produce

It's a great question, and one I have in my head conceptually, but it's time to start writing this down:
  1. A specification for how some things are going to work (basically a FHIR IG).
  2. Two or more deployable FHIR Server images (1 .NET and 1 Java based) that can be used to support reporting of utilization data from hospitals, and viewing of that data by public health.
  3. Interfaces delivered by major EHR, Health IT and Medical Device vendors that can populate the server above.
  4. One or more components that can take data from that server and produce visualizations including:
    1. Maps of facilities or regions that depict information about utilization
    2. Graphs of trends over time for a specific facility or region that shows resource utilization and/or capacity.
  5. One or more components that can consume data from that server and provide alerts about facilities or regions having (or no longer) being challenged.
  6. One or more components that can consume data from that server and provide measure reports based on the available data.
  7. Components that can bridge between what is delivered and existing solutions:
    1. A way to quickly classify beds in an institution.
    2. A way to transform the resulting data into  
  8. A classification system for beds that allows systems to participate with some minimal level of supporting information and evolve to providing more granular data without disruption.
  9. A classification system for reporting about mission critical equipment and supplies, and perhaps even a process or set of principles to use for establishing such classifications for future crises.
  10. A virtual testing event where these materials can be tested safely, with simulated data.
  11. A real world pilot where some of the components are deployed in secure environment and used with real world data.
  12. A directory of endpoints where data can be sent to, or consumed from.
Components should be able to deploy with commonly available components on multiple platforms, including open-source database servers (SQL or NoSQL), and where cloud is available, to work with readily available cloud components (including both Windows and Linux servers).

No single organization will deliver all of these components.  Instead, members of The SANER Project will produce individual components that they can make readily available and easily installed for their customers, with minimal disruption.


Wednesday, March 25, 2020

The SANER Project

I don't usually talk about "work" that isn't part of my volunteer standards activities here.  What I do in standards is public information, and I can have my own opinions about it, so that's safe. The rest of my work is really between me and my employer, and when I blog for them, I do it on their site as I've done for the past decade and a half.  But there are times to break the rules, even my own.

The SANER Project is called that because if I couldn't do something about COVID-19, I would go crazy.  At lot of my colleagues at Audacious Inquiry feel that same way.  That's why we've started a project to work on this.  There's an implementation guide, and people, and meetings.  If you are in the same boat, become one of the people, join the meetings, and help us develop the implementation guide, test it, and even pilot it.  So, how am I breaking my rules?  This isn't a Standards project.  It WILL become one, it is using standards, but first and foremost, it's about how to rapidly deploy a solution that can become the way to address our needs in the US.

I grew up outside Philadelphia during the gratefully brief Legionnaire's disease crisis, and I remember the trauma, and the daily news cycles as a child, and my a) inability to escape the constant barrage of information, and b) complete inability to do anything about it.  It must have had an impact on me, I can still recall it vividly today as we enter the same sort of barrage around COVID.

I've been thinking about what the Health IT Community could do about COVID-19 for weeks, I listened to what people were saying, and where the attention was (or wasn't). Discussions around situational awareness have been getting louder again as this crisis began (as is usual for Public Health in the ongoing approximately triennial cycle of novel fill-in-the-blank).  Six new diseases SARS, Bird Flu (H5N1), Swine Flu, MERS, Bird Flu (H7N9), and now COVID-19 (SAR-Cov-2) in the last 17 years.

Situation awareness is something I first heard about in standards circles with the introduction of what many of us called the "Bird Flu" Use Case in ANSI/HITSP.  HITSP adopted OASIS HAVE and some HL7 V2 messages in C47 Resource Utilization Component.  HL7 and OASIS worked on a Cross-Paradigm specification.  And AHRQ took up the ball with HAvBED2.  It didn't work because it involved manual entry of data for the most part. 

The big ask from Public health are very much the same now as they have been over the past decade an a half: How are hospitals doing?  Break that down, and it's Beds (by type), Equipment and Supplies, and Staff.

The challenges our healthcare providers are having now is dealing with a crisis, and not having resources to get public health the information they need because we failed to automate it when we had the time.  And if the situation keeps repeating itself the way it has over the last decade an a half, they won't be able to address it the next time either, because nothing can be done in the middle of a crisis, and real efforts to resolve it after haven't considered the real challenges of automating the process, because once the crisis is over, public health doesn't get to take center stage (if there's ever a group of under-recognized people, it's those who work in public health... it's somewhat like what I've heard others say about the military 98% waiting, 2% sheer panic).

Bed Availability is a math problem.  Most think about it as:

Bed Availability = Total Beds - Beds In Use

But that's half the story because a bed cannot be used if the people needed aren't available to staff it.  A bed is a location of care... if the staff cannot provide the care, an empty bed is not going to help.  The real math is closer to:

Available Capacity = Total Beds - Beds in Use
Usable Capacity = f(Available Capacity, Available Staff)

And f() varies by type of bed and so many other things.

And there are other challenges.  Total Beds is not a constant for a facility, it's a fungible figure, because someone can roll out overflow beds into the ER in minutes, reconfigure an OR as a temporary ICU in hours, crank up a temporary treatment center in hours, build hospitals in days, or field treatment centers, activate resources, and people and even Health IT systems (my employer developed PULSE for this -- That came out of the "Katrina" use case).

Equipment is fungible too, ventilators have been modified in real treatment (NOTE: This is off-label use, and not a recommendation from me, just a note that it's been done by experts), and there are makers building them.

But situation awareness isn't about exact numbers, it's about close enough to make plans (kind of like software project management).  And there's an incremental path to getting there, because we can track the outputs needed (Bed Availability) back through it's dependent inputs Available Capacity, Total Beds and Beds in Use, and start generating those numbers, and eventually, as we track it, we start to derive the f() based on what we learn.  But before we get to point C, we have to go through A and B first.

There's value in reporting available capacity and beds in use, even if that doesn't get to usable capacity immediately, because someone who works in this space can take what else they know about staffing shortages and compute an approximation of f() in their head.  And Beds in Use is an important signal alone. It can be used to alert public health to changing trends.

And tracking those two and applying analytics can eventually get us to a data driven value for f(), and other analytics can improve the performance of f() even further.

And yes, I know, there's International value in what we are doing, and I certainly welcome International contributions, but for now, this project is going to focus on the US; others are welcome to contribute and use the materials it produces, but for now, it's going to be hard enough to manage 56 or so jurisdictions and public health agencies in my own country, so it's going to be focused at least for now, on US efforts.  After all, there's only so many cats one can possibly herd.  Thankfully, I'm not trying to do this alone.

    Keith