A Test Case Generator for FHIR and SUSHI (and SANER )

I've often heard the complaint of combinatorial explosion with respect to creating test cases to fully test a system.  The problem is acute.  One part of the solution is good analysis, but the other part of it is automation.

It must be my week for mini-languages, because here is another example of a mini-language, this time used for test case generation.  I think I might have caught the language virus.

TestCase Case1:
    Patient patient X 30 with (
        identifier.value = Identifier,
        identifier.system = "http://sanerproject.org/testdata/patients",
        name.given in "firstnames",
        name.family in "lastnames",
        gender in "genders",
        birthDate within '@1930-09-09' to '@2020-09-09'
    )
Values
    // This is a set of common last names, it is purposefully of prime length
    "lastnames": {
        Smith
        Johnson
        Williams
        Brown
        Jones
        Garcia
        Miller
        Davis
        Rodriguez
        Martinez
        Hernandez
        Lopez
        Gonzalez
        Wilson
        Anderson
        Thomas
        Taylor
        Moore
        Jackson
        Martin
        Lee
        Perez
        Thompson
    }
    // This is a set of first names that are gender free, also of prime length
    // and mutually prime with the set of last names.
    "firstnames": {
        Alex
        James
        Blake
        Kyle
        Drew
        Taylor
        Kennedy
        Jordan
        Parker
        Avery
        Ryan
        Brooklyn
        Cameron
        Logan
        Emerson
        Charlie
        Ezra
    }
    "genders": {
        male
        female
    }

This example says: Generate a test case (in a bundle) containing the resource "Patient" with identifier "patient" and do it 30 times.  Take the identifier.value from an autoincrementing counter.  Set the identifier.system to a fixed value.  Pull given and family names from predefined list of values, iterating over them until done.  Take gender from another list with only two codes.  Generate birth dates from a range of values.

Now, if that was all my language did, you'd not be terribly impressed (or at least I wouldn't be).

But, what if, you could generate multiple resources, and link them correctly by identifiers.  Now we are starting to get somewhere, but it still isn't all that much better than what one can already do with an excel spreadsheet (as we did manually for the first set of test data for SANER automation).

But I also need test cases where I have encounters with, and without reasonReference as variations, with with and without reasonCode values matching a certain value set, and observation and condition resources that match or don't match selection criteria.

So, what if I could specify variation within a field like this:

Patient patient1 

    /* as before */

Condition condition1 with (

   code in COVID19Diagnosis OR in NotACovid19Diagnosis,

   patient = patient1

)

Encounter encounter1 with (

   reasonReference = condition1 OR missing,

   reasonCode in COVID19Diagnsoses OR missing,

   subject = patient1

)

And what if the test case generator spit out were six different bundles where each bundle contained a patient1, condition1 and encounter1 meeting all the appropriate mixes of criteria.

Bundle1: 

condition1 with COVID19Diagnosis, 

encounter1 with reasonReference to condition1 and reasonCode in COVID19Diagnosis

Bundle2:

condition1 with COVID19Diagnosis,

encounter1 with reasonReference to condition1 and reasonCode missing

Bundle3:

condition1 with COVID19Diagnosis

encounter1 with reasonRererence missing and reasonCode in COVID19Diagnosis

Bundle4:

condition1 with NotACovid19Diagnosis

encounter1 with reasonReference to condition1 and reasonCode in COVID19Diagnosis

Bundle5:

condition1 with NotACovid19Diagnosis

encounter1 with reasonReference to condition1 and reasonCode missing

Bundle6:

condition1 with NotACovid19Diagnosis

encounter1 with reasonReference to condition1 and reasonCode in COVID19Diagnosis

OK, now we are talking about something useful.

And if patient1 can vary (non-essentially) across these encounters, and so can encounter location, we've gone a step better.

Yes, this could produce a muck-ton of data.  But, the computer did it, you didn't have to.  All you had to do was give it the correct instructions to do something useful, and it produced something that you can use.

Anyway, more on this later as I continue my experiments.   So far, this was about two days of work, and what I have to show for it is this sample output:

Instance: patient3

InstanceOf: Patient

Description: "Generate sample patients with random characteristics"

* birthDate = "1964-06-09"

* extension[0].extension[0].url = "ombCategory"

* extension[0].extension[0].valueCoding = urn:oid:2.16.840.1.113883.6.238#2054-5 "Black or African American"

* extension[0].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-race"

* extension[1].extension[0].url = "ombCategory"

* extension[1].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity"

* gender = #male

* identifier.system = "http://sanerproject.org/testdata/patients"

* identifier.value = "3"

* name.family = "Williams"

* name.given = "Drew"

* name.given[1] = "Taylor"

Instance: patient4

InstanceOf: Patient

Description: "Generate sample patients with random characteristics"

* birthDate = "1975-09-09"

* extension[0].extension[0].url = "ombCategory"

* extension[0].extension[0].valueCoding = urn:oid:2.16.840.1.113883.6.238#2076-8 "Native Hawaiian or Other Pacific Islander"

* extension[0].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-race"

* extension[1].extension[0].url = "ombCategory"

* extension[1].extension[0].valueCoding = urn:oid:2.16.840.1.113883.6.238#2135-2 "Hispanic or Latino"

* extension[1].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity"

* gender = #female

* identifier.system = "http://sanerproject.org/testdata/patients"

* identifier.value = "4"

* name.family = "Brown"

* name.given = "Kennedy"

* name.given[1] = "Jordan"

Instance: patient5

InstanceOf: Patient

Description: "Generate sample patients with random characteristics"

* birthDate = "1986-12-09"

* extension[0].extension[0].url = "ombCategory"

* extension[0].extension[0].valueCoding = urn:oid:2.16.840.1.113883.6.238#2106-3 "White"

* extension[0].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-race"

* extension[1].extension[0].url = "ombCategory"

* extension[1].extension[0].valueCoding = urn:oid:2.16.840.1.113883.6.238#2186-5 "Non Hispanic or Latino"

* extension[1].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity"

* gender = #male

* identifier.system = "http://sanerproject.org/testdata/patients"

* identifier.value = "5"

* name.family = "Jones"

* name.given = "Parker"

* name.given[1] = "Avery"

Instance: patient6

InstanceOf: Patient

Description: "Generate sample patients with random characteristics"

* birthDate = "1998-03-10"

* extension[0].extension[0].url = "ombCategory"

* extension[0].extension[0].valueCoding = http://terminology.hl7.org/CodeSystem/v3-NullFlavor#UNK "Unknown"

* extension[0].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-race"

* extension[1].extension[0].url = "ombCategory"

* extension[1].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity"

* gender = #female

* identifier.system = "http://sanerproject.org/testdata/patients"

* identifier.value = "6"

* name.family = "Garcia"

* name.given = "Ryan"

* name.given[1] = "Brooklyn"

Instance: patient7

InstanceOf: Patient

Description: "Generate sample patients with random characteristics"

* birthDate = "2009-06-09"

* extension[0].extension[0].url = "ombCategory"

* extension[0].extension[0].valueCoding = http://terminology.hl7.org/CodeSystem/v3-NullFlavor#ASKU "Asked but no answer"

* extension[0].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-race"

* extension[1].extension[0].url = "ombCategory"

* extension[1].extension[0].valueCoding = urn:oid:2.16.840.1.113883.6.238#2135-2 "Hispanic or Latino"

* extension[1].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity"

* gender = #male

* identifier.system = "http://sanerproject.org/testdata/patients"

* identifier.value = "7"

* name.family = "Miller"

* name.given = "Cameron"

* name.given[1] = "Logan"

Generating your own FHIR Narrative with XML using SUSHI

Building an implementation guide relies on a stack of transformations.  It's a good thing we know how to stack things up on top of each other these days.  But what happens when something in the stack isn't quite adequate?  How do you manipulate things to make it all work.

This example a particularly interesting challenge:  I want to be able to generate my own Narrative from the XML version of the FHIR Resource.  I won't have the XML when SUSHI is done, only after the IG Builder completes.  I can easily get the XML Version from the JSON version by running this little program:

    public static void main(String args[]) {
        IParser op = null;
        IParser ip = null;
        for (String arg: args) {
            IBaseResource r;
            File fin = new File(arg);
            File fout;
            if (arg.endsWith(".json")) {
                fout = new File(fin.getParent(), fin.getName().replace(".json", ".xml"));
                op = xp;
                ip = jp;
            } else if (arg.endsWith(".xml")) {
                fout = new File(fin.getParent(), fin.getName().replace(".xml", ".json"));
                op = jp;
                ip = xp;
            } else {
                System.err.println("Do not know how to convert " + arg);
                continue;
            }

            try (FileReader fr = new FileReader(fin); FileWriter fw = new FileWriter(fout)) {
                r = ip.parseResource(fr);
                op.encodeResourceToWriter(r, fw);
            } catch (IOException e) {
                System.err.printf("Cannot convert %s: %s\n", arg, e.getMessage());
            }
        }
    }

But then, how do I insert my narrative into the build process?  If it has to happen
post-SUSHI processing, then IG-builder won't execute it.  I'm only doing this for ONE resource (or perhaps a few), not the 97 or so that I otherwise generate using SUSHI.  It will get infrequently updated, and I don't want to give up the convenience of having the automated SUSHI run in IG Publisher do most of the work for me.  

But hey, if you can make your transform stack run in a circle the way the Crazy Russian does with Pringles, you can make it work.

Here's what I decided to do:

In the Resource that I want generated text, I include the following:

Instance: ComputableCDCPatientImpactAndHospitalCapacity
InstanceOf: PublicHealthMeasure
Title: "Computable CDC Patient Impact and Hospital Capacity"
...
* insert ComputableCDCPatientImpactAndHospitalCapacityText

Then I create a RuleSet named ComputableCDCPatientImpactAndHospitalCapacityText and modify my XSLT to generate that ruleset instead of content of the DIV.  The template to generate the ruleset looks like this:

<xsl:template match="/">
    <xsl:text>RuleSet: </xsl:text><xsl:value-of select="fhir:Measure/fhir:id/@value"/><xsl:text>Text&#xA;</xsl:text>
    <xsl:text>* text.status = #generated&#xA;</xsl:text>
    <xsl:text>* text.div = """&#xA;</xsl:text>
    <div xmlns="http://www.w3.org/1999/xhtml">
        <xsl:apply-templates select="/fhir:Measure"/>
    </div>
    <xsl:text>"""&#xA;</xsl:text>
</xsl:template> 

Finally, my process to update the generated narrative is to do the following:

1. Generate the resources using SUSHI
2. Run my XSLT to generate the Narrative
3. Rerun the IG Publisher

Basically, I'm using SUSHI to generate an input to a second run SUSHI (run by the IG Publisher).

I'm not happy with this.  There should be a way to insert this generation step somewhere else in the tool chain, and update the tool chain (perhaps with templates) so that I don't have to rely on the FHIR Generated narrative, but can rather build my own from the XML resource.

It's a bit uglier than that even, because some of the resource content (e.g., description, definition, et cetera) is actually Markdown, rather than HTML, and I had originally used markdown tags to support bulleted lists or definition lists and links in some of that content where appropriate.  I punted on that problem by simply replacing that limited markup with its (again limited) HTML equivalents since HTML markup is allowed in Markdown.

Configuring Eclipse to run SUSHI over your Fish Tank to generate FHIR Resources and Profiles

As I write more and more FHIR Shorthand, I just want it to work better

in MY development environment.  I principally use Eclipse based tools (Eclipse and Oxygen XML Editor).

1. I want a quick launcher.
2. I want easy navigation to errors.
3. I want better error messages.
4. I want syntax highlighting.

Like I sometimes tell my kids.  Now you know how it feels to want.

I did manage to create a launch configuration, and you can see the content below.  So, now I've got #1, and you can have it as well.

Save this in a text file with the .launch extension

<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<launchConfiguration type="org.eclipse.ui.externaltools.ProgramLaunchConfigurationType">

    <stringAttribute key="org.eclipse.debug.ui.ATTR_CONSOLE_ENCODING" value="UTF-8"/>

    <stringAttribute key="org.eclipse.ui.externaltools.ATTR_LOCATION"
       value="C:\Program Files\nodejs\node.exe"/>

    <stringAttribute key="org.eclipse.ui.externaltools.ATTR_TOOL_ARGUMENTS"
      value="&quot;C:\Users\{YOURUSERNAME}\AppData\Roaming\npm\node_modules\fsh-sushi\dist\app.js&quot; fsh -o ."/>

    <stringAttribute key="org.eclipse.ui.externaltools.ATTR_WORKING_DIRECTORY" value="${workspace_loc:/fhir-saner}"/>

</launchConfiguration>

Edit this file to fix the location where Node and Sushi are installed for you, and what fishtank you want it to run on, and then import it using "File | Import | Launch Configuration" into your eclipse environment.

Now, if you want to run SUSHI, you can easily do it.  I'm sure there's more that could be done to parameterize things like where node.exe is, and who the current user is, and how to run it on the default fish tank folder of the current project.  When I have time to mess with it, I may get around to it.  

I'm sharing what I've learned though, in the hopes that somebody else who knows how to make 'clipse stuff work might take up the task.

It's that time again, but what? AGAIN!?!

One of the many unwritten rules of the Ad Hoc Harley award is that you can't earn it more than once.  I'm not going to change that now, but one of the previous recipients has done it again.  And for that, I'm awarding the rare (and heretofore never previous awarded) serious chrome upgrade for this contributors latest piece of work.

If you haven't heard about SUSHI (and no, I'm not talking about what I eat when I travel to standards meetings), and you build FHIR implementation guides, or implement FHIR resources, you NEED to learn about it, and the FHIR Shorthand language for creating FHIR Implementation Guide Resources.

Here's why:  Starting at 10:30 on Saturday, and working about 16 hours straight, I was able to do with SUSHI what in the past would have taken me 5 to 10 days of implementation effort.  As a result, I managed to put together an IHE Profile implementation guide from scratch in record time.

The ability to create implementation guides, value sets, capability statements, coding systems, any sample resource you might ever want in this simple language is simply fantastic.

So, here goes:

This certifies that

Chris Moesel of MITRE

has hereby been recognized AGAIN for awesome work with the rarely awarded and highly coveted Serious Chrome Upgrade.

And furthermore, no man is an island, and no open source project like this can possibly be done alone. So I get to repeat the Bike Week award (originally awarded to the Redoc Dozen).

Chris is working with a team of developers at MITRE that includes:

ngfreiter (Nicolas Freiter)

mint-thompson (Mint Thompson)

kjmahalingam (Dylan Mahalingam)

jafeltra (Julia Afeltra)

Let's give them all, and MITRE a round of applause.

Ad Hoc Bike Week Award