A Test Case Generator for FHIR and SUSHI (and SANER )

I've often heard the complaint of combinatorial explosion with respect to creating test cases to fully test a system.  The problem is acute.  One part of the solution is good analysis, but the other part of it is automation.

It must be my week for mini-languages, because here is another example of a mini-language, this time used for test case generation.  I think I might have caught the language virus.

TestCase Case1:
    Patient patient X 30 with (
        identifier.value = Identifier,
        identifier.system = "http://sanerproject.org/testdata/patients",
        name.given in "firstnames",
        name.family in "lastnames",
        gender in "genders",
        birthDate within '@1930-09-09' to '@2020-09-09'
    )
Values
    // This is a set of common last names, it is purposefully of prime length
    "lastnames": {
        Smith
        Johnson
        Williams
        Brown
        Jones
        Garcia
        Miller
        Davis
        Rodriguez
        Martinez
        Hernandez
        Lopez
        Gonzalez
        Wilson
        Anderson
        Thomas
        Taylor
        Moore
        Jackson
        Martin
        Lee
        Perez
        Thompson
    }
    // This is a set of first names that are gender free, also of prime length
    // and mutually prime with the set of last names.
    "firstnames": {
        Alex
        James
        Blake
        Kyle
        Drew
        Taylor
        Kennedy
        Jordan
        Parker
        Avery
        Ryan
        Brooklyn
        Cameron
        Logan
        Emerson
        Charlie
        Ezra
    }
    "genders": {
        male
        female
    }

This example says: Generate a test case (in a bundle) containing the resource "Patient" with identifier "patient" and do it 30 times.  Take the identifier.value from an autoincrementing counter.  Set the identifier.system to a fixed value.  Pull given and family names from predefined list of values, iterating over them until done.  Take gender from another list with only two codes.  Generate birth dates from a range of values.

Now, if that was all my language did, you'd not be terribly impressed (or at least I wouldn't be).

But, what if, you could generate multiple resources, and link them correctly by identifiers.  Now we are starting to get somewhere, but it still isn't all that much better than what one can already do with an excel spreadsheet (as we did manually for the first set of test data for SANER automation).

But I also need test cases where I have encounters with, and without reasonReference as variations, with with and without reasonCode values matching a certain value set, and observation and condition resources that match or don't match selection criteria.

So, what if I could specify variation within a field like this:

Patient patient1 

    /* as before */

Condition condition1 with (

   code in COVID19Diagnosis OR in NotACovid19Diagnosis,

   patient = patient1

)

Encounter encounter1 with (

   reasonReference = condition1 OR missing,

   reasonCode in COVID19Diagnsoses OR missing,

   subject = patient1

)

And what if the test case generator spit out were six different bundles where each bundle contained a patient1, condition1 and encounter1 meeting all the appropriate mixes of criteria.

Bundle1: 

condition1 with COVID19Diagnosis, 

encounter1 with reasonReference to condition1 and reasonCode in COVID19Diagnosis

Bundle2:

condition1 with COVID19Diagnosis,

encounter1 with reasonReference to condition1 and reasonCode missing

Bundle3:

condition1 with COVID19Diagnosis

encounter1 with reasonRererence missing and reasonCode in COVID19Diagnosis

Bundle4:

condition1 with NotACovid19Diagnosis

encounter1 with reasonReference to condition1 and reasonCode in COVID19Diagnosis

Bundle5:

condition1 with NotACovid19Diagnosis

encounter1 with reasonReference to condition1 and reasonCode missing

Bundle6:

condition1 with NotACovid19Diagnosis

encounter1 with reasonReference to condition1 and reasonCode in COVID19Diagnosis

OK, now we are talking about something useful.

And if patient1 can vary (non-essentially) across these encounters, and so can encounter location, we've gone a step better.

Yes, this could produce a muck-ton of data.  But, the computer did it, you didn't have to.  All you had to do was give it the correct instructions to do something useful, and it produced something that you can use.

Anyway, more on this later as I continue my experiments.   So far, this was about two days of work, and what I have to show for it is this sample output:

Instance: patient3

InstanceOf: Patient

Description: "Generate sample patients with random characteristics"

* birthDate = "1964-06-09"

* extension[0].extension[0].url = "ombCategory"

* extension[0].extension[0].valueCoding = urn:oid:2.16.840.1.113883.6.238#2054-5 "Black or African American"

* extension[0].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-race"

* extension[1].extension[0].url = "ombCategory"

* extension[1].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity"

* gender = #male

* identifier.system = "http://sanerproject.org/testdata/patients"

* identifier.value = "3"

* name.family = "Williams"

* name.given = "Drew"

* name.given[1] = "Taylor"

Instance: patient4

InstanceOf: Patient

Description: "Generate sample patients with random characteristics"

* birthDate = "1975-09-09"

* extension[0].extension[0].url = "ombCategory"

* extension[0].extension[0].valueCoding = urn:oid:2.16.840.1.113883.6.238#2076-8 "Native Hawaiian or Other Pacific Islander"

* extension[0].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-race"

* extension[1].extension[0].url = "ombCategory"

* extension[1].extension[0].valueCoding = urn:oid:2.16.840.1.113883.6.238#2135-2 "Hispanic or Latino"

* extension[1].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity"

* gender = #female

* identifier.system = "http://sanerproject.org/testdata/patients"

* identifier.value = "4"

* name.family = "Brown"

* name.given = "Kennedy"

* name.given[1] = "Jordan"

Instance: patient5

InstanceOf: Patient

Description: "Generate sample patients with random characteristics"

* birthDate = "1986-12-09"

* extension[0].extension[0].url = "ombCategory"

* extension[0].extension[0].valueCoding = urn:oid:2.16.840.1.113883.6.238#2106-3 "White"

* extension[0].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-race"

* extension[1].extension[0].url = "ombCategory"

* extension[1].extension[0].valueCoding = urn:oid:2.16.840.1.113883.6.238#2186-5 "Non Hispanic or Latino"

* extension[1].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity"

* gender = #male

* identifier.system = "http://sanerproject.org/testdata/patients"

* identifier.value = "5"

* name.family = "Jones"

* name.given = "Parker"

* name.given[1] = "Avery"

Instance: patient6

InstanceOf: Patient

Description: "Generate sample patients with random characteristics"

* birthDate = "1998-03-10"

* extension[0].extension[0].url = "ombCategory"

* extension[0].extension[0].valueCoding = http://terminology.hl7.org/CodeSystem/v3-NullFlavor#UNK "Unknown"

* extension[0].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-race"

* extension[1].extension[0].url = "ombCategory"

* extension[1].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity"

* gender = #female

* identifier.system = "http://sanerproject.org/testdata/patients"

* identifier.value = "6"

* name.family = "Garcia"

* name.given = "Ryan"

* name.given[1] = "Brooklyn"

Instance: patient7

InstanceOf: Patient

Description: "Generate sample patients with random characteristics"

* birthDate = "2009-06-09"

* extension[0].extension[0].url = "ombCategory"

* extension[0].extension[0].valueCoding = http://terminology.hl7.org/CodeSystem/v3-NullFlavor#ASKU "Asked but no answer"

* extension[0].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-race"

* extension[1].extension[0].url = "ombCategory"

* extension[1].extension[0].valueCoding = urn:oid:2.16.840.1.113883.6.238#2135-2 "Hispanic or Latino"

* extension[1].url = "http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity"

* gender = #male

* identifier.system = "http://sanerproject.org/testdata/patients"

* identifier.value = "7"

* name.family = "Miller"

* name.given = "Cameron"

* name.given[1] = "Logan"