FluentQuery for FHIRPath

 The SANER Project has been using FHIRPath to define situational awareness measures.  It has need to query at least FHIR Server to evaluate data for the measure, and the server to query is something that can be defined by the implementer.  Although perhaps I should say "servers" to query, because it's possible that data may be available from more than one location, as a recent commentator on our measures suggested.

For example, in some placees, laboratory data may be available from multiple sources, both the hospital lab, and a health information exchange for example.  The existence of any lab data from either would be sufficient to rule a patient into (or out of) a measurement.  We are using the resolve() method to resolve the content of these queries (and yes, I have taken into account pagination in my own implementation of resolve), which leads to measures that have somewhat unreadable measures for those not deeply versed in FHIR search.  

Here's a couple of examples:

( %Base + 'Encounter?' +
  '_include=Encounter:subject&_include=Encounter:condition&' +
  '_include=Encounter:reasonReference' +
  '&status=in-progress,finished' +
  '&date=ge' + %ReportingPeriod.start.toString() +
  '&date=lt' + %ReportingPeriod.end.toString()
).resolve().select(entry.resource)

Patient.distinct().where(
  %Base + 'Observation?_count=1' +
  '&status=registered,preliminary,final,amended,corrected' +
  '&patient=' + $this.id +
  '&date=gt' + (%ReportingPeriod.start - 14 'days').toString() +
  '&code:in=' + %Covid19Labs.url +
  '&value-concept:in=' + %PositiveResults.url
).resolve().select(entry.resource as Observation)

Sure, you can read these.  Of course you can, but what about your staff, leadership, or customers.  You'd probably have to explain that the first looks for Encounters, and their referenced subject, condition and reasonReference resources where Encounter is either in-progress or finished and the date is within the reporting period.  The second is an existence test that succeeds if there is at least Observation resource for a patient where the date of the observation is up to two weeks before the reporting period, showing a positive result on a COVID lab test.

But wouldn't something like below be MUCH easier for not just you, but also for your analysts, leadership and customers to read?

findAll('Encounter',
  including('subject','condition','reasonReference'),
  with('status').equalTo('in-progress'|'finished'),
  with('date').within(%ReportingPeriod)
).onServers(%Base)

Patient.distinct()
  .whereExists('Observation',
    for('patient', $this),
    with('status').equalTo(
     'registered' | 'preliminary' | 'final' | 'amended' | 'corrected'),
    with('date').greaterThan(%ReportingPeriod.start - 14 'days'),
    with('code').in(%CovidLabs),
    with('value-concept).in(%PositiveResults)
   ).onServers(%Base)

Of course it probably would.  The Reference Implementation of FHIRPath that Grahame developed provides support for custom functions, but unfortunately, custom functions don't have access (yet) to the left has side (the focus) of the expression.  I'm in the process of fixing that on my fork of his FHIRPathEngine code.

Once they do, here's how I see this working.  A query builder expression returns a URL that is a query being built, taking as input any part of the previously constructed query.  For the most part, these are simply specialized concatenations.

There are two functions: findAll() and whereExists() that start a query builder expression. These functions return a string containing the currently built URL.  findAll('resourceType') would simply return the string 'resourceType?', while whereExists(resourceType) would return 'resourceType?_count=1 (supporting an existence test).

The returned URL is evaluated by an onServers() function which is like resolve() except that onServers takes a list of base urls and the search is executed on each combination of search url and server (with pagination resolved).  In the case of "whereExists()", resolution is allowed to stop after finding the first matching resource on any server (although the queries might be executed in parallel).

onServers() is effectively equivalent to resolve().select(entry.resource), but there's one little bit of extra juice, because it can do this for more than one server at a time.  OnServers is the principle reason that my function set needs access to the focus, so that it can produce the cartesian product of of the URLs and the server base addresses.  And if there are multiple queries, onServers could be smart enough to send a batch query or use FHIR bulk data to get it's results.

The work above is really out of scope for SANER, although we might consider including it as an appendix for others to consider (that's probably the easiest way to address a couple of issues in the ballot wrt to scope, making sure we've got it documented, but not requiring it to be used for successful implementation).

One of the values of FluentQuery is that it provides a means for someone to write a query without it being completely depedendent on how the query URL is constructed.  The lightweight implementation can construct a URL as it goes (and that will be the first implementation that I write).  But other implementations could do some smart things like:

1. Limiting queries to what a FHIR Server supports, and handling some of the filter parameters differently.  
1. :in and :not-in
   Not every server supports code:in queries on Observation for example.  But it's a really valuable way to simplify the writing of the query.  
2. _has
   Rewriting has queries for servers that don't support them.
   Resource1?_has:Resource2:name=value can be handled as first querying for Resource2?name=value&_elements=name with a post filter that collects all the references and then performs Resource2?_id=Reference1,Reference2, et cetera.
2. Return lists that defer pagination of results until needed.

Search Functions

findAll(ResourceType [, QueryFunctionExpression]*)

The findAll function constructs a relative search URL for the specified resource type, appending the queries specified by any of the QueryFunctionExpression values separated by an &, and returns this in a string.  When executed, this query will return all results including those returned via pagination.

whereExists(ResourceType [, QueryFunctionExpression]*)

The whereExists function constructs a relative search URL for the specified resource type, adds _count=1 to it, and appends the queries specified by any of the QueryFunctionExpression values separated by an &, and returns this in a string.  This query is used for existence tests.

Query Execution Functions

onServers([reliability, ] Servers*)

The onServers function executes a search on the specified servers.  Servers is a list of fully qualified base URLs.  findAll queries resolve the data on all servers (including all pages from each server), and then selects entry.resource from all returned Bundle resources.  The whereExists query returns the first resource found after finding at least one match on any server, returning the first matching resource (and any included resources associated with it) .  Implementations are free to execute searches serially or in parallel.  

The optional reliability parameter indicates how to handle failures during a search, and can take the value of 'skip', or 'fail'.  Skip means if any server fails to respond, or responds with an error code (or exception of some sort), that the resolution should act as if a Bundle was returned with a singular OperationOutcome resource that describes the kind of error that occurred.  This ensures that queries succeed, but there may be missing data. If the value is fail, it indicates that if any server should fail to respond, the expression should throw an exception.  This allows expression writers some limited control over how to handle error conditions when a server is being queried.  Implementations should consider retrying failed queries in either case.

The following executions are equivalent:

onServers('http:example.com/server1','http://example.com/server2')

onServers('http:example.com/server1'|'http://example.com/server2')

Query Parameter Name Functions

The query parameter name functions start the first half of a query parameter to add to the query url.  They simply return the name as a string

with(Name)

The with function constructs the first half of a named query parameter.  It should be followed by a Query Parameter Value Function to construct the second half (the part containing the equals sign).

for(Name, Reference|Resource|Identifier)

The for function constructs a complete query parameter that matches a resource or identifier.

If the first parameter is a reference or resource id, the query parameter is written as Name=Reference.

If the second parameter is a resource, this is converted to a reference to that resource, and treated as above.

If the second parameter is an identifier, this is converted to a reference by identifier search, and written as Name:identifer=Identifier

including(Names)

The including function specifies what other resources should be included.  If any value in Names doesn't start with a resource type, it is prepended with the type of resource specified in _

has(Name)

has constructs the first half of a named query parameter supporting _has searches.  The value will be _has:Name.  Chained has are possible. It should be followed by a Query Parameter Value function to construct the second half (the part containing the equals sign).

Query Parameter Value Functions

Query parameter value functions produce the second half of a query parameter (the part containing the equals sign).

A value can be a primitive, Quantity, Coding, or CodeableConcept, or Reference type.

For Quantity type, value will be expressed in number|system|code form as required by Quantity parameters.  If system and code are empty, but unit is present, a Quantity value will be expressed as number||unit with no system.  If neither of system, code or unit are present, a quantity value will be expressed as number.

For Coding type, value will be expressed in system|code form as required of Token parameters.

Period parameter values work with Date, DateTime, Instant and Period data types.  A date promotes to an appropriate Period in these cases.

equalTo(Values*)

Appends =Values[1],Values[2],...,Values[n] to the query parameter.  Note that Values can repeat, or be a list, or both.  The following three expressions are equivalent:

equalTo('in-progress','finished')

equalTo('in-progress'|'finished')

equalTo('in-progress,finished')

equalToComposite(Value1, Value2)

Appends =Value1$Value2 to the query.

notEqualTo(Values*)

Appends =neValues[1],Values[2],...,Values[n] to the query parameter. Note that Values can repeat, or be a list, or both. The following three expressions are equivalent:

notEqualTo('in-progress','finished')

notEqualTo('in-progress'|'finished')

notEqualTo('in-progress,finished')

greaterThan(Value), greaterThanOrEqualTo(Value), lessThan(Value), lessThanOrEqualTo(Value), approximately(Value)

Appends =prefixValue to the query parameter, where prefix is gt, ge, lt, le, or ap appropriately.  Value must be a singular value.  

startsAfter(Period), endsBefore(Period)

If Period is any date type, promotes that to a Period first.

Appends =prefixValue to the query parameter, where prefix is sa or eb respectively, and Value is Period.start or Period.end respectively.

within(Period)

If Period is any date type, promotes that to a Period first.

Given name is the Query Parameter name, appends =gtPeriod.start&name=ltPeriod.end to the query parameter.  If end is not present (an open period at the end), then it only appends =gtPeriod.start to the query parameter (this is one of the functions that needs access to the focus)

not(token), text(token), above(token|uri), below(token|uri), in(uri), notIn(uri)

Appends :modifier=token|uri to the query parameter, where modifier is not, text, above, below, in, or not-in appropriately.