Thursday, November 7, 2019

ACID, BASE, HATEOAS and Pagination in HAPI on FHIR

Let's start with some definitions:
Two common transaction models are ACID and BASE.  I've talked about these before with regard to EHR and HIE solutions.
HATEOAS is the RESTful model of interaction in a cloud based, BASE oriented world.

Which brings us to Pagination, the ability to look at the set of resources matching a particular query.

Pagination is a feature in a FHIR Server in which a query can specify a page size to indicate the maximum number of matches to return.

HAPI on FHIR has native support for pagination, but most implementation models come closer to an ACID model of the world.  Essential, the implementation memorizes the list of matching resource ids, and handle pagination from there, retrieving data as needed from the list associated with the query.  Thus, pages represent content at a given snapshot in time (which might be updated when results get stale or removed from any sort of cache).  In this model, you won't get unrepeatable reads or other issues wrt to transaction isolation, BUT, you could be looking at data that is not consistent with the current system state.

One of the points of pagination is to reduce server load.  Just getting the IDs and retrieving only what you need for a single page is a good start (and reduces network overhead), but it doesn't reduce the computational load to perform the actual query (and yes, I know, properly implemented, you can reduce it).

I use a more BASE oriented perspective, which is where results are retrieved a page at a time, and NEXT and PREV links are computed based on Resource.id.  Since the id values I use have the property that they are monotonically increasing from creation time, I can return resources in reverse order from creation time.  I could (and have) used last update time as the sort critiera as well.  Any sort key (including a composite key) that uniquely identifies a resources works in this model.

My NEXT and PREV links simply provide the sort key of the first and last resources in the current response in special parameters (I use _a and _b for after and before respectively).  To make things a little bit more complex, results are generally returned from most recent to least recent order (descending order), so I have to reverse my normal thought processes wrt order, but it works with either ascending or descending orders.

In the model I use, if something changes is the result set between one query and the next, you'll see the changes (which you won't always depending on how others implement the HAPI Pagination provider).  I could probably figure out how to handle this better and refactor my work to align it with the HAPI Pagination stuff, but what I've got is working.

Well, almost.  There is this issue that I've managed to workaround, so yeah it works, but I had to hack around a bug.