Tuesday, September 6, 2011

For the Next HL7 Ballot

Balloting HL7 Documents is always a challenge.  The CDA Consolidation Guide is 356 pages.  I made over 125 comments.  The easiest way to make comments for me is to make them inline in the document.  The easiest way to keep track of comments and manage them is to use a spreadsheet.  Transferring data from the document to the spreadsheet is really tedious and time consuming.

When I started on the CDA Consolidation guide this morning, I knew I was in trouble (I had intended to start over the weekend -- but it was a holiday weekend, and my family won out).  So this morning, I spend a few precious hours writing a Word Macro (the Lord only knows what I would have done had I needed to deal with PDF).  It paid off, and I was able to do my review quickly using Word, and then gather the comments in a nearly decent table format, which I could transfer to the spreadsheet.  I could (and did) submit my marked up word document to HL7 (to ensure my comments were registered).  But I knew that if I hadn't submitted them in the spreadsheet format, there would be hell for me to pay later.  So, I built a tool, and used it, and submitted both.

PDF also has tools for revision marking, but unfortunately, I don't know Acrobat well enough to even know if I could gather the revision marks into something nearly as useful.  I can also use this Word Macro to comment on IHE documents (since I can always find the Word versions of the PDFs).  You can find the macro in the window below, or download it from Google Docs.  It may not help you this ballot cycle, but it could be very useful in future ballot cycles for you.  I expect to get a lot of mileage out of it.



Code fixes are welcome.  There are still a couple of things that don't work:

  1. It doesn't grab the heading number correctly, I had to fix that manually.  It worked during design-time testing, but not when I really needed it.
  2. It doesn't grab the page number.  This too worked at one point, but I don't know why it isn't anymore.  I ignored this problem for now.
  3. Comments spanning multiple lines should replace paragraph breaks (in both the commment and the text it spans) with a special character so that the table rows don't get messed up.  I fixed that with global search and replace manually.
  4. Some things are slow and could use speeding up.  I used very simple algorithms.  Linear search, bubble sort, et cetera.  Simple was good to get it working, but when I had more than 100 comments, I found I had to make at least two optimizations to the linear search algorithm to make it run acceptably (less than 10 minutes).
One of the things we need to look at as we develop tools to create ballot content is to also ensure that we have good tools to support commenting on it.  If we have great tools to generate content, but lousy support for comments, we'll have a problem generating good content, because it will be hard for people to comment on issues.  This is one thing that some folks in the OpenEHR space understand really well.  Some folks over there have created some excellent tools for collaborative review of archetypes.  It's something that HL7 can learn from.

2 comments:

  1. This is a great macro and idea. I am so glad you started on this effort, so much so that I decided to expand on your efforts. It includes - per some of your identified needed improvements grabbing the heading numbers, page numbers, does quick sort instead of bubble sort, and support comments for multiple paragraphs), as well as a few of my own additions - macro in a separate xls file that can load a commented word document (so you do not have to paste the macro into new documents), imports comments directly into a blank HL7 ballot response xls file with push button ease (no copy paste needed), and populates the ballot WG column. I would like to commit this to the HL7 public domain once I have a sizable comment file to test its performance. Once again, thank you Keith.

    ReplyDelete