Navigating Standards and Regulation

Compare four different use cases with me:

From an IHE IUA (Internet User Authentication a.k.a. OAuth) profile discussion: to authorize the patient's application to access health information.

From the ABBI Charter: Allowing a third party application of the consumer’s choosing to privately and securely access personal health data on demand.

From the Omnibus Final Rule:
if requested by an individual, a covered entity must transmit the copy of protected health information directly to another person designated by the individual. In contrast to other requests under § 164.524, when an individual directs the covered entity to send the copy of protected health information to another designated person, the request must be made in writing, signed by the individual, and clearly identify the designated person and where to send the copy of the protected health information.

From the HITECH Act (see subsection (e) at the top of page 7748):
the individual shall have a right to obtain from such covered entity a copy of such information in an electronic format and, if the individual chooses, to direct the covered entity to transmit such copy directly to an entity or person designated by the individual, provided that any such choice is clear, conspicuous, and specific;

In each of the above: individual, patient, consumer also includes in some way their authorized representative, although that may be variously defined (e.g., parent, guardian, holder of a power of attorney, et cetera) and implemented, and may be stated or implied in the use case.

In each use case, we talk about authorizing an application, in others, an entity, and in others, designated person as the recipient of the electronic information.

In the IHE use case I described above (which is really just one example of many similar use cases), it mentions the "patient's application", as if we are discussing an application the patient owns (or licenses).  But in the IHE profile, that could also be an application developed by a third party that the consumer authorizes.  I can see uses where a patient could use this same capability to authorize an application devised by the SSA to access data on a to enable adjudication of disability for example.

In the Omnibus Final Rule, it clearly states: the request must be made in writing, signed by the individual, and clearly identify the designated person 
However, I would note, that writing does not mean "printed on paper", signed by the individual can be represented by some form of electronic signature, and clearly identifying a designated person need not mean "by name", but could also be by role (e.g., privacy officer, licensed healthcare provider, et cetera).

By paying attention, one can navigate all of these use cases to produce one common solution that meets the needs for all of them.  But it truly isn't easy.  Welcome to my world.

OAuth 2.0 Headaches for ABBI

Now I understand why Eran Hammer wrote OAuth 2.0 and the Road to Hell.  I've been playing around with how to address client registration issues.  Right now, I have the following open in my browser:

1. JSON Web Signature (JWS) [IETF Draft 4]
2. JSON Web Token (JWT) [IETF Draft 6]
3. JSON Web Token (JWT) Bearer Token Profiles for OAuth 2.0 [IETF Draft 4]
4. JSON Web Algorithms (JWA) [IETF Draft 8]
5. Assertion Framework for OAuth 2.0 [IETF Draft 8]
6. The OAuth 2.0 Authorization Framework [RFC 6749]

Does this sound vaguely familiar?  Recall these seven: MU Stage 1, HITSP C32, C80, C83, IHE XPHR, HL7 CCD, HL7 CDA

Here's the break down:
OAuth 2.0 is being used to support capturing the authorization of the patient [or authorized representative] to allow a client application instance to access PHI on the patient's behalf.   The Assertion Framework for OAuth 2.0 specifies the parameters that tell how the client assertion of its identity is being exchanged in the OAuth 2.0 protocol.  JWT Bearer token profiles describes how a JWT bearer token is constructed for OAuth 2.0 for client identity and other uses.  JWS explains how a signed JWT is exchanged.  JWA indicates the algorithms that are used in JWS. And two of these don't yet agree on whether the key for who the principal is "prn" or "sub", or perhaps I don't understand the distinctions being made.


You'll note that NOT listed above is the OAuth Dynamic Client Registration Protocol [IETF Draft 04].  That's because I traded one problem [Dynamic Registration], for five others [to support a signed assertion of client identity] .  I'm not sure it was a good trade, but I'm going to plug away at this anyway.


I think we need to develop the Joined JSON Profile for OAuth 2.0 Web Authorization, which I'm going to call J-JWA, as a pun on C-CDA.  The biggest challenge for me in all of this is that with the exception of the OAuth 2.0 Framework, the rest of these are in draft form.  However, maybe we can address it in J-JWA, and we'll never have to go down that path we've been through before.

-- Keith

Slap me

While I've been at the HL7WGM, I've also been spending time thinking about ABBI and OAuth 2.0 and dynamic client registration.  I still don't have the details and capabilities of dynamic client registration fixed firmly in my head.  Part of the reason I'm working through this so hard is due to a threat to the ABBI Ecosystem that doesn't yet seem to be mitigated.  Let me start with some assumptions:

ABBI PULL will use OAuth 2.0 for authentication.  It will need dynamic client registration to support authentication of client applications.  The reason that authentication of clients is necessary is that we want to establish trust in the ABBI ecosystem.  That means that applications need to be trustable, or not trustable, and that trust once granted could subsequently be revoked.

To identify a client application installed on a user's system, it must be provisioned with credentials so that an authorizer can identify that installation, and grant trust (or not) based on both the user authorization, and the client application identity.  The client application identity could be specific to an installation, or cover all installations of the same client, or an instance's identity could be linked to  an "application class".  Revoking trust on an instance by instance basis is tedious, and means that having seen one threat, you address it, but that doesn't help you with the next one.  In the linked instance to class model, trust could be revoked from a specific instance, or from all instances of the same client, depending on what happened in the ecosystem.

The threat to an application is this:  If an instance of an application can be impersonated, then an attacker can deploy numerous instances of a rogue application that impersonates a real application (e.g., through a bot-net).  These rogue applications behave badly in a way that is detectable by the authorizer.  This results initially in a revocation of trust, first from each badly behaving instance, but eventually to any instance reporting to be that application.

The attacker in this case has successfully denied users of that application from the services it provides in the eco-system. The reputation of the organization supplying the application also suffers.

Denial of service might seem to be a pretty mimimal attack.  No PHI was lost, damaged or stolen.  Service will be restored eventually.  This isn't about not being able to access Twitter though. This is healthcare data.  People have died because they didn't have their medication list readily available when they needed care.  Am I over-responding?  I don't know.

The problem is that in the OAuth threat model, application code isn't secure.  If you've ever used a Java decompiler, you know what I mean.  I've seen C/C++ decompilers that work with x86 architectures that are also pretty good.  So the code isn't safe, nor is the data in your application, so it cannot be safely pre-configured with secrets. Even assuming that an attacker has limited resources, modern tools can reconstruct the application code readily enough to enable serious hacking (been there, done that, all for legal purposes).

Post-configuration of the application with a secret is possible (because the application can store the secret in protected storage), but there is a chicken and egg problem.  Because the application code isn't secure, the protocol by which the installed application obtains its post-configured secret is known, and an attacker can use then that protocol to obtain an installed instance of an application's post-configured secret.  Having that, they can now impersonate the installed instance.  So anyone could masquerade as an application for which they had the code for, and if they can do it once, they can do it a thousand times (or more).

The responsibilities of members of the ABBI ecosystem is not equally shared.  Applications (running on a device) need to protect a single (or perhaps a few) end-user's data and their own secrets in an installation.  Data holders must protect the data of thousands or even millions of patients.  Authorizers enable the trust in the ecosystem, and have one of the highest levels of responsibility, because if they break down, the ecosystem breaks down.

And what I cannot figure out is how to enable an authorizer to protect an application from being impersonated.

This is not my area of expertise.  I know enough about security (which is quite a bit actually) to know how not to get my face slapped, and when I need expert assistance.  This is one of those cases.  Yet, because I understand more of OAuth than many others (there are a few exceptions, including Josh M, whose been really helpful), I'm putting my face right out front.  Slap away.

What-duhl?

WADL stands for Web Application Description Language.  It is a member submission to the W3C by Sun Microsystems (now Oracle), written by Marc Hadley (now at MITRE).  WADL is to REST as WSDL is to SOAP, which interestingly enough, makes it both a useful documentation and code generation tool for RESTful web service development, and anathema to the RESTful in-crowd (of which, I'm apparently not a member).  I'd also note that you can use WSDL for the same purpose (which is apparently even worse).

WADL isn't a standard, but it was certainly meant to fill a gap in standards for RESTful web services.  I've been playing around with WADL a bit to define and document the ABBI protocol.  What I found useful about WADL is that it makes me think about (and document) all the necessary  aspects of services that an implementer needs.  The advantages of using WADL are pretty significant.  I can get a lot of documentation out of WADL that would be difficult to create in the same way using a word processor.

There are several tools available that will take a WADL description and turn it into documentation of a RESTful API, others that will turn it into code, and yet others that will generate WADL from code.  All-in-all, useful stuff when you have a small team (as I do).  Sure, I don't need any tools to do this by hand, but why not use them if they make my job easier.  One of the things I like about WADL is that it helps me to order my thinking, and to make it easier to understand the API.  I can still read the XML and understand what it is doing.

One of my experiments in playing with WADL was to restructure the OAuth 2.0 RFC in WADL form, since we are also talking about using OAuth 2.0 for ABBI.  OAuth 2.0 doesn't define the resource URLs, it just defines what they need to do, which makes them a <resource_type> in WADL instead of a <resource> proper.  The tool I was using to generate documentation didn't support <resource_type> so I fixed it to do so (and made some other tweaks to it).

I haven't finished documenting either the ABBI API or the OAuth 2.0 specification in WADL, but the results are promising enough that I've posted them both to my prototype implementation site.  You can find them at the links above.

A Four-legged OAuth for ABBI

I've been thinking about how to address the million registration problem that I identified last week (actually, it was Josh Mandel who first pointed it out).  I think I have a solution to the problem, and it revolves around the way the the consumer key is exchanged. 

In Twitter, the same client application on different consumers’ systems uses the same key and secret, to connect to a singular data holder (twitter.com).  In ABBI, an application will want to use the same key, yet different secrets on for each different data holder it connects to.  The reason to use the same key is so that the client application can be consistently identified in the same way across multiple data holders.  It also provides a way to avoid collisions between client applications.  The reason to use a different secret with each data holder is to ensure that no data holder's secret can be used with another data holder.  This prevents someone from pretending to be a data holder in order to obtain all of an application's secrets.

The consumer key I propose to use is a URL identifying the application’s web page.  It’s not unreasonable to assume that a commercial application will have its own web site, and while a little bit more challenging for the typical hacker, not a huge hurdle.  In fact, I run a secure web site for free.  It’s also not too hard to believe that someone would willingly host pages for “garage-apps” (things a developer like me would use to manage their own data).

The reason to use a different secret is so that the secret shared between a client application and a data holder can only be used with that data holder.  This prevents a rogue from creating a system pretending to be a data holder in order to gain access to a client application’s secret.  Surely they could obtain access to a secret, but it would do them no good with any other data holder, because the client application would just use a different secret for the other data holder.

It is important that the secrets used by the client application with the same data holder always be the same regardless of which consumer system is accessing that data holder, because most (if not all) OAuth implementations expect each key to be associated with one secret.   Centralizing control of secrets isn’t something that you can expect the client application itself to manager.  In fact, a new secret would have to be generated for every new data holder that appears in the ecosystem.  This is another reason why the client application needs to be associated with a web site, because that becomes the single point of secret distribution for that client application.

Here is the workflow I’m presuming:

1)      Client Application MyABBI is installed on my personal device.

2)      During Device Configuration, the application is pointed to MyDataHolder.org as being one of the sources of data it needs to query.

3)      Client Application MyABBI connects to its home web-server (MyAbbiApps.com/MyABBI), and asks for the secret key to use with MyDataHolder.org, as it has never connected to that Data Holder before.  We don’t need to say much about how the client application communicates to the web site, nor do we need to say how the client application authenticates itself to that site, since both are under control of the client application developer.  However, we do need to say that the communication must occur over a secure channel, and that the Client Application must authenticate itself.  In fact, the client application could use the first step of the token request workflow to authenticate itself.

4)      The Client Application Web Site for MyABBI responds with a secret key to MyABBI on my personal device.  If the client application used the first step of the token request workflow, the response could be the same as how a server responds with a request token.  During this stage, one of two things happens:

a)      https://MyAbbiApps.com/ABBI has never encountered https://MyDataHolder.org before.  In this case, the site makes a determination whether to trust MyDataHolder.org or not.  If it chooses to trust MyDataHolder.org, The website behind MyABBIAps.com/ABBI creates and stores a new secret associated with that data holder.b)      https://MyAbbiApps.com/ABBI has encountered https://MyDataHolder.org.  If it is trusted, the web site returns the secret associated with that data holder, and it isn’t trusted it returns some sort error response.

5)      Having obtained a consumer_secret, the MyABBI can now begin the Authorization Workflow with https://MyDataHolder.org.  The first step is for the client application to obtain a request token from the Data Holder.

6)      About half way through the process of trying to verify the request just made in the previous step, the https://MyDataHolder.org will now need to obtain the client secret associated with client application.  If it already has that information, it could just reuse what it knows, since it usually will not have changed. But if it doesn’t it needs to follow these steps:

a)      Identify itself as https://MyDataHolder.org to https://MyAbbiApps.com/ABBI, and indicate that it is requesting the secret for the client application that identified itself to the Data Holder as being supported by that site.b)      Reject the OAuth Request Token request with an error code.

7)      The https://MyABBIAps.com/ABBI website will accept a request for a secret from a Data Holder.  If the data holder trusts the client application’s website certificate (at https://MyDataHolder.org), communication continues, otherwise it stops at this point, and the Authorization workflow fails.

8)      That request must contain the base URL for the Data Holder’s ABBI API (e.g., https://MyDataHolder.org/ABBI/api) and also contains a nonce associated with this request for the client application’s key.

9)      The Client Application’s website (https://MyAbbiApps.com/ABBI) will send a request the “Secret Exchange” end-point for the Data Holder (https://MyDataHolder.org).  That request will contain: The sending website URL (https://MyAbbiApps.com/ABBI), the nonce given in step 8, and the client applications secret needed to access resources.

10)   The Data Holder will store the client secret associated with the client key if it recognizes the nonce, and return 200 OK.  If it doesn’t recognize the nonce, it sends back an error message.

11)   If https://MyAbbiApps.com gets back an error message, indicating that MyDataHolder.org didn’t recognize the nonce associated with the client secret, it should invalidate that secret.

12)   The client application will restart the Authorization workflow with the data holder, and this time, it will succeed.

There are a couple of key points here that I’d like to explain further.  I’m assuming TLS without client authentication, because that is just about how every consumer facing web site works (even those with patient portals).  Making either the data holder or the client application’s web site use TLS with client authentication would simplify the steps here, and enable them to establish mutual trust.  However, it would also make MyDataHolder.org have to provide a public certificate, which the web-site developer may not have access to, or vice-versa.  I’d be very challenged to get access to the certificate securing https://abbi-motorcycleguy.rhcloud.com. It’s not mine.  It belongs to https://rhcloud.com and so I can’t use it to verify a TLS connection as a client endpoint, it can only be used to verify the identity of the web server.

Without mutual authentication, the challenge in step 7 is when MyAbbiApp.com gets a request purporting to be from MyDataHolder.org asking for the secret associated with MyAbbiApp.com.  It has no way to determine that the requester is indeed MyDataHolder.org without without client authentication. 

To resolve that problem, step 8 and 9 come into play.  Step 8 ensures that there is a way to synchronize the request with the response that comes asynchronously in step 9.

In Step 9, DataHolder.org finally gets the secret it asked for at step 7. 

Note that we haven’t delayed the authorization request in step 6, we simply rejected it, expecting the client to retry in step 12.  It is certainly feasible to delay the authorization step, but synchronizing these two separate threads of activity can be challenging, and isn’t necessary.  It’s just necessary to get the client application to try again.  With a sufficient time delay (e.g., one minute), the client application will be able to complete the authentication workflow just fine most of the time.

Step 10 and 11 ensures that data holders respond appropriately to requests to send them a secret, and the client web site revokes a secret it handed out that wasn’t accepted by a Data Holder.

The applications running behind MyAbbiApps.com and MyDataHolder.org need establish no mutual trust relationship initially.  It is only when they become aware of each other that each can separately make a trust determination about the other based upon policies.  MyDataHolder.org could refuse to trust a site whose certificate it didn’t like.  Similarly, MyAbbiApps.com could refuse to trust a Data Holder it didn’t like.  It’s completely up to them to configure the policy.  In fact, the trust relationships need not even be based on the certifications, but could be based on the site URL.  Blacklists could be used by either site as well to reject requests.

Now that I’ve worked out the Authorization Workflow, what about the normal API Authorization case?  It shouldn’t be necessary for the data holder to get the consumer secret again for MyAbbiApps.com/ABBI, but if it were, it could simply start at step 6 with the request to obtain the secret.  In this way, applications don’t even need to maintain any sort of persistent record of the client secret, because they have a way to obtain it again later.

What I’ve just described is what I call 4-legged OAuth.  The three actors in 3-legged OAuth are

1.       Consumer (End User)

2.       Client Application

3.       Server (Data Holder)

We’ve just added the Client Registrar.

Thousand of Providers and Apps for ABBI

The vision for ABBI is to enable thousands of providers to allow patients to automatically download data, possibly using thousands of various health applications.  The challenge this creates is that for the eco-system to work, we need to ensure that 1000 apps X 1000 providers doesn't equal 1,000,000 application registrations for every app to work with every provider.  I was reminded of this by a comment made by Josh Mandel yesterday on another of my ABBI posts.

I've been prototyping my efforts using OAuth 1.0, and have been following Twitter's pattern.  With Twitter, you have to register your app, and get a client key and secret that your app will use later in the Authorization Workflow.  But providers don't want to be in the credentialing business at all, and many would love to off-load that responsibility even for patients to others, so why would they now want to credential applications as well.

In the case of Twitter, it is the singular provider of the Twitter API, yet what we want for ABBI is to have numerous providers of the ABBI API.   We don't want to have each of those applications to have to register with all of the possible data holders, because then the applications would have to have foreknowledge of all data holders.  In fact, based on the consent model, there should not need to be any sort of business relationship between the application developers and the data holders.  But we do want to make sure (from the patient's perspective) that these applications aren't just some rogue actor trying to obtain access to patient information.

I looked to see what the Rhex Project had done, considering they were implementing OAuth 2.0, but no luck.  Page 11 of the Rhex OAuth 2.0 Profile (.doc) states:

Confidential Clients register a JSON Web Key (JWK) with a trusted Authorization Server prior to conducting health exchange transactions.  

So even they require a registration step.

There are a couple of ways to avoid registration.

Don't Use oauth_consumer_key

OAuth 1.0 doesn't require the oauth_consumer_key to be present in the Authorization workflow or in API calls, nor is it required to have a value.  The whole idea that the client application has to register could be ignored.  But, one advantage that oauth_consumer_key and oauth_consumer_secret support is the idea that activities performed by applications on behalf of a user can be traced back to a specific application.  This is rather beneficial.  If my data was downloaded, I'd really like to know which app did it when I check the logs.

I'd like to avoid dropping the oauth_consumer_key if possible.  There's simply no way to convey the client application identity if you do that.

Negotiate oauth_consumer_key without a Registration

OAuth 1.0 leaves completely unspecified the means by which the consumer_key and consumer_secret values are exchanged with the client application. This is by design.  With Twitter, there's a developer's API page you use, and you request keys via the web.  But there could be any number of other mechanisms by which client keys and secrets are exchanged, some of which could be done without a registration step.

Key features of the client key and secret that need to be preserved:

1. The key needs to uniquely identify the client application, and be distinct from any other client application.
2. The secret needs to be something that that only the server (data holder) and the client application could have knowledge of.

I have a solution in mind that involves consumer key and secret exchanges with the client application's web-site.  There are several challenges that have to be surmounted in order for this to work, but I'm going to spend some time thinking on this.

OAuth Enabling a Web Application

While most of what I post is of interest to just a Healthcare crowd, this particular post is of general interest, because the code I've put together should work with any kind of web site.


One of the discussions in the ABBI pull workgroup is about how to secure access to the API.  We'd discussed using OAuth, but there are some concerns that it is too difficult to implement on an existing website.  Having developed both client and server side RFC 5849 implementations (based on OAuth 1.0a), I decided to see what it would actually take.  It turns out to be about 2000 lines of code and about a week of my time.  Having done so, it should only take an implementer using the tools I've produced a day.

To do this exercise, I needed to refactor my OAuth Provider implementation.  There are two parts of OAuth.  The first part is the Authorization Workflow, which involves a sequence of exchanges to obtain a credential used to access OAuth protected APIs.  The second part is Request Validation Process used to verify authorization credentials used to access various pages.  My original implementation was complicated because various parts of the Authorization Workflow also go through Request Validation Process.

What follows below is a description of the code I put together that allows you to OAuth enable a web site.  It is a fairly long post.  If you want to cut to the chase, I won't be offended.

On a side note, although everything done here is Java based, the same kind of thing could be done for other kinds of sites.  You should also be able to use the OAuthFilter and OAuthServlet in conjunction with IIS driven web sites that run inside of a Tomcat web server (or other JSP 2.3/Servlet 1.2 compliant container).

Authorization Workflow and Request Validation

The image below shows both the Authorization Workflow and use of OAuth Protected API calls.  As you can see, validating that a request is authorized by an OAuth token appears in both places.

What I wound up doing in my refactoring was:

1. Creating an Authentication Filter to Validate Requests.  The OAuthFilter validates a request.  On failure, it returns a 401 Unauthorized error to the User Agent.  On success, it passes a wrapped request onto the web application that provides access to the authenticated user (the application), and the authorizing user, via the getUserPrincipal, getRemoteUser and isUserInRole API's of HttpServletRequestWrapper (and HttpServletRequest). 
2. Creating a Servlet that handles the Data Holder's side of the Authorization Workflow (the Data Holder is the Server in 
3. Creating a demonstration JSP Page to handle the "Authorization".

OAuthFilter

The OAuthFilter class is fairly simple.  It relies on two pluggable components, an OAuthProvider and a Repository.  The OAuthProvider class implements the verification and much of the Authorization Workflow. It delegates to the Repository to get access to tokens, users and applications.  These objects are structured according to the model shown below (it was the lack of this model that led to the complexity of my original Provider implementation).

During initialization, it constructs instances of specified OAuthProvider and Repository classes from the Filter initialization parameters, using defaults if they haven't been specified.

The filter protects all API pages, the OAuthServlet, and the Authorization page.  The first thing it does it put the OAuthProvider class into the Request context under the OAuth.Provider key.  This is how the OAuthServlet and the Authorization page get the provider it needs to use to manage the Authorization Workflow.

Next it checks to see if this is a request to the Authorization page.  If it is, it just forwards the request on to that page.  If it isn't then the filter must validate the request.  It extracts the OAuth parameters from the Authorization header, gets the tokens, and verifies that the signature is correctly computed.  If the tokens aren't there, the authorization header is missing, or the signature is incorrect, it returns a 401 Unauthorized to the caller, with a WWW-Authenticate: OAuth ream="realm" response header to the caller.

If the request is valid, the an OAuthRequestWrapper is created that will return an OAuthPrincipal in response to the getUserPrincipal() HttpServletRequest API call.

OAuthUser

This class represents the means by which any application can get access to the name (actually the identifier) of the Application which is making request, and the name of the User on whose behalf the requests are being made.  This principal represents not the User, but rather the Application, because it is the Application that is the actor doing things, not the user.  OAuthUser has one method that allows access to the id of the user being "impersonated" for the requests the application is performing (getImpersonating()).

Any API call protected by the OAuthFilter can obtain a reference to the OAuthUser class as follows:

  OAuthUser u = null;

  Principal p = request.getUserPrincipal();

  if (p instanceof OAuthUser)

u = (OAuthUser)p;

This is essential so that the API knows which user's data to return.

Repository

The repository provides access to Tokens (by their key), Users (by ID), and Applications (by ID), provides an API to construct tokens used in the Authentication Workflow, to remove used tokens, and to check and store nonce values.  Most tokens don't need persistent storage in a database, but client (or consumer) and access tokens need long term persistence.  For testing, I developed a repository that just stores all of these in memory.  Users and Applications do need long term persistence, and so the MemoryRepository class is only a starting point.

Nonces only need to be stored for about 5 minutes or so (depending on the maximum time difference between an application request and response you want to support).  I use two hash tables to deal with storing nonces.  The first one is populated with all nonces created in a particular 5 minute period.  The second one is populated with the nonces created in the prior period.  To check whether a nonce was already used, it is looked up in both tables, but when saving a nonce that has been used, it is stored in the first table. Every five minutes I throw away the oldest table, and transfer the newer table as the older table, and create a new empty table.  It's very quick, and means that I don't need to do much to manage nonces.

OAuthProvider

The OAuthProvider class supports a method to validate an HTTP Request, get tokens (from the repository) for each stage of the Authorization Workflow, check for prior authorization, authorize an application, and manage logging.

OAuthServlet

The OAuthServlet supports several steps of the Authorization Workflow.  It exposes two endpoints in the web application context: /oauth/request and /oauth/access.  The first endpoint is used by the client application to obtain a request token.  The second endpoint is used by the client application to convert a temporary token into an access token, which can be used in subsequent API requests.

The Servlet obtains the OAuthProvider to use from the request context (stored under the OAuth.Provider key by the OAuthFilter). It checks to see if the request is for a request token or access token, and also determines whether the request being performed is appropriate to the role associated with the tokens used in the request.  On failure here, the OAuthServlet returns a 404 Not Found error.

Otherwise, it performs the appropriate next step in the Authorization Workflow.

Authorization Page

The Authorization page is what connects OAuth to the rest of the web application.  This page has two halves.  The first half deals with fulfilling the Authorization Workflow.  The second half deals with displaying to the user what they can authorize for the application.  These are really two independent pieces, where only the second part should be customized by web applications.  I could refactor this just a bit more by placing some of this in the OAuthServlet, and having the Authorization page be configured so that the servlet could forward to it as needed.

The page expects an authorization request containing a request token coming from the user agent (via a redirect from the data holder).  When the page posts, it needs to include the roles that the application has been authorized to access on behalf of the authorizing user.

The Authorization page needs to be protected by container managed security.  This is because the OAuthProvider used by this page associates the user identified by the Principal associated with the request by container managed security in the application.  The ramification here is that Principal.getName() must be a unique identifier for each user.

Putting it All Together

OAuth enabling an application is pretty straightforward with the tools I've put together.

Here are the prerequisites:

1. A JSP 2.3/Servlet 1.2 (or higher) enabled Web Container (my test Environment is Tomcat 5.0).
2. A Web Application (existing site) that uses Container Managed Security to support login to your existing application.

And here are the steps to OAuth enable your application with these tools:

1. Add the jar file containing the OAuthFilter and OAuthServlet to WEB-INF/lib in your application.
2. Add about 50 lines to your applications web.xml to configure the OAuthFilter and OAuthServlet.
3. Update the Authenticate.jsp page to support the roles needed by your application, and the user interface you want to present to users.
4. Add the Authenticate.jsp page to the list of protected pages that require a login for your application.
5. Create API pages that allow access to the services you want to expose.
6. Ensure that your API pages are protected by the OAuthFilter.

All the code can be found here.  

Next Steps

My next steps are to build a SQL Database backed Repository, and do a little more thorough delivery on this code base (including creating appropriate Jar and documentation files which you have to build yourself today).  The DatabaseRepository will implement a bit more capabilities around users and applications, providing some of the attributes you saw in the class diagram at the beginning of this post.

An OAuth Provider for Java based on Scribe

If you've been following my travels with ABBI, you know I recently embarked on developing a sample OAuth provider service.  A day later, and I've managed to implement it.  There's one bug with the URL decoding that I need to work on (basically, what starts out as a + winds up as a space later on).  This is intermittent, but not sufficiently so that I was unable to find it.  It shows up when dealing with tokens, so the quick fix is to swap out some characters when I create them.  The longer fix will be to figure out where the double decode is occurring.

I started with a Java OAuth client library known as Scribe, which has a lot going for it.   The solution includes the Scribe library, and 9 additional classes.  All told, it's less than 650 lines of code, and relatively sparse on documentation.  And I managed to get it to work in two days. Two days.  I'm done listening to complaints about OAuth complexity.

So, given my lack of copious documentation, here's an explanation of how this stuff works.

InvalidOAuthRequestException

public class InvalidOAuthRequestException extends Exception {
   public InvalidOAuthRequestException(String reason) {
   super(reason);
  }
}

This is an exception class that I use to report a variety of different reasons why an OAuth request might not be valid.  As far as my code is concerned, I don't need to differentiate these.  If someone sent the wrong data, the error message is enough to tell you why my provider rejected the request.  Classes don't get much shorter than this.

OAuthProvider 

public interface OAuthProvider {

  public Token getRequestToken(HttpServletRequest req) 
    throws InvalidOAuthRequestException;

public String authorizeUser(HttpServletRequest req) throws     
    throws InvalidOAuthRequestException;

public String getCallback(HttpServletRequest req) throws 
    throws InvalidOAuthRequestException;

public Token getAccessToken(HttpServletRequest req)
    throws InvalidOAuthRequestException;

public void validate(HttpServletRequest req)
    throws InvalidOAuthRequestException;

   public TokenRepository getRepository();
}

This interface describes a provider service, and contains six key methods.  In the OAuth workflow, there are four main steps:

1. The first step is to get a request token.  The getRequestToken() method of OAuthProvider handles most of the effort.  It takes a HttpServletRequest that has been generated by calling the request token end point appropriately, and generates a new token. 
2. The next step is dealing with the login flow.  Once it has a request token, the client needs to pass it the "login" page of the service provider.  The service provider can either accept or reject the users credentials, and the user can agree or cancel on the authorization.  
1. If the user credentials are accepted AND the user authorizes the calling application to access their data, then a temporary token and verification string is created and the user is redirected to the client application with those values.  The temporary token is created using the authorizeUser() method.  That method returns the callback string which can be used to redirect the user appropriately.
2. If the user credentials are not accepted, OR the user does not authorize the calling application, we need to return back to the calling application WITHOUT passing a temporary token.  That's what getCallback() helps with.  It generates the appropriate callback string to handle this "unauthorized" path.
3. Having been authorized (following from 2.1 above), the client application now must request an access token.  The getAccessToken() method of OAuthProvider again, handles most of that effort.
4. Finally, API requests need to verify that they are being accessed with a valid access token.  That's what the validate() method does.
5. Tokens, Tokens, Tokens.  Every provider deals with a ton of tokens.  It has to be configured with a place to create and store these things persistently.  getRepository() provides access to the provider's token repository.

OAuthProvider10aImpl

There are number of variations on a theme that could be used to implement an OAuth provider.  I picked a few reasonable settings and implemented them.  It uses the HMAC-SHA1 signature method as the only method that it expects clients to use. I don't need the complexity of RSA-based signing, but I don't want to support PLAINTEXT signing.  

The core methods of my implementation follow the same general pattern:

• Map the ServletRequest into an OAuthRequest.
• Validate the signature of the OAuthRequest.
• Perform some business logic with various OAuth parameters, manipulating tokens as necessary.
• Return the token (or other data) needed for the next step.

TokenRepository

import org.scribe.model.Token;

public interface TokenRepository {

  enum TokenType {

    CLIENT_KEY,

    REQUEST_TOKEN,

    TEMP_TOKEN,

    ACCESS_TOKEN

  };

  public Token get(TokenType type, String key);

  public void put(TokenType type, Token t);

  public Token remove(TokenType type, String key);

  public Token create(String data);

  public boolean isNonceUsed(String nonce);

  public void saveNonce(String nonce);

}

The TokenRepository interface defines an interface to a persistent store of tokens, and provides a mechanism to create them.  It also keeps track of "nonce" values, single use values that ensure that replay attacks cannot be used in the protocol.  The get/put/remove methods work like the related Map methods.  In fact, my implementation of a memory based token repository uses just that as the backing store.  The isNonceUsed and saveNonce methods are interfaces to something like a Set.  The saveNonce() method adds the nonce to the set of previously seen nonce values.  The isNonceUsed method checks to see if a nonce has been seen before.  In the real world, you don't save every nonce forever.  Typically, you save those that have been used in the last 5-10 minutes, and refuse to process any request that has a time stamp that is older than that.

MemoryTokenRepositoryImpl

MemoryTokenRepositoryImpl is an implementation of the TokenRepository class.  It uses a HashMap to keep track of tokens and nonces.  The get/put/remove methods map into get/put/remove on  its hashmap with the TokenType prefixed to the key.  The same hashmap is also used to store nonce values, with "nonce:" prepended to the nonce value serving as the key.  It's a totally simple and useless implementation, unless of course you need something quick and dirty for testing.  In which case, it works just fine.

Tokens (and secrets) are created by generating about 24 bytes worth of random bits (using java.util.Random), and then base64 encoding the results, and splitting the string into two parts.  The first part becomes the key, and the second part the secret.

Util

This was really intended to be a utility class, but instead it is what does most of the heavy lifting.

public static OAuthRequest createRequest(HttpServletRequest req)

This method creates an OAuthRequet object (a class existing in Scribe) from an existing HttpServletRequest object.  OAuthRequest is the class that Scribe uses to generate signatures from, which is why I do this.  I don't want to have to regenerate the signature creation logic.  This method is called very early by each of the request processing methods found in OAuthProvider10aImpl.

public static Map<String,String> parseHeader(String header)

This method is used to parse the "Authorization" header of a request.  OAuth parameters can appear either in query parameters, as POST parameters, or in the Authorization header.  The parser is dead simple.  It splits content at sequences of commas and spaces, and then splits those into a name and value part.  Because of the very smart way that OAuth defined the OAuth schema for this header parameter, you can use a dump parser like this.  This method is called by createRequest above to read the Authorization header of a request and store the parameters into the OAuthRequest.

public static Verb getVerb(String method) 

This method maps a string into an appropriate enumeration type, turning "GET" into Verb.GET, and "POST" into Verb.POST.

public static Token validate(OAuthRequest r, TokenRepository tokens) throws InvalidOAuthRequestException

This method does the real work .  It checks the OAuth Version (1.0 only), ensures the request is not too old, verifies the nonce hasn't been used before, maps the consumer_key and token to a consumer_secret and token_secret, regenerates the signature and compares it to the existing one.  If, somewhere along the way, one of these tests fails, it throws an InvalidOAuthRequestException explaining the stage at which the failure occured.

public static String getSignature(OAuthRequest r, String consumerSecret, String tokenSecret)

This method computes the signature from the request, given the consumer and token secrets.  This is five lines of code.  It simply puts together the right parts from elsewhere in Scribe to generate the signature.

OAuthRequest

This is a servlet that implements the token request and access request endpoints.  It expects something implementing the OAuthProvider interface I described above to be stored in "OAuth.Provider" attribute of the application.  The doGet() method usually returns a 404 (but if the URL includes the word test, it duplicates doPut()).  The doPost() method gets the operation to perform from the end of the URL Path (either /request or /access).  It calls getRequestToken() or getAccessToken() appropriately from the OAuthProvider.  Then it writes the returned token to the response in application/x-www-form-urlencoded format.

init.jsp

The OAuthRequest servlet needs a provider.  This JSP creates one and stuffs it into the appropriate application attribute.

Authorize.jsp

This is my login page.  Right now, it looks something like this:

By entering your name and password below and clicking on "I Agree", you agree to the following:

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Username: 
Password: 


The point demonstrated is that Authorization can be worded as "consent".

validateCredentials.jsp

 The whole point of OAuth is to validate that requests have been authorized.  So this page is a dummied up API call.  It looks something like this right now:


<%  response.setHeader("Content-Type", "application/json"); 
    OAuthProvider provider =
      (OAuthProvider)application.getAttribute("OAuth.Provider");
    try
    { provider.validate(request); }
    catch (InvalidOAuthRequestException ex)
    { ex.printStackTrace();
      response.sendError(401);
    }
%>{ 
 "Test": "This is a test",
 "name": "Hi Keith",
 "screen_name": "Did this work?"
}

It fails if you aren't sending an OAuth signed request, and succeeds if you are.  The content is just some JSON that my ABBI application knows how to process.

Download

Everything you need to build this OAuth provider is available for download on Google code.  Just download the ABBI-Auth project from svn here. 

An OAuth Provider for ABBI

My exercise for the past couple of days has been to build an OAuth provider for Java.  There are several reasons why I'm doing this.  The first is because when I started doing the security risk analysis, I realized that ABBI needs to be an OAuth provider, not a client.  My first implementation of Monday implemented the client.  Oh well, live and learn.

While there are plenty of OAuth Java frameworks for the client side, there were only a few for the provider side, and many of those required use of frameworks more complicated than I wanted to handle (e.g., Spring).  I suppose that someday I'll have to learn how to use these frameworks, but not right now.  I had already been using Scribe as my OAuth client framework.  It's really simple to use, and I was able to implement an OAuth client in about 4 hours with several interruptions included in that time.

Because both the OAuth client and the provider have to go through many of the same steps to verify a request, I wondered how difficult it would actually be to create a provider on my own.  So I decided to figure out how to take Scribe and use it to create an OAuth provider implementation.  I spent a good part of the day understanding the Scribe internals.  It's pretty clean code, and I found that I could reuse quite a bit of it to create the provider side.  One of the reasons I decided to stick with Scribe was because it was very light-weight.  Even so, like all gems there's quite a bit of capability if you look deep enough.

About halfway through this exercise, we had the ABBI Pull call, and several folks had raised concerns about OAuth and the effort their clients would need implement it instead of some other framework for accessing patient data.  For a lot of good reasons, I don't trust code not developed by security experts to secure my information.  OAuth has a lot of features that I understand to secure a connection.  Some OAuth-like structure that works in different ways with different sites just doesn't cut it for me.

So, yesterday's discussion about the access framework for ABBI reassured me that understanding how complex it would be to implement OAuth starting with just what I know, and a few additional tools, would probably be a good idea.

Today I'm about halfway there.  Having the client implementation helps, because it becomes part of how I test my implementation.  I did my original tests with ABBI using Twitter as my OAuth provider.  What I now needed to do was develop an OAuth provider implementation that I could direct ABBI at instead of Twitter.  Once that was able to complete the workflow correctly, I knew I'd be done.  I've completed testing about half the workflow.

I should be finished with the provider tonight, and I'll post about how it works tomorrow, along with the source code for anyone else who might be seeking a Java OAuth provider.  I expect that it will be fewer than 500 lines of code.  It won't be production code, in that it won't have all the cool enterprisy stuff like user creation and management, access controls, persistence, et cetera that you need for a full blown authentication server.  What it will have is the stuff you need to integrate what you already use for those things with OAuth, so that you can authorize other applications to access data via your application API.

-- Keith