Who do you trust?

If you've been involved in software for any period of time, you've heard the terms validation and verification.

If you've been involved in standards for any period of time, you've also heard the terms certified and accredited.

By non-pedants, these two pairs of terms are often slightly confusing, because one term of each pair is often used incorrectly to mean the other. 

The differences are subtle:

• Validation is the process of ensuring that a specification meets the customer requirements.
• Verification is the process of ensuring that a product meets the specifications.

• Accreditation is a third party process of ensuring that an organization has the capacity (skill set and processes) to create validated and verified products.
• Certification is a third party process of ensuring that a product meets a specification, and may include verification that a particular set of skills and processes were used during development.

In the end, Validation and Verification are everyday engineering processes in an organization. Accreditation is something that occurs periodically, and which ensures that organizations are following those everyday processes.  And Certification happens sometime around product delivery to ensure that a product is verified.

The word accreditation includes the same stem as credible, and is about establishing trust in an organization.

In software development, these words are all meaningful when you think about the acquisition of third party software products for use in your own software development.  When you acquire that software (either through purchase, or via open source), you need to validate that it meets your needs, and may also need to verify it.  It depends on your organizations process requirements.

In my past, one of the ways to acquire new software included a step where we basically "accredited" an organization, essentially convinced ourselves (or our leaders) that the organization that developed the materials had good processes and followed them, which mean that we had less effort to go through when we verified that the software met our needs.  You basically do the same when you look at a brand to make a decision (and yes, Apache is a brand)

This can be very helpful, because a full-blown verification of something like an XSLT processor is a pretty extensive (and expensive) task.

As I look at the recent discussions about the exchange of Situation Awareness data in the media recently, what we face is this challenge of trust.  The question of who you trust is important.  CDC has a trusted brand, HSS Protect is brand new, it has yet to establish such a brand, and the consequent trust.

The last thing any Software Engineering manager in the non-governmental world of software engineering is going to allow in the development of a system is the replacement of a trusted branded system with a novel system in the middle of a product release.  It's just NOT the way we've learned to do things successfully.  Yet, there have been times when it has been necessary (I'm not saying it is at this time).  What has to happen then, is that the NEW system has to be validated and verified.  It has to be thoroughly tested, and that also means that there's a lot about how it works that has to be made transparent to people who are going to rely on that system. That's how trust works.

The challenge of validation is a real one.  I've worked on software projects where we built a fully verified system, but failed to validate one of the requirements (that it was sufficient to be as accurate as a human), and that caused the product to fail.  Humans can explain their rationale (right or wrong), but the product we build (and verified was as accurate as a human), could NOT explain it's rationale, and so failed to meet a fundamental requirement of its users, which was to be something that could be trusted.  And since trust failed, the product failed.  

I don't know if HHS Protect is going to be successful in the long run. HHS Protect's  primary requirement is to be a trusted system reporting the data about what's going on with COVID. It won't succeed if it cannot be trusted.  That's not a statement of opinion, or about politics.  That's a statement of experience.