Driving Second-Order Change from Security Assessment Reports

I’ve conducted and lead many assessments of customer computing environments.  Network infrastructure, server configurations, application environment–you name it, I’ve reviewed it.  For each of these engagements there is a particular cadence; a rhythm, if you will, unique to each customer and to the players involved.  But in the end, the assessment team completes its labors and delivers unto the customer, “The Report.”

At many points during my career, I have performed year-over-year assessments for customers.  While some clients change more than others over the intervening period, all too often, the subsequent engagement turns out to be a familiar song and dance.  Particularly when it comes to issues identified in the resulting report.

In general, “The Report” is full of observations, recommendations and the like, typically translated into a number of action items for the client to address.  These findings are usually distributed to the parties within the organization responsible for their resolution.  Organizations with mature vulnerability management processes often use some sort of workflow system that tracks the assignment of these responsibilities, the subsequent responses, and ensures that the resolution is effective and the issue closed. So why is it that in the following year, the assessment team identifies that some subset of these issues has found its way back into the organization?  The answer is simple: assessments and similar activities are seen as catalysts for First-Order change.

First-Order change means doing more (or less) of something that’s already being done.  For instance, the organization may  be patching systems and applications, but The Report notes that some patches are missing–even ones thought to have been applied.  Driving First-Order change is like treating the symptoms of an illness rather than curing the disease itself. As a result, the symptoms continue to manifest themselves. This is because a key characteristic of First-Order change is that it is reversible. For instance, to resolve a high-priority enterprise issue, a developer might change the application code in production, but that update doesn’t make it to the authoritative code repository.  As a result, the next time a software update is pushed up to production, the developer’s ad-hoc change is over-written and the issue is re-instantiated.

Identifying oversights and tactical issues in the production environment is certainly of value, but this First-Order level of due diligence, though necessary, isn’t transformative. In order to have a material impact on the organization’s security posture over time, a focus on driving Second-Order change is needed.

Second-Order change involves doing something significantly or fundamentally different from what has been done before–strategic, evolutionary change that  alters the nature of downstream tactical issues. Opportunities for Second-Order change can be identified by viewing the issues singled out in security assessment reports as key indicators of the relative effectiveness of the organization’s policies, processes and procedures.

Take the example of Cross Site Scripting, a computer security vulnerability typically found in web applications and a mainstay on the Open Web Application Security Project’s Top Ten vulnerabilities list.  When Cross Site Scripting is identified in an assessment report, the organization often engages in First-Order change by resolving the specific instances of the flaw in the application code.  However, many organizations fail to take the next step and update application coding standards to account for this issue or to search for similar issues in other areas of their application portfolio.  It’s only by evaluating the policies, processes and procedures surrounding the development lifecycle–and by making material changes to prevent such flaws from being coded into the applications in the first place—that the organization drives Second-Order change.

I realize that implementing such changes in the organization can be both challenging and time consuming.  However it is this kind of strategic change that, over time, helps advance the organization’s security capabilities and ultimately allows the security team to focus on emerging technologies and business-enabling security solutions as opposed to addressing a recurring (and ever increasing) catalogue of tactical remediation activities.

If you haven’t done so lately, revisit your last few security assessment reports. Look for issues that continue to appear, and put together a team to address the root cause of the problem. If you do, then maybe the next time around your assessors will sing a different tune.

What types of issues do you see appearing in your reports?
How does your organization identify first-order vs. second-order changes?
Mike Klepper Consulting Application Security Practice Director AT&T About Mike