To Test or Not To Test…

As a member of the application security practice for AT&T Consulting, I often get asked questions regarding application testing and web application firewalls.

  • Which is better?
  • Which is most cost effective?
  • Which addresses compliance requirements that the organization is subject to?

These are great questions and ones you’d expect from organizations trying to find that delicate balance between cost, compliance, and security.  But do these two separate modalities of application security address the same, or highly similar, internal control objective – that of application security assurance?

Application Testing

Application testing is a procedural control intended to identify and detect flaws that might be exploitable within a piece of software. Such flaws can be detected through automated means (i.e., using vulnerability scanners to check quickly for thousands of potential flaws) or by a team of individuals using purely manual methods: an optimum testing plan leverages both approaches.  This blended methodology generally results in the fewest occurrences of false-positive results and the best chance to identify complex, high-risk security-impacting conditions in the application prior to the application’s deployment to production.

Assessments provide a “point in time” look at the security posture of the application, a look that can become outdated quickly in highly dynamic environments. Due to emerging research by security experts, or as a result of oversights in the patching, upgrading, and maintenance processes, ongoing, post-production assessments are needed to detect the presence of conditions in the application that become threats over time. Also, there is always the risk of unidentified flaws making their way through this process and into production.

Web Application Firewalls

Dedicated Web Application Firewalls (WAFs), and the multi-feature products which provide WAF services as part of an overall solution (referred to as “WAF+”), are technical controls intended to prevent attacks at the web application layer from being successful. At a high level, these solutions seek to apply a set of rules to an HTTP conversation in order to thwart Cross Site Scripting, SQL Injection, and other common web application attacks in applications after they have been deployed.  For legacy applications, WAF solutions can be used as a compensating control for identified security flaws; such as vulnerabilities that have been revealed via a penetration test or code review and cannot be fixed within the application in a timely or cost-effective manner.

While WAFs provide ongoing protection, they are not a substitute for secure coding practices and code-level resolution of identified flaws.  A key example of this can be seen in the recent compromise of an application at Barracuda Networks. While the company had a WAF solution in place, due to an administrative oversight, the solution allowed malicious web traffic through to a website which was vulnerable to SQL Injection.

Working as a Team

Individually, application testing and web application firewalls each provide an organization with a measure of insight and protection from application security exposures, but they are not equivalent controls.  They are complimentary controls meant to work together to reduce the risk of application compromise. I suggest you test your applications to understand what exposures are present without a WAF.  Then validate those findings once the WAF is in place to understand what issues are mitigated, or not mitigated, by the technology.  Use the results to prioritize the issues for remediation at the application layer and to fine-tune your WAF configuration for optimal coverage.


The question is not, “To test or not to test?” but whether or not application test results are used most effectively within the organization to vet complimentary controls and drive long lasting, strategic, second order change within the enterprise.

Mike Klepper Consulting Application Security Practice Director AT&T About Mike