Do better penetration tests – for buyers and testers

We do lots of penetration tests. Along the way, we’ve learned a few things worth sharing to get more out of penetration testing.  There are, of course, caveats; specific situations will require specific solutions, however, there are generalizations that apply in most cases.

Much of this is of use to those buying penetration tests; if you are in the enviable position of having a budget that does not need to be justified this may be of less use, however for the other 97% of people tasked with securing the organization’s systems and data, this one’s for you.

First, don’t confuse vulnerability scans with penetration tests. An automated scan of systems, which then dumps results into a report is not a pen test—it’s a scan. Scans only find low-hanging fruit, which may be helpful to keep you safe from some attacks, help identify vulnerabilities, and check some of your processes, however, they should not be signed off as a penetration test.

Second, if a “penetration test” is done in two days, it’s not a penetration test.

Next up, while possibly a useful exercise, testing the external perimeter for holes in isolation will likely do little to address the organization’s larger cyber and business risk issues in the long run.

Often, while the organization focuses singularly on fixing vulnerabilities listed in the pen test report, the opportunity to improve security in a fundamental and lasting way is missed.  This tunnel vision ignores the emergence of new and future vulnerabilities as well as how the organization actually operates.

This approach exemplifies the perception of security as a ‘business cost,’ rather than demonstrating the business value of a security budget. To be useful, a penetration test must determine the actual risk to your organization of being hacked, AND the impact of an incursion on key aspects of the organization or business processes.

So, how to set the ‘business case’ and maximize the benefits of your penetration test?  Think about the critical assets/processes within your organization and make that the target. This is sometimes known asa “trophy” based assessment. If the objective is to breach your perimeter and get to some important database or show how an important business process can be disrupted, then the test suddenly can be put in “business terms”.

The technology assessed in the pen test, and technicalities of the identified vulnerabilities are incidental. The true value in the test is assessing the ways that business processes can be disrupted or assets stolen.  Understanding and mitigating this is what improves the overall defense against the risks that your organization faces.

This also helps the test avoid the peril of ‘intellectual masturbation’ (as it was described to me) –   there are some really cool technical holes found during assessments, however, if they are not practical for actual use by a bad guy, and don’t actually have an impact on your organization, then it’s a waste of time and money. It may help the test team get a spot talking at a conference, but it’s not going to help your organization.

Similarly, even if you do just test the perimeter, you need some context provided for the findings.  For example, if a vulnerability is found on a web server, but that web server is not connected to anything else, and is actually for some low-level, mostly uninteresting app, then who cares?  If you can’t chain an attack from that box to another, then your risk is low.

In addition to determining the cyber and associated business risk, results from your assessment will ideally be used to improve your monitoring controls.  Since you have an attack that was executed to get at your critical assets, you now know what you need to do to defend them.   Take the findings from your trophy-based assessment, augment with other possible targets and see how you can catch someone. Suddenly your pen test is increasing its usefulness.

On the subject of monitoring, except in a few specific circumstances, all controls should be left on. Anyone who is offering to test your environment and wants you to turn off all your external controls is looking for an easy ride. Your test needs to be as close to the real world as possible. In the real world all those controls are on, ‘eyes are on glass’, so why not use the pen test to assess them also?

Reading all of this, many may say, “Okay, sounds good, but my budget is only $5k and I really want a penetration test?”. First – reconsider, there are a variety of other activities you can do that will be of more benefit. If you really want one done consider buying your own vulnerability scanner license key, it will probably cost you about half that budget. Run some scans, see how good or bad your environment is and fix what it finds. Yes, you will be missing out on context, business processes etc., but you’re trying to do what you can with what you’ve got.

Go through a couple of cycles with the scans, and over time see if you have some systemic issues i.e. Patch management issues, poor coding practices, etc.… then, when your next budget cycle comes around, give the results of your scans to the pen testers, and give them a target. Ask them how reasonable it is to get that done in the time they have and see what they come back with. Challenge them, maybe it can be done, maybe it can’t. However, a badly performed penetration test can create a false sense of security that’s’ more harmful than not having one done to begin with.

Set in the right context, penetration tests can be really useful.  They can help you identify vulnerabilities that, if left unaddressed, could harm the organization.  They can be used to improve controls and provide business cases to get stuff fixed. However, there will always be constraints, and many organizations such as ours are working on ways to address those and provide assessments that provide  more value. In the meantime, consider what you are doing, how it’s being done, and what question(s) you want answered.

Finally, there is no magic A.I (yet) which can do this, no matter what someone tells you.

Contact Us Today to Get a Quote for Penetration Testing