Breach and Attack Simulation vs Penetration Testing

There is a lot of confusion in the market around the difference between “Breach and Attack Simulation” and “Automated Penetration Testing”. They are different technologies that deliver different outcomes. Let’s clarify the difference for you.

Breach and Attack Simulation

The primary aim of a BAS technology is to test the effectiveness of your operational security controls by emulating security breaches within your internal network.To get the full capabilities out of a BAS technology, you must deploy BAS agents across all of your internal hosts and deploy virtual machines in key zones throughout your security architecture.

BAS host-based agents are typically used to identify vulnerabilities on the hosts by gathering missing patches and to simulate host-based breach scenarios. Many BAS technologies use the MITRE ATT&CK framework as the basis for their breach simulations, which may include simulating malware infections to determine if your host-based security controls detect the activity and alert your security operations team.

BAS virtual machines are used to simulate network-based attacks between each other to test the effectiveness of the network-based IDS/IPS or next-generation firewalls and whether they will alert your security operations team.

BAS certainly adds value to organisations; however, there are some critical limitations to BAS technologies that you need to consider:
 

Since the BAS agents are deployed on internal systems, there is no simulation of internet-based attacks against your perimeter systems, which is pretty important considering that the attackers are on the internet. In fact, we would argue that testing your perimeter defenses against internet attacks is one of the most important aspects of a pentest, and the BAS simply cannot provide that.

Since the BAS virtual machines are typically deployed internally, the network-based simulations are only tested internally. If you get creative, you could deploy a virtual machine on the internet to test your internet-facing threat detections.All authenticated or agent-based vulnerability scans report an absolute huge number of vulnerabilities, with most of them not having any working exploits and therefore not really introducing risk to your business.

BAS technologies don’t perform real attacks and actual exploitation of vulnerabilities to verify that they are real, which means that around 99% of the vulnerabilities are not going to be exploitable. BAS technologies also don’t touch your web applications, which means that critical areas of your business are not being assessed.

Around 80% of all security breaches originate from leaked passwords from third-party security breaches, which BAS technologies do not monitor or test for.

BAS attack simulations are often not recognized as a threat and are less effective than emulation of real attacks BAS is unable to safely detonate destructive attacks such as malware and ransomware, which puts into question the reality of the simulations

This demonstrates that there is certainly value delivered through a BAS solution by testing the effectiveness of your operational security controls; however, it is clearly not a penetration test, so let’s now understand what an Automated Penetration Test encompases.

What Problems Does BAS Attempt to Solve?

With the increasing sophistication of cyberattacks, it is critical for organizations to have confidence in their security defenses and ensure that they are able to withstand potential threats.

Breach and Attack Simulation (BAS) is primarily used to test an organization's security defenses and determine whether they are effective at detecting and preventing simulated attacks. BAS helps organizations to assess their security posture by simulating different types of cyber attacks and measuring the response of their security controls, such as firewalls, intrusion prevention systems, and endpoint protection tools. This testing can help organizations identify gaps in their security controls and take corrective action to improve their defenses.

Automated Penetration Testing

The primary aim of Automated Penetration Testing is to perform continuous penetration testing of your organisation to identify and verify the real risks to your business across your external and internal systems, applications and even your supply chain (third party vendors).

This is achieved through black box assessments without requiring any agents to be installed onto any systems, allowing a fast and cost-effective deployment.

TYPES OF AUTOMATED PENETRATION TESTING

Features vary per vendor, with many focusing only on internal infrastructure, so we will use the wider range of Automated Penetration Testing capabilities offered within our Evolve Security Automation Cloud: 

• Evolve Automated External Penetration Testing
• Evolve Automated Internal Penetration Testing
• Evolve Automated Supply Chain Penetration Testing
• Evolve Automated DevOps Application Security Testing
• Web Applications and APIs

AUTOMATED PENETRATION TESTING METHODOLOGY

The EvolvePT, our automated penetration testing tool, covers a full five-stage penetration test:

• Automated Internet Reconnaissance
• Automated Fingerprinting and Scanning
• Automated Attack and Exploitation
• Automated Post-Exploitation and Lateral Movement
• Automated Reporting

Rather than performing simulations, Automated Penetration Testing performs contextual attacks specific to your organisation that real-world attackers would perform in order to reveal actual risks to your business. These contextual attacks include:

• Extracting employee details from social media networks in order to predict employee email addresses and locating their leaked passwords from thousands of third-party security breaches to breach exposed administrative services
• Real-time identification of vulnerabilities, intelligent safe contextual exploitation and post-exploitation, password cracking and lateral movement attacks to demonstrate and prioritise actual exploitable vulnerabilities and the corresponding impact
• Passive Supply Chain Penetration Testing against third-party vendors using intelligence sources to map out employees, email addresses, leaked passwords, domain names and IP addresses, software versions, vulnerabilities, latest exploits and recommended exploit configurations.
• Web application and API vulnerability identification using intelligent automation that utilises contextual requests specific to the application to ensure that business flows are followed and real application data is used to provide both broad and deep application security coverage

To provide an insight into the deployment effort required compared to BAS, there is very little setup required for Automated Penetration Testing, which varies for external and internal.

There is next-to-no setup required for “Automated External Penetration Testing” and “Automated Supply Chain Penetration Testing” so they can literally both be up and running in less than 5 minutes.

The “Automated Internal Penetration Testing” simply needs a single pre-configured virtual appliance that is deployed through a simple “download-and-boot”, which supports proxies and authentication. No changes to firewalls are required, which means Automated Internal Penetration Testing can be deployed within minutes.

The “Automated DevOps Application Security Testing” can be integrated with DevOps pipelines in as little as 10 minutes and will automatically orchestrate an Automated Application Security Testing environment upon the next code commit, without any further actions from any team member.

Since Automated Penetration Testing sends attacks across the network, both internally and externally, IDS/IPS and next-generation firewall detections are triggered using a wide range of attacks allowing your operational security controls to be tested. 

Since safe intelligent exploitation is used to actively compromise systems, perform privilege escalation and execute post-exploitation, host-based security controls are tested for their effectiveness and often highlights unexpected gaps in security operations. One key example is where malicious code is detected, but the security operations team is unable to locate where the exploit originated due to connections passing through proxies or load balancers, or that network connection information simply doesn’t exist.

BAS VS AUTOMATED PENETRATION TESTING: WHICH ONE SHOULD YOUR ENTERPRISE CHOOSE?

Automated Penetration Testing and Breach and Attack Simulation (BAS) are two approaches that enterprises can use to assess their security defenses. While both approaches use automated tools to test an organization's security posture, they differ in their perspective and objectives. Penetration testing typically begins from the perspective of an unauthenticated attacker, with the goal of gaining unauthorized access to the system and demonstrating its impact. On the other hand, BAS assumes that the system has already been breached and starts from an authenticated perspective. The objective of BAS is to test the effectiveness of in-place security controls in preventing or detecting malicious activities.

So, which approach should an enterprise choose?

While the choice between automated penetration testing and Breach and Attack Simulation ultimately depends on the specific needs and goals of the enterprise, the following factors should be carefully considered when making a decision:

Does the testing tool offer minimal false positive and false negative findings?

How precise are the tool's reconnaissance and fingerprinting capabilities?

Will the tool provide high-quality reports?

Is the tool easy to use?

To what extent can the tool automate difficult-to-automate human tasks, such as 100% bug validation using machine learning?

Conclusion

If you are purely looking at testing the effectiveness of your internal operational security controls, such as the effectiveness of your SOC to respond to a security breach, then BAS is likely to be the technology that you are after.

However, if your business needs to identify, verify and manage real risks to your business, across your external and internal infrastructure and applications, as well as your supply chain, to proactively prevent a security breach, whilst also gaining the added benefit of streamlining your security team through prioritised remediation activities and also testing your security operations, then you need Automated Penetration Testing.

To get started with Automated Penetration Testing within minutes, request a demo suited to your environment now.