Testing Types

Black Box

Most security testing is "Black Box". The tester is not provided with any internal knowledge of the system and approaches testing from the same position as an external or internal attacker with the same level of assumed knowledge. The tester is effectively 'blind' in this scenario, and aims to discover and exploit vulnerabilities that can be discovered from this perspective.

White Box

In contrast, white box testing involves thorough inspection and testing of the internal structures and workings of an application or system. Full visibility of the source code, system and service configuration, architecture diagrams, and other crucial documentation is typically provided. This complete access allows them to scrutinize the codebase and configurations for potential security issues and misconfigurations, including issues that might not be as obvious from a black box perspective or require additional time or brute force type methods to discover. White box testing is methodical and comprehensive, targeting known risky areas in source code such as data validation, code execution paths, and sensitive data handling and all aspects of a service configuration including those affecting developers and system admins for greater defence in depth. Highlighted issues may not be security vulnerabilities within their own right, but help systems adhere to best practice relating to hardening and hygiene, hopefully making systems services more robust against attackers.

Penetration Testing

Penetration Testing is often Black Box but can also be conducted with credentials and or from the perceptive of specific users or networks. It aims to simulate real-world hacking attempts and reveal how the system responds to unexpected or incorrect types of input and misconfiguration and weaknesses that could result in unauthorised data access and elevation of privileges. Penetration testing aims to quantify and validate impacts of vulnerabilities through exploitation and uses real world attack techniques and tools to chain vulnerabilities and weaknesses together to fully ascertain the severity of issues and test defences in depth. A Black box penetration test can and should be scoped widely to include websites, hostnames, IP addresses and Public Cloud and will include all manner of attacks against applications, infrastructure and network protocols. Tests can also be scenario based with specific goals which can help focus testing on higher risk areas, or areas of concern (high value assets or data for example). Tests are conducted during a fixed time period at the end of which a report will be produced detailing all the discovered vulnerabilities and how to fix them. Regular penetration testing is encouraged to ensure security is maintained against the evolving threat and technology landscape.

Web Application Assessment

Web Application Assessments are typically Black Box but can be augmented with access to source code and are often conducted from the perceptive of different user roles within the application. They aim to be a comprehensive assessment of an applications security posture from various attack vectors. Like Penetration Testing, Web Applications Assessments aim to simulate real-world attacks to quantify and validate impacts of vulnerabilities through exploitation and chain vulnerabilities together to fully ascertain the severity of issues and test defences in depth measures. Unlike penetration testing, a Web Application assessment will be scoped to a specific website or websites and focus on web application vulnerabilities like those described in the OWASP Top 10. Tests are conducted during a fixed time period at the end of which a report will be produced detailing all the discovered vulnerabilities and how to fix them. It is recommended to perform assessments as part of the development lifecycle before significant changes are released into production.

Bug Bounties

Bug bounties are a form of Black Box testing and have risen in popularity as a means of harnessing the collective expertise of the global security community. Companies and organizations incentivize independent researchers by offering rewards for reporting cybersecurity vulnerabilities in their products or services. By capitalizing on the varied skill set of numerous ethical hackers, these entities can uncover security flaws that might have otherwise gone unnoticed in the traditional testing process. This model helps create a feedback loop where the cycle of finding and fixing security bugs is continuous and not conducted during a fixed time period like a typical Black Box engagement. Bounties can also be scoped to include websites, hostnames and IP addresses and because issues are solicited from the general security community will require someone to triage submissions and determine the validity and impacts before applying fixes or paying out a bounty.

Cloud Configuration Reviews

A cloud configuration review consists of a white box assessment of cloud environments (AWS, Azure, GCP, OCI, M365, Kubernetes Etc) to assess compliance with security best practice, and identify weaknesses and common misconfigurations. The configuration review will assess the security posture of the cloud management environment (control plane) as well as the applications, containers and infrastructure hosted within (data plane) and is a useful adjunct to other black box Pen Testing activities. These reviews aim to ensure cloud services, such as infrastructure or platforms, are configured correctly to protect data and maintain privacy. These reviews scrutinize the entirety of cloud deployment—from network access controls and storage encryption to identity and access management policies. Being a White Box assessment Cloud Config Reviews require full read access to the target account and can be scoped for single accounts/tenancies, multiple accounts or organisations and also include supporting services used to deploy resources into the cloud environment, for example Terraform Scripts, GitHub Actions, Azure DevOps, Jenkins etc. Assessments typically take a few days and will produce a report detailing all issues along with a severity rating and remediation advice.

Server Build Reviews

Server build reviews are another form of White Box assessment and are a technical deep-dive into cloud hosted or on prem Linux and Windows server configurations. This assessment focuses on the operating system, installed software, system services, and access controls with the primary goal being to verify that the servers have been hardened, meaning they are set up to minimize the surface area for attacks and includes aspects like patch levels, unnecessary service removal, and adherence to security best practices. Build reviews are often included in a larger security assessment along side black box penetration testing and white box configuration reviews. Requires Admin and network level access to target hosts and will produce a detailed report highlighting all security vulnerabilities and items that do not align with best practice guidance.