OutsourcingDifferent Types of Security Tests
The below table describes the most common types of security testing that may be required from partners working with Google.
| Vulnerability Scan | Static Source Code Analysis | Penetration Test | Source Code Audit | |
|---|---|---|---|---|
| Method | Automated | Automated | Manual1 | Manual |
| Purpose | Identify known vulnerabilities based on pre-configured signatures. | Identify a specific set of vulnerabilities in source code. | In-depth, manual security assessment of a defined scope. | Very in-depth review of a single application/product. |
| Benefits | Fast way to identify misconfigurations and missing security updates. Since these scans are automated, they can be run frequently and complement the vulnerability management process very well. | Fast way to identify vulnerabilities newly introduced into code. | Can identify previously unknown vulnerabilities. Very low false positive rate. Can find subtle logic errors that lead to security issues. Can follow vulnerability chains (exploiting vulnerabilities to identify further issues). Humans are still best at assessing risks. | Can identify pretty much all types of vulnerabilities and security weaknesses in an application's code. Also able to identify design issues and recommend improvements. |
| Limitations | Vulnerabilities in custom or uncommon software cannot be identified. Only finds vulnerabilities a signature has been created for. False positive rate is often very high. | Still in its infancy. Usually reports a very high number of false positives, making it hard to identify real findings. Limited support for discovery of logic errors (which includes entire classes of vulnerabilities, such as authentication bypass, some types of privilege escalation, etc.) | Time-intensive and expensive. Quality highly dependent on the provider. Scoping often dictates the quality of the test. | Very time-intensive and expensive. |
| VSA/IPA Requirements2 | Attestation of Quarterly scanning | N/A | Annual third-party penetration test report | N/A |
| Example Providers3 | Nessus, Nexpose, Qualys, Trustwave App Scanner, … | Fortify, Veracode, CheckMarx, Whitehat, … | NCC Group, Bishop Fox, Leviathan Security, … | |
1 Penetration testing that is marketed as Automated penetration testing or as validated scans (e.g. Whitehat Business Logic Assessments) do not meet the threshold to be accepted as manual penetration testing under this definition.
2 Please check your Information Protection Addendum (IPA) for further guidance on these requirements for your specific project. Where additional types of testing are required, these will be clearly requested as part of the IPA and VSA process.
3 This list includes security companies that are known to provide these services, and does not constitute a recommendation on the quality or coverage provided.