Penetration Testing Guidelines

Summary

Penetration tests requested by Google must fulfill the requirements listed below. Note that requirements may differ between vendors due to special agreements.

1. Penetration Test and Report Requirements – Report the result of the penetration test in writing and include:
   • Appropriate contextual information
   • The findings and their risk severity
2. Assessment Duration and Dates – Plan an appropriate total testing effort (in person days) according to the number and scale of systems in scope. Penetration tests of a small scope can take only a few days, while a large scope can require multiple weeks. The penetration test must have been performed in the past 12 months.
3. Scope of the Assessment – Scope the penetration test to include all infrastructure used to provide services to Google. If there is a security dependency (i.e. the infrastructure used to provide services to Google depends on other systems for its security), those systems need to be included in the scope of testing.
4. Attribution Information – Entrust an appropriately qualified third-party provider with the penetration test. The quality between providers can vary significantly. Google recommends carefully comparing providers. Upon request, Google can share a list of providers whose reports have met our quality expectations in the past.
5. Assessment Methodology – Employ manual methods for identifying security vulnerabilities when penetration testing. Manually verifying automated results is not sufficient. Additionally, perform penetration tests using both unauthenticated and authenticated (i.e. the testers must be provided with credentials to the application, if an authentication exists) methodologies.
6. Documentation of Findings – When reporting findings, assign a risk severity and include technical details to allow Google to independently assess the risk and impact of the finding.
7. Remediation Planning / Confirmation – Remediate findings with a critical or high risk severity, these should not be risk-accept. In addition to the penetration test report, also share a remediation plan. For findings that have already been addressed, include a remediation confirmation.

Adherence to a security standard, automated vulnerability scanning/testing, and Bug Bounty and Vulnerability Rewards Programs are not sufficient replacements for penetration test reports.

Google’s Requirements for Penetration Testing and Reporting

The following subsections describe the requirements set out by Google for penetration testing and the penetration testing report that you will submit to Google during the VSA process.

Some resources providing background information:

• For an understanding of Google’s definition of penetration testing, see Different Types of Security Tests.
• For an understanding of why Google requires third-party testing, see The Importance of Third-Party Testing.
• For an understanding of the difference between compliance and penetration testing, see Compliance and Penetration Testing.

Penetration Test and Report Requirements

Ensure that the penetration test report is complete. If the information you share leaves some points unanswered, there may be gaps in the quality of testing that was performed that will need to be clarified. This can result in delays to the approval process.

To ensure your report is complete, verify that it answers these key questions clearly before submitting it to Google:

1. Was the penetration test performed within the past 12 months?
2. Was the total testing effort in line with the number of systems which were tested?
3. Was the penetration test scoped to include all systems used to provide services to Google?
4. Was the penetration test performed by a third-party provider?
5. Did the penetration test include manual methods of analysis for identifying security vulnerabilities?
6. Was the penetration test performed using suitable authenticated test accounts?
7. Have findings with high and critical risk severity been addressed (not risk-accepted)?
   • Where the report doesn't cover remediation, provide a separate remediation plan / confirmation document.

Go to Top

If the penetration testing report does not cover ALL of these points, please provide an attestation letter from the penetration testing provider that clearly responds to these questions in full.

Assessment Duration and Dates

Capture the dates on which the assessment was conducted, as well as the testing effort (in person days).

If the penetration test is part of a periodic assessment, include the dates of the previous and upcoming assessments. This information enables Google to understand how recently the penetration test was conducted as well as the regularity of testing.

Requirements Covered

1. Was the penetration test performed within the past 12 months?
2. Was the total testing effort in line with the number of systems which were tested?

Go to Top

Scope of the Assessment

A well-defined scope must include application-level testing of any application related to the processing or storage of data in the service provided to Google.

As a consequence, a good penetration test report should include the following elements when describing the scope:

• A well-defined scope definition that demonstrates sufficient coverage of the various threat models the application is subjected to.
• If there were areas that could not be assessed for whatever reason, mention this in the scope definition as well.
• If the penetration test consists of multiple areas, indicate the testing duration devoted to each area.
• For areas where active penetration testing is explicitly not permitted (i.e. select third-party SaaS services), include a configuration review.

Following these points helps to reduce any ambiguity on whether a particular area was sufficiently covered during the assessment. It also helps Google identify any residual risks that need a closer examination after a penetration test.

Requirements Covered

3. Was the penetration test scoped to include all systems used to provide services to Google?

Go to Top

Attribution Information

The report must include the name of the security company involved in the assessment as well as the names and titles of the testers.

If you need support in picking a qualified penetration testing provider, see Penetration Testing Provider Selection.

Requirements Covered

4. Was the penetration test performed by a third-party provider?

Go to Top

Assessment Methodology

Google requires that penetration testers use an assessment methodology with the following characteristics:

• Covers both authenticated and unauthenticated testing if the asset has an authenticated section. A comparison of the two approaches can be found at Unauthenticated vs. Authenticated Testing.
• Enforces that the majority of the work is performed manually. Automated testing or manual verification of automated testing does not meet this requirement. For more information read Manual vs Automated Testing.

A good penetration test report should clearly outline the assessment methodology followed during the assessment and call out if the assessment was performed authenticated and manually. Ideally, the penetration testers follow an industry-recognized standard that is relevant to the environment (i.e. OWASP, PTES, or OSSTMM). Finally, the report should also include a section outlining the tools used during the assessment.

Requirements Covered

5. Did the penetration test include manual methods of analysis for identifying security vulnerabilities?
6. Was the penetration test performed using suitable authenticated test accounts?

Go to Top

Documentation of Findings

The section on findings is the core of a penetration test report. Unfortunately, the structure and information provided in this section may vary greatly between different penetration test providers.

Make sure your penetration testing provider includes the following information in this section:

• An overview table of all findings, listed by title and indicating the risk severity
• For each finding, detailed documentation covering:
• Risk severity of the finding.
• The finding’s impact from a business and technical perspective.
• Clear, concise steps on how to reproduce the finding.
• Screenshots or other form of evidence confirming the finding (if applicable).
• Recommendations on how to resolve the finding. These recommendations should be tailored to the application and environment, avoid generic recommendations.

If this is part of a retest, the report should provide a history of findings which persisted from previous assessments. Please include information about why remediation efforts were not effective. If remediation was not carried out, business owners should provide supporting information behind that decision.

Requirements Covered

5. Did the penetration test include manual methods of analysis for identifying security vulnerabilities?
7. Have findings with high and critical risk severity been addressed (not risk-accepted)?

Go to Top

Remediation Planning / Confirmation

Your penetration test should always conclude with a remediation plan. This is not something that your penetration test provider creates, but is instead part of your action plan regarding the identified findings.

For findings with critical and high severity that have not been mitigated at the time of sharing your penetration test report, create a remediation plan that outlines:

• The expected remediation date.
• The planned remediation action and how to prevent future regression of identified findings.
• Considered residual risk of findings that could only be partially remediated.

For findings with critical and high severity that have been mitigated at the time of sharing your penetration test report, create a remediation confirmation statement containing:

• Confirmation that all findings with critical and high severity have been remediated (as opposed to being risk-accepted).
• Information about residual risk for findings that could only be partially remediated.
• Supporting information from business owners on how they inted to prevent regressions of identified findings (e.g. unit test, SDLC, and so on).

Requirements Covered

7. Have findings with high and critical risk severity been addressed (not risk-accepted)?

Go to Top