Part 4. People

In my previous related articles, I talked about what is wrong with the current state of pentesting, and two parts of the solution – the governance structure & process, and technology to enable it.

This purpose of this article is to identify the People component of the solution. Let’s look at the list of roles necessary to implement the process and structure described in the second article:

1. Owner of the Penetration Testing Program Scope

2. Owner of the Pentesting Methodologies

3. Owner of the Vulnerability Library

4. Pentesting program coordinator

5. Remediation activities coordinator

6. Pentest scoping coordinator

7. Pentesters

8. Analytics and process improvement

A small organization may have these roles performed by a single individual or combined with other roles. Whereas large corporates might need several specialists to fulfill each role. Based on the analysis of the process (see second article) – these roles are necessary for the process to be complete and effective.

Let’s take a look into each of the roles:

Owner of the Penetration Testing Program Scope

The primary responsibility for this role is to determine and maintain the Scope of the entire security testing program (including penetration testing). This role would usually sit within the information security risk management and compliance team, or within the pentesting team. In any case, this role requires significant collaboration between people who know the information security risk and compliance profile of the organization and understand penetration testing.

This role must determine what assets should be the subject of the penetration testing program. This role also determines the frequency and trigger points for penetration testing for each asset in scope of the program.

Types of assets that should be considered within scope will usually include the following:

• internal and external applications

• internet exposed infrastructure

• classified environments

• compliance related assets (such as Cardholder Data Environment)

• wired, optical and wireless networks

• people (for social engineering testing)

• buildings and other physical assets (for physical penetration testing)

• other assets determined by risk and compliance profile of the organization

Frequency and trigger points would be dependent on the nature of assets and associated compliance requirements, and risk appetite of the organization.

Owner of the Pentesting Methodologies

This role carries responsibilities to establish and maintain pentesting methodologies for each category of assets in scope of the program; and to ensure that each methodology matches the risk profile of the organization and target assets.

When deciding on which methodologies to use, organizations can start with existing industry accepted standards defined by various respectable organizations, such as:

• OSSTMM, developed by ISECOM,

• ASVS, MASVS, and CSVS, developed by OWASP,

• Penetration Testing Execution Standard (PTES), and others

This role must develop organization-specific methodologies matching each methodology to the categories of assets and their risk profiles. Even when industry accepted standards are used, they might need to be adjusted to reflect organizations’ assurance requirements.

For example, ASVS is a natural choice for a foundation for Web Applications & Web Services category of assets. However, different risk profiles associated with various web applications might require a distinct subset of the original standard. ASVS is provided in three different levels – Level 1, Level 2 and Level 3. Level 1 focuses on the essential items that needs to be tested. Level 2 requires a deeper level of knowledge & familiarity with the assets in scope. Level 3 is considered to be the most through approach, requiring collaboration between technical auditors/pentesters and the engineers/developers.

Every methodology should provide a distinct set of test cases for the pentesters to follow when performing a pentest. 

Some of the tools described in article three come with prepackaged sets of popular methodologies or ways of importing the required methodologies into the tools.

The next important step is to maintain the methodologies. Penetration testing continuously evolves and should keep up with the organization as well. New methodologies are developed, existing methodologies require updates and revisions. For example, organization might develop a need for Red Team based security testing. Methodologies for Red Team exercises are usually specific to every organisation. With each new exercise, new approaches emerge, and established tactics improve. Each change would need to be reviewed and documented in the methodologies.

Owner of Vulnerability Library

Having consistent terminology used to record penetration testing findings is an absolute necessity for mature effective pentesting programs. If different pentesters would call the same vulnerability differently - engineers will have a hard time with remediation, and evaluation of results would be nearly impossible.

There are multiple industry accepted libraries of vulnerability definitions specified by reputable organizations such as:

• CWE and CAPEC from MITRE Organization, or

• Various OWASP lists.

This role is responsible for creating a list of vulnerabilities that are most common for the organization to guide pentesters when they record their findings. Having a tool (like those described in the previous article) to establish the vulnerability library would be advantageous. Also, some of the tools may come with prepackaged vulnerability libraries.

The vulnerability library must continuously evolve as internal security specialists and external researchers are finding new vulnerabilities regularly. These vulnerabilities must be added to the library to ensure consistency and currency of penetration testing results. Pentesters are expected to find new organization-specific types of vulnerabilities, for example related to business logic. Another responsibility for this role is to moderate the input from pentesters to continuously maintain and update the library.

Pentesting program coordinator

Each pentesting engagement consists of multiple tasks; requires multiple stakeholders to collaborate; and many pieces of information to be prepared upfront for a pentest to go smoothly.

Typically this role will receive and manage new requests for pentests, as projects require them. This role may also initiate pentests for BAU assets.

Once a pentest is requested or initiated, this role will work with the pentest scoping coordinator (see next role below) in order to determine the technical approach for testing, and identify what is considered to be in-scope and out-of-scope.

This role is responsible for the following activities to be completed before, during and after every pentesting engagement:

• Collaborating with pentest scoping coordinator to determine scope and technical approach for testing

• Stakeholder management such as:

  • Establishing lines of communication with all stakeholders for each engagement.

  • Establishing Rules of engagement (what can and cannot be done within environment) for each of asset. Communicating them to pentesters.

  • Ensuring that Change Management process is followed before, during and after pentesting engagement.

  • Coordination with internal business and technology partners.

  • Coordination with customers, technology providers, and other external parties.

  • Other stakeholder management activities.

• Gathering of Logistical data such as:

  • IP addresses, URLs and endpoint details,

  • API specifications,

  • software versions and build details,

  • testing credentials,

  • physical addresses, and

  • other engagement specific details.

• Coordination and allocation of actual pentesting activities between different pentesters. This includes the following:

  • Allocating test cases to individual pentesters

  • Ensuring senior pentesting resources are available to guide and QA junior resources

  • Ensuring that vulnerabilities are communicated in the expected approved format (Vulnerability Library) to engineering and business stakeholders (if not already enforced by any workflow tools).

 Remediation activities coordinator

The primary responsibility for this role is to ensure that any vulnerabilities discovered during any of the pentests, are subject to organization vulnerability management process. This typically would include:

• Provide vulnerabilities to asset owners.

Asset owners, in assistance from risk managers, can determine the relevant risk of the vulnerability against their affected assets.

• Follow up on any vulnerabilities exceeding the organizational SLAs (or implicit expectations for smaller organizations). This usually involves line reporting to designated escalation paths.

• Coordinate retesting of vulnerabilities. This involves working with the pentesting program coordinator to ensure vulnerabilities are scheduled for retesting, and results of retesting are then shared with the asset owners.

• Generating reports on organization remediation. These reports are usually provided to senior management, risk and audit committee and boards – to help them understand how the organization is tracking and overall security posture.

Pentest scoping coordinator

Each pentesting engagement must have a clearly defined scope. The primary responsibility for this role is to accurately define the scope for every pentest within the program. This usually includes the following activities:

• Working with the pentesting program coordinator to initiate scoping exercise for every new pentest as part of the program.

• Working with the owner of the penetration testing program scope to identify assets that need to be tested for a particular engagement, and what should be considered out-of-scope. This includes understanding of the business context, physical aspects, and technologies associated with the engagement.

• Determining which methodologies must be used for the engagement.

Pentesters

The primary responsibility of the pentester is to perform a technical assessment for assets in scope of the engagement, in accordance with the methodology prescribed for the engagement.

Common phases performed by this role for most of the pentests will include the following activities:

• Reconnaissance / footprinting / intelligence gathering

• Execution of test cases

• Vulnerability analysis / exploitation / collection of evidence

• Documenting findings, evidence and recommendations

• Communicating findings and recommendations to the relevant stakeholders

Analytics and process improvement

The primary responsibility for this role is to analyse the data generated by the pentesting program and present the relevant findings to the stakeholders such as boards, security committees, risk and compliance teams, audit teams etc.

This role would usually perform analysis activities on a regular basis, as required by stakeholders. The purpose of performing this regularly is to understand how the security posture changes over time.

The usual activities this role would perform includes:

• Collecting reporting requirements based on the metrics for the pentesting program.

• Understanding and collecting available data as part of pentesting program. This data usually includes the following:

  • Vulnerabilities and their types and characteristics such as vulnerability classes, ratings & scores, timestamps, affected assets, etc.

  • Passed and failed test cases and their characteristics such as what was tested/not tested, timestamps, evidence, linkage to actual vulnerabilities found, etc.

  • Details for all assets which have and have not been tested and their characteristics such as when they were tested, who tested it, what was found, etc.

  • Perform analysis on captured data. This usually includes the following:

  • Comparison of business units & platforms against each other.

  • Tracking security posture for related groups of assets, by their business units, platforms, development teams, compliance requirements, portfolios, time periods, etc.

  • Deriving a mean time to remediate for vulnerabilities. This helps the organization understand whether they are getting better or worse at fixing known issues.

  • Determining patterns such as most common vulnerabilities across entire organization, business units etc.

• Gap analysis to determine what data as part of pentesting program is not captured and is required to be captured, in order to meet reporting requirements for the pentesting program.

This list of roles might not be exhaustive, however I consider these to be the most important roles and activities required to support a robust governance structure and its processes. These roles will help to establish and maintain the efficiency of a pentesting program, and to improve overall understanding of the organization security profile.

This concludes this series of articles. I strongly believe that the approach proposed in this series of articles would significantly improve any pentesting program; reduce the overall stress on everyone involved in pentesting; decrease organization's risk by helping to close vulnerabilities faster; and make it easier to focus limited resources.

For those who survived to the end of this article - I would love to learn what you think about the proposed structure. And all of the articles in general!