AI Pentesting

The Question Every SOC 2 Auditor Will Ask That Most Engineering Teams Can't Answer

Amartya | CodeAnt AI Code Review Platform
Sonali Sood

Founding GTM, CodeAnt AI

The auditor sits across from your engineering lead. The SOC 2 Type II audit is in its third day. The auditor pulls up the penetration testing section of the audit program.

"Can you show me evidence that exploitable vulnerabilities identified during your penetration test were corrected and that the corrections were tested?"

Not: "Did you do a penetration test?" Not: "Can you show me the pentest report?" The specific question is about the correction of exploitable vulnerabilities and the verification of those corrections.

The engineering lead can produce the penetration test report. Finding a retest report that specifically confirms each vulnerability was patched and verified in production is harder. Documentation showing the timeline from finding to remediation to verification, even harder. Evidence that the retest was conducted against the same environment as the original test, with the same methodology, almost no one has this.

The auditor marks the control as having an exception. The SOC 2 report goes out with a qualified opinion on CC7.1. The company's enterprise sales cycle just got harder.

This scenario plays out thousands of times per year across companies going through SOC 2 audits for the first time or upgrading from Type I to Type II. Not because companies aren't doing security testing, many are, but because they're doing security testing without understanding what SOC 2 specifically requires the testing to produce as evidence.

This guide covers every dimension of SOC 2 penetration testing requirements:

  • what the Trust Services Criteria actually say

  • what auditors actually look for (which is different from what the criteria literally say)

  • how to scope the test correctly for SOC 2 compliance

  • what the evidence package must contain

  • how Type I and Type II requirements differ

  • how to build a testing program that produces audit-ready evidence the first time

The SOC 2 Framework: What it is and Why Penetration Testing is Required

SOC 2 at a Technical Level

SOC 2 (Service Organization Control 2) is an auditing standard developed by the American Institute of Certified Public Accountants (AICPA). It evaluates whether a service organization's controls meet the Trust Services Criteria (TSC) relevant to their service commitments and system requirements.

SOC 2 is NOT a certification. It is an attestation, a formal opinion from an independent CPA firm that your controls meet the criteria. The distinction matters operationally: there is no SOC 2 body that grants certificates. Your auditor firm produces a report that contains their opinion on your controls. That report is what you share with customers.




The Five Trust Services Criteria Categories

SOC 2 audits can cover any combination of five Trust Services Criteria categories. Security (CC) is always required, the other four are optional:

TSC Category

Abbreviation

Always Required?

Penetration Testing Relevance

Security

CC

Yes

Directly required — CC6, CC7, CC8, CC9

Availability

A

Optional

Infrastructure testing, DDoS resilience

Processing Integrity

PI

Optional

Business logic testing, data validation

Confidentiality

C

Optional

Data access controls, encryption validation

Privacy

P

Optional

PII handling, access control verification

Most SaaS companies pursuing SOC 2 cover Security + Availability at minimum. Companies handling significant personal data or providing financial processing add Confidentiality, Privacy, or Processing Integrity.

Why Penetration Testing is Required

The SOC 2 framework does not use the words "penetration testing" in many places. This is where most companies go wrong, they look for an explicit requirement, do not find it stated plainly, and conclude pentesting is optional. It is not.

The requirement emerges from multiple TSC points, particularly CC6.6 (protection against external threats) and CC7.1 (detection of security events). The AICPA's supplemental guidance explicitly references penetration testing as an appropriate procedure for meeting these criteria. Every Big Four firm and most regional CPA firms with SOC 2 practices treat annual penetration testing as a table-stakes control for the Security category.

The question is not whether you need a pentest. It is whether the pentest you ran produces the specific evidence auditors are trained to look for.

The TSC Controls That Penetration Testing Directly Supports

The Complete Control Mapping

Understanding exactly which controls penetration testing evidence supports is the prerequisite for scoping the test correctly. A penetration test that doesn't generate evidence for the right controls is a compliance gap regardless of how thorough the technical testing was.

DETAILED SOC 2 TSC → PENETRATION TESTING EVIDENCE MAPPING:

CC6: Logical and Physical Access Controls
├── CC6.1: The entity implements logical access security software,
│   infrastructure, and architectures over protected information assets
│   to protect them from security events.
│
│   Penetration testing evidence required:
│   □ Authentication bypass testing results
│   □ Authorization control effectiveness (IDOR testing)
│   □ Privilege escalation attempts and outcomes
│   □ Session management security assessment
│   □ API access control verification
│   Finding implication: Authentication bypass or IDOR findings are
│   direct CC6.1 control failures
│
├── CC6.2: Prior to issuing system credentials and granting system access,
│   the entity registers and authorizes new internal and external users.
│
│   Penetration testing evidence required:
│   □ Test of account creation flows without proper authorization
│   □ Verification that guest/anonymous accounts have minimum permissions
│   □ API endpoint testing for unauthorized account creation
│   Finding implication: Unauthenticated account creation is CC6.2 failure
│
├── CC6.3: The entity authorizes, modifies, or removes access to data,
│   software, functions, and other protected information assets based on
│   approved and documented access requests and the results of access reviews.
│
│   Penetration testing evidence required:
│   □ Role-based access control verification
│   □ Testing that terminated employee accounts are deactivated
│   □ Privilege creep testing (can regular user access admin functions?)
│   □ Cross-tenant access testing (for multi-tenant systems)
│
├── CC6.6: The entity implements logical access security measures to protect
│   against threats from sources outside its system boundaries.
│
│   Penetration testing evidence required:
│   □ External penetration testing results
│   □ Attack surface enumeration
│   □ Testing of all externally accessible endpoints
│   □ Firewall/WAF bypass testing
│   □ VPN and remote access security testing
│   THIS IS WHERE EXTERNAL PENETRATION TESTING IS MOST DIRECTLY REQUIRED
│
└── CC6.7: The entity restricts the transmission, movement, and removal
    of information to authorized internal and external users and processes.

    Penetration testing evidence required:
    □ Data exfiltration testing (simulated)
    □ CORS policy verification
    □ API response filtering testing
    □ Encryption verification for data in transit

CC7: System Operations
├── CC7.1: [See above — detection and monitoring]
│
│   Penetration testing evidence required:
│   □ Complete penetration test report showing findings
│   □ Evidence of remediation for each finding
│   □ Retest report confirming remediation effectiveness
│   □ Timeline documentation (finding → fix → verification)
│   THIS IS THE PRIMARY CONTROL WHERE PENTEST EVIDENCE LIVES
│
├── CC7.2: [See above — system monitoring]

DETAILED SOC 2 TSC → PENETRATION TESTING EVIDENCE MAPPING:

CC6: Logical and Physical Access Controls
├── CC6.1: The entity implements logical access security software,
│   infrastructure, and architectures over protected information assets
│   to protect them from security events.
│
│   Penetration testing evidence required:
│   □ Authentication bypass testing results
│   □ Authorization control effectiveness (IDOR testing)
│   □ Privilege escalation attempts and outcomes
│   □ Session management security assessment
│   □ API access control verification
│   Finding implication: Authentication bypass or IDOR findings are
│   direct CC6.1 control failures
│
├── CC6.2: Prior to issuing system credentials and granting system access,
│   the entity registers and authorizes new internal and external users.
│
│   Penetration testing evidence required:
│   □ Test of account creation flows without proper authorization
│   □ Verification that guest/anonymous accounts have minimum permissions
│   □ API endpoint testing for unauthorized account creation
│   Finding implication: Unauthenticated account creation is CC6.2 failure
│
├── CC6.3: The entity authorizes, modifies, or removes access to data,
│   software, functions, and other protected information assets based on
│   approved and documented access requests and the results of access reviews.
│
│   Penetration testing evidence required:
│   □ Role-based access control verification
│   □ Testing that terminated employee accounts are deactivated
│   □ Privilege creep testing (can regular user access admin functions?)
│   □ Cross-tenant access testing (for multi-tenant systems)
│
├── CC6.6: The entity implements logical access security measures to protect
│   against threats from sources outside its system boundaries.
│
│   Penetration testing evidence required:
│   □ External penetration testing results
│   □ Attack surface enumeration
│   □ Testing of all externally accessible endpoints
│   □ Firewall/WAF bypass testing
│   □ VPN and remote access security testing
│   THIS IS WHERE EXTERNAL PENETRATION TESTING IS MOST DIRECTLY REQUIRED
│
└── CC6.7: The entity restricts the transmission, movement, and removal
    of information to authorized internal and external users and processes.

    Penetration testing evidence required:
    □ Data exfiltration testing (simulated)
    □ CORS policy verification
    □ API response filtering testing
    □ Encryption verification for data in transit

CC7: System Operations
├── CC7.1: [See above — detection and monitoring]
│
│   Penetration testing evidence required:
│   □ Complete penetration test report showing findings
│   □ Evidence of remediation for each finding
│   □ Retest report confirming remediation effectiveness
│   □ Timeline documentation (finding → fix → verification)
│   THIS IS THE PRIMARY CONTROL WHERE PENTEST EVIDENCE LIVES
│
├── CC7.2: [See above — system monitoring]

DETAILED SOC 2 TSC → PENETRATION TESTING EVIDENCE MAPPING:

CC6: Logical and Physical Access Controls
├── CC6.1: The entity implements logical access security software,
│   infrastructure, and architectures over protected information assets
│   to protect them from security events.
│
│   Penetration testing evidence required:
│   □ Authentication bypass testing results
│   □ Authorization control effectiveness (IDOR testing)
│   □ Privilege escalation attempts and outcomes
│   □ Session management security assessment
│   □ API access control verification
│   Finding implication: Authentication bypass or IDOR findings are
│   direct CC6.1 control failures
│
├── CC6.2: Prior to issuing system credentials and granting system access,
│   the entity registers and authorizes new internal and external users.
│
│   Penetration testing evidence required:
│   □ Test of account creation flows without proper authorization
│   □ Verification that guest/anonymous accounts have minimum permissions
│   □ API endpoint testing for unauthorized account creation
│   Finding implication: Unauthenticated account creation is CC6.2 failure
│
├── CC6.3: The entity authorizes, modifies, or removes access to data,
│   software, functions, and other protected information assets based on
│   approved and documented access requests and the results of access reviews.
│
│   Penetration testing evidence required:
│   □ Role-based access control verification
│   □ Testing that terminated employee accounts are deactivated
│   □ Privilege creep testing (can regular user access admin functions?)
│   □ Cross-tenant access testing (for multi-tenant systems)
│
├── CC6.6: The entity implements logical access security measures to protect
│   against threats from sources outside its system boundaries.
│
│   Penetration testing evidence required:
│   □ External penetration testing results
│   □ Attack surface enumeration
│   □ Testing of all externally accessible endpoints
│   □ Firewall/WAF bypass testing
│   □ VPN and remote access security testing
│   THIS IS WHERE EXTERNAL PENETRATION TESTING IS MOST DIRECTLY REQUIRED
│
└── CC6.7: The entity restricts the transmission, movement, and removal
    of information to authorized internal and external users and processes.

    Penetration testing evidence required:
    □ Data exfiltration testing (simulated)
    □ CORS policy verification
    □ API response filtering testing
    □ Encryption verification for data in transit

CC7: System Operations
├── CC7.1: [See above — detection and monitoring]
│
│   Penetration testing evidence required:
│   □ Complete penetration test report showing findings
│   □ Evidence of remediation for each finding
│   □ Retest report confirming remediation effectiveness
│   □ Timeline documentation (finding → fix → verification)
│   THIS IS THE PRIMARY CONTROL WHERE PENTEST EVIDENCE LIVES
│
├── CC7.2: [See above — system monitoring]

SOC 2 Type I vs Type II: Radically Different Evidence Requirements

What Changes Between Type I and Type II

The difference between Type I and Type II evidence requirements for penetration testing is more than a matter of degree, it's a fundamental difference in what the audit is measuring.


Type I

Type II

Core question

Are controls suitably designed?

Did controls operate effectively over time?

What it means for pentest

Do you have a penetration testing program, and does it appear appropriately designed?

Did your penetration testing program actually function and produce results that were acted upon throughout the period?

Type I: Minimum Evidence Required

Type I auditors are checking for the existence and design of your program, not whether it produced results. You need:

  • Penetration testing policy (written, approved, dated)

  • One completed penetration test report within the observation period

  • Evidence that the test was conducted by a qualified party

  • Finding list with severity classifications

  • Remediation plan, it doesn't have to be complete, just documented

What auditors will ask:

  • "Do you have a penetration testing policy?"

  • "When was your last penetration test?"

  • "Who conducted it?"

  • "What were the findings?"

  • "Do you have a plan to remediate them?"

What auditors will not ask for Type I:

  • "Show me evidence that the remediations actually worked"

  • "Show me the retest report"

  • "Demonstrate the testing covered all critical systems"

  • "Show me how findings feed into your risk register"

Type II: Minimum Evidence Required

Type II auditors are checking whether your program actually ran, produced findings, and those findings were acted on. The evidence bar is substantially higher:

  • Penetration testing policy (written, approved, current)

  • Penetration test report(s) covering the observation period

  • Evidence that critical and high findings were remediated

  • Retest report confirming remediation effectiveness, this is the most commonly missing item

  • Timeline evidence: finding date, remediation date, retest date

  • Remediation SLA documentation showing how fast findings were closed

  • Evidence that testing covered systems processing customer data

  • Evidence that testing scope was appropriate for the risk profile

  • Risk acceptance documentation for any findings not remediated

What auditors will ask for Type II:

  • "Show me the pentest report and the retest report"

  • "How long did it take to remediate each critical finding?"

  • "Which systems were in scope and why?"

  • "Show me that a critical finding was remediated and verified"

  • "What happened to finding X from the previous period?"

  • "How do findings feed into your risk management process?"

  • "What was your SLA for critical finding remediation?"

  • "Show me evidence you met that SLA"

  • "Were any findings accepted rather than remediated? Show me the risk acceptance"

Common Type II Failures

These are the patterns that produce exceptions in otherwise well-run programs:

Failure

Why it matters

Pentest report exists, retest report does not

Without a retest, there is no evidence remediations worked

Retest report does not cover all original findings

Partial retesting leaves open findings without disposition

Findings remediated in staging, not production

The audited system is production — staging evidence does not count

Remediation took longer than the stated SLA

Creates a gap between your documented policy and actual practice

Risk acceptances not formally documented

Accepted risks without documentation appear as unresolved findings

Testing was done on a different system version than production

The evidence does not reflect the system the audit covers

Gap between testing periods exceeds 12 months

Leaves portions of the observation period without testing coverage

The Observation Period and Testing Frequency

SOC 2 Type II has a minimum observation period of six months, per AICPA guidance. In practice, enterprise buyers expect twelve months. Your penetration testing program needs to provide evidence across that full window, not just a single point within it.

Three Ways to Structure Your Testing

Each option below is a valid approach. The right choice depends on how fast your attack surface changes and how much coverage your auditor expects.


Option A

Option B

Option C

Cadence

Annual

Semi-annual

Continuous

Tests per period

1 test + 1 retest

2 tests + 2 retests

Monthly assessments

Coverage

System as it existed on test date

Two snapshots across the period

Full observation period

Best for

Stable systems, low-change environments

Most SaaS teams

High-velocity development

Evidence strength

Acceptable

Stronger

Strongest

Option A: Annual testing

One penetration test at the start or middle of the observation period, followed by one retest after remediation. The risk: anything that changes in the months after the test date is untested. Auditors may ask whether the test was conducted timely relative to the period, especially if it happened early in the year and significant infrastructure changes followed.

Option B: Semi-annual testing

A penetration test at the start of the period, a retest after first-period remediations, a second test at the six-month mark, and a retest after second-period remediations. This produces two coverage snapshots and is substantially stronger evidence for any team shipping frequently. This is the approach most commonly recommended for SaaS companies.

Option C: Continuous testing with monthly reporting

Ongoing testing aligned to deployment cadence, with monthly comprehensive assessments and continuous remediation verification. This provides complete observation period coverage and is the strongest possible evidence that controls operated effectively throughout. The tradeoff is operational overhead.

Timing Rules

These are the rules that catch companies off guard most often:

  • The test must occur within the observation period, not before it. A test conducted two months before the audit period started provides evidence of prior-period controls, it does not count toward the current period.

  • For twelve-month periods, at least one test must occur within those twelve months.

  • The retest must occur after remediation and also within the period.

  • The gap between the last test and the end of the observation period should not exceed six months. Auditor judgment applies, but anything beyond six months is a conversation.

The Most Common Timing Mistake

Many companies run their penetration test in Q1 for a Q1 through Q4 audit period and never schedule a follow-up. When the auditor arrives, they ask: "What security testing covers Q3 and Q4?"

The answer, nothing, is a gap in the evidence record. It does not cause an automatic qualified opinion, but it requires an explanation, and the explanation is rarely satisfying. The fix is simple: schedule a second test or at minimum a targeted retest at the six-month mark.

Part 4: Scoping the SOC 2 Penetration Test: What Must Be Included

The Scoping Principle: Follow the Data

The most important scoping principle for SOC 2 penetration testing: test everything that touches, stores, processes, or transmits the data your SOC 2 is about.

SOC 2 is fundamentally about the security of your service delivery, the systems and processes that handle your customers' data. The penetration test must cover those systems. Testing systems that don't touch customer data while excluding the systems that do is a scoping failure that auditors will identify.

def scope_soc2_penetration_test(
    organization_profile: dict,
    tsc_categories: list,
    customer_data_flows: list
) -> dict:
    """
    Generate appropriate SOC 2 penetration test scope based on:
    - Which TSC categories are in scope for the audit
    - How customer data flows through the system
    - Risk profile of individual system components
    """

    scope = {
        'required_components': [],  # Must test for SOC 2 compliance
        'recommended_components': [],  # Strong recommendation
        'optional_components': [],  # At discretion based on risk
        'explicitly_excluded': [],  # Never test (third-party, etc.)
        'rationale': {}
    }

    # ═══════════════════════════════════════════════════════
    # SECURITY (CC) — ALWAYS IN SCOPE
    # ═══════════════════════════════════════════════════════

    # CC6.6 requires testing of all external-facing attack surface
    scope['required_components'].append({
        'component': 'External web application',
        'tsc_control': 'CC6.6',
        'test_types': [
            'Authentication bypass testing',
            'Authorization and access control testing',
            'Session management testing',
            'Input validation (injection testing)',
            'Business logic testing',
            'API security testing'
        ],
        'evidence_required': [
            'Findings report with CVSS scores',
            'Proof of concept for all confirmed findings',
            'Remediation evidence per finding'
        ]
    })

    scope['required_components'].append({
        'component': 'API layer (all authenticated endpoints)',
        'tsc_control': 'CC6.1, CC6.3, CC6.6',
        'test_types': [
            'Authentication enforcement',
            'Authorization (IDOR, BOLA)',
            'Rate limiting',
            'Input validation',
            'JWT/token security',
            'Mass assignment',
            'API versioning security'
        ],
        'evidence_required': [
            'Complete endpoint coverage documentation',
            'Authentication test results',
            'Authorization control verification'
        ]
    })

    scope['required_components'].append({
        'component': 'Network infrastructure (externally accessible)',
        'tsc_control': 'CC6.6',
        'test_types': [
            'Port scanning and service enumeration',
            'Service vulnerability assessment',
            'Firewall rule testing',
            'SSL/TLS configuration testing',
            'Open redirect testing',
            'CORS policy testing'
        ],
        'evidence_required': [
            'Network scan results',
            'Service enumeration report',
            'TLS configuration report'
        ]
    })

    # Customer data systems are always required
    for data_flow in customer_data_flows:
        if data_flow.get('contains_pii') or data_flow.get('is_primary_storage'):
            scope['required_components'].append({
                'component': f"Customer data system: {data_flow['name']}",
                'tsc_control': 'CC6.1, CC6.3, C1.1 (if confidentiality in scope)',
                'test_types': [
                    'Data access control testing',
                    'Unauthorized access attempts',
                    'SQL/NoSQL injection testing',
                    'Data exposure testing'
                ],
                'evidence_required': [
                    'Access control test results',
                    'Data exposure findings documentation'
                ]
            })

    # ═══════════════════════════════════════════════════════
    # CLOUD INFRASTRUCTURE — REQUIRED IF CLOUD-HOSTED
    # ═══════════════════════════════════════════════════════

    if organization_profile.get('cloud_provider'):
        scope['required_components'].append({
            'component': 'Cloud infrastructure security',
            'tsc_control': 'CC6.6, CC8.1',
            'test_types': [
                'IAM role and permission review',
                'S3/GCS/Blob storage access control',
                'Security group and network ACL review',
                'Secrets management review',
                'Logging and monitoring configuration'
            ],
            'evidence_required': [
                'Cloud security posture assessment',
                'IAM privilege audit results',
                'Storage access control verification'
            ]
        })

    # ═══════════════════════════════════════════════════════
    # AVAILABILITY (A1) — IF IN SCOPE
    # ═══════════════════════════════════════════════════════

    if 'availability' in tsc_categories:
        scope['recommended_components'].append({
            'component': 'Availability and resilience testing',
            'tsc_control': 'A1.2',
            'test_types': [
                'Rate limiting effectiveness',
                'Resource exhaustion testing (low intensity)',
                'Failover testing',
                'Recovery procedure testing'
            ],
            'note': 'DoS testing requires separate authorization and careful scoping'
        })

    # ═══════════════════════════════════════════════════════
    # WHAT SHOULD NOT BE IN SCOPE
    # ═══════════════════════════════════════════════════════

    scope['explicitly_excluded'] = [
        {
            'component': 'Third-party SaaS systems (Salesforce, Workday, etc.)',
            'reason': 'These vendors have their own SOC 2 reports — test your integration, not their system',
            'correct_approach': 'Review vendor SOC 2 reports as part of vendor management (CC9.2)'
        },
        {
            'component': 'Payment processor infrastructure (Stripe, Braintree)',
            'reason': 'PCI-DSS scoping exemption; testing the processor directly is not permitted',
            'correct_approach': 'Test your integration security, not the processor itself'
        },
        {
            'component': 'Other customers\\' data or systems',
            'reason': 'Never in scope — testing would itself be a SOC 2 control failure',
            'correct_approach': 'Use test accounts with synthetic data'
        }
    ]

    return scope
def scope_soc2_penetration_test(
    organization_profile: dict,
    tsc_categories: list,
    customer_data_flows: list
) -> dict:
    """
    Generate appropriate SOC 2 penetration test scope based on:
    - Which TSC categories are in scope for the audit
    - How customer data flows through the system
    - Risk profile of individual system components
    """

    scope = {
        'required_components': [],  # Must test for SOC 2 compliance
        'recommended_components': [],  # Strong recommendation
        'optional_components': [],  # At discretion based on risk
        'explicitly_excluded': [],  # Never test (third-party, etc.)
        'rationale': {}
    }

    # ═══════════════════════════════════════════════════════
    # SECURITY (CC) — ALWAYS IN SCOPE
    # ═══════════════════════════════════════════════════════

    # CC6.6 requires testing of all external-facing attack surface
    scope['required_components'].append({
        'component': 'External web application',
        'tsc_control': 'CC6.6',
        'test_types': [
            'Authentication bypass testing',
            'Authorization and access control testing',
            'Session management testing',
            'Input validation (injection testing)',
            'Business logic testing',
            'API security testing'
        ],
        'evidence_required': [
            'Findings report with CVSS scores',
            'Proof of concept for all confirmed findings',
            'Remediation evidence per finding'
        ]
    })

    scope['required_components'].append({
        'component': 'API layer (all authenticated endpoints)',
        'tsc_control': 'CC6.1, CC6.3, CC6.6',
        'test_types': [
            'Authentication enforcement',
            'Authorization (IDOR, BOLA)',
            'Rate limiting',
            'Input validation',
            'JWT/token security',
            'Mass assignment',
            'API versioning security'
        ],
        'evidence_required': [
            'Complete endpoint coverage documentation',
            'Authentication test results',
            'Authorization control verification'
        ]
    })

    scope['required_components'].append({
        'component': 'Network infrastructure (externally accessible)',
        'tsc_control': 'CC6.6',
        'test_types': [
            'Port scanning and service enumeration',
            'Service vulnerability assessment',
            'Firewall rule testing',
            'SSL/TLS configuration testing',
            'Open redirect testing',
            'CORS policy testing'
        ],
        'evidence_required': [
            'Network scan results',
            'Service enumeration report',
            'TLS configuration report'
        ]
    })

    # Customer data systems are always required
    for data_flow in customer_data_flows:
        if data_flow.get('contains_pii') or data_flow.get('is_primary_storage'):
            scope['required_components'].append({
                'component': f"Customer data system: {data_flow['name']}",
                'tsc_control': 'CC6.1, CC6.3, C1.1 (if confidentiality in scope)',
                'test_types': [
                    'Data access control testing',
                    'Unauthorized access attempts',
                    'SQL/NoSQL injection testing',
                    'Data exposure testing'
                ],
                'evidence_required': [
                    'Access control test results',
                    'Data exposure findings documentation'
                ]
            })

    # ═══════════════════════════════════════════════════════
    # CLOUD INFRASTRUCTURE — REQUIRED IF CLOUD-HOSTED
    # ═══════════════════════════════════════════════════════

    if organization_profile.get('cloud_provider'):
        scope['required_components'].append({
            'component': 'Cloud infrastructure security',
            'tsc_control': 'CC6.6, CC8.1',
            'test_types': [
                'IAM role and permission review',
                'S3/GCS/Blob storage access control',
                'Security group and network ACL review',
                'Secrets management review',
                'Logging and monitoring configuration'
            ],
            'evidence_required': [
                'Cloud security posture assessment',
                'IAM privilege audit results',
                'Storage access control verification'
            ]
        })

    # ═══════════════════════════════════════════════════════
    # AVAILABILITY (A1) — IF IN SCOPE
    # ═══════════════════════════════════════════════════════

    if 'availability' in tsc_categories:
        scope['recommended_components'].append({
            'component': 'Availability and resilience testing',
            'tsc_control': 'A1.2',
            'test_types': [
                'Rate limiting effectiveness',
                'Resource exhaustion testing (low intensity)',
                'Failover testing',
                'Recovery procedure testing'
            ],
            'note': 'DoS testing requires separate authorization and careful scoping'
        })

    # ═══════════════════════════════════════════════════════
    # WHAT SHOULD NOT BE IN SCOPE
    # ═══════════════════════════════════════════════════════

    scope['explicitly_excluded'] = [
        {
            'component': 'Third-party SaaS systems (Salesforce, Workday, etc.)',
            'reason': 'These vendors have their own SOC 2 reports — test your integration, not their system',
            'correct_approach': 'Review vendor SOC 2 reports as part of vendor management (CC9.2)'
        },
        {
            'component': 'Payment processor infrastructure (Stripe, Braintree)',
            'reason': 'PCI-DSS scoping exemption; testing the processor directly is not permitted',
            'correct_approach': 'Test your integration security, not the processor itself'
        },
        {
            'component': 'Other customers\\' data or systems',
            'reason': 'Never in scope — testing would itself be a SOC 2 control failure',
            'correct_approach': 'Use test accounts with synthetic data'
        }
    ]

    return scope
def scope_soc2_penetration_test(
    organization_profile: dict,
    tsc_categories: list,
    customer_data_flows: list
) -> dict:
    """
    Generate appropriate SOC 2 penetration test scope based on:
    - Which TSC categories are in scope for the audit
    - How customer data flows through the system
    - Risk profile of individual system components
    """

    scope = {
        'required_components': [],  # Must test for SOC 2 compliance
        'recommended_components': [],  # Strong recommendation
        'optional_components': [],  # At discretion based on risk
        'explicitly_excluded': [],  # Never test (third-party, etc.)
        'rationale': {}
    }

    # ═══════════════════════════════════════════════════════
    # SECURITY (CC) — ALWAYS IN SCOPE
    # ═══════════════════════════════════════════════════════

    # CC6.6 requires testing of all external-facing attack surface
    scope['required_components'].append({
        'component': 'External web application',
        'tsc_control': 'CC6.6',
        'test_types': [
            'Authentication bypass testing',
            'Authorization and access control testing',
            'Session management testing',
            'Input validation (injection testing)',
            'Business logic testing',
            'API security testing'
        ],
        'evidence_required': [
            'Findings report with CVSS scores',
            'Proof of concept for all confirmed findings',
            'Remediation evidence per finding'
        ]
    })

    scope['required_components'].append({
        'component': 'API layer (all authenticated endpoints)',
        'tsc_control': 'CC6.1, CC6.3, CC6.6',
        'test_types': [
            'Authentication enforcement',
            'Authorization (IDOR, BOLA)',
            'Rate limiting',
            'Input validation',
            'JWT/token security',
            'Mass assignment',
            'API versioning security'
        ],
        'evidence_required': [
            'Complete endpoint coverage documentation',
            'Authentication test results',
            'Authorization control verification'
        ]
    })

    scope['required_components'].append({
        'component': 'Network infrastructure (externally accessible)',
        'tsc_control': 'CC6.6',
        'test_types': [
            'Port scanning and service enumeration',
            'Service vulnerability assessment',
            'Firewall rule testing',
            'SSL/TLS configuration testing',
            'Open redirect testing',
            'CORS policy testing'
        ],
        'evidence_required': [
            'Network scan results',
            'Service enumeration report',
            'TLS configuration report'
        ]
    })

    # Customer data systems are always required
    for data_flow in customer_data_flows:
        if data_flow.get('contains_pii') or data_flow.get('is_primary_storage'):
            scope['required_components'].append({
                'component': f"Customer data system: {data_flow['name']}",
                'tsc_control': 'CC6.1, CC6.3, C1.1 (if confidentiality in scope)',
                'test_types': [
                    'Data access control testing',
                    'Unauthorized access attempts',
                    'SQL/NoSQL injection testing',
                    'Data exposure testing'
                ],
                'evidence_required': [
                    'Access control test results',
                    'Data exposure findings documentation'
                ]
            })

    # ═══════════════════════════════════════════════════════
    # CLOUD INFRASTRUCTURE — REQUIRED IF CLOUD-HOSTED
    # ═══════════════════════════════════════════════════════

    if organization_profile.get('cloud_provider'):
        scope['required_components'].append({
            'component': 'Cloud infrastructure security',
            'tsc_control': 'CC6.6, CC8.1',
            'test_types': [
                'IAM role and permission review',
                'S3/GCS/Blob storage access control',
                'Security group and network ACL review',
                'Secrets management review',
                'Logging and monitoring configuration'
            ],
            'evidence_required': [
                'Cloud security posture assessment',
                'IAM privilege audit results',
                'Storage access control verification'
            ]
        })

    # ═══════════════════════════════════════════════════════
    # AVAILABILITY (A1) — IF IN SCOPE
    # ═══════════════════════════════════════════════════════

    if 'availability' in tsc_categories:
        scope['recommended_components'].append({
            'component': 'Availability and resilience testing',
            'tsc_control': 'A1.2',
            'test_types': [
                'Rate limiting effectiveness',
                'Resource exhaustion testing (low intensity)',
                'Failover testing',
                'Recovery procedure testing'
            ],
            'note': 'DoS testing requires separate authorization and careful scoping'
        })

    # ═══════════════════════════════════════════════════════
    # WHAT SHOULD NOT BE IN SCOPE
    # ═══════════════════════════════════════════════════════

    scope['explicitly_excluded'] = [
        {
            'component': 'Third-party SaaS systems (Salesforce, Workday, etc.)',
            'reason': 'These vendors have their own SOC 2 reports — test your integration, not their system',
            'correct_approach': 'Review vendor SOC 2 reports as part of vendor management (CC9.2)'
        },
        {
            'component': 'Payment processor infrastructure (Stripe, Braintree)',
            'reason': 'PCI-DSS scoping exemption; testing the processor directly is not permitted',
            'correct_approach': 'Test your integration security, not the processor itself'
        },
        {
            'component': 'Other customers\\' data or systems',
            'reason': 'Never in scope — testing would itself be a SOC 2 control failure',
            'correct_approach': 'Use test accounts with synthetic data'
        }
    ]

    return scope

The SOC 2 Scope Decision Matrix

System Component

Always In Scope?

TSC Controls

Evidence Required

Notes

Public-facing web application

Yes

CC6.6

Full pentest report + retest

Primary requirement

Authenticated API (all endpoints)

Yes

CC6.1, CC6.3

Coverage map + auth test results

Document endpoint count

Admin panel/back office

Yes

CC6.1, CC6.3

Privilege escalation test results

High-value target

Customer data database

Yes

CC6.1, C1.1

Access control verification

Often tested via app

Cloud IAM and permissions

Yes (cloud-hosted)

CC6.6, CC8.1

IAM privilege audit

Cloud security review

Internal admin tools

Recommended

CC6.3

Internal access control testing

If accessible by employees

CI/CD pipeline

Recommended

CC8.1

Pipeline secrets review

High supply chain risk

Mobile applications

Recommended

CC6.6

Mobile app pentest

If customer-facing

Internal network (if on-prem)

Yes (on-prem)

CC6.6

Internal network assessment

N/A for pure SaaS

Staging/QA environments

No

N/A

N/A

Test production

Third-party SaaS tools

No

CC9.2

Vendor SOC 2 review

Review their reports

Employee workstations

No

CC6.6

Endpoint management evidence

Different control

The SOC 2 Penetration Test Report: What Auditors Need to See

A technically excellent penetration test report and a SOC 2-compliant penetration test report are not the same thing. Security reports are designed to tell engineers what to fix. SOC 2 evidence reports are designed to tell auditors that controls are working. These are different communication goals that require different document structures.

ELEMENTS REQUIRED IN A SOC 2 PENETRATION TEST REPORT:

1. TESTING FIRM QUALIFICATIONS
   ├── Firm name and relevant certifications (CREST, OSCP, CISSP, etc.)
   ├── Individual tester qualifications
   ├── Years of experience in the relevant domain
   └── Independence declaration (no conflicts of interest)

   Why auditors need this: CC7.1 requires "qualified" personnel conduct
   vulnerability assessment. The report must demonstrate qualification.

2. SCOPE STATEMENT (EXPLICIT)
   ├── Exact systems tested (IPs, URLs, account IDs)
   ├── Systems NOT tested and why (with acknowledgment of limitations)
   ├── Testing methodology (black box, gray box, white box)
   ├── Testing dates and duration
   └── Testing environment (production confirmed explicitly)

   Why auditors need this: Auditor must verify scope covered
   customer-data-processing systems. "Our web application" is insufficient.
   The report must specifically name the systems.

3. METHODOLOGY DESCRIPTION
   ├── Testing phases executed
   ├── Standards followed (OWASP, PTES, NIST SP 800-115)
   ├── Tools used (Burp Suite, Nmap, Metasploit, etc.)
   └── Manual vs automated component breakdown

   Why auditors need this: Demonstrates systematic coverage,
   not opportunistic testing.

4. FINDINGS — REQUIRED STRUCTURE PER FINDING
   ├── Unique finding identifier (for retest tracking)
   ├── Finding title
   ├── CVSS 4.0 score AND vector string
   ├── Severity classification (Critical/High/Medium/Low/Informational)
   ├── Affected system(s) — specific, not "the web application"
   ├── Technical description (enough for auditor to understand the issue)
   ├── Proof of concept (screenshot, HTTP request/response, command output)
   ├── Business impact (in plain language for the auditor)
   ├── Root cause classification (authentication, authorization, configuration, etc.)
   ├── Remediation recommendation (specific, actionable)
   └── Compliance mapping (which CC or other criterion this affects)

   SOC 2 specific addition: REMEDIATION STATUS FIELD
   Status: [Open / Remediated / Accepted Risk / Transferred]
   This field tracks the lifecycle for audit evidence purposes.

5. EXECUTIVE SUMMARY (NON-TECHNICAL)
   ├── Overall security posture assessment
   ├── Key risk areas identified
   ├── Highest severity findings summary
   └── Trend analysis (if re-engagement)

   Why auditors need this: Management review evidence for CC5.3

6. FINDINGS SUMMARY TABLE
   Required columns:
   Finding ID | Title | CVSS | Severity | Status | Remediation Date | Retest Date

   Why auditors need this: At-a-glance evidence for CC7.1
   Auditors will scan this table before reading any findings detail.
   If it's absent, they'll ask for it separately and note the gap.

7. COVERAGE ATTESTATION
   A statement from the testing firm that the scope covered
   [specific systems] and that limitations are documented.

   Example: "This assessment covered all external-facing systems
   processing customer data as of [date]. The systems listed in
   Section 3 represent the complete scope of customer-data-processing
   infrastructure as confirmed by [Organization]

ELEMENTS REQUIRED IN A SOC 2 PENETRATION TEST REPORT:

1. TESTING FIRM QUALIFICATIONS
   ├── Firm name and relevant certifications (CREST, OSCP, CISSP, etc.)
   ├── Individual tester qualifications
   ├── Years of experience in the relevant domain
   └── Independence declaration (no conflicts of interest)

   Why auditors need this: CC7.1 requires "qualified" personnel conduct
   vulnerability assessment. The report must demonstrate qualification.

2. SCOPE STATEMENT (EXPLICIT)
   ├── Exact systems tested (IPs, URLs, account IDs)
   ├── Systems NOT tested and why (with acknowledgment of limitations)
   ├── Testing methodology (black box, gray box, white box)
   ├── Testing dates and duration
   └── Testing environment (production confirmed explicitly)

   Why auditors need this: Auditor must verify scope covered
   customer-data-processing systems. "Our web application" is insufficient.
   The report must specifically name the systems.

3. METHODOLOGY DESCRIPTION
   ├── Testing phases executed
   ├── Standards followed (OWASP, PTES, NIST SP 800-115)
   ├── Tools used (Burp Suite, Nmap, Metasploit, etc.)
   └── Manual vs automated component breakdown

   Why auditors need this: Demonstrates systematic coverage,
   not opportunistic testing.

4. FINDINGS — REQUIRED STRUCTURE PER FINDING
   ├── Unique finding identifier (for retest tracking)
   ├── Finding title
   ├── CVSS 4.0 score AND vector string
   ├── Severity classification (Critical/High/Medium/Low/Informational)
   ├── Affected system(s) — specific, not "the web application"
   ├── Technical description (enough for auditor to understand the issue)
   ├── Proof of concept (screenshot, HTTP request/response, command output)
   ├── Business impact (in plain language for the auditor)
   ├── Root cause classification (authentication, authorization, configuration, etc.)
   ├── Remediation recommendation (specific, actionable)
   └── Compliance mapping (which CC or other criterion this affects)

   SOC 2 specific addition: REMEDIATION STATUS FIELD
   Status: [Open / Remediated / Accepted Risk / Transferred]
   This field tracks the lifecycle for audit evidence purposes.

5. EXECUTIVE SUMMARY (NON-TECHNICAL)
   ├── Overall security posture assessment
   ├── Key risk areas identified
   ├── Highest severity findings summary
   └── Trend analysis (if re-engagement)

   Why auditors need this: Management review evidence for CC5.3

6. FINDINGS SUMMARY TABLE
   Required columns:
   Finding ID | Title | CVSS | Severity | Status | Remediation Date | Retest Date

   Why auditors need this: At-a-glance evidence for CC7.1
   Auditors will scan this table before reading any findings detail.
   If it's absent, they'll ask for it separately and note the gap.

7. COVERAGE ATTESTATION
   A statement from the testing firm that the scope covered
   [specific systems] and that limitations are documented.

   Example: "This assessment covered all external-facing systems
   processing customer data as of [date]. The systems listed in
   Section 3 represent the complete scope of customer-data-processing
   infrastructure as confirmed by [Organization]

ELEMENTS REQUIRED IN A SOC 2 PENETRATION TEST REPORT:

1. TESTING FIRM QUALIFICATIONS
   ├── Firm name and relevant certifications (CREST, OSCP, CISSP, etc.)
   ├── Individual tester qualifications
   ├── Years of experience in the relevant domain
   └── Independence declaration (no conflicts of interest)

   Why auditors need this: CC7.1 requires "qualified" personnel conduct
   vulnerability assessment. The report must demonstrate qualification.

2. SCOPE STATEMENT (EXPLICIT)
   ├── Exact systems tested (IPs, URLs, account IDs)
   ├── Systems NOT tested and why (with acknowledgment of limitations)
   ├── Testing methodology (black box, gray box, white box)
   ├── Testing dates and duration
   └── Testing environment (production confirmed explicitly)

   Why auditors need this: Auditor must verify scope covered
   customer-data-processing systems. "Our web application" is insufficient.
   The report must specifically name the systems.

3. METHODOLOGY DESCRIPTION
   ├── Testing phases executed
   ├── Standards followed (OWASP, PTES, NIST SP 800-115)
   ├── Tools used (Burp Suite, Nmap, Metasploit, etc.)
   └── Manual vs automated component breakdown

   Why auditors need this: Demonstrates systematic coverage,
   not opportunistic testing.

4. FINDINGS — REQUIRED STRUCTURE PER FINDING
   ├── Unique finding identifier (for retest tracking)
   ├── Finding title
   ├── CVSS 4.0 score AND vector string
   ├── Severity classification (Critical/High/Medium/Low/Informational)
   ├── Affected system(s) — specific, not "the web application"
   ├── Technical description (enough for auditor to understand the issue)
   ├── Proof of concept (screenshot, HTTP request/response, command output)
   ├── Business impact (in plain language for the auditor)
   ├── Root cause classification (authentication, authorization, configuration, etc.)
   ├── Remediation recommendation (specific, actionable)
   └── Compliance mapping (which CC or other criterion this affects)

   SOC 2 specific addition: REMEDIATION STATUS FIELD
   Status: [Open / Remediated / Accepted Risk / Transferred]
   This field tracks the lifecycle for audit evidence purposes.

5. EXECUTIVE SUMMARY (NON-TECHNICAL)
   ├── Overall security posture assessment
   ├── Key risk areas identified
   ├── Highest severity findings summary
   └── Trend analysis (if re-engagement)

   Why auditors need this: Management review evidence for CC5.3

6. FINDINGS SUMMARY TABLE
   Required columns:
   Finding ID | Title | CVSS | Severity | Status | Remediation Date | Retest Date

   Why auditors need this: At-a-glance evidence for CC7.1
   Auditors will scan this table before reading any findings detail.
   If it's absent, they'll ask for it separately and note the gap.

7. COVERAGE ATTESTATION
   A statement from the testing firm that the scope covered
   [specific systems] and that limitations are documented.

   Example: "This assessment covered all external-facing systems
   processing customer data as of [date]. The systems listed in
   Section 3 represent the complete scope of customer-data-processing
   infrastructure as confirmed by [Organization]

The Retest Report: The Most Underspecified Document

The retest report is the document auditors are most likely to cite as missing or insufficient. Here is the minimum structure:

SOC 2 RETEST REPORT REQUIREMENTS:

HEADER INFORMATION:
  Original Engagement Reference: [ID]
  Original Test Date: [DATE]
  Retest Date: [DATE]
  Testing Environment: Production (MUST BE EXPLICIT)
  Production Version at Retest: [Version/Commit/Build ID]
  Retest Performed By: [Names — same firm as original]

EXECUTIVE SUMMARY:
  Total original findings: N
  Findings retested: N (must equal total — cannot skip)
  Status: X Remediated, Y Partially Remediated, Z Not Remediated

CVSS DELTA:
  Original aggregate risk: [Score]
  Post-remediation aggregate risk: [Score]
  Risk reduction: [Percentage]

  This section is increasingly expected by sophisticated auditors
  as a quantitative measure of control improvement.

PER-FINDING RETEST RESULTS:
  Finding ID: [ID matching original report]
  Original Finding: [Title]
  Original CVSS: [Score]
  Remediation Applied: [Summary of what was changed]
  Deployment Confirmation: [Evidence that fix is in production]
  Retest Method: [What was tested to verify]
  Retest Result: [What was observed]
  Status: REMEDIATED | PARTIALLY_REMEDIATED | NOT_REMEDIATED
  New CVSS: [Score if not fully remediated, 0.0 if remediated]

  Auditor note: The retest result must show the SPECIFIC HTTP request
  or command that was executed and the SPECIFIC response observed.
  "The finding was confirmed remediated" is insufficient.
  "GET /api/users/43 returned 403 Forbidden (previously returned 200 OK
   with user data from account 43)" is sufficient.

OPEN FINDINGS SECTION:
  For any finding NOT remediated:
  Finding ID: [ID]
  Reason Not Remediated: [Technical explanation]
  Risk Acceptance: [Documented? Yes/No]
  Risk Owner: [Name and title]
  Target Remediation Date: [Date]
  Compensating Controls: [What mitigates risk in the interim]

SOC 2 RETEST REPORT REQUIREMENTS:

HEADER INFORMATION:
  Original Engagement Reference: [ID]
  Original Test Date: [DATE]
  Retest Date: [DATE]
  Testing Environment: Production (MUST BE EXPLICIT)
  Production Version at Retest: [Version/Commit/Build ID]
  Retest Performed By: [Names — same firm as original]

EXECUTIVE SUMMARY:
  Total original findings: N
  Findings retested: N (must equal total — cannot skip)
  Status: X Remediated, Y Partially Remediated, Z Not Remediated

CVSS DELTA:
  Original aggregate risk: [Score]
  Post-remediation aggregate risk: [Score]
  Risk reduction: [Percentage]

  This section is increasingly expected by sophisticated auditors
  as a quantitative measure of control improvement.

PER-FINDING RETEST RESULTS:
  Finding ID: [ID matching original report]
  Original Finding: [Title]
  Original CVSS: [Score]
  Remediation Applied: [Summary of what was changed]
  Deployment Confirmation: [Evidence that fix is in production]
  Retest Method: [What was tested to verify]
  Retest Result: [What was observed]
  Status: REMEDIATED | PARTIALLY_REMEDIATED | NOT_REMEDIATED
  New CVSS: [Score if not fully remediated, 0.0 if remediated]

  Auditor note: The retest result must show the SPECIFIC HTTP request
  or command that was executed and the SPECIFIC response observed.
  "The finding was confirmed remediated" is insufficient.
  "GET /api/users/43 returned 403 Forbidden (previously returned 200 OK
   with user data from account 43)" is sufficient.

OPEN FINDINGS SECTION:
  For any finding NOT remediated:
  Finding ID: [ID]
  Reason Not Remediated: [Technical explanation]
  Risk Acceptance: [Documented? Yes/No]
  Risk Owner: [Name and title]
  Target Remediation Date: [Date]
  Compensating Controls: [What mitigates risk in the interim]

SOC 2 RETEST REPORT REQUIREMENTS:

HEADER INFORMATION:
  Original Engagement Reference: [ID]
  Original Test Date: [DATE]
  Retest Date: [DATE]
  Testing Environment: Production (MUST BE EXPLICIT)
  Production Version at Retest: [Version/Commit/Build ID]
  Retest Performed By: [Names — same firm as original]

EXECUTIVE SUMMARY:
  Total original findings: N
  Findings retested: N (must equal total — cannot skip)
  Status: X Remediated, Y Partially Remediated, Z Not Remediated

CVSS DELTA:
  Original aggregate risk: [Score]
  Post-remediation aggregate risk: [Score]
  Risk reduction: [Percentage]

  This section is increasingly expected by sophisticated auditors
  as a quantitative measure of control improvement.

PER-FINDING RETEST RESULTS:
  Finding ID: [ID matching original report]
  Original Finding: [Title]
  Original CVSS: [Score]
  Remediation Applied: [Summary of what was changed]
  Deployment Confirmation: [Evidence that fix is in production]
  Retest Method: [What was tested to verify]
  Retest Result: [What was observed]
  Status: REMEDIATED | PARTIALLY_REMEDIATED | NOT_REMEDIATED
  New CVSS: [Score if not fully remediated, 0.0 if remediated]

  Auditor note: The retest result must show the SPECIFIC HTTP request
  or command that was executed and the SPECIFIC response observed.
  "The finding was confirmed remediated" is insufficient.
  "GET /api/users/43 returned 403 Forbidden (previously returned 200 OK
   with user data from account 43)" is sufficient.

OPEN FINDINGS SECTION:
  For any finding NOT remediated:
  Finding ID: [ID]
  Reason Not Remediated: [Technical explanation]
  Risk Acceptance: [Documented? Yes/No]
  Risk Owner: [Name and title]
  Target Remediation Date: [Date]
  Compensating Controls: [What mitigates risk in the interim]

Building the Complete SOC 2 Evidence Package

The Evidence Package Structure

The evidence package is the collection of documents you provide to auditors. It must be organized, navigable, and complete:

SOC 2 PENETRATION TESTING EVIDENCE PACKAGE STRUCTURE:

FOLDER 1: POLICY AND PROCEDURE
  ├── penetration_testing_policy_v[X]_[DATE].pdf
  │     Required contents:
  │     □ Testing frequency (annual minimum)
  │     □ Who is responsible for commissioning tests
  │     □ Scope definition process
  │     □ Qualified vendor requirements
  │     □ Finding SLA (critical, high, medium, low)
  │     □ Remediation verification requirements
  │     □ Risk acceptance process
  │     □ Evidence retention period
  │     □ Approval signatures (CISO or equivalent)
  │
  ├── vulnerability_management_policy_v[X]_[DATE].pdf
  │     The SLA table that auditors will test:
  │     Critical: 48 hours
  │     High: 7 days
  │     Medium: 30 days
  │     Low: 90 days
  │     (Your actual SLAs — must be met, not aspirational)
  │
  └── penetration_testing_vendor_qualification_criteria.pdf
        Evidence that the firm used meets your qualification standards

FOLDER 2: CURRENT PERIOD PENETRATION TEST
  ├── engagement_authorization_letter_signed.pdf
  │     Proof of authorized testing
  │
  ├── penetration_test_report_[FIRM]_[DATE].pdf
  │     The complete finding report
  │
  ├── pentest_scope_confirmation.pdf
  │     Confirmation that scope covered customer data systems
  │
  └── pentest_executive_briefing_[DATE].pdf
        Evidence that management reviewed findings (CC5.3)

FOLDER 3: REMEDIATION EVIDENCE
  ├── remediation_tracking_log.xlsx
  │     Columns: Finding ID | Title | CVSS | Discovery Date |
  │     Assigned To | Remediation Date | PR/Ticket | Deployed Date | Status
  │     This is the SLA compliance evidence auditors check
  │
  ├── per_finding_remediation/
  │   ├── FIND-001_remediation_evidence.pdf
  │   │   Contains: Code diff or config change + deployment proof
  │   ├── FIND-002_remediation_evidence.pdf
  │   └── [one file per finding]
  │
  └── risk_acceptance_register.pdf
        Signed risk acceptances for any unremediated findings
        Required: Finding description, risk owner name/title,
        acceptance date, compensating controls, review date

FOLDER 4: RETEST EVIDENCE
  ├── retest_report_[FIRM]_[DATE].pdf
  │     Complete retest with per-finding verification
  │
  └── production_deployment_confirmation.pdf
        Evidence that remediations are in production
        (deployment log, version confirmation, commit hash)

FOLDER 5: HISTORICAL EVIDENCE (Type II requirement)
  ├── prior_period_pentest_report.pdf
  │     Shows continuous testing across observation period
  │
  └── prior_period_remediation_log.pdf
        Shows findings from previous period were addressed

FOLDER 6: SUPPORTING EVIDENCE
  ├── vendor_qualification_evidence/
  │   ├── [Testing firm] CREST certification.pdf
  │   ├── [Testing firm] professional liability insurance.pdf
  │   └── [Testing firm] qualifications statement.pdf
  │
  └── management_review_evidence/
      ├── security_committee_meeting_minutes_[DATE].pdf
      │     Minutes showing pentest findings were reviewed by management
      └── board_security_briefing_[DATE]

SOC 2 PENETRATION TESTING EVIDENCE PACKAGE STRUCTURE:

FOLDER 1: POLICY AND PROCEDURE
  ├── penetration_testing_policy_v[X]_[DATE].pdf
  │     Required contents:
  │     □ Testing frequency (annual minimum)
  │     □ Who is responsible for commissioning tests
  │     □ Scope definition process
  │     □ Qualified vendor requirements
  │     □ Finding SLA (critical, high, medium, low)
  │     □ Remediation verification requirements
  │     □ Risk acceptance process
  │     □ Evidence retention period
  │     □ Approval signatures (CISO or equivalent)
  │
  ├── vulnerability_management_policy_v[X]_[DATE].pdf
  │     The SLA table that auditors will test:
  │     Critical: 48 hours
  │     High: 7 days
  │     Medium: 30 days
  │     Low: 90 days
  │     (Your actual SLAs — must be met, not aspirational)
  │
  └── penetration_testing_vendor_qualification_criteria.pdf
        Evidence that the firm used meets your qualification standards

FOLDER 2: CURRENT PERIOD PENETRATION TEST
  ├── engagement_authorization_letter_signed.pdf
  │     Proof of authorized testing
  │
  ├── penetration_test_report_[FIRM]_[DATE].pdf
  │     The complete finding report
  │
  ├── pentest_scope_confirmation.pdf
  │     Confirmation that scope covered customer data systems
  │
  └── pentest_executive_briefing_[DATE].pdf
        Evidence that management reviewed findings (CC5.3)

FOLDER 3: REMEDIATION EVIDENCE
  ├── remediation_tracking_log.xlsx
  │     Columns: Finding ID | Title | CVSS | Discovery Date |
  │     Assigned To | Remediation Date | PR/Ticket | Deployed Date | Status
  │     This is the SLA compliance evidence auditors check
  │
  ├── per_finding_remediation/
  │   ├── FIND-001_remediation_evidence.pdf
  │   │   Contains: Code diff or config change + deployment proof
  │   ├── FIND-002_remediation_evidence.pdf
  │   └── [one file per finding]
  │
  └── risk_acceptance_register.pdf
        Signed risk acceptances for any unremediated findings
        Required: Finding description, risk owner name/title,
        acceptance date, compensating controls, review date

FOLDER 4: RETEST EVIDENCE
  ├── retest_report_[FIRM]_[DATE].pdf
  │     Complete retest with per-finding verification
  │
  └── production_deployment_confirmation.pdf
        Evidence that remediations are in production
        (deployment log, version confirmation, commit hash)

FOLDER 5: HISTORICAL EVIDENCE (Type II requirement)
  ├── prior_period_pentest_report.pdf
  │     Shows continuous testing across observation period
  │
  └── prior_period_remediation_log.pdf
        Shows findings from previous period were addressed

FOLDER 6: SUPPORTING EVIDENCE
  ├── vendor_qualification_evidence/
  │   ├── [Testing firm] CREST certification.pdf
  │   ├── [Testing firm] professional liability insurance.pdf
  │   └── [Testing firm] qualifications statement.pdf
  │
  └── management_review_evidence/
      ├── security_committee_meeting_minutes_[DATE].pdf
      │     Minutes showing pentest findings were reviewed by management
      └── board_security_briefing_[DATE]

SOC 2 PENETRATION TESTING EVIDENCE PACKAGE STRUCTURE:

FOLDER 1: POLICY AND PROCEDURE
  ├── penetration_testing_policy_v[X]_[DATE].pdf
  │     Required contents:
  │     □ Testing frequency (annual minimum)
  │     □ Who is responsible for commissioning tests
  │     □ Scope definition process
  │     □ Qualified vendor requirements
  │     □ Finding SLA (critical, high, medium, low)
  │     □ Remediation verification requirements
  │     □ Risk acceptance process
  │     □ Evidence retention period
  │     □ Approval signatures (CISO or equivalent)
  │
  ├── vulnerability_management_policy_v[X]_[DATE].pdf
  │     The SLA table that auditors will test:
  │     Critical: 48 hours
  │     High: 7 days
  │     Medium: 30 days
  │     Low: 90 days
  │     (Your actual SLAs — must be met, not aspirational)
  │
  └── penetration_testing_vendor_qualification_criteria.pdf
        Evidence that the firm used meets your qualification standards

FOLDER 2: CURRENT PERIOD PENETRATION TEST
  ├── engagement_authorization_letter_signed.pdf
  │     Proof of authorized testing
  │
  ├── penetration_test_report_[FIRM]_[DATE].pdf
  │     The complete finding report
  │
  ├── pentest_scope_confirmation.pdf
  │     Confirmation that scope covered customer data systems
  │
  └── pentest_executive_briefing_[DATE].pdf
        Evidence that management reviewed findings (CC5.3)

FOLDER 3: REMEDIATION EVIDENCE
  ├── remediation_tracking_log.xlsx
  │     Columns: Finding ID | Title | CVSS | Discovery Date |
  │     Assigned To | Remediation Date | PR/Ticket | Deployed Date | Status
  │     This is the SLA compliance evidence auditors check
  │
  ├── per_finding_remediation/
  │   ├── FIND-001_remediation_evidence.pdf
  │   │   Contains: Code diff or config change + deployment proof
  │   ├── FIND-002_remediation_evidence.pdf
  │   └── [one file per finding]
  │
  └── risk_acceptance_register.pdf
        Signed risk acceptances for any unremediated findings
        Required: Finding description, risk owner name/title,
        acceptance date, compensating controls, review date

FOLDER 4: RETEST EVIDENCE
  ├── retest_report_[FIRM]_[DATE].pdf
  │     Complete retest with per-finding verification
  │
  └── production_deployment_confirmation.pdf
        Evidence that remediations are in production
        (deployment log, version confirmation, commit hash)

FOLDER 5: HISTORICAL EVIDENCE (Type II requirement)
  ├── prior_period_pentest_report.pdf
  │     Shows continuous testing across observation period
  │
  └── prior_period_remediation_log.pdf
        Shows findings from previous period were addressed

FOLDER 6: SUPPORTING EVIDENCE
  ├── vendor_qualification_evidence/
  │   ├── [Testing firm] CREST certification.pdf
  │   ├── [Testing firm] professional liability insurance.pdf
  │   └── [Testing firm] qualifications statement.pdf
  │
  └── management_review_evidence/
      ├── security_committee_meeting_minutes_[DATE].pdf
      │     Minutes showing pentest findings were reviewed by management
      └── board_security_briefing_[DATE]

The Remediation SLA Compliance Evidence

This is where most companies fail their Type II audit without realizing it. The auditor doesn't just check that findings were remediated, they check whether they were remediated within the stated SLA.

import json
from datetime import datetime, timedelta
from typing import List, Dict, Optional

def verify_sla_compliance(
    findings: List[Dict],
    sla_policy: Dict,  # {'CRITICAL': 2, 'HIGH': 7, 'MEDIUM': 30, 'LOW': 90}
    audit_period_start: datetime,
    audit_period_end: datetime
) -> Dict:
    """
    Verify penetration test finding SLA compliance for SOC 2 audit evidence.
    Produces the evidence document auditors will examine.
    """

    results = {
        'audit_period': {
            'start': audit_period_start.isoformat(),
            'end': audit_period_end.isoformat()
        },
        'sla_policy': sla_policy,
        'findings_analysis': [],
        'compliance_summary': {},
        'exceptions': [],
        'audit_ready': False
    }

    compliance_by_severity = {severity: {'total': 0, 'within_sla': 0, 'breached': 0}
                               for severity in sla_policy}

    for finding in findings:
        finding_id = finding['id']
        severity = finding['severity'].upper()
        discovery_date = datetime.fromisoformat(finding['discovery_date'])
        remediation_date = finding.get('remediation_date')
        retest_confirmed = finding.get('retest_confirmed', False)
        retest_date = finding.get('retest_date')
        status = finding.get('status', 'OPEN')

        # Skip findings outside audit period
        if discovery_date < audit_period_start or discovery_date > audit_period_end:
            continue

        sla_days = sla_policy.get(severity, 90)
        sla_deadline = discovery_date + timedelta(days=sla_days)

        analysis = {
            'finding_id': finding_id,
            'title': finding.get('title'),
            'severity': severity,
            'cvss': finding.get('cvss'),
            'discovery_date': discovery_date.isoformat(),
            'sla_days': sla_days,
            'sla_deadline': sla_deadline.isoformat(),
            'status': status,
        }

        if status == 'REMEDIATED' and remediation_date:
            rem_date = datetime.fromisoformat(remediation_date)
            days_to_remediate = (rem_date - discovery_date).days
            within_sla = rem_date <= sla_deadline

            analysis.update({
                'remediation_date': remediation_date,
                'days_to_remediate': days_to_remediate,
                'within_sla': within_sla,
                'retest_confirmed': retest_confirmed,
                'retest_date': retest_date,
                'audit_evidence_status': (
                    'COMPLETE' if within_sla and retest_confirmed
                    else 'INCOMPLETE_MISSING_RETEST' if within_sla and not retest_confirmed
                    else 'SLA_BREACH'
                )
            })

            if severity in compliance_by_severity:
                compliance_by_severity[severity]['total'] += 1
                if within_sla:
                    compliance_by_severity[severity]['within_sla'] += 1
                else:
                    compliance_by_severity[severity]['breached'] += 1

                    results['exceptions'].append({
                        'finding_id': finding_id,
                        'severity': severity,
                        'days_over_sla': days_to_remediate - sla_days,
                        'exception_type': 'SLA_BREACH',
                        'auditor_risk': 'HIGH — auditor will flag this as control exception',
                        'mitigation': 'Document root cause and corrective action'
                    })

        elif status == 'ACCEPTED_RISK':
            risk_acceptance = finding.get('risk_acceptance', {})
            analysis.update({
                'risk_acceptance_date': risk_acceptance.get('date'),
                'risk_owner': risk_acceptance.get('owner'),
                'risk_owner_title': risk_acceptance.get('owner_title'),
                'compensating_controls': risk_acceptance.get('compensating_controls'),
                'review_date': risk_acceptance.get('review_date'),
                'audit_evidence_status': (
                    'COMPLETE' if all([
                        risk_acceptance.get('date'),
                        risk_acceptance.get('owner'),
                        risk_acceptance.get('compensating_controls')
                    ])
                    else 'INCOMPLETE_RISK_ACCEPTANCE'
                )
            })

            if not all([risk_acceptance.get('date'),
                        risk_acceptance.get('owner'),
                        risk_acceptance.get('compensating_controls')]):
                results['exceptions'].append({
                    'finding_id': finding_id,
                    'severity': severity,
                    'exception_type': 'INCOMPLETE_RISK_ACCEPTANCE',
                    'auditor_risk': 'HIGH — risk acceptance without proper documentation',
                    'mitigation': 'Complete risk acceptance form with owner signature'
                })

        elif status == 'OPEN':
            days_open = (datetime.now() - discovery_date).days
            days_past_sla = max(0, days_open - sla_days)

            analysis.update({
                'days_open': days_open,
                'days_past_sla': days_past_sla,
                'audit_evidence_status': 'SLA_BREACH' if days_past_sla > 0 else 'WITHIN_SLA',
            })

            if days_past_sla > 0:
                results['exceptions'].append({
                    'finding_id': finding_id,
                    'severity': severity,
                    'days_past_sla': days_past_sla,
                    'exception_type': 'OPEN_PAST_SLA',
                    'auditor_risk': 'CRITICAL — open finding past SLA is a direct control failure',
                    'mitigation': 'Remediate immediately or obtain signed risk acceptance'
                })

        results['findings_analysis'].append(analysis)

    # Build compliance summary
    for severity, counts in compliance_by_severity.items():
        if counts['total'] > 0:
            compliance_rate = counts['within_sla'] / counts['total'] * 100
            results['compliance_summary'][severity] = {
                'total_findings': counts['total'],
                'within_sla': counts['within_sla'],
                'sla_breached': counts['breached'],
                'compliance_rate': f"{compliance_rate:.1f}%",
                'status': 'PASS' if compliance_rate == 100 else
                          'ACCEPTABLE' if compliance_rate >= 90 else
                          'FAIL'
            }

    # Determine overall audit readiness
    critical_exceptions = [e for e in results['exceptions']
                           if e.get('auditor_risk') in ['CRITICAL', 'HIGH']]

    results['audit_ready'] = len(critical_exceptions) == 0
    results['exception_count'] = len(results['exceptions'])
    results['critical_exception_count'] = len(critical_exceptions)

    return results
import json
from datetime import datetime, timedelta
from typing import List, Dict, Optional

def verify_sla_compliance(
    findings: List[Dict],
    sla_policy: Dict,  # {'CRITICAL': 2, 'HIGH': 7, 'MEDIUM': 30, 'LOW': 90}
    audit_period_start: datetime,
    audit_period_end: datetime
) -> Dict:
    """
    Verify penetration test finding SLA compliance for SOC 2 audit evidence.
    Produces the evidence document auditors will examine.
    """

    results = {
        'audit_period': {
            'start': audit_period_start.isoformat(),
            'end': audit_period_end.isoformat()
        },
        'sla_policy': sla_policy,
        'findings_analysis': [],
        'compliance_summary': {},
        'exceptions': [],
        'audit_ready': False
    }

    compliance_by_severity = {severity: {'total': 0, 'within_sla': 0, 'breached': 0}
                               for severity in sla_policy}

    for finding in findings:
        finding_id = finding['id']
        severity = finding['severity'].upper()
        discovery_date = datetime.fromisoformat(finding['discovery_date'])
        remediation_date = finding.get('remediation_date')
        retest_confirmed = finding.get('retest_confirmed', False)
        retest_date = finding.get('retest_date')
        status = finding.get('status', 'OPEN')

        # Skip findings outside audit period
        if discovery_date < audit_period_start or discovery_date > audit_period_end:
            continue

        sla_days = sla_policy.get(severity, 90)
        sla_deadline = discovery_date + timedelta(days=sla_days)

        analysis = {
            'finding_id': finding_id,
            'title': finding.get('title'),
            'severity': severity,
            'cvss': finding.get('cvss'),
            'discovery_date': discovery_date.isoformat(),
            'sla_days': sla_days,
            'sla_deadline': sla_deadline.isoformat(),
            'status': status,
        }

        if status == 'REMEDIATED' and remediation_date:
            rem_date = datetime.fromisoformat(remediation_date)
            days_to_remediate = (rem_date - discovery_date).days
            within_sla = rem_date <= sla_deadline

            analysis.update({
                'remediation_date': remediation_date,
                'days_to_remediate': days_to_remediate,
                'within_sla': within_sla,
                'retest_confirmed': retest_confirmed,
                'retest_date': retest_date,
                'audit_evidence_status': (
                    'COMPLETE' if within_sla and retest_confirmed
                    else 'INCOMPLETE_MISSING_RETEST' if within_sla and not retest_confirmed
                    else 'SLA_BREACH'
                )
            })

            if severity in compliance_by_severity:
                compliance_by_severity[severity]['total'] += 1
                if within_sla:
                    compliance_by_severity[severity]['within_sla'] += 1
                else:
                    compliance_by_severity[severity]['breached'] += 1

                    results['exceptions'].append({
                        'finding_id': finding_id,
                        'severity': severity,
                        'days_over_sla': days_to_remediate - sla_days,
                        'exception_type': 'SLA_BREACH',
                        'auditor_risk': 'HIGH — auditor will flag this as control exception',
                        'mitigation': 'Document root cause and corrective action'
                    })

        elif status == 'ACCEPTED_RISK':
            risk_acceptance = finding.get('risk_acceptance', {})
            analysis.update({
                'risk_acceptance_date': risk_acceptance.get('date'),
                'risk_owner': risk_acceptance.get('owner'),
                'risk_owner_title': risk_acceptance.get('owner_title'),
                'compensating_controls': risk_acceptance.get('compensating_controls'),
                'review_date': risk_acceptance.get('review_date'),
                'audit_evidence_status': (
                    'COMPLETE' if all([
                        risk_acceptance.get('date'),
                        risk_acceptance.get('owner'),
                        risk_acceptance.get('compensating_controls')
                    ])
                    else 'INCOMPLETE_RISK_ACCEPTANCE'
                )
            })

            if not all([risk_acceptance.get('date'),
                        risk_acceptance.get('owner'),
                        risk_acceptance.get('compensating_controls')]):
                results['exceptions'].append({
                    'finding_id': finding_id,
                    'severity': severity,
                    'exception_type': 'INCOMPLETE_RISK_ACCEPTANCE',
                    'auditor_risk': 'HIGH — risk acceptance without proper documentation',
                    'mitigation': 'Complete risk acceptance form with owner signature'
                })

        elif status == 'OPEN':
            days_open = (datetime.now() - discovery_date).days
            days_past_sla = max(0, days_open - sla_days)

            analysis.update({
                'days_open': days_open,
                'days_past_sla': days_past_sla,
                'audit_evidence_status': 'SLA_BREACH' if days_past_sla > 0 else 'WITHIN_SLA',
            })

            if days_past_sla > 0:
                results['exceptions'].append({
                    'finding_id': finding_id,
                    'severity': severity,
                    'days_past_sla': days_past_sla,
                    'exception_type': 'OPEN_PAST_SLA',
                    'auditor_risk': 'CRITICAL — open finding past SLA is a direct control failure',
                    'mitigation': 'Remediate immediately or obtain signed risk acceptance'
                })

        results['findings_analysis'].append(analysis)

    # Build compliance summary
    for severity, counts in compliance_by_severity.items():
        if counts['total'] > 0:
            compliance_rate = counts['within_sla'] / counts['total'] * 100
            results['compliance_summary'][severity] = {
                'total_findings': counts['total'],
                'within_sla': counts['within_sla'],
                'sla_breached': counts['breached'],
                'compliance_rate': f"{compliance_rate:.1f}%",
                'status': 'PASS' if compliance_rate == 100 else
                          'ACCEPTABLE' if compliance_rate >= 90 else
                          'FAIL'
            }

    # Determine overall audit readiness
    critical_exceptions = [e for e in results['exceptions']
                           if e.get('auditor_risk') in ['CRITICAL', 'HIGH']]

    results['audit_ready'] = len(critical_exceptions) == 0
    results['exception_count'] = len(results['exceptions'])
    results['critical_exception_count'] = len(critical_exceptions)

    return results
import json
from datetime import datetime, timedelta
from typing import List, Dict, Optional

def verify_sla_compliance(
    findings: List[Dict],
    sla_policy: Dict,  # {'CRITICAL': 2, 'HIGH': 7, 'MEDIUM': 30, 'LOW': 90}
    audit_period_start: datetime,
    audit_period_end: datetime
) -> Dict:
    """
    Verify penetration test finding SLA compliance for SOC 2 audit evidence.
    Produces the evidence document auditors will examine.
    """

    results = {
        'audit_period': {
            'start': audit_period_start.isoformat(),
            'end': audit_period_end.isoformat()
        },
        'sla_policy': sla_policy,
        'findings_analysis': [],
        'compliance_summary': {},
        'exceptions': [],
        'audit_ready': False
    }

    compliance_by_severity = {severity: {'total': 0, 'within_sla': 0, 'breached': 0}
                               for severity in sla_policy}

    for finding in findings:
        finding_id = finding['id']
        severity = finding['severity'].upper()
        discovery_date = datetime.fromisoformat(finding['discovery_date'])
        remediation_date = finding.get('remediation_date')
        retest_confirmed = finding.get('retest_confirmed', False)
        retest_date = finding.get('retest_date')
        status = finding.get('status', 'OPEN')

        # Skip findings outside audit period
        if discovery_date < audit_period_start or discovery_date > audit_period_end:
            continue

        sla_days = sla_policy.get(severity, 90)
        sla_deadline = discovery_date + timedelta(days=sla_days)

        analysis = {
            'finding_id': finding_id,
            'title': finding.get('title'),
            'severity': severity,
            'cvss': finding.get('cvss'),
            'discovery_date': discovery_date.isoformat(),
            'sla_days': sla_days,
            'sla_deadline': sla_deadline.isoformat(),
            'status': status,
        }

        if status == 'REMEDIATED' and remediation_date:
            rem_date = datetime.fromisoformat(remediation_date)
            days_to_remediate = (rem_date - discovery_date).days
            within_sla = rem_date <= sla_deadline

            analysis.update({
                'remediation_date': remediation_date,
                'days_to_remediate': days_to_remediate,
                'within_sla': within_sla,
                'retest_confirmed': retest_confirmed,
                'retest_date': retest_date,
                'audit_evidence_status': (
                    'COMPLETE' if within_sla and retest_confirmed
                    else 'INCOMPLETE_MISSING_RETEST' if within_sla and not retest_confirmed
                    else 'SLA_BREACH'
                )
            })

            if severity in compliance_by_severity:
                compliance_by_severity[severity]['total'] += 1
                if within_sla:
                    compliance_by_severity[severity]['within_sla'] += 1
                else:
                    compliance_by_severity[severity]['breached'] += 1

                    results['exceptions'].append({
                        'finding_id': finding_id,
                        'severity': severity,
                        'days_over_sla': days_to_remediate - sla_days,
                        'exception_type': 'SLA_BREACH',
                        'auditor_risk': 'HIGH — auditor will flag this as control exception',
                        'mitigation': 'Document root cause and corrective action'
                    })

        elif status == 'ACCEPTED_RISK':
            risk_acceptance = finding.get('risk_acceptance', {})
            analysis.update({
                'risk_acceptance_date': risk_acceptance.get('date'),
                'risk_owner': risk_acceptance.get('owner'),
                'risk_owner_title': risk_acceptance.get('owner_title'),
                'compensating_controls': risk_acceptance.get('compensating_controls'),
                'review_date': risk_acceptance.get('review_date'),
                'audit_evidence_status': (
                    'COMPLETE' if all([
                        risk_acceptance.get('date'),
                        risk_acceptance.get('owner'),
                        risk_acceptance.get('compensating_controls')
                    ])
                    else 'INCOMPLETE_RISK_ACCEPTANCE'
                )
            })

            if not all([risk_acceptance.get('date'),
                        risk_acceptance.get('owner'),
                        risk_acceptance.get('compensating_controls')]):
                results['exceptions'].append({
                    'finding_id': finding_id,
                    'severity': severity,
                    'exception_type': 'INCOMPLETE_RISK_ACCEPTANCE',
                    'auditor_risk': 'HIGH — risk acceptance without proper documentation',
                    'mitigation': 'Complete risk acceptance form with owner signature'
                })

        elif status == 'OPEN':
            days_open = (datetime.now() - discovery_date).days
            days_past_sla = max(0, days_open - sla_days)

            analysis.update({
                'days_open': days_open,
                'days_past_sla': days_past_sla,
                'audit_evidence_status': 'SLA_BREACH' if days_past_sla > 0 else 'WITHIN_SLA',
            })

            if days_past_sla > 0:
                results['exceptions'].append({
                    'finding_id': finding_id,
                    'severity': severity,
                    'days_past_sla': days_past_sla,
                    'exception_type': 'OPEN_PAST_SLA',
                    'auditor_risk': 'CRITICAL — open finding past SLA is a direct control failure',
                    'mitigation': 'Remediate immediately or obtain signed risk acceptance'
                })

        results['findings_analysis'].append(analysis)

    # Build compliance summary
    for severity, counts in compliance_by_severity.items():
        if counts['total'] > 0:
            compliance_rate = counts['within_sla'] / counts['total'] * 100
            results['compliance_summary'][severity] = {
                'total_findings': counts['total'],
                'within_sla': counts['within_sla'],
                'sla_breached': counts['breached'],
                'compliance_rate': f"{compliance_rate:.1f}%",
                'status': 'PASS' if compliance_rate == 100 else
                          'ACCEPTABLE' if compliance_rate >= 90 else
                          'FAIL'
            }

    # Determine overall audit readiness
    critical_exceptions = [e for e in results['exceptions']
                           if e.get('auditor_risk') in ['CRITICAL', 'HIGH']]

    results['audit_ready'] = len(critical_exceptions) == 0
    results['exception_count'] = len(results['exceptions'])
    results['critical_exception_count'] = len(critical_exceptions)

    return results

What SOC 2 Auditors Actually Ask: The Real Questions

Here's the section rewritten for readability, structured to match how auditors actually move through the conversation.

What SOC 2 Auditors Actually Ask

Auditors follow a predictable pattern. The questions below are not hypothetical, they are the actual questions experienced SOC 2 auditors ask during fieldwork. Knowing them in advance means you show up with the right documentation rather than scrambling to explain gaps.

The Standard Interview

These questions appear in virtually every Type II audit that involves penetration testing.

Question

What they actually want

What fails

Do you have a penetration testing policy?

Written policy, approved by management, with a date

"Yes, we do pentests" without a formal document

When was your last test?

A date that falls inside the observation period

A test from six months before the period started

Who conducted it?

A named external firm with documented credentials

"Our developer ran some scans"

What systems were in scope?

An explicit list that includes customer-data systems

"Our main application" without specifics

What were the findings?

Complete finding list with CVSS scores

"A few medium things we fixed"

How did you remediate critical and high findings?

Per-finding evidence with ticket or PR numbers

"We patched everything" no documentation

How do you know remediations worked?

A retest report from the testing firm

Internal testing only, or self-attestation

Show me a critical finding was remediated within your SLA

Discovery date, fix date, and retest date, all within your stated SLA

Any of those dates outside the policy timeline

What about findings you didn't remediate?

Signed risk acceptance with a named owner and compensating controls

"We decided not to fix that one"

How do findings feed into risk management?

Finding entries visible in your risk register

"Security handles it separately"

What covered changes made after the test?

Evidence of ongoing testing or sprint-level reviews

"The annual test covers everything"

If a critical vulnerability was found, how would you know?

A defined escalation procedure, email or call within hours

"We'd see it in the report when it arrived"

The perfect answer to each question is the same pattern: present the document, reference the date, name the firm, show the timeline.

The Questions That Catch Organizations Off Guard

These appear less frequently, but they are the ones organizations fail. They target gaps that look fine on paper but collapse under scrutiny.

"Was the test conducted against your production environment?"

Many companies test staging to avoid disruption. Auditors know this, and they ask. SOC 2 controls apply to production, a staging test does not evidence production control effectiveness. Document explicitly that the test targeted production systems.

"Did the test cover your cloud infrastructure?"

Application testing and cloud configuration review are not the same thing. IAM misconfigurations, exposed S3 buckets, and overpermissioned service accounts are CC6.1 and CC6.6 findings, and they live in the cloud layer, not the application layer. If your test report does not include cloud infrastructure review, you have a scope gap.

"What version of the application was running when the test occurred?"

A test of v1.2 does not evidence the security of v2.8. Have the production version number documented on the date of testing. This is a one-line addition to your test authorization letter that most organizations skip.

"How do you ensure the testing firm is independent?"

Tests conducted by internal staff or affiliated consultants do not satisfy the independence requirement under CC4. An arm's-length engagement agreement with an external firm is the clean answer.

"Has your system scope changed significantly since the test?"

New features, new infrastructure, and new integrations added after the test date are not covered by the test. Auditors in renewal audits ask this to identify coverage gaps. Have security review evidence for any significant changes made post-test.

"Do you test your APIs specifically, or just the web interface?"

Most breaches exploit API layers. Many testing programs cover only what renders in a browser. API-specific coverage documentation, endpoint list, authentication tests, BOLA and IDOR testing results, answers this cleanly.

"What about findings from year to year, are you getting better?"

This appears in renewal audits. Auditors want to see a mature control environment, not a compliance checkbox. CVSS severity trend data showing improvement over successive test periods is the strongest answer. A flat or worsening trend is a flag.

The Three Failure Modes

Almost every audit shortcoming traces back to one of three problems.

The first is missing the retest report. Organizations complete the penetration test, remediate the findings, and consider the work done. The auditor asks how they know the remediations worked. Without a third-party retest report, there is no clean answer.

The second is timing. A test completed before the observation period started, or a Q1 test with no follow-up for a Q1 through Q4 audit period, leaves months of the audit window uncovered. The auditor asks what security testing covered Q3 and Q4. The answer is nothing, and that is a gap.

The third is scope. Testing the staging environment, skipping the cloud layer, or covering only the web interface while leaving the API untested all create the same problem: the test exists, but it does not satisfy the controls it is supposed to evidence.

SOC 2-Compliant Penetration Testing by Company Stage

SOC 2 penetration testing requirements are not one-size-fits-all. The program that satisfies a first Type I audit looks nothing like what an enterprise with customer MSA commitments needs to maintain. Here is what each stage actually requires.

The Startup: First SOC 2 Type I

The goal at this stage is simple: get a qualified test done, remediate the findings, verify the remediations, and compile the evidence before the auditor arrives.

Timeline: start four months before your audit, not four weeks.

Month

What to complete

1

Write and approve a penetration testing policy. Select a qualified external vendor. Define scope, every system that touches customer data. Execute the test. Receive the finding report.

2

Triage all findings with owners and target dates. Remediate all critical findings before the audit starts. Remediate high findings, strongly recommended. Document risk acceptances for anything unresolved. Engage the firm for a retest.

3

Receive the retest report. Compile the full evidence package: policy, report, remediation evidence, retest report. Have management review the findings and minute it. Hand the package to your auditor.

Budget to expect: A small SaaS with fewer than 50 endpoints should budget $8,000–$20,000 for the test, $2,000–$6,000 for the retest, and $1,000–$3,000 for policy writing support if starting from scratch. Total: $11,000–$29,000. These numbers move up with surface area and complexity.

What goes wrong at this stage: Starting too late is the most common problem. Four weeks is not enough time to complete a test, remediate findings, and get a retest done. Auditors have seen this many times and will not grant extensions.

Skipping the retest is the second most common failure. The test report shows you have findings. Without the retest report, you have no evidence the findings were actually fixed.

Testing staging instead of production is the third. Auditors ask which environment was tested. The answer needs to be production.

The Growth-Stage Company: Type II with Engineering Velocity

Active development teams face a problem that startups do not: the codebase changes continuously, but the audit asks whether controls operated effectively over the entire twelve-month period. A single annual test does not cover code shipped in month nine.

A balanced testing cadence: One comprehensive external penetration test per year establishes the compliance baseline and satisfies the primary evidence requirement. That test should be supplemented with quarterly targeted testing of high-change areas, the endpoints your team has been actively building. Any authentication or authorization changes during a sprint should receive a security review before release. Continuous automated scanning through DAST, SAST, and SCA tools fills in the gaps between human-led tests.

Evidence collection across twelve months: The annual test is one data point. Auditors want to see that security was operating throughout the period, not just during test week. Automate what you can, CI/CD pipeline scan results are legitimate audit evidence. Run a monthly security metrics report and take it to management review. Update the risk register quarterly. Brief the executive team annually and document it.

Common Type II gaps at this stage: Months seven through twelve with no security testing evidence is the most frequent audit finding. A test in January followed by silence until the next January leaves half the period uncovered.

New features deployed without security review violate CC8.1. If your team shipped a new payment flow, a new authentication method, or a significant API change after the annual test, the auditor will ask what security review that change received.

Risk acceptances that were documented once and never revisited create another problem. Accepted risks need to be reviewed and renewed annually, an acceptance signed eighteen months ago does not satisfy a current-period auditor.

The Enterprise: Type II with Customer Commitments

Enterprise programs face the baseline SOC 2 requirements plus a second layer: what customers have contractually required in their MSAs. These are separate obligations, and failing a customer commitment is a business problem independent of the audit outcome.

Enhanced testing at this stage: The annual test needs to be comprehensive, external perimeter, internal network, API layer, and cloud infrastructure reviewed together, not separately. Application security testing should be embedded in the SDLC rather than bolted on annually. A red team engagement every two to three years provides a level of adversarial simulation that point-in-time penetration tests cannot replicate. Cloud security posture assessments for AWS, GCP, or Azure, third-party vendor assessments, and physical security reviews for any in-scope data centers round out the program.

Evidence depth: At the enterprise level, auditors and customers want to see a maturing program, not just a compliant one. CVSS severity trend data across multiple years demonstrates that each test cycle produces fewer and less severe findings. Threat modeling documentation, security architecture reviews, and bug bounty results, where applicable, contribute to the picture of a security function operating beyond minimum requirements.

Customer-specific obligations: Some enterprise customers require quarterly penetration tests rather than annual. Some require access to the full report rather than an executive summary. Some specify particular tester certifications such as CREST or ISO 27001. Some require testing of their specific integration with your platform. Some require notification of critical findings within a defined window, often 24 or 48 hours.

These are contractual commitments, not SOC 2 requirements. Your audit evidence package should include compliance evidence for each customer commitment, separate from the standard SOC 2 evidence package. Conflating the two creates confusion during fieldwork and risks missing an obligation that lives in a contract rather than a control framework.

SOC 2 Penetration Testing Across the Five TSC Categories

Security (CC): The Mandatory Category

Every SOC 2 engagement requires the Security category. The specific common criteria that penetration testing most directly satisfies:

CC Control

Description

Penetration Testing Evidence

Auditor Priority

CC6.1

Logical access security controls

Authentication bypass test results, IDOR findings, privilege escalation results

High

CC6.2

Access credential registration

Unauthenticated account creation testing

Medium

CC6.3

Access modification and removal

Role-based access testing, terminated user testing

High

CC6.6

External threat protection

External penetration test results (primary evidence)

Critical

CC6.7

Information transmission controls

CORS, TLS, data exposure testing

High

CC7.1

Detection and monitoring

Full pentest findings + remediation + retest

Critical

CC7.2

Anomaly monitoring

SIEM/detection evidence during testing

Medium

CC7.3

Security event evaluation

Finding triage and classification documentation

Medium

CC8.1

Change management testing

Pre-deployment security testing evidence

High

CC9.1

Risk mitigation

Risk register showing pentest findings as input

Medium

Availability (A): When Required

The availability category adds infrastructure resilience testing to the scope:




Confidentiality (C): When Required




Selecting a Penetration Testing Provider for SOC 2

SOC 2 does not name specific certifications or firms. What it requires is that testing be conducted by a "qualified" party, and auditors apply professional judgment to determine whether that bar was met. The practical implication is that you need to make a defensible case for why you chose your provider. Here is what that case needs to look like.

What Qualifies a Provider

There is a floor and a ceiling. The floor is what every SOC 2-ready provider must have. The ceiling is what gives auditors more confidence, particularly in competitive enterprise sales cycles where customers may review your report directly.

The minimum bar

The provider must be external to your organization. Independence is a fundamental audit requirement, internal teams and affiliated consultants do not satisfy it, regardless of their technical skill.

Beyond independence, the provider needs to demonstrate actual security testing experience, carry professional liability insurance, and commit to producing a formal written report. That last point matters more than it sounds. A spreadsheet of findings is not audit evidence. A structured report with a unique ID per finding, CVSS scores, proof-of-concept documentation, and a narrative of methodology is what auditors expect to see.

The provider also needs to offer a formal retest. Not "we'll review your remediation notes" a retest where they re-execute their test cases against the patched systems and produce a second report confirming which findings were resolved. Without this, you cannot answer the auditor's question about how you know the remediations worked.

Preferred qualifications

CREST organizational certification is the international benchmark for penetration testing quality. Individual certifications that carry weight with auditors include OSCP from Offensive Security and CISSP from ISC2. ISO 27001 certification at the organizational level is a secondary signal of process maturity.

More practically: experience with SOC 2 engagements specifically matters. A provider who has run these engagements before knows what the report needs to contain, how to structure finding IDs for remediation tracking, and what "per-finding verification" means to an auditor. You should not be the engagement where they learn this.

Questions to Ask Before Signing

These are the questions that separate providers who understand audit evidence from those who do not.

"Have you conducted penetration tests for SOC 2 compliance before? Can you show me a sanitized sample report?"

A sample report tells you more than any sales conversation. Look for unique finding IDs, CVSS scores, proof-of-concept evidence, and a clear executive summary table. If the sample looks like it was generated from a scanner with minimal human analysis, that is what your report will look like.

"Is retest included in the engagement scope, and will you produce a separate retest report?"

The answer needs to be yes to both. A provider who treats retest as an optional add-on is signaling that they are not accustomed to compliance-oriented engagements.

"Can you confirm you will test the production environment, not staging?"

A good provider will ask this of you. If you have to ask it of them, that is a yellow flag. Some providers default to staging to reduce operational risk, which is reasonable, but it produces evidence that auditors will question.

"What is your process when you find a critical vulnerability during testing?"

There should be a defined escalation path, an out-of-band call or email to your security team, not waiting for the final report. The answer to this question tells you how the engagement will feel under pressure.

"Will you provide a CVSS delta summary comparing this year's findings to last year's?"

For renewal audits, trend data matters. Providers experienced with Type II engagements understand this. Providers who are new to compliance work often do not.

Red Flags

Some of these end the conversation. Others are context-dependent but worth probing.

Signal

What it means

Report is automated scanner output with minimal narrative

Not qualified as a "formal" penetration test for audit purposes

No retest service offered

You will not be able to evidence remediation effectiveness

No unique finding IDs in the report

Remediation tracking and per-finding verification become difficult

Provider cannot explain how their report maps to SOC 2 controls

They have not done this before

Provider defaults to staging environment

Audit evidence will be questioned

No professional liability insurance

Financial and reputational risk if the engagement causes an incident

"Report" is a spreadsheet

Not structured audit evidence

The last point is worth expanding. A spreadsheet of vulnerabilities is useful for a development team. It is not what a SOC 2 auditor expects to receive when they ask for the penetration test report. The format of the deliverable matters as much as the content.

The Practical Decision

For most companies pursuing SOC 2 Type I or early Type II, the decision comes down to three things: independence from your organization, a report format that satisfies audit requirements, and an explicit retest commitment. Everything else, certifications, tooling, methodology, is secondary to those three.

For enterprise Type II programs with customer MSA obligations, the bar is higher. Customers who require CREST certification, named testers with specific credentials, or access to the full report are imposing contractual requirements that go beyond SOC 2 minimums. Know which customer commitments you are carrying before you issue an RFP, because those commitments determine your provider shortlist.

The SOC 2 Penetration Testing Vendor Evaluation Matrix

Use this matrix when evaluating providers for SOC 2-compliance purposes:

Evaluation Criterion

Weight

Score 1–5

Notes

External independent firm

Required

Pass/Fail

Non-negotiable

Formal written report delivery

Required

Pass/Fail

Non-negotiable

Retest included in scope

Required

Pass/Fail

Non-negotiable

Firm certifications (CREST, ISO 27001)

High

1–5

+1 per relevant cert

Individual tester certifications (OSCP, CISSP)

High

1–5

Per lead tester

SOC 2 engagement experience

High

1–5

Ask for references

Report includes CVSS scores

High

1–5

Required for CC7.1

Critical finding escalation SLA

High

1–5

4–24 hours preferred

Production environment testing

High

1–5

Confirm explicit commitment

Cloud infrastructure review

Medium

1–5

If cloud-hosted

Per-finding remediation verification

Medium

1–5

In retest report

CVSS delta reporting

Medium

1–5

For trend analysis

API-specific testing capability

Medium

1–5

Critical for SaaS

Code review capability (white box)

Low

1–5

Enhanced testing

Availability for auditor questions

Low

1–5

Some auditors ask

Common SOC 2 Penetration Testing Mistakes and How to Avoid Them

Most SOC 2 audit exceptions related to penetration testing trace back to the same twelve problems. None of them require advanced security knowledge to prevent. They all require process.

Process Failures

These mistakes happen before the test runs. They are the easiest to prevent and the most embarrassing to explain to an auditor.

Testing staging instead of production

The fear is that a penetration test will disrupt production. The reality is that most testing does not cause downtime, and disruptive tests can be scheduled during off-hours. The engagement letter should explicitly state the production scope. If your report says "tested against staging environment," expect an exception on CC6.6 and CC7.1.

No retest report

Fixing the bug is not the same as evidencing that the fix worked. Auditors know this. The retest must be conducted by the same external firm that ran the original test, and it must produce a second report confirming which findings were resolved. Internal testing can supplement but cannot replace external verification. Include retest in the engagement contract before signing and budget for it during scoping.

Critical findings open at audit time

An open critical finding at the start of an audit is a direct control failure. Auditors view it seriously. Start testing at least ninety days before your audit date. That window gives you time to receive the report, remediate critical findings, get the retest done, and compile evidence before fieldwork begins.

Test conducted before the observation period

A test completed in Q4 of the prior year does not evidence controls for a Q1 through Q4 audit period. Schedule testing within the observation period. If a test overlaps a period boundary, confirm with your auditor whether the evidence covers the period being assessed.

Different testing firm each year

Price shopping each engagement year means every test starts from scratch with no baseline. A multi-year relationship with one firm enables the CVSS delta trend that Type II renewals benefit from. Auditors respond better to "here is how our finding severity has improved over three years" than to three disconnected reports from three different firms.

Documentation Failures

These mistakes happen after the test. The findings exist, the remediations happen, but the paper trail is missing.

Finding triage not documented

If findings were reviewed and prioritized but there is no record of it, the auditor cannot verify the review happened. Use a ticketing system, Jira, Linear, or equivalent, for every finding. Link each finding to a ticket with owner, status, and resolution date. The ticket history is the evidence.

Management review not evidenced

CC5.3 requires evidence that leadership reviewed the security posture. A CISO reviewing the report and making decisions without documenting it creates a gap. Security committee meeting minutes that reference the penetration test findings, or an executive briefing document with attendee signatures, closes it.

Risk acceptances not formally documented

An open finding without a risk acceptance looks like unmanaged risk. A verbal decision to accept a risk looks the same way. Every finding that is not remediated needs a written risk acceptance with a named individual as risk owner, a person, not a team, along with compensating controls and a review date. Accepted risks also need to be revisited annually.

No evidence of scope appropriateness

If the auditor cannot verify that the test covered customer-data-processing systems, they will question whether the test satisfied CC6.6. Management sign-off on the scope document, confirming it includes the right systems, closes this gap.

Policy Failures

These are the mistakes that turn a process gap into something that looks like systematic organizational failure.

SLA policy not matched to actual practice

A policy that says critical findings must be remediated within 48 hours, combined with a tracking log showing 30-day remediation times, is worse than having no SLA policy at all. It is evidence of a documented commitment that the organization routinely fails to meet.

Set SLAs you will actually achieve. Fourteen days for critical findings that you consistently hit is a better audit outcome than 48 hours that you consistently miss. Auditors want to see consistent execution, not ambitious targets.

Scope does not include cloud infrastructure

"Cloud is managed by AWS" is not a scope rationale that satisfies an auditor. IAM misconfigurations, overpermissioned service accounts, and exposed storage buckets are CC6.1 and CC6.6 findings that live in the cloud layer. They need to be explicitly included in the test scope.

Summary Table

Mistake

Primary TSC impact

Prevention

Testing staging not production

CC6.6, CC7.1

Engagement letter explicitly states production scope

No retest report

CC7.1

Include retest in contract before signing

Retest by internal team

CC7.1

External firm must conduct retest

Critical findings open at audit

CC6.6, CC6.1

Start testing 90 days before audit

SLA policy not met in practice

CC7.1

Set SLAs you will actually achieve

Cloud not in scope

CC6.1, CC6.6

Explicitly include IAM, storage, security groups

Finding triage not documented

CC7.1, CC7.3

Use a ticketing system for every finding

Management review not evidenced

CC5.3

Meeting minutes referencing findings

Test before observation period

CC7.1

Schedule test within the audit window

Risk acceptances not documented

CC9.1

Written acceptance with named owner per finding

Scope not signed off by management

CC6.6

Management approval on scope document

Different firm each year

CC7.1 (trend data)

Multi-year relationship with one firm

The SOC 2 Penetration Testing Timeline: Working Backwards from Your Audit Date

The 90-Day Countdown




How CodeAnt AI Builds the SOC 2 Evidence Package Automatically

A CodeAnt AI SOC 2 engagement is structured specifically to produce the complete audit evidence package, not just a technical report.

What the engagement produces:

  • Phase 1: Reconnaissance: Surfaces every external asset, subdomains, open ports, exposed APIs, CVEs on accessible machines. This covers CC6.6 completely and surfaces assets internal teams are often not tracking.

  • Phase 2: Source Code Intelligence: Traces authentication and authorization flows, identifies middleware misconfigurations, finds injection vulnerabilities invisible to external testing. Covers CC6.1, CC6.3 at the code level.

  • Phase 3: JS Bundle Analysis: Detects hardcoded secrets, exposed internal endpoints, leaked credentials. Covers CC6.7 data handling controls.

  • Phase 4–6: Active Testing and Chain Construction: 500+ exploit agents run concurrently with WAF evasion. Every finding is confirmed exploitable before reporting. Attack chains constructed, three medium findings becoming one critical finding are reported as a chain with combined CVSS, not individually as three medium findings your engineering team ignores.

  • Phase 7: Evidence-Based Report: Every finding includes working proof-of-exploit, root cause to file and line, CVSS 4.0 with metric justification, and compliance mapping to specific TSC control IDs. Not general SOC 2 references, specific CC6.1, CC6.6, CC7.1 control numbers.

  • Unlimited retesting: Every fix retested at no additional cost. Retest report issued confirming remediation in the production environment.

  • Data deletion certificate: Issued on engagement close. Included as standard.

  • 48-hour turnaround: Full report delivered within 48 hours of engagement start. This makes it operationally viable to run the semi-annual or continuous testing structure that produces the strongest SOC 2 Type II evidence.

  • Security researcher sign-off: Every report is signed off by a named security researcher providing the third-party attestation your auditor requires. This is not automated output — it is a researcher-validated engagement that satisfies the "qualified party" requirement in your penetration testing policy.

Book a 30-minute scoping call. Testing starts within 24 hours.

Continue reading:

FAQs

Does SOC 2 require a penetration test?

How long does the penetration test report remain valid for SOC 2 purposes?

What is the difference between SOC 2 Type I and Type II penetration testing requirements?

How often do I need to run a penetration test for SOC 2?

What does the SOC 2 penetration test evidence package need to contain?

Table of Contents

Start Your 14-Day Free Trial

AI code reviews, security, and quality trusted by modern engineering teams. No credit card required!

Share blog: