Testland
Browse all skills & agents

attack-surface-test-checklist

Maps a code change to the security tests worth running against it. Classifies changed paths and file contents into nine attack surfaces (authentication, session management, input handling, file upload, deserialization, access control, API and web service, cryptography, data protection), attaches the matching OWASP ASVS 4.0.3 verification requirements, OWASP Top 10 2021 category IDs, and OWASP WSTG section numbers to each active surface, then emits a per-surface manual and automated test checklist bounded by what actually changed. Surfaces with no changed lines are excluded rather than carried as filler. Use when a pull request, release branch, or feature is about to be security tested and the team needs a targeted test list instead of a generic application-wide checklist.

Install with skills.sh (any agent)

npx skills add testland/qa --skill attack-surface-test-checklist
View source

attack-surface-test-checklist

Overview

A change set has an attack surface: the set of security-relevant behaviors its files participate in. A migration adding an avatar_url column and one adding a password_reset_token column look identical in a diff stat and need entirely different tests. This skill turns a change set into a bounded security test checklist in four moves:

  1. Classify each changed file into one or more of nine attack surfaces.
  2. Attach the OWASP ASVS verification requirements for each active surface.
  3. Tag each active surface with its OWASP Top 10 2021 category and WSTG test sections.
  4. Emit the per-surface manual and automated test items, and list the excluded surfaces.

Differentiation axis

What this owns: the mapping from a specific change to the specific security tests worth running against it, expressed as checkable items with standards references.

What it deliberately is not:

  • Not a scanner configuration. The automated items are specifications of what to run, not tool settings. Triaging findings a scanner already produced is a different job.
  • Not a penetration-test methodology. WSTG describes a full assessment of a deployed target; this selects a small subset because a change touched a surface, and stops when that surface is covered.
  • Not a compliance-control audit. ASVS chapters are a catalog of things worth testing, not controls to attest against. A completed checklist is evidence tests ran, never evidence the change is secure or that any ASVS level is met.

Which ASVS version this uses

All requirement IDs below use OWASP ASVS 4.0.3 numbering. ASVS renumbers requirements between major versions, so an ID with no version attached is ambiguous. ASVS 5.0.0 was released on 2025-05-30 and 4.0.3 remains published as the previous stable release (OWASP ASVS project page (opens in new window)). On a project standardized on 5.0.0, treat the chapter names below as the lookup key and re-resolve the numbers against that edition; the surfaces and test items do not change.

ASVS defines three verification levels: Level 1 is a low assurance level that is completely penetration testable, Level 2 is for applications containing sensitive data and is the recommended level for most applications, and Level 3 is for the most critical applications such as those handling high value transactions or sensitive medical data (ASVS 4.0.3, Using the ASVS (opens in new window)). For a per-change checklist, default to Level 1 items and escalate to Level 2 when the change touches credential storage or session token issuance. That default is a scoping convention of this skill, not an ASVS rule.

The nine attack surfaces

Classify each changed file by path first, then confirm or correct with content. A file can belong to more than one surface.

SurfacePath signalsContent signals
Authenticationauth/, login/, oauth/, sso/, mfa/, token/Password hashing, session creation, JWT issuance, credential validation
Session managementsession/, cookie, middleware/Cookie attribute setting, token expiry, invalidation on logout
Input handlingroutes/, controllers/, validators/, forms/, parsers/SQL queries, template rendering, shell invocation, XML or JSON parsing
File uploadupload/, storage/, media/, attachments/Multipart handling, MIME validation, storage path construction
Deserializationserializ, marshal, pickle, yaml.load, JSON.parseObject hydration from an untrusted source
Access controlpermissions/, policy/, roles/, authz/, acl/Role checks, ownership assertions, resource-level guards
API and web serviceapi/, graphql/, soap/, rest/Schema validation, HTTP method checks, rate limiting
Cryptographycrypto/, cipher, hmac, hash, tls/Key generation, algorithm selection, IV or nonce handling
Data protectionpii/, gdpr/, models/, db/, cache/Sensitive fields stored in plaintext, logging of secrets, caching policy

Two rules keep the classification honest:

  1. Path first, content second. The path signal decides unless file content contradicts it. A file under utils/ that calls yaml.load is a deserialization surface; a file under crypto/ that only re-exports constants is not a cryptography surface.
  2. A surface with zero changed lines is excluded, not marked "not applicable". Overscoping is the failure mode that kills these checklists: when every change produces the same forty items, the list stops being read.

Step 1 - Mark the active surfaces

For each changed file, record the surface or surfaces it matched and the count of changed lines attributed to each. A surface is active when at least one changed line lands in it. Keep the file list per surface: it is what makes the test items concrete later ("SQL injection probes on the two new query parameters in orders_controller", not "test for SQL injection"). Record the excluded surfaces explicitly too. A reader needs to see that file upload was considered and found untouched, otherwise they cannot tell whether it was skipped or forgotten.

Step 2 - Attach ASVS requirements

SurfaceASVS 4.0.3 chapterRequirement areas to pull items from
AuthenticationV2 AuthenticationV2.2 General Authenticator Security, V2.4 Credential Storage, V2.5 Credential Recovery (V2 (opens in new window))
Session managementV3 Session ManagementV3.2 Session Binding, V3.3 Session Termination, V3.4 Cookie-based Session Management (V3 (opens in new window))
Input handlingV5 Validation, Sanitization and EncodingV5.1 Input Validation, V5.2 Sanitization and Sandboxing, V5.3 Output Encoding and Injection Prevention (V5 (opens in new window))
File uploadV12 Files and ResourcesV12.1 File Upload, V12.2 File Integrity, V12.3 File Execution, V12.4 File Storage, V12.6 SSRF Protection (V12 (opens in new window))
DeserializationV5 Validation, Sanitization and EncodingV5.5 Deserialization Prevention (V5 (opens in new window))
Access controlV4 Access ControlV4.1 General Access Control Design, V4.2 Operation Level Access Control, V4.3 Other Access Control Considerations (V4 (opens in new window))
API and web serviceV13 API and Web ServiceV13.1 Generic Web Service Security, V13.2 RESTful Web Service, V13.4 GraphQL (V13 (opens in new window))
CryptographyV6 Stored CryptographyV6.2 Algorithms, V6.3 Random Values, V6.4 Secret Management (V6 (opens in new window))
Data protectionV8 Data Protection, plus V7 Error Handling and LoggingV8.1 General Data Protection, V8.2 Client-side Data Protection, V8.3 Sensitive Private Data (V8 (opens in new window)); V7.1 Log Content (V7 (opens in new window))

Data protection spans two chapters on purpose. Secrets leaking into logs is a logging requirement (V7.1), not a data protection one, and a checklist that only reads V8 misses it.

Step 3 - Tag Top 10 2021 categories and WSTG sections

Three tagging notes that catch people out:

  • XSS is inside A03:2021 Injection. The 2021 category maps 33 CWEs including CWE-79 Cross-site Scripting, CWE-89 SQL Injection, and CWE-78 OS Command Injection (A03:2021 (opens in new window)). Do not file XSS under a separate category.
  • XXE is inside A05:2021 Security Misconfiguration. Its notable CWEs include CWE-611 Improper Restriction of XML External Entity Reference (A05:2021 (opens in new window)), even though the test that finds it lives in WSTG 4.7 Input Validation.
  • WSTG-<CAT>-<NN> shorthand IDs are not printed on the stable index pages. The section numbers used here (4.4.3, 4.5.4, and so on) are what the stable guide publishes, so they are the citable form.

Step 4 - Per-surface test items

Pull only the sections whose surface is active. Rewrite each item with the specific files, parameters, and endpoints recorded in Step 1.

Authentication

Manual:

  • Inspect changed credential storage: passwords must be salted and hashed with an approved one-way key derivation or password hashing function (ASVS 4.0.3 2.4.1). If PBKDF2 is used, the iteration count should be as large as verification server performance allows, typically at least 100,000 iterations (2.4.3). If bcrypt is used, the work factor should be as large as performance allows, with a minimum of 10 (2.4.4) (V2 (opens in new window)).
  • Confirm changed login endpoints keep anti-automation controls effective against breached credential testing, brute force, and account lockout attacks; ASVS states no more than 100 failed attempts per hour should be possible on a single account (2.2.1). Pair with WSTG 4.4.3 Testing for Weak Lock Out Mechanism (WSTG 4.4 (opens in new window)).
  • If the change adds an alternative sign-in path, confirm weak authenticators such as SMS and email are limited to secondary verification and transaction approval rather than replacing a stronger method (2.2.2). Pair with WSTG 4.4.10 Testing for Weaker Authentication in Alternative Channel.
  • Exercise any changed recovery path: a system generated activation or recovery secret must not be sent in clear text (2.5.1), knowledge-based "secret questions" must not be present (2.5.2), and recovery must not reveal the current password (2.5.3). Pair with WSTG 4.4.9 Testing for Weak Password Change or Reset Functionalities.

Automated:

  • Run static-analysis rules for plaintext or reversibly-stored credentials against the changed files only.
  • Scan the changed lines for hard-coded credentials and API keys.

The remaining eight surfaces (session management, input handling, file upload, deserialization, access control, API and web service, cryptography, data protection) follow the same Manual / Automated shape with their own ASVS 4.0.3, Top 10 2021, and WSTG citations. Pull the full catalog from references/per-surface-test-items.md.

Output format

Emit one Markdown block containing, in order:

  1. Header naming the change set (repository, branch or pull request identifier, and the commit the plan was built from) and the date.
  2. Surfaces touched, with the changed-line count per surface.
  3. ASVS target level and one sentence saying why that level was chosen.
  4. Test items by surface, as checkboxes tagged [MANUAL] or [AUTO], each carrying its ASVS 4.0.3 requirement ID and, where one applies, its WSTG section number. Name the actual files, parameters, or endpoints.
  5. Surfaces excluded, listed by name, so a reader can see what was considered and found untouched.
  6. Findings already visible in the change, if any: something spotted while classifying that is a defect now, not a test to run later.
  7. Not established by this checklist: an explicit line stating that completing the items is evidence tests ran, not evidence the change is secure, and not an ASVS level attestation.

Keep item 7 even when it feels redundant. It is what stops a green checklist from being read as a sign-off.

Worked example

Change set: a pull request on a document management service adding shared-link downloads. Changed files:

src/api/routes/share_links.py          +142
src/services/share_link_service.py     +96
src/storage/attachment_fetcher.py      +54
src/models/share_link.py               +31
migrations/0042_share_link.sql         +18
tests/api/test_share_links.py          +210

Classification: api/routes/ and the new query parameters make input handling and API and web service active. storage/ plus a fetch by URL makes file upload active (its SSRF sub-area specifically). The model and migration add a token column, making data protection active. share_link tokens are generated in the service, making cryptography active. The link grants object access without a session, making access control active. No login, cookie, or deserialization code changed. Test files are excluded from classification.

## Security test plan - docs-service PR #814 - 3f9a1c2

**Surfaces touched:** input handling (142), API and web service (142),
access control (96), file upload / SSRF (54), data protection (49),
cryptography (96)
**ASVS target level:** L2 (the change issues a bearer-style access token)
**Produced:** 2026-07-19

### Input handling
- [ ] [MANUAL] SQL metacharacters on `?token=` and `?expires=` in `share_links.py`; confirm parameterized queries (ASVS 4.0.3 5.3.4; WSTG 4.7.5)
- [ ] [MANUAL] Positive allow-list validation on both new parameters (ASVS 4.0.3 5.1.3)
- [ ] [AUTO]   Injection static-analysis rules scoped to `src/api/routes/`

### API and web service
- [ ] [MANUAL] Verb tampering on `/share/{id}`: PUT, DELETE, PATCH (ASVS 4.0.3 13.2.1; WSTG 4.7.3)
- [ ] [MANUAL] Wrong and missing `Content-Type` rejected (ASVS 4.0.3 13.2.5, 13.1.5)
- [ ] [MANUAL] `token` is not exposed in the URL path or logs (ASVS 4.0.3 13.1.3)

### Access control
- [ ] [MANUAL] Fetch another tenant's document id with a valid token (ASVS 4.0.3 4.2.1; WSTG 4.5.4)
- [ ] [MANUAL] Force an exception in the token guard; confirm deny (ASVS 4.0.3 4.1.5)

### File upload / SSRF
- [ ] [MANUAL] `attachment_fetcher.py` remote fetch against 169.254.169.254 and file:// ; confirm allow list (ASVS 4.0.3 12.6.1; A10:2021)

### Cryptography
- [ ] [MANUAL] Token generated via CSPRNG, not `random` (ASVS 4.0.3 6.3.1)
- [ ] [AUTO]   Flag `Math.random` / `random.random` in `share_link_service.py`

### Data protection
- [ ] [MANUAL] Token column encrypted or stored hashed (ASVS 4.0.3 8.3.7)
- [ ] [MANUAL] No token value in the new log lines (ASVS 4.0.3 7.1.1)

### Surfaces excluded (no changed lines)
Authentication, session management, deserialization

### Findings already visible in the change
- `share_link_service.py:41` builds the token with `random.randint`. This is a defect now, not a test item. Route it to defect triage.

### Not established by this checklist
Completing these items is evidence that the listed tests ran against this change. It is not a statement that the change is secure, and it is not an ASVS L2 attestation.

Anti-patterns

Anti-patternWhy it failsCorrect behavior
Applying the generic Top 10 list unchangedEvery change gets the same forty items and the team stops reading themFilter to active surfaces in Step 1 and list the excluded ones explicitly
Expanding to a full application assessmentThe checklist balloons past what anyone will execute, so nothing gets testedBound the plan by the changed surface. A file upload change does not trigger a full authentication review
Building the checklist from scanner outputThat is triage of findings that already exist, a different input and a different decisionBuild this before scanners run; feed findings to defect triage instead
Inventing ASVS requirement numbersFabricated IDs misroute the tester and destroy trust in every other ID on the pageCite only IDs read from the published chapter; when unsure, name the chapter and section instead
Quoting requirement IDs without a versionASVS renumbers across major versions, so a bare V2.4 is ambiguousWrite the version next to the ID, as in ASVS 4.0.3 2.4.3
Marking an item PASS with no evidenceProduces false confidence that survives into releaseItems are binary: tested with recorded evidence, or not tested
Burying an obvious defect as a checklist itemA hard-coded key becomes a task someone might do next sprintSplit it out as a finding in its own section and route it now
Treating a complete checklist as a security sign-offThe checklist covers the changed surface only, at one ASVS levelKeep the "not established by this checklist" line in the output

Limitations

  • Path classification is a heuristic. A shared utility in the change set can reach an authentication flow that no path signal touched. Content signals reduce this but do not remove it; a short human read of the change set is still warranted.
  • The level default is conservative. The Level 1 default, escalating to Level 2 for credential storage or session token issuance (defined in "Which ASVS version this uses"), suits most changes; teams in regulated or safety-critical domains should raise it rather than accept Level 1.
  • No threat model. Surfaces map to known categories. Attacker motivation, likelihood, and business impact are not modeled, so the checklist cannot rank its own items by risk.
  • No dynamic analysis is performed. The [AUTO] items are specifications of what to run. Something else has to run them.
  • Requirement IDs are version-bound. Every ID here is ASVS 4.0.3. On a project standardized on 5.0.0 the chapter names still route correctly but the numbers must be re-resolved against that edition (OWASP ASVS project page (opens in new window)).
  • WSTG shorthand IDs are not used. The stable guide's index pages publish section numbers, not WSTG-<CAT>-<NN> identifiers, so section numbers are what this skill cites.

Source index

Every requirement ID, section number, and quoted phrase above carries its source link at the point it is used. These are the three roots those links come from, for anyone who wants to browse rather than verify one claim:

Per-surface test items

View source (opens in new window)

Per-surface test items

Deep reference for the attack-surface-test-checklist SKILL.md, Step 4. The Authentication surface is kept inline in SKILL.md as the worked example of the item shape; this file carries the other eight surfaces. Pull only the sections whose surface is active, and rewrite each item with the specific files, parameters, and endpoints recorded in Step 1.

Session management

Manual:

  • Check every cookie the change sets or modifies for the Secure attribute (ASVS 4.0.3 3.4.1), the HttpOnly attribute (3.4.2), and a SameSite attribute limiting cross-site request forgery exposure (3.4.3) (V3 (opens in new window)). Pair with WSTG 4.6.2 Testing for Cookies Attributes (WSTG 4.6 (opens in new window)).
  • Log out through the changed flow, then replay the old token: logout and expiration must invalidate the session token such that the back button or a downstream relying party cannot resume the session (3.3.1). Pair with WSTG 4.6.6 Testing for Logout Functionality.
  • Capture the pre-authentication token and re-check it after login: the application must generate a new session token on user authentication (3.2.1). Pair with WSTG 4.6.3 Testing for Session Fixation.
  • If token generation changed, confirm tokens still carry at least 64 bits of entropy (3.2.2) and are generated with approved cryptographic algorithms (3.2.4).

Automated:

  • Assert on Set-Cookie response headers for the changed endpoints in an integration test, so the attributes cannot silently regress.

Input handling

Manual:

  • Drive every new or modified parameter with SQL metacharacters. Data selection and queries must use parameterized queries, ORMs, or entity frameworks (ASVS 4.0.3 5.3.4) (V5 (opens in new window)). Pair with WSTG 4.7.5 Testing for SQL Injection (WSTG 4.7 (opens in new window)).
  • Drive the same parameters with XSS payloads and check that context-aware output escaping protects against reflected, stored, and DOM based XSS (5.3.3). Pair with WSTG 4.7.1 and 4.7.2.
  • If the change renders a server-side template with user input, probe for template injection: user input must be sanitized or sandboxed (5.2.5). Pair with WSTG 4.7.18 Testing for Server-side Template Injection.
  • If an XML parser is touched, confirm it uses the most restrictive configuration available and that resolving external entities is disabled (5.5.2). Pair with WSTG 4.7.7 Testing for XML Injection.
  • Confirm new inputs are validated with positive validation (allow lists) (5.1.3) and that structured data is strongly typed and validated against a defined schema covering allowed characters, length, and pattern (5.1.4).
  • If the change binds request data onto a model, confirm protection against mass parameter assignment such as marking fields private (5.1.2).

Automated:

  • Run injection-family static analysis rules scoped to the changed route, controller, and query-building files.
  • Add negative-path tests for the rejected inputs so the validation itself has regression coverage.

File upload

Manual:

  • Upload a file whose extension and content disagree. Files obtained from untrusted sources must be validated to be of the expected type based on the file's content, not the extension (ASVS 4.0.3 12.2.1) (V12 (opens in new window)). Pair with WSTG 4.10.8 Test Upload of Unexpected File Types (WSTG 4.10 (opens in new window)).
  • Confirm the destination: untrusted files must be stored outside the web root with limited permissions (12.4.1), and scanned by antivirus scanners (12.4.2).
  • Send traversal sequences and absolute paths in the filename. User-submitted filename metadata must not be used directly by system or framework filesystems (12.3.1), and must be validated or ignored to prevent disclosure, creation, updating, or removal of local files (12.3.2). Pair with WSTG 4.10.9 Test Upload of Malicious Files.
  • Push size and compression limits: the application must not accept large files that could fill storage or cause denial of service (12.1.1), and compressed files must be checked against a maximum uncompressed size and a maximum number of files (12.1.2).
  • If the feature fetches a file from a user-supplied URL, confirm the server is configured with an allow list of resources or systems it may send requests to (12.6.1). SSRF flaws occur whenever an application fetches a remote resource without validating the user-supplied URL (A10:2021 (opens in new window)).

Automated:

  • Add a fixture upload of a polyglot file (a valid image with an embedded script payload) through the changed endpoint and assert it is rejected.

Deserialization

Manual:

  • Confirm the changed path avoids deserializing untrusted data, or protects it, in both custom code and third-party JSON, XML, and YAML parsers (ASVS 4.0.3 5.5.3) (V5 (opens in new window)).
  • If serialized objects cross a trust boundary, confirm they use integrity checks or are encrypted to prevent hostile object creation or data tampering (5.5.1). The Top 10 2021 scenario for this category is exactly that shape: a serialized user state passed back and forth per request, re-signed by an attacker into remote code execution (A08:2021 (opens in new window)).
  • In browser or JavaScript-based backends, confirm JSON.parse is used and eval() is not (5.5.4).

Automated:

  • Add static-analysis rules scoped to the changed files that flag pickle.loads, yaml.load without safe_load, and Java ObjectInputStream.

Access control

Manual:

  • Authenticate as user A, then request user B's resources through every changed endpoint. Sensitive data and APIs must be protected against Insecure Direct Object Reference attacks targeting creation, reading, updating, and deletion of records (ASVS 4.0.3 4.2.1) (V4 (opens in new window)). Pair with WSTG 4.5.4 Testing for Insecure Direct Object References (WSTG 4.5 (opens in new window)).
  • Call the changed privileged operations as a lower-privileged role: least privilege requires that users only access functions, data files, URLs, controllers, services, and other resources for which they hold specific authorization (4.1.3). Pair with WSTG 4.5.3 Testing for Privilege Escalation.
  • Manipulate role, tenant, and ownership parameters in the request. Attributes and policy information used by access controls must not be manipulable by end users unless specifically authorized (4.1.2). Pair with WSTG 4.5.2 Testing for Bypassing Authorization Schema.
  • Confirm the check runs server-side: access control rules must be enforced on a trusted service layer, especially where a client-side check exists and could be bypassed (4.1.1).
  • Force an exception inside the changed guard and confirm access controls fail securely (4.1.5).
  • If the change adds an administrative interface, confirm it uses appropriate multi-factor authentication (4.3.1).

Automated:

  • Add a static-analysis or route-inventory rule that flags changed handlers lacking an authorization guard, decorator, or policy attachment.

API and web service

Manual:

  • Replay each changed endpoint with unexpected HTTP verbs. Enabled RESTful HTTP methods must be a valid choice for the user or action (ASVS 4.0.3 13.2.1) (V13 (opens in new window)). Pair with WSTG 4.7.3 Testing for HTTP Verb Tampering (WSTG 4.7 (opens in new window)).
  • Send a wrong and a missing Content-Type. REST services must explicitly check the incoming content type is the expected one (13.2.5), and requests with unexpected or missing content types must be rejected with appropriate headers (13.1.5).
  • Confirm JSON schema validation is in place and verified before input is accepted (13.2.2).
  • Probe every changed endpoint unauthenticated: authorization decisions must be made at the URI, enforced programmatically or declaratively (13.1.4).
  • Inspect changed API URLs for sensitive values. URLs must not expose information such as API keys or session tokens (13.1.3).
  • For a changed GraphQL schema or resolver, send deeply nested and batched queries: a query allow list, or depth limiting and amount limiting, must prevent GraphQL denial of service (13.4.1). Confirm authorization logic sits in the business logic layer rather than the GraphQL layer (13.4.2). Pair with WSTG 4.12.1 Testing GraphQL (WSTG 4.12 (opens in new window)).

Automated:

  • Diff the API schema or route inventory between base and head, and fail the check when a new endpoint or field appears without a declared auth requirement.

Cryptography

Manual:

  • Read the changed algorithm selection. Known insecure block modes such as ECB, padding modes such as PKCS#1 v1.5, ciphers with small block sizes such as Triple-DES and Blowfish, and weak hashing algorithms such as MD5 and SHA1 must not be used unless required for backwards compatibility (ASVS 4.0.3 6.2.5) (V6 (opens in new window)). The Top 10 2021 guidance is the same: avoid deprecated cryptographic functions and padding schemes such as MD5, SHA1, and PKCS number 1 v1.5 (A02:2021 (opens in new window)).
  • Inspect IV and nonce handling: nonces, initialization vectors, and other single use numbers must not be used more than once with a given encryption key, and the generation method must suit the algorithm (6.2.6).
  • Confirm any new random value used for security purposes comes from the cryptographic module's approved cryptographically secure random number generator (6.3.1), and that new GUIDs use the v4 algorithm with a CSPRNG (6.3.2).
  • Confirm new keys and secrets live in a secrets management solution such as a key vault (6.4.1) rather than in source or config, and that key material is not exposed to the application where an isolated module can perform the operation (6.4.2).
  • Check transport on changed paths: all data in transit should be encrypted with secure protocols such as TLS with forward secrecy ciphers (A02:2021 (opens in new window)). Pair with WSTG 4.9.1 Testing for Weak Transport Layer Security and 4.9.4 Testing for Weak Encryption (WSTG 4.9 (opens in new window)).

Automated:

  • Static-analysis rules scoped to the changed files that flag hard-coded key and IV literals and any use of MD5, SHA1, DES, or ECB mode.
  • Static-analysis rule that flags non-cryptographic randomness such as Math.random() in a security context.

Data protection

Manual:

  • Review changed model, schema, and serializer fields for sensitive values. Sensitive or private information that is required to be encrypted must use approved algorithms providing both confidentiality and integrity (ASVS 4.0.3 8.3.7) (V8 (opens in new window)).
  • Read every log statement the change adds. The application must not log credentials or payment details, and session tokens should only appear in logs in an irreversible hashed form (7.1.1); other sensitive data defined under local privacy laws or security policy must not be logged either (7.1.2) (V7 (opens in new window)).
  • Check changed caching behavior: sensitive data must be protected from being cached in server components such as load balancers and application caches (8.1.1), and cached or temporary server-side copies must be purged or invalidated after the authorized user accesses the data (8.1.2). On the client side, confirm sufficient anti-caching headers are set (8.2.1).
  • If the change writes to browser storage, confirm localStorage, sessionStorage, IndexedDB, and cookies do not hold sensitive data (8.2.2).
  • Check changed request shapes: sensitive data must travel in the HTTP message body or headers, and query string parameters of any HTTP verb must not carry it (8.3.1).
  • For changed endpoints returning personal data, minimize the response. The application should minimize the number of parameters in a request, including hidden fields, Ajax variables, cookies, and header values (8.1.3), and the same discipline applies to what is sent back.
  • Pair transport checks with WSTG 4.9.3 Testing for Sensitive Information Sent via Unencrypted Channels (WSTG 4.9 (opens in new window)).

Automated:

  • Static-analysis rule that flags log statements in the changed files which interpolate model fields without a redaction or masking helper.
  • Static-analysis rule that flags changed serializers exposing sensitive fields without an explicit exclusion.

Related skills

definition-of-done

The team's Definition of Done (DoD), both halves of the lifecycle: authoring and auditing. Explains the Scrum Guide's DoD definition ("a formal description of the state of the Increment when it meets the quality measures required for the product"), proposes a starter DoD with the 7-10 lines most teams need (code reviewed, unit tests, docs, AC met, deployed to staging, smoke passed, no a11y regressions, telemetry wired), emits a per-PR checklist a reviewer enforces, and audits work against an existing DoD line by line with repository evidence (review records, diffs, CI runs, coverage reports), tagging every line met, not met, or unverifiable - never passing a line on self-attestation. Use when the team doesn't have a DoD, wants to revise theirs, or is about to mark a story or PR done and nobody has checked the work against the committed checklist.

e2e-suite-budget

Caps E2E suite size by computing per-test ROI - (regressions caught × value) ÷ (runtime × flake rate × maintenance) - then ranks every end-to-end test and recommends which bottom-decile ones to retire, move to a lower layer, or fix. Use when CI is slow or E2E-dominated, flaky failures are rising, or quarterly to keep suite size within maintenance capacity. For strategic unit:service:UI layer ratios use test-pyramid-balancer, for the minimal per-deploy critical-path gate use smoke-suite-gate, and for quarantining flaky tests use flaky-test-quarantine; this prunes low-signal tests by ROI.

framework-choice-advisor

Reference catalog for picking a test automation framework or QA tool - covers Playwright / Cypress / Selenium / WebdriverIO / Appium / Espresso / XCUITest / RestAssured / Karate / k6 / Locust with side-by-side tradeoffs on speed, cross-browser, mobile, parallelisation, language support, ecosystem maturity, CI integration; a decision tree matching project NFRs to framework choice; and reference layouts for the chosen stack. references/ extends the same decision to commercial procurement (seven-axis vendor evaluation for TCM platforms, no-code tools, visual-regression services) and to recording the outcome (ADR-based tool-selection decision record with signal, one recommendation, flip conditions). This is the upstream selection step: it decides which tool to adopt, not how to configure one already chosen. Use when starting a new test-automation suite, evaluating commercial QA vendors, or writing down a tool decision.

post-mortem-author

Build-an-X workflow that produces a blameless post-mortem from an incident - captures the timeline (chronological event sequence with sources), root cause analysis (what + why, not who), impact (users / revenue / SLO debt), action items (with owners + due dates + measurable success criteria), and "what went well" (intentional). Per Google SRE: "Blameless postmortems are a tenet of SRE culture." Use after every user-visible incident, not just severe ones.

risk-matrix

The risk-based testing (RBT) umbrella: risk matrix and risk register authoring, likelihood x impact scoring, risk storming, calibration, and risk-to-test coverage mapping. Produces the per-feature / per-release matrix artifact (structured intake: feature, category, impact 1-5 by likelihood 1-5, score; heatmap; mitigations with owners and due dates), supporting lightweight and heavyweight (FMEA / Cost of Exposure) methods per RBT canon, plus a risk coverage mapping workflow that proves which tests, cases, or monitors back each registered risk. references/ carries the product-risk and project-risk register variants, the risk-storming facilitation guide, matrix calibration against observed defect data, and a register review checklist. Use for any risk-based-testing artifact: building a matrix or register, running a risk-storming session, calibrating ratings against defects, or mapping risks onto test coverage.

smoke-suite-gate

Build-an-X workflow for a critical-path smoke suite that runs in <5 minutes - picks the 5-15 highest-business-value journeys (login, hero flow, checkout, payment, primary read), implements as fast E2E or API tests, gates per-deploy, retries on transient failures with quarantine. Use as the canary-precursor or per-deploy verification gate; the team's "if this fails, the build can't proceed" floor.

test-case-from-live-feature

Build-an-X workflow that produces a test-case matrix from a **live, undocumented feature** - running app at a URL, screen recording, screenshot, or verbal brief - by combining structured exploration (Playwright trace / DevTools / accessibility tree) with the four canonical heuristic test-design models bundled in references/ (Bach's HTSM / SFDPOT product elements, Whittaker's How-to-Break-Software attacks, Bolton's FEW HICCUPPS consistency oracles, ISO/IEC 25010 quality characteristics). Output is a structured case matrix, not an exploratory session charter. Use when there is no story, no AC, and no documentation - only a live feature - or as the heuristic reference layer for zero-documentation test design.

test-case-ideation-from-story

Turns a thin or ambiguous story into a reviewable test list - a backlog item that is a short paragraph plus the click-through support recorded for themselves, a spec that is mostly a list of accepted formats, or a tech design pasted into the ticket while the last few releases still shipped missed cases. Takes the story or feature spec and emits a markdown test-case matrix, one row per case (id, title, precondition, steps, expected, tier), covering happy path, alternate paths, boundaries, and negative paths, before any test code is written. Output is the human-reviewable matrix that goes into TestRail / Qase / Xray, not Gherkin scenarios. Use when a story needs its cases enumerated and agreed before automation starts.

test-effort-estimation

Turns a list of testable areas plus a change-shape distribution into a PERT three-point test effort estimate, reporting every row as a range around the expected value rather than a single number, requiring a named assumptions ledger across six mandatory categories, and recommending a per-layer ownership split across developer, automation, and exploratory roles. Bundles the change-shape classifier (pure-logic / service-layer / ui-heavy / data-heavy from git-history path and content signals, with the relative per-layer cost model) as a reference, so the shape distribution the estimate consumes can be produced here too. Does not choose which tests to run or how deep coverage should go. Use when an epic or release has been broken into testable areas and someone is about to commit test capacity for a sprint, or when a change set needs its shape classified before planning.

test-pyramid-balancer

Build-an-X workflow that analyzes a repo's test mix (unit / integration / E2E counts + runtimes) and recommends rebalancing toward the test pyramid ratios per the change-set shape - pure-logic-heavy repo wants ~80/15/5; UI-heavy repo wants ~60/25/15. Detects 'ice-cream cone' (E2E-heavy) and 'hourglass' (integration-thin) anti-patterns. Use when the user asks about test distribution, test strategy, test balance, too many E2E tests, slow CI caused by tests, testing best practices, or rebalancing their test suite; also suitable for quarterly calibration of the test mix to codebase reality.

test-strategy-author

Authors a test strategy document (a master test plan) for a project, release, or feature - covers scope, in/out, test types per layer (unit / integration / contract / E2E / perf / security / a11y), risk-based test prioritization that maps top risks to test investment (per `risk-matrix`), tooling stack, environments, exit criteria, and ownership. Includes a risk-based test-planning workflow that turns a feature scope plus the risk matrix into a budgeted per-risk test plan with owners, effort estimates, and an explicit risks-not-addressed section. Use when a team needs the release-readiness artifact stakeholders sign off on before significant test investment, or a risk-prioritized test plan for a feature or quarter.