test-strategy-author

Overview

A test strategy document answers the question "how do we know we're testing the right things?" Without it, test investment is gut-feel; with it, the team has a defensible artifact tying test work to risk and business goals.

The ISTQB glossary defines a test strategy as "A high-level description of the test levels to be performed and the testing within those levels" (istqb-strategy (opens in new window)). This skill produces that document, then the per-feature risk-based plan under it (Step 5).

When to use

• A new product / large feature is in planning; the team needs to scope test investment.
• A multi-quarter initiative needs a published strategy stakeholders can review.
• An audit / compliance review requires a documented strategy.
• A new team forming around a product needs onboarding context.

For per-feature test plans (smaller scope), use the lighter-weight risk-matrix - strategy is for larger horizons.

Step 1 - Document structure

# Test Strategy - `<product / release>`

**Author:** _______________   **Date:** YYYY-MM-DD
**Status:** Draft | Review | Approved
**Stakeholders:** Engineering, Product, QA, Compliance

## 1. Scope

### In scope
- (List of features / surfaces this strategy covers)

### Out of scope
- (Explicit list of what's NOT covered, with rationale)

### Assumptions
- (List of context the strategy depends on)

## 2. Risk basis

(Reference the risk matrix per `risk-matrix`)

| Risk class | Top risks (from matrix) | Test investment |
|------------|-------------------------|-----------------|
| Business    | Promo math, Tax calculations | Property-based + UAT |
| Technical   | Webhook reliability, DB migrations | Chaos + integration |
| Regulatory  | EU GDPR, CCPA          | UAT + privacy review |
| Performance | Checkout latency        | Load + canary       |

## 3. Test types per layer (the pyramid)

Per the test pyramid (`test-pyramid-balancer`). The coverage targets
below are example values to replace with the team's own; no standard
fixes them.

| Layer       | Coverage target (example) | Tools | Owner |
|-------------|----------------:|-------|-------|
| Unit         |       80%       | Jest, pytest, JUnit | Devs |
| Integration  |       60%       | Testcontainers, supertest | Devs |
| Contract     |       100% of consumer-provider pairs | Pact | Devs |
| E2E          |       Critical paths (5-10 flows) | Playwright | QA |
| Performance  |       Critical endpoints | k6 + Lighthouse CI | QA + SRE |
| Security     |       OWASP Top 10 (2025) | Schemathesis + manual pen test | Security |
| A11y         |       WCAG 2.2 AA | axe + manual review | QA |

## 4. Tooling

(Stack inventory per layer; references to specific skills)

## 5. Environments

- **Local dev** - per-engineer; Testcontainers backing services.
- **Staging** - shared; smoke + UAT.
- **Canary** - 5% prod traffic; 30-min observation per
  prod-canary-validator.
- **Prod** - synthetic monitors per
  synthetic-monitor-author.

## 6. Test data

- **Synthetic accounts** - per synthetic-data-toolkit.
- **PII handling** - per synthetic-pii-generator.
- **Database state** - per database snapshot / restore.

## 7. Exit criteria

Release ships when:

- All AC for in-scope features pass.
- All unit/integration tests green; no flake in last 3 main runs.
- E2E critical-path suite green.
- Coverage targets met per Section 3.
- Performance targets met (p95 within budget).
- A11y regression scan green.
- Threat model reviewed for security-touching changes.
- Risk matrix Critical (>=15) all mitigated.

## 8. Ownership

| Activity              | Owner | Backup |
|-----------------------|-------|--------|
| Unit test reviews      | Devs  | Tech lead |
| E2E suite maintenance  | QA    | Dev TPM |
| Perf budget approval   | SRE   | QA |
| Threat model authorship | Security | Dev TPM |
| Synthetic monitors     | SRE   | QA |
| Risk matrix updates    | QA    | Product |

## 9. Cadence

- **Per-PR:** Lint, unit, integration, smoke E2E, coverage delta.
- **Per-merge to main:** Full E2E, perf gate.
- **Nightly:** Full regression, mutation testing weekly.
- **Pre-release:** Manual UAT sign-off, full security scan,
  release-readiness check.

## 10. Risk register snapshot

(Top 10 risks from the matrix as of strategy authoring date)

## 11. Open questions / decisions needed

- (List of unresolved items requiring stakeholder input)

## Approval

- [ ] Engineering manager
- [ ] QA lead
- [ ] Product manager
- [ ] Security (if applicable)

Sources behind the template sections:

• Section 2 (risk basis). ISTQB defines risk-based testing as an approach "to reduce the level of product risks and inform stakeholders of their status", which "involves the identification of product risks and the use of risk levels to guide the test process" (istqb-rbt (opens in new window)). That is why every layer's investment in Section 3 traces back to a risk row.
• Section 3 (layers). The pyramid's two rules from practical-test-pyramid (opens in new window): "Write tests with different granularity" and "The more high-level you get the fewer tests you should have". Keep E2E to the critical paths; per the same article, "Due to their high maintenance cost you should aim to reduce the number of end-to-end tests to a bare minimum."
• Section 3 (security and a11y rows). "The OWASP Top 10 is the reference standard for the most critical web application security risks" (owasp-top10 (opens in new window)); WCAG 2.2 is a W3C Recommendation (12 December 2024) with "three levels of conformance ... A (lowest), AA, and AAA (highest)" (wcag22 (opens in new window)). Write down which level the product must meet and who set it (a law, a contract, a policy).
• Section 7 (exit criteria). ISTQB: exit criteria are conditions "agreed upon with the stakeholders", and "Exit criteria are used to report against and to plan when to stop testing" (istqb-exit (opens in new window)). That is why each one must be measurable and signed off in Approval.

Step 1a - Validate the draft before approval

Before circulating for sign-off, verify the draft against its own exit gate:

• Assert every in-scope feature in Section 1 maps to a risk class in Section 2, a layer + coverage target in Section 3, and at least one exit criterion in Section 7.
• Assert every Section 7 exit criterion is measurable (a number or a green/red check), not a subjective phrase.

If a section cannot be filled - for example the risk matrix in Section 2 does not exist yet - record it as an open question in Section 11 rather than leaving the section blank, resolve it, then re-run this check. Collect the Approval sign-offs only once no Section 11 item still blocks release.

Step 2 - Tailor per project size

Step 3 - Review cadence

Step 4 - Reference patterns

Common strategy patterns by product type:

Step 5 - Risk-based test planning (per feature / per quarter)

The strategy sets the frame; a per-feature plan allocates the hours. Given a feature scope, the current risk matrix (per risk-matrix), an engineer-week budget, and the team's tooling inventory, produce a risk-prioritized test plan:

1. Identify scope-relevant risks. Direct hits (the feature modifies code in a risk's source paths) plus area-level hits (same product area as an existing risk row). No matrix, no plan - author the matrix first.
2. Map each risk to test types per risk-matrix Step 4 (business logic -> unit + property-based + UAT; technical -> integration + chaos + load; regulatory -> stakeholder UAT + compliance review; and so on).
3. Estimate effort per test type (rough starting points the team calibrates): unit 0.5h, property-based 2h, integration / contract 4h, E2E 8h, manual / UAT 4h per scenario, visual regression 2h per baseline, load 1d, chaos 2d, threat model 1d.
4. Emit the plan with a per-risk table (risk, score, class, test types, effort, owner), a budget-allocation summary with ~20% headroom for incident response and retests, and a "Risks NOT addressed (intentional)" table naming each skipped risk and why - the audit trail that distinguishes prioritization from neglect.

## Test plan - Feature `Promo banner v2`

**Risks implicated:** 6 (of 23) · **Budget:** 2 engineer-weeks · **Estimated:** 1.6 weeks (within budget)

| Risk | Score | Class | Test types | Effort | Owner |
|---|---:|---|---|---|---|
| R-1 Promo math | 15 | Business logic | + 4 unit + 1 property-based | 4 hours | Alice |
| R-2 Stripe webhook | 16 | Technical | + 1 integration + 1 chaos | 1.5 days | Bob |
| R-3 EU tax calc | 10 | Regulatory | + 1 UAT with finance | 4 hours | Carol |

### Risks NOT addressed (intentional)

| Risk | Score | Why skipped |
|---|---:|---|
| R-15 Old promo CMS migration | 3 | Below threshold; manual smoke covers. |

Planning guardrails: never plan more than 20% over budget; never skip the highest-scored Critical risk (>=15) under budget pressure - escalate to product instead; never pick test types the team has no tooling for. Version plans in markdown alongside the matrix and track revisions rather than silently rewriting prior versions.

Anti-patterns

Limitations

• Strategy ≠ execution. A great strategy doesn't ensure great testing - pair with per-feature plans + per-PR enforcement.
• Quarterly cadence is a floor. Fast-moving products may need more frequent revisits.
• Stakeholder buy-in required. A strategy nobody read is paperwork; circulate, present, get sign-off.

References

• istqb-strategy (opens in new window) - ISTQB glossary: test strategy.
• istqb-rbt (opens in new window) - ISTQB glossary: risk-based testing.
• istqb-exit (opens in new window) - ISTQB glossary: exit criteria.
• pyramid (opens in new window) - Martin Fowler site, "The Practical Test Pyramid": test granularity, fewer high-level tests.
• owasp-top10 (opens in new window) - OWASP Top 10 (2025 edition).
• wcag22 (opens in new window) - WCAG 2.2, W3C Recommendation, conformance levels.
• risk-matrix - feeds Section 2.
• test-pyramid-balancer - feeds Section 3.
• definition-of-done - feeds Section 7.