test-strategy-author

Authors a test strategy document (a master test plan) for a project, release, or feature - covers scope, in/out, test types per layer (unit / integration / contract / E2E / perf / security / a11y), risk-based test prioritization that maps top risks to test investment (per `risk-matrix`), tooling stack, environments, exit criteria, and ownership. Includes a risk-based test-planning workflow that turns a feature scope plus the risk matrix into a budgeted per-risk test plan with owners, effort estimates, and an explicit risks-not-addressed section. Use when a team needs the release-readiness artifact stakeholders sign off on before significant test investment, or a risk-prioritized test plan for a feature or quarter.

Install with skills.sh (any agent)

npx skills add testland/qa --skill test-strategy-author

View source

test-strategy-author

Overview

A test strategy document answers the question "how do we know we're testing the right things?" Without it, test investment is gut-feel; with it, the team has a defensible artifact tying test work to risk and business goals.

This skill produces the document.

When to use

A new product / large feature is in planning; the team needs to scope test investment.
A multi-quarter initiative needs a published strategy stakeholders can review.
An audit / compliance review requires a documented strategy.
A new team forming around a product needs onboarding context.

For per-feature test plans (smaller scope), use the lighter-weight risk-matrix - strategy is for larger horizons.

Step 1 - Document structure

# Test Strategy - `<product / release>`

**Author:** _______________   **Date:** YYYY-MM-DD
**Status:** Draft | Review | Approved
**Stakeholders:** Engineering, Product, QA, Compliance

## 1. Scope

### In scope
- (List of features / surfaces this strategy covers)

### Out of scope
- (Explicit list of what's NOT covered, with rationale)

### Assumptions
- (List of context the strategy depends on)

## 2. Risk basis

(Reference the risk matrix per `risk-matrix`)

| Risk class | Top risks (from matrix) | Test investment |
|------------|-------------------------|-----------------|
| Business    | Promo math, Tax calculations | Property-based + UAT |
| Technical   | Webhook reliability, DB migrations | Chaos + integration |
| Regulatory  | EU GDPR, CCPA          | UAT + privacy review |
| Performance | Checkout latency        | Load + canary       |

## 3. Test types per layer (the pyramid)

Per the test pyramid (`test-pyramid-balancer`):

| Layer       | Coverage target | Tools | Owner |
|-------------|----------------:|-------|-------|
| Unit         |       80%       | Jest, pytest, JUnit | Devs |
| Integration  |       60%       | Testcontainers, supertest | Devs |
| Contract     |       100% of consumer-provider pairs | Pact | Devs |
| E2E          |       Critical paths (5-10 flows) | Playwright | QA |
| Performance  |       Critical endpoints | k6 + Lighthouse CI | QA + SRE |
| Security     |       OWASP Top 10 | Schemathesis + manual pen test | Security |
| A11y         |       WCAG 2.2 AA | axe + manual review | QA |

## 4. Tooling

(Stack inventory per layer; references to specific skills)

## 5. Environments

- **Local dev** - per-engineer; Testcontainers backing services.
- **Staging** - shared; smoke + UAT.
- **Canary** - 5% prod traffic; 30-min observation per
  prod-canary-validator.
- **Prod** - synthetic monitors per
  synthetic-monitor-author.

## 6. Test data

- **Synthetic accounts** - per synthetic-data-toolkit.
- **PII handling** - per synthetic-pii-generator.
- **Database state** - per database snapshot / restore.

## 7. Exit criteria

Release ships when:

- All AC for in-scope features pass.
- All unit/integration tests green; no flake in last 3 main runs.
- E2E critical-path suite green.
- Coverage targets met per Section 3.
- Performance targets met (p95 within budget).
- A11y regression scan green.
- Threat model reviewed for security-touching changes.
- Risk matrix Critical (>=15) all mitigated.

## 8. Ownership

| Activity              | Owner | Backup |
|-----------------------|-------|--------|
| Unit test reviews      | Devs  | Tech lead |
| E2E suite maintenance  | QA    | Dev TPM |
| Perf budget approval   | SRE   | QA |
| Threat model authorship | Security | Dev TPM |
| Synthetic monitors     | SRE   | QA |
| Risk matrix updates    | QA    | Product |

## 9. Cadence

- **Per-PR:** Lint, unit, integration, smoke E2E, coverage delta.
- **Per-merge to main:** Full E2E, perf gate.
- **Nightly:** Full regression, mutation testing weekly.
- **Pre-release:** Manual UAT sign-off, full security scan,
  release-readiness check.

## 10. Risk register snapshot

(Top 10 risks from the matrix as of strategy authoring date)

## 11. Open questions / decisions needed

- (List of unresolved items requiring stakeholder input)

## Approval

- [ ] Engineering manager
- [ ] QA lead
- [ ] Product manager
- [ ] Security (if applicable)

Step 1a - Validate the draft before approval

Before circulating for sign-off, verify the draft against its own exit gate:

Assert every in-scope feature in Section 1 maps to a risk class in Section 2, a layer + coverage target in Section 3, and at least one exit criterion in Section 7.
Assert every Section 7 exit criterion is measurable (a number or a green/red check), not a subjective phrase.

If a section cannot be filled - for example the risk matrix in Section 2 does not exist yet - record it as an open question in Section 11 rather than leaving the section blank, resolve it, then re-run this check. Collect the Approval sign-offs only once no Section 11 item still blocks release.

Step 2 - Tailor per project size

Project size	Strategy length	Scope
Small (1-3 mo)	2-3 pages	Sections 1-3 + 7
Medium (1-2 quarters)	5-7 pages	All sections
Large (multi-quarter)	10+ pages	All + sub-strategies per major feature

Step 3 - Review cadence

Trigger	Action
Quarterly	Re-review; update sections 2 (risks), 7 (exit), 10 (snapshot).
Major architectural change	Re-author Sections 3 + 6.
Post-incident	Update Section 2 (new risks); Section 7 (new exit criterion).
New compliance requirement	Section 1 + Section 7.

Step 4 - Reference patterns

Common strategy patterns by product type:

Product type	Strategy emphasis
E-commerce	Business risks (promo math, currency, tax); UAT-heavy.
SaaS B2B	Security + multi-tenancy; contract testing critical.
Mobile native	Per-platform matrix; manual tier; performance.
Internal tooling	Light strategy; integration over E2E.
Regulated (fintech, health)	Heavyweight RBT (FMEA); auditable trail.

Step 5 - Risk-based test planning (per feature / per quarter)

The strategy sets the frame; a per-feature plan allocates the hours. Given a feature scope, the current risk matrix (per risk-matrix), an engineer-week budget, and the team's tooling inventory, produce a risk-prioritized test plan:

Identify scope-relevant risks. Direct hits (the feature modifies code in a risk's source paths) plus area-level hits (same product area as an existing risk row). No matrix, no plan - author the matrix first.
Map each risk to test types per risk-matrix Step 4 (business logic -> unit + property-based + UAT; technical -> integration + chaos + load; regulatory -> stakeholder UAT + compliance review; and so on).
Estimate effort per test type (rough starting points the team calibrates): unit 0.5h, property-based 2h, integration / contract 4h, E2E 8h, manual / UAT 4h per scenario, visual regression 2h per baseline, load 1d, chaos 2d, threat model 1d.
Emit the plan with a per-risk table (risk, score, class, test types, effort, owner), a budget-allocation summary with ~20% headroom for incident response and retests, and a "Risks NOT addressed (intentional)" table naming each skipped risk and why - the audit trail that distinguishes prioritization from neglect.

## Test plan - Feature `Promo banner v2`

**Risks implicated:** 6 (of 23) · **Budget:** 2 engineer-weeks · **Estimated:** 1.6 weeks (within budget)

| Risk | Score | Class | Test types | Effort | Owner |
|---|---:|---|---|---|---|
| R-1 Promo math | 15 | Business logic | + 4 unit + 1 property-based | 4 hours | Alice |
| R-2 Stripe webhook | 16 | Technical | + 1 integration + 1 chaos | 1.5 days | Bob |
| R-3 EU tax calc | 10 | Regulatory | + 1 UAT with finance | 4 hours | Carol |

### Risks NOT addressed (intentional)

| Risk | Score | Why skipped |
|---|---:|---|
| R-15 Old promo CMS migration | 3 | Below threshold; manual smoke covers. |

Planning guardrails: never plan more than 20% over budget; never skip the highest-scored Critical risk (>=15) under budget pressure - escalate to product instead; never pick test types the team has no tooling for. Version plans in markdown alongside the matrix and track revisions rather than silently rewriting prior versions.

Anti-patterns

Anti-pattern	Why it fails	Fix
Generic strategy template not tailored to product	Reads like boilerplate; nobody references.	Tailor per Section 1 (scope) and Section 4 (tooling).
Plan ignores the effort budget	Idealistic; team can't execute.	Match plan to budget (Step 5).
Skipping Critical risks under budget pressure	Defeats the prioritization purpose.	Escalate to product instead (Step 5 guardrails).
No "risks NOT addressed" section in the plan	Audit gap; prioritization indistinguishable from neglect.	Always include it (Step 5).
Strategy authored once; never reviewed	Becomes obsolete; team distrusts.	Quarterly cadence (Step 3).
No exit criteria	"Done" is subjective; release decisions arbitrary.	Section 7 - explicit + measurable.
Strategy = test plan	Strategy is high-level; per-feature plans are detailed (different artifact).	Use `risk-matrix` for per-feature.
Owner column missing	Activities slip; "everyone owns it = nobody."	Section 8 explicit owners.
Strategy in slides, not version-controlled	History lost; updates invisible.	Markdown + git.

Limitations

Strategy ≠ execution. A great strategy doesn't ensure great testing - pair with per-feature plans + per-PR enforcement.
Quarterly cadence is a floor. Fast-moving products may need more frequent revisits.
Stakeholder buy-in required. A strategy nobody read is paperwork; circulate, present, get sign-off.

References

risk-matrix - feeds Section 2.
test-pyramid-balancer - feeds Section 3.
definition-of-done - feeds Section 7.

Related skills

attack-surface-test-checklist

Maps a code change to the security tests worth running against it. Classifies changed paths and file contents into nine attack surfaces (authentication, session management, input handling, file upload, deserialization, access control, API and web service, cryptography, data protection), attaches the matching OWASP ASVS 4.0.3 verification requirements, OWASP Top 10 2021 category IDs, and OWASP WSTG section numbers to each active surface, then emits a per-surface manual and automated test checklist bounded by what actually changed. Surfaces with no changed lines are excluded rather than carried as filler. Use when a pull request, release branch, or feature is about to be security tested and the team needs a targeted test list instead of a generic application-wide checklist.

definition-of-done

The team's Definition of Done (DoD), both halves of the lifecycle: authoring and auditing. Explains the Scrum Guide's DoD definition ("a formal description of the state of the Increment when it meets the quality measures required for the product"), proposes a starter DoD with the 7-10 lines most teams need (code reviewed, unit tests, docs, AC met, deployed to staging, smoke passed, no a11y regressions, telemetry wired), emits a per-PR checklist a reviewer enforces, and audits work against an existing DoD line by line with repository evidence (review records, diffs, CI runs, coverage reports), tagging every line met, not met, or unverifiable - never passing a line on self-attestation. Use when the team doesn't have a DoD, wants to revise theirs, or is about to mark a story or PR done and nobody has checked the work against the committed checklist.

e2e-suite-budget

Caps E2E suite size by computing per-test ROI - (regressions caught × value) ÷ (runtime × flake rate × maintenance) - then ranks every end-to-end test and recommends which bottom-decile ones to retire, move to a lower layer, or fix. Use when CI is slow or E2E-dominated, flaky failures are rising, or quarterly to keep suite size within maintenance capacity. For strategic unit:service:UI layer ratios use test-pyramid-balancer, for the minimal per-deploy critical-path gate use smoke-suite-gate, and for quarantining flaky tests use flaky-test-quarantine; this prunes low-signal tests by ROI.

framework-choice-advisor

Reference catalog for picking a test automation framework or QA tool - covers Playwright / Cypress / Selenium / WebdriverIO / Appium / Espresso / XCUITest / RestAssured / Karate / k6 / Locust with side-by-side tradeoffs on speed, cross-browser, mobile, parallelisation, language support, ecosystem maturity, CI integration; a decision tree matching project NFRs to framework choice; and reference layouts for the chosen stack. references/ extends the same decision to commercial procurement (seven-axis vendor evaluation for TCM platforms, no-code tools, visual-regression services) and to recording the outcome (ADR-based tool-selection decision record with signal, one recommendation, flip conditions). This is the upstream selection step: it decides which tool to adopt, not how to configure one already chosen. Use when starting a new test-automation suite, evaluating commercial QA vendors, or writing down a tool decision.

post-mortem-author

Build-an-X workflow that produces a blameless post-mortem from an incident - captures the timeline (chronological event sequence with sources), root cause analysis (what + why, not who), impact (users / revenue / SLO debt), action items (with owners + due dates + measurable success criteria), and "what went well" (intentional). Per Google SRE: "Blameless postmortems are a tenet of SRE culture." Use after every user-visible incident, not just severe ones.

risk-matrix

The risk-based testing (RBT) umbrella: risk matrix and risk register authoring, likelihood x impact scoring, risk storming, calibration, and risk-to-test coverage mapping. Produces the per-feature / per-release matrix artifact (structured intake: feature, category, impact 1-5 by likelihood 1-5, score; heatmap; mitigations with owners and due dates), supporting lightweight and heavyweight (FMEA / Cost of Exposure) methods per RBT canon, plus a risk coverage mapping workflow that proves which tests, cases, or monitors back each registered risk. references/ carries the product-risk and project-risk register variants, the risk-storming facilitation guide, matrix calibration against observed defect data, and a register review checklist. Use for any risk-based-testing artifact: building a matrix or register, running a risk-storming session, calibrating ratings against defects, or mapping risks onto test coverage.

smoke-suite-gate

Build-an-X workflow for a critical-path smoke suite that runs in <5 minutes - picks the 5-15 highest-business-value journeys (login, hero flow, checkout, payment, primary read), implements as fast E2E or API tests, gates per-deploy, retries on transient failures with quarantine. Use as the canary-precursor or per-deploy verification gate; the team's "if this fails, the build can't proceed" floor.

test-case-from-live-feature

Build-an-X workflow that produces a test-case matrix from a **live, undocumented feature** - running app at a URL, screen recording, screenshot, or verbal brief - by combining structured exploration (Playwright trace / DevTools / accessibility tree) with the four canonical heuristic test-design models bundled in references/ (Bach's HTSM / SFDPOT product elements, Whittaker's How-to-Break-Software attacks, Bolton's FEW HICCUPPS consistency oracles, ISO/IEC 25010 quality characteristics). Output is a structured case matrix, not an exploratory session charter. Use when there is no story, no AC, and no documentation - only a live feature - or as the heuristic reference layer for zero-documentation test design.

test-case-ideation-from-story

Turns a thin or ambiguous story into a reviewable test list - a backlog item that is a short paragraph plus the click-through support recorded for themselves, a spec that is mostly a list of accepted formats, or a tech design pasted into the ticket while the last few releases still shipped missed cases. Takes the story or feature spec and emits a markdown test-case matrix, one row per case (id, title, precondition, steps, expected, tier), covering happy path, alternate paths, boundaries, and negative paths, before any test code is written. Output is the human-reviewable matrix that goes into TestRail / Qase / Xray, not Gherkin scenarios. Use when a story needs its cases enumerated and agreed before automation starts.

test-effort-estimation

Turns a list of testable areas plus a change-shape distribution into a PERT three-point test effort estimate, reporting every row as a range around the expected value rather than a single number, requiring a named assumptions ledger across six mandatory categories, and recommending a per-layer ownership split across developer, automation, and exploratory roles. Bundles the change-shape classifier (pure-logic / service-layer / ui-heavy / data-heavy from git-history path and content signals, with the relative per-layer cost model) as a reference, so the shape distribution the estimate consumes can be produced here too. Does not choose which tests to run or how deep coverage should go. Use when an epic or release has been broken into testable areas and someone is about to commit test capacity for a sprint, or when a change set needs its shape classified before planning.

test-pyramid-balancer

Build-an-X workflow that analyzes a repo's test mix (unit / integration / E2E counts + runtimes) and recommends rebalancing toward the test pyramid ratios per the change-set shape - pure-logic-heavy repo wants ~80/15/5; UI-heavy repo wants ~60/25/15. Detects 'ice-cream cone' (E2E-heavy) and 'hourglass' (integration-thin) anti-patterns. Use when the user asks about test distribution, test strategy, test balance, too many E2E tests, slow CI caused by tests, testing best practices, or rebalancing their test suite; also suitable for quarterly calibration of the test mix to codebase reality.