Testland
Browse all skills & agents

definition-of-done

The team's Definition of Done (DoD), both halves of the lifecycle: authoring and auditing. Explains the Scrum Guide's DoD definition ("a formal description of the state of the Increment when it meets the quality measures required for the product"), proposes a starter DoD with the 7-10 lines most teams need (code reviewed, unit tests, docs, AC met, deployed to staging, smoke passed, no a11y regressions, telemetry wired), emits a per-PR checklist a reviewer enforces, and audits work against an existing DoD line by line with repository evidence (review records, diffs, CI runs, coverage reports), tagging every line met, not met, or unverifiable - never passing a line on self-attestation. Use when the team doesn't have a DoD, wants to revise theirs, or is about to mark a story or PR done and nobody has checked the work against the committed checklist.

Install with skills.sh (any agent)

npx skills add testland/qa --skill definition-of-done
View source

definition-of-done

Overview

Per scrum-guide (opens in new window):

"The Definition of Done is a formal description of the state of the Increment when it meets the quality measures required for the product."

"If the Definition of Done for an increment is part of the standards of the organization, all Scrum Teams must follow it as a minimum. If it is not an organizational standard, the Scrum Team must create a Definition of Done appropriate for the product." (scrum-guide (opens in new window))

The DoD is the team's contract with itself. Without it, "done" varies per PM / per developer / per sprint.

When to use

  • A new team needs to author its first DoD.
  • An existing team's DoD is stale (years old; doesn't reflect current quality bar).
  • A retro identified that "done" means different things to different team members.
  • An audit / compliance review needs a documented quality bar.

Starter DoD (recommended baseline)

Most teams converge on something like:

# Definition of Done - `<team>`

A story / PR is "Done" only when ALL of the following are true:

1. **Code reviewed** by at least one other engineer; review approval
   recorded.
2. **Tests** for new behavior:
   - Unit test coverage on changed files >=80%.
   - At least one test per acceptance criterion.
3. **Documentation** updated:
   - User-facing changes have updated docs (or "no user-facing
     change" is documented).
   - API changes have updated OpenAPI / Swagger / type definitions.
4. **Acceptance criteria** from the story all pass (manual or
   automated; per AC ID).
5. **Deployed to staging** and smoke test passed (per
   `smoke-suite-gate`).
6. **No new accessibility regressions** (axe / pa11y / Lighthouse
   a11y category green vs main).
7. **Telemetry / observability** wired for new features (per
   `synthetic-monitor-author` in the qa-shift-right plugin).
8. **Security review** for changes that touch auth / payments /
   PII - threat-model entry recorded.
9. **No new tech debt** introduced without an issue logged.
10. **Build green** on the target branch (CI check required).

Customize per team - not all 10 apply to every project.

Per-organization vs per-team

Per scrum-guide (opens in new window):

"If the Definition of Done for an increment is part of the standards of the organization, all Scrum Teams must follow it as a minimum."

If the org has a security review requirement, every team's DoD includes it. The team's DoD can add stricter team-specific lines on top, but can't drop org-wide minimums.

Categorization of DoD lines

CategoryExamplesVerifier
Code qualityReviewed, no new lint errorsCode review tool
Test coverageUnit test for new behavior, coverage thresholdCI coverage gate
DocumentationUser-facing docs, API spec updatedManual review
Spec complianceAll AC metManual or automated AC tests
Deploy stateStaging deploy + smokeCI deploy job
Quality barA11y regression, perf budgetCI gate
ObservabilityTelemetry / monitoring wiredManual review of diff
SecurityThreat model entry for sensitive changesManual review
ProcessIssue logged, retro feedback recordedManual

The categorization helps when adding lines: "do we have a coverage of the security category?"

Generated per-PR checklist

The DoD becomes a PR template:

<!-- .github/pull_request_template.md -->

## Description

(what changed and why)

## Definition of Done checklist

- [ ] Code reviewed by ≥1 engineer.
- [ ] Unit tests added for new behavior; coverage check passes.
- [ ] Documentation updated (or "no user-facing change" noted below).
- [ ] All AC from the story pass.
- [ ] Deployed to staging; smoke suite green.
- [ ] No new a11y regressions (axe report attached for UI changes).
- [ ] Telemetry / monitoring wired for new features.
- [ ] Security review for auth / payment / PII changes (link to threat model).

## Notes

(any DoD line marked N/A - explain)

A reviewer reads this template + the actual PR state and verifies each line.

DoD evolution

The DoD should evolve with the team's quality bar:

  • After an a11y incident → add the a11y line.
  • After a security incident → add the threat-model line.
  • After production-canary saved a regression → add the canary-passed line.

Don't lower the DoD without explicit retro discussion. Lowering because "we kept failing the gate" usually means the team needs better tooling, not a lower bar.

Review cadence

CadenceTrigger
QuarterlySchedule a 30-min DoD review.
Post-incidentAdd lines that would have prevented the incident.
New team member onboardingReread the DoD; pick up new perspective.

Auditing adherence

The other half of the lifecycle: a compliance audit of work against the checklist that already exists. The checklist is an input, treated as fixed for the duration of the audit; the output is a per-line verdict backed by artifacts. Authoring is collaborative and can trade a line away; auditing is adversarial and cannot. If a line turns out to be unenforceable, that is a finding to hand back to the authoring conversation, not a licence to skip it.

Two facts from the 2020 Scrum Guide (scrumguides.org/scrum-guide.html (opens in new window)) anchor the audit: the team or organization owns the lines (so the audit never invents or edits a line), and a miss has a defined consequence - an item that does not meet the DoD "cannot be released or even presented at the Sprint Review" and returns to the Product Backlog, so "not met" is not advisory. Do not over-attribute: the November 2020 revision made the DoD a formal commitment attached to the Increment (scrumguides.org/revisions.html (opens in new window)); the Guide prescribes no checklist content, coverage number, or verdict vocabulary - those are practitioner convention. Many teams also run an entry-side "ready for development" checklist; it is auditable with the same method but has no standing in the Scrum Guide, so declare which stage is being audited before starting ("acceptance criteria" at entry means they exist and are testable; at exit it means covered by tests that ran and passed).

Audit setup

  • Pin the checklist revision and the evidence window. Record the checklist file path and commit / date, and what counts as evidence: for a PR, the diff range and the CI runs attached to the head commit; for a release, the commit range in the tag. A green CI run for a different SHA proves nothing about this one.
  • Split the checklist into atomic lines. A compound line ("coverage >=80% and one test per AC") is audited as two lines. Do not derive lines from prose - if the checklist is a paragraph, hand it back for restructuring rather than substituting the auditor's reading for the team's intent.
  • Handle conditional lines explicitly. "Docs updated (or no user-facing change)" has two satisfying branches; show evidence for whichever branch is claimed. "No files under the UI or API surface changed in the diff" is checkable; a bare assertion of the escape branch is not.
  • Scope one audit to one artifact - one PR, one story, or one release candidate.

Line-pattern to evidence mapping

Line patternArtifact to readHow to read it
"reviewed by at least N engineers"gh pr view --json reviews,reviewDecisionreviews and reviewDecision are documented JSON fields of gh pr view (cli.github.com/manual/gh_pr_view (opens in new window)). Count approving reviews whose author is not the change author. Review actions are APPROVE, REQUEST_CHANGES, COMMENT (docs.github.com/en/rest/pulls/reviews (opens in new window)).
"coverage on changed files at or above X%"coverage/lcov.info or the CI coverage reportIn an LCOV tracefile each file section carries LF: (instrumented lines) and LH: (lines hit) (manpages.debian.org geninfo(1) (opens in new window)). Per-file rate is LH / LF, computed only over files present in the diff.
"no new lint or type errors"CI status checks for the head commitgh pr view --json statusCheckRollup; compare against the same checks on the base.
"documentation updated"git diff --name-only <base>..<head> (git-scm.com/docs/git-diff (opens in new window))Require at least one changed path under the docs surface. Necessary, not sufficient: it shows a doc changed, not that it describes this change.
"every AC is covered by a passing test"Criterion IDs in the ticket + test names / tags in the diffNeeds a team-declared convention linking criterion IDs to tests. Without one, unverifiable, not not met.
"no new accessibility violations"The a11y scan output for both branchesAn axe run returns a violations array (github.com/dequelabs/axe-core (opens in new window)). Compare head against base, not against zero, unless the line says zero.
"deployed to staging and smoke passed"CI runs bound to the head commitA completed deploy job and a completed smoke job, both for that exact SHA.
"telemetry wired for new code paths"The diff itselfSearch for the project's instrumentation calls on the newly added paths. A statement in the description is not evidence.
"security review for auth / payments / PII changes"Linked review or threat-model recordThe link must resolve and be dated within the change window. Unlinked = unverifiable.
"no new tech debt without a logged issue"Linked issueunverifiable by construction unless the team requires an issue link in the description.

Thresholds come from the team's line, never from a default; if the line names no number ("good test coverage"), it is unverifiable and the finding is that the line is unenforceable as written.

Three states, four evidence standards, one verdict

StateMeaningRequired to assign it
metArtifact exists and shows the line satisfiedName the artifact and the value read from it
not metArtifact located and read; shows the line unsatisfiedName the artifact and the shortfall (actual vs required)
unverifiableCannot be settled from artifacts in the windowState what was searched, why inconclusive, and what the team must supply

The three states apply per line and are never averaged - a checklist is not scored out of ten. unverifiable is a real finding: the line names no checkable condition, the convention does not exist, or the artifact was never produced.

Evidence standards: self-attestation is not evidence (a ticked PR-template checkbox is the claim under audit, not proof); a doc claim with no diff is not met; pre-existing failure is not a free pass (a budget already breached on base does not become an acceptable baseline); urgency and size do not waive lines (a faster hotfix bar is a separate written checklist, not an in-flight waiver).

ConditionVerdict
Every line metACCEPT
Any line not metREJECT
No not met, at least one unverifiableREJECT, pending named human confirmation
No checklist found, or the checklist is unsplit proseINCONCLUSIVE

unverifiable never auto-passes - it clears only when a named person supplies the missing evidence, recorded like any other artifact. No checklist means no verdict: inventing a plausible generic one substitutes the auditor's judgment for the team's, which the Scrum Guide assigns to the team or the organization.

Audit report shape

## DoD adherence review - <artifact under audit>

**Stage:** stage-1-entry | stage-2-exit
**Checklist source:** <path> @ <revision>, <N> lines split into <M> audited lines
**Evidence window:** <base>..<head>, CI runs for <sha>
**Verdict:** ACCEPT | REJECT | INCONCLUSIVE
**met: <x>   not met: <y>   unverifiable: <z>**

| # | Checklist line | State | Evidence sought | What was found |
|---|---|---|---|---|
| 1 | ... | met | ... | ... |

Follow with a "Not met" block (required vs actual vs artifact per line), an "Unverifiable" block (searched / blocked by / to resolve), and a one-sentence recommended action tied to the verdict rules. A full stage-2 audit worked end to end is in references/adherence-worked-example.md.

Audit limitations: roughly half of a typical checklist is not machine-checkable (review quality, "no new tech debt") and resolves to unverifiable by construction; presence is not correctness (nothing reads the doc to confirm it describes the change); evidence outside the window is invisible unless recorded; and the verdict is advice, not enforcement - deciding to ship anyway is a human decision worth recording next to the report.

Anti-patterns

Anti-patternWhy it failsFix
No DoDPer scrum-guide (opens in new window), the team must define one if not org-mandated.Adopt the starter (above).
30-line DoD that nobody can satisfy per-PRTeam marks everything "N/A"; defeats the purpose.Trim to 7-10 lines covering the highest-value gates.
DoD as aspirational, not enforced"We have a DoD but PRs ship without meeting it."Enforce it; PRs blocked on unmet lines.
Per-PR DoD different from team's "done"Drift; team can't tell what's done.One DoD; one PR template referencing it.
Lowering the DoD when it gets in the wayQuality bar erodes silently.Discuss in retro before lowering; document the reason.
DoD never updated after an incidentSame incident class recurs.Post-incident DoD review (above cadence).
Treating unverifiable as a pass in an auditHides the unknown; the team believes it is done when nothing was checked.Verdict rules: unverifiable blocks until a named human supplies evidence.
Auditing against a generic checklist because the team has noneSubstitutes the auditor's bar for the team's (scrum-guide (opens in new window)).Report INCONCLUSIVE; author the DoD first (this skill's other half).
Rewriting or softening a line mid-auditThe checklist stops being a fixed bar and becomes negotiable under deadline.Record the finding; renegotiate in the authoring conversation afterwards.
Passing a coverage line using the repo-wide percentageA high repo-wide number hides an untested new file.Compute per-file rates over the diff's files only.

Limitations

  • Subjective per-team. "What's good enough" varies; the DoD reflects the team's context, not a universal standard.
  • Coverage thresholds are arbitrary. "≥80%" is convention; the right number depends on the codebase.
  • Some lines are unverifiable automatically ("docs updated" needs human review). The DoD acknowledges these - mark them "unverifiable" rather than auto-passing.

References

  • sg (opens in new window) - Scrum Guide DoD: "a formal description of the state of the Increment when it meets the quality measures required for the product"; team owns it (or follows org-mandated minimum); items not Done return to the Product Backlog.
  • smoke-suite-gate - CI gate that satisfies the "staging deploy + smoke passed" line.

Worked example - DoD adherence review

View source (opens in new window)

Worked example - DoD adherence review

Deep reference for the definition-of-done SKILL.md "Auditing adherence" section. A full stage-2 audit end to end, using the audit report shape in the spine. Note at the end what the example deliberately does NOT do - those are authoring-side moves the audit never makes.

Checklist (docs/definition-of-done.md @ a91c4f2, stage 2), audited against pull request 4567, diff main..feat/promo-codes:

1. Code reviewed by at least one other engineer.
2. Unit test coverage on changed files >= 80%.
3. User-facing documentation updated (or no user-facing change).
4. All acceptance criteria from the story pass.
5. Deployed to staging and smoke suite green.
6. No new accessibility violations.
7. Telemetry wired for new features.

Output:

## DoD adherence review - PR #4567 "Add promo codes to checkout"

**Stage:** stage-2-exit
**Checklist source:** docs/definition-of-done.md @ a91c4f2, 7 lines split into 7 audited lines
**Evidence window:** main..feat/promo-codes (head 3f0be21), CI runs for 3f0be21
**Verdict:** REJECT
**met: 4   not met: 2   unverifiable: 1**

| # | Checklist line | State | Evidence sought | What was found |
|---|---|---|---|---|
| 1 | Reviewed by >=1 other engineer | met | approving reviews from non-authors | 2 approving reviews, neither by the author |
| 2 | Coverage on changed files >= 80% | not met | LH/LF per changed file in coverage/lcov.info | src/checkout/promo.ts LF:112 LH:73, 65.2% |
| 3 | Docs updated, or no user-facing change | met | changed docs paths, or absence of user-facing diff | escape branch: diff touches no UI, API schema, or copy files |
| 4 | All acceptance criteria pass | unverifiable | tests named or tagged for AC-1..AC-3 | no criterion-to-test convention exists in this repo |
| 5 | Deployed to staging, smoke green | not met | deploy + smoke jobs for 3f0be21 | no deploy job ran for this SHA |
| 6 | No new accessibility violations | met | violations array, head vs base | 4 violations on both branches, same rule IDs, none new |
| 7 | Telemetry wired | met | instrumentation calls on new code paths | track('promo.applied') added in src/checkout/promo.ts |

### Not met

**Line 2 - coverage on changed files**
- Required: >= 80% per changed file
- Actual: src/checkout/promo.ts at 65.2% (LF:112, LH:73); other changed files pass
- Artifact: coverage/lcov.info from CI run for 3f0be21

**Line 5 - staging deploy and smoke**
- Required: deployed to staging, smoke suite green
- Actual: no deploy job present in any CI run bound to 3f0be21
- Artifact: CI run list for 3f0be21

### Unverifiable

**Line 4 - acceptance criteria**
- Searched: test names and tags in the diff for AC-1, AC-2, AC-3 as written in the story
- Blocked by: no declared convention linking criterion IDs to tests, so absence of a match proves nothing
- To resolve: either adopt a criterion-ID tag on tests, or have the person who ran the criteria confirm each one and link the run

### Recommended action

REJECT: two lines not met and one unverifiable. Line 4 does not become a pass by
default; a named person confirms it with a linked artifact.

Note what the example does not do: it does not suggest lowering line 2 to 65%, does not drop line 4 as unenforceable, and does not average the seven lines into a percentage. Those are all authoring-side moves.

Related skills

attack-surface-test-checklist

Maps a code change to the security tests worth running against it. Classifies changed paths and file contents into nine attack surfaces (authentication, session management, input handling, file upload, deserialization, access control, API and web service, cryptography, data protection), attaches the matching OWASP ASVS 4.0.3 verification requirements, OWASP Top 10 2021 category IDs, and OWASP WSTG section numbers to each active surface, then emits a per-surface manual and automated test checklist bounded by what actually changed. Surfaces with no changed lines are excluded rather than carried as filler. Use when a pull request, release branch, or feature is about to be security tested and the team needs a targeted test list instead of a generic application-wide checklist.

e2e-suite-budget

Caps E2E suite size by computing per-test ROI - (regressions caught × value) ÷ (runtime × flake rate × maintenance) - then ranks every end-to-end test and recommends which bottom-decile ones to retire, move to a lower layer, or fix. Use when CI is slow or E2E-dominated, flaky failures are rising, or quarterly to keep suite size within maintenance capacity. For strategic unit:service:UI layer ratios use test-pyramid-balancer, for the minimal per-deploy critical-path gate use smoke-suite-gate, and for quarantining flaky tests use flaky-test-quarantine; this prunes low-signal tests by ROI.

framework-choice-advisor

Reference catalog for picking a test automation framework or QA tool - covers Playwright / Cypress / Selenium / WebdriverIO / Appium / Espresso / XCUITest / RestAssured / Karate / k6 / Locust with side-by-side tradeoffs on speed, cross-browser, mobile, parallelisation, language support, ecosystem maturity, CI integration; a decision tree matching project NFRs to framework choice; and reference layouts for the chosen stack. references/ extends the same decision to commercial procurement (seven-axis vendor evaluation for TCM platforms, no-code tools, visual-regression services) and to recording the outcome (ADR-based tool-selection decision record with signal, one recommendation, flip conditions). This is the upstream selection step: it decides which tool to adopt, not how to configure one already chosen. Use when starting a new test-automation suite, evaluating commercial QA vendors, or writing down a tool decision.

post-mortem-author

Build-an-X workflow that produces a blameless post-mortem from an incident - captures the timeline (chronological event sequence with sources), root cause analysis (what + why, not who), impact (users / revenue / SLO debt), action items (with owners + due dates + measurable success criteria), and "what went well" (intentional). Per Google SRE: "Blameless postmortems are a tenet of SRE culture." Use after every user-visible incident, not just severe ones.

risk-matrix

The risk-based testing (RBT) umbrella: risk matrix and risk register authoring, likelihood x impact scoring, risk storming, calibration, and risk-to-test coverage mapping. Produces the per-feature / per-release matrix artifact (structured intake: feature, category, impact 1-5 by likelihood 1-5, score; heatmap; mitigations with owners and due dates), supporting lightweight and heavyweight (FMEA / Cost of Exposure) methods per RBT canon, plus a risk coverage mapping workflow that proves which tests, cases, or monitors back each registered risk. references/ carries the product-risk and project-risk register variants, the risk-storming facilitation guide, matrix calibration against observed defect data, and a register review checklist. Use for any risk-based-testing artifact: building a matrix or register, running a risk-storming session, calibrating ratings against defects, or mapping risks onto test coverage.

smoke-suite-gate

Build-an-X workflow for a critical-path smoke suite that runs in <5 minutes - picks the 5-15 highest-business-value journeys (login, hero flow, checkout, payment, primary read), implements as fast E2E or API tests, gates per-deploy, retries on transient failures with quarantine. Use as the canary-precursor or per-deploy verification gate; the team's "if this fails, the build can't proceed" floor.

test-case-from-live-feature

Build-an-X workflow that produces a test-case matrix from a **live, undocumented feature** - running app at a URL, screen recording, screenshot, or verbal brief - by combining structured exploration (Playwright trace / DevTools / accessibility tree) with the four canonical heuristic test-design models bundled in references/ (Bach's HTSM / SFDPOT product elements, Whittaker's How-to-Break-Software attacks, Bolton's FEW HICCUPPS consistency oracles, ISO/IEC 25010 quality characteristics). Output is a structured case matrix, not an exploratory session charter. Use when there is no story, no AC, and no documentation - only a live feature - or as the heuristic reference layer for zero-documentation test design.

test-case-ideation-from-story

Turns a thin or ambiguous story into a reviewable test list - a backlog item that is a short paragraph plus the click-through support recorded for themselves, a spec that is mostly a list of accepted formats, or a tech design pasted into the ticket while the last few releases still shipped missed cases. Takes the story or feature spec and emits a markdown test-case matrix, one row per case (id, title, precondition, steps, expected, tier), covering happy path, alternate paths, boundaries, and negative paths, before any test code is written. Output is the human-reviewable matrix that goes into TestRail / Qase / Xray, not Gherkin scenarios. Use when a story needs its cases enumerated and agreed before automation starts.

test-effort-estimation

Turns a list of testable areas plus a change-shape distribution into a PERT three-point test effort estimate, reporting every row as a range around the expected value rather than a single number, requiring a named assumptions ledger across six mandatory categories, and recommending a per-layer ownership split across developer, automation, and exploratory roles. Bundles the change-shape classifier (pure-logic / service-layer / ui-heavy / data-heavy from git-history path and content signals, with the relative per-layer cost model) as a reference, so the shape distribution the estimate consumes can be produced here too. Does not choose which tests to run or how deep coverage should go. Use when an epic or release has been broken into testable areas and someone is about to commit test capacity for a sprint, or when a change set needs its shape classified before planning.

test-pyramid-balancer

Build-an-X workflow that analyzes a repo's test mix (unit / integration / E2E counts + runtimes) and recommends rebalancing toward the test pyramid ratios per the change-set shape - pure-logic-heavy repo wants ~80/15/5; UI-heavy repo wants ~60/25/15. Detects 'ice-cream cone' (E2E-heavy) and 'hourglass' (integration-thin) anti-patterns. Use when the user asks about test distribution, test strategy, test balance, too many E2E tests, slow CI caused by tests, testing best practices, or rebalancing their test suite; also suitable for quarterly calibration of the test mix to codebase reality.

test-strategy-author

Authors a test strategy document (a master test plan) for a project, release, or feature - covers scope, in/out, test types per layer (unit / integration / contract / E2E / perf / security / a11y), risk-based test prioritization that maps top risks to test investment (per `risk-matrix`), tooling stack, environments, exit criteria, and ownership. Includes a risk-based test-planning workflow that turns a feature scope plus the risk matrix into a budgeted per-risk test plan with owners, effort estimates, and an explicit risks-not-addressed section. Use when a team needs the release-readiness artifact stakeholders sign off on before significant test investment, or a risk-prioritized test plan for a feature or quarter.