test-case-from-live-feature
Build-an-X workflow that produces a test-case matrix from a **live, undocumented feature** - running app at a URL, screen recording, screenshot, or verbal brief - by combining structured exploration (Playwright trace / DevTools / accessibility tree) with the heuristic models in `heuristic-test-design-reference` (SFDPOT, Whittaker attacks, FEW HICCUPPS, ISO 25010). Output is a structured case matrix, not an exploratory session charter. Use when there is no story, no AC, and no documentation - only a live feature.
Install with skills.sh (any agent)
npx skills add testland/qa --skill test-case-from-live-featuretest-case-from-live-feature
Overview
A tester is told "test the new checkout flow" with no story, no AC, no design doc, but the feature is deployed to staging. The right path is to reverse-engineer a test-case matrix from the live feature, anchored on the four heuristic models in heuristic-test-design-reference, and emit a structured matrix that downstream skills (manual-test-script-author, gherkin-from-stories, ai-test-generator) can consume.
The output is the same shape as test-case-ideation-from-story - one row per case with id / title / tier / precondition / steps / expected / source claim - but the source claim column points at observed behaviour rather than a story sentence, and each row is tagged with the heuristic that surfaced it so the team can audit the coverage logic.
When to use
Do not use this skill when:
Step 1 - Probe the live feature
Capture concrete observations from the running surface. Sources, in order of preference:
| Source | What to capture | Tool |
|---|---|---|
| Live URL / app | All visible actions, fields, validation messages, error states; the URL pattern; the network requests; the rendered DOM | Browser DevTools, Playwright trace, axe-core accessibility tree |
| Screen recording / Loom | The flow the engineer / PM walked through; the implicit assumptions about state | Annotate the recording with timestamps |
| Screenshot set | Static state; what fields exist; what labels say | Inspect element labels and ARIA |
| Verbal brief from an engineer | "It does X and Y" - capture as a quote, do not transcribe as fact | Mark as [verbal, unconfirmed] |
| Existing code (the spec-in-code case) | Public API surface, route definitions, validation rules, DB schema | git log to see recent change scope |
Output of Step 1 is an observation log:
## Observation log - checkout flow @ staging.example.com (2026-05-11 14:00 UTC)
### URLs probed
- `/cart` - cart view; lists line items.
- `/cart/checkout` - multi-step flow: address → shipping → payment → review → confirm.
- `/cart/confirm/:order_id` - confirmation page.
### Network calls observed
- `POST /api/cart/items` (add to cart) → 201, body `{ sku, qty, addedAt }`.
- `POST /api/coupons/apply` → 200 on valid, 409 on already-applied, 422 on expired.
- `POST /api/checkout/payment` → 201 on success, 402 on declined, 5xx on provider-down.
### UI affordances observed
- Coupon field accepts up to 32 chars; case-insensitive in client validation (DOM `text-transform: uppercase`).
- "Place order" button disabled on submit (good - prevents double-click).
- No client-side qty boundary; server returns 422 above qty=99.
### Accessibility tree (axe-core)
- 3 violations on /cart/checkout: missing label on shipping-method radios; insufficient contrast on disabled button; missing live-region on validation errors.
### Verbal brief (engineer Slack message, 2026-05-10)
- "It uses Stripe for cards and PayPal for wallets, and we have a feature flag `new_checkout_v2` defaulting on." [verbal, unconfirmed]Inputs that cannot be confirmed by direct observation are tagged [verbal, unconfirmed] or [claim, unverified] and tracked through the matrix as source claim: observation + [unverified]. This is the audit trail that lets the team disambiguate "tester observed" from "tester was told."
Step 2 - Walk the heuristic models
Apply each model in heuristic-test-design-reference to the observation log, in order:
The full walk applied to the checkout observation log - the SFDPOT table, the per-function Whittaker attacks, the FEW HICCUPPS pre-flight, and the ISO 25010 cross-check - is in references/heuristic-walk-example.md.
Step 3 - Emit the matrix
Same shape as test-case-ideation-from-story output, with two added columns:
| Column | Notes |
|---|---|
| ID | <feature>-LIVE-<n>, e.g. CHECKOUT-LIVE-03. The LIVE infix marks it as heuristically-derived. |
| Title | Imperative single sentence. |
| Tier | smoke / regression / edge / negative / a11y / perf / sec. |
| Precondition | Observed (or [unverified - confirm with PM]). |
| Steps | Numbered, declarative (per Cucumber better-Gherkin (opens in new window)). |
| Expected | Observed behaviour or the FEW HICCUPPS-derived expectation. |
| Source claim | Observation log line + heuristic that surfaced the case (e.g., obs:cart.qty boundary @ DevTools; Whittaker input-attack). |
| Heuristic (new) | Which model surfaced this: SFDPOT-F, Whittaker-input, FEW-HICCUPPS-comparable-products, ISO25010-security, etc. |
| Confidence (new) | observed (saw it directly), inferred (heuristic surfaced it but not yet probed), verbal-unverified (came from a non-canonical source). |
Worked example row
| ID | Title | Tier | Pre | Steps | Expected | Source claim | Heuristic | Confidence |
|---|---|---|---|---|---|---|---|---|
| CHECKOUT-LIVE-07 | Rejects coupon when length exceeds 32 chars | negative | Authenticated session | 1. Open /cart/checkout. 2. Enter coupon of 33 chars. 3. Submit. | Either client validation blocks at 32; or server returns 422. Both behaviours are defensible - observe which the team chose and document. | obs:coupon-input maxlength=32 in DOM; Whittaker input-attack | Whittaker-input | inferred |
| CHECKOUT-LIVE-08 | Idempotent re-POST on /api/checkout/payment | regression | Authenticated session; payment about to submit | 1. Submit payment. 2. Network-throttle the response. 3. Re-submit with the same idempotency key. | Returns the original order id, does not charge twice. | obs:idempotency-key header observed; FEW HICCUPPS-purpose | FEW-HICCUPPS-purpose | inferred |
| CHECKOUT-LIVE-09 | Shipping-method radios have accessible labels | a11y | Authenticated session, address completed | 1. Inspect shipping-method radios. 2. Verify each has an associated <label> or aria-label. | Each radio has an accessible name; screen reader announces it. | obs:axe-core violation @ /cart/checkout; ISO25010-usability; WCAG 2.2 AA | ISO25010-usability | observed |
Confidence-tagged rows give the team an explicit gradient: observed cases can be run immediately; inferred cases are the heuristic's prediction the team should confirm-or-falsify on first run; verbal-unverified cases need product-side validation before they go into the regression suite.
Step 4 - Reconcile with downstream skills
The matrix is the input to the same downstream chain as test-case-ideation-from-story:
The matrix should also be filed with the team's PM / engineer as a documentation byproduct - the heuristic walk often surfaces things the team didn't realise were unspecified, and the matrix becomes the de facto spec for the feature going forward.
Step 5 - Tracker / test-management integration
Per the same conventions as test-case-ideation-from-story: import as CSV into TestRail / Qase / Xray; preserve the Heuristic and Confidence columns as tags so the team can filter "all SFDPOT-F-derived smoke cases" or "all inferred cases awaiting first-run confirmation."
Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Skipping the observation log; jumping straight to heuristic walk | Without the observation log, the matrix's "source claim" column is empty - the team cannot audit which case came from where. | Step 1 produces the observation log first; it is the load-bearing artifact. |
Treating inferred rows as authoritative | Heuristics generate hypotheses, not facts; an inferred row that doesn't reproduce is the heuristic doing its job. | The Confidence column gates downstream automation - inferred cases are probed on first run, not blindly automated. |
| Filing FEW HICCUPPS-derived bugs without naming the lens | The bug report reads "this feels wrong" - undefensible. | Always cite the lens (e.g., FEW-HICCUPPS: Comparable-products + User-expectations). |
| Transcribing the engineer's verbal brief as fact | The brief is the engineer's mental model; mental models leak. | Tag verbal input [verbal, unconfirmed] and probe it against the live surface in Step 1. |
| Running this skill on a feature that already has a story | The story-driven path (test-case-ideation-from-story) is faster and more traceable when a story exists. | Use this skill only when no story / AC / spec exists; combine with the story-driven matrix for thin specs. |
| Probing production directly (instead of staging / canary) | Side effects on real users, real data, real money. | Step 1's "live URL" means staging / canary by default; production probes require a separate authorisation. |
Limitations
Hand-off targets
References
Heuristic walk - worked example
View source (opens in new window)Heuristic walk - worked example
Deep reference for the test-case-from-live-feature SKILL.md, Step 2. The four heuristic models from heuristic-test-design-reference applied to the checkout observation log from Step 1, turning observations into candidate test-case rows. The spine keeps the four-substep method; this file shows the walk in full.
2a - SFDPOT coverage walk
Per HTSM (James Bach (opens in new window)), enumerate cases per Product Element:
| Guideword | From the observation log |
|---|---|
| S - Structure | cart service, payment service, coupon service, idempotency layer (observed via network calls). |
| F - Function | add to cart, edit qty, apply coupon, choose shipping, choose payment, place order, see confirmation. |
| D - Data | SKU, qty, price, coupon code, address, payment method, order id, idempotency key. |
| P - Platform | desktop Chrome / Safari / Firefox; mobile iOS / Android web; observed responsive layout via DevTools. |
| O - Operations | feature flag new_checkout_v2 (verbal, unverified); rollback path unknown. |
| T - Time | cart expiry (unknown - to probe), coupon expiry (422 on expired observed), payment timeout (unknown). |
Each non-empty cell becomes one or more test-case rows.
2b - Whittaker attack overlay
For each function, enumerate the attacks from the Whittaker catalog (opens in new window) (in heuristic-test-design-reference):
2c - FEW HICCUPPS oracle pre-flight
For each observation that already looked wrong, pre-classify with Bolton's FEW HICCUPPS (opens in new window) so the test row carries a defensible verdict frame:
2d - ISO 25010 quality cross-check
Walk the eight (+2) ISO/IEC 25010 (opens in new window) characteristics; add rows for the quality dimensions SFDPOT didn't surface:
Related skills
attack-surface-test-checklist
Maps a code change to the security tests worth running against it. Classifies changed paths and file contents into nine attack surfaces (authentication, session management, input handling, file upload, deserialization, access control, API and web service, cryptography, data protection), attaches the matching OWASP ASVS 4.0.3 verification requirements, OWASP Top 10 2021 category IDs, and OWASP WSTG section numbers to each active surface, then emits a per-surface manual and automated test checklist bounded by what actually changed. Surfaces with no changed lines are excluded rather than carried as filler. Use when a pull request, release branch, or feature is about to be security tested and the team needs a targeted test list instead of a generic application-wide checklist.
code-change-shape-classifier
Classifies a code change set into four shapes (pure-logic, service-layer, ui-heavy, data-heavy) from file-path and file-content signals, computes the shape distribution over a window of git history, and attaches a relative per-layer test cost model (unit 1x, service 3x, UI 10x) so downstream planning works from one shared input. Produces the classification only: it does not prescribe a target unit:service:UI ratio, does not estimate hours, and does not select which tests to run. Use when a pull request, release branch, or epic needs its change shape labelled before test effort, pyramid balance, or coverage depth is decided.
definition-of-done
Pure-reference + checklist-generator for the team's Definition of Done (DoD) - explains the Scrum Guide's DoD definition ("a formal description of the state of the Increment when it meets the quality measures required for the product"), proposes a starter DoD with the 7-10 lines most teams need (code reviewed, unit tests, docs, AC met, deployed to staging, smoke passed, no a11y regressions, telemetry wired, observability in place), and emits a per-PR checklist a reviewer enforces. Use when the team doesn't have a DoD or wants to revise theirs.
dod-adherence-review
Audits an existing Definition of Done checklist line by line against repository evidence (review records, diffs, CI runs, coverage reports, scan output) and tags every line met, not met, or unverifiable, refusing to pass a line on self-attestation or on a claim with no matching diff. Covers the line-pattern-to-evidence mapping for the common checklist shapes (code reviewed, coverage threshold, docs updated, acceptance criteria covered, staging deploy plus smoke, no new accessibility regressions, telemetry wired), the entry-stage versus exit-stage split many teams run, the roll-up verdict rules, and the audit table that gets emitted. Does not author, revise, or soften the checklist. Use when a story or pull request is about to be marked done and a committed Definition of Done exists that nobody has actually checked the work against.
e2e-suite-budget
Caps E2E suite size by computing per-test ROI - (regressions caught × value) ÷ (runtime × flake rate × maintenance) - then ranks every end-to-end test and recommends which bottom-decile ones to retire, move to a lower layer, or fix. Use when CI is slow or E2E-dominated, flaky failures are rising, or quarterly to keep suite size within maintenance capacity. For strategic unit:service:UI layer ratios use test-pyramid-balancer, for the minimal per-deploy critical-path gate use smoke-suite-gate, and for quarantining flaky tests use flaky-test-quarantine; this prunes low-signal tests by ROI.
framework-choice-advisor
Pure reference catalog for picking a test automation framework - covers Playwright / Cypress / Selenium / WebdriverIO / Appium / Espresso / XCUITest / RestAssured / Karate / k6 / Locust with side-by-side tradeoffs on speed, cross-browser, mobile, parallelisation, language support, ecosystem maturity, CI integration; a decision tree for matching project NFRs to framework choice; and reference directory / fixture / CI layouts for the chosen stack. This is the **upstream selection step**: it decides which tool to adopt, not how to configure a tool already chosen, and not how to rebalance the unit / integration / E2E mix of an existing suite. Use when starting a new test-automation suite from scratch, before installing any tool.
heuristic-test-design-reference
Reference catalog of the four canonical heuristic test-design models - Bach's Heuristic Test Strategy Model (HTSM) with SFDPOT product elements, Whittaker's 'How to Break Software' attack patterns, Bolton's FEW HICCUPPS consistency oracles, and the ISO/IEC 25010 quality characteristics - for use when the tester has no user story, no acceptance criteria, and no documentation. This is the zero-documentation case: it does not read from a written story, and it yields test-case ideas rather than session charters. Use as the reference layer when generating coverage for a feature with no documented input.
post-mortem-author
Build-an-X workflow that produces a blameless post-mortem from an incident - captures the timeline (chronological event sequence with sources), root cause analysis (what + why, not who), impact (users / revenue / SLO debt), action items (with owners + due dates + measurable success criteria), and "what went well" (intentional). Per Google SRE: "Blameless postmortems are a tenet of SRE culture." Use after every user-visible incident, not just severe ones.
product-risk-register-builder
Build-an-X workflow that produces a product-level risk register catalogue - per-feature / per-component product risks (functionality, performance, security, usability, compatibility, reliability) that persist across releases, distinct from per-release risk matrices. Walks the author through risk identification by ISO 25010 quality characteristic, scoring per impact × likelihood, and linking each register entry to mitigations + owners + review cadence. Output is a Markdown register the team reviews quarterly and that seeds release-level risk matrices. Use for long-lived product-quality risks; complements risk-matrix for per-release risks.
project-risk-register-builder
Build-an-X workflow producing a project-level risk register - risks to project execution (schedule slippage, environment instability, people / staffing, vendor / dependency, scope creep) rather than the product itself. Walks the author through ISO 31000-aligned identification, impact × likelihood scoring, mitigation strategy (avoid / mitigate / transfer / accept), and ownership; the project manager reviews it weekly. Use for release-execution risk. For product-quality risks use product-risk-register-builder, for the per-release product risk table use risk-matrix, and to sign off accepting one specific risk use risk-acceptance-decision-author.
qa-okr-author
Build-an-X workflow that drafts a QA team's quarterly OKR set - one to three Objectives, each with 3 - 5 measurable Key Results - from the team's current state (risk matrix, defect-trend narrative, test-run history, test-pyramid balance, compliance coverage). Every numeric target cites its source artifact (e.g., a defect-trend baseline's 2026-Q1 escape rate). QA-specific by design - generic OKR generators (Tability, Asana, ClickUp) don't know test metrics; the differentiation is the domain. Produces the OKR set itself - not the test-strategy document it sits inside, and not the risk-score calibration behind the baselines. Use at the start of each quarter to draft the OKR set the manager edits and the team commits to.
qa-vendor-evaluator
Build-an-X workflow that produces a side-by-side **commercial-vendor** evaluation matrix for QA tools - test-management platforms (TestRail / Qase / Xray / Zephyr / TestCollab), no-code platforms (mabl / Testim / Functionize / TestSigma / Reflect), visual regression services (Applitools / Percy / Chromatic), and commercial AI copilots - scoring each on capability fit, cost model, integration depth, vendor lock-in risk, exit cost, contractual posture, and customer-reference data. Scoped to commercial procurement - contract, lock-in, and exit-cost axes - not to choosing an open-source code-first framework on architectural fit. Use for commercial procurement decisions only - refuses to recommend a winner; the team owns the procurement choice.
risk-acceptance-decision-author
Build-an-X workflow that produces a structured risk-acceptance decision document - for risks the team has decided to accept (rather than mitigate / transfer / avoid). Walks the author through the ISO 31000 risk-acceptance criteria (rationale, sign-off, scope, review trigger, exit conditions), captures stakeholder approval, and links to the originating risk register entry. Output is a Markdown decision artefact that lives alongside the risk register and provides audit-defensible justification for the team's acceptance choice. Use when a risk register entry's Strategy column is set to Accept, or an already-accepted risk comes up for its scheduled re-review, an audit, or a post-incident look-back.
risk-coverage-mapper
Build-an-X workflow that produces a risk-to-test-coverage matrix - maps each risk in the product/release register to the tests / cases / monitoring that mitigate it. Walks the author through ingesting risks (from risk-matrix / product-risk-register-builder), inventorying test coverage (test cases via traceability-matrix-builder, automated tests via repo scan, production monitoring via observability dashboards), and computing per-risk coverage depth + identifying orphan risks (no coverage) + orphan tests (not linked to risks). Output is a Markdown matrix + executive summary. Use before a release sign-off or compliance audit, when the team must show which tests, cases, or monitors back each registered risk and which risks have nothing behind them.
risk-matrix
Produces the per-feature / per-release risk-matrix artifact itself: a structured intake (feature, category, impact 1-5 by likelihood 1-5, score), mitigations with owners and due dates, supporting both lightweight (impact by likelihood) and heavyweight (FMEA / Cost of Exposure) methods per RBT canon, output as a Markdown / spreadsheet the team reviews each sprint. Use when building the matrix artifact; to facilitate the live risk-storming meeting use risk-storming-facilitator, to calibrate scores across raters use risk-matrix-calibration, and to map the resulting risks onto test coverage use risk-coverage-mapper.
risk-matrix-calibration
Checks an already-written risk matrix against what actually happened. Maps each row's likelihood rating to observed defect density, test failure rate and code churn, maps its impact rating to the severity mix and escape rate, then classifies each row as over-stated, under-stated, in-agreement, or not calibrated, using stated reporting thresholds so small differences are not treated as findings. Every proposed rating change carries the observation that produced it, and every proposal is handed to the matrix owner rather than applied. Also emits candidate new entries for areas that show up in defect data but have no row. Owns calibration only: choosing a scoring methodology, designing the matrix structure, picking risk categories, mapping risks to test types, FMEA scoring, review cadence and file storage are all out of scope. Use when a matrix has been driving test decisions for at least three releases and nobody has yet checked whether its ratings match the defects, escapes and incidents that followed.
risk-storming-facilitator
Reference guide for planning and facilitating a risk-storming session yourself - meeting structure, participant roster, per-category brainstorm prompts (categories from risk-matrix), affinity grouping, impact by likelihood scoring, and mitigation assignment. Static reference only, not an active runner that writes the matrix file. Use to learn or teach the facilitation pattern, or to run a feature-kickoff session without agent assistance. For the matrix artifact itself use risk-matrix, to calibrate its ratings against real defect data use risk-matrix-calibration, and to map the resulting risks onto test coverage use risk-coverage-mapper.
smoke-suite-gate
Build-an-X workflow for a critical-path smoke suite that runs in <5 minutes - picks the 5-15 highest-business-value journeys (login, hero flow, checkout, payment, primary read), implements as fast E2E or API tests, gates per-deploy, retries on transient failures with quarantine. Use as the canary-precursor or per-deploy verification gate; the team's "if this fails, the build can't proceed" floor.
tdd-stuck-pattern-resolver
Pattern catalog for "I can't write the test first" moments - recognizes common testability blockers (singletons / static dependencies, network in constructors, time / random as hidden inputs, deeply nested construction, untestable boundaries) and proposes the refactor that unblocks TDD (extract interface, dependency injection, seam, ports-and-adapters). Use as TDD coaching when an engineer is stuck on a class of code. For a catalog of what-to-test heuristics with no story use heuristic-test-design-reference, to label a change's shape before planning test effort use code-change-shape-classifier, and for conventions on writing the test well once the code is testable use test-code-conventions.
test-case-ideation-from-story
Takes a user story or feature spec and emits a markdown test-case matrix - one row per case (id, title, precondition, steps, expected, tier) covering happy path, alternate paths, boundaries, and negative paths - before any test code is written. Output is the human-reviewable matrix that goes into TestRail / Qase / Xray. Emits the human-reviewable case matrix itself - not Gherkin scenarios written against locked acceptance criteria, and not executable test code. Use as the first artifact a manual tester or three-amigos session produces from a story, ahead of automation.
test-effort-estimation
Turns a list of testable areas plus a change-shape distribution into a PERT three-point test effort estimate, reporting every row as a range around the expected value rather than a single number, requiring a named assumptions ledger across six mandatory categories, and recommending a per-layer ownership split across developer, automation, and exploratory roles. Owns the hours and the ownership recommendation only: it consumes a change-shape distribution rather than producing one, and it does not choose which tests to run or how deep coverage should go. Use when an epic or release has been broken into testable areas and someone is about to commit test capacity for a sprint.
test-pyramid-balancer
Build-an-X workflow that analyzes a repo's test mix (unit / integration / E2E counts + runtimes) and recommends rebalancing toward the test pyramid ratios per the change-set shape - pure-logic-heavy repo wants ~80/15/5; UI-heavy repo wants ~60/25/15. Detects 'ice-cream cone' (E2E-heavy) and 'hourglass' (integration-thin) anti-patterns. Use when the user asks about test distribution, test strategy, test balance, too many E2E tests, slow CI caused by tests, testing best practices, or rebalancing their test suite; also suitable for quarterly calibration of the test mix to codebase reality.
test-strategy-author
Authors a test strategy document (a master test plan) for a project, release, or feature - covers scope, in/out, test types per layer (unit / integration / contract / E2E / perf / security / a11y), risk-based test prioritization that maps top risks to test investment (per `risk-matrix`), tooling stack, environments, exit criteria, and ownership. Use when a team needs the release-readiness artifact stakeholders sign off on before significant test investment, and the reference engineering teams return to when scope or quality questions arise.
tool-selection-decision-record
Defines the output contract for writing down a chosen developer tool as a portable decision record: the observed project signal, exactly one primary recommendation, rationale that names the rejected alternative, what to read next, and a mandatory list of the conditions that would flip the choice. Adapts Architecture Decision Record conventions (context, decision, consequences, status, supersede rather than edit) to tool selection, and refuses any recommendation inferred from a README or a folder name instead of a manifest, lockfile, config file, or existing test directory. Distinct from a catalog or advisor that compares candidate tools on their merits: this owns the shape of the written record, not the comparison. Use when a tool has just been chosen (test framework, build tool, linter, package manager, migration tool) and the choice needs to be recorded so a later reader can see the signal, the rejected alternative, and what would reverse it.