test-effort-estimation

Turns a list of testable areas plus a change-shape distribution into a PERT three-point test effort estimate, reporting every row as a range around the expected value rather than a single number, requiring a named assumptions ledger across six mandatory categories, and recommending a per-layer ownership split across developer, automation, and exploratory roles. Bundles the change-shape classifier (pure-logic / service-layer / ui-heavy / data-heavy from git-history path and content signals, with the relative per-layer cost model) as a reference, so the shape distribution the estimate consumes can be produced here too. Does not choose which tests to run or how deep coverage should go. Use when an epic or release has been broken into testable areas and someone is about to commit test capacity for a sprint, or when a change set needs its shape classified before planning.

Install with skills.sh (any agent)

npx skills add testland/qa --skill test-effort-estimation

View source

test-effort-estimation

Overview

A test effort estimate is a probability distribution over hours. Reporting it as one number destroys the only information that makes it useful: how wide the distribution is, and what has to stay true for it to hold.

This skill produces three things per epic:

A per-area, per-layer effort range computed with the PERT three-point method.
A ledger of named assumptions the ranges depend on.
A recommended split of which role owns which layer of the work.

Differentiation axis

This skill owns hours and ownership. The taxonomy of change shapes, the path and content signals used to detect them, and the relative per-layer cost model live in references/change-shape-classifier.md, and the estimating steps consume that classifier's output: a distribution over pure-logic, service-layer, ui-heavy, and data-heavy, each already mapped to the test layer where its verification lands. Do not re-derive that mapping inside the estimate. If the shape definitions get copied into the estimator's own tables, the two answers drift and stop agreeing.

It also stops short of two downstream decisions. It does not select which existing tests to run for a given change, and it does not set coverage depth (how many tests of what type, with what entry and exit criteria). Both consume this estimate; neither is produced by it.

Read the boundary as: shape in, hours and owners out.

When to use

An epic is broken into stories and someone is about to promise a sprint's worth of test capacity.
A release plan needs a defensible number rather than a gut feel, and the number will be challenged.
Two people disagree about how long a test area will take and a structured three-point round would settle it.
A previous estimate was blown and the team wants to know which assumption failed, which is only answerable if a ledger existed.

Step 1 - Decompose into testable areas and weight them for risk

Split the epic into discrete testable areas. Each area maps to one or more acceptance criteria and to a coherent chunk of behaviour a person can be assigned and tracked against. Record for each:

Area name: a short label such as "checkout flow", "discount-code API".
Story IDs: which stories contribute to it.
Change shape and layer: taken from the change-shape distribution, not re-derived. Each shape already carries the layer where most of its failure-detection value sits.

Then assign a risk weight on a coarse 1 to 3 scale:

Weight	Meaning
1	Internal only, easily rolled back, small blast radius
2	Customer facing, recoverable if broken
3	Payment, authentication, data integrity, or compliance

Risk weight shifts effort upward: a risk-3 service area gets more test work than a risk-1 service area of the same size. The 1 to 3 scale is a practitioner convention used here as an effort modifier, not a standard risk-scoring method. It is deliberately too coarse to serve as a formal risk assessment. If the programme needs one, run a structured risk-scoring method separately and feed its output in as the weight.

An area with no risk weight is not estimable. Force at least one risk-1 and one risk-3 assignment per epic, or the scale collapses to "medium for everything" and stops carrying signal.

Step 2 - Elicit three points per area and layer

For each (area, layer) pair, produce three values in hours:

Symbol	Meaning
`a`	Optimistic: everything goes smoothly, no environment or data problems
`m`	Most likely
`b`	Pessimistic: blockers, missing test data, flaky infrastructure

These are the standard three-point inputs: a is "the best-case estimate", m is "the most likely estimate", b is "the worst-case estimate" (Wikipedia, Three-point estimation (opens in new window)).

Gathering the three points with more than one estimator

When several people could estimate the area, run a Wideband Delphi round rather than averaging opinions in a meeting (Wikipedia, Wideband delphi (opens in new window)). Boehm's sequence, adapted to test effort:

The coordinator gives each estimator the area description and a blank form.
The group meets and discusses estimation issues, including what is in scope.
Estimators fill out a, m, b anonymously.
The coordinator summarizes and distributes the spread.
The group meets again, focusing only on the widely varying rows.
Estimators fill out the forms anonymously again. Repeat steps 4 to 6 as needed.

The anonymity in steps 3 and 6 is the mechanism, not a formality: it stops the loudest or most senior estimate from anchoring the rest. Every disagreement that surfaces in step 5 is a candidate assumption for Step 4's ledger, because two people estimating the same area differently are usually assuming different things about scope, environment, or data.

Step 3 - Compute the expected value and the spread

Combine the three points with the PERT formulas:

E  = (a + 4m + b) / 6
SD = (b - a) / 6

These are the classical PERT definitions, valid on the assumption that a PERT distribution governs the data (Wikipedia, Three-point estimation (opens in new window); Brunel University (opens in new window)). The 1/6:4/6:1/6 weighting "is essentially fixed and cannot be altered" (Brunel); to weight the pessimistic case more heavily, raise b, do not rewrite the formula.

Report every row as a range, E - SD to E + SD. Never collapse a row to a point. The three estimates define a distribution in which "all times are possible (with an associated probability)" (Brunel); a single number discards that.

Aggregating rows to an epic total

Expected values add directly: the epic total E is the sum of the row E values.

Spreads do not. Sum the variances (SD squared) and take the square root of the total, which is how PERT combines activity spreads along a path (Brunel University, Network analysis: uncertain completion times (opens in new window)):

E_total  = sum(E_i)
SD_total = sqrt( sum(SD_i^2) )

That aggregation assumes the rows are independent. The cited PERT treatment makes the same independence assumption explicitly and warns that when it does not hold "the probability figures calculated may be inaccurate" (Brunel University, Network analysis: uncertain completion times (opens in new window)). Test rows are frequently not independent: one missing staging environment blows out every row at once. When rows share a dependency, also report the fully-correlated bound, sum(SD_i), as a worst case, and name the shared dependency in the ledger.

Step 4 - The assumptions ledger (mandatory)

An estimate without a ledger is not an estimate. It is a guess with a decimal point. The ledger is what makes the number auditable: it is the only artifact that lets someone later ask "which assumption failed?" instead of "who was wrong?".

This is not a local house rule. Federal cost-estimating guidance makes documentation one of the four pillars of a reliable estimate: a well-documented estimate identifies "rationales, assumptions, original source data, and methodologies used for calculations" (IRS IRM 1.33.9.3 (opens in new window)).

Every row in the effort table cites at least one assumption ID. Six categories are mandatory: an epic-level ledger that is missing any of them is incomplete, and the missing category is usually where the estimate later breaks.

#	Category	What it pins down	Example entry
1	Scope boundary	What is explicitly excluded	"Stories 12 to 15 only. Story 16 (dark mode) is excluded."
2	Environment	What must exist, and when	"Staging environment available from sprint day 2."
3	Test data	What data exists and who produces it	"Fixture generator covers all discount-code scenarios."
4	Dependency	Interfaces and teams outside the estimate	"Auth service API is stable; no interface churn expected."
5	Skill	Who is available and what they can already do	"One automation engineer with browser-automation experience on the team."
6	Risk rating	Why each risk weight was assigned	"Checkout rated risk-3 because it processes real payments."

Two rules make the ledger load-bearing:

An estimate is a distribution, not a commitment. State plainly, in the output, that the range describes uncertainty and is not a delivery promise. When someone converts E into a date, they have made a commitment the estimate does not support.
A violated assumption invalidates the estimate. It does not get padded. If assumption 2 fails because staging slips to day 6, do not add hours. Change the input and recompute the affected rows. Padding hides which assumption broke and destroys the audit trail that the ledger existed to provide.

Step 5 - Recommend the per-layer ownership split

Each (area, layer) row gets an owner. The default split below is a practitioner convention for a conventional team structure, not a standard; it is grounded in where the work naturally sits, not in a cited allocation rule.

Layer	Default owner	Why
Unit	The developer writing the production code	Tests land in the same PR as the code; unit tests run fast, keeping the loop inside the edit cycle
Service	Automation engineer	API and integration tests need harness and environment work and run slower than stubbed unit tests
UI / E2E	Automation engineer, or a manual tester for the long tail	Automate happy paths only; end-to-end UI tests are brittle, expensive to write, and slow to run
Exploratory	Manual tester	A manual approach for the tester's freedom to spot issues a scripted row cannot cover

Layer characteristics per Fowler and Vocke, The Practical Test Pyramid (opens in new window) and Fowler, TestPyramid (opens in new window).

Three allocation rules:

Every risk-2 and risk-3 area gets an exploratory row, with its own three points, owned by a manual tester. Exploratory time that is not estimated does not happen.
data-heavy areas get a dedicated data-checking row in addition to their service row: schema and contract changes fail in ways a request-level test does not see.
Risk-3 areas touching authentication, access control, or payment get a separate security-review row. Do not fold that time into the service row, because it is usually done by a different person on a different schedule.

Ownership is a recommendation. If capacity figures were supplied, compare each role's summed E against its available sprint hours and flag every role whose estimate exceeds capacity, with a suggested redistribution. If the split implies a large shift in the balance between layers (for example, an epic that adds 40 percent more UI-layer test work), say so explicitly so someone can review the overall test mix before the work starts.

Worked example

The full row-by-row PERT arithmetic for the "Promo codes at checkout" epic - six (area, layer) rows, the variance-sum aggregation, and the independent vs fully-correlated ranges - is in references/worked-example.md.

Output format

Emit one Markdown document with these sections, in order:

Header - epic name and date, total expected effort, the independent-rows range, and the shared-dependency worst-case range.
"Distribution, not a commitment" line - state that the range is invalidated, not padded, if any assumption changes.
Effort by area and layer - per-row table: area, layer, risk, a/m/b, E, range, owner, assumption IDs.
Assumptions ledger - one row per assumption (ID, category, statement, rows affected), with all six mandatory categories present.
Ownership summary - per role: rows, summed expected hours, capacity, over/under flag.
Capacity flags - each over-allocated role with a suggested redistribution.
Method - the PERT formulas, the variance-sum aggregation, and the note that change shape was consumed, not derived.

Keep the "distribution, not a commitment" line and the Method section even when they feel redundant; they are what stop a reader from turning E into a date. The full filled template is in references/effort-estimate-output-template.md.

Anti-patterns

Anti-pattern	Why it fails	Fix
Reporting a single number ("this will take 8 hours")	Hides uncertainty and anchors the team to false precision	Report `E - SD` to `E + SD` for every row (Wikipedia, Three-point estimation (opens in new window))
Emitting the table with no assumptions ledger	The range means nothing without knowing what it assumes; nobody can later tell which assumption failed	Require at least one assumption ID per row and all six categories per epic (IRS IRM 1.33.9.3 (opens in new window))
Padding hours when an assumption breaks	Destroys the audit trail and makes the next estimate worse	Change the input and recompute the affected rows
Treating `E` as a delivery date	Converts a probability distribution into a promise the estimate does not support	State the invalidating assumptions explicitly next to the total
Adding row standard deviations to get the epic spread	Overstates the spread for independent rows	Sum variances and take the square root; report the summed-SD figure separately as the correlated worst case (Brunel University (opens in new window))
Reweighting the PERT formula to "be more pessimistic"	The 1/6:4/6:1/6 weighting "is essentially fixed and cannot be altered" (Brunel University (opens in new window))	Raise `b` instead
Rating every area risk-2	Erases the signal that drives effort allocation and the exploratory rows	Force at least one risk-1 and one risk-3 per epic
Estimating without a layer for each row	Hours cannot be mapped to an owner, so the ownership split is unassignable	Attach a layer to every row before computing anything
Re-deriving change shapes inside the estimate	Two components then own the same taxonomy and drift apart	Consume the distribution from references/change-shape-classifier.md unchanged
Leaving exploratory work unestimated	Unestimated work is unbudgeted work and does not happen	Give every risk-2 and risk-3 area its own exploratory row with three points
Collecting three points in an open group meeting	The loudest or most senior estimate anchors everyone else	Use anonymous rounds (Wikipedia, Wideband delphi (opens in new window))

Limitations

Three-point inputs are only as good as the estimator's domain knowledge. When nobody has built the area before, even b is usually too low. Record "first-time implementation" as a skill-category assumption and widen b deliberately.
No historical velocity is used. The estimate derives from area structure, risk weight, and change shape, not from past sprint actuals. A team with a points-to-hours baseline should apply that conversion after Step 3 and record the conversion factor in the ledger.
The PERT distribution assumption may not fit. The formulas rest on the assumption "that a PERT distribution governs the data" (Wikipedia, Three-point estimation (opens in new window)), and a triangular distribution is a documented alternative for some applications (same source). Test work with a long tail of rare blockers is poorly described by any of them.
Row independence rarely holds exactly, which is why the correlated bound is reported alongside. When one environment or one person gates most rows, the correlated bound is the honest number.
The 1 to 3 risk weight is a coarse effort modifier, not a risk assessment. It cannot substitute for a structured risk-scoring method.
The ownership split assumes a conventional structure with developers, automation engineers, and manual testers as distinct roles. A solo tester or a fully automated team collapses all rows onto fewer owners, which changes the capacity flags but not the hours.
Non-functional test effort is out of scope. Load, security, and accessibility test work is scoped by their own methods; only the security review row above is reserved, and it is reserved rather than estimated.
Predicted change shapes are weaker than measured ones. For an epic with no code yet, the distribution comes from story text. Record that in the ledger and recompute once code exists.

References

Wikipedia, Three-point estimation (opens in new window): definitions of a, m, b; E = (a + 4m + b) / 6; SD = (b - a) / 6; the PERT distribution assumption and the triangular-distribution alternative.
Brunel University, Network analysis: uncertain completion times (opens in new window): the classical PERT expected time (t1 + 4t2 + t3)/6 and standard deviation (t3 - t1)/6; use of the beta and normal distributions; the fixed 1/6:4/6:1/6 weighting; all times possible with an associated probability; variances summed and square-rooted; the independence assumption and its inaccuracy warning.
Wikipedia, Wideband delphi (opens in new window): Barry Boehm and John A. Farquhar originated the wideband variant in the 1970s; named for "greater interaction and more communication among those participating"; popularized through Software Engineering Economics (1981); the six-step anonymous-round sequence.
IRS Internal Revenue Manual 1.33.9, Cost Estimating Guidelines (opens in new window): the four pillars of a reliable estimate; the well-documented pillar requiring rationales, assumptions, source data, and methodologies; documenting the rationale behind ground rules and assumptions; the credible pillar requiring documented limitations, risks, and uncertainty.
Fowler, TestPyramid (opens in new window): tests running end-to-end through the UI are "brittle, expensive to write, and time consuming to run".
Fowler and Vocke, The Practical Test Pyramid (opens in new window): unit tests "run very fast"; integration with filesystems and databases is "much slower"; end-to-end tests "require a lot of maintenance and run pretty slowly"; exploratory testing is "a manual testing approach that emphasises the tester's freedom and creativity to spot quality issues in a running system".

Change-shape classifier

View source

Change-shape classifier

Deep reference for the test-effort-estimation SKILL.md. Produces the change-shape distribution the estimator consumes: classifies a code change set into four shapes (pure-logic, service-layer, ui-heavy, data-heavy) from file-path and file-content signals, computes the shape distribution over a window of git history, and attaches a relative per-layer test cost model (unit 1x, service 3x, UI 10x) so downstream planning works from one shared input.

Every change set has a shape: the mix of layers its files touch. A sprint of tax-rule refactors and a sprint of checkout-screen redesign produce the same commit count and very different verification needs. The pyramid model names three test layers (unit, service, UI) and shows cost and execution speed increasing as you move up toward the higher-level tests (Fowler, TestPyramid (opens in new window), which credits Mike Cohn's 2009 book Succeeding with Agile for popularizing the model). That model describes where tests live. This classifier describes where the change lives, which is the input the model needs.

The classification stops here. It does not carry a recommended unit:service:UI target ratio, does not convert shapes into hours, and does not pick which existing tests to run. Those are three separate downstream decisions: pyramid balancing, effort estimation (the SKILL.md spine), and test selection. Keeping the taxonomy in one place is the point: if the shape definitions are copied into the estimator and the balancer separately, they drift and the two answers stop agreeing.

The four change shapes

Classification is per file first, then reduced to per commit (Step 3).

Shape	What it means	Primary path signals	Content tie-breakers
`pure-logic`	Domain rules, calculations, transformations. No user-visible surface and no wire surface.	`src/domain/`, `core/`, `lib/`, `rules/`, `calc/`, plain model and value-object files	No import of an HTTP framework, ORM session, or view library. Pure functions, arithmetic, branching on domain state.
`service-layer`	Request handling, orchestration, persistence access, outbound integration.	`routes/`, `controllers/`, `handlers/`, `api/`, `services/`, `repositories/`, `resolvers/`, `consumers/`, `clients/`	Imports a router, request or response types, an ORM or query builder, an external SDK client, a queue consumer.
`ui-heavy`	Anything a user sees or clicks.	`components/`, `views/`, `pages/`, `screens/`, route segment files, `.tsx`, `.vue`, `*.svelte`, templates, stylesheets	Imports a view library, declares a component, contains markup or event handlers.
`data-heavy`	Schema, stored shape, and data movement.	`migrations/`, `db/migrate/`, `schema.sql`, `schema.prisma`, `models/` in a dbt project, `pipelines/`, `etl/`, `.proto`, `.avsc`, seed files	Contains DDL, an up/down migration pair, a schema version bump, a column type change, a data contract field.

Two rules keep the table honest:

Path first, content second. The path signal decides unless the file content contradicts it. A file under services/ that is a pure calculator with no I/O is pure-logic; a file under lib/ that opens a database connection is not.
Test files and config are excluded from shape classification. They are the output of the decision, not evidence about the change.

Step 1 - Collect the change set

For a window of history, list each non-merge commit with the files it touched. --since=<date> shows commits more recent than the date, --no-merges drops commits with more than one parent, --name-only prints the changed paths, and --pretty=format: controls the header line (git-log documentation (opens in new window)):

git log --since="90 days ago" --no-merges --name-only \
        --pretty=format:"COMMIT %H" > /tmp/changeset.txt

For a single pull request, use the diff against the merge base instead:

git diff --name-only origin/main...HEAD

For an epic that has not been implemented yet, there is no history to read. Derive shapes from the story text and the areas it names, and record that the distribution is predicted rather than measured (see Limitations).

Step 2 - Classify each file

Apply the path rules first, then the content overrides, then drop excluded paths (tests, config, lockfiles, generated). A file matches at most one shape; with no surface signal it defaults to pure-logic. The runnable classifier that mechanizes this is in the "Classification script" section below.

Read file content only when the path yields no match or when the path match is being challenged. Reading every file in a 90-day window is slow and rarely changes the answer.

Step 3 - Reduce files to a per-commit shape

A commit gets one shape:

Count classified files per shape (ignore the excluded ones).
The plurality shape wins.
On a tie, break toward the shape whose verification is more expensive, in this order: data-heavy, then ui-heavy, then service-layer, then pure-logic.

The tie-break ordering is a convention of this reference, not a claim from the cited sources. Its rationale: the cited pyramid shows cost rising toward the top layers (Fowler, TestPyramid (opens in new window)), and schema changes carry irreversible-migration risk, so a mixed commit is safest labelled by its costliest component.

Record a mixed flag on any commit where no shape holds more than 50 percent of its files. Mixed commits are the ones a human should look at.

Step 4 - Compute the shape distribution

Aggregate per-commit shapes into a distribution. Report both commit share and changed-file share: they diverge when one large screen rewrite lands in a single commit while forty small logic commits land beside it.

| Shape         | Commits | % commits | Files changed | % files |
|---------------|--------:|----------:|--------------:|--------:|
| pure-logic    |      42 |       30% |           118 |     22% |
| service-layer |      49 |       35% |           201 |     37% |
| ui-heavy      |      35 |       25% |           186 |     34% |
| data-heavy    |      14 |       10% |            38 |      7% |
| (mixed)       |       9 |         - |             - |       - |

Name the window explicitly (dates and commit count). A distribution without its window is not comparable to the next one.

Step 5 - Attach the relative test cost model

Each shape has a layer where most of its failure-detection value sits, and each layer has a relative cost:

Layer	Typical scope	Relative cost weight
Unit	Domain rules, isolated functions, a single class or method	1x
Service	API contracts, integration points, database queries	3x
UI / E2E	User-visible flows, cross-browser, accessibility	10x

Change shape	Layer where verification lands	Weight
`pure-logic`	Unit	1x
`service-layer`	Service	3x
`ui-heavy`	UI / E2E	10x
`data-heavy`	Service, plus dedicated data checks	3x

Be clear about what these numbers are. The ordering is grounded: the pyramid shows cost and execution speed increasing as you move up the layers (Fowler, TestPyramid (opens in new window)), unit tests run "very fast" while integration tests are "much slower" because of external dependencies and end-to-end tests are "notoriously flaky" and maintenance-heavy (Fowler and Vocke, The Practical Test Pyramid (opens in new window)), and UI-driven end-to-end tests are "brittle, expensive to write, and time consuming to run" (Fowler, TestPyramid (opens in new window)). The specific values 1, 3, and 10 are illustrative relative weights, not measured constants. No cited source publishes them. They exist so downstream steps can do arithmetic against a shared scale. A team with a fast headless E2E rig and a slow container-backed service suite should measure its own per-layer wall-clock and runner cost and substitute real numbers.

Derive a cost-weighted shape index so two distributions can be compared:

weighted_index = sum(share_of_shape * layer_weight for each shape)

For the Step 4 table, using commit share:

0.30*1 + 0.35*3 + 0.25*10 + 0.10*3 = 0.30 + 1.05 + 2.50 + 0.30 = 4.15

A weighted index near 1 means the change stream is cheap to verify. An index above roughly 5 means most verification pressure sits in the expensive layers. The threshold is a reading aid, not a rule: what matters is the trend across windows and the gap between two repositories, not the absolute number.

Classification script

The runnable per-file classifier that mechanizes Step 2: path rules first, content overrides second, excluded paths dropped. Save it as scripts/classify-change-shape.py and pipe a git log --name-only stream (Step 1) through it.

# scripts/classify-change-shape.py
import re

PATH_RULES = [
    ("data-heavy",    r"(^|/)(migrations?|db/migrate|etl|pipelines?)/|"
                      r"schema\.(sql|prisma|graphql)$|\.(proto|avsc)$|seeds?/"),
    ("ui-heavy",      r"(^|/)(components?|views?|pages?|screens?|layouts?)/|"
                      r"\.(tsx|jsx|vue|svelte|css|scss|html)$"),
    ("service-layer", r"(^|/)(routes?|controllers?|handlers?|api|services?|"
                      r"repositor(y|ies)|resolvers?|consumers?|clients?)/"),
    ("pure-logic",    r"(^|/)(domain|core|lib|rules|calc|models?)/"),
]

CONTENT_OVERRIDES = [
    ("data-heavy",    r"CREATE TABLE|ALTER TABLE|ADD COLUMN|def upgrade\("),
    ("ui-heavy",      r"<[A-Z][A-Za-z]*|useState\(|render\(|@Component\b"),
    ("service-layer", r"@(Get|Post|Put|Delete)Mapping|app\.(get|post|put)\(|"
                      r"session\.query\(|createConnection\(|fetch\(|HttpClient"),
]

EXCLUDE = re.compile(r"(^|/)(tests?|spec|__tests__|e2e|fixtures?)/|"
                     r"\.(test|spec)\.|(^|/)(\.github|docs)/|"
                     r"(package-lock|yarn\.lock|Gemfile\.lock)")

def classify_file(path, content=""):
    if EXCLUDE.search(path):
        return None                      # not evidence about the change
    for shape, pattern in PATH_RULES:
        if re.search(pattern, path):
            path_shape = shape
            break
    else:
        path_shape = None
    for shape, pattern in CONTENT_OVERRIDES:
        if content and re.search(pattern, content):
            return shape if path_shape is None else path_shape
    return path_shape or "pure-logic"    # default: no surface signal found

How the code maps back to the rules: EXCLUDE is Rule 2 (test files and config are not evidence); the PATH_RULES order encodes "path first"; CONTENT_OVERRIDES are the content tie-breakers applied only when a path match is contested. Keep the three tables in sync with the "four change shapes" table above - if a signal is added there, add it here too, or the code and the prose drift apart.

Output format

Emit one Markdown block containing, in order:

Header with the repository or change-set identifier and the exact window (dates and commit count).
Distribution table: shape, commits, percent of commits, files changed, percent of files, mapped layer, cost weight.
Cost-weighted shape index with the arithmetic shown, not just the result.
Dominant shape in one sentence, naming the directories that drove it.
Divergences between commit share and file share, when any shape differs by more than about 10 percentage points between the two views.
Mixed commits table for human triage.
Not decided here: an explicit line stating that target ratios, effort hours, and test selection are downstream decisions.

Keep section 7 even when it feels redundant. It is what stops a reader from treating a classification as a plan.

Anti-patterns

Anti-pattern	Why it fails	Fix
Emitting a target unit:service:UI ratio alongside the distribution	Two capabilities then own the same decision and drift apart	Emit the distribution and the layer mapping only; let the balancing step choose the ratio
Counting test files as evidence of shape	The existing test mix is the thing under review, so using it as input makes the analysis circular	Exclude test paths in Step 2
Reporting only commit share	One large screen rewrite in a single commit disappears	Report commit share and file share side by side
Classifying by path with no content check on ambiguous files	A calculator under `services/` gets labelled service-layer and inflates the expensive share	Apply the content tie-breakers when the path match is contested
Silently forcing every commit into one shape	Genuinely cross-cutting commits get an arbitrary label and no one notices	Flag commits with no >50 percent shape as `mixed` and list them
Treating 1x / 3x / 10x as measured facts	They are relative weights chosen for arithmetic, and no cited source publishes them	State them as illustrative and substitute measured per-layer cost when available
Comparing distributions from different-length windows	A 30-day and a 90-day window are not comparable	Always print the window; keep the window length fixed across reviews

Limitations

Path-based classification is a heuristic. It labels location, not behavior. The classic failure is the mirror image on the test side: a file under __tests__/cart.test.ts that opens a real database connection is an integration test, because tests that involve databases are integration tests rather than unit tests (Fowler and Vocke, The Practical Test Pyramid (opens in new window)). The same confusion happens to production files: a module under lib/ that issues queries is service-layer work no matter where it sits. Content tie-breakers reduce this but do not eliminate it, so review the mixed list.
Monorepos need per-package runs. A single distribution across a frontend package and three backend services averages away the signal that would drive any decision. Classify per package, then aggregate if you must.
Vendored, generated, and lockfile paths pollute file counts. Exclude generated clients, build output, and lockfiles explicitly, or a codegen refresh reads as a large service-layer change.
Predicted shapes are weaker than measured ones. For unimplemented epics the classification comes from story text, which reflects how the work was described rather than how it will land. Mark the output as predicted and re-run it once code exists.
Squash-merge history hides shape. When every pull request lands as one squashed commit, commit share and file share converge and the mixed-commit count rises. Prefer the file-share column on squash-merge repositories.
The layer mapping is a default, not a law. A data-heavy change to a contract consumed by a browser client may need UI-layer verification too. The mapping picks the layer where most of the value sits, not the only layer that applies.

References

Fowler, TestPyramid (opens in new window) - the three layers (unit, service, UI); "you should have many more low-level UnitTests than high level BroadStackTests running through a GUI"; UI-driven end-to-end tests are "brittle, expensive to write, and time consuming to run"; cost and execution speed increase toward the top of the pyramid; Mike Cohn popularized the model in Succeeding with Agile (2009).
Fowler and Vocke, The Practical Test Pyramid (opens in new window): Cohn's original three layers; unit tests run "very fast" while integration tests are "much slower" and end-to-end tests are "notoriously flaky"; tests involving databases are integration tests, not unit tests.
git-log documentation (opens in new window) - --since=<date> shows commits more recent than the date; --no-merges excludes commits with more than one parent; --name-only is a supported non-patch diff format; --pretty=format: takes a printf-like format string.

Test effort estimate output template

View source

Test effort estimate output template

Deep reference for the test-effort-estimation SKILL.md. The full Markdown document the skill emits, filled for the "Promo codes at checkout" epic from the worked example. Copy the section order; keep the "distribution, not a commitment" line and the Method section even when they feel redundant - they are what stop a reader from turning E into a delivery date.

Emit one Markdown document with these sections in order.

## Test effort estimate - Promo codes at checkout - 2026-07-19

**Total expected effort:** 33.6 h
**Range (independent rows):** 30.1 to 37.1 h
**Range (shared-dependency worst case):** 25.6 to 41.6 h

This is a distribution, not a commitment. It is invalidated, not padded, if any
assumption below changes.

### Effort by area and layer

| # | Area | Layer | Risk | a | m | b | E | Range | Owner | Assumptions |
|---|------|-------|-----:|--:|--:|--:|--:|-------|-------|-------------|
| 1 | Checkout flow | Service | 3 | 4 | 8 | 16 | 8.7 | 6.7 - 10.7 | Automation | A1, A2, A4, A6 |
| 2 | Checkout flow | UI / E2E | 3 | 2 | 5 | 10 | 5.3 | 4.0 - 6.6 | Automation | A1, A2, A5 |
| 3 | Discount-code API | Service | 2 | 2 | 4 | 8 | 4.3 | 3.3 - 5.3 | Automation | A2, A3 |
| 4 | Discount rules | Unit | 2 | 3 | 5 | 9 | 5.3 | 4.3 - 6.3 | Developer | A1, A3 |
| 5 | Promo schema | Data checks | 3 | 2 | 6 | 14 | 6.7 | 4.7 - 8.7 | Developer + data | A2, A4, A6 |
| 6 | Checkout | Exploratory | 3 | 2 | 3 | 6 | 3.3 | 2.6 - 4.0 | Manual tester | A1, A6 |

### Assumptions ledger

| ID | Category | Statement | Rows affected |
|----|----------|-----------|---------------|
| A1 | Scope boundary | Stories 12 to 15 only; story 16 (dark mode) excluded | 1, 2, 4, 6 |
| A2 | Environment | Staging available from sprint day 2 | 1, 2, 3, 5 |
| A3 | Test data | Fixture generator covers all discount-code cases | 3, 4 |
| A4 | Dependency | Payment provider sandbox contract unchanged this sprint | 1, 5 |
| A5 | Skill | One automation engineer with browser-automation experience | 2 |
| A6 | Risk rating | Checkout and promo schema rated risk-3: real payments, irreversible migration | 1, 2, 5, 6 |

All six mandatory categories present.

### Ownership summary

| Role | Rows | Expected hours | Capacity | Flag |
|------|------|---------------:|---------:|------|
| Developer (unit + data checks) | 4, 5 | 12.0 | 16 | ok |
| Automation engineer (service + E2E) | 1, 2, 3 | 18.3 | 14 | OVER by 4.3 h |
| Manual tester (exploratory) | 6 | 3.3 | 8 | ok |

### Capacity flags

Automation engineer is over-allocated by 4.3 h. Options: move row 3 to the
developer who owns the discount-code endpoint, or descope row 2 to the single
happy path and cover the variants in row 6.

### Method

Three-point PERT per row: E = (a + 4m + b) / 6, SD = (b - a) / 6. Row ranges
are E - SD to E + SD. Epic spread aggregated as sqrt(sum of variances), with the
fully-correlated sum reported alongside because A2 is shared across four rows.
Three-point inputs from a two-round Wideband Delphi with three estimators.
Change-shape distribution consumed as an input, not derived here.

Keep the "distribution, not a commitment" line and the "Method" section even when they feel redundant. They are what stop a reader from turning E into a date.

Worked example - PERT arithmetic

View source

Worked example - PERT arithmetic

Deep reference for the test-effort-estimation SKILL.md. The row-by-row PERT arithmetic behind the "Promo codes at checkout" epic, using the formulas from the spine (E = (a + 4m + b) / 6, SD = (b - a) / 6). The filled output document is in effort-estimate-output-template.md (opens in new window).

Epic: Promo codes at checkout (stories 12 to 15). Change-shape distribution supplied by the classifier: 30 percent service-layer, 25 percent ui-heavy, 25 percent pure-logic, 20 percent data-heavy. Three-point values gathered in a two-round Wideband Delphi with three estimators.

Row arithmetic:

Checkout flow (service):    E = (4 + 32 + 16)/6 = 8.7    SD = (16 - 4)/6 = 2.0
Checkout flow (UI/E2E):     E = (2 + 20 + 10)/6 = 5.3    SD = (10 - 2)/6 = 1.3
Discount-code API (service):E = (2 + 16 +  8)/6 = 4.3    SD = ( 8 - 2)/6 = 1.0
Discount rules (unit):      E = (3 + 20 +  9)/6 = 5.3    SD = ( 9 - 3)/6 = 1.0
Promo schema (data checks): E = (2 + 24 + 14)/6 = 6.7    SD = (14 - 2)/6 = 2.0
Checkout (exploratory):     E = (2 + 12 +  6)/6 = 3.3    SD = ( 6 - 2)/6 = 0.7

Aggregation:

E_total  = 8.7 + 5.3 + 4.3 + 5.3 + 6.7 + 3.3 = 33.6 h
SD_total = sqrt(2.0^2 + 1.3^2 + 1.0^2 + 1.0^2 + 2.0^2 + 0.7^2)
         = sqrt(4.00 + 1.69 + 1.00 + 1.00 + 4.00 + 0.49)
         = sqrt(12.18) = 3.5 h
Independent-rows range:   30.1 to 37.1 h
Fully-correlated bound:   sum(SD) = 8.0  ->  25.6 to 41.6 h

The two ranges are reported together because assumption A2 (staging availability) is shared by four of the six rows, so the independence assumption behind SD_total is known to be imperfect (Brunel University, Network analysis: uncertain completion times (opens in new window)).

Related skills

attack-surface-test-checklist

Maps a code change to the security tests worth running against it. Classifies changed paths and file contents into nine attack surfaces (authentication, session management, input handling, file upload, deserialization, access control, API and web service, cryptography, data protection), attaches the matching OWASP ASVS 4.0.3 verification requirements, OWASP Top 10 2021 category IDs, and OWASP WSTG section numbers to each active surface, then emits a per-surface manual and automated test checklist bounded by what actually changed. Surfaces with no changed lines are excluded rather than carried as filler. Use when a pull request, release branch, or feature is about to be security tested and the team needs a targeted test list instead of a generic application-wide checklist.

definition-of-done

The team's Definition of Done (DoD), both halves of the lifecycle: authoring and auditing. Explains the Scrum Guide's DoD definition ("a formal description of the state of the Increment when it meets the quality measures required for the product"), proposes a starter DoD with the 7-10 lines most teams need (code reviewed, unit tests, docs, AC met, deployed to staging, smoke passed, no a11y regressions, telemetry wired), emits a per-PR checklist a reviewer enforces, and audits work against an existing DoD line by line with repository evidence (review records, diffs, CI runs, coverage reports), tagging every line met, not met, or unverifiable - never passing a line on self-attestation. Use when the team doesn't have a DoD, wants to revise theirs, or is about to mark a story or PR done and nobody has checked the work against the committed checklist.

e2e-suite-budget

Caps E2E suite size by computing per-test ROI - (regressions caught × value) ÷ (runtime × flake rate × maintenance) - then ranks every end-to-end test and recommends which bottom-decile ones to retire, move to a lower layer, or fix. Use when CI is slow or E2E-dominated, flaky failures are rising, or quarterly to keep suite size within maintenance capacity. For strategic unit:service:UI layer ratios use test-pyramid-balancer, for the minimal per-deploy critical-path gate use smoke-suite-gate, and for quarantining flaky tests use flaky-test-quarantine; this prunes low-signal tests by ROI.

framework-choice-advisor

Reference catalog for picking a test automation framework or QA tool - covers Playwright / Cypress / Selenium / WebdriverIO / Appium / Espresso / XCUITest / RestAssured / Karate / k6 / Locust with side-by-side tradeoffs on speed, cross-browser, mobile, parallelisation, language support, ecosystem maturity, CI integration; a decision tree matching project NFRs to framework choice; and reference layouts for the chosen stack. references/ extends the same decision to commercial procurement (seven-axis vendor evaluation for TCM platforms, no-code tools, visual-regression services) and to recording the outcome (ADR-based tool-selection decision record with signal, one recommendation, flip conditions). This is the upstream selection step: it decides which tool to adopt, not how to configure one already chosen. Use when starting a new test-automation suite, evaluating commercial QA vendors, or writing down a tool decision.

post-mortem-author

Build-an-X workflow that produces a blameless post-mortem from an incident - captures the timeline (chronological event sequence with sources), root cause analysis (what + why, not who), impact (users / revenue / SLO debt), action items (with owners + due dates + measurable success criteria), and "what went well" (intentional). Per Google SRE: "Blameless postmortems are a tenet of SRE culture." Use after every user-visible incident, not just severe ones.

risk-matrix

The risk-based testing (RBT) umbrella: risk matrix and risk register authoring, likelihood x impact scoring, risk storming, calibration, and risk-to-test coverage mapping. Produces the per-feature / per-release matrix artifact (structured intake: feature, category, impact 1-5 by likelihood 1-5, score; heatmap; mitigations with owners and due dates), supporting lightweight and heavyweight (FMEA / Cost of Exposure) methods per RBT canon, plus a risk coverage mapping workflow that proves which tests, cases, or monitors back each registered risk. references/ carries the product-risk and project-risk register variants, the risk-storming facilitation guide, matrix calibration against observed defect data, and a register review checklist. Use for any risk-based-testing artifact: building a matrix or register, running a risk-storming session, calibrating ratings against defects, or mapping risks onto test coverage.

smoke-suite-gate

Build-an-X workflow for a critical-path smoke suite that runs in <5 minutes - picks the 5-15 highest-business-value journeys (login, hero flow, checkout, payment, primary read), implements as fast E2E or API tests, gates per-deploy, retries on transient failures with quarantine. Use as the canary-precursor or per-deploy verification gate; the team's "if this fails, the build can't proceed" floor.

test-case-from-live-feature

Build-an-X workflow that produces a test-case matrix from a **live, undocumented feature** - running app at a URL, screen recording, screenshot, or verbal brief - by combining structured exploration (Playwright trace / DevTools / accessibility tree) with the four canonical heuristic test-design models bundled in references/ (Bach's HTSM / SFDPOT product elements, Whittaker's How-to-Break-Software attacks, Bolton's FEW HICCUPPS consistency oracles, ISO/IEC 25010 quality characteristics). Output is a structured case matrix, not an exploratory session charter. Use when there is no story, no AC, and no documentation - only a live feature - or as the heuristic reference layer for zero-documentation test design.

test-case-ideation-from-story

Turns a thin or ambiguous story into a reviewable test list - a backlog item that is a short paragraph plus the click-through support recorded for themselves, a spec that is mostly a list of accepted formats, or a tech design pasted into the ticket while the last few releases still shipped missed cases. Takes the story or feature spec and emits a markdown test-case matrix, one row per case (id, title, precondition, steps, expected, tier), covering happy path, alternate paths, boundaries, and negative paths, before any test code is written. Output is the human-reviewable matrix that goes into TestRail / Qase / Xray, not Gherkin scenarios. Use when a story needs its cases enumerated and agreed before automation starts.

test-pyramid-balancer

Build-an-X workflow that analyzes a repo's test mix (unit / integration / E2E counts + runtimes) and recommends rebalancing toward the test pyramid ratios per the change-set shape - pure-logic-heavy repo wants ~80/15/5; UI-heavy repo wants ~60/25/15. Detects 'ice-cream cone' (E2E-heavy) and 'hourglass' (integration-thin) anti-patterns. Use when the user asks about test distribution, test strategy, test balance, too many E2E tests, slow CI caused by tests, testing best practices, or rebalancing their test suite; also suitable for quarterly calibration of the test mix to codebase reality.

test-strategy-author

Authors a test strategy document (a master test plan) for a project, release, or feature - covers scope, in/out, test types per layer (unit / integration / contract / E2E / perf / security / a11y), risk-based test prioritization that maps top risks to test investment (per `risk-matrix`), tooling stack, environments, exit criteria, and ownership. Includes a risk-based test-planning workflow that turns a feature scope plus the risk matrix into a budgeted per-risk test plan with owners, effort estimates, and an explicit risks-not-addressed section. Use when a team needs the release-readiness artifact stakeholders sign off on before significant test investment, or a risk-prioritized test plan for a feature or quarter.