test-removal-criteria

The axis: removal, not selection

Two questions about a large test suite get confused, and they have very different risk profiles:

This skill owns the second question only. Anything about narrowing a single run (impacted-set computation, previously-failing sets, fallback triggers, nightly safety runs) belongs to per-change selection and is deliberately out of scope here.

The asymmetry is the whole reason this skill is written conservatively. Meszaros names the end state of a bad removal a Lost Test, whose symptom is that "the number of tests being executed in a test suite has dropped (or has not increased as much as expected)", and lists it as a direct cause of Production Bugs: untested code has "no automated tests to tell the developer when they have introduced a problem" (Meszaros, Production Bugs). A deleted test and a disabled test produce the same silence.

The burden of proof sits on removal

Five rules govern every decision below. They are not advisory.

1. Keep is the default verdict. A test stays unless a removal case is affirmatively made. "No case was made either way" resolves to keep, never to delete.
2. Every removal carries a stated reason and a named reviewer. The reason names the removal class and the evidence; the reviewer is a person, not a team alias, who signed off on that specific row. A removal without both is not eligible to ship. (Practitioner convention, not a standard: teams that skip the named-reviewer column end up with rubber-stamped bulk deletes.)
3. "It never fails" is not evidence a test is worthless. A test that has passed for years while its subject churned may be the load-bearing guard that is passing because the code is still correct. Absence of failure is absence of evidence, in both directions.
4. Coverage-tool output alone is not grounds for removal. Coverage finds untested code; it does not measure test quality. Per Fowler, "if you make a certain level of coverage a target, people will try to attain it. The trouble is that high coverage numbers are too easy to reach with low quality testing" (Fowler, TestCoverage). A redundant-coverage number is one input to the delete gate below, never the whole case.
5. Never remove a test to make a signal go away. Meszaros is explicit about the temptation with an intermittently failing test: "we may be tempted to remove the failing test from the suite to 'Keep the Bar Green' but this would result in an (intentional) Lost Test" (Meszaros, Erratic Test). Flakiness is a diagnosis problem, not a removal class. Quarantine and fix.

Step 1 - Classify the candidate into one removal class

Each candidate gets exactly one class. If two fit, take the more conservative one (the one lower in this table).

A candidate that fits none of these five classes is not a removal candidate. "Old", "long", "slow", "nobody remembers why it is there", and "the coverage report says it is redundant" are not classes.

The never-remove list

Regardless of class, a test is off limits when any of these holds:

• It is labeled @critical, @regression-guard, or any team-configured do-not-delete marker.
• It has failed inside the signal window. A failure is the exact evidence that says this is a real test.
• A test-code quality review has flagged it for rewrite. Obscure or badly structured tests get refactored, not deleted: "test code is just as important as the production code and it needs to be refactored just as often" (Meszaros, Obscure Test).
• It lives in vendored or third-party paths the team does not own.
• It is currently flaky (see rule 5).

Step 2 - Apply the class test

duplicate

Compare a normalized signature, not assertion text. Cosmetic differences (toEqual vs toBe, reordered setup lines, renamed local variables) hide real duplicates, and identical text across different describe paths is often not a duplicate at all. The signature is the tuple:

(describe-path, normalized-setup, normalized-assertion-arguments)

Two tests are duplicates only when all three match. Keep the copy in the canonical location, normally the file co-located with the subject under test; remove the stray copy left behind by a move or a copy-paste.

Before removing, check whether the pair is really a fold candidate instead (Step 4). Meszaros treats repeated test code as a smell whose fix is extraction and parameterization, not deletion (Meszaros, Test Code Duplication).

tautology

A tautology re-implements the subject under test inside the assertion, so it passes for any implementation:

// Tautology: the expected side redoes the arithmetic.
test('add adds', () => {
  expect(add(2, 3)).toBe(2 + 3);
});

// Tautology: the expected side calls the subject under test.
test('formats price', () => {
  expect(formatPrice(100)).toBe(formatPrice(100));
});

// Rewritten: a known-good value the test author committed to.
test('add adds', () => {
  expect(add(2, 3)).toBe(5);
});

Detection heuristic: the expected side of the assertion should contain only literals, expected-value constants, or test-fixture lookups. If it contains a call that resolves into a production module import, the assertion is tautological. Matching on the literal shape expect(x).toBe(x) is not the heuristic and misses nearly every real case, because the recomputation is usually spelled differently on each side.

The default action for this class is rewrite, not remove. The test names a behavior somebody cared about; only the assertion is broken. Remove only when the rewrite produces something already covered elsewhere, which turns it into a duplicate decision with its own evidence.

trivial

test('it works', () => {
  expect(true).toBe(true);   // self-satisfying
});

test('placeholder', () => {});   // no assertion at all

These are usually scaffolding left over from a test-first cycle. They execute code and contribute to coverage numbers while verifying nothing, which is exactly why coverage cannot be the deciding input: Fowler's account of an assertion-free suite notes "you can do this and have 100% code coverage, which is one reason why you have to be careful on interpreting code coverage data" (Fowler, Assertion-Free Testing). Empirical work points the same way: assertion quantity and assertion coverage correlate strongly with a suite's fault-detection ability measured by mutation score, and the relationship is not explained by suite size alone (Zhang and Mesbah, ESEC/FSE 2015, Assertions Are Strongly Correlated with Test Suite Effectiveness).

One caveat before removing: a body with no assertion still executes the code and can surface crashes. Fowler concedes "some faults do show up through code execution, eg null pointer exceptions" (Fowler, Assertion-Free Testing). If the smoke value is the point, the fix is to add the missing assertion, not to delete the test.

dead-signal

This is the class that most often gets removed wrongly. A test qualifies as a dead-signal candidate when it has recorded zero failures across the signal window while the source files it covers have churned repeatedly. Candidacy is not a verdict.

Typical thresholds, all practitioner conventions rather than standards, and all worth tuning per team: a signal window of 180 days for the no-failure check, at least 10 commits touching the covered files in that window, and a hard minimum of 90 days of per-test history before the class is used at all. Below that minimum there is no signal basis, and every candidate resolves to keep.

Reviewer checklist. Ask all three questions per test, in order:

1. Has the subject's semantics that this test asserts been re-architected? If yes and the test still passes, the test may be a passive regression guard. Keep.
2. Is this test asserting trivially true behavior (the file imports, the class instantiates)? If yes, remove (and note that this is really a trivial removal, with that class's evidence).
3. Is this test asserting a business-critical invariant? Keep regardless of failure history. A regression here would be costly, and its cost is not reduced by the test's quiet record.

Dead-signal removals are never batched. Each one gets its own reviewer sign-off, because the evidence is statistical and the failure mode is silent.

orphan

The test imports a module or calls a symbol that no longer resolves, normally the residue of a refactor that deleted the module but left the test behind, broken or skipped. Confirm the target is genuinely gone rather than moved or re-exported, then remove. Record the missing module in the reason so the reviewer can tell "the module went away" from "the import path went stale".

Step 3 - The delete gate: all four conditions must hold

Classification proposes. This gate disposes. For a removal to be eligible, every one of these must hold. If any one fails, the verdict is keep.

1. No caught regression in the window. The test has never gone from pass to fail to pass-after-fix inside the signal window. That transition is the signature of a test catching a real regression, and one occurrence is enough to end the discussion.
2. Redundant coverage, proven per test. Every source path this test covers is also covered by at least one other test, confirmed against a per-test to source-file map, not a merged coverage report. The bidirectional mapping this requires is the same one test impact analysis builds: "one test (from many) exercises a subset of the production sources" and "one prod source is exercised by a subset of the tests" (Hammant and Fowler, The Rise of Test Impact Analysis). No per-test map means condition 2 cannot be evaluated, which means no removal in that pass. Merged line coverage cannot answer "which other test covers this line".
3. No do-not-delete label. Not @critical, @regression-guard, or any team-configured equivalent.
4. Not flagged for rewrite. A test that a test-code quality review marked as obscure, weakly asserted, or badly structured is a fix, not a delete.

Two properties of this gate matter more than the conditions themselves:

• It is conjunctive and unweighted. Three strong conditions do not outvote one failing condition. Do not convert it into a score.
• A missing input is a failed condition, not a skipped one. No signal history means condition 1 fails. No per-test map means condition 2 fails. Silence never counts in favor of removal.

Step 4 - Choose the action

Removal is one of four possible outcomes, and it is the last one to reach for.

A fold looks like this:

// Before: 4 tests, 4 setup blocks.
test('addItem accepts 1',   () => { /* ... */ });
test('addItem accepts 5',   () => { /* ... */ });
test('addItem accepts 100', () => { /* ... */ });
test('addItem rejects 0',   () => { /* ... */ });

// After: 1 test, per-row failure messages preserved by the name template.
test.each([
  { qty: 1,   expected: 'accepted' },
  { qty: 5,   expected: 'accepted' },
  { qty: 100, expected: 'accepted' },
  { qty: 0,   expected: 'rejected' },
])('addItem qty=$qty is $expected', ({ qty, expected }) => { /* ... */ });

Confidence to action

Confidence is a property of the classification, not a license to skip the gate. Every row below still passes Step 3 before anything is removed.

Keep classes in separate change sets. A single change set mixing high and low confidence rows trains the reviewer to skim, and the low-confidence rows are exactly the ones that need reading.

Step 5 - Record every removal

One row per proposed removal, in the change description, before anyone approves. This ledger is the artifact the rules above exist to produce.

| Test | Class | Reason | Gate 1 no regression | Gate 2 redundant coverage | Gate 3 no label | Gate 4 not flagged | Reviewer |
|------|-------|--------|----------------------|---------------------------|-----------------|--------------------|----------|
| `cart.spec.ts:34 Cart > addItem` | duplicate | Identical signature to `cart.spec.ts:12`; kept copy is the co-located one. | pass (0 fails, 365d) | pass (`src/cart.ts` L12-40 also covered by `cart.spec.ts:12`) | pass | pass | A. Rivera |
| `legacy/report.spec.ts:8 renders` | orphan | Imports `../src/reportBuilder`, deleted in 3f21ac9. | pass | n/a (test cannot run) | pass | pass | A. Rivera |

Alongside it, state what the change set does not do:

**Kept despite classification**

| Test | Class | Why kept |
|------|-------|----------|
| `payment.spec.ts stripe_3ds_failure` | dead-signal | Gate 1 fail: caught a regression 2026-02-12. |
| `auth.spec.ts session_token_rotation` | dead-signal | Gate 3 fail: `@critical:auth-flow`. |
| `parseDate.spec.ts millennium_bug_edge` | dead-signal | Gate 2 fail: only test covering the pre-1970 branch. |

The kept table is not filler. It is how a reviewer checks that the gate was actually run rather than asserted, and it is the record that explains, a year later, why a quiet test was left alone.

Worked example

A suite of 1,283 tests, 41 candidates surfaced, 214 days of per-test history.

Result: 3 removals across two change sets (one for duplicate plus trivial, one for orphan), 1 rewrite, 3 tests kept with recorded reasons. Candidate 5 is the case the whole gate exists for: it looked like the strongest dead-signal row in the batch until the per-test map showed it was the only test on that branch.

Expected output shape

A removal pass produces exactly three artifacts:

1. The removal ledger (Step 5), one row per proposed removal, each with a class, a reason, four gate results, and a named reviewer.
2. The kept table (Step 5), every classified candidate that the gate stopped, with the failing condition named.
3. A separate rewrite and fold list, which are not removals and do not ship in the same change set as deletions.

If the pass produces a list of test names with no gate columns and no reviewer column, it is not a removal proposal. It is a list of suspicions, and nothing on it is eligible to be deleted.

Anti-patterns

Limitations

• A removal decision cannot be fully automated. Classification is mechanical; the dead-signal checklist and the reviewer sign-off are not. Every rule here assumes a person reads the ledger.
• No semantic understanding of value. A test that scores as low-signal on every mechanical input may carry architectural or regression-guard value that no heuristic sees. This is why gate condition 3 depends on labels, and why unlabeled load-bearing tests are the standing risk: the team has to label them for the gate to protect them.
• Redundancy evidence is only as good as the per-test map. Tests that exercise a function indirectly through an integration path may not appear in a per-test coverage map at all, which makes redundancy look higher than it is and over-flags the dead-signal class.
• Short history means no dead-signal decisions. Under the 90-day minimum, the signal-driven classes are unusable. Duplicate, trivial, and orphan still work, because their evidence is structural rather than historical.
• Removing tests does not make a suite fast. It reduces count and maintenance surface. Per-test speed and per-change run size are different problems with different tools.