Testland
Browse all skills & agents

flag-removal-runbook-author

Workflow-driven skill that builds the runbook for safely removing a feature flag from the codebase + the flag platform. Walks through: pre-removal verification (flag fully rolled out, no usage variance in evaluations, dependent code paths identified), the code-removal steps (delete the if-branches, simplify, restore types), the platform-side removal (archive in LaunchDarkly / Unleash / Flagsmith / GrowthBook), the verification post-removal, and the rollback plan. Use when removing a flag that has finished its mission (rollout-complete, experiment-shipped, kill-switch retired).

Install with skills.sh (any agent)

npx skills add testland/qa --skill flag-removal-runbook-author
View source

flag-removal-runbook-author

Overview

Stale feature flags are technical debt: they bloat the codebase, expand the flag-state matrix per feature-flag-test-matrix-reference, and confuse readers. But removing a flag wrong - deleting code that's actually still gated for some users - is a regression.

This skill produces the runbook for safely removing a flag.

When to use

  • A flag is at 100% rollout for ≥ 2 weeks.
  • An experiment has shipped; the variant code paths need consolidation.
  • A kill-switch hasn't been activated for ≥ 6 months.
  • Stale-flag audit surfaced the flag.

Step 1 - Pre-removal verification

Before any code touches:

CheckPass criterion
Rollout 100% for all segmentsLD/Unleash/etc. dashboard confirms
No alternate behaviour gated on this flagPer-flag-page in platform shows no targeting variation
Evaluation count stableNot a recently-changed flag; no recent variation flips
Dependent flags identifiedGrep for the flag + any flag whose targeting references this flag's value
Owner / sponsor identifiedPerson who can authorise removal
Rollback plan existsIf removal breaks, how to restore

Failure of any check = abort and resolve first.

Step 2 - Identify dependent code

Grep:

grep -rn 'show-new-ui\|"show-new-ui"' --include='*.{ts,js,py,go,java}' .

For each match, classify:

PatternAction
if (client.isOn('flag')) → branch A else branch BDelete branch B; remove the if
Conditional import / requireDelete the conditional; keep the new module
Conditional CSS / stylingDelete the old class; keep the new
Switch on multi-variant flagKeep only the shipped variant
Test fixtures with the flagUpdate or delete
Configuration with default = the old valueUpdate default; document

Step 3 - Code removal

For boolean flag at 100% rollout = true:

// Before
if (client.isOn('show-new-ui')) {
  return <NewHeader />;
} else {
  return <OldHeader />;
}

// After
return <NewHeader />;

Delete <OldHeader /> component if not used elsewhere. Use TypeScript / linter to find unused imports.

For multi-variant flag with treatment-b shipped:

// Before
const variant = client.getFeatureValue('checkout-experiment', 'control');
if (variant === 'treatment-a') { ... }
else if (variant === 'treatment-b') { ... }
else { ... }  // control

// After (only treatment-b retained)
// (inline treatment-b's logic here)

Keep the shipped variant's logic, inlined. Delete the others.

Step 4 - Update tests

Per flag-state-coverage-builder:

  • Remove the per-flag isolation tests for this flag.
  • Update any matrix that included this flag.
  • Update the flag-coverage.yaml to remove entries.
  • Keep any test that asserts the new behaviour (it's no longer flag-gated - it's just the behaviour).
# Find tests referencing the flag
grep -rn 'show-new-ui' tests/

Step 5 - Platform-side removal

PlatformRemoval action
LaunchDarklyArchive the flag (UI: flag → ⋯ → Archive). Don't delete - archived flags retain analytics. Per launchdarkly.com/docs (opens in new window)
UnleashArchive in the UI; deletion is permanent
FlagsmithDelete feature (UI); the environment.json on next regenerate will lack it
GrowthBookArchive in the UI

Archive instead of delete where possible - retains historical analytics in case of regression.

Step 6 - Deploy

Standard deploy. The flag is now:

  • Removed from code
  • Archived in the platform
  • No longer in the test matrix

Production should see identical behaviour as before removal.

Step 7 - Post-removal verification

Within 24h of deploy:

CheckPass
Error rate unchangedNo 5xx spike
Performance unchangedp95 latency unchanged
User-facing behaviour stableNo "broken UI" support tickets
Platform shows zero evaluations of the archived flagConfirms no code still references it

If any check fails → roll back (revert the removal commit).

Step 8 - Rollback plan

# If removal regresses something
git revert <removal-commit>
git push
# Then in the platform: un-archive the flag and re-set its prior value

The rollback plan must be in the PR description for the removal - not memorized.

Step 9 - Document the removal

Add to the project's flag-history.md:

## 2026-05-20 - Removed `show-new-ui`

- Rolled out 100% on 2026-04-15
- Stable for 5 weeks at 100%
- Removed in commit abc1234
- Archived in LaunchDarkly
- Owner sign-off: @engineering-lead
- Rollback plan: revert + un-archive

Multi-flag removal batching

For stale-flag audits surfacing multiple removals: batch is faster but riskier:

Batch sizeWhen
1 flag per PRDefault - easy to revert, clear analytics
3-5 flags per PRSame area of codebase; same owner
10+ flags per PRPeriodic audit; experienced reviewer required

Anti-patterns

Anti-patternWhy it failsFix
Remove flag at <100% rolloutSome users still see the old path; you break themWait for 100% + soak
Delete in platform but leave code"Removed" but still evaluated → SDK errors on missing flagCode first, then platform
Leave code but archive in platformCode still references; defaults kick in; behaviour changeEither both or neither
No rollback planHard to recover if regressionAlways plan revert + un-archive
Skip post-removal verification"It compiled; ship it"Watch error rate / latency / tickets for 24h
Bulk remove without owner sign-offUnilateral removal of someone else's workPer-flag owner sign-off
Re-use flag name laterConfuses analytics, archive-then-create-newNew flag, new name
Forget to update testsTests still gate on the removed flag; mock returns default → wrong outcomeStep 4
Leave dead variants from multi-variantHalf-clean removalInline shipped variant's logic

Output

This skill produces:

  • A per-flag removal PR (code + tests + coverage-doc updated).
  • A platform-side checklist for the archive action.
  • A 24h post-deploy verification plan.
  • A rollback-ready PR description.
  • A line in flag-history.md.

References

  • LaunchDarkly archive doc: launchdarkly.com/docs (opens in new window).
  • Companion catalogs: feature-flag-test-matrix-reference, flag-state-coverage-builder.
  • Sibling SDKs (for platform-specific archive steps): launchdarkly-testing, unleash-testing, flagsmith-testing, growthbook-testing.

Related skills

feature-flag-test-matrix-reference

Pure-reference catalog of feature-flag test matrix design. Defines the flag-state combinatorics problem (N flags × M variants × K user-segments = N×M×K test cases), the canonical coverage strategies (pairwise interaction coverage; default-only smoke; full matrix; risk-driven matrix), the kill-switch + percentage-rollout test patterns, and the relationship between flags + experiments (flags toggle behaviour; experiments measure outcome). Use when designing the flag-test surface for a new project or auditing existing flag-test coverage.

flag-state-coverage-builder

Workflow-driven skill that builds a flag-state coverage matrix from the project's flag inventory and risk register. Walks through: inventorying flags (grep for flag-evaluation calls), classifying each (boolean / multi-variant / kill-switch / experiment), choosing the coverage strategy (per-flag-isolation / pairwise / full / risk-driven per feature-flag-test-matrix-reference), generating the test matrix (PICT for pairwise; manual for risk-driven), and emitting test skeletons. Use when introducing flag-test coverage to a new codebase or when a flag-related incident exposes a coverage gap.

flagsmith-testing

Wraps Flagsmith server-side SDK testing patterns (feature flags / feature toggles) so tests run without calling the Flagsmith API: offline mode with LocalFileHandler + a downloaded environment.json snapshot, local-evaluation mode (no per-request network), and default_flag_handler for per-feature mocked fallbacks, via the get_environment_flags / get_identity_flags evaluation paths. Use when writing feature-flag tests for code that uses Flagsmith, mocking flag values, or testing feature toggles offline in CI.

growthbook-testing

Wraps GrowthBook Node SDK testing patterns: GrowthBookClient initialization with direct payload (initSync; no network), isOn / getFeatureValue / evalFeature, scoped instances (createScopedInstance) for per-request user context, inline experiment (runInlineExperiment) tests, and tracking-callback assertion patterns. Use when writing tests for code using GrowthBook for flags + experiments.

killswitch-test-author

Workflow-driven skill that authors the four test categories specific to kill-switch (ops-toggle) flags: switch-OFF graceful degradation, fail-static default when the flag service is unreachable, latency budget for the kill decision, and no-data-corruption mid-flight. Distinct from flag-state-coverage-builder (which builds a full coverage matrix across all flag types) and feature-flag-test-matrix-reference (which catalogs patterns without producing tests). Use when a kill-switch flag exists in the codebase and needs dedicated, production-incident-rehearsing tests authored for it.

launchdarkly-testing

Wraps LaunchDarkly server-side SDK testing patterns: TestData data source for hermetic tests (no network), file-based data source for fixture-driven tests, flag override patterns (TestData.update for per-test flag values), and assignment-integrity tests. Use when writing tests for code that uses LaunchDarkly flags.

openfeature-sdk-testing

Wraps OpenFeature (CNCF vendor-neutral SDK abstraction) testing patterns: the InMemoryProvider for hermetic tests without network calls, provider registration via OpenFeature.setProvider, the getBooleanValue/getBooleanDetails evaluation API with EvaluationDetails (value, variant, reason, errorCode), hooks for evaluation side-effects, and evaluation context for targeting-rule tests. Covers TypeScript, Java, and Python SDKs. Use when writing tests for code that resolves feature flags through the OpenFeature SDK regardless of the underlying flag management platform.

unleash-testing

Wraps Unleash (Open Source / SaaS) SDK testing patterns: bootstrap with a static toggles array (no network), the test mode (disableMetrics + disablePolling), the custom strategy testing pattern (implement a Strategy class + assert isEnabled), and assignment-integrity tests. Use when writing tests for code that uses Unleash for feature flags.