qa-flake-triage
Flaky test triage: find the condition a flake depends on, quarantine it with an expiry date, fix known flake patterns, and track flake rates. Agents (e2e-flake-bisector, regression-bisector) rerun a test under varied conditions with a shared-state isolation stage, and run git bisect to find the commit that introduced a regression, with a perf-measurement mode.
Install this plugin
View source (opens in a new tab)/plugin install qa-flake-triage@testland-qaFirst time? Add the marketplace with /plugin marketplace add testland/qa
Plugin
- Category
- Operations & resilience
- Version
v1.5.5- Contents
- 4 skills · 2 agents
Skills4
flake-axis-bisectionUse when one test is known to fail intermittently and reading its code has not explained why.skillflake-dashboard-authorUse when a team needs a lasting view of test reliability from CI history (JUnit XML or JSON): a flake-rate metric, a Grafana panel or Datadog CI Visibility view, the quarantine-candidate query, trend alerts, and a weekly or monthly test-health report for Slack or a wiki.skillflake-pattern-referenceReference catalog of the eight flake patterns - async/timing, test ordering, shared parallel state, resource leaks, network, locator drift, environment variance, randomness - with detection heuristics, remediation per pattern, and the concrete code-level fixes: replacing fixed sleeps with framework auto-waits, isolating state in beforeEach fixtures, per-worker DB schemas via workerIndex, try/finally teardown, mocking network + clock at the boundary, stable role-based locators, TZ pinning, and RNG seeding.skillflaky-test-quarantineBuilds a quarantine workflow for flaky tests - marks the test with the framework's skip/fixme/retry annotation, records the failure-rate observation and a bisect link in the annotation body, sets an auto-expiry date, and produces a CI report listing every quarantined test that has expired and needs re-evaluation.skill
Agents2
e2e-flake-bisectorRuns a target end-to-end test N times under varied conditions (worker isolation, test order, viewport, network throttling, parallelism) to identify the axis along which the flake reproduces, then - when the bisect implicates parallel execution - runs a stage-2 shared-state isolation check that instruments DB rows, env vars, files, ports, and global module state to name the specific resource two workers are colliding on, with file:line evidence.agentregression-bisectorOrchestrates git bisect against a target test or build script to identify the introducing commit of a regression.agent
About qa-flake-triage
Flake triage workflow: axis bisector with shared-state isolation checking, regression bisector (pass/fail and perf-measurement modes), flake pattern catalog with per-pattern code fixes, dashboards with periodic trend reports, and the quarantine workflow.