qa-resilience
Chaos engineering, fault injection, and resilience drills: 7 skills (chaos-experiment-author, chaos-mesh, toxiproxy-chaos, failure-injection-test-author, chaos-drill-protocol, dr-drill-runner, error-budget-tests). Chaos-experiment design with steady-state pre-flight validation and LitmusChaos/Gremlin routing, the chaos-drill run protocol with abort gates, Chaos Mesh, Toxiproxy, and combined HTTP+TCP fault injection, plus the DR-drill discipline (backup verification + restore-time / RTO measurement in references) and error-budget gating with burn-rate alerting and incident metrics.
/plugin install qa-resilience@testland-qaFirst time? Add the marketplace with /plugin marketplace add testland/qa
- Category
- Operations & resilience
- Version
v1.0.1- Contents
- 7 skills
- Part of
- qa-role-performance
Skills 7
chaos-drill-protocolRun protocol and run workflow for a chaos experiment that has already been designed: the four pre-flight gates (non-production target, measured healthy baseline, live observability, a rollback that has actually been exercised), how to pick a conservative blast-radius bound, the sampling cadence and abort criteria fixed in writing before injection, the per-runner inject and abort commands (Chaos Mesh / Litmus / Gremlin / Toxiproxy), the refuse-to-start rules (no blast-radius bound, production context, degraded baseline, offline observability, unexercised rollback), and the recovery-validation step with its tolerance and timeout.chaos-experiment-authorBuild-an-X workflow for a chaos experiment per the Principles of Chaos Engineering - defines steady-state hypothesis, picks the variables (real-world events: network latency, node failure, region outage), sets the blast radius (which percentage / namespace / user cohort), automates execution, and emits the verdict (steady-state held / didn't hold).chaos-meshConfigures Chaos Mesh for Kubernetes-native chaos engineering - picks fault types (PodChaos, NetworkChaos, StressChaos, IOChaos, TimeChaos, DNSChaos, KernelChaos, HTTPChaos), targets via label selectors, controls blast radius via namespace whitelists + selector filters, schedules via CronJobs, observes via dashboard.dr-drill-runnerThe full DR-drill discipline for one service: author the runbook (per-tier RTO + RPO), pre-drill checklist (data sync state, alert silencing, customer comms), drill workflow (announce, fail-over, verify, fail-back) with timestamps, the supervised run protocol (refuse without declared RTO/RPO or against production, RTO/RPO monitoring cadence, abort-on-breach), and an auditor-ready post-drill report.error-budget-testsBuild error-budget gate tests - SLO + error-budget calculation per Google SRE workbook ("difference between target uptime and actual uptime"); burn-rate alerting; monthly-budget exhaustion test; freeze-trigger when budget consumed.failure-injection-test-authorOrchestrates WireMock fault stubs (HTTP-level fault: 500s, malformed JSON, slow responses) with Toxiproxy (TCP-level: latency, packet loss, reset) into a single resilience test scenario - the test starts both, applies fault per scenario, runs the SUT against the impaired endpoints, verifies the SUT's resilience patterns.toxiproxy-chaosConfigures Toxiproxy for TCP-level fault injection - runs as a sidecar / proxy between client and upstream, applies toxics (latency, bandwidth, slow_close, timeout, slicer, limit_data, reset_peer) via control API.
About qa-resilience
Chaos engineering, fault injection, and resilience drills. The unrehearsed half: structured chaos-experiment authoring with steady-state pre-flight validation and per-tool routing (Chaos Mesh standalone; LitmusChaos and Gremlin as references), the chaos-drill run protocol with abort gates and recovery validation, Toxiproxy (TCP-level), and combined HTTP+TCP fault injection. The rehearsed half: measured, scheduled DR drills with backup verification and restore-time SLAs worked in references, plus error budgets and MTTR/MTBF tracking.