perf-budget-gate

Builds a unified release-readiness gate that aggregates verdicts from any combination of k6 / JMeter / Gatling / Locust load runners and Lighthouse CI Web Vitals, applies severity-aware pass/fail thresholds, and emits a single go / no-go decision with per-metric deltas vs the main-branch baseline. Posts the delta as a PR comment when the team has the integration set up. Use when authoring a CI step that gates a deployment on cross-runner perf compatibility.

Install with skills.sh (any agent)

npx skills add testland/qa --skill perf-budget-gate

View source

perf-budget-gate

Overview

Modern teams measure perf at multiple layers:

Layer	Runner
Backend load	`k6-load-testing`, `jmeter-load-testing`, `gatling-load-testing`, `locust-load-testing`
Frontend	`lighthouse-perf` - Web Vitals via Lighthouse CI

Each runner has its own pass/fail criterion. This gate unifies them into a single go / no-go verdict with per-metric deltas vs. main, and emits a markdown summary suitable for $GITHUB_STEP_SUMMARY or a PR comment.

This is the perf counterpart to data-quality-gate, visual-baseline-gate, and contract-compatibility-gate - same artifact shape, different domain.

When to use

The team uses two or more perf runners and wants one CI gate.
Per-PR perf delta vs. main is the team's regression-detection signal.
Some metrics should be advisory rather than blocking - e.g. block on p95 latency regression but warn on Lighthouse score drift.
Per-metric ratchet behavior is needed (existing budget breaches grandfathered, new breaches block).

If the project has only one runner, defer this gate - use the runner's native CI integration directly.

How to use

Collect each runner's output artifact (k6 summary.json, Lighthouse lhr-*.json, and the rest) - the per-runner source map is in references/ci-wiring-and-metric-sources.md.
Flatten every runner's output into the unified metric record (below), one record per measured subject + metric.
Fetch the last green main-branch baseline and compute each record's delta.
Apply the gate decision rule to produce a single go / no-go verdict.
Emit the markdown + JSON artifact; a no-go exits non-zero and halts CI. Full CI wiring and per-metric budgets are in references/ci-wiring-and-metric-sources.md.

The unified metric record

Flatten every runner's output into one shape:

{
  "runner":   "k6",
  "subject":  "GET /api/orders",
  "metric":   "p95_latency_ms",
  "value":    320,
  "baseline": 280,
  "delta":    "+14.3%",
  "budget":   500,
  "status":   "pass",
  "severity": "blocker"
}

Field	Source
`runner`	`k6` / `jmeter` / `gatling` / `locust` / `lighthouse`.
`subject`	URL path / sampler name / story ID - what was measured.
`metric`	`p95_latency_ms` / `error_rate` / `lcp_ms` / `inp_ms` / `cls`.
`value`	Current run's value.
`baseline`	Last green main-branch run's value (from artifact storage / Grafana / Lighthouse CI server).
`delta`	Percent change vs. baseline.
`budget`	Configured threshold from the runner.
`status`	`pass` / `fail` based on `value` vs `budget`.
`severity`	`blocker` / `warn`.

The gate decision rule

def gate_decision(records, *,
                  block_on_regression_pct=10,   # block if any blocker regresses >10%
                  warn_on_regression_pct=3):    # warn if anything regresses >3%
    blockers = []
    warnings = []
    for r in records:
        if r["status"] == "fail" and r["severity"] == "blocker":
            blockers.append((r, "budget breach"))
        elif r["delta_pct"] > block_on_regression_pct and r["severity"] == "blocker":
            blockers.append((r, f"regression > {block_on_regression_pct}%"))
        elif r["delta_pct"] > warn_on_regression_pct:
            warnings.append((r, f"regression > {warn_on_regression_pct}%"))

    return {
        "verdict": "no-go" if blockers else "go",
        "blocker_count": len(blockers),
        "warning_count": len(warnings),
        "blockers": blockers,
        "warnings": warnings,
    }

Two regression triggers:

Budget breach - value > budget (absolute threshold).
Regression - delta_pct > N% vs. baseline (relative).

Both matter: a metric within budget but trending up still warrants a warning.

Emit the artifact

Markdown summary:

# Perf Budget Gate - verdict: NO-GO

**Blockers: 2**

| Runner     | Subject              | Metric            | Current | Baseline | Δ      | Budget | Status |
|------------|----------------------|-------------------|--------:|---------:|-------:|-------:|--------|
| k6         | POST /api/orders     | p95 latency       | 620ms   | 280ms    | +121% | 500ms  | FAIL   |
| lighthouse | /dashboard           | LCP               | 3200ms  | 2100ms   | +52%  | 2500ms | FAIL   |

**Warnings: 3**

| Runner     | Subject              | Metric            | Current | Baseline | Δ     |
|------------|----------------------|-------------------|--------:|---------:|------:|
| k6         | GET /api/orders      | p95 latency       | 240ms   | 220ms    | +9%   |
| lighthouse | /                    | INP               | 180ms   | 150ms    | +20%  |
| locust     | GET /search          | p95 latency       | 410ms   | 380ms    | +8%   |

Plus a JSON sibling for downstream tooling:

{
  "verdict": "no-go",
  "blocker_count": 2,
  "warning_count": 3,
  "blockers": [...],
  "warnings": [...]
}

A no-go verdict exits non-zero - CI halts.

Worked example

Run the gate on one build: read the k6 and Lighthouse artifacts, flatten each into records, apply the rule, print the verdict, and exit with its code.

# scripts/run_perf_gate.py
import json, csv, sys, os
from pathlib import Path

records = []

# Source: k6 summary.json
k6_path = Path("k6-summary.json")
if k6_path.exists():
    s = json.loads(k6_path.read_text())
    p95 = s["metrics"]["http_req_duration"]["values"]["p(95)"]
    error_rate = s["metrics"]["http_req_failed"]["values"]["rate"]
    records += [
        {"runner": "k6", "subject": "global", "metric": "p95_latency_ms",
         "value": p95, "budget": 500, "severity": "blocker",
         "status": "fail" if p95 > 500 else "pass"},
        {"runner": "k6", "subject": "global", "metric": "error_rate",
         "value": error_rate, "budget": 0.01, "severity": "blocker",
         "status": "fail" if error_rate > 0.01 else "pass"},
    ]

# Source: Lighthouse CI lhr-*.json
for lhr in Path(".lighthouseci/").glob("lhr-*.json"):
    r = json.loads(lhr.read_text())
    url = r["finalUrl"]
    lcp = r["audits"]["largest-contentful-paint"]["numericValue"]
    inp = r["audits"]["interaction-to-next-paint"]["numericValue"]
    cls = r["audits"]["cumulative-layout-shift"]["numericValue"]
    records += [
        {"runner": "lighthouse", "subject": url, "metric": "lcp_ms",
         "value": lcp, "budget": 2500, "severity": "blocker",
         "status": "fail" if lcp > 2500 else "pass"},
        {"runner": "lighthouse", "subject": url, "metric": "inp_ms",
         "value": inp, "budget": 200, "severity": "blocker",
         "status": "fail" if inp > 200 else "pass"},
        {"runner": "lighthouse", "subject": url, "metric": "cls",
         "value": cls, "budget": 0.1, "severity": "blocker",
         "status": "fail" if cls > 0.1 else "pass"},
    ]

# Apply gate
blockers = [r for r in records if r["status"] == "fail" and r["severity"] == "blocker"]
verdict = "no-go" if blockers else "go"

print(f"# Perf Budget Gate - verdict: {verdict.upper()}")
for r in blockers:
    print(f"- {r['runner']} :: {r['subject']} :: {r['metric']} = {r['value']} (budget {r['budget']})")

sys.exit(0 if verdict == "go" else 1)

Anti-patterns

Anti-pattern	Why it fails	Fix
Hardcoded budgets in the gate script	Budgets evolve; updating requires code review.	Externalize to `.perf-budgets.yml` consumed by the gate.
Comparing against the previous run, not the main baseline	Drift compounds; the team approves a 5% regression every PR.	Compare against the last-known-green main commit.
Block on every metric	The gate becomes "the perf gate that always fails."	Block on the team's documented NFRs only; everything else is `warn`.
Skipping baseline storage	First-run-after-budget-update has nothing to compare against.	Persist baseline JSON as a build artifact uploaded on every main-branch run.
Asserting on Lighthouse score (categories:performance)	The score conflates LCP/INP/CLS; one bad Web Vital tanks the score uninterpretably.	Assert on individual Web Vitals; category score is supplementary.

References

Per-runner source artifacts, per-metric budgets, and the full CI wiring: references/ci-wiring-and-metric-sources.md.
data-quality-gate, visual-baseline-gate, contract-compatibility-gate - sibling gates with the same artifact shape.
non-functional-requirement-extractor - upstream skill that produces the threshold-bound budgets this gate enforces.

Perf budget gate - runner sources, metric budgets, and CI wiring

View source

Perf budget gate - runner sources, metric budgets, and CI wiring

Deep reference for perf-budget-gate SKILL.md. Consult when wiring the gate into CI, mapping each runner's output artifact, or setting per-metric budgets.

Runner sources and artifacts

Each runner writes a machine-readable artifact the gate flattens into the unified metric record. Persist each as a CI build artifact (if: always()) so the gate input is reproducible and triageable.

Source	Artifact	Produce with	Yields
k6	`summary.json`	`k6 run --summary-export summary.json`	Per-metric values + threshold pass/fail.
JMeter	`results.jtl` + `report/statistics.json`	`jmeter -n -t plan.jmx -l results.jtl -e -o report`	Per-sampler percentiles + counts.
Gatling	`js/stats.json` under `target/gatling/<sim>-<ts>/`	Gatling run (writes the HTML report bundle)	Per-request percentiles + assertion outcomes.
Locust	`<prefix>_stats.csv`	`locust --csv <prefix> --headless`	Per-endpoint percentiles.
Lighthouse CI	`.lighthouseci/lhr-*.json`	`lhci autorun` / `lhci collect`	Per-URL audit results including Web Vitals.

Per-metric budgets

Back-end latency budgets come from the team's NFRs; the Core Web Vitals "good" thresholds are the canonical front-end defaults (web-vitals (opens in new window)).

Metric	Budget	Severity (typical)
`p95_latency_ms`	team NFR (e.g. 500)	blocker
`error_rate`	0.01	blocker
`lcp_ms` (Largest Contentful Paint)	2500	blocker
`inp_ms` (Interaction to Next Paint)	200	blocker
`cls` (Cumulative Layout Shift)	0.1	blocker
Lighthouse category score	advisory	warn

Assert on individual Web Vitals, not the aggregate Lighthouse performance score - the score conflates LCP / INP / CLS, so one bad vital tanks it uninterpretably.

CI wiring

Run each runner, upload its artifact, then run the gate as the final step. The gate writes its markdown summary to $GITHUB_STEP_SUMMARY and exits non-zero on a no-go verdict, which fails the job (gha-summary (opens in new window)).

# .github/workflows/perf-gate.yml
- name: k6
  run: k6 run --summary-export k6-summary.json load.js
- name: Lighthouse CI
  run: npx lhci autorun
- name: Upload perf artifacts
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: perf-artifacts
    path: |
      k6-summary.json
      .lighthouseci/
- name: Perf budget gate
  run: python scripts/run_perf_gate.py >> "$GITHUB_STEP_SUMMARY"

Store the last green main-branch record set as a baseline artifact uploaded on every main-branch run, so per-PR deltas compare against a real baseline rather than the previous PR.

Related skills

db-query-plan-analyzer

Reads `EXPLAIN` / `EXPLAIN ANALYZE` output from PostgreSQL, MySQL, or SQLite - identifies the dominant cost (sequential scan, nested loop, sort spill, missing index, type-cast preventing index use), proposes the specific index or query rewrite to fix it, and emits the candidate `CREATE INDEX` statement. Use when load testing or production telemetry shows the database as the bottleneck and the team needs targeted query-level remediation.

flame-graph-analyzer

Reads CPU flame-graph output from py-spy (Python), async-profiler (JVM), Go pprof, or Node.js `perf_hooks` / clinic.js: identifies the hot path (top sample-time frames), classifies the bottleneck (CPU-bound vs lock contention vs allocator pressure), and proposes the next investigation step. Use when a perf regression is bisected to a commit but the hot path inside it is unclear; for tail-latency percentiles use latency-percentile-analyzer, for GC pauses specifically use jvm-gc-tuning, and for a slow SQL hot path use db-query-plan-analyzer.

gatling-load-testing

Authors Gatling simulations in Java / Kotlin / Scala (or JS / TS) using the Simulation class plus http() / scenario() / exec() DSL builders, ramps virtual users via injectOpen (arrival rate) or injectClosed (concurrent count), runs via Maven / Gradle / sbt with the Gatling plugin, and gates CI on assertions defined in setUp(). Use when the project is on the JVM and the team prefers code-first load tests over JMeter's XML or k6's JavaScript-only authoring.

jmeter-load-testing

Authors Apache JMeter `.jmx` test plans (Thread Groups + HTTP samplers + assertions + listeners) in the JMeter GUI, runs them headlessly via `jmeter -n -t plan.jmx -l results.jtl`, generates an HTML dashboard with `-e -o`, and gates CI on JTL parsing. Use when the project has an existing JMeter investment, needs JVM-native load tooling, or works in domains with strong JMeter community support (banking, telecom, enterprise).

jvm-gc-tuning

Diagnoses JVM garbage-collection behaviour under load: reads and interprets unified GC logs (-Xlog:gc*), selects the right collector (G1 vs ZGC vs Parallel vs Serial), tunes heap sizing and pause-time targets, quantifies allocation rate, and traces the GC-pause-to-latency-tail link using GCViewer and Java Flight Recorder (JFR). Use when a load test reveals p99/p999 latency spikes that correlate with GC activity, or when heap sizing and collector selection need justification before a performance baseline is locked.

k6-load-testing

Authors k6 JavaScript load-test scripts (VU loops + checks + sleeps), configures the `options` block with `stages` (ramp-up patterns) and `thresholds` (p(95) latency, error rate), runs via `k6 run script.js` or `--vus / --duration` ad-hoc flags, and uses thresholds as the CI pass/fail signal. Use when the project ships HTTP / WebSocket / gRPC load tests and the team wants developer-friendly JavaScript authoring.

latency-percentile-analyzer

Interprets latency distributions from k6 load tests beyond the p95/p99 gate: reads percentile summaries and JSON exports to identify tail shape, computes the tail ratio (p99/p50) as a distribution-spread signal, detects bimodal distributions, explains coordinated omission and why naive p99 values are optimistic under sustained load, and distinguishes request-rate from concurrency models. Use when a k6 threshold passes but the system still feels slow, when p99 is suspiciously low during ramp-up, or when the team needs to explain why tail latency is high rather than just observing that it is.

lighthouse-budget-author

Drafts a `lighthouserc.js` (or `budget.json`) at design time - picks Web Vitals thresholds (LCP / INP / CLS) per route based on traffic class (cached / dynamic / API-heavy / form-heavy) and the team's NFRs, plus resource-size budgets (JS / CSS / images / total bytes). Emits the config file ready for the lighthouse-perf runner. Use when starting Lighthouse coverage on a project that has no budgets yet, or when the existing budgets need a redesign.

lighthouse-perf

Configures Lighthouse CI (`@lhci/cli`) to audit Web Vitals (LCP, INP, CLS) on every PR, asserts against canonical thresholds (LCP ≤2.5s, INP ≤200ms, CLS ≤0.1 at the 75th percentile), uploads Lighthouse reports as build artifacts, and posts deltas as PR comments. Use when the project ships a web frontend and the team needs continuous Web Vitals monitoring tied to PR gating.

load-testing-overview

Teaches load and performance testing from zero: how to choose between k6, JMeter, Gatling, Locust, and Artillery based on observable project facts (team language, tests-as-code vs GUI authoring, protocols beyond HTTP, CI gating needs); the six load profiles (smoke, average-load, stress, spike, soak, breakpoint) and the question each one answers; the difference between open workload models that hold arrival rate constant and closed models that hold concurrent users constant; why percentiles rather than averages are the unit of measurement; and how to turn a run into a pass/fail CI gate, with a first runnable k6 script. Use when a service needs performance coverage and the tool, the load profile, or the pass/fail threshold has not been decided yet.

locust-load-testing

Authors Locust load tests as Python classes - HttpUser with @task-decorated methods plus on_start hooks and between() wait_time - runs via `locust -f locustfile.py` headless mode (or distributed via `--master` / `--worker`), and exports CSV / JUnit reports for CI gating. Use when the project's primary stack is Python and the team wants load tests in the same language as the application.

slo-load-test-plan

Turns a service's SLOs and endpoint traffic mix into a named scenario matrix: one scenario per SLO boundary condition, a load profile (smoke, average-load, stress, soak, spike, breakpoint) per scenario, an open or closed workload injection model, a threshold expression derived from the SLO the scenario guards, and an error-budget calculation that sets the soak run's failure allowance. Stays runner-agnostic and fixes the pass/fail line before any tool is configured. Use when an SLO document and an endpoint list both exist but nobody has decided which load runs to make, what shape of load each carries, or what number would count as a failure.