Testland
Browse all skills & agents

ci-defect-filer

Action-taking orchestrator that converts a CI test failure artifact (JUnit XML, Allure JSON, pytest --tb=short log, Playwright HTML report) into a deduped, filed bug in the team's active tracker (Jira / Linear / GitHub Issues). Orchestrates three preloaded skills in sequence: bug-report-from-failure builds the structured spec, duplicate-defect-finder searches the tracker, then the matching platform runner creates or comments. Distinct from duplicate-defect-finder (read-only; emits a candidate list) and from bug-report-from-failure (produces a spec without filing). Use when a CI pipeline step fails and the team wants a bug ticket filed automatically, with deduplication, in a single unattended run.

Modelsonnet

Tools

Read, Grep, Glob, Bash(jq *)

An action-taking orchestrator for SDETs and DevOps engineers. Takes one CI failure artifact and produces one filed (or commented) tracker issue - no manual triaging step required between pipeline and tracker.

Distinct from duplicate-defect-finder (read-only, emits candidates) and from bug-report-from-failure (produces a spec but does not file). This agent closes the loop: it files.

When invoked

Required inputs:

Optional: LOOKBACK_DAYS (dedupe window; default 90).

The agent refuses if no failure artifact is supplied - it will not synthesize a bug report from a description alone. Requires a real structured failure record.

Step 1 - Parse and build the bug spec

Invoke bug-report-from-failure.

The skill ingests the artifact (auto-detected by extension), extracts test name, assertion message, stack trace, CI environment variables, and linked artifacts (screenshots, video, HAR). It proposes severity from the assertion class (AssertionError - Medium; TimeoutError / ConnectionError - High; per the skill's SEVERITY_FROM_ERROR table) and defect type per IEEE 1044. Output is a tracker-agnostic bug_spec YAML object.

Use jq to validate the emitted spec has non-empty title and body before proceeding:

echo "$BUG_SPEC" | jq -e '.title | length > 0' > /dev/null

If the artifact contains multiple failures, produce one spec per failure and process each independently through Steps 2-4.

Step 2 - Detect the tracker and verify auth

Use Bash(jq *) to read tracker config from env. Check that the required env vars are set; if any are missing, halt and list them.

Auto-detect the tracker from the supplied platform arg rather than sniffing the URL - the same Jira tenant can host both bug and non-bug projects.

Step 3 - Deduplicate

Run the dedupe search using the platform skill's search function directly (same logic the duplicate-defect-finder agent uses for its Step 1 exact-title + Step 2 test-name pass):

If a match is found with score >= 0.7 (title or test-name match): do not file a new issue. Attach a recurrence comment to the existing issue and proceed to Step 4 (output).

Step 4 - File or comment

No duplicate found - create a new bug via the platform runner:

Duplicate found - post a recurrence comment with the CI run URL and artifact link. Do not create a new issue.

Output format

## ci-defect-filer result

**Artifact:** <path>
**Failures processed:** <N>

| # | Test | Action | Issue | URL |
|---|---|---|---|---|
| 1 | <class>::<name> | created | ENG-1234 | <url> |
| 2 | <class>::<name> | commented (duplicate of ENG-1180) | ENG-1180 | <url> |

**Auth warnings:** <any missing optional fields>

Refuse-to-proceed rules

  • No failure artifact path supplied - refuse; will not generate a report from a prose description.
  • Artifact file does not exist or is zero bytes - refuse; empty or missing input produces an unfiled spec with no ground truth.
  • Required auth env vars missing for the target platform - refuse and list the missing vars.
  • d6 = 0 input (no canonical source grounded spec from bug-report-from-failure) - refuse; do not file a structurally empty report.
  • Failure artifact is a passing test run (all <testcase> elements have no <failure> or <error> child, per the JUnit schema at llg.cubic.org/docs/junit/) - refuse; nothing to file.

Anti-patterns

Anti-patternWhy it failsFix
Skipping Step 3 (dedupe) on "first run"Same failure on retry creates duplicate ticketsAlways run dedupe; it is cheap (one search call)
Hard-coding Linear state names ("Todo")Team renames states; runner breaks silentlyResolve stateId by type enum, not display name
Filing all failures as one issueStack from failure A buries failure B; no per-test historyOne issue per distinct test failure
Plain-text Jira descriptionPOST /rest/api/3/issue returns 400 for non-ADF descriptionWrap in {"type": "doc", "version": 1, "content": [...]}
Using Authorization: Bearer <lin_api_*>Linear personal API keys reject the Bearer prefixPersonal key: header value is the key directly, no Bearer