tcm-case-management
Test case management (TCM) across the five major platforms - TestRail, Xray, Zephyr Scale, Allure TestOps, and Qase - one tool-agnostic workflow for pre-execution case authoring and repository management: create and update cases, organise suites / sections / folders, attach structured steps with per-step expected results, link cases to requirements (Jira / Linear / GitHub), bulk import from CSV / JSON with idempotent re-runs, and sync from CI. The body works the workflow end to end against TestRail's API v2; references/ carries the per-vendor API specifics (auth model, endpoints, steps shape, enums, rate limits) for all five tools. Use for test case management in any of the five TCMs - authoring cases from a spec, bulk-importing legacy cases, migrating between tools, or mass-editing a case repository. Do NOT use for posting test-run results (pass/fail): result sync is the qa-test-reporting plugin's *-integration surface.
Install with skills.sh (any agent)
npx skills add testland/qa --skill tcm-case-managementtcm-case-management
Overview
Every mainstream TCM stores the same canonical case anatomy (test-case-anatomy-reference: identifier, objective, preconditions, steps, expected results, traceability) under different field names, containers, and auth models. The workflow is identical across tools:
This skill works that workflow end to end using TestRail (the largest install base) as the primary example, then routes the per-vendor API deltas to references/.
Differentiation vs result sync: this skill operates on the case repository - create, update, organise, traceability - a strictly pre-execution concern. Posting test-run results (pass/fail, status updates) is the qa-test-reporting plugin's test-management-sync / test-management-sync / test-management-sync surface.
When to use
Vendor routing table
| Vendor | Reference | Auth | Case container | Steps shape | Distinctive feature |
|---|---|---|---|---|---|
| TestRail | references/testrail.md | HTTP Basic (email + API key) | project → suite → section | custom_steps_separated array (Steps template) | Template system (Steps / Text / Exploratory); refs free-text requirement links |
| Xray (Jira) | references/xray.md | OAuth client credentials → JWT (separate from Jira auth) | Jira issue with issuetype: Test | GraphQL steps (action / data / result) | Manual / Cucumber / Generic test types; .feature import; preconditions as linked issues |
| Zephyr Scale (Jira) | references/zephyr-scale.md | Bearer token (per-user) | project → folder | separate /teststeps endpoint, OVERWRITE / APPEND modes | Folder hierarchy per entity type; Jira-native links |
| Allure TestOps | references/allure-testops.md | Bearer token | project → suite + layer + feature | nested scenario.steps (sub-steps) | Links automated allure-results back to manual cases (@allure.testcase) |
| Qase | references/qase-io.md | Token header (not Authorization) | project → suite | flat steps array (action / expected_result / data) | Shared steps reused across cases; inverted priority enum (1=High) |
Each reference carries the vendor's auth setup, create / update / list code, bulk-import loop, migration field map, response shapes, rate limits, and anti-patterns.
The workflow, worked with TestRail
TestRail organises tests as cases inside sections inside suites inside projects. The API v2 covers full CRUD on each plus templates, custom fields, references, and types. Authentication is HTTP Basic with email + API key per the TestRail support docs (support.testrail.com/hc/en-us/articles/7077871398036-Cases - Cloudflare-protected, cite by stable URL).
Step 1 - Authenticate
export TR_BASE="https://your-tenant.testrail.io"
export TR_EMAIL="you@company.com"
export TR_KEY="<api-key-from-user-profile>"import requests, base64, os, json
auth = base64.b64encode(
f"{os.environ['TR_EMAIL']}:{os.environ['TR_KEY']}".encode()
).decode()
HEADERS = {
"Authorization": f"Basic {auth}",
"Content-Type": "application/json",
}
BASE = os.environ["TR_BASE"]
def api(path, method="GET", body=None):
url = f"{BASE}/index.php?/api/v2/{path.lstrip('/')}"
r = requests.request(method, url, headers=HEADERS,
data=json.dumps(body) if body else None)
r.raise_for_status()
return r.json()Step 2 - Create a case with structured steps
POST /index.php?/api/v2/add_case/:section_id:
def create_case(section_id, title, template_id=1, type_id=1, priority_id=2,
preconditions=None, steps=None, refs=None):
"""
template_id: 1=Steps, 2=Text, 3=Exploratory
type_id: per project - discover via get_case_types
priority_id: 1=Low, 2=Medium, 3=High, 4=Critical (default project enum)
steps: list of {"content": "Action", "expected": "Outcome"}
refs: comma-separated requirement IDs (e.g., "REQ-123,REQ-124")
"""
body = {
"title": title,
"template_id": template_id,
"type_id": type_id,
"priority_id": priority_id,
}
if preconditions:
body["custom_preconds"] = preconditions
if steps and template_id == 1:
body["custom_steps_separated"] = steps
if refs:
body["refs"] = refs
return api(f"add_case/{section_id}", method="POST", body=body)
steps = [
{"content": "Navigate to /login", "expected": "Login form rendered"},
{"content": "Enter alice@example.com + correct password",
"expected": "Submit button enabled"},
{"content": "Click Submit",
"expected": "Redirected to /dashboard within 2 s"},
]
new_case = create_case(
section_id=42,
title="Login with valid credentials redirects to dashboard",
template_id=1,
steps=steps,
preconditions="User `alice@example.com` exists; password set to 'pw123'.",
refs="REQ-AUTH-001",
)
print(new_case["id"]) # e.g., 1234One action per step, each paired with its expected result - the per-step result tracking is why steps go in the structured field, never a text blob.
Step 3 - Discover tenant enums at runtime
Each tenant defines its own custom fields, case types, and priorities; discover their IDs at runtime rather than hard-coding them:
fields = api("get_case_fields") # system_name, label, type_id per field
types = api("get_case_types") # id + name per case type
prios = api("get_priorities") # id + name per priority
type_ids = {t["name"]: t["id"] for t in types}
prio_ids = {p["name"]: p["id"] for p in prios}This discovery step exists in every vendor: Zephyr's statusName / priorityName enums are project-scoped, Allure TestOps layers / features are per-project, Qase types are configurable. Never hard-code the integers.
Step 4 - Update, get, and list with pagination
def update_case(case_id, **fields):
return api(f"update_case/{case_id}", method="POST", body=fields)
update_case(1234, refs="REQ-AUTH-001,REQ-AUTH-002") # bulk re-tag
case = api(f"get_case/1234")
# List with pagination:
cases = []
offset = 0
while True:
page = api(f"get_cases/{project_id}&suite_id={suite_id}"
f"&limit=250&offset={offset}")
cases.extend(page.get("cases", page) if isinstance(page, dict) else page)
if isinstance(page, dict) and page.get("size", 0) < 250:
break
offset += 250Newer TestRail versions return {"offset", "limit", "size", "_links", "cases": [...]}; older return a bare array. Handle both. Every vendor paginates differently (Zephyr isLast, Allure last, Qase entities < limit) - see the references.
Step 5 - Bulk import from CSV, idempotently
Build name-to-id maps once, then inside the loop check for an existing case by title before create so re-runs stay idempotent. get_cases accepts filter (substring match on title):
import csv
created, skipped = [], []
for row in csv.DictReader(open("legacy-cases.csv")):
title = row["title"]
existing = api(f"get_cases/{project_id}&suite_id={suite_id}"
f"&filter={requests.utils.quote(title[:60])}")
if existing.get("cases"):
skipped.append((title, existing["cases"][0]["id"]))
continue # already present; skip and log
steps = [
{"content": s, "expected": e}
for s, e in zip(row["steps"].split("|"), row["expected"].split("|"))
]
case = create_case(
section_id=int(row["section_id"]),
title=title,
template_id=1,
type_id=type_ids[row["type"]],
priority_id=prio_ids[row["priority"]],
preconditions=row.get("preconditions"),
steps=steps,
refs=row.get("refs"),
)
created.append(case["id"])Verify: assert len(created) + len(skipped) equals the row count before declaring the import done; a shortfall means a row raised - inspect it, fix the source row, and re-run (already-created cases are skipped by the title check). On rate-limited vendors (Xray, Zephyr, Qase throttle around 60 req/min) add a time.sleep(1) per row and back off further on 429.
Step 6 - CI sync
Sync cases from a tests/ directory layout. One pattern: each spec file has front-matter declaring the case ID; on PR merge, post updates back:
- name: Sync test cases to TestRail
env:
TR_BASE: ${{ vars.TR_BASE }}
TR_EMAIL: ${{ vars.TR_EMAIL }}
TR_KEY: ${{ secrets.TR_KEY }}
run: python scripts/sync-testrail.pyHave the sync script exit non-zero when any case fails to sync, and verify the repository case count matches the source count after the run - a mismatch means a silent partial sync.
Parsing results
Create responses return the full case object (id, created_on, updated_on, ...). Permalink:
url = f"{BASE}/index.php?/cases/view/{case_id}"Per-vendor response shapes and permalink formats are in each reference.
Anti-patterns (cross-vendor)
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Hard-coded type / priority / status IDs | Enums differ per project and tenant | Discover at runtime (Step 3) |
| Steps as a single text blob | Per-step results unavailable | Use the structured steps field (Step 2) |
| Plain title with no requirement refs | Coverage reports show 0% requirement coverage | Always link cases to requirement IDs |
| Creating cases in the root container | Hard to find later; organisation chaos | Always pick a section / folder / suite; create as needed |
| No pagination on list endpoints | Misses cases beyond the first page | Loop per the vendor's pagination contract (Step 4) |
| Storing API keys in code | Token leak | Environment variable / CI secret store |
| Bulk-create without throttling or idempotency | 429s; duplicate cases on re-run | Rate-limit + title-check pattern (Step 5) |
| Vendor-specific case structure as the authoring source | Migration cost exploded | Author in the canonical anatomy (test-case-anatomy-reference); let the tracker mapping be additive |
Limitations
References
Allure TestOps - API specifics
View source (opens in new window)Allure TestOps - API specifics
Per-vendor reference for the tcm-case-management SKILL.md. Author and manage Allure TestOps test cases via the REST API - create cases, manage projects + suites, attach scenarios with nested steps, link to issue trackers, sync with allure-results from CI runs. Per docs.qameta.io/allure-testops (Cloudflare-protected; cite by stable URL).
Allure TestOps (formerly Allure Server, by Qameta Software) combines test case management with the Allure reporting framework that automated tests already emit. The unique feature: automated tests' allure-results JSON link back to manual case definitions in the TCM, providing automation-coverage visibility at the case level.
Authentication
Allure TestOps uses Bearer tokens (API tokens generated per user):
export ATO_BASE="https://your-tenant.qatools.cloud" # or self-hosted URL
export ATO_TOKEN="<api-token>"import requests, os
BASE = os.environ["ATO_BASE"]
HEADERS = {
"Authorization": f"Bearer {os.environ['ATO_TOKEN']}",
"Content-Type": "application/json",
}Create a test case
POST /api/rs/testcase:
def create_case(project_id, name, description=None, precondition=None,
scenario_steps=None, status="Active", layer_id=None,
feature_id=None):
"""
scenario_steps: list of {"keyword": "Given|When|Then|And",
"name": "...", "expectedResult": "..."}
status: Active / Outdated / Archived
layer_id: testing layer (UI / API / Component / Unit) - discover via
/api/rs/testlayer
"""
body = {
"projectId": project_id,
"name": name,
"description": description,
"precondition": precondition,
"status": status,
"layerId": layer_id,
"featureId": feature_id,
}
r = requests.post(f"{BASE}/api/rs/testcase", json=body, headers=HEADERS)
r.raise_for_status()
created = r.json()
if scenario_steps:
attach_scenario(created["id"], scenario_steps)
return createdScenario + nested steps
Allure TestOps' scenario shape supports nested steps (steps with sub-steps), uniquely among the five TCMs covered by the umbrella:
def attach_scenario(case_id, steps):
"""
steps: nested tree, e.g.,
[
{"keyword": "Given", "name": "User is logged in",
"steps": [
{"name": "Navigate to /login"},
{"name": "Enter credentials"},
{"name": "Submit form"},
]},
{"keyword": "When", "name": "User clicks Checkout"},
{"keyword": "Then", "name": "Order is placed",
"expectedResult": "Confirmation page shows order ID"},
]
"""
r = requests.post(
f"{BASE}/api/rs/testcase/{case_id}/scenario",
json={"steps": steps},
headers=HEADERS,
)
r.raise_for_status()Nested steps unlock BDD-style hierarchies and step-reuse.
Layers + features
# Discover layers in a project
layers = requests.get(f"{BASE}/api/rs/testlayer",
params={"projectId": project_id},
headers=HEADERS).json()
# [{"id": 1, "name": "API"}, {"id": 2, "name": "UI"}, ...]
# Discover features
features = requests.get(f"{BASE}/api/rs/feature",
params={"projectId": project_id},
headers=HEADERS).json()Layers + features are how Allure TestOps organises cases (in addition to suites).
Link to issue tracker (Jira / GitHub / Linear)
Allure TestOps integrates with multiple trackers; configure the integration in the UI, then link a case to an issue:
def link_to_issue(case_id, integration_id, issue_key):
r = requests.post(
f"{BASE}/api/rs/testcase/{case_id}/issue",
json={"integrationId": integration_id, "key": issue_key},
headers=HEADERS,
)
r.raise_for_status()Tags + custom fields
def set_tags(case_id, tag_names):
r = requests.post(
f"{BASE}/api/rs/testcase/{case_id}/tag",
json={"tags": [{"name": t} for t in tag_names]},
headers=HEADERS,
)
r.raise_for_status()Tags can be key-value: tags = ["severity:critical", "team:checkout"].
Get + list
case = requests.get(f"{BASE}/api/rs/testcase/{case_id}",
headers=HEADERS).json()
# Paginated list
def list_cases(project_id, page_size=100):
out = []
page = 0
while True:
r = requests.get(f"{BASE}/api/rs/testcase",
params={"projectId": project_id,
"page": page, "size": page_size},
headers=HEADERS)
r.raise_for_status()
data = r.json()
out.extend(data.get("content", []))
if data.get("last", False):
break
page += 1
return outResponse shape: {"content": [...], "page", "totalPages", "totalElements", "last"}.
Sync allure-results into manual cases
The unique Allure TestOps integration - link automated tests' results to manual case IDs:
# In your automated test (using allure-pytest, allure-junit, etc.):
import allure
@allure.testcase("ATO-1234") # links to Allure TestOps case ID 1234
def test_checkout_flow():
...When the CI run uploads allure-results to Allure TestOps, results auto-attach to the linked case. Coverage report shows which manual cases have automation backing.
Migration field map
| Source field | Allure TestOps field |
|---|---|
| Title | name |
| Description / objective | description |
| Precondition | precondition |
| Steps | scenario.steps (with optional nesting) |
| Type | layerId (UI / API / Component / Unit) |
| Component | featureId |
| Status | status (Active / Outdated / Archived) |
| Severity | tags or custom field |
| Priority | tags or custom field |
| Requirement traceability | /testcase/{id}/issue |
Response shapes + permalink
POST /testcase returns {"id", "name", "createdDate", "createdBy", ...}. Permalink:
url = f"{BASE}/project/{project_id}/test-cases/{case_id}"CI integration
Upload allure-results after every CI run; Allure TestOps ingests and links to manual cases by ID:
- name: Run tests with Allure
run: pytest --alluredir=allure-results
- name: Upload to Allure TestOps
env:
ATO_TOKEN: ${{ secrets.ALLURE_TESTOPS_TOKEN }}
ATO_BASE: ${{ vars.ALLURE_TESTOPS_BASE }}
ATO_PROJECT_ID: ${{ vars.ATO_PROJECT_ID }}
run: |
allurectl upload allure-results \
--endpoint $ATO_BASE \
--token $ATO_TOKEN \
--project-id $ATO_PROJECT_ID \
--launch-name "CI run ${GITHUB_RUN_NUMBER}"(allurectl is the official CLI; pip install allurectl.)
Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Flat steps for hierarchical workflows | Loses Allure TestOps' nested-step advantage | Use nested scenario.steps |
| No layer assignment | Coverage reports lose the UI/API/Unit split | Always set layerId |
| Manual cases not linked to automation | Coverage shows 0% even when tests exist | Use @allure.testcase("ATO-1234") decorator + sync |
| Tag-based severity / priority without convention | Cross-team queries break | Adopt prefix convention (severity: / priority:) |
| Single project for everything | Hard to navigate | Project per product / service |
Skipping allurectl for results upload | Manual upload error-prone | Always use allurectl upload |
Limitations
References
Qase.io - API specifics
View source (opens in new window)Qase.io - API specifics
Per-vendor reference for the tcm-case-management SKILL.md. Author and manage Qase.io test cases via the Public API v1 - create cases, organise into suites, attach structured steps, link to Jira / Linear / GitHub, manage shared steps, and bulk-import via JSON. Per developers.qase.io (Cloudflare-protected; cite by stable URL).
Qase.io is a modern lightweight TCM popular with smaller / agile teams that find TestRail / Xray heavy. It offers a clean Public API v1 (Token-based auth, REST + OpenAPI spec) and a simpler data model than its competitors.
Authentication
Qase Public API v1 uses Token header authentication:
export QASE_TOKEN="<api-token-from-qase.io-settings>"import requests, os
BASE = "https://api.qase.io/v1"
HEADERS = {
"Token": os.environ["QASE_TOKEN"],
"Content-Type": "application/json",
}Note the header is literally Token (not Authorization), which is unusual.
Create a case
POST /case/{project_code}:
def create_case(project_code, title, description=None, preconditions=None,
postconditions=None, steps=None, suite_id=None,
severity=4, priority=2, type=1, automation=0,
status=1, params=None):
"""
severity: 1=Blocker, 2=Critical, 3=Major, 4=Normal, 5=Minor, 6=Trivial
priority: 1=High, 2=Medium, 3=Low
type: 1=Functional, 2=Smoke, 3=Regression, 4=Security, etc. (per project enum)
automation: 0=Manual, 1=Automated, 2=To-be-automated
status: 0=Actual, 1=Draft, 2=Deprecated
steps: list of {"action": "...", "expected_result": "...", "data": "..."}
"""
body = {
"title": title,
"description": description,
"preconditions": preconditions,
"postconditions": postconditions,
"severity": severity,
"priority": priority,
"type": type,
"automation": automation,
"status": status,
"suite_id": suite_id,
"steps": steps or [],
"params": params or {},
}
r = requests.post(f"{BASE}/case/{project_code}",
json=body, headers=HEADERS)
r.raise_for_status()
return r.json()Field enums
| Field | Values |
|---|---|
severity | 1=Blocker, 2=Critical, 3=Major, 4=Normal, 5=Minor, 6=Trivial |
priority | 1=High, 2=Medium, 3=Low |
type | 1=Functional, 2=Smoke, 3=Regression, 4=Security, 5=Usability, 6=Performance, 7=Acceptance, 8=Compatibility (defaults; configurable) |
automation | 0=Manual, 1=Automated, 2=To-be-automated |
status | 0=Actual, 1=Draft, 2=Deprecated |
Note Qase priority is inverted (1=High here vs 1=Critical in IEEE 1044; map per severity-vs-priority-reference in the qa-defect-management plugin).
Steps
steps = [
{"action": "Navigate to /login",
"expected_result": "Login form rendered",
"data": ""},
{"action": "Enter alice@example.com + correct password",
"expected_result": "Submit button enabled",
"data": "alice@example.com / pw123"},
{"action": "Click Submit",
"expected_result": "Redirected to /dashboard within 2 s",
"data": ""},
]
case = create_case("AUTH", "Login redirects to dashboard",
steps=steps, suite_id=42,
severity=3, priority=1)Suites (test suite hierarchy)
def create_suite(project_code, title, description=None, parent_id=None):
body = {"title": title, "description": description,
"parent_id": parent_id}
r = requests.post(f"{BASE}/suite/{project_code}",
json=body, headers=HEADERS)
r.raise_for_status()
return r.json()Suites nest; create the hierarchy first, then place cases.
Shared steps
A unique Qase feature: define a step once, reuse across cases.
def create_shared_step(project_code, title, action, expected_result, data=None):
r = requests.post(
f"{BASE}/shared_step/{project_code}",
json={"title": title, "action": action,
"expected_result": expected_result, "data": data},
headers=HEADERS,
)
r.raise_for_status()
return r.json()
# Reference shared step in a case
steps = [
{"shared_step_hash": shared_step_hash},
{"action": "...", "expected_result": "..."},
]Update a case
PATCH /case/{project_code}/{id}:
def update_case(project_code, case_id, **fields):
r = requests.patch(f"{BASE}/case/{project_code}/{case_id}",
json=fields, headers=HEADERS)
r.raise_for_status()
return r.json()Get + list
case = requests.get(f"{BASE}/case/{project_code}/{case_id}",
headers=HEADERS).json()
def list_cases(project_code, limit=100):
cases = []
offset = 0
while True:
r = requests.get(f"{BASE}/case/{project_code}",
params={"limit": limit, "offset": offset},
headers=HEADERS)
r.raise_for_status()
data = r.json().get("result", {})
cases.extend(data.get("entities", []))
if len(data.get("entities", [])) < limit:
break
offset += limit
return casesEndpoint map
| Verb + path | Purpose |
|---|---|
POST /case/{project_code} | Create a case |
PATCH /case/{project_code}/{id} | Update a case |
GET /case/{project_code}/{id} | Get one case |
GET /case/{project_code} | List cases (paginate with limit / offset) |
POST /suite/{project_code} | Create a suite |
POST /shared_step/{project_code} | Create a shared step |
POST /result/{project_code} | Post run results (different surface; not case authoring) |
Bulk import via CSV
Wrap each row in try/except, throttle to ~60 req/min, and tally successes so one bad row does not abort the run:
import csv, time
rows = list(csv.DictReader(open("legacy.csv")))
created, failed = [], []
for row in rows:
steps = [
{"action": s, "expected_result": e, "data": d}
for s, e, d in zip(
row["steps"].split("|"),
row["expected"].split("|"),
(row.get("data") or "").split("|"),
)
]
try:
r = create_case(
project_code=row["project"],
title=row["title"],
preconditions=row.get("preconditions"),
steps=steps,
severity=int(row.get("severity", 4)),
priority=int(row.get("priority", 2)),
suite_id=int(row["suite_id"]),
)
created.append(r["result"]["id"])
except Exception as e:
failed.append((row["title"], str(e)))
time.sleep(1) # ~60 req/min; avoids 429sVerify: assert len(created) + len(failed) == len(rows) and that failed is empty before treating the import as done. If rows failed, inspect each (title, error), fix the source row (or back off on 429s), and re-run only the failed titles.
Link cases to issues
Qase supports linking via the tags / external_issues field (per project integration):
update_case(project_code, case_id, tags=["jira:ENG-123"])The platform supports first-class integrations with Jira / GitHub / Linear; configure in Qase UI.
Response shapes + permalink
CI integration
- name: Sync Qase cases
env:
QASE_TOKEN: ${{ secrets.QASE_TOKEN }}
run: python scripts/sync-qase.pyHave sync-qase.py exit non-zero when any case fails to sync, and after the run verify the repository case count matches the source count (compare list_cases length against the CSV row count); a mismatch means a silent partial sync - fail the job and re-run.
For result reporting after CI runs, use the qase-pytest / qase-cypress / qase-playwright reporters that post to /result/{project_code} (different surface from this case-management API).
Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
Authorization: Bearer <token> | Qase uses Token header, not Authorization | Set Token header directly |
| Hard-coded severity/priority integers | Easy to mix up the inverted Qase convention | Use named constants per the enum table |
| Inlining shared steps everywhere | Repeated maintenance, drift | Define shared steps; reference via shared_step_hash |
| Single suite for everything | Hard to navigate at scale | Suite per feature area |
Skipping automation field | Coverage reports incomplete | Set automation field per case |
| Bulk-create without rate throttling | 429s on >100 cases / min | Throttle to ~60 req / min |
Limitations
References
TestRail - API specifics
View source (opens in new window)TestRail - API specifics
Per-vendor reference for the tcm-case-management SKILL.md. The spine works the full workflow (auth, create, discover, list, bulk import, CI sync) against TestRail; this file carries the remaining TestRail API v2 specifics. Per the TestRail API docs (support.testrail.com/hc/en-us/articles/7077871398036-Cases; Cloudflare-protected, cite by stable URL).
Hierarchy
Project -> suite -> section (nests via parent_id) -> case. Keep section trees at 3 levels or fewer; the UI display is shallow.
api("add_suite/123", method="POST", body={"name": "Authentication"})
api("add_section/123", method="POST",
body={"suite_id": 7, "name": "Login flows", "parent_id": None})Templates
template_id: 1=Steps, 2=Text, 3=Exploratory. Use the Steps template (custom_steps_separated) for hand-executed cases - per-step results are unavailable on the Text template (custom_steps).
Custom-field discovery
Each tenant defines its own custom fields. Discover IDs once:
fields = api("get_case_fields")
for f in fields:
print(f["system_name"], f["label"], f["type_id"])
# custom_preconds Preconditions 3
# custom_severity Severity 6
# custom_automation_type Automation Type 6type_id values per TestRail docs: 1=String, 2=Integer, 3=Text, 4=URL, 5=Checkbox, 6=Dropdown, 7=User, 8=Date, 9=Milestone, 10=Steps, 11=Multi-select.
Types + priorities
types = api("get_case_types") # -> [{"id", "name", "is_default"}, ...]
prios = api("get_priorities") # -> [{"id", "name", "priority", ...}, ...]Build name-to-id maps from these; never hard-code the integers (they differ per project). Default priority enum: 1=Low, 2=Medium, 3=High, 4=Critical.
Traceability
refs is a comma-separated free-text field of requirement IDs (e.g., "REQ-123,REQ-124"). TestRail doesn't validate that referenced IDs exist in Jira / Linear / etc.; pair with traceability-matrix reconciliation.
Response shapes + permalink
TestRail-specific anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
Steps in custom_steps (Text template) | Per-step results unavailable | Use custom_steps_separated (Steps template) |
| Polling for case existence on every CI run | Rate-limited | Cache case-ID-by-title within the CI run |
Limitations
References
Xray (Jira) - API specifics
View source (opens in new window)Xray (Jira) - API specifics
Per-vendor reference for the tcm-case-management SKILL.md. Author and manage Xray test cases (Jira issues with Test issue type) via the GraphQL + REST APIs - create tests, attach steps, link preconditions, set testType (Manual / Cucumber / Generic), associate with requirements, bulk import via JSON. Per docs.getxray.app/display/XRAYCLOUD/REST+API (Cloudflare-protected; cite by stable URL).
In Xray, tests are first-class Jira issues with issuetype: Test. Xray augments them with test-specific data (steps, type, preconditions) accessible via either the REST v2 API or a GraphQL endpoint.
Authentication - OAuth client credentials
Xray Cloud requires OAuth client credentials (different from Jira auth):
# From Xray Global Settings → API Keys, create a client
export XRAY_CLIENT_ID="..."
export XRAY_CLIENT_SECRET="..."
export JIRA_BASE="https://your-tenant.atlassian.net"import requests, os
# Exchange client credentials for a JWT
def get_token():
r = requests.post("https://xray.cloud.getxray.app/api/v2/authenticate",
json={"client_id": os.environ["XRAY_CLIENT_ID"],
"client_secret": os.environ["XRAY_CLIENT_SECRET"]})
r.raise_for_status()
return r.text.strip('"') # response is a quoted string
XRAY_BASE = "https://xray.cloud.getxray.app/api/v2"
GRAPHQL = "https://xray.cloud.getxray.app/api/v2/graphql"
def xray_headers(token):
return {"Authorization": f"Bearer {token}", "Content-Type": "application/json"}JWT expires in 24 hours; cache + refresh.
Create a Manual test (GraphQL)
CREATE_TEST_MUTATION = """
mutation CreateTest($input: CreateTestInput!) {
createTest(input: $input) {
test { issueId jira(fields: ["key", "summary"]) }
warnings
}
}
"""
def create_manual_test(token, project_key, summary, steps,
preconditions_issue_id=None):
"""
steps: list of {"action": "...", "data": "...", "result": "..."}
"""
variables = {
"input": {
"testType": {"name": "Manual"},
"jira": {
"fields": {
"summary": summary,
"project": {"key": project_key},
"issuetype": {"name": "Test"},
}
},
"steps": steps,
"preconditionIssueIds": [preconditions_issue_id] if preconditions_issue_id else [],
}
}
r = requests.post(GRAPHQL,
json={"query": CREATE_TEST_MUTATION, "variables": variables},
headers=xray_headers(token))
r.raise_for_status()
return r.json()Test types
Per Xray docs, the three canonical types:
| testType | Steps storage | When to use |
|---|---|---|
Manual | Structured step list with action / data / result | Default; hand-executed cases |
Cucumber | Gherkin scenario text | BDD-driven tests (composes with the qa-bdd plugin) |
Generic | Single free-text definition | Automated tests where the script is the spec |
Bulk import (REST)
POST /api/v2/import/test/bulk accepts JSON arrays of tests:
def bulk_import(token, project_key, tests):
"""
tests: list of {"testtype": "Manual", "fields": {...}, "steps": [...]}
"""
r = requests.post(
f"{XRAY_BASE}/import/test/bulk",
json={"projectKey": project_key, "tests": tests},
headers=xray_headers(token),
)
r.raise_for_status()
return r.json() # returns job id + status poll URLBulk import returns a job ID - poll /api/v2/import/test/bulk/{jobId}/status until complete. Expect ~50 ms per test.
Cucumber feature import
# Upload .feature file via /api/v2/import/feature
with open("checkout.feature", "rb") as f:
r = requests.post(
f"{XRAY_BASE}/import/feature",
files={"file": f},
params={"projectKey": "ENG"},
headers={"Authorization": f"Bearer {token}"},
)One scenario = one Test issue; one feature = one Pre-Condition (if Background present).
Link to a requirement
Requirements in Xray are arbitrary Jira issues marked with a Requirement issue type or in a configured project. Link tests to requirements via Jira issue links:
def link_test_to_requirement(jira_base, jira_email, jira_token,
test_key, req_key, link_type="Tests"):
r = requests.post(
f"{jira_base}/rest/api/3/issueLink",
json={
"type": {"name": link_type},
"inwardIssue": {"key": req_key},
"outwardIssue": {"key": test_key},
},
auth=(jira_email, jira_token),
)
r.raise_for_status()The Tests/TestedBy link type is Xray-recommended; configure per project.
Preconditions as separate issues
Xray stores preconditions as Jira issues with type Pre-Condition. Reuse a precondition across tests by creating the Pre-Condition issue via the Jira REST API v3, then attaching via preconditionIssueIds: [...] in the create-test mutation.
Migration field map
| Source field | Xray field |
|---|---|
| Title | Jira summary |
| Steps | steps array |
| Preconditions text | Linked Pre-Condition issue |
| Type | testType.name (Manual / Cucumber / Generic) |
| Refs | Jira issue links to requirement issues |
Updating a test
GraphQL updateTest mutation (signature similar to createTest). For step changes:
UPDATE_STEPS_MUTATION = """
mutation UpdateTestSteps($issueId: String!, $steps: [UpdateStepInput!]!) {
updateTestSteps(issueId: $issueId, steps: $steps) {
test { issueId }
warnings
}
}
"""Response shapes + permalink
createTest response includes test.issueId (internal Jira ID) and test.jira (the requested Jira fields, e.g., key). Permalink:
url = f"{os.environ['JIRA_BASE']}/browse/{jira_key}"CI integration
Sync Cucumber .feature files on every PR merge:
- name: Sync feature files to Xray
env:
XRAY_CLIENT_ID: ${{ secrets.XRAY_CLIENT_ID }}
XRAY_CLIENT_SECRET: ${{ secrets.XRAY_CLIENT_SECRET }}
run: |
TOKEN=$(curl -s -X POST -H "Content-Type: application/json" \
-d '{"client_id":"'$XRAY_CLIENT_ID'","client_secret":"'$XRAY_CLIENT_SECRET'"}' \
https://xray.cloud.getxray.app/api/v2/authenticate | tr -d '"')
for f in features/*.feature; do
curl -X POST -H "Authorization: Bearer $TOKEN" \
-F "file=@$f" \
"https://xray.cloud.getxray.app/api/v2/import/feature?projectKey=ENG"
doneAnti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Storing preconditions in step 1 of every test | Duplication; preconditions evolve and tests drift | Use Pre-Condition issues + preconditionIssueIds |
| Manual tests as Generic (single text blob) | Per-step results lost | Use testType: Manual with structured steps |
| Skipping the OAuth client setup | Xray Cloud rejects Jira-auth-only requests | Always use Xray's OAuth credentials |
| Hard-coding the JWT | Expires in 24 h | Cache + refresh on 401 |
| Creating tests via Jira REST without Xray-specific fields | Tests look right in Jira but Xray's data layer (steps, type) is empty | Use GraphQL createTest or REST /import/test/bulk |
| Polling job status without backoff | Rate-limited | Exponential backoff |
| One feature = many tests imported individually | Slow | Import via /import/feature (single call) |
Limitations
References
Zephyr Scale (Jira) - API specifics
View source (opens in new window)Zephyr Scale (Jira) - API specifics
Per-vendor reference for the tcm-case-management SKILL.md. Author and manage Zephyr Scale Cloud test cases via the REST API v2 - create tests, attach steps, link to Jira issues, organise into folders. Per smartbear.com/test-management/zephyr-scale (Cloudflare-protected; cite by stable URL).
Zephyr Scale Cloud (formerly Adaptavist TM4J, now SmartBear) exposes a REST API v2 with Bearer-token authentication.
Authentication
Zephyr Scale Cloud uses Bearer tokens generated from the Zephyr Scale UI (API access tokens, per-user):
export ZS_TOKEN="<bearer-token-from-zephyr-scale-ui>"import requests, os
BASE = "https://api.zephyrscale.smartbear.com/v2"
HEADERS = {
"Authorization": f"Bearer {os.environ['ZS_TOKEN']}",
"Content-Type": "application/json",
}Create a test case
POST /testcases:
def create_test_case(project_key, name, objective=None, precondition=None,
steps=None, owner=None, folder_id=None,
labels=None, components=None, priority="Normal",
status="Approved"):
"""
steps: list of {"inline": {"description": "...", "expectedResult": "..."}}
priority: Highest / High / Normal / Low / Lowest (per project config)
status: Approved / Draft / Deprecated
"""
body = {
"projectKey": project_key,
"name": name,
"objective": objective,
"precondition": precondition,
"ownerId": owner,
"folderId": folder_id,
"labels": labels or [],
"componentId": components,
"priorityName": priority,
"statusName": status,
}
r = requests.post(f"{BASE}/testcases", json=body, headers=HEADERS)
r.raise_for_status()
created = r.json()
if steps:
attach_steps(created["key"], steps)
return createdTest script + steps
Steps are managed via a separate testScript endpoint:
def attach_steps(test_case_key, steps):
"""
steps: list of {"inline": {"description": str, "expectedResult": str,
"testData": str}}
"""
body = {"mode": "OVERWRITE", "items": [{"inline": s} for s in steps]}
r = requests.post(f"{BASE}/testcases/{test_case_key}/teststeps",
json=body, headers=HEADERS)
r.raise_for_status()
return r.json()
steps = [
{"description": "Navigate to /login",
"expectedResult": "Login form rendered", "testData": ""},
{"description": "Enter credentials and click Submit",
"expectedResult": "Redirected to /dashboard",
"testData": "alice@example.com / pw123"},
]
attach_steps("PROJ-T123", steps)mode: OVERWRITE replaces the existing step list; APPEND adds to it.
Folders
def create_folder(project_key, name, folder_type="TEST_CASE", parent_id=None):
r = requests.post(f"{BASE}/folders", json={
"projectKey": project_key, "name": name,
"folderType": folder_type, # TEST_CASE / TEST_PLAN / TEST_CYCLE
"parentId": parent_id,
}, headers=HEADERS)
r.raise_for_status()
return r.json()Folders nest; create the hierarchy first, then place cases.
Linking to Jira issues
def link_to_jira(test_case_key, issue_key):
r = requests.post(f"{BASE}/testcases/{test_case_key}/links/issues",
json={"issueId": resolve_jira_issue_id(issue_key)},
headers=HEADERS)
r.raise_for_status()Requires the Jira REST API to resolve issue key -> issue ID separately. Trace back from cases to requirements via the linked-issues endpoint.
Get + list
case = requests.get(f"{BASE}/testcases/PROJ-T123", headers=HEADERS).json()
# Paginated list
def list_cases(project_key, max_results=100):
cases = []
start_at = 0
while True:
r = requests.get(f"{BASE}/testcases",
params={"projectKey": project_key,
"startAt": start_at,
"maxResults": max_results},
headers=HEADERS)
r.raise_for_status()
data = r.json()
cases.extend(data["values"])
if data["isLast"]:
break
start_at += max_results
return casesBulk import via CSV to JSON
Wrap each row in try/except, throttle to ~60 req/min (the tenant limit), and tally successes so one failing row does not abort the batch:
import csv, time
rows = list(csv.DictReader(open("legacy.csv")))
created, failed = [], []
for row in rows:
try:
case = create_test_case(
project_key=row["project"],
name=row["title"],
objective=row.get("objective"),
precondition=row.get("precondition"),
priority=row.get("priority", "Normal"),
labels=row.get("labels", "").split(",") if row.get("labels") else None,
)
steps = [
{"description": s, "expectedResult": e, "testData": d}
for s, e, d in zip(
row["steps"].split("|"),
row["expected"].split("|"),
(row.get("data") or "").split("|"),
)
]
attach_steps(case["key"], steps)
created.append(case["key"])
except Exception as e:
failed.append((row["title"], str(e)))
time.sleep(1) # ~60 req/min; a 429 means back off further
print(f"created {len(created)}, failed {len(failed)}")Verify: assert len(created) + len(failed) == len(rows) and that failed is empty before treating the import as complete. On any 429, increase the delay; on other errors, fix the source row and re-run the failed titles.
Migration field map
| Source field | Zephyr field |
|---|---|
| Title | name |
| Objective | objective |
| Preconditions | precondition |
| Steps | testScript items |
| Owner | ownerId (Jira user ID) |
| Priority | priorityName (project enum) |
| Status | statusName |
| Labels | labels[] |
| Component | componentId (Jira component) |
| Requirement traceability | /links/issues |
Response shapes + permalink
CI integration
- name: Sync to Zephyr Scale
env:
ZS_TOKEN: ${{ secrets.ZEPHYR_SCALE_TOKEN }}
run: python scripts/sync-zephyr-scale.py specs/Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Inline steps in test-case body (objective field) | Per-step pass/fail unavailable | Use /teststeps testScript endpoint |
| Hard-coded priority names | Project-specific enum may differ | Discover via project config endpoint |
| Flat folder hierarchy | Hundreds of cases unfindable | Create folders matching feature areas |
Mixing componentId and labels[] randomly | Cross-team queries break | Component for ownership; labels for cross-cutting concerns |
| Bulk-create without rate throttling | 429s on >100 cases / min | Limit to ~60 req / min |
| Storing the Bearer token in repo | Token leak | Environment variable / secret store |
Limitations
References
Related skills
test-case-anatomy-reference
Pure-reference catalog of test-case anatomy and review quality - what fields a well-formed test case must have, what each field means, and how to score the content once the fields are filled. Enumerates the ISO/IEC/IEEE 29119-3:2021 test-case template fields (identifier, objective, preconditions, inputs, steps, expected results, postconditions, environment, traceability) and the ISTQB CTAL-TM specification-technique-driven additions (equivalence partition, boundary value, decision table, state transition), maps the canonical anatomy to five tracker-specific schemas (TestRail, Xray, Zephyr Scale, Allure TestOps, Qase), and carries the review rubric: six per-case quality axes plus six set-level axes with PASS / WEAK / FAIL verdicts derived without averaging, each threshold marked standard-backed or practitioner convention. Use as the authoritative source when authoring a case template, reviewing a batch of test cases for quality, or migrating between tools.
traceability-matrix-builder
Build-an-X workflow that produces a requirements-to-tests traceability matrix from a TCM case repository + a requirements source (Jira / Linear / GitHub Issues). Walks the author through (1) extracting requirements with stable IDs, (2) extracting cases + their refs, (3) computing coverage (which requirements have at least one test, which tests verify which requirements, orphaned cases / orphaned requirements), (4) emitting a CSV / Markdown / HTML matrix, and (5) producing an executive summary (X% requirement coverage, Y orphans, Z over-tested). Use for test coverage audits, finding requirements-coverage gaps, sprint-end coverage reviews, compliance documentation, and traceability in regulated industries.