os-matrix-runner

Configures a CI matrix that runs tests across operating systems (Linux / macOS / Windows) and runtime versions (Node 18/20/22; Python 3.10/3.11/3.12; Java 17/21; .NET 6/8). Wires GitHub Actions matrix syntax, addresses OS-specific quirks (path separators, line endings, file permissions). Use when the product ships across OS / runtime combinations and the team needs continuous cross-platform coverage.

Install with skills.sh (any agent)

npx skills add testland/qa --skill os-matrix-runner

View source

os-matrix-runner

When to use

The product runs on multiple OSes (CLI tools, libraries, desktop apps).
The team supports multiple runtime versions.
A bug report says "works on Linux, broken on Windows."

For browsers specifically, see browser-matrix-runner.

Step 1 - Define the OS matrix

# .github/workflows/os-matrix.yml
strategy:
  fail-fast: false
  matrix:
    os: [ubuntu-latest, macos-latest, windows-latest]

GitHub Actions provides:

Runner	Use
`ubuntu-latest`	Default; cheapest; most CI runs here.
`ubuntu-22.04`	Pin specific Ubuntu LTS.
`macos-latest`	macOS; needed for iOS / Safari testing.
`macos-15`	Pin macOS version.
`windows-latest`	Windows; tests Windows-specific paths.
`windows-2022`	Pin Windows version.

Step 2 - Define the runtime matrix

Per language:

# Node.js
strategy:
  matrix:
    node: [18, 20, 22]
    os: [ubuntu-latest, macos-latest, windows-latest]

# Python
strategy:
  matrix:
    python: ['3.10', '3.11', '3.12']
    os: [ubuntu-latest, macos-latest, windows-latest]

# Java
strategy:
  matrix:
    java: [17, 21]
    os: [ubuntu-latest, macos-latest, windows-latest]

# .NET
strategy:
  matrix:
    dotnet: ['6.0.x', '8.0.x']
    os: [ubuntu-latest, macos-latest, windows-latest]

The full cross-product is 3 OSes × 3 runtimes = 9 jobs. For larger matrices, use include + exclude to skip uninteresting combinations.

Step 3 - Address OS-specific quirks

Path separators

// Bad - hardcoded /
const configPath = projectRoot + '/config/app.json';

// Good - path.join
const path = require('node:path');
const configPath = path.join(projectRoot, 'config', 'app.json');

Line endings

# .gitattributes
*.sh text eol=lf
*.bat text eol=crlf
*.json text

Without .gitattributes, Windows users may commit CRLF; tests that compare output strings break.

Case sensitivity

// On Linux: import './Utils' fails if file is './utils'
// On macOS / Windows (default): both work

// Always match file case exactly:
import { foo } from './utils';   // matches utils.js

Shell

- name: Run script (cross-platform)
  shell: bash
  run: ./scripts/setup.sh

shell: bash works on Linux + macOS + Windows (via Git Bash on Windows runners).

Step 4 - Per-OS conditional steps

When OS-specific setup is needed:

- name: Install Linux deps
  if: runner.os == 'Linux'
  run: sudo apt-get install -y libssl-dev

- name: Install macOS deps
  if: runner.os == 'macOS'
  run: brew install openssl

- name: Install Windows deps
  if: runner.os == 'Windows'
  run: choco install openssl

Step 5 - Aggregate per-OS results

## OS / runtime matrix results - `<sha>`

| OS        | Runtime  | Tests | Pass | Fail | Time |
|-----------|----------|------:|-----:|-----:|-----:|
| Linux     | Node 22  |  142  |  142 |    0 | 2m   |
| Linux     | Node 20  |  142  |  142 |    0 | 2m   |
| Linux     | Node 18  |  142  |  140 |    2 | 2m   |  ← Node 18 incompat
| macOS     | Node 22  |  142  |  141 |    1 | 3m   |  ← macOS path issue
| macOS     | Node 20  |  142  |  141 |    1 | 3m   |
| Windows   | Node 22  |  142  |  140 |    2 | 4m   |  ← Windows path issue
| ...

Step 6 - Per-OS conditional tests

Some tests are OS-specific:

// jest.config.js
module.exports = {
  testPathIgnorePatterns: process.platform === 'win32'
    ? ['unix-only.test.js']
    : ['windows-only.test.js'],
};

Or via test framework conditionals:

test.skipIf(process.platform === 'win32')('uses fork()', () => {
  // POSIX-specific test
});

Step 7 - Cost management

Matrix size grows multiplicatively. Manage cost:

Tier	Cadence	Matrix size
Per-PR (smoke)	Per push	1 × 1 = 1 job (Linux + latest runtime).
Per-merge to main	Per merge	3 × 1 = 3 jobs (3 OSes + latest runtime).
Nightly	Cron	3 × 3 = 9 jobs (full matrix).
Pre-release	Tag	Full matrix + extra exotic combinations.

The "smoke matrix" per-PR keeps CI cheap; the full matrix runs less frequently.

Anti-patterns

Anti-pattern	Why it fails	Fix
Hardcoded `/` path separators	Breaks on Windows.	`path.join` (Step 3).
`fail-fast: true` on the matrix	One OS fails; can't see others.	`fail-fast: false`.
Same matrix every commit	CI cost explodes; team disables.	Tiered cadence (Step 7).
Per-OS code in production	If/else by OS; high maintenance.	Cross-platform abstractions in production; OS-specific code in glue layer only.
Skipping `.gitattributes`	CRLF / LF mixing; tests fail mysteriously.	Always set (Step 3).

Limitations

GitHub Actions runner cost. Windows + macOS runners are more expensive than Linux; matrix design tradeoff.
Per-OS bugs may surface only at runtime. Static analysis catches some; integration tests are the safety net.
macOS-specific issues often only reproduce on real macOS; Linux CI doesn't catch them.
Older OS versions rarely available on hosted runners; need self-hosted for legacy.

References

GitHub Actions runners docs at docs.github.com/en/actions/using-github-hosted-runners.
browser-matrix-runner - sibling: browser-specific.
compatibility-budget - conventions for matrix sizing.

Related skills

browser-matrix-runner

Configures a CI matrix that runs the smoke / regression suite across multiple browsers per Playwright's three-engine support (Chromium, Firefox, WebKit / Safari) plus branded variants (chrome, msedge channels). Wires GitHub Actions / GitLab CI matrix syntax, captures per-browser screenshots, and aggregates per-browser pass/fail. Use when the product targets multiple browsers and the team wants automated cross-browser testing or browser-compatibility regression - e.g. a 'works in Chrome, broken in Safari' bug that needs Chrome / Firefox / Safari coverage.

browser-matrix-strategy-reference

Pure-reference for designing and reviewing a browser / OS / device test matrix from traffic data - the T1/T2/T3 tier-membership heuristics (T1 >=5% traffic, T2 1-5% or statutory, T3 <1% with customer demand), the traffic-share sources (own analytics, StatCounter, MDN browser-compat-data), a worked matrix template with tier-change log, and how to justify dropping a legacy browser (IE11, old iOS Safari). Use when designing an initial matrix, running a quarterly re-tier review, or making the case to drop a browser. This is the WHAT-to-test strategy reference - to execute the matrix use the runners browser-matrix-runner (bundled engines) or selenium-grid-4-runner (self-hosted); to cap and publish committed support tiers use compatibility-budget.

compatibility-budget

Pure-reference for deciding how large a compatibility matrix a team can afford and for publishing that commitment - defines tier-1 (must work; per-PR) vs tier-2 (must work; nightly) vs tier-3 (should work; pre-release) vs unsupported, with example budgets per product type (web / desktop / mobile / library), the matrix-size cost / coverage trade-off, and 'what we support' external templates. Use when a team must cap how many browser / OS / runtime combos it commits to, or must publish a support policy. This is the BUDGET and support-statement gate - for the traffic-share analysis that picks WHICH specific browsers belong in each tier use browser-matrix-strategy-reference; to execute the resulting matrix use the runners browser-matrix-runner (bundled engines) or selenium-grid-4-runner (self-hosted).

selenium-grid-4-runner

Author and operate Selenium Grid 4 - self-hosted distributed WebDriver. Covers the six-component architecture (Router / Distributor / Session Map / Event Bus / New Session Queue / Node), standalone vs hub-and-node modes, the Docker-image stack (selenium/standalone-chrome, selenium/hub, selenium/node-chrome), node registration, session-queue tuning, and observability. Use for self-hosted cross-browser testing when data residency or cost-control require an on-prem grid. This is the self-hosted execution RUNNER - for the zero-infra alternative use browser-matrix-runner (Playwright bundled engines); for managed cloud grids use browserstack-automate, saucelabs-automate, or lambdatest-automate; to decide WHICH browsers and tiers to run use browser-matrix-strategy-reference.