Testland
Browse all skills & agents

playwright-testing

Authors and remediates Playwright E2E tests across Chromium, Firefox, WebKit - `npm init playwright@latest` scaffolding, `playwright.config.ts` browser projects, accessibility-first locators (`getByRole`/`getByLabelText`) to replace brittle CSS selectors, web-first assertions to eliminate `waitForTimeout` flakiness, Page Object pattern, trace viewer debugging, sharded parallel execution with merged HTML reporting, mobile-web emulation via the `devices` catalog (viewport / DPR / touch per-device projects), the cross-browser matrix with branded channels (chrome / msedge) in references/browser-matrix.md, and GitHub Actions CI integration. Use for new test authoring, flakiness remediation, mobile-breakpoint regression, cross-browser matrix setup, and CI setup; for reviewing codegen output specifically, use a dedicated codegen-review pass.

Install with skills.sh (any agent)

npx skills add testland/qa --skill playwright-testing
View source

playwright-testing

Overview

Per pw-intro (opens in new window):

"Playwright Test is an end-to-end test framework for modern web apps. It bundles test runner, assertions, isolation, parallelization and rich tooling."

"The framework supports Chromium, WebKit, and Firefox across Windows, Linux, and macOS." (pw-intro (opens in new window))

When to use

  • New web E2E project; pick Playwright as the modern default.
  • Cross-browser coverage matters (see references/browser-matrix.md) - Playwright's three-engine support is the differentiator.
  • Migration from Selenium / WebDriver-based stacks (see selenium-testing).

Step 1 - Scaffold

Per pw-intro (opens in new window):

npm init playwright@latest

The init prompts choose TypeScript/JavaScript, tests folder, GitHub Actions CI, and browser binaries.

What lands: playwright.config.ts + tests/example.spec.ts + package.json updates.

Step 2 - Author tests with accessibility-first locators

import { test, expect } from '@playwright/test';

test('checkout flow happy path', async ({ page }) => {
  await page.goto('/');

  await page.getByRole('link', { name: /sign in/i }).click();
  await page.getByLabel('Email').fill('user@example.com');
  await page.getByLabel('Password').fill('test-password');
  await page.getByRole('button', { name: /sign in/i }).click();

  await expect(page.getByRole('heading', { name: /welcome/i })).toBeVisible();

  await page.getByRole('link', { name: /shop/i }).click();
  await page.getByRole('link', { name: /BOOK-001/i }).click();
  await page.getByRole('button', { name: /add to cart/i }).click();
  await expect(page.getByTestId('cart-count')).toHaveText('1');
});

Prefer getByRole / getByLabelText / getByText over CSS class / XPath. Web-first assertions (await expect(...)) auto-wait within the test timeout.

Step 3 - Page Object pattern

// tests/page-objects/CheckoutPage.ts
import { Page, expect } from '@playwright/test';

export class CheckoutPage {
  constructor(private page: Page) {}

  async signIn(email: string, password: string) {
    await this.page.getByLabel('Email').fill(email);
    await this.page.getByLabel('Password').fill(password);
    await this.page.getByRole('button', { name: /sign in/i }).click();
  }

  async addToCart(sku: string) {
    await this.page.getByRole('link', { name: new RegExp(sku, 'i') }).click();
    await this.page.getByRole('button', { name: /add to cart/i }).click();
  }

  async expectConfirmation() {
    await expect(this.page.getByRole('heading', { name: /order confirmed/i })).toBeVisible();
  }
}

Tests import the Page Object:

test('checkout', async ({ page }) => {
  const checkout = new CheckoutPage(page);
  await checkout.signIn('user@example.com', 'pwd');
  await checkout.addToCart('BOOK-001');
  await checkout.expectConfirmation();
});

Step 4 - Configuration

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 4 : undefined,
  reporter: [
    ['html'],
    ['junit', { outputFile: 'reports/junit.xml' }],
  ],
  use: {
    baseURL: process.env.BASE_URL || 'http://localhost:3000',
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
  },
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
    { name: 'firefox',  use: { ...devices['Desktop Firefox'] } },
    { name: 'webkit',   use: { ...devices['Desktop Safari'] } },
  ],
});

trace: 'on-first-retry' captures rich debug info (DOM snapshots, network, console) only when needed - avoids storage cost on passing runs.

The same devices catalog covers mobile-web emulation: add projects spreading devices['iPhone 15'] / devices['Pixel 7'] for mobile-breakpoint regression (viewport, DPR, user agent, .tap() touch synthesis, per-device snapshots, CI matrix) - see references/mobile-emulation.md. For the full cross-browser matrix - branded chrome / msedge channel projects, the per-browser CI matrix with fail-fast: false, and the engine-vs-channel decision table - see references/browser-matrix.md.

Step 5 - Run

Per pw-intro (opens in new window):

# All tests, all browsers, headless, parallel
npx playwright test

# Specific browser
npx playwright test --project=chromium

# Headed (see the browser)
npx playwright test --headed

# UI Mode (watch + debug)
npx playwright test --ui

# Single test file
npx playwright test tests/checkout.spec.ts

# Single test by name
npx playwright test -g "checkout flow"

Step 6 - Trace viewer

When a test fails, the trace contains everything needed to debug:

# After a failure
npx playwright show-trace test-results/<...>/trace.zip

The viewer shows:

  • DOM snapshot at each action.
  • Network requests + responses.
  • Console output.
  • Screenshots.
  • Source code with highlighted line.

Step 7 - Sharded execution

For large suites:

# Run 4 of 4 shards (one per CI job)
npx playwright test --shard=1/4
npx playwright test --shard=2/4
# ... etc.
# CI matrix
strategy:
  matrix:
    shard: [1/4, 2/4, 3/4, 4/4]
runs-on: ubuntu-latest
steps:
  - run: npx playwright test --shard=${{ matrix.shard }}

Step 8 - CI integration

# .github/workflows/playwright.yml
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v4
        with: { node-version: '22' }
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: playwright-report
          path: playwright-report/

Step 9 - Reporting

Per pw-intro (opens in new window): "The HTML Reporter provides a filterable dashboard showing results by browser, status (passed/failed/skipped), and flaky tests."

npx playwright show-report

For programmatic / CI consumption, the JUnit reporter (Step 4) feeds junit-xml-analysis (in the qa-test-reporting plugin).

Anti-patterns

Anti-patternWhy it failsFix
CSS-class / XPath selectorsBrittle to DOM changes.getByRole / getByLabelText.
page.waitForTimeout(2000)Flaky on slow CI; slow on fast.Web-first assertions (auto-wait).
One mega-test that spans multiple flowsFailure mid-test obscures cause.Per-flow tests; share setup via Page Objects.
Skipping --with-deps in CILinux runner missing browser deps.Always --with-deps (Step 8).
trace: 'on' alwaysWasted storage on passing runs.trace: 'on-first-retry' (Step 4).

Limitations

  • No real Safari. WebKit ≠ Safari (per references/browser-matrix.md); iOS Safari needs real-device testing.
  • Per-test runtime ~2-30s. E2E expensive vs unit tests; use pyramid balance per test-pyramid-balancer (in the qa-process plugin).
  • Browser version drift. Playwright N+1 ahead of stable; some tests pass in Playwright but fail in shipped Chrome.
  • Migrating from Puppeteer: Playwright supersedes it for testing; the APIs are close enough that scripts port near-mechanically.

References

Cross-browser matrix and branded channels

View source (opens in new window)

Cross-browser matrix and branded channels

Reference for playwright-testing: configure a CI matrix that runs the smoke / regression suite across Playwright's three engines (Chromium, Firefox, WebKit / Safari) plus branded variants (chrome, msedge channels), aggregate per-browser pass/fail, and choose engine vs branded channel per tier. Consult for a "works in Chrome, broken in Safari" bug that needs cross-browser coverage.

Overview

Per pw-browsers (opens in new window):

Playwright supports three browser engines:

  1. Chromium - "Open source builds used by default. Playwright versions support Chromium N+1 before branded browsers release it."
  2. Firefox - "Playwright's Firefox version matches the recent Firefox Stable build."
  3. WebKit - "Derived from latest WebKit main branch sources, often before Safari incorporation."

Plus branded variants (pw-browsers (opens in new window)):

"Playwright can operate against Google Chrome and Microsoft Edge when installed on the machine. Available channels include: chrome, chrome-beta, chrome-dev, chrome-canary, msedge, msedge-beta, msedge-dev, msedge-canary."

When to use

  • The product runs on multiple browsers and the team wants automated cross-browser coverage.
  • A bug report says "works in Chrome, broken in Safari."
  • Pre-release: scheduled cross-browser smoke.

Step 1 - Install browsers

Per pw-browsers (opens in new window):

# All Playwright-bundled browsers
npx playwright install

# Specific browsers
npx playwright install chromium firefox webkit

# With system dependencies (CI-friendly)
npx playwright install --with-deps

Branded:

npx playwright install msedge   # only on Windows / macOS
npx playwright install chrome

Step 2 - Configure projects

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
    { name: 'firefox',  use: { ...devices['Desktop Firefox'] } },
    { name: 'webkit',   use: { ...devices['Desktop Safari'] } },
    { name: 'edge',     use: { ...devices['Desktop Edge'], channel: 'msedge' } },
    { name: 'chrome',   use: { ...devices['Desktop Chrome'], channel: 'chrome' } },
  ],
});

Step 3 - Run

# All browsers
npx playwright test

# Specific browser
npx playwright test --project=firefox

# Multiple
npx playwright test --project=chromium --project=webkit

Step 4 - CI matrix

# .github/workflows/cross-browser.yml
jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        browser: [chromium, firefox, webkit]
        include:
          - browser: edge
            os: windows-latest    # msedge on Windows
          - browser: chrome
            os: ubuntu-latest
    runs-on: ${{ matrix.os || 'ubuntu-latest' }}
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v4
        with: { node-version: '20' }
      - run: npm ci
      - run: npx playwright install --with-deps ${{ matrix.browser }}
      - run: npx playwright test --project=${{ matrix.browser }}
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: playwright-${{ matrix.browser }}
          path: playwright-report/

fail-fast: false ensures Chromium failure doesn't cancel WebKit / Firefox.

Verify each browser leg: assert npx playwright install --with-deps exited 0 and a playwright-report/ artifact was produced before counting the browser as passed. If install fails, the cause is almost always a missing system dependency - re-run with --with-deps (wired above) and surface the missing package from the installer's error output, then re-run the leg.

Step 5 - Per-browser failure analysis

When per-browser failures appear:

## Cross-browser results - `<sha>`

| Browser   | Tests | Pass | Fail | Time   |
|-----------|------:|-----:|-----:|-------:|
| Chromium  |   42  |   42 |    0 |  120s  |
| Firefox   |   42  |   42 |    0 |  135s  |
| WebKit    |   42  |   38 |    4 |  140s  |   ← WebKit-specific issues
| Edge      |   42  |   42 |    0 |  125s  |
| Chrome    |   42  |   42 |    0 |  120s  |

### WebKit-only failures

| Test                                | Symptom                                   |
|-------------------------------------|-------------------------------------------|
| `checkout.spec.ts > apply promo`     | `100vh` calculated wrong on iOS Safari   |
| `cart.spec.ts > add item`            | `IntersectionObserver` callback timing diff |
| ...                                  |                                           |

WebKit-only failures often cluster around well-known iOS Safari quirks (viewport units, scroll behavior, font rendering).

Step 6 - Engine vs channel choice

Use the engine (Playwright-bundled) whenUse the branded channel when
Default; reproducible across team / CINeed to test specific Chrome / Edge release behavior
Faster CI (lighter download per pw-browsers (opens in new window))Compliance / contract requires the branded build
Pre-release behavior (Chromium N+1)Bug reports against branded versions

For most teams: bundled engines for CI; branded channels for manual / spot-check.

Anti-patterns

Anti-patternWhy it failsFix
Single-browser CICross-browser regressions invisible.Matrix (Step 4).
fail-fast: true on the matrixChromium fails; team can't see WebKit / Firefox.fail-fast: false (Step 4).
Skipping --with-deps installLinux runner missing browser dependencies; tests fail.Always --with-deps in CI (Step 1).
Per-browser conditional code in testsCouples tests to browser implementation; fragile.Test what users observe; abstract per-browser quirks in production code.
Not testing branded channels at allMisses Edge / Chrome-specific bugs.Schedule weekly branded-channel runs (Step 6).

Limitations

  • WebKit on CI is approximate. Bundled WebKit ≠ Safari; per-browser quirks (especially iOS) need real-device testing via mobile-device-matrix-toolkit (in the qa-mobile plugin).
  • Edge / Chrome require host OS. msedge needs Windows or macOS; not Linux.
  • Per-browser CI cost adds up. 5 browsers × N tests = 5x CI time / cost. Use selectively for full regression; run smoke cross-browser per-PR.

References

  • pwb (opens in new window) - Playwright supported browsers (Chromium / Firefox / WebKit + branded channels), install commands, disk-space estimates.
  • github-actions-test-jobs (in the qa-ci-integration plugin, references/os-matrix.md) - OS / runtime matrices.
  • browser-matrix-strategy-reference (references/compatibility-budget.md) - conventions for choosing and capping the matrix.
  • mobile-emulation.md (opens in new window) - mobile viewport variants of the same browsers via the devices catalog.

Mobile-web emulation - devices catalog, per-device projects, CI matrix

View source (opens in new window)

Mobile-web emulation - devices catalog, per-device projects, CI matrix

Deep reference for playwright-testing. Consult when a responsive web app needs mobile-breakpoint regression without a real-device farm: viewport + DPR + user-agent + touch emulation via Playwright's devices catalog.

Emulation covers mobile web only - for native apps use the qa-mobile plugin (appium-testing, detox-testing, flutter-testing).

Device profiles

Playwright ships a devices catalog with realistic viewport / DPR / user-agent combinations:

import { devices } from '@playwright/test';

// Common modern profiles:
devices['iPhone 15']
devices['iPhone 15 Pro Max']
devices['iPhone 14']
devices['Pixel 7']
devices['Pixel 5']
devices['Galaxy S9+']
devices['iPad Pro 11']
devices['iPad Mini']

Each entry includes:

{
  viewport: { width: 393, height: 852 },
  deviceScaleFactor: 3,
  isMobile: true,
  hasTouch: true,
  userAgent: 'Mozilla/5.0 (iPhone; ...) AppleWebKit/...',
}

isMobile: true triggers Playwright's mobile-mode quirks (meta viewport handling); hasTouch: true enables touch-event synthesis.

Per-device project config

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  projects: [
    {
      name: 'desktop-chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'mobile-iphone-15',
      use: { ...devices['iPhone 15'] },
    },
    {
      name: 'mobile-pixel-7',
      use: { ...devices['Pixel 7'] },
    },
    {
      name: 'tablet-ipad-pro',
      use: { ...devices['iPad Pro 11'] },
    },
  ],
});

Run all projects: npx playwright test. Run only mobile:

npx playwright test --project=mobile-iphone-15 --project=mobile-pixel-7

Mobile-specific assertions

Tests should distinguish desktop-only from mobile-aware behavior:

import { test, expect, devices } from '@playwright/test';

test.describe('Cart page - mobile layout', () => {
  test.use(devices['iPhone 15']);

  test('shows mobile drawer, not sidebar', async ({ page }) => {
    await page.goto('/cart');
    // Mobile-specific: drawer behind hamburger
    await expect(page.getByRole('button', { name: /menu/i })).toBeVisible();
    await expect(page.getByRole('navigation')).not.toBeVisible();   // hidden until open
  });

  test('tap (not click) on add-to-cart', async ({ page }) => {
    await page.goto('/products/BOOK-001');
    await page.getByRole('button', { name: /add to cart/i }).tap();   // .tap not .click
    await expect(page.getByRole('alert', { name: /added/i })).toBeVisible();
  });
});

.tap() synthesizes a touch event (enabled by hasTouch: true); .click() synthesizes mouse events. Prefer .tap() on mobile profiles - .click() misses touch-handler bugs.

Visual regression per device

test('home page mobile layout snapshot', async ({ page }) => {
  await page.goto('/');
  await expect(page).toHaveScreenshot('home-iphone-15.png');
});

Per-device screenshots produce per-device baselines; layout regressions at iPhone width catch issues desktop-only tests miss. Pair with playwright-snapshots (in the qa-visual-regression plugin).

CI matrix

jobs:
  e2e:
    strategy:
      fail-fast: false
      matrix:
        project:
          - desktop-chromium
          - mobile-iphone-15
          - mobile-pixel-7
          - tablet-ipad-pro
    runs-on: ubuntu-latest
    name: ${{ matrix.project }}
    steps:
      - uses: actions/checkout@v5
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test --project=${{ matrix.project }}

Each project runs as a separate matrix job; fail-fast: false ensures a failure on iPhone doesn't cancel Pixel. Because shards fail independently, gate the merge on all shards green - one red shard must block, never be averaged away.

Cypress equivalent

Cypress doesn't ship a devices catalog as rich as Playwright's; viewport sizing is the primary control:

beforeEach(() => {
  cy.viewport('iphone-15');   // built-in preset
  // OR
  cy.viewport(393, 852, 'portrait');
});

For touch-event synthesis, use cy.realTouch() (via the cypress-real-events plugin).

Anti-patterns

Anti-patternWhy it failsFix
Setting viewport: { width: 375 } onlyMisses DPR / touch / user-agent differencesSpread a devices[...] profile
.click() on mobile projectsSynthesizes mouse events; misses touch-handler bugs.tap() when hasTouch: true
One desktop+mobile mega-testif (viewport.width < ...) branching cluttersPer-project tests
Mobile-only baselines without desktop comparisonMisses desktop regressions at the mobile breakpointBoth desktop + mobile projects in CI
Treating emulation as a real-device substituteMisses real-device perf, touch sensitivity, browser quirksPair with farm runs for the release tier

Limitations

  • Emulation ≠ real device. Mobile Safari has quirks Chromium emulation doesn't reproduce (100vh viewport behavior, iOS-specific gestures).
  • No native APIs. Camera / push / geolocation accuracy / biometrics are out of reach.
  • Performance is the runner's CPU. Use Lighthouse's mobile profile for mobile perf budgets.

Related skills

browser-matrix-strategy-reference

Pure-reference for designing and reviewing a browser / OS / device test matrix from traffic data - the T1/T2/T3 tier-membership heuristics (T1 >=5% traffic, T2 1-5% or statutory, T3 <1% with customer demand), the traffic-share sources (own analytics, StatCounter, MDN browser-compat-data), a worked matrix template with tier-change log, the matrix review checklist (staleness, T1 oversize, below-threshold T1 entries, missing real-device coverage), how to justify dropping a legacy browser (IE11, old iOS Safari), and the compatibility budget (tier caps, CI cost formula, published support statement) in references/compatibility-budget.md. Use when designing an initial matrix, capping or publishing a support policy, running a quarterly re-tier review, or making the case to drop a browser. This is the WHAT-to-test strategy reference - to execute the matrix use playwright-testing browser projects (bundled engines), selenium-grid-4-runner (self-hosted), or cloud-grid-e2e (managed grids).

cloud-grid-e2e

Author and run E2E tests on a cloud browser grid - BrowserStack Automate, Sauce Labs, or LambdaTest. All three follow one pattern: username + access-key env vars, a W3C WebDriver hub URL, a vendor options dict inside the capabilities (bstack:options / sauce:options / LT:Options), a local tunnel binary for internal apps, session pass/fail reporting, and a CI matrix throttled to the plan's parallel-session limit. Worked example uses BrowserStack; per-vendor deltas live in references/. Use for cross-browser regression on real devices + browsers beyond the engines bundled on the local machine - distinct from a local matrix runner and from self-hosted Selenium Grid.

cypress-testing

Authors and improves Cypress E2E tests - installs Cypress, configures `cypress.config.ts`, authors `cy.*` command chains, refactors existing specs (`cy.wait(ms)` sleeps into assertions, repeated flows into `cy.session` custom commands), and debugs with the time-travel GUI; Cypress Cloud for parallel runs and recording. Use for both greenfield test authoring and improving hand-written specs already in the codebase. For automated refactor of raw Cypress Studio recordings specifically, use a dedicated codegen-review pass.

selenium-grid-4-runner

Author and operate Selenium Grid 4 - self-hosted distributed WebDriver. Covers the six-component architecture (Router / Distributor / Session Map / Event Bus / New Session Queue / Node), standalone vs hub-and-node modes, the Docker-image stack (selenium/standalone-chrome, selenium/hub, selenium/node-chrome), node registration, session-queue tuning, and observability. Use for self-hosted cross-browser testing when data residency or cost-control require an on-prem grid. This is the self-hosted execution RUNNER - for the zero-infra alternative use playwright-testing browser projects (bundled engines); for managed cloud grids use cloud-grid-e2e (BrowserStack / Sauce Labs / LambdaTest); to decide WHICH browsers and tiers to run use browser-matrix-strategy-reference.

selenium-testing

Authors Selenium WebDriver tests in any of its 6+ supported languages (Java, Python, JavaScript, C#, Ruby, Kotlin, PHP) - picks the appropriate language binding, configures WebDriver per browser, uses `By.*` locators with the team's accessibility-first preference where supported, runs locally + via Selenium Grid for distributed execution, parses results to JUnit XML. Use for legacy Selenium-locked stacks; new projects pick Playwright or Cypress.

web-e2e-overview

Teaches web end-to-end testing from first principles: what browser-driven E2E covers and how it differs from unit and integration tests, a decision table for choosing between Playwright, Cypress, Selenium WebDriver, WebdriverIO, Puppeteer, TestCafe and the BrowserStack / Sauce Labs / LambdaTest cloud grids based on files already present in the repo, install and first-run commands for each, and the flakiness traps (fixed sleeps, CSS and XPath selectors, state shared between tests) that sink new suites. Use when a web application has no E2E coverage yet, when picking or replacing an E2E framework, or when a first browser test needs to go green end to end.

webdriverio-testing

Authors WebdriverIO E2E tests - `npm init wdio@latest` scaffolding, services architecture (sauce, browserstack, appium, devtools), reporters (spec, allure, junit), built-in Mocha/Jasmine/Cucumber framework integrations. WebdriverIO sits between Selenium (W3C protocol) and Playwright (modern API) - Selenium-protocol-compatible with rich plugin ecosystem. Use when the team needs WebDriver protocol + service-based device-farm integration.