lighthouse-pwa-audit

Overview

Lighthouse is the canonical PWA audit tool. The current release line is v13.3.0 per github.com/GoogleChrome/lighthouse (released May 2026). The PWA category itself was deprecated - per developer.chrome.com/docs/lighthouse/pwa: "PWA testing in Lighthouse is deprecated. For more information on its deprecation see Chrome's updated Installability Criteria." - but the individual audits remain available and run on demand under a custom Lighthouse config. This skill covers both running them and reading the LHR (Lighthouse Result) JSON the audits emit.

The companion @lhci/cli package per github.com/GoogleChrome/lighthouse-ci is how the audits gate CI: it wraps Lighthouse runs and asserts category / audit scores per a .lighthouserc.json config.

When to use

• A PWA needs a pre-release audit gate covering installability, service-worker registration, and the maskable-icon contract.
• A regression bisect points at a manifest / icon change and you need a precise installable-manifest verdict.
• CI needs to fail PRs that drop below a fixed PWA-audit threshold even though the category badge is deprecated.
• A site-reliability dashboard wants per-audit time-series - the LHR JSON is the canonical input.

Authoring

Step 1 - Install Lighthouse + Lighthouse CI

npm install --save-dev lighthouse @lhci/cli
# Or globally for one-off CLI use:
npm install -g lighthouse @lhci/cli

Step 2 - Inventory the PWA audits

Per lh-pwa, the audits previously grouped under the PWA category:

Step 3 - Run audits from the CLI

The basic invocation per lh-gh:

lighthouse https://localhost:3000 \
  --output=json \
  --output-path=./lhr.json \
  --form-factor=mobile \
  --throttling-method=simulate \
  --chrome-flags="--headless --window-size=412,660"

CLI flags from lh-gh:

To restrict to the still-supported audits without invoking the deprecated category, list the audit IDs directly:

lighthouse https://localhost:3000 \
  --only-audits=installable-manifest,service-worker,maskable-icon,viewport,themed-omnibox,splash-screen,content-width,apple-touch-icon,is-on-https \
  --output=json \
  --output-path=./lhr.json

Step 4 - Author a Lighthouse CI config

Create .lighthouserc.json per lhci-config:

{
  "ci": {
    "collect": {
      "url": ["http://localhost:3000/"],
      "numberOfRuns": 3,
      "settings": {
        "onlyAudits": [
          "installable-manifest",
          "service-worker",
          "maskable-icon",
          "viewport",
          "themed-omnibox",
          "splash-screen",
          "content-width",
          "apple-touch-icon",
          "is-on-https"
        ],
        "throttlingMethod": "devtools"
      }
    },
    "assert": {
      "assertions": {
        "installable-manifest": ["error", { "minScore": 1 }],
        "service-worker": ["error", { "minScore": 1 }],
        "maskable-icon": ["error", { "minScore": 1 }],
        "viewport": ["error", { "minScore": 1 }],
        "is-on-https": ["error", { "minScore": 1 }],
        "themed-omnibox": ["warn", { "minScore": 1 }],
        "splash-screen": ["warn", { "minScore": 1 }],
        "content-width": ["warn", { "minScore": 1 }],
        "apple-touch-icon": ["warn", { "minScore": 1 }]
      },
      "aggregationMethod": "median-run"
    },
    "upload": { "target": "temporary-public-storage" }
  }
}

Per lhci-config, the assertion shape is "<audit-id-or-categories:<id>>": [severity, { minScore | maxNumericValue | ... }] with severity one of off, warn, error. The "error" path fails the build; "warn" surfaces a warning without failing.

The aggregationMethod per lhci-config supports median, optimistic, pessimistic, median-run - median-run "represents the most typical run" and is the right choice for noisy mobile PWA audits.

Step 5 - Programmatic invocation from Node.js

For per-test invocation outside Lighthouse CI:

// tests/lighthouse-pwa.spec.ts
import { test, expect } from 'vitest';
import lighthouse from 'lighthouse';
import { launch } from 'chrome-launcher';

test('PWA audits pass on the build', async () => {
  const chrome = await launch({ chromeFlags: ['--headless'] });
  try {
    const { lhr } = await lighthouse(
      'http://localhost:3000/',
      {
        port: chrome.port,
        output: 'json',
        onlyAudits: [
          'installable-manifest',
          'service-worker',
          'maskable-icon',
          'viewport',
          'is-on-https',
        ],
        formFactor: 'mobile',
        throttlingMethod: 'devtools',
      } as any
    );

    for (const id of [
      'installable-manifest',
      'service-worker',
      'maskable-icon',
      'viewport',
      'is-on-https',
    ]) {
      expect(lhr.audits[id].score).toBe(1);
    }
  } finally {
    await chrome.kill();
  }
});

The lighthouse npm export returns a Promise resolving to { lhr, report, artifacts }. The LHR is the parsed JSON; report is the rendered HTML (when requested).

Running

Local one-off run

# Smoke
lighthouse https://localhost:3000/ \
  --only-audits=installable-manifest,service-worker,maskable-icon \
  --output=html --output-path=./pwa-smoke.html
open pwa-smoke.html

Lighthouse CI autorun

npm install -g @lhci/cli@0.15.x
lhci autorun

lhci autorun per lhci-gh is the umbrella command that "orchestrates the workflow" - it sequences lhci collect → lhci assert → lhci upload.

CI (GitHub Actions)

Per lhci-gh:

name: CI
on: [push]
jobs:
  lighthouseci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm install && npm install -g @lhci/cli@0.15.x
      - run: npm run build
      - run: lhci autorun

Parsing results

The LHR (Lighthouse Result) JSON is the canonical machine-readable output. Key paths:

Per lh-gh (release v13.3.0), the LHR schema is stable across patch releases; major releases may add / remove audits. Pin the Lighthouse version in CI for stable assertions.

For the installable-manifest audit specifically, the details field contains a list of failing requirements (e.g. "Manifest does not have a maskable icon", "Page does not work offline"). A failed-audit triage step is:

const a = lhr.audits['installable-manifest'];
if (a.score !== 1) {
  console.log('installable-manifest failures:');
  for (const item of a.details?.items ?? []) {
    console.log('  -', item.failureReason || item.message || JSON.stringify(item));
  }
}

CI integration

For projects that ship a PWA: gate PRs on the still-supported install audits as a baseline.

jobs:
  pwa-audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v4
      - run: npm ci && npm run build
      - run: npm install -g @lhci/cli@0.15.x
      - run: lhci autorun
      - if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: lhr-failure
          path: .lighthouseci/

The upload-artifact-on-failure step is essential - lhci's default output is a temporary-public-storage URL that disappears after the job retention window. Persisting the .lighthouseci/ directory gives engineers the LHR JSON to triage offline.

Anti-patterns

Limitations

• The PWA category is deprecated per lh-pwa; the individual audits work but the aggregate category badge is no longer surfaced in Chrome DevTools UI. Asserting per-audit (Step 4) is the forward-compatible posture.
• load-fast-enough-for-pwa and works-offline scores depend on the network throttling profile; cross-environment comparisons require pinning --throttling-method and --form-factor.
• Per-run variance. Even with median-run aggregation, mobile emulation introduces ±5-point swings on perf metrics; installable-manifest and service-worker are binary and stable, but other audits drift.
• Headless CI cannot test iOS install criteria. Lighthouse drives Chromium only; the apple-touch-icon audit checks the link tag presence, not the resulting iOS install behavior. See pwa-install-flow-reference Stage 3 for the iOS-specific path.
• Authenticated routes require the auth recipe from lh-gh docs - programmatic invocation with a logged-in Chrome user data dir, not covered by the default lighthouse <url> form.

References

• Lighthouse PWA audits (deprecation notice, audit list, per-audit purpose statements) - lh-pwa.
• Lighthouse repo (v13.3.0, CLI flags, --only-categories, --throttling-method, programmatic API) - lh-gh.
• Lighthouse CI repo (lhci autorun, GitHub Actions workflow) - lhci-gh.
• Lighthouse CI config (.lighthouserc.json shape, preset values, category vs audit assertions, aggregation methods) - lhci-config.
• Differentiation: this skill is the audit reader. The pwa-install-flow-reference is the contract the installable-manifest audit checks against; the workbox-tests skill covers the runtime cache behavior Lighthouse can't fully inspect.
• Sibling skills: pwa-install-flow-reference, web-push-test, service-worker-lifecycle-test.