Testland
Browse all skills & agents

chart-render-tests

Chart-render regression testing across the three chart-library families - Canvas (Chart.js: locator screenshot snapshot + `canvas.toDataURL()` diff with animations disabled), SVG (D3: `outerHTML` structural snapshot with generated-ID normalization + per-element data-binding tests), and declarative specs (Vega / Vega-Lite: JSON Schema validation + Vega-Lite → Vega compile test). Detects the family from package.json imports (chart.js / d3 / vega-lite), then applies the matching recipe; full per-library depth with citations in references/chartjs.md, references/d3.md, references/vega.md. Use when a dashboard or data product renders charts and their output needs regression coverage - before a chart-library major upgrade, after a theming change, or when runtime-generated Vega specs must be proven valid before render.

Install with skills.sh (any agent)

npx skills add testland/qa --skill chart-render-tests
View source

chart-render-tests

Overview

Chart regressions hide behind green unit tests: the data pipeline is correct but the rendered output drifted - a theme change recolored a series, a library upgrade dropped an axis group, a spec generator emitted an encoding the compiler rejects. The right regression test depends on the rendering family, and each family gets one recipe:

FamilyLibrariesTest surfaceReference
CanvasChart.jsPixel snapshot of the <canvas> locator; toDataURL() diffreferences/chartjs.md
SVGD3, Observable Plot, D3-based React libs (Visx, Nivo, Recharts)outerHTML structural snapshot + data-binding assertionsreferences/d3.md
Declarative specVega, Vega-LiteJSON Schema validation + compile test, before renderreferences/vega.md

General UI-screen snapshots belong to the sibling skills (playwright-snapshots, percy-visual-regression-testing, chromatic-visual-regression-testing); this skill covers the chart-specific layers those miss - canvas pixel capture, SVG structure, and spec validity.

Detecting the chart library

Read package.json dependencies and source imports:

# Which family is in use?
jq -r '.dependencies, .devDependencies | keys[]?' package.json | grep -E '^(chart\.js|d3|d3-[a-z-]+|vega|vega-lite|@observablehq/plot)$'
grep -rn "from 'chart.js'\|from 'd3'\|from 'vega-lite'" src/ | head
  • chart.js (or react-chartjs-2) → Canvas recipe.
  • d3 / d3-* modules / @observablehq/plot / Visx / Nivo / Recharts → SVG recipe.
  • vega-lite / vega → spec-validation recipe; add the SVG or Canvas recipe for the rendered output depending on the renderer option.

A project can hit two rows (a BI tool generating Vega-Lite specs rendered to SVG); apply each matching recipe.

Canvas (Chart.js)

Per the Chart.js docs (opens in new window), Chart.js renders to <canvas>. Two rules make snapshots stable, then one Playwright assertion locks the render:

  1. Disable animation and responsive resizing in the chart config (options.animation = false, responsive: false) - otherwise snapshots capture mid-animation frames randomly.
  2. Pin deviceScaleFactor: 1 in playwright.config.ts so dev and CI machines rasterize identically.
test('revenue bar chart matches snapshot', async ({ page }) => {
  await page.goto('https://localhost:3000/dashboard');
  await page.waitForFunction(() => {
    const canvas = document.querySelector('canvas#revenue-chart');
    return canvas && canvas.toDataURL().length > 1000;
  });
  await expect(page.locator('canvas#revenue-chart')).toHaveScreenshot(
    'revenue-chart.png',
    { maxDiffPixels: 50 } // absorbs anti-aliasing variance
  );
});

Tooltips and legends render in DOM, not canvas - test those interactions separately. Full depth (tooltip / legend interaction tests, multi-DPI handling, toDataURL() programmatic diff, jsdom + canvas-mock unit tests, non-visual data assertions): references/chartjs.md and references/chartjs-alternative-approaches.md.

SVG (D3)

Per the D3 getting-started docs (opens in new window), D3 generates SVG - text DOM, so the structural contract is diffable as markup. Normalize generated IDs first or every run false-positives:

function normalizeSvg(svg: string): string {
  return svg
    .replace(/id="[^"]*-\d+"/g, 'id="ID"')
    .replace(/\s+/g, ' ')
    .trim();
}

test('bar chart SVG has expected structure', async ({ page }) => {
  await page.goto('https://localhost:3000/d3-bar');
  await page.waitForSelector('svg.bar-chart');
  const svgHtml = await page.locator('svg.bar-chart').evaluate(el => el.outerHTML);
  expect(normalizeSvg(svgHtml)).toMatchSnapshot('bar-chart.svg.txt');
});

Pair the structural snapshot with a data-binding test (one element per data point; per-element attributes track the data) and disable transition() in test mode. Full depth (rendered-image snapshot, jsdom unit tests, update-join enter / update / exit tests, SVG a11y metadata): references/d3.md and references/d3-advanced-tests.md.

Declarative specs (Vega / Vega-Lite)

When application code generates Vega-Lite JSON at runtime (BI builders, spec templating), validate the spec before render - the compiler's errors on invalid specs are cryptic. Three gates per the Vega-Lite docs (opens in new window):

import Ajv from 'ajv';
import vlSchema from 'vega-lite/build/vega-lite-schema.json';
import * as vl from 'vega-lite';

const validate = new Ajv({ strict: false }).compile(vlSchema);

test('generated bar spec is valid, correctly encoded, and compiles', () => {
  const spec = generateBarSpec({ x: 'quarter', y: 'revenue' });
  expect(validate(spec)).toBe(true);                 // Gate 1 - schema-valid
  expect(spec.mark.type).toBe('bar');                // Gate 2 - intended encoding
  expect(spec.encoding.y.type).toBe('quantitative');
  expect(() => vl.compile(spec)).not.toThrow();      // Gate 3 - compiles to Vega
});

A spec can be schema-valid yet semantically broken (references a missing field) - the compile gate catches that. Full depth (render-to-SVG assertions, multi-view composition, transforms, interaction parameters, spec snapshots): references/vega.md and references/vega-advanced-spec-tests.md.

Anti-patterns

Anti-patternWhy it failsFix
Snapshot the whole page for one chartUnrelated layout shifts break the testSnapshot the chart locator only
Skip animation: false (Chart.js) or transition() disable (D3)Mid-animation frames make snapshots flakyDisable motion in test mode
maxDiffPixels: 0 on canvas snapshotsAnti-aliasing flake across machinesAllow ~50 pixels; pin DPR to 1
Diff SVG outerHTML with generated IDs intactFalse positive every runNormalize IDs first
Compile Vega-Lite without schema validation firstCompiler errors are crypticSchema gate before compile gate
Test only the rendered output of generated specsSpec-generator bugs hide behind a correct-looking renderAssert mark + encoding on the spec itself

Limitations

  • Canvas snapshots cannot catch SVG-only regressions and vice versa; classify the library family first (Detection above).
  • jsdom computes no SVG layout (getBBox() missing); measured-position tests need a real browser.
  • Vega schema validation is slow on large spec corpora; cache the compiled Ajv validator.
  • Tooltips and legends usually render in DOM, not the chart surface; cover them with interaction tests, not snapshots.

References

Chart.js - alternative approaches

View source (opens in new window)

Chart.js - alternative approaches

Secondary approaches beyond the core Playwright canvas snapshot (Step 2 in SKILL.md). Reach for these when the primary screenshot workflow does not fit.

Programmatic canvas dataURL diff

For finer control without Playwright's screenshot helper:

test('chart canvas data URL is stable', async ({ page }) => {
  await page.goto('https://localhost:3000/dashboard');
  await page.waitForFunction(() => /* render complete */);

  const dataUrl = await page.evaluate(() => {
    const canvas = document.querySelector('canvas#revenue-chart') as HTMLCanvasElement;
    return canvas.toDataURL('image/png');
  });

  // Compare to baseline saved as PNG
  const baseline = await readBaseline('revenue-chart.png');
  const diff = imagePixelDiff(dataUrl, baseline);
  expect(diff.diffRatio).toBeLessThan(0.005);
});

jsdom + canvas-mock unit testing

For unit-test-speed feedback (no browser):

// jest.setup.js
import 'canvas';  // node-canvas package
import { Chart } from 'chart.js/auto';

test('chart renders with expected dataset count', () => {
  const canvas = document.createElement('canvas');
  document.body.appendChild(canvas);

  const chart = new Chart(canvas, {
    type: 'bar',
    data: { labels: ['Q1', 'Q2'], datasets: [{ data: [10, 20] }] },
    options: { animation: false, responsive: false },
  });

  expect(chart.data.datasets).toHaveLength(1);
  expect(chart.data.labels).toEqual(['Q1', 'Q2']);
});

The canvas package (Node native) lets jsdom render Chart.js output without a browser. Use for fast assertions on dataset shape + config; rely on the core snapshot workflow for visual regression.

Data-driven assertion (without snapshot)

For non-visual assertions, query Chart.js internal state via the chart instance:

test('chart shows all 12 months', async ({ page }) => {
  const labels = await page.evaluate(() => {
    const chart = (window as any).Chart.getChart('revenue-chart');
    return chart.data.labels;
  });
  expect(labels).toHaveLength(12);
});

Chart.js (Canvas) snapshot tests

View source (opens in new window)

Chart.js (Canvas) snapshot tests

Chart.js reference for chart-render-tests: render via headless Chromium / jsdom + canvas mock, capture canvas pixels via canvas.toDataURL() + image-diff, disable animations (options.animation = false) for stable snapshots, test tooltip + legend interactions.

Per the Chart.js docs (opens in new window), Chart.js renders to <canvas>, testable via canvas.toDataURL() snapshot diff.

When to use

  • Dashboards or analytics products where chart accuracy is product surface.
  • Library upgrade gate (Chart.js v4 → v5 changes default styles).
  • Custom theme integration - verify the brand styling renders correctly.

Step 1 - Disable animations for stable snapshots

Per the Chart.js docs (opens in new window), the basic config object accepts options:

new Chart(ctx, {
  type: 'bar',
  data: {...},
  options: {
    animation: false,    // disable for snapshots
    responsive: false,   // fix the canvas dimensions
    plugins: {
      legend: { display: true },
    },
    scales: { y: { beginAtZero: true } },
  },
});

Without animation: false, snapshots capture mid-animation frames randomly.

Step 2 - Playwright canvas snapshot

import { test, expect } from '@playwright/test';

test('revenue bar chart matches snapshot', async ({ page }) => {
  await page.goto('https://localhost:3000/dashboard');

  // Wait for chart to render (no animation, just initial draw)
  await page.waitForFunction(() => {
    const canvas = document.querySelector('canvas#revenue-chart');
    return canvas && canvas.toDataURL().length > 1000;
  });

  const canvas = page.locator('canvas#revenue-chart');
  await expect(canvas).toHaveScreenshot('revenue-chart.png', {
    maxDiffPixels: 50,
  });
});

maxDiffPixels allows for sub-pixel anti-aliasing variance across runs.

For alternatives to the Playwright screenshot helper - a programmatic toDataURL() diff, jsdom + canvas-mock unit tests, and non-visual data-driven assertions - see chartjs-alternative-approaches.md (opens in new window).

Step 3 - Tooltip + legend interaction

test('tooltip shows data point value on hover', async ({ page }) => {
  await page.goto('https://localhost:3000/dashboard');
  await waitForChartReady(page);

  // Hover over a known data point coordinate
  await page.mouse.move(150, 200);
  await page.waitForSelector('.chartjs-tooltip', { state: 'visible' });

  const tooltipText = await page.locator('.chartjs-tooltip').textContent();
  expect(tooltipText).toContain('Q1: 10');
});

test('legend click toggles dataset visibility', async ({ page }) => {
  await page.goto('https://localhost:3000/dashboard');
  await waitForChartReady(page);

  await page.click('.chartjs-legend-item:has-text("Revenue")');

  // Re-snapshot; revenue dataset should be hidden
  await expect(page.locator('canvas#revenue-chart')).toHaveScreenshot(
    'revenue-chart-revenue-hidden.png'
  );
});

Step 4 - Multi-DPI handling

Pin device pixel ratio so CI and dev machines produce identical snapshots:

// playwright.config.ts
use: {
  deviceScaleFactor: 1,  // pin to 1× for snapshot stability
}

Anti-patterns

Anti-patternWhy it failsFix
Skip animation: falseSnapshots flakyStep 1 mandatory
Snapshot whole pageLayout shifts unrelated to chart break testsSnapshot the canvas locator only (Step 2)
maxDiffPixels: 0Anti-aliasing flakeAllow ~50 pixels (Step 2)
Test only static dataDynamic data behavior untestedSnapshot per scenario (filter, range)
Skip DPR pinningCI machines vs dev machines render differentlyStep 4

Limitations

  • Canvas snapshots can't catch SVG-only regressions (Chart.js is canvas-only); for SVG charts see d3.md (opens in new window).
  • Chart.js plugins (annotation, datalabels) may have separate init paths; verify they render before snapshotting.
  • Tooltips render in DOM (not canvas), so canvas snapshot misses them - test interactions separately (Step 3).

References

D3 - advanced correctness tests

View source (opens in new window)

D3 - advanced correctness tests

Deeper D3 test patterns split out of the SKILL spine: the update join (enter / update / exit) and SVG accessibility metadata.

Update join correctness

D3's update join (enter / update / exit) is the hardest D3 concept to test. Test the three states:

test('update join handles insert + remove + reorder', async ({ page }) => {
  await page.goto('https://localhost:3000/d3-update');

  // Initial: [A, B, C]
  await page.evaluate(() => (window as any).updateChart(['A', 'B', 'C']));
  expect(await page.locator('rect[data-key="A"]').count()).toBe(1);

  // After: [A, B, D] - remove C, add D
  await page.evaluate(() => (window as any).updateChart(['A', 'B', 'D']));
  expect(await page.locator('rect[data-key="C"]').count()).toBe(0);
  expect(await page.locator('rect[data-key="D"]').count()).toBe(1);

  // After: [B, D, A] - reorder; element identity preserved
  await page.evaluate(() => (window as any).updateChart(['B', 'D', 'A']));
  // 'A' should be the same DOM node (just repositioned)
  // Verify via attribute or event listener attached pre-reorder
});

Use a stable key function: data-bind by .data(arr, d => d.id).

Accessibility metadata

D3 generates SVG; SVG has accessibility primitives. Tests verify:

test('chart has title + desc for screen readers', async ({ page }) => {
  await page.goto('https://localhost:3000/d3-bar');

  await expect(page.locator('svg.bar-chart > title')).toContainText('Revenue by Quarter');
  await expect(page.locator('svg.bar-chart > desc')).toContainText('Bar chart showing');
});

test('rects have aria-labels', async ({ page }) => {
  const labels = await page.locator('svg.bar-chart rect').evaluateAll(els =>
    els.map(el => el.getAttribute('aria-label'))
  );
  expect(labels[0]).toBe('Q1 revenue: $10k');
});

Cross-ref qa-accessibility plugin for broader a11y patterns.

D3 (SVG) structural snapshot tests

View source (opens in new window)

D3 (SVG) structural snapshot tests

D3 reference for chart-render-tests: use outerHTML snapshot for static structure, toHaveScreenshot for rendered SVG, jsdom for headless render in unit tests, disabled transitions for stable snapshots, and per-element data-binding correctness tests.

Per the D3 getting-started docs (opens in new window), D3 "generates SVG output (not Canvas). Code examples show creation of SVG elements with d3.create('svg') and DOM manipulation via selections." SVG is text-DOM, so snapshots can be the rendered SVG markup OR a rendered image - different test patterns for each.

When to use

  • Custom D3 viz library where DOM structure is the contract.
  • Dashboards using Observable Plot / D3-based React libs (Visx, Nivo, Recharts).
  • Pre-deploy gate before D3 major upgrade (D3 v6 → v7 changed module imports).

How to use

  1. Confirm the chart renders SVG via d3.create('svg') or D3 selections (Canvas belongs to chartjs.md (opens in new window)).
  2. Pick the snapshot mode: outerHTML for the structural contract (Step 1) or toHaveScreenshot for rendered pixels (Step 2).
  3. Disable D3 transition() in test mode so snapshots are stable (Step 3).
  4. Add a data-binding test: one element per data point, per-element attributes track the data (Step 4).
  5. For fast headless runs, render the generator under jsdom in a unit test (Step 5).
  6. Cover the update join and SVG a11y metadata via d3-advanced-tests.md (opens in new window).
  7. Normalize generated IDs before diffing; run the whole suite as a gate before any D3 major-version upgrade.

Worked example

A revenue bar chart renders svg.bar-chart from [10, 20, 30, 40]. Capture its outerHTML, pass it through normalizeSvg (Step 1) to strip generated IDs, and store bar-chart.svg.txt. Add a data-binding test (Step 4) asserting 4 rects and heights[3] > heights[0] (data 40 > 10). The 750 ms enter transition is swapped for the identity function in test mode (Step 3) so the snapshot is deterministic. Suite runs green. Later a D3 v6 → v7 upgrade drops a <g> wrapper import; the outerHTML snapshot diff flags the missing group before it ships.

Step 1 - outerHTML structural snapshot

For tests where SVG structure should match exactly:

import { test, expect } from '@playwright/test';

test('bar chart SVG has expected structure', async ({ page }) => {
  await page.goto('https://localhost:3000/d3-bar');
  await page.waitForSelector('svg.bar-chart');

  const svgHtml = await page.locator('svg.bar-chart').evaluate(el => el.outerHTML);

  // Compare to stored fixture
  expect(normalizeSvg(svgHtml)).toMatchSnapshot('bar-chart.svg.txt');
});

normalizeSvg strips dynamically-generated IDs (__id__123) + whitespace differences:

function normalizeSvg(svg: string): string {
  return svg
    .replace(/id="[^"]*-\d+"/g, 'id="ID"')
    .replace(/\s+/g, ' ')
    .trim();
}

Step 2 - Rendered-image snapshot (Playwright)

For visual-regression-style:

test('scatter plot renders correctly', async ({ page }) => {
  await page.goto('https://localhost:3000/d3-scatter');
  await page.waitForSelector('svg.scatter');

  await expect(page.locator('svg.scatter')).toHaveScreenshot('scatter.png', {
    maxDiffPixels: 50,
  });
});

Per Chart.js docs (opens in new window) equivalent works for D3 too - snapshot the locator, not the page.

Step 3 - Disable transitions

D3 transition() calls animate. Disable for tests:

// Instead of d3.select(...).transition().duration(750).attr(...)
// In test mode:
const transition = process.env.NODE_ENV === 'test'
  ? (sel) => sel  // identity
  : (sel) => sel.transition().duration(750);

transition(d3.select('.bars').selectAll('rect'))
  .attr('width', d => x(d.value));

Or use d3.transition().duration(0) if API can't be conditional.

Step 4 - Per-element data-binding test

D3's strength is data-driven DOM. Test the binding holds:

test('one rect per data point', async ({ page }) => {
  const data = [10, 20, 30, 40];
  await page.goto(`https://localhost:3000/d3-bar?data=${JSON.stringify(data)}`);
  await page.waitForSelector('svg.bar-chart rect');

  const rects = await page.locator('svg.bar-chart rect').count();
  expect(rects).toBe(data.length);

  // Per-rect height matches data
  const heights = await page.locator('svg.bar-chart rect').evaluateAll(els =>
    els.map(el => parseFloat(el.getAttribute('height')!))
  );
  expect(heights[3]).toBeGreaterThan(heights[0]);  // data[3]=40 > data[0]=10
});

Step 5 - jsdom unit test (fast)

import { JSDOM } from 'jsdom';
import * as d3 from 'd3';

test('bar generator emits N rects for N data points', () => {
  const dom = new JSDOM('<svg id="chart"></svg>');
  global.document = dom.window.document;

  const data = [1, 2, 3];
  d3.select(dom.window.document.body)
    .select('svg')
    .selectAll('rect')
    .data(data)
    .join('rect')
    .attr('height', d => d);

  const rects = dom.window.document.querySelectorAll('rect');
  expect(rects).toHaveLength(3);
});

Per the D3 getting-started docs (opens in new window), D3 imports cleanly under ESM - jsdom + native ESM works.

Advanced correctness tests

Update join (enter / update / exit) and SVG accessibility metadata are the deepest D3 test patterns - see d3-advanced-tests.md (opens in new window).

Anti-patterns

Anti-patternWhy it failsFix
outerHTML diff with all generated IDsFalse positives every runNormalize (Step 1)
Skip transition disableSnapshot flakeStep 3
No update-join testenter/exit bugs shipd3-advanced-tests.md (opens in new window)
Test only happy data shapeEmpty / single-element / overflow data shapes breakBoundary value testing
Mix Chart.js + D3 in same chartCanvas + SVG mix: snapshots inconsistentOne library per chart

Limitations

  • jsdom doesn't compute SVG layout (no getBBox()). Tests that require measured positions need a real browser.
  • D3's modular structure means d3 (full bundle) ≠ individual modules (d3-selection, d3-array); pin import strategy.
  • D3 generates <title> tooltip natively; some libs override - test the actual rendering.

References

Vega/Vega-Lite advanced spec tests

View source (opens in new window)

Vega/Vega-Lite advanced spec tests

Deep reference for the Vega side of chart-render-tests (vega.md (opens in new window)). Consult after the core schema + structure + compilation gates pass, when adding render-time, composition, transform, interaction, or spec-snapshot coverage.

Render-to-SVG test

import { Vega } from 'react-vega';
import { create } from 'jsdom';

test('renders SVG with expected mark count', async () => {
  const dom = create('<div id="vis"></div>');
  global.document = dom.window.document;

  const view = new vega.View(vega.parse(vegaSpec))
    .renderer('svg')
    .initialize(dom.window.document.querySelector('#vis'))
    .run();

  const svg = await view.toSVG();
  // Use a parser to count <path>/<rect> elements
  const rectCount = (svg.match(/<rect/g) || []).length;
  expect(rectCount).toBe(4);  // 4 quarters
});

Multi-view composition test

Per the Vega-Lite docs (opens in new window), Vega-Lite supports faceting, layering, concatenation, repeating. Test each composition:

test('layered spec has 2 layers', () => {
  const spec = {
    layer: [
      { mark: 'line', encoding: {...} },
      { mark: 'point', encoding: {...} },
    ],
  };

  expect(validate(spec)).toBe(true);
  expect(spec.layer).toHaveLength(2);
});

test('faceted spec creates one view per category', async () => {
  const spec = {
    facet: { field: 'category', type: 'nominal' },
    spec: { mark: 'bar', encoding: {...} },
  };

  const compiled = vl.compile(spec).spec;
  const view = new vega.View(vega.parse(compiled)).renderer('svg').initialize(...).run();
  // Inspect view's data tables to verify N facets emerged
});

Data transform test

Per the Vega-Lite docs (opens in new window), transforms include "Aggregate, filter, bin, calculate, fold, pivot." Test transform output:

test('aggregate transform produces correct sum', () => {
  const spec = {
    data: { values: [
      { region: 'NA', revenue: 100 },
      { region: 'NA', revenue: 200 },
      { region: 'EU', revenue: 150 },
    ]},
    transform: [
      { aggregate: [{ op: 'sum', field: 'revenue', as: 'total' }],
        groupby: ['region'] },
    ],
    mark: 'bar',
    encoding: { x: { field: 'region' }, y: { field: 'total', type: 'quantitative' } },
  };

  const view = new vega.View(vega.parse(vl.compile(spec).spec));
  await view.runAsync();
  const data = view.data('source_0');
  expect(data.find(d => d.region === 'NA').total).toBe(300);
  expect(data.find(d => d.region === 'EU').total).toBe(150);
});

Interaction (parameters / selections)

Per the Vega-Lite docs (opens in new window), "Interactive parameters - Selections and value bindings" enable interaction. Test parameters resolve:

test('selection parameter filters data', async () => {
  const spec = {
    params: [{ name: 'brush', select: 'interval' }],
    data: {...},
    mark: 'point',
    encoding: {...},
    transform: [{ filter: { param: 'brush' } }],
  };

  // Compile, render, inject brush event, verify filtered data
  ...
});

Spec-snapshot regression

For complex spec-generation logic, snapshot the spec output:

test('quarterly-revenue spec snapshot stable', () => {
  const spec = generateBarSpec({
    x: 'quarter',
    y: 'revenue',
    title: 'Quarterly Revenue',
  });

  expect(spec).toMatchSnapshot('quarterly-revenue.spec.json');
});

When intentionally changing spec generation, regenerate snapshots: UPDATE_SNAPSHOTS=1 npm test.

Vega / Vega-Lite spec validation

View source (opens in new window)

Vega / Vega-Lite spec validation

Vega reference for chart-render-tests: validate Vega + Vega-Lite specifications against the JSON Schema (vega.github.io/schema), test cross-engine compatibility (Vega-Lite compiles to Vega per the canonical compiler), and verify data-binding correctness. Pair with d3.md (opens in new window) when Vega specs render to SVG; pair with chartjs.md (opens in new window) when rendered to Canvas.

Vega-Lite specs (Mark, Encoding, Data) compile to Vega, which renders to SVG/Canvas, per the Vega-Lite docs (opens in new window). Tests validate a spec is well-formed AND produces the expected rendered output.

When to use

  • BI tool / data product builds Vega-Lite specs from user input; verify generated specs are valid before render.
  • Library upgrade (Vega 5 → Vega 6) - verify existing specs still compile + render.
  • Custom encoding rules in spec generation - assert the produced spec matches the expected mark+encoding shape.

How to use

  1. Install a JSON Schema validator (Ajv) plus the bundled Vega-Lite schema; compile the validator once and cache it (Step 1).
  2. Assert schema validity of every generated spec before anything else - compiler errors on an invalid spec are cryptic (Step 1).
  3. Add structural assertions that the mark + encoding match the intended chart shape, beyond bare schema validity (Step 2).
  4. Compile Vega-Lite to Vega to catch semantically-broken specs that pass the schema but reference missing fields (Step 3).
  5. Confirm the three gates in one end-to-end test (Worked example).
  6. For render-time, composition, transform, interaction, and spec-snapshot coverage, see vega-advanced-spec-tests.md (opens in new window).

Step 1 - JSON Schema validation

Per the Vega-Lite docs (opens in new window), the spec is JSON; it has a published JSON Schema. Validate:

import Ajv from 'ajv';
import schema from 'vega-lite/build/vega-lite-schema.json';

const ajv = new Ajv({ strict: false });
const validate = ajv.compile(schema);

test('generated bar spec is valid Vega-Lite', () => {
  const spec = generateBarSpec({ x: 'quarter', y: 'revenue' });
  const valid = validate(spec);
  if (!valid) {
    console.log(validate.errors);
  }
  expect(valid).toBe(true);
});

Schema URL pattern: https://vega.github.io/schema/vega-lite/v5.json (track current version per the Vega-Lite docs (opens in new window)).

Step 2 - Spec structural assertions

Beyond schema validity, assert business-relevant structure:

test('spec uses correct mark + encoding for bar chart', () => {
  const spec = generateBarSpec({ x: 'quarter', y: 'revenue' });

  expect(spec.mark.type).toBe('bar');
  expect(spec.encoding.x.field).toBe('quarter');
  expect(spec.encoding.x.type).toBe('nominal');
  expect(spec.encoding.y.field).toBe('revenue');
  expect(spec.encoding.y.type).toBe('quantitative');
});

Step 3 - Compilation test (Vega-Lite → Vega)

Per the Vega-Lite docs (opens in new window): "Vega-Lite compiles a Vega-Lite specification into a lower-level, more detailed Vega specifications and rendered using Vega's compiler."

import * as vl from 'vega-lite';

test('Vega-Lite compiles to valid Vega', () => {
  const vlSpec = generateBarSpec(...);
  const vegaSpec = vl.compile(vlSpec).spec;

  // Validate Vega spec against Vega schema
  const vegaValid = validateVegaSchema(vegaSpec);
  expect(vegaValid).toBe(true);
});

Failed compilation indicates the Vega-Lite spec is well-formed schema-wise but semantically broken (e.g., references a non-existent field).

Worked example

One test that proves a BI builder's bar-spec generator produces a spec that is schema-valid, correctly encoded, AND compiles - the three core gates in a single first run:

import Ajv from 'ajv';
import vlSchema from 'vega-lite/build/vega-lite-schema.json';
import * as vl from 'vega-lite';

const validate = new Ajv({ strict: false }).compile(vlSchema);

test('generated bar spec is valid, correctly encoded, and compiles', () => {
  const spec = generateBarSpec({ x: 'quarter', y: 'revenue' });

  // Gate 1 - schema-valid
  expect(validate(spec)).toBe(true);

  // Gate 2 - correct mark + encoding
  expect(spec.mark.type).toBe('bar');
  expect(spec.encoding.x.field).toBe('quarter');
  expect(spec.encoding.y.type).toBe('quantitative');

  // Gate 3 - compiles to Vega without throwing
  expect(() => vl.compile(spec)).not.toThrow();
});

That single test is the minimum a runtime-generated spec must pass before render. Render-time and semantic verification build on it - see the advanced reference below.

Advanced tests

Render-to-SVG assertions, multi-view composition (facet / layer / concat / repeat), data-transform verification, interaction parameters, and spec-snapshot regression each get a full worked test in vega-advanced-spec-tests.md (opens in new window).

Anti-patterns

Anti-patternWhy it failsFix
Skip JSON Schema validation; compile directCompiler errors are crypticStep 1 first
Test only the rendered output, not the specSpec gen bugs hide behind correct renderStep 2 + spec snapshot (references)
Hardcoded Vega-Lite v4 schemaSchema upgrades change validityPin AND track
Skip transform testsAggregate / filter bugs ship silentlyTransform test (references)
Use mark: 'bar' shorthand mixed with object formSchema accepts both; downstream code may notPick one form per project

Limitations

  • Vega specs are large; full schema validation is slow on big spec corpora. Cache compiled validators.
  • Vega-Lite is opinionated - some custom visuals require dropping to plain Vega.
  • Per-engine renderers (browser SVG vs canvas vs Vega-Embed + worker) differ subtly; pin engine in tests.

References

Related skills

chromatic-visual-regression-testing

Authors and runs Chromatic visual tests on Storybook, Playwright, or Cypress projects via the `chromatic` CLI; configures baselines, TurboSnap, UI Review, and CI gating; reads exit codes for change-vs-error classification. Use when the project ships visual regression coverage to Chromatic Cloud.

percy-visual-regression-testing

Authors Percy visual snapshot tests via the @percy/cli + framework SDK (Playwright, Cypress, Selenium, Storybook), runs them with `percy exec -- {test command}`, configures viewports / masking / ignored regions, and reviews diffs in the Percy build UI. Use when the project ships visual regression coverage to BrowserStack Percy.

playwright-snapshots

Authors Playwright `expect(page).toHaveScreenshot()` assertions, configures masks / clips / threshold / maxDiffPixels per test, manages the per-OS / per-browser snapshot directory, and runs the update flow with `--update-snapshots`; references/ carry the responsive-breakpoint viewport matrix (one project per breakpoint, cross-breakpoint matrix report, plus Chromatic / Percy / Storybook test-runner viewport syntax). Use when the project ships self-hosted visual regression coverage in Playwright (no external snapshot service), or needs a unified multi-viewport breakpoint matrix.

storybook-visual-regression-testing

Sets up visual regression coverage for a Storybook project - either via the official @chromatic-com/storybook addon (hosted) or via @storybook/test-runner with a postVisit hook that calls Playwright's toHaveScreenshot (self-hosted). Covers test-runner install, lifecycle hooks (setup / preVisit / postVisit), and CI integration. Use when a repo already has a working `.storybook/` config and the team wants per-story visual coverage rather than page-level snapshots.

visual-baseline-conventions

Reference catalog for visual regression coverage decisions - which Storybook stories or pages get baselines, how to choose breakpoints, when to mask vs adjust threshold, when to add or remove a baseline, and a decision matrix for picking among Percy / Chromatic / Playwright / Storybook test-runner. Use when designing visual coverage for a new project or auditing an existing baseline set.

visual-baseline-gate

Consumes pre-classified visual-diff JSON and a reviewer-signed acceptance log to produce a single go/no-go CI verdict for visual regression. Blocks when intentional baseline changes lack a non-author reviewer sign-off or when regressions are present, and emits the binding gate artifacts - visual-gate.json + visual-gate.md - with fail-closed handling of a missing classifier run and author-cannot-self-approve enforcement, so the pipeline can exit non-zero on BLOCK. Use when the gate's input is pre-classified diff data and the enforcement concern is reviewer approval and a binding CI verdict.