launchdarkly-testing

Wraps LaunchDarkly server-side SDK testing patterns: TestData data source for hermetic tests (no network), file-based data source for fixture-driven tests, flag override patterns (TestData.update for per-test flag values), and assignment-integrity tests. Use when writing tests for code that uses LaunchDarkly flags; to decide which flag combinations those tests should cover in the first place, see feature-flag-test-matrix-reference.

Install with skills.sh (any agent)

npx skills add testland/qa --skill launchdarkly-testing

View source

launchdarkly-testing

Overview

LaunchDarkly's server-side SDK exposes a TestData data source that lets tests configure flag values without any network call - the canonical hermetic-test pattern. Per launchdarkly.com/docs/sdk (opens in new window), TestData replaces the production data source; everything else about the SDK (variation evaluation, targeting rules, default handling) runs the real code paths.

When to use

Unit / integration tests for code that calls client.variation() / client.boolVariation().
Tests asserting on targeting-rule behaviour (segment matching, percentage rollout).
Assignment-integrity tests per ab-test-validity-checklist.

Authoring

Install

npm install --save-dev launchdarkly-node-server-sdk
pip install launchdarkly-server-sdk

Initialize with TestData

import * as LaunchDarkly from 'launchdarkly-node-server-sdk';

const td = LaunchDarkly.TestData.dataSource();
const client = LaunchDarkly.init('sdk-test-key', {
  updateProcessor: td,
  sendEvents: false,         // No event uploads in tests
});

await client.waitForInitialization();

// Configure a flag's default + per-user variation
td.update(td.flag('show-new-ui').booleanFlag().on(true));

Variation evaluation

test('flag on returns true', async () => {
  td.update(td.flag('show-new-ui').booleanFlag().on(true));
  const user = { key: 'user-1' };
  const enabled = await client.variation('show-new-ui', user, false);
  expect(enabled).toBe(true);
});

test('flag off returns default', async () => {
  td.update(td.flag('show-new-ui').booleanFlag().on(false));
  const enabled = await client.variation('show-new-ui', { key: 'user-1' }, false);
  expect(enabled).toBe(false);
});

Targeting rules

test('only premium users get treatment', async () => {
  td.update(
    td.flag('premium-feature')
      .booleanFlag()
      .variationForUser('premium-user-1', true)
      .fallthroughVariation(false)
  );
  expect(await client.variation('premium-feature', { key: 'premium-user-1' }, false)).toBe(true);
  expect(await client.variation('premium-feature', { key: 'free-user-1' }, false)).toBe(false);
});

Percentage rollout (deterministic assignment)

test('rollout is deterministic per user', async () => {
  td.update(td.flag('half-rollout').booleanFlag().on(true).variations(true, false));
  // Bucketing is deterministic on user key
  const r1 = await client.variation('half-rollout', { key: 'user-1' }, false);
  const r2 = await client.variation('half-rollout', { key: 'user-1' }, false);
  expect(r1).toBe(r2);
});

File-based data source (alternative)

For fixture-shared tests:

const fileSource = LaunchDarkly.FileDataSource({
  paths: ['./tests/fixtures/ld-flags.json'],
});
const client = LaunchDarkly.init('sdk-test-key', {
  updateProcessor: fileSource,
  sendEvents: false,
});

The JSON file uses LaunchDarkly's flag-snapshot format. Useful for cross-language test fixtures.

Teardown

afterAll(async () => {
  await client.close();
});

Running

npm test

CI integration

jobs:
  ld-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v4
      - run: npm ci && npm test

No LAUNCHDARKLY_SDK_KEY needed in CI - TestData/file source replaces it.

Anti-patterns

Anti-pattern	Why it fails	Fix
Real SDK key in tests	Test traffic pollutes prod analytics	TestData / FileDataSource
`sendEvents: true` in tests	Event upload from CI	`sendEvents: false`
Multiple init per test	Slow init; leaks	One global client; TestData.update per test
Asserting exact variation result without `td.update`	Race vs SDK state	Always `update` first
Forgetting `await client.waitForInitialization()`	Race; variation returns default	Always wait
Sharing `td` across test files	Cross-test pollution	Per-file or per-test TestData
No teardown `client.close()`	Network handles leak	Always close in afterAll

Limitations

TestData is server-side-SDK-specific. Client-side SDKs have their own offline modes.
Doesn't validate LaunchDarkly's UI-side rollout math. Platform-side bucketing is separately tested.
No experiment-results validation. That's feature-flag-experiment-validator (in the qa-shift-right plugin).
FileDataSource schema is LaunchDarkly-internal. Refer to LD docs for the format.

References

LaunchDarkly SDK docs: launchdarkly.com/docs/sdk (opens in new window).
TestData data source guide: launchdarkly.com/docs/sdk/features/test-data-sources (opens in new window).
File-based data source: launchdarkly.com/docs/sdk/features/flags-from-files (opens in new window).
Deciding WHICH flag combinations to cover is a separate step this skill does not do: when code branches on several flags, load feature-flag-test-matrix-reference for the matrix, then come back here to pin each flag's value per test.
Sibling SDK umbrella: openfeature-sdk-testing (OpenFeature + Unleash / Flagsmith / GrowthBook references).
Cross-plugin: feature-flag-test-harness, ab-test-validity-checklist.

Related skills

ab-test-validity-checklist

Workflow skill that builds an A/B-test validity checklist from an experiment proposal, walking the canonical design-correctness gates - pre-registered OEC/power/guardrails, randomization unit + SRM check, assignment integrity, telemetry, peeking discipline, novelty/primacy, post-experiment SRM re-check - into a per-experiment checklist + sign-off form. Use when launching, auditing, or governing an experiment. For pitfall mechanics (guardrails, peeking) see experiment-results-interpreter's references; to read an already-valid result use experiment-results-interpreter; for per-SDK harness tests use experiment-sdk-testing - this gates DESIGN, not SDK code.

experiment-results-interpreter

Interprets the results of a valid online controlled experiment, one whose harness, SRM, and telemetry have already been confirmed. Covers the distinction between practical and statistical significance, reading confidence intervals instead of binary p-values, novelty and primacy week-over-week decay that causes post-ship reversion, interaction effects from concurrent experiments, Simpson's paradox in segmented results, and the ordered guardrail-check sequence required before a ship decision - with the deep methodology in references/: the peeking problem and its corrections (fixed-horizon, alpha-spending, always-valid mSPRT) in references/peeking.md, and guardrail-metric methodology (taxonomy, OEC relationship, pre-commitment, thresholds) in references/guardrails.md. Use when a data scientist or PM is ready to draw conclusions from an experiment, when designing a stop-early policy, or when declaring an experiment's guardrail set. Distinct from ab-test-validity-checklist (harness setup and SRM detection).

experiment-sdk-testing

Umbrella for experimentation-SDK test harnesses: the shared offline-datafile / hermetic-init pattern (commit a point-in-time flag/experiment config fixture, initialize the SDK with no network, pin arms per test, assert assignment integrity), with per-vendor references for Statsig (localMode + overrideGate), Optimizely (datafile + forced decisions), Split.io / Harness FME (localhost mode + features map or YAML fixture), Amplitude Experiment (local evaluation + bootstrap), and VWO (settings file + deterministic bucketing). Use when writing tests for application code instrumented with any of these five experimentation SDKs; for experiment DESIGN gates use ab-test-validity-checklist, and to read results use experiment-results-interpreter.

feature-flag-test-matrix-reference

Feature-flag test matrix design: the flag-state combinatorics problem (N flags × M variants × K user-segments = N×M×K test cases), the canonical coverage strategies (pairwise interaction coverage; default-only smoke; full matrix; risk-driven matrix), the workflow for building the coverage suite from a flag inventory (grep-based inventory, per-flag classification, PICT pairwise generation, per-cell test skeletons), the dedicated kill-switch test categories (references/killswitch.md: graceful degradation, fail-static default, kill latency, mid-flight consistency), and the flags-vs-experiments distinction. Use when designing the flag-test surface for a new project, building or auditing flag-test coverage, or authoring kill-switch tests.

openfeature-sdk-testing

Wraps OpenFeature (CNCF vendor-neutral SDK abstraction) testing patterns: the InMemoryProvider for hermetic tests without network calls, provider registration via OpenFeature.setProvider, the getBooleanValue/getBooleanDetails evaluation API with EvaluationDetails (value, variant, reason, errorCode), hooks for evaluation side-effects, and evaluation context for targeting-rule tests. Covers TypeScript, Java, and Python SDKs, plus per-vendor hermetic-bootstrap references for Unleash (bootstrap toggles), Flagsmith (offline LocalFileHandler), and GrowthBook (initSync payload). Use when writing tests for code that resolves feature flags through the OpenFeature SDK or the Unleash / Flagsmith / GrowthBook native SDKs; LaunchDarkly has its own skill (launchdarkly-testing).