Testland
Browse all skills & agents

openfeature-sdk-testing

Wraps OpenFeature (CNCF vendor-neutral SDK abstraction) testing patterns: the InMemoryProvider for hermetic tests without network calls, provider registration via OpenFeature.setProvider, the getBooleanValue/getBooleanDetails evaluation API with EvaluationDetails (value, variant, reason, errorCode), hooks for evaluation side-effects, and evaluation context for targeting-rule tests. Covers TypeScript, Java, and Python SDKs, plus per-vendor hermetic-bootstrap references for Unleash (bootstrap toggles), Flagsmith (offline LocalFileHandler), and GrowthBook (initSync payload). Use when writing tests for code that resolves feature flags through the OpenFeature SDK or the Unleash / Flagsmith / GrowthBook native SDKs; LaunchDarkly has its own skill (launchdarkly-testing).

Install with skills.sh (any agent)

npx skills add testland/qa --skill openfeature-sdk-testing
View source

openfeature-sdk-testing

Overview

The OpenFeature SDK ships an InMemoryProvider in every language that substitutes real flag-management infrastructure with in-process flag state, letting unit and integration tests run without any network call. The production evaluation path (targeting logic, type coercion, defaults) is exercised in full; only the data source is swapped. Per openfeature.dev/docs/reference/concepts/provider (opens in new window), "an application integrator can register one provider at a time."

Scope: this skill is the SDK-testing umbrella. The body covers the vendor-neutral OpenFeature layer teams adopt to keep application code decoupled from a specific provider; the vendor-native hermetic-bootstrap patterns (each an offline data-source variant of the same idea) live in references/: references/unleash.md (bootstrap toggles + custom strategies), references/flagsmith.md (offline LocalFileHandler + default_flag_handler), and references/growthbook.md (initSync payload + inline experiments). LaunchDarkly's TestData source has enough distinct surface for its own skill: launchdarkly-testing.

When to use

  • Tests for code that calls client.getBooleanValue() / get_boolean_value() through the OpenFeature SDK, regardless of which provider runs in production.
  • Tests asserting on evaluation details: variant name, reason code, or error code (e.g., FLAG_NOT_FOUND, TYPE_MISMATCH).
  • Tests that inject hook logic (telemetry, validation) exercised during flag evaluation.
  • Teams migrating between flag providers who want tests that survive the swap.
  • Assignment-integrity tests per ab-test-validity-checklist.

How to use

TypeScript/Node.js is the canonical language below. The Java and Python flow is identical; only the API names differ, listed in references/multi-language-and-spec.md.

TypeScript / Node.js

Install (per github.com/open-feature/js-sdk README (opens in new window)):

npm install --save @openfeature/server-sdk

Configure the InMemoryProvider with flag variants and a default variant:

import { OpenFeature, InMemoryProvider } from '@openfeature/server-sdk';

const flags = {
  'show-new-ui': {
    variants: { on: true, off: false },
    disabled: false,
    defaultVariant: 'on',
  },
  'checkout-v2': {
    variants: { enabled: true, disabled: false },
    disabled: false,
    defaultVariant: 'disabled',
  },
} as const;

await OpenFeature.setProvider(new InMemoryProvider(flags));
const client = OpenFeature.getClient();

Evaluate flags using the typed evaluation API (per openfeature.dev/docs/reference/concepts/evaluation-api (opens in new window)):

// Returns the resolved value; falls back to default on error
const enabled = await client.getBooleanValue('show-new-ui', false);

// Returns full EvaluationDetails
const details = await client.getBooleanDetails('show-new-ui', false);
// details.value    - the resolved boolean
// details.variant  - e.g. "on"
// details.reason   - e.g. "STATIC", "DEFAULT", "TARGETING_MATCH"
// details.errorCode- e.g. "FLAG_NOT_FOUND" when the flag is absent

Typical test pattern:

import { OpenFeature, InMemoryProvider } from '@openfeature/server-sdk';

describe('checkout feature', () => {
  let client: Client;

  beforeAll(async () => {
    await OpenFeature.setProvider(new InMemoryProvider({
      'checkout-v2': {
        variants: { enabled: true, disabled: false },
        disabled: false,
        defaultVariant: 'disabled',
      },
    }));
    client = OpenFeature.getClient();
  });

  afterAll(() => OpenFeature.close());

  it('returns false when flag defaults to disabled', async () => {
    const value = await client.getBooleanValue('checkout-v2', false);
    expect(value).toBe(false);
  });

  it('returns details with reason STATIC for static flags', async () => {
    const details = await client.getBooleanDetails('checkout-v2', false);
    expect(details.reason).toBe('STATIC');
  });

  it('returns FLAG_NOT_FOUND error code for unknown flag', async () => {
    const details = await client.getBooleanDetails('unknown-flag', false);
    expect(details.errorCode).toBe('FLAG_NOT_FOUND');
  });
});

Java and Python install/configure/evaluate blocks live in references/multi-language-and-spec.md.

EvaluationDetails and reason codes

EvaluationDetails carries value, flagKey, variant, reason, errorCode, errorMessage, and flagMetadata. Canonical reason values include STATIC, DEFAULT, TARGETING_MATCH, SPLIT, CACHED, DISABLED, UNKNOWN, STALE, ERROR; canonical error codes include FLAG_NOT_FOUND, TYPE_MISMATCH, TARGETING_KEY_MISSING, and PROVIDER_NOT_READY. The full field table with spec requirement numbers is in references/multi-language-and-spec.md.

Evaluation context for targeting-rule tests

Evaluation context is "a container for arbitrary contextual data that can be used as a basis for dynamic evaluation" (per openfeature.dev/docs/reference/concepts/evaluation-context (opens in new window)). The targeting key is a unique identifier (user ID, session ID) that providers use for deterministic bucketing. Custom attributes carry additional data (email, plan, region).

Context can be set at three levels: global (via the API object), client, and per invocation. Lower levels override duplicate keys from higher levels.

// TypeScript - per-invocation context for targeting-rule tests
const ctx = { targetingKey: 'user-42', email: 'user@example.com' };
const details = await client.getBooleanDetails('beta-access', false, ctx);
expect(details.reason).toBe('TARGETING_MATCH');

Java and Python context builders (ImmutableContext, EvaluationContext) are in references/multi-language-and-spec.md.

Hooks for test-time side-effects

Hooks intercept the flag evaluation lifecycle at four stages - before (can modify evaluation context), after (validate the returned value), error (on resolution failure), and finally (unconditionally). The stage table and execution order (per specification Requirement 4.4.2) are in references/multi-language-and-spec.md.

Register hooks at global, client, or invocation level:

// Global hook - runs for every flag evaluation
OpenFeature.addHooks({
  before(ctx) {
    // ctx carries flagKey, flagValueType, defaultValue, evaluationContext
    console.log(`Evaluating ${ctx.flagKey}`);
  },
  after(ctx, details) {
    // details is the EvaluationDetails for this evaluation
    expect(details.errorCode).toBeUndefined();
  },
  error(ctx, err) {
    console.error(`Flag ${ctx.flagKey} failed: ${err.message}`);
  },
});

Test use case: attach an after hook to assert that no evaluation returns an error code, surfacing FLAG_NOT_FOUND regressions across the entire test run without asserting each flag individually.

Anti-patterns

Anti-patternWhy it failsFix
Registering a real (networked) provider in unit testsNetwork calls; non-deterministic; slowInMemoryProvider for unit tests
Evaluating without setProviderAndWait / await setProviderReturns default with PROVIDER_NOT_READY error code before initAlways await provider readiness
Sharing a single InMemoryProvider instance across test filesCross-test state pollutionCreate a fresh provider per describe block
Asserting only value; ignoring reason and errorCodeHides fallback-to-default failures (flag absent, type mismatch)Assert details.reason and details.errorCode explicitly
Testing provider internals (variant weighting, targeting logic)That is the provider's responsibility, not the application'sTest what the application does with the evaluated value
Omitting OpenFeature.close() in teardownLeaks provider state and background threadsAlways call close() / shutdown() in afterAll

Limitations

  • InMemoryProvider does not replicate production targeting rules. To test that a real provider evaluates a segment correctly, write an integration test against that provider's native test harness - launchdarkly-testing, or the vendor references in references/ (unleash.md, flagsmith.md, growthbook.md).
  • Client-side OpenFeature SDKs (@openfeature/web-sdk) use a different setContext pattern; this skill targets server-side SDKs.
  • Hook before stages can modify the evaluation context in some SDK versions but the specification marks this as optional behavior - verify against your SDK version.
  • InMemoryProvider does not fire PROVIDER_CHANGED events on putIfAbsent in all SDK versions; check release notes when relying on event-driven updates in tests.

References

Flagsmith test modes, anti-patterns, and limitations

View source (opens in new window)

Flagsmith test modes, anti-patterns, and limitations

Deeper detail for flagsmith.md (opens in new window). Offline mode (covered in the skill spine) is the default for tests; the modes below are secondary.

Local-evaluation mode

Polls the Flagsmith API periodically and evaluates flags locally between refreshes (no per-request network, but not zero network):

flagsmith = Flagsmith(
    environment_key="server-key",
    enable_local_evaluation=True,
    environment_refresh_interval_seconds=60,
)

Local mode polls; offline mode does not. For tests, offline is usually preferred.

default_flag_handler - per-flag mock

Programmatic fallback used when the offline environment.json does not have the flag under test yet:

from flagsmith import Flagsmith
from flagsmith.models import DefaultFlag

def default_flag_handler(feature_name: str) -> DefaultFlag:
    if feature_name == "secret_button":
        return DefaultFlag(enabled=False, value='{"colour": "#b8b8b8"}')
    return DefaultFlag(enabled=False, value=None)

flagsmith = Flagsmith(
    environment_key="test-key",
    default_flag_handler=default_flag_handler,
)

Useful when the offline environment.json does not have the flag you are testing yet.

Anti-patterns

Anti-patternWhy it failsFix
Production env_key in testsReal API calls + analytics pollutionoffline_mode + LocalFileHandler
environment.json not committedTest flakes when prod changesCommit; refresh deliberately
Mixing offline_mode + local_evaluation_modeConflicting; one takes precedencePick one
default_flag_handler returns DefaultFlag with no valueTests for value-based flags fail silentlyAlways set value
Skipping flagsmith.get_identity_flags for identity-scoped testsBypasses per-user logicUse identity API
Per-test new Flagsmith clientSlow initSession-scoped fixture

Limitations

  • environment.json is point-in-time. Drift invisible.
  • default_flag_handler only fires for missing flags in local-eval mode. In offline mode it can be used for fallback.
  • No granular per-user override API. Use traits + segments via the environment.json.
  • Doesn't validate Flagsmith's own logic. Platform-side evaluation is separate.

Flagsmith SDK testing

Wraps Flagsmith server-side SDK testing patterns so tests run without calling the Flagsmith API: offline mode with LocalFileHandler + a downloaded environment.json snapshot, local-evaluation mode (no per-request network), and default_flag_handler for per-feature mocked fallbacks, via the get_environment_flags / get_identity_flags evaluation paths.

Flagsmith (open-source, also SaaS at flagsmith.com) supports three test-friendly modes per docs.flagsmith.com/clients/server-side (opens in new window):

  1. Offline mode with LocalFileHandler - loads a downloaded environment.json snapshot; zero network.
  2. Local-evaluation mode - fetches environment + flags periodically, evaluates locally without per-request network.
  3. Default flag handler - programmatic fallback for any flag (mock-flag-only mode).

Offline mode is the default choice for tests; local-evaluation and default_flag_handler detail live in flagsmith-modes.md (opens in new window).

How to use

  1. Install the SDK: pip install flagsmith (Python) or npm install --save-dev flagsmith-nodejs (Node).
  2. Download the environment snapshot with the Flagsmith CLI (command below) and commit it as a test fixture.
  3. Construct the client in offline mode with LocalFileHandler pointed at that fixture, so tests make zero network calls.
  4. For flags not yet in the snapshot, register a default_flag_handler returning a DefaultFlag with the value under test (see flagsmith-modes.md (opens in new window)).
  5. Evaluate via get_environment_flags() for environment-scoped flags, or get_identity_flags(identifier, traits=...) for per-user flags.
  6. Assert on is_feature_enabled(...) and get_feature_value(...), and add an integrity test that the same identity resolves consistently.
  7. Verify: with offline_mode=True and no environment_key set, get_environment_flags() must return the fixture's flags with zero network calls (run the suite with the machine offline or under a network-blocking fixture to confirm). If it raises or attempts an API call, the LocalFileHandler environment_document_path is wrong or offline_mode is unset - fix the path and re-run.
  8. Wire pytest tests/flagsmith/ into CI - offline mode needs no env vars.

Offline mode with LocalFileHandler

Per docs.flagsmith.com (opens in new window):

from flagsmith import Flagsmith
from flagsmith.offline_handlers import LocalFileHandler

local_file_handler = LocalFileHandler(environment_document_path="tests/fixtures/flagsmith-environment.json")

flagsmith = Flagsmith(offline_mode=True, offline_handler=local_file_handler)

Download the environment.json via the Flagsmith CLI, then commit it (refresh deliberately):

flagsmith environment-document --api-key=<server-key> --output=tests/fixtures/flagsmith-environment.json

Evaluate flags

def test_environment_flag():
    flags = flagsmith.get_environment_flags()
    assert flags.is_feature_enabled("secret_button") is False
    assert flags.get_feature_value("secret_button") == '{"colour": "#b8b8b8"}'

def test_identity_flag():
    flags = flagsmith.get_identity_flags(
        identifier="user@example.com",
        traits={"plan": "premium"},
    )
    assert flags.is_feature_enabled("premium_feature") is True

Integrity test - the same identity must resolve consistently:

def test_same_identity_consistent():
    f1 = flagsmith.get_identity_flags("u1")
    f2 = flagsmith.get_identity_flags("u1")
    assert f1.is_feature_enabled("flag-x") == f2.is_feature_enabled("flag-x")

CI integration

No env vars needed in offline mode:

jobs:
  flagsmith-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-python@v5
      - run: pip install flagsmith
      - run: pytest tests/flagsmith/

Worked example

A service reads a secret_button flag (boolean plus a JSON colour value) and a premium_feature flag gated on a plan trait, tested fully offline:

  1. Download the environment document and commit it (step 2 above).
  2. Build the offline client with LocalFileHandler (offline-mode snippet above).
  3. secret_button is not in the snapshot yet, so register a default_flag_handler for it (see flagsmith-modes.md (opens in new window)).
  4. Assert on get_environment_flags() and get_identity_flags(...) (evaluate snippet above).

Result: pytest tests/flagsmith/ runs green in CI with zero network access and no analytics pollution.

Sources

GrowthBook SDK testing

View source (opens in new window)

GrowthBook SDK testing

Wraps GrowthBook Node SDK testing patterns: GrowthBookClient initialization with direct payload (initSync; no network), isOn / getFeatureValue / evalFeature, scoped instances (createScopedInstance) for per-request user context, inline experiment (runInlineExperiment) tests, and tracking-callback assertion patterns.

Per docs.growthbook.io/lib/node (opens in new window), the GrowthBook Node SDK's GrowthBookClient supports an initSync({ payload }) pattern that fully bypasses the network - pass the feature definitions directly. createScopedInstance lets each test (or request) bind its own user context cleanly.

Install

npm install --save-dev @growthbook/growthbook

Initialize with payload (offline)

Per docs.growthbook.io (opens in new window):

import { GrowthBookClient } from '@growthbook/growthbook';

const gbClient = new GrowthBookClient().initSync({
  payload: {
    features: {
      'show-new-ui': { defaultValue: true },
      'experiment-x': {
        defaultValue: false,
        rules: [{ condition: { id: 'test-user-1' }, force: true }],
      },
    },
  },
});

initSync is purpose-built for tests - no async wait.

isOn / getFeatureValue

test('flag on', () => {
  const userContext = { attributes: { id: 'user-1' } };
  expect(gbClient.isOn('show-new-ui', userContext)).toBe(true);
});

test('typed feature value', () => {
  const userContext = { attributes: { id: 'user-1' } };
  const color = gbClient.getFeatureValue('button-color', 'blue', userContext);
  expect(['blue', 'red', 'green']).toContain(color);
});

Scoped instance per test

test('user-specific evaluation', () => {
  const instance = gbClient.createScopedInstance({
    attributes: { id: 'user-1', plan: 'premium' },
  });
  expect(instance.isOn('premium-feature')).toBe(true);
});

Avoids passing context everywhere.

Inline experiments

test('inline experiment returns one variant', () => {
  const userContext = { attributes: { id: 'user-1' } };
  const { value } = gbClient.runInlineExperiment({
    key: 'my-experiment',
    variations: ['red', 'blue', 'green'],
    coverage: 1.0,
    weights: [0.33, 0.34, 0.33],
  }, userContext);
  expect(['red', 'blue', 'green']).toContain(value);
});

Tracking-callback assertions

test('experiment tracking fires', () => {
  const tracked: any[] = [];
  const client = new GrowthBookClient({
    trackingCallback: (exp, result, ctx) => {
      tracked.push({ key: exp.key, variation: result.key });
    },
  }).initSync({ payload: {} });

  client.runInlineExperiment(
    { key: 'exp-x', variations: [0, 1] },
    { attributes: { id: 'user-1' } }
  );

  expect(tracked).toHaveLength(1);
  expect(tracked[0].key).toBe('exp-x');
});

Feature-usage callback

const evaluated: any[] = [];

test('feature usage logged', () => {
  const userContext = {
    attributes: { id: 'user-1' },
    onFeatureUsage: (key: string, result: any) => {
      evaluated.push({ key, value: result.value });
    },
  };

  gbClient.evalFeature('feature-x', userContext);
  expect(evaluated).toContainEqual(expect.objectContaining({ key: 'feature-x' }));
});

TypeScript-strict feature contract

interface AppFeatures {
  'button-color': string;
  'font-size': number;
  'newForm': boolean;
}

const gbClient = new GrowthBookClient<AppFeatures>({}).initSync({ payload: {} });

const color = gbClient.getFeatureValue('button-color', 'blue', ctx);  // typed
// gbClient.isOn('buton-color', ctx);  // typo → compile error

CI integration

Fully offline; no GrowthBook key needed:

jobs:
  growthbook-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v4
      - run: npm ci && npm test

Anti-patterns

Anti-patternWhy it failsFix
Live apiHost in testsNetwork requests; flakyinitSync({ payload })
Skipping tracking-callback assertionExposure-event regressions silentPer-test callback + assert
Sharing scopedInstance across testsCross-test statePer-test create
TypeScript any-typed featuresLose compile-time safetyGeneric AppFeatures
coverage: 0.1 in test without large NNot enough samples to see all variationsUse coverage: 1.0 for deterministic tests
Missing user.id in contextBucketing degenerateAlways pass attributes.id
Tests assume default weights is 50/50Default behaviour driftsExplicit weights

Limitations

  • Payload is point-in-time. Drift from GrowthBook UI invisible.
  • No targeting-rule-test helper beyond writing conditions. Complex targeting tests need many fixture variants.
  • Tracking-callback is synchronous. Async exposure logging needs a separate test.
  • TypeScript strict-mode required for the typed feature contract.

Sources

OpenFeature testing: Java / Python SDKs and specification tables

View source (opens in new window)

OpenFeature testing: Java / Python SDKs and specification tables

Companion reference for openfeature-sdk-testing. The SKILL.md spine shows the full TypeScript/Node.js flow; the flow is identical here, only the API names differ.

Java

Install (Maven, per github.com/open-feature/java-sdk README (opens in new window)):

<dependency>
  <groupId>dev.openfeature</groupId>
  <artifactId>sdk</artifactId>
  <version>1.20.2</version>
</dependency>

Configure the InMemoryProvider:

import dev.openfeature.sdk.OpenFeatureAPI;
import dev.openfeature.sdk.Client;
import dev.openfeature.sdk.providers.memory.Flag;
import dev.openfeature.sdk.providers.memory.InMemoryProvider;

Map<String, Flag<?>> flags = new HashMap<>();
flags.put("show-new-ui", Flag.builder()
    .variant("on", true)
    .variant("off", false)
    .defaultVariant("on")
    .build());

OpenFeatureAPI api = OpenFeatureAPI.getInstance();
api.setProviderAndWait(new InMemoryProvider(flags));
Client client = api.getClient();

Evaluate flags:

boolean enabled = client.getBooleanValue("show-new-ui", false);

FlagEvaluationDetails<Boolean> details =
    client.getBooleanDetails("show-new-ui", false);
// details.getValue()    - resolved value
// details.getVariant()  - "on" or "off"
// details.getReason()   - "STATIC", "DEFAULT", etc.
// details.getErrorCode()- ErrorCode.FLAG_NOT_FOUND, TYPE_MISMATCH, etc.

Per-invocation evaluation context:

Map<String, Value> attrs = new HashMap<>();
attrs.put("email", new Value("user@example.com"));
EvaluationContext ctx = new ImmutableContext("user-42", attrs);
boolean value = client.getBooleanValue("beta-access", false, ctx);

Python

Install (per github.com/open-feature/python-sdk README (opens in new window)):

pip install openfeature-sdk==0.10.0

Configure the InMemoryProvider:

from openfeature import api
from openfeature.provider.in_memory_provider import InMemoryFlag, InMemoryProvider

flags = {
    "show-new-ui": InMemoryFlag(
        default_variant="on",
        variants={"on": True, "off": False}
    ),
}

api.set_provider_and_wait(InMemoryProvider(flags))
client = api.get_client()

Evaluate flags:

enabled = client.get_boolean_value("show-new-ui", False)

details = client.get_boolean_details("show-new-ui", False)
# details.value       - resolved value
# details.variant     - "on" / "off"
# details.reason      - "STATIC", "DEFAULT", etc.
# details.error_code  - "FLAG_NOT_FOUND", "TYPE_MISMATCH", etc.

Per-invocation evaluation context:

from openfeature.evaluation_context import EvaluationContext
ctx = EvaluationContext(targeting_key="user-42",
                        attributes={"email": "user@example.com"})
details = client.get_boolean_details("beta-access", False, ctx)

EvaluationDetails and reason codes

Per the OpenFeature specification (openfeature.dev/specification/sections/flag-evaluation (opens in new window), Requirements 1.4.3-1.4.15), EvaluationDetails carries:

FieldTypeMeaning
valueTResolved flag value (spec req. 1.4.3)
flagKeystringThe requested flag identifier (1.4.5)
variantstringProvider-supplied variant name (1.4.6)
reasonstringResolution rationale (1.4.7)
errorCodeenumFailure classification (1.4.8)
errorMessagestringOptional context for the error (1.4.13)
flagMetadatamapImmutable provider-supplied data (1.4.14)

Canonical reason values (per openfeature.dev/specification/sections/providers (opens in new window), Requirement 2.2.5): STATIC, DEFAULT, TARGETING_MATCH, SPLIT, CACHED, DISABLED, UNKNOWN, STALE, ERROR.

Canonical error codes include FLAG_NOT_FOUND, TYPE_MISMATCH, PARSE_ERROR, TARGETING_KEY_MISSING, INVALID_CONTEXT, GENERAL, PROVIDER_NOT_READY, PROVIDER_FATAL.

Hook lifecycle stages and execution order

Per openfeature.dev/docs/reference/concepts/hooks (opens in new window) and the specification (openfeature.dev/specification/sections/hooks (opens in new window), Requirement 4.3.1-4.3.8), the four stages are:

StageRuns when
beforeBefore flag resolution; can modify evaluation context
afterAfter successful resolution; can validate the returned value
errorOn resolution failure or unhandled before-hook error
finallyUnconditionally after all other stages

Execution order for before: API - Client - Invocation. For after, error, finally: reverse order (Invocation - Client - API), per specification Requirement 4.4.2.

Unleash SDK testing

Wraps Unleash (Open Source / SaaS) SDK testing patterns: bootstrap with a static toggles array (no network), the test mode (disableMetrics + disablePolling), the custom strategy testing pattern (implement a Strategy class + assert isEnabled), and assignment-integrity tests.

Install

npm install --save-dev unleash-client
pip install UnleashClient

Bootstrap with toggles (offline)

import { initialize } from 'unleash-client';

const unleash = initialize({
  url: 'http://localhost:4242/api/',
  appName: 'test-app',
  disableMetrics: true,        // No metrics upload
  disablePolling: true,        // No background polling
  bootstrap: {
    data: [
      {
        name: 'show-new-ui',
        enabled: true,
        strategies: [
          { name: 'default' },
        ],
      },
    ],
  },
});

// Wait for initialization
await new Promise<void>((resolve) => unleash.once('synchronized', resolve));

The bootstrap.data array is the SDK's initial flag state; since polling is disabled, that state persists for the test session.

isEnabled tests

test('flag enabled', () => {
  expect(unleash.isEnabled('show-new-ui')).toBe(true);
});

test('flag with context', () => {
  expect(unleash.isEnabled('premium-only', { userId: 'u1', properties: { tier: 'premium' } })).toBe(true);
});

Custom strategy

Unleash's extensibility: custom strategies implement an isEnabled(parameters, context) method.

import { Strategy } from 'unleash-client';

class TenantStrategy extends Strategy {
  constructor() { super('tenantStrategy'); }
  isEnabled(parameters: any, context: any): boolean {
    const allowedTenants = (parameters.tenants ?? '').split(',');
    return allowedTenants.includes(context.tenantId);
  }
}

const unleash = initialize({
  url: '...',
  appName: 'test',
  strategies: [new TenantStrategy()],
  bootstrap: {
    data: [
      {
        name: 'flag-x',
        enabled: true,
        strategies: [{ name: 'tenantStrategy', parameters: { tenants: 'A,B' } }],
      },
    ],
  },
});

test('strategy allows tenant A', () => {
  expect(unleash.isEnabled('flag-x', { tenantId: 'A' })).toBe(true);
});

test('strategy rejects tenant C', () => {
  expect(unleash.isEnabled('flag-x', { tenantId: 'C' })).toBe(false);
});

Percentage-rollout determinism

const unleash = initialize({
  // ...
  bootstrap: {
    data: [
      {
        name: 'gradual-rollout',
        enabled: true,
        strategies: [{ name: 'flexibleRollout', parameters: { rollout: '50', stickiness: 'userId' } }],
      },
    ],
  },
});

test('rollout deterministic per user', () => {
  const r1 = unleash.isEnabled('gradual-rollout', { userId: 'u1' });
  const r2 = unleash.isEnabled('gradual-rollout', { userId: 'u1' });
  expect(r1).toBe(r2);
});

Teardown

afterAll(() => unleash.destroy());

CI integration

Fully offline - no Unleash server URL needed since polling is disabled:

jobs:
  unleash-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v4
      - run: npm ci && npm test

Anti-patterns

Anti-patternWhy it failsFix
disableMetrics: false in CISpurious metrics POSTsAlways disableMetrics: true in tests
disablePolling: false without test-only URLNetwork calls; CI flakesAlways disable polling in offline tests
bootstrap.data is staleDrift from prod definitionsPull from Unleash periodically; commit fixture
Custom strategies not unit-testedLogic bugs in the strategy itselfTest the strategy class in isolation too
Skipping synchronized event waitRace: isEnabled returns defaultAlways await synchronized
Forgetting unleash.destroy()Goroutine / timer leakAlways destroy in afterAll
Tests use real Unleash serverSlow; flaky if server downBootstrap mode

Limitations

  • Bootstrap is point-in-time. Drift between fixture + Unleash UI is invisible.
  • Custom strategies tested in isolation lose context. Integration test with the full SDK still needed.
  • Metric-side tests need real server. If you're testing the metrics export path, bootstrap mode is wrong.
  • Stickiness is parameter-driven. A misconfigured stickiness parameter shows up in production, not in tests.

Sources

Related skills

ab-test-validity-checklist

Workflow skill that builds an A/B-test validity checklist from an experiment proposal, walking the canonical design-correctness gates - pre-registered OEC/power/guardrails, randomization unit + SRM check, assignment integrity, telemetry, peeking discipline, novelty/primacy, post-experiment SRM re-check - into a per-experiment checklist + sign-off form. Use when launching, auditing, or governing an experiment. For pitfall mechanics (guardrails, peeking) see experiment-results-interpreter's references; to read an already-valid result use experiment-results-interpreter; for per-SDK harness tests use experiment-sdk-testing - this gates DESIGN, not SDK code.

experiment-results-interpreter

Interprets the results of a valid online controlled experiment, one whose harness, SRM, and telemetry have already been confirmed. Covers the distinction between practical and statistical significance, reading confidence intervals instead of binary p-values, novelty and primacy week-over-week decay that causes post-ship reversion, interaction effects from concurrent experiments, Simpson's paradox in segmented results, and the ordered guardrail-check sequence required before a ship decision - with the deep methodology in references/: the peeking problem and its corrections (fixed-horizon, alpha-spending, always-valid mSPRT) in references/peeking.md, and guardrail-metric methodology (taxonomy, OEC relationship, pre-commitment, thresholds) in references/guardrails.md. Use when a data scientist or PM is ready to draw conclusions from an experiment, when designing a stop-early policy, or when declaring an experiment's guardrail set. Distinct from ab-test-validity-checklist (harness setup and SRM detection).

experiment-sdk-testing

Umbrella for experimentation-SDK test harnesses: the shared offline-datafile / hermetic-init pattern (commit a point-in-time flag/experiment config fixture, initialize the SDK with no network, pin arms per test, assert assignment integrity), with per-vendor references for Statsig (localMode + overrideGate), Optimizely (datafile + forced decisions), Split.io / Harness FME (localhost mode + features map or YAML fixture), Amplitude Experiment (local evaluation + bootstrap), and VWO (settings file + deterministic bucketing). Use when writing tests for application code instrumented with any of these five experimentation SDKs; for experiment DESIGN gates use ab-test-validity-checklist, and to read results use experiment-results-interpreter.

feature-flag-test-matrix-reference

Feature-flag test matrix design: the flag-state combinatorics problem (N flags × M variants × K user-segments = N×M×K test cases), the canonical coverage strategies (pairwise interaction coverage; default-only smoke; full matrix; risk-driven matrix), the workflow for building the coverage suite from a flag inventory (grep-based inventory, per-flag classification, PICT pairwise generation, per-cell test skeletons), the dedicated kill-switch test categories (references/killswitch.md: graceful degradation, fail-static default, kill latency, mid-flight consistency), and the flags-vs-experiments distinction. Use when designing the flag-test surface for a new project, building or auditing flag-test coverage, or authoring kill-switch tests.

launchdarkly-testing

Wraps LaunchDarkly server-side SDK testing patterns: TestData data source for hermetic tests (no network), file-based data source for fixture-driven tests, flag override patterns (TestData.update for per-test flag values), and assignment-integrity tests. Use when writing tests for code that uses LaunchDarkly flags; to decide which flag combinations those tests should cover in the first place, see feature-flag-test-matrix-reference.