Unreal Automation System Skill for Claude Code

Overview

This skill wraps the C++ Automation Test Framework (UE 4.x / 5.x) plus the two most commonly composed sub-systems. Per Epic's Automation Test Framework documentation (opens in new window), the framework spans five test categories (Unit, Feature, Smoke, Content Stress, Screenshot Comparison) and supports multiple authoring styles (traditional, BDD Spec, UI Driver, Functional, Python / Blueprint).

Automation Spec - BDD-style Describe / It / BeforeEach per Automation Spec documentation (opens in new window).
Automation Driver - UI input simulation per Automation Driver documentation (opens in new window).

The full authoring code (macros, Spec, Driver) lives in references/authoring-macros-and-apis.md; running, report parsing, and CI wiring live in references/running-and-reporting.md. This file is the decision surface.

Composes with:

platform-cert-overview-reference for the canonical six test categories Unreal's five categories map to, and for cert-gated requirements automation tests should cover.
multiplayer-state-machine-coverage for replication / dedicated-server state coverage authored as automation tests.

When to use

Unit under test is C++ code in an Unreal Engine project (UE 4.x or UE 5.x) that needs the engine runtime, editor APIs, or UMG UI surface.
You want CI-runnable tests via Unreal's command-line Automation entry point.
You need BDD-style readable specs (Automation Spec) or scripted UI input simulation (Automation Driver) on top of plain assertion tests.

For Python / Blueprint editor tests outside C++, see Editor Automation in Unreal Engine (opens in new window). For end-to-end pipeline / build-farm orchestration, use Gauntlet (opens in new window) on top of the framework this skill covers.

How to use

Pick the category from the table below (Unit / Feature / Smoke / Content Stress / Screenshot Comparison) and the matching EAutomationTestFlags filter + application-context mask.
Pick the authoring style - plain RunTest macro, data-driven complex macro, BDD Spec, or Automation Driver for UI - and lift the pattern from references/authoring-macros-and-apis.md.
Name the test path as dot-separated segments (MyGame.Health.Damage_…); the tree keys the Session Frontend.
Run in-editor via Session Frontend to iterate, then run headless via UnrealEditor-Cmd.exe … -ExecCmds="Automation RunTests …" for CI.
Parse the -ReportOutputPath JSON - treat any "state": "Fail" as a failed build; full schema in references/running-and-reporting.md.
Split the CI matrix - SmokeFilter on PRs (sub-second), ProductFilter | StressFilter plus screenshot comparison nightly.

Test categories and flags

Per the Automation Test Framework page (opens in new window), Epic's five categories are:

Category	Purpose
Unit	"API level verification tests."
Feature	"System-level tests that verify such things as PIE, in-game stats, and changing resolution."
Smoke	Tests that "complete within 1 second" and run automatically.
Content Stress	"More thorough testing of a particular system to avoid crashes."
Screenshot Comparison	For comparing renders "between versions or builds".

Each test declares flags from EAutomationTestFlags that mix:

Filter - SmokeFilter, EngineFilter, ProductFilter, PerfFilter, StressFilter, NegativeFilter.
Application context - EditorContext, ClientContext, ServerContext, CommandletContext, plus the convenience mask ApplicationContextMask.

Typical combo for a product-level test runnable in editor / client / server contexts: EAutomationTestFlags::ProductFilter | EAutomationTestFlags::ApplicationContextMask.

Authoring styles

Pick the style that matches the unit under test, then lift the full pattern from references/authoring-macros-and-apis.md:

Style	Macro / API	Use when
Simple test	`IMPLEMENT_SIMPLE_AUTOMATION_TEST` + `RunTest`	One assertion body over plain C++
Data-driven	`IMPLEMENT_COMPLEX_AUTOMATION_TEST` + `GetTests`	One sub-test per enumerated row (assets, configs)
Multi-frame	`ADD_LATENT_AUTOMATION_COMMAND`	Test must yield to the tick loop across frames
BDD Spec	`DEFINE_SPEC` / `BEGIN_DEFINE_SPEC` + `Describe` / `It` / `LatentIt`	Readable specs; `.spec.cpp` files; async via `FDoneDelegate`
UI Driver	`IAutomationDriverModule::Get().CreateDriver()` + `By::Id`	Simulate cursor / click / type on UMG; runs off the GameThread

Running

Iterate in-editor via Window → Test Automation (Session Frontend), then run headless for CI:

UnrealEditor-Cmd.exe MyGame.uproject \
    -ExecCmds="Automation RunTests MyGame.Inventory; Quit" \
    -unattended -nopause -testexit="Automation Test Queue Empty" \
    -ReportOutputPath="artifacts/automation" \
    -log

Command variants, the -ReportOutputPath JSON schema, Gauntlet, and a full GitHub Actions job are in references/running-and-reporting.md.

Worked example

Goal: cover an inventory system's stacking rule as a CI-gating BDD spec, plus a menu-close UI check.

Category + flags. Stacking logic is pure C++ product logic, so ProductFilter | ApplicationContextMask; it runs in well under a second, so a second SmokeFilter copy joins the PR job.
Authoring. Write FInventorySpec with DEFINE_SPEC and a Describe("AddItem") block holding two It("should …") cases - one asserting count increases by the stack amount, one asserting an over-cap add is rejected (pattern in references/authoring-macros-and-apis.md). Save it as InventorySpec.spec.cpp.
UI check. Add FMenuDriverSpec with the Driver enabled in BeforeEach on TaskGraphMainThread, the It running on EAsyncExecution::ThreadPool, By::Id("SubmitButton") clicked, and Disable() in AfterEach.
Run headless. UnrealEditor-Cmd.exe MyGame.uproject -ExecCmds="Automation RunTests MyGame.Inventory; Quit" -unattended -nopause -ReportOutputPath="artifacts/automation" -log.
Gate. CI reads artifacts/automation/index.json; the over-cap It shows "state": "Fail" when the rule regresses, so the job fails the build. Nightly re-runs the same specs under ProductFilter | StressFilter.

Anti-patterns

Anti-pattern	Why it fails	Fix
All tests as `SmokeFilter`	Smoke tests "complete within 1 second" per framework docs (opens in new window) - long tests break the smoke contract	Use `ProductFilter` for tests > 1 s
Running Automation Driver on GameThread	API "cannot run on the GameThread" per Driver docs (opens in new window)	Use `EAsyncExecution::ThreadPool` on the driver-using `It`
Spec without `It` descriptions starting with "should"	Runner output reads poorly	Per Spec docs (opens in new window), start descriptions with "should"
Trusting `By::Path` locators	"Brittle" per Driver docs (opens in new window)	Prefer `By::Id` with tagged metadata
`Sleep`-style waits in latent commands	Flaky under CI load	Use `FDoneDelegate` (LatentIt) or custom `IAutomationLatentCommand::Update()` polling
Spec test file without `.spec.cpp` extension	Build system may not pick it up	Per Spec docs (opens in new window), use `.spec.cpp` suffix and no "Test" in filename
Cloning non-thread-safe shared pointers in async `It`	Crash under threadpool	Per Driver docs (opens in new window), cache them on the test class
No `BeforeEach` / `AfterEach` cleanup of `IAutomationDriverModule`	Driver state leaks between specs	Pair `Enable()` / `Disable()` calls in `BeforeEach` / `AfterEach` per Driver docs (opens in new window)

Limitations

C++ build required. Tests live in C++ modules; pure-content Blueprint projects need a code module added to use this framework. (Pure Blueprint projects can use Blueprint Functional Tests in-level - see Epic's docs on Functional Testing.)
No common exit code definition comparable to Unity's caveat - parse the -ReportOutputPath JSON or scrape the log for LogAutomationController Fail lines.
Spec parameterised tests are loop-generated per Spec docs (opens in new window); there is no [TestCase] analogue from NUnit. The framework is NUnit-inspired but not NUnit-derived (unlike Unity's UTF).
Screenshot comparison baseline storage + tolerance configuration is engine-version-specific - consult per-version Epic docs.
Editor-context tests cannot run in dedicated-server only builds - declare EditorContext on those tests; client / server tests need their own flag set.
Documentation source. The dev.epicgames.com docs (opens in new window) are the public mirror; deeper detail (full macro implementations, precise JSON report schema) lives in the engine source under Engine/Source/Runtime/AutomationController/ and Engine/Source/Developer/AutomationMessages/ - partners with engine source access should consult those for authoritative details.

References

Automation Test Framework - dev.epicgames.com (opens in new window).
Automation Spec (BDD) - dev.epicgames.com (opens in new window).
Automation Driver (UI input) - dev.epicgames.com (opens in new window).
Full authoring code (macros, Spec, Driver): references/authoring-macros-and-apis.md.
Running, report parsing, and CI wiring: references/running-and-reporting.md.
Category mapping + cert-gated requirements: platform-cert-overview-reference.
Replication / dedicated-server coverage: multiplayer-state-machine-coverage.

Reference

Authoring macros and APIs

View source

Deep reference for unreal-automation-system SKILL.md - the full macro, Automation Spec, and Automation Driver code. Consult after picking the test category and flags in the spine, when writing the actual test body.

Simple automation test

Per Automation Test Framework documentation (opens in new window), the canonical macro pair is IMPLEMENT_SIMPLE_AUTOMATION_TEST + RunTest:

#include "Misc/AutomationTest.h"

IMPLEMENT_SIMPLE_AUTOMATION_TEST(
    FHealthComponentDamageTest,
    "MyGame.Health.Damage_DeductsCorrectAmount",
    EAutomationTestFlags::ProductFilter |
        EAutomationTestFlags::ApplicationContextMask)

bool FHealthComponentDamageTest::RunTest(const FString& Parameters)
{
    UHealthComponent* Health = NewObject<UHealthComponent>();
    Health->Initialize(/* MaxHealth */ 100.f);

    Health->ApplyDamage(35.f);

    TestEqual(TEXT("Current HP after 35 damage"),
              Health->GetCurrent(),
              65.f);
    return true;
}

Naming convention: the second macro argument ("MyGame.Health.Damage_…") is the test path shown in the Session Frontend. Dot-separated segments build a tree.

Complex automation test (data-driven)

IMPLEMENT_COMPLEX_AUTOMATION_TEST generates a sub-test per row returned from GetTests:

IMPLEMENT_COMPLEX_AUTOMATION_TEST(
    FAllAssetsLoadCleanlyTest,
    "MyGame.Assets.LoadCleanly",
    EAutomationTestFlags::ProductFilter |
        EAutomationTestFlags::ApplicationContextMask)

void FAllAssetsLoadCleanlyTest::GetTests(
    TArray<FString>& OutBeautifiedNames,
    TArray<FString>& OutTestCommands) const
{
    // Enumerate every .uasset under /Game/Characters
    // and emit one sub-test per asset path.
}

bool FAllAssetsLoadCleanlyTest::RunTest(const FString& Parameters)
{
    // Parameters is the row from GetTests.
    UObject* Loaded = StaticLoadObject(UObject::StaticClass(),
                                       nullptr, *Parameters);
    return TestNotNull(TEXT("Asset loaded"), Loaded);
}

Latent commands

For tests that must span multiple frames, use ADD_LATENT_AUTOMATION_COMMAND to chain commands that yield back to the engine tick loop:

ADD_LATENT_AUTOMATION_COMMAND(
    FEngineWaitLatentCommand(/* Seconds */ 2.0f));

Custom latent commands derive from IAutomationLatentCommand and override Update() (returns true when complete).

Automation Spec (BDD style)

Per the Automation Spec documentation (opens in new window), specs are "built following the Behavior Driven Design (BDD) methodology" and use Describe / It / BeforeEach / AfterEach instead of one RunTest body. Spec files use .spec.cpp extension.

Simple spec:

DEFINE_SPEC(
    FInventorySpec,
    "MyGame.Inventory",
    EAutomationTestFlags::ProductFilter |
        EAutomationTestFlags::ApplicationContextMask)

void FInventorySpec::Define()
{
    Describe("AddItem", [this]()
    {
        It("should increase count by the stack amount", [this]()
        {
            FInventory Inv;
            Inv.AddItem(EItem::Potion, /* Count */ 3);
            TestEqual(TEXT("Potion count"),
                      Inv.GetCount(EItem::Potion),
                      3);
        });

        It("should reject items past max stack", [this]()
        {
            FInventory Inv;
            const bool bOk = Inv.AddItem(EItem::Potion, 999);
            TestFalse(TEXT("Over-cap add rejected"), bOk);
        });
    });
}

Per the same docs, It() descriptions should "start with 'should'" so the runner output reads as full sentences ("Inventory AddItem should increase count by the stack amount").

For specs with shared state, use BEGIN_DEFINE_SPEC / END_DEFINE_SPEC:

BEGIN_DEFINE_SPEC(
    FBackendSpec,
    "MyGame.Backend",
    EAutomationTestFlags::ProductFilter |
        EAutomationTestFlags::ApplicationContextMask)
    TSharedPtr<FMyBackendClient> Client;
END_DEFINE_SPEC(FBackendSpec)

void FBackendSpec::Define()
{
    BeforeEach([this]()
    {
        Client = MakeShared<FMyBackendClient>();
    });

    LatentIt("should return items asynchronously",
        [this](const FDoneDelegate& Done)
    {
        Client->QueryItemsAsync([Done](const TArray<FItem>& Items)
        {
            // assert on Items here, then signal completion
            Done.Execute();
        });
    });
}

LatentIt (per the Automation Spec docs (opens in new window)) gives the test a FDoneDelegate it must invoke when the async work has finished - Unreal's analogue of a Promise / future-based test.

Disabling: prefix the spec function with x (xIt(…), xDescribe(…)) per the same docs.

Automation Driver (UI input simulation)

Per the Automation Driver documentation (opens in new window), Automation Driver "enabling programmers to simulate user input … cursor movement, clicks, pressing, typing, scrolling, drag-and-drop, and more". It pairs with Automation Spec; the synchronous Driver API cannot run on the GameThread so the test must execute on a ThreadPool context.

Pattern (paraphrased from the same page):

BEGIN_DEFINE_SPEC(FMenuDriverSpec,
    "MyGame.Menu.Driver",
    EAutomationTestFlags::ProductFilter |
        EAutomationTestFlags::ApplicationContextMask)
    FAutomationDriverPtr Driver;
END_DEFINE_SPEC(FMenuDriverSpec)

void FMenuDriverSpec::Define()
{
    BeforeEach(EAsyncExecution::TaskGraphMainThread, [this]()
    {
        IAutomationDriverModule::Get().Enable();
        Driver = IAutomationDriverModule::Get().CreateDriver();
    });

    It("Submit button should close the menu",
       EAsyncExecution::ThreadPool, [this]()
    {
        FDriverElementRef Submit = Driver->FindElement(
            By::Id("SubmitButton"));
        Submit->Click();
        // assert menu state changed …
    });

    AfterEach(EAsyncExecution::TaskGraphMainThread, [this]()
    {
        IAutomationDriverModule::Get().Disable();
    });
}

Locator hierarchy (per the Automation Driver page (opens in new window)):

Locator	When	Caveat
`By::Id("SubmitButton")`	Most reliable	Requires explicit metadata tagging on the widget
`By::Path("…")`	Powerful	"Brittle" per the docs - depends on widget hierarchy
`By::Cursor()`	Returns widget under cursor	Mainly for hover tests
`By::Delegate(…)`	Custom lambda discovery	Power-user fallback

FindElement returns a FDriverElementRef; actions like Click(), Type(), TypeChord() are exposed on the element. "All Automation Driver actions automatically wait the configured ImplicitWait timespan for any dependent scenarios" per the same page.

Reference

Running and reporting

View source

Deep reference for unreal-automation-system SKILL.md - how to execute the authored tests (editor and command line), how to parse the JSON report, and how to wire the run into CI. Consult after the tests are written.

Session Frontend (Editor)

Per the Automation Test Framework documentation (opens in new window), open Window → Test Automation (or equivalent Session Frontend) inside the Editor. Tests appear as a tree keyed on the test path (the dot-separated string from the test macro). Click Run Tests for selected leaves; results show pass / fail

log output inline.

Command line

Run tests from the command line by launching the editor / commandlet with -ExecCmds:

UnrealEditor-Cmd.exe MyGame.uproject \
    -ExecCmds="Automation RunTests MyGame.Inventory; Quit" \
    -unattended -nopause -testexit="Automation Test Queue Empty" \
    -ReportOutputPath="artifacts/automation" \
    -log

Variants:

Automation RunTests <Filter> - run tests matching the path prefix.
Automation RunAll - run every registered test.
Automation Quit - quit when queue drains.

On Windows the editor binary is UnrealEditor-Cmd.exe; on macOS / Linux the launcher script is UnrealEditor with the same flags.

Gauntlet for build-farm runs

For full build-and-test pipelines (deploy a packaged build to a target devkit, run automation, collect artifacts), the next layer up is Gauntlet (Unreal's automation harness referenced on the Automation Test Framework page (opens in new window)). Out of scope for this skill - gauntlet wraps the same Automation RunTests command-line surface internally.

Parsing results

The -ReportOutputPath directory contains a JSON index.json plus per-test JSON / HTML detail. Top-level shape:

{
  "devices": [{
    "deviceName": "WindowsEditor",
    "instance": "WindowsEditor"
  }],
  "reportCreatedOn": "2026-05-21T10:33:00Z",
  "succeeded": 38,
  "failed": 1,
  "succeededWithWarnings": 2,
  "notRun": 0,
  "totalDuration": 142.7,
  "tests": [
    {
      "fullTestPath": "MyGame.Inventory.AddItem should increase count by the stack amount",
      "testDisplayName": "should increase count by the stack amount",
      "state": "Success",
      "entries": [/* per-step log entries */]
    },
    {
      "fullTestPath": "MyGame.Inventory.AddItem should reject items past max stack",
      "state": "Fail",
      "entries": [/* assertion failure detail */]
    }
  ]
}

(Field names paraphrased from observed Unreal output; the precise schema ships with the engine version under Engine/Source/Runtime/AutomationController/. For CI gating, treat any "state": "Fail" as a failed build.)

There is no public common exit code; parse the JSON or scrape the log for LogAutomationController: Display: Test ...: Result: Fail lines.

CI integration

GitHub Actions example (paraphrased from common Unreal CI patterns; Unreal CI itself is documented inside the editor SDK):

jobs:
  unreal-automation:
    runs-on: windows-2022
    steps:
      - uses: actions/checkout@v4
        with:
          lfs: true
      - name: Run automation tests
        shell: pwsh
        run: |
          $UE = "C:\Program Files\Epic Games\UE_5.4\Engine\Binaries\Win64\UnrealEditor-Cmd.exe"
          & $UE "$PWD\MyGame.uproject" `
            -ExecCmds="Automation RunTests MyGame; Quit" `
            -unattended -nopause `
            -testexit="Automation Test Queue Empty" `
            -ReportOutputPath="$PWD\artifacts\automation" `
            -log
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: automation-report
          path: artifacts/automation

CI pipelines typically split the test run into:

PR job - EAutomationTestFlags::SmokeFilter only ("complete within 1 second" per the framework docs (opens in new window)), for sub-minute PR feedback.
Nightly job - ProductFilter | StressFilter plus screenshot comparison.

For a sketch of how internal QA categories (the test-category taxonomy in platform-cert-overview-reference) map to Unreal's flags, see that reference.