unreal-automation-system
Author and run Unreal Engine's Automation Test Framework - Epic's C++ test framework for UE 4.x / 5.x, documented at dev.epicgames.com/documentation/en-us/unreal-engine. Covers the five test categories Epic defines (Unit / Feature / Smoke / Content Stress / Screenshot Comparison), the IMPLEMENT_SIMPLE_AUTOMATION_TEST and IMPLEMENT_COMPLEX_AUTOMATION_TEST macros, the BDD-style Automation Spec API (DEFINE_SPEC / BEGIN_DEFINE_SPEC / Describe / It / BeforeEach / LatentIt / xIt), latent commands (ADD_LATENT_AUTOMATION_COMMAND), the Automation Driver for UI input simulation (IAutomationDriverModule::Get().CreateDriver(), By::Id / By::Path locators), running via Session Frontend (Window > Test Automation) and command line (-ExecCmds="Automation RunTests …"), and CI integration. Use when the unit under test is C++ Unreal code that needs the UE runtime, editor, or UMG UI surface.
Install with skills.sh (any agent)
npx skills add testland/qa --skill unreal-automation-systemunreal-automation-system
Overview
This skill wraps the C++ Automation Test Framework (UE 4.x / 5.x) plus the two most commonly composed sub-systems. Per Epic's Automation Test Framework documentation (opens in new window), the framework spans five test categories (Unit, Feature, Smoke, Content Stress, Screenshot Comparison) and supports multiple authoring styles (traditional, BDD Spec, UI Driver, Functional, Python / Blueprint).
The full authoring code (macros, Spec, Driver) lives in references/authoring-macros-and-apis.md; running, report parsing, and CI wiring live in references/running-and-reporting.md. This file is the decision surface.
Composes with:
When to use
For Python / Blueprint editor tests outside C++, see Editor Automation in Unreal Engine (opens in new window). For end-to-end pipeline / build-farm orchestration, use Gauntlet (opens in new window) on top of the framework this skill covers.
How to use
Test categories and flags
Per the Automation Test Framework page (opens in new window), Epic's five categories are:
| Category | Purpose |
|---|---|
| Unit | "API level verification tests." |
| Feature | "System-level tests that verify such things as PIE, in-game stats, and changing resolution." |
| Smoke | Tests that "complete within 1 second" and run automatically. |
| Content Stress | "More thorough testing of a particular system to avoid crashes." |
| Screenshot Comparison | For comparing renders "between versions or builds". |
Each test declares flags from EAutomationTestFlags that mix:
Typical combo for a product-level test runnable in editor / client / server contexts: EAutomationTestFlags::ProductFilter | EAutomationTestFlags::ApplicationContextMask.
Authoring styles
Pick the style that matches the unit under test, then lift the full pattern from references/authoring-macros-and-apis.md:
| Style | Macro / API | Use when |
|---|---|---|
| Simple test | IMPLEMENT_SIMPLE_AUTOMATION_TEST + RunTest | One assertion body over plain C++ |
| Data-driven | IMPLEMENT_COMPLEX_AUTOMATION_TEST + GetTests | One sub-test per enumerated row (assets, configs) |
| Multi-frame | ADD_LATENT_AUTOMATION_COMMAND | Test must yield to the tick loop across frames |
| BDD Spec | DEFINE_SPEC / BEGIN_DEFINE_SPEC + Describe / It / LatentIt | Readable specs; .spec.cpp files; async via FDoneDelegate |
| UI Driver | IAutomationDriverModule::Get().CreateDriver() + By::Id | Simulate cursor / click / type on UMG; runs off the GameThread |
Running
Iterate in-editor via Window → Test Automation (Session Frontend), then run headless for CI:
UnrealEditor-Cmd.exe MyGame.uproject \
-ExecCmds="Automation RunTests MyGame.Inventory; Quit" \
-unattended -nopause -testexit="Automation Test Queue Empty" \
-ReportOutputPath="artifacts/automation" \
-logCommand variants, the -ReportOutputPath JSON schema, Gauntlet, and a full GitHub Actions job are in references/running-and-reporting.md.
Worked example
Goal: cover an inventory system's stacking rule as a CI-gating BDD spec, plus a menu-close UI check.
Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
All tests as SmokeFilter | Smoke tests "complete within 1 second" per framework docs (opens in new window) - long tests break the smoke contract | Use ProductFilter for tests > 1 s |
| Running Automation Driver on GameThread | API "cannot run on the GameThread" per Driver docs (opens in new window) | Use EAsyncExecution::ThreadPool on the driver-using It |
Spec without It descriptions starting with "should" | Runner output reads poorly | Per Spec docs (opens in new window), start descriptions with "should" |
Trusting By::Path locators | "Brittle" per Driver docs (opens in new window) | Prefer By::Id with tagged metadata |
Sleep-style waits in latent commands | Flaky under CI load | Use FDoneDelegate (LatentIt) or custom IAutomationLatentCommand::Update() polling |
Spec test file without .spec.cpp extension | Build system may not pick it up | Per Spec docs (opens in new window), use .spec.cpp suffix and no "Test" in filename |
Cloning non-thread-safe shared pointers in async It | Crash under threadpool | Per Driver docs (opens in new window), cache them on the test class |
No BeforeEach / AfterEach cleanup of IAutomationDriverModule | Driver state leaks between specs | Pair Enable() / Disable() calls in BeforeEach / AfterEach per Driver docs (opens in new window) |
Limitations
References
Authoring macros and APIs
View source (opens in new window)Authoring macros and APIs
Deep reference for unreal-automation-system SKILL.md - the full macro, Automation Spec, and Automation Driver code. Consult after picking the test category and flags in the spine, when writing the actual test body.
Simple automation test
Per Automation Test Framework documentation (opens in new window), the canonical macro pair is IMPLEMENT_SIMPLE_AUTOMATION_TEST + RunTest:
#include "Misc/AutomationTest.h"
IMPLEMENT_SIMPLE_AUTOMATION_TEST(
FHealthComponentDamageTest,
"MyGame.Health.Damage_DeductsCorrectAmount",
EAutomationTestFlags::ProductFilter |
EAutomationTestFlags::ApplicationContextMask)
bool FHealthComponentDamageTest::RunTest(const FString& Parameters)
{
UHealthComponent* Health = NewObject<UHealthComponent>();
Health->Initialize(/* MaxHealth */ 100.f);
Health->ApplyDamage(35.f);
TestEqual(TEXT("Current HP after 35 damage"),
Health->GetCurrent(),
65.f);
return true;
}Naming convention: the second macro argument ("MyGame.Health.Damage_…") is the test path shown in the Session Frontend. Dot-separated segments build a tree.
Complex automation test (data-driven)
IMPLEMENT_COMPLEX_AUTOMATION_TEST generates a sub-test per row returned from GetTests:
IMPLEMENT_COMPLEX_AUTOMATION_TEST(
FAllAssetsLoadCleanlyTest,
"MyGame.Assets.LoadCleanly",
EAutomationTestFlags::ProductFilter |
EAutomationTestFlags::ApplicationContextMask)
void FAllAssetsLoadCleanlyTest::GetTests(
TArray<FString>& OutBeautifiedNames,
TArray<FString>& OutTestCommands) const
{
// Enumerate every .uasset under /Game/Characters
// and emit one sub-test per asset path.
}
bool FAllAssetsLoadCleanlyTest::RunTest(const FString& Parameters)
{
// Parameters is the row from GetTests.
UObject* Loaded = StaticLoadObject(UObject::StaticClass(),
nullptr, *Parameters);
return TestNotNull(TEXT("Asset loaded"), Loaded);
}Latent commands
For tests that must span multiple frames, use ADD_LATENT_AUTOMATION_COMMAND to chain commands that yield back to the engine tick loop:
ADD_LATENT_AUTOMATION_COMMAND(
FEngineWaitLatentCommand(/* Seconds */ 2.0f));Custom latent commands derive from IAutomationLatentCommand and override Update() (returns true when complete).
Automation Spec (BDD style)
Per the Automation Spec documentation (opens in new window), specs are "built following the Behavior Driven Design (BDD) methodology" and use Describe / It / BeforeEach / AfterEach instead of one RunTest body. Spec files use .spec.cpp extension.
Simple spec:
DEFINE_SPEC(
FInventorySpec,
"MyGame.Inventory",
EAutomationTestFlags::ProductFilter |
EAutomationTestFlags::ApplicationContextMask)
void FInventorySpec::Define()
{
Describe("AddItem", [this]()
{
It("should increase count by the stack amount", [this]()
{
FInventory Inv;
Inv.AddItem(EItem::Potion, /* Count */ 3);
TestEqual(TEXT("Potion count"),
Inv.GetCount(EItem::Potion),
3);
});
It("should reject items past max stack", [this]()
{
FInventory Inv;
const bool bOk = Inv.AddItem(EItem::Potion, 999);
TestFalse(TEXT("Over-cap add rejected"), bOk);
});
});
}Per the same docs, It() descriptions should "start with 'should'" so the runner output reads as full sentences ("Inventory AddItem should increase count by the stack amount").
For specs with shared state, use BEGIN_DEFINE_SPEC / END_DEFINE_SPEC:
BEGIN_DEFINE_SPEC(
FBackendSpec,
"MyGame.Backend",
EAutomationTestFlags::ProductFilter |
EAutomationTestFlags::ApplicationContextMask)
TSharedPtr<FMyBackendClient> Client;
END_DEFINE_SPEC(FBackendSpec)
void FBackendSpec::Define()
{
BeforeEach([this]()
{
Client = MakeShared<FMyBackendClient>();
});
LatentIt("should return items asynchronously",
[this](const FDoneDelegate& Done)
{
Client->QueryItemsAsync([Done](const TArray<FItem>& Items)
{
// assert on Items here, then signal completion
Done.Execute();
});
});
}LatentIt (per the Automation Spec docs (opens in new window)) gives the test a FDoneDelegate it must invoke when the async work has finished - Unreal's analogue of a Promise / future-based test.
Disabling: prefix the spec function with x (xIt(…), xDescribe(…)) per the same docs.
Automation Driver (UI input simulation)
Per the Automation Driver documentation (opens in new window), Automation Driver "enabling programmers to simulate user input … cursor movement, clicks, pressing, typing, scrolling, drag-and-drop, and more". It pairs with Automation Spec; the synchronous Driver API cannot run on the GameThread so the test must execute on a ThreadPool context.
Pattern (paraphrased from the same page):
BEGIN_DEFINE_SPEC(FMenuDriverSpec,
"MyGame.Menu.Driver",
EAutomationTestFlags::ProductFilter |
EAutomationTestFlags::ApplicationContextMask)
FAutomationDriverPtr Driver;
END_DEFINE_SPEC(FMenuDriverSpec)
void FMenuDriverSpec::Define()
{
BeforeEach(EAsyncExecution::TaskGraphMainThread, [this]()
{
IAutomationDriverModule::Get().Enable();
Driver = IAutomationDriverModule::Get().CreateDriver();
});
It("Submit button should close the menu",
EAsyncExecution::ThreadPool, [this]()
{
FDriverElementRef Submit = Driver->FindElement(
By::Id("SubmitButton"));
Submit->Click();
// assert menu state changed …
});
AfterEach(EAsyncExecution::TaskGraphMainThread, [this]()
{
IAutomationDriverModule::Get().Disable();
});
}Locator hierarchy (per the Automation Driver page (opens in new window)):
| Locator | When | Caveat |
|---|---|---|
By::Id("SubmitButton") | Most reliable | Requires explicit metadata tagging on the widget |
By::Path("…") | Powerful | "Brittle" per the docs - depends on widget hierarchy |
By::Cursor() | Returns widget under cursor | Mainly for hover tests |
By::Delegate(…) | Custom lambda discovery | Power-user fallback |
FindElement returns a FDriverElementRef; actions like Click(), Type(), TypeChord() are exposed on the element. "All Automation Driver actions automatically wait the configured ImplicitWait timespan for any dependent scenarios" per the same page.
Running and reporting
View source (opens in new window)Running and reporting
Deep reference for unreal-automation-system SKILL.md - how to execute the authored tests (editor and command line), how to parse the JSON report, and how to wire the run into CI. Consult after the tests are written.
Session Frontend (Editor)
Per the Automation Test Framework documentation (opens in new window), open Window → Test Automation (or equivalent Session Frontend) inside the Editor. Tests appear as a tree keyed on the test path (the dot-separated string from the test macro). Click Run Tests for selected leaves; results show pass / fail
Command line
Run tests from the command line by launching the editor / commandlet with -ExecCmds:
UnrealEditor-Cmd.exe MyGame.uproject \
-ExecCmds="Automation RunTests MyGame.Inventory; Quit" \
-unattended -nopause -testexit="Automation Test Queue Empty" \
-ReportOutputPath="artifacts/automation" \
-logVariants:
On Windows the editor binary is UnrealEditor-Cmd.exe; on macOS / Linux the launcher script is UnrealEditor with the same flags.
Gauntlet for build-farm runs
For full build-and-test pipelines (deploy a packaged build to a target devkit, run automation, collect artifacts), the next layer up is Gauntlet (Unreal's automation harness referenced on the Automation Test Framework page (opens in new window)). Out of scope for this skill - gauntlet wraps the same Automation RunTests command-line surface internally.
Parsing results
The -ReportOutputPath directory contains a JSON index.json plus per-test JSON / HTML detail. Top-level shape:
{
"devices": [{
"deviceName": "WindowsEditor",
"instance": "WindowsEditor"
}],
"reportCreatedOn": "2026-05-21T10:33:00Z",
"succeeded": 38,
"failed": 1,
"succeededWithWarnings": 2,
"notRun": 0,
"totalDuration": 142.7,
"tests": [
{
"fullTestPath": "MyGame.Inventory.AddItem should increase count by the stack amount",
"testDisplayName": "should increase count by the stack amount",
"state": "Success",
"entries": [/* per-step log entries */]
},
{
"fullTestPath": "MyGame.Inventory.AddItem should reject items past max stack",
"state": "Fail",
"entries": [/* assertion failure detail */]
}
]
}(Field names paraphrased from observed Unreal output; the precise schema ships with the engine version under Engine/Source/Runtime/AutomationController/. For CI gating, treat any "state": "Fail" as a failed build.)
There is no public common exit code; parse the JSON or scrape the log for LogAutomationController: Display: Test ...: Result: Fail lines.
CI integration
GitHub Actions example (paraphrased from common Unreal CI patterns; Unreal CI itself is documented inside the editor SDK):
jobs:
unreal-automation:
runs-on: windows-2022
steps:
- uses: actions/checkout@v4
with:
lfs: true
- name: Run automation tests
shell: pwsh
run: |
$UE = "C:\Program Files\Epic Games\UE_5.4\Engine\Binaries\Win64\UnrealEditor-Cmd.exe"
& $UE "$PWD\MyGame.uproject" `
-ExecCmds="Automation RunTests MyGame; Quit" `
-unattended -nopause `
-testexit="Automation Test Queue Empty" `
-ReportOutputPath="$PWD\artifacts\automation" `
-log
- uses: actions/upload-artifact@v4
if: always()
with:
name: automation-report
path: artifacts/automationCI pipelines typically split the test run into:
For a sketch of how internal QA categories (game-test-categories-reference) map to Unreal's flags, see the category reference.
Related skills
game-perf-profiling
Profiles game builds against frame-time, memory, GPU draw-call, and GC-spike budgets using Unity Profiler + Profile Analyzer + Performance Testing package and Unreal Insights + stat commands. Establishes pass/fail thresholds (16.6 ms at 60 fps, 33.3 ms at 30 fps), writes automated performance regression tests that run in CI, and emits a structured budget report per SKU. Use when a title must hit a declared frame-time or memory budget before a milestone gate or platform-cert submission, or when a recent change needs a performance regression check.
game-test-categories-reference
Pure-reference catalog of the testing categories that apply to a video-game build before it ships. Defines the six canonical buckets the industry tests against - functional / compliance / compatibility / performance / localization / accessibility - plus the multiplayer and content-rating sub-axes. Cross-references each bucket to the platform-holder vocabulary that drives it (Microsoft Xbox Requirements / XR test cases, Sony TRC, Nintendo Lotcheck, Steam Direct review). Use as the taxonomy lookup when planning a game test pass, scoping QA effort, mapping platform-cert findings back to internal test categories, preparing a submission checklist, reviewing first-party certification requirements, or triaging cert testing failures against internal categories.
gameplay-recording-replay
Build a deterministic gameplay record/replay test artefact for Unity, Unreal, or Godot - record a player session, save it to disk, replay it bit-for-bit, and assert that the resulting game state matches the original. Covers Unity Input System's InputEventTrace API (Enable / Disable / WriteTo / ReadFrom / Replay) for input-level capture, Unreal's Replay System (DemoRec / DemoPlay / DemoStop console commands plus DemoNetDriver + NetworkReplayStreamer, default storage at %LOCALAPPDATA%/{Project}/Saved/Demos) for replication-stream capture, and Godot's community-pattern deterministic-RNG + input-script replay since Godot ships no first-party replay system. Use when authoring a regression-test artefact for player-recorded sessions, building a netcode replay for spectator / esports, or producing reproducible bug repros for cert teams.
godot-gut-tests
Author and run GUT (Godot Unit Test) - the community-canonical GDScript test framework at github.com/bitwes/Gut and gut.readthedocs.io. Covers install (Godot Asset Library or manual `addons/gut/` copy + plugin enable), GUT panel inside the editor, writing tests that extend GutTest with `test_` prefix methods, the assertion family (assert_eq / assert_almost_eq / assert_true / assert_signal_emitted), lifecycle hooks (before_each / after_each / before_all / after_all), inner classes for grouping, parameterized tests via `params=[...]`, doubles / stubs / spies, async / coroutine tests, the command-line runner (`-d -s addons/gut/gut_cmdln.gd -gdir=res://test -gjunit_xml_file=... -gexit`), JUnit XML export, and CI integration. Godot 4.x uses GUT 9.x (current main branch supports 4.6.x; godot_4_7 branch for 4.7.x); Godot 3.x uses GUT 7.x. Use when the unit under test is GDScript code in a Godot project.
multiplayer-state-machine-coverage
Build a coverage matrix for a networked-game state machine that exercises connect / authority-handoff / disconnect / reconnect / host-migration paths across Unity Netcode for GameObjects, Unreal Engine replication, and Mirror Networking. Workflow: enumerate the engine's connection states + ownership states + replicated-property update rules, cross them against latency / loss / out-of-order packet injection, encode each combination as a test fixture, and emit a go / no-go gate. Use before submitting a multiplayer title to platform cert - Microsoft's cert guide lists 'Multiplayer does not work as expected' as one of the most common Hold reasons, and Xbox XR-067 (MPSD session state) is failed by uncovered state-machine paths.
platform-cert-overview-reference
Pure-reference catalog of the four platform-holder certification regimes a multi-platform title submits to before release: Microsoft Xbox Requirements (XR) / Xbox certification on learn.microsoft.com, Sony Technical Requirements Checklist (TRC) on the gated PlayStation DevNet portal, Nintendo Lotcheck on the gated Nintendo Developer Portal, and Steam Direct review on partner.steamgames.com. Documents the submission workflow, severity / pass-fail vocabulary, test-bench configurations, and known SLAs for each platform. Cites public sources inline; cites gated NDA portals by stable ID per PLUGIN_AUTHORING.md Step 4 fallback. Use when planning a cert calendar, mapping internal QA findings to the platform's vocabulary, or sequencing submissions across platforms.
unity-test-framework
Author and run the Unity game-engine Test Framework (`com.unity.test-framework`, currently v1.8). Distinct from the ThrowTheSwitch Unity C testing library at throwtheswitch.org/unity - the two tools share only a name. Covers package install via Package Manager, the EditMode vs PlayMode split, the [Test] / [UnityTest] / [SetUp] / [TearDown] / [UnityPlatform] attributes, assembly-definition setup (Editor folder vs asmdef with `includePlatforms` / `optionalUnityReferences: [TestAssemblies]`), Test Runner window, command-line batch invocation with `-runTests` / `-testPlatform` / `-testResults` / `-testFilter` / `-testCategory`, NUnit 3.5 assertion API, and CI integration. Use when the unit under test is C# Unity code that needs to exercise the Unity runtime or editor.