platform-cert-overview-reference
Reference catalog of game-QA test categories and the four platform-holder certification regimes a multi-platform title submits to before release. Defines the six canonical test categories (functional / compliance / compatibility / performance / localization / accessibility) plus the multiplayer and content-rating cross-axes; documents the submission workflow, severity vocabulary, test-bench configurations, and known SLAs for Microsoft Xbox Requirements (XR), Sony TRC (gated PlayStation DevNet), Nintendo Lotcheck (gated Nintendo Developer Portal), and Steam Direct review; and includes the building-a-cert-checklist workflow that maps a target platform's requirement items to the six categories. Cites public sources inline; cites gated NDA portals by stable ID per PLUGIN_AUTHORING.md Step 4 fallback. Use when planning a game test pass or cert calendar, mapping internal QA findings to the platform's vocabulary, sequencing submissions across platforms, or emitting a pre-submission checklist.
Install with skills.sh (any agent)
npx skills add testland/qa --skill platform-cert-overview-referenceplatform-cert-overview-reference
Overview
A multi-platform title that ships on console plus PC submits to four distinct certification regimes, each with its own requirements document, severity vocabulary, test-bench configuration, and SLA. This skill is the pure reference that maps those four regimes onto a common structure so QA leads can plan a cert calendar without re-learning each portal.
| Platform | Regime | Document name | Source |
|---|---|---|---|
| Xbox | Xbox Certification | Xbox Requirements (XR) | learn.microsoft.com (opens in new window) (public) |
| PlayStation | Sony TRC | Technical Requirements Checklist | PlayStation DevNet (gated) |
| Nintendo | Lotcheck | Submission Guidelines | Nintendo Developer Portal (gated) |
| Steam | Steam Direct | App Review Process | partner.steamgames.com (opens in new window) (public) |
NDA note. Sony TRC and Nintendo Lotcheck live behind gated portals; their exact clause numbers, SLAs, severity vocabulary, and bench matrices are cited by stable ID ("Sony TRC" / "Nintendo Lotcheck") per PLUGIN_AUTHORING.md Step 4 fallback. Xbox and Steam requirements are public and cited inline.
The six canonical test categories (functional / compliance / compatibility / performance / localization / accessibility) that map across all four regimes are defined in "The six test categories" below.
When to use
The six test categories
A game build is tested against six canonical categories the industry has converged on over three console generations (sources: Microsoft's Certification step-by-step guide (opens in new window), Xbox Accessibility Guidelines (XAG) v3.2 (opens in new window), Steam's App Review Process (opens in new window), and Schultz & Bryant, Game Testing All-In-One, 3rd ed., ISBN 9781305077133):
| # | Category | Scope | Typical owner |
|---|---|---|---|
| 1 | Functional | Mechanics, scripting, AI, UI flow, save/load, economy/progression match the design spec. | Engine-specific automation + manual QA |
| 2 | Compliance | Build conforms to platform holder's Requirements / TRC / Lotcheck document. | Cert / submission QA |
| 3 | Compatibility | Runs across all required SKUs, OS versions, hardware generations, and display/audio/storage configurations. | Lab / device-cloud QA |
| 4 | Performance | Hits declared frame-time, load-time, memory, thermal, and battery budgets on each target SKU. | Performance engineers + profilers |
| 5 | Localization | Translated strings fit their UI, render in their script, and behave under RTL + CJK + length expansion; VO coverage correct. | LQA agencies + automation |
| 6 | Accessibility | Meets platform-required or -recommended accessibility guidelines (Microsoft XAG, AbleGamers, CVAA/EAA where applicable). | Accessibility QA + design |
Category notes:
Triage workflow
Given a bug report: identify the symptom → map to a category (cheatsheet below) → assign the owner from the table → set severity in the platform holder's vocabulary (CFR / SRI / ION for Xbox; comparable NDA tiers for Sony / Nintendo - triage CFRs first) → record the XR / TRC / Lotcheck ID in the bug tracker so cert QA can verify the fix → confirm the category and requirement ID still resolve in the current requirements list before closing (a stale XR number is the common triage error).
| Symptom | Category | Severity hint |
|---|---|---|
| Crash on save load | Functional | High - cert-blocker |
| XR-024 fail (unhandled controller disconnect) | Compliance | CFR - cert-blocker |
| Black screen on PS4 base, fine on PS5 | Compatibility | High - gens-affecting |
| 22 fps in raid encounter (target 30) | Performance | Triage vs. budget |
| German "Zurück" overflows button | Localization | Sev 3 - UI |
| Subtitles missing in cutscene 7 | Accessibility | XAG fail |
| Host migration drops players | Multiplayer (functional + compliance) | High |
Cross-axes: multiplayer and content rating
Multiplayer is not a seventh category - it cuts across functional + compliance + compatibility + performance: state-machine correctness under packet loss / latency / disconnect (see multiplayer-state-machine-coverage), matchmaking + session lifecycle, anti-cheat compatibility with platform integrity systems. Per Microsoft's certification-guide (opens in new window), "Multiplayer does not work as expected" is one of the most common reasons a title is placed on Hold during an Xbox cert pass.
Content rating (ESRB / PEGI / CERO / USK / IARC) is a regulatory classification, not a testing category - but Steam requires the Content Survey (opens in new window) before review, platform holders gate distribution on the local rating board's certificate, and localization may interact with rating (removing gambling minigames for the Belgian release).
Certification regimes
Per-regime detail lives in one reference file each:
Versioning and deprecation
Time-sensitive version facts, isolated here so the body stays evergreen:
Cross-platform comparison summary
| Dimension | Xbox | Sony | Nintendo | Steam |
|---|---|---|---|---|
| Requirements doc | XR (versioned per GDK) | TRC (NDA) | Lotcheck (NDA) | App Review Process |
| Public visibility | Full | Gated | Gated | Full |
| Submission stages | 3 (val + BVT + XR) | Multi-step (NDA) | Multi-step (NDA) | 2 parallel (store + build) |
| Test-pass types | Optional + Final | (NDA) | (NDA) | One pass per submission |
| Severity vocab | CFR / SRI / ION | NDA (comparable) | NDA (comparable) | Pass / Fail (informal) |
| Console SLA (digital) | 5 BD | (NDA - practitioner-weeks) | (NDA - practitioner-weeks) | 3-5 BD (plan for 7) |
| Content-update SLA | 3 BD (1 BD bypass on PC) | (NDA) | (NDA) | Same as initial review |
| Bench matrix | 5 consoles × SKU / display / audio / power | PS4 / PS4 Pro / PS5 / PS5 Pro (NDA) | Switch / OLED / Lite / Switch 2 + handheld/docked (NDA) | Per-OS startup |
| Accessibility surface | XAG v3.2 (recommended; MGATS service available) | TRC clauses + Sony's accessibility guidance (NDA) | Lotcheck clauses (NDA) | Not gated |
| Achievements / Trophies surface | XR-055 / -057 / -058 / -060 / -062 | Trophies (NDA) | None native to Lotcheck | Steam Achievements (no cert gate) |
Sequencing across platforms
A pattern that surfaces issues earliest:
This is a heuristic, not a mandate - partners with strong in-house TRC expertise often reverse the Xbox / Sony ordering.
Building a cert checklist
To turn a target platform into a ready-to-run pre-submission checklist, follow this workflow (inputs: target platform - xbox / playstation / nintendo / steam / all; optional milestone context, title-features list, target hardware SKUs):
Refuse to emit a checklist item whose claim has no available source - surface the gap instead of inventing text. Engine test scenario files are a different job: the game-test-scenario-author agent authors those.
Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Treating cert as a Q4 milestone | Late-found CFRs delay launch by weeks | Run XR / TRC checklists from milestone 1 |
| Treating "QA" as one undifferentiated bucket | Loses ownership; compliance gaps surface at cert | Use the six categories explicitly in test plans + bug-tracker fields |
| Compatibility on the dev kit only | Retail SKUs (Series S, PS4 base, Switch handheld) behave differently | Lab test on retail hardware per the bench layout in references/platform-cert-details.md |
| Performance averaged over the level | Spikes cause CFRs; averages hide them | Frame-time histograms, p99, sustained-window measurements |
| Localization screenshot review | Misses runtime overflow, missing VO, lip-sync drift | Playthrough every locale; not a glanced screenshot |
| Accessibility as a launch checklist | Architectural changes (rebindable input) can't be retrofitted | XAG scoping questions at design milestone, not at submission |
| Multiplayer tested only in low-latency LAN | Drops, NAT, host migration fail in the wild | Inject loss / jitter; see multiplayer-state-machine-coverage |
| Optional submission "to find out where we are" without prep | Burns paid slot to find bugs internal QA could have | Run internal XR pre-pass before any paid Optional |
| Submitting before partner-service stubs are ready | Held titles cost real calendar days per Microsoft's common hold reasons (opens in new window) | Stand up CERT.DEBUG / CERT sandbox partner accounts before submission |
| Using XR-only language with Sony / Nintendo teams | Cross-platform vocabulary collapses; finger-pointing | Translate findings via the comparison table above |
| Treating Steam Direct review as "real cert" | Misses console requirements that wouldn't surface on Steam | Use the four regimes' actual requirements docs - Steam is not a proxy |
| Cross-network play without privilege checks | Xbox XR-045 / XR-007 fail - CFR | Use XPRIVILEGE_* constants per the XR-045 privilege table (opens in new window) |
| Achievements that "drift" between updates | Xbox XR-060 fail - published achievements can't change unlock rules / rewards | Lock achievement design before launch; only modify name / description / icon post-launch |
| Local-storage writes > 1 GiB / 5 min | Xbox XR-133 fail | Rate-limit save / cache flushes |
Limitations
Sony TRC and Nintendo Lotcheck - reference
View source (opens in new window)Sony TRC and Nintendo Lotcheck - reference
Full detail for the two NDA-gated console regimes, for platform-cert-overview-reference.
NDA note (applies to every clause below). All Sony TRC and Nintendo Lotcheck specifics are behind gated portals - PlayStation DevNet and the Nintendo Developer Portal. They are cited by stable ID ("Sony TRC" / "Nintendo Lotcheck") per PLUGIN_AUTHORING.md Step 4 fallback; partners with portal access should consult the current authoritative revision. Individual clauses are not re-marked NDA in the sections below.
Sony PlayStation Certification (TRC)
Source of truth
Structure
The Sony TRC is a numbered checklist organised into roughly comparable buckets to the Xbox XR list:
Submission workflow
Sony documents its high-level publishing workflow publicly via playstation.com/en-us/develop (opens in new window); the per-step TRC details are NDA.
Severity vocabulary
Sony classifies findings on a severity scale comparable to the Microsoft CFR / SRI split. Each finding lists the TRC clause it violates and the steps to reproduce.
SLAs
Sony does not publish SLAs publicly. Practitioners budget calendar-weeks for a console-game format QA pass; patch / minor-update SLAs are shorter than initial-release SLAs.
Nintendo Lotcheck
Source of truth
Structure
Nintendo Lotcheck covers analogous buckets:
Workflow
The Lotcheck submission workflow is comparable in shape to Sony TRC: pre-cert dry-run, master upload, Nintendo QA pass, fail ticket list keyed to Lotcheck clauses, re-submission. Exact step names and SLAs are NDA.
Notable distinguishing constraints
Platform certification detail
View source (opens in new window)Platform certification detail
Per-platform requirement documents, severity codes, and the compatibility bench layout that back the six-category taxonomy in SKILL.md.
Requirement documents by platform (Compliance)
| Platform | Document name | Source |
|---|---|---|
| Xbox console | Xbox Requirements (XRs) | learn.microsoft.com/.../console/certification-requirements (opens in new window) |
| Xbox PC | Xbox Network Policies for PC and Mobile | learn.microsoft.com/.../pc/live-policies-pc (opens in new window) |
| PlayStation | Technical Requirements Checklist (TRC) | Gated NDA portal; cite by stable ID "Sony TRC" |
| Nintendo | Lotcheck / Submission Guidelines | Gated NDA portal; cite by stable ID "Nintendo Lotcheck" |
| Steam | App Review Process | partner.steamgames.com/doc/store/review_process (opens in new window) |
Each requirement maps to test cases with explicit Test Steps, Expected Behavior, and Pass/Fail examples - see Microsoft's Certification Tested Xbox Requirements for Xbox Console Games (opens in new window).
Severity model (Xbox; broadly applicable)
| Code | Name | Effect |
|---|---|---|
| CFR | Condition for Resubmission | Title fails cert; must be fixed before resubmit |
| SRI | Standard Reporting Issue | Related to an XR but not severe enough to fail; fix recommended |
| ION | Issue of Note | Not tied to an XR; informational |
| Non-Tested | - | XR could not be tested in this pass |
Sony TRC and Nintendo Lotcheck use comparable severity tiers internally (NDA - cite by stable ID).
Compatibility bench layout
The Xbox bench - per the Certification step-by-step guide (opens in new window) - exemplifies the required matrix:
| Console | SKU | Resolution | Storage | Audio |
|---|---|---|---|---|
| Console 1 | Xbox One / Xbox Series X | 720p | Internal HDD | Stereo |
| Console 2 | Xbox One X / Xbox Series S | 4k | Internal HDD | 5.1 Bitstream |
| Console 3 | Xbox One S / Xbox Series X | 1080p | Internal HDD | Stereo |
| Console 4 | Xbox Series S | 1080p | USB HDD | Headset (Windows Sonic) |
| Console 5 | Xbox Series X | 720p | USB HDD | Stereo |
The same matrix logic applies to Sony (PS4 base / PS4 Pro / PS5 / PS5 Pro), Nintendo (Switch original / OLED / Switch 2), and PC (GPU vendor × driver version × OS × DirectX/Vulkan). Microsoft tracks cross-generation requirements under XR-130: Xbox Console Families and Generations (opens in new window). Each bench console also varies Console Language to exercise localized assets in the cert pass.
Steam Direct - reference
View source (opens in new window)Steam Direct - reference
Full Steam Direct regime detail for platform-cert-overview-reference.
Source of truth
Workflow
Per the App Review Process (opens in new window):
Per the same page: "Once your game has been reviewed and approved, there is no need to go through review again."
What is reviewed
Per the same page, the Store Presence Review checks:
The Product Build Review checks:
Special categories
Per the same page:
Comparison to console cert
Steam Direct is dramatically lighter than the three console regimes: no equivalent of Xbox XR test cases, no equivalent of Sony TRC bench matrix, no equivalent of Nintendo handheld / docked test surface. The bar is "starts up + features match store page + uses Steam Wallet for IAP" rather than a multi-stage cert pass.
Microsoft Xbox Certification - reference
View source (opens in new window)Microsoft Xbox Certification - reference
Full Xbox regime detail for platform-cert-overview-reference. Version-specific notes (XR v16.1, GDK monikers, retired XRs) are consolidated in the skill's "Versioning and deprecation" section.
Source of truth
Stages
Per the Certification step-by-step guide (opens in new window), a submission goes through three stages in order:
Submission types and SLAs
Per the same guide:
| Submission type | Console SLA | PC (MSIXVC) SLA |
|---|---|---|
| Disc | 6 business days | N/A |
| Digital | 5 business days | 3 business days |
| Digital Content-Update | 3 business days | 1 business day (incl. bypass) |
There are two test-pass types:
XR categories
XRs are organised into the categories enumerated in the XR document (opens in new window):
| Category | Representative XRs |
|---|---|
| Base requirements | XR-001 (Title Stability), XR-003 (Title Quality), XR-022 (Naming), XR-074 (Service Loss), XR-130 (Console Families and Generations), XR-131 (Display Mode Support), XR-132 (Service Access Limits), XR-133 (Local Storage Write Limit, 1 GiB / 5 min) |
| Security | XR-009 (Secure Title Development, per Microsoft Store policies 10.2 / 10.2.2 / 10.2.3 / 10.2.4) |
| Online Safety and Privacy | XR-013 (Account Linking), XR-014 (Player Data, child / teen handling), XR-015 (Player Communication, with CommunicateUsingText / CommunicateUsingVoice privilege checks), XR-017 (Title Ratings), XR-018 (UGC) |
| Content packages and updates | XR-034 (Streaming Install Initial Play Marker), XR-037 (DLC Dependencies), XR-123 (DLC unlock without relaunch), XR-129 (Intelligent Delivery via PackageInstallChunksAsync) |
| Purchasing | XR-036 (In-Title Pricing), XR-039 (Common Purchase UI) |
| User profiles | XR-045 (Privileges, e.g. XPRIVILEGE_MULTIPLAYER_SESSIONS ID 254, XPRIVILEGE_COMMUNICATIONS ID 252, XPRIVILEGE_USER_CREATED_CONTENT ID 247), XR-046 (Gamertag display), XR-047 (User-Profile Access), XR-048 (Profile Settings Usage), XR-052 (Save Roaming), XR-112 (User and Controller activation), XR-115 (User / Controller add and remove) |
| Achievements and Awards | XR-055 (Counts and Gamerscore - min 10, max 100 launch / 500 lifetime, single ach ≤ 200 GS), XR-057 (No real-money unlocks), XR-058 (No cross-title sharing), XR-060 (No modifying active achievements), XR-062 (PEGI 12 / ESRB EVERYONE 10+ name and description content) |
| Multiplayer sessions | XR-064 (Joinable via shell), XR-067 (MPSD session state), XR-070 (Friends Lists), XR-124 (Game Invitations) |
| Betas and Game Previews | XR-117 (Beta notification splash) |
XRs that are tested in Cert are marked with an asterisk on the source page; not all XRs are tested every submission. See the test cases page above for per-XR Test Steps + Expected Behavior + Pass / Fail examples.
Issue severity vocabulary
Per the Certification step-by-step guide (opens in new window):
| Code | Name | Effect |
|---|---|---|
| CFR | Condition for Resubmission | Title fails cert; must be fixed before resubmit |
| SRI | Standard Reporting Issue | Tied to an XR but not fail-severe; fix recommended |
| ION | Issue of Note | Not tied to an XR; informational |
| Non-Tested | (no code) | XR could not be tested in this pass |
CFR severity is determined by Failure Mode Analysis (FMA) (opens in new window) which combines severity × probability × repeatability.
Common hold reasons
Per the same guide, titles are most often placed on Hold because of:
(Multiplayer issues are a particularly common cert blocker - see multiplayer-state-machine-coverage for the test patterns that catch them pre-submission.)
Bench layout (compatibility matrix)
Per the Certification test bench configuration (opens in new window) section of the guide:
| Console | SKU | Resolution | Storage | Audio | Power |
|---|---|---|---|---|---|
| 1 | Xbox One / Xbox Series X | 720p | Internal HDD | Stereo | Instant-On |
| 2 | Xbox One X / Xbox Series S | 4k (alt HDR after 4 h) | Internal HDD | 5.1 Bitstream | Instant-On |
| 3 | Xbox One S / Xbox Series X | 1080p | Internal HDD | Stereo | Energy Saving |
| 4 | Xbox Series S | 1080p | USB HDD | Headset (Windows Sonic) | Instant-On |
| 5 | Xbox Series X | 720p | USB HDD | Stereo | Energy Saving |
Reports
Three report types (per same guide):
Exception requests
Per the guide: "If your title is not fully compliant with any Xbox Requirement (XR), you can request an exception for that XR. This might occur if your title is introducing innovative features or if strong technical limitations would prevent the title from meeting the XR." Work with Microsoft contacts well in advance - "approval of an exception is not guaranteed."
Related skills
game-perf-profiling
Profiles game builds against frame-time, memory, GPU draw-call, and GC-spike budgets using Unity Profiler + Profile Analyzer + Performance Testing package and Unreal Insights + stat commands. Establishes pass/fail thresholds (16.6 ms at 60 fps, 33.3 ms at 30 fps), writes automated performance regression tests that run in CI, and emits a structured budget report per SKU. Use when a title must hit a declared frame-time or memory budget before a milestone gate or platform-cert submission, or when a recent change needs a performance regression check.
gameplay-recording-replay
Build a deterministic gameplay record/replay test artefact for Unity, Unreal, or Godot - record a player session, save it to disk, replay it bit-for-bit, and assert that the resulting game state matches the original. Covers Unity Input System's InputEventTrace API (Enable / Disable / WriteTo / ReadFrom / Replay) for input-level capture, Unreal's Replay System (DemoRec / DemoPlay / DemoStop console commands plus DemoNetDriver + NetworkReplayStreamer, default storage at %LOCALAPPDATA%/{Project}/Saved/Demos) for replication-stream capture, and Godot's community-pattern deterministic-RNG + input-script replay since Godot ships no first-party replay system. Use when authoring a regression-test artefact for player-recorded sessions, building a netcode replay for spectator / esports, or producing reproducible bug repros for cert teams.
godot-gut-tests
Author and run GUT (Godot Unit Test) - the community-canonical GDScript test framework at github.com/bitwes/Gut and gut.readthedocs.io. Covers install (Godot Asset Library or manual `addons/gut/` copy + plugin enable), GUT panel inside the editor, writing tests that extend GutTest with `test_` prefix methods, the assertion family (assert_eq / assert_almost_eq / assert_true / assert_signal_emitted), lifecycle hooks (before_each / after_each / before_all / after_all), inner classes for grouping, parameterized tests via `params=[...]`, doubles / stubs / spies, async / coroutine tests, the command-line runner (`-d -s addons/gut/gut_cmdln.gd -gdir=res://test -gjunit_xml_file=... -gexit`), JUnit XML export, and CI integration. Godot 4.x uses GUT 9.x (current main branch supports 4.6.x; godot_4_7 branch for 4.7.x); Godot 3.x uses GUT 7.x. Use when the unit under test is GDScript code in a Godot project.
multiplayer-state-machine-coverage
Build a coverage matrix for a networked-game state machine that exercises connect / authority-handoff / disconnect / reconnect / host-migration paths across Unity Netcode for GameObjects, Unreal Engine replication, and Mirror Networking. Workflow: enumerate the engine's connection states + ownership states + replicated-property update rules, cross them against latency / loss / out-of-order packet injection, encode each combination as a test fixture, and emit a go / no-go gate. Use before submitting a multiplayer title to platform cert - Microsoft's cert guide lists 'Multiplayer does not work as expected' as one of the most common Hold reasons, and Xbox XR-067 (MPSD session state) is failed by uncovered state-machine paths.
unity-test-framework
Author and run the Unity game-engine Test Framework (`com.unity.test-framework`, currently v1.8). Distinct from the ThrowTheSwitch Unity C testing library at throwtheswitch.org/unity - the two tools share only a name. Covers package install via Package Manager, the EditMode vs PlayMode split, the [Test] / [UnityTest] / [SetUp] / [TearDown] / [UnityPlatform] attributes, assembly-definition setup (Editor folder vs asmdef with `includePlatforms` / `optionalUnityReferences: [TestAssemblies]`), Test Runner window, command-line batch invocation with `-runTests` / `-testPlatform` / `-testResults` / `-testFilter` / `-testCategory`, NUnit 3.5 assertion API, and CI integration. Use when the unit under test is C# Unity code that needs to exercise the Unity runtime or editor.
unreal-automation-system
Author and run Unreal Engine's Automation Test Framework - Epic's C++ test framework for UE 4.x / 5.x, documented at dev.epicgames.com/documentation/en-us/unreal-engine. Covers the five test categories Epic defines (Unit / Feature / Smoke / Content Stress / Screenshot Comparison), the IMPLEMENT_SIMPLE_AUTOMATION_TEST and IMPLEMENT_COMPLEX_AUTOMATION_TEST macros, the BDD-style Automation Spec API (DEFINE_SPEC / BEGIN_DEFINE_SPEC / Describe / It / BeforeEach / LatentIt / xIt), latent commands (ADD_LATENT_AUTOMATION_COMMAND), the Automation Driver for UI input simulation (IAutomationDriverModule::Get().CreateDriver(), By::Id / By::Path locators), running via Session Frontend (Window > Test Automation) and command line (-ExecCmds="Automation RunTests …"), and CI integration. Use when the unit under test is C++ Unreal code that needs the UE runtime, editor, or UMG UI surface.