What the validator checks
JUnit XML has no single official schema: each test runner writes its own dialect and each CI reader accepts its own subset. The validator checks what makes readers disagree about a file, in three levels.
Errors make readers reject the file or show wrong numbers:
| Rule | What it catches |
|---|---|
xml-malformed | The file is not well-formed XML: an unclosed tag, a stray &, a second root. Often a runner killed mid-write. |
root-element | The root is not <testsuites> or <testsuite>. |
count-mismatch | tests, failures, errors or skipped on a suite disagrees with the test cases inside it. |
count-not-integer | A count is not a whole number. |
time-not-number | A time is not a number of seconds, including a decimal comma such as 1,75 from a localized writer. |
time-negative | A time below zero. |
case-name-missing | A <testcase> without a name. |
Warnings are valid XML that loses information:
| Rule | What it catches |
|---|---|
case-duplicate | Two test cases with the same class and name in one suite, so history and flaky tracking can't tell them apart. |
case-multiple-results | A test case with both <failure> and <skipped>, or two results; readers pick different ones. |
classname-missing | No classname, so the test is grouped under the suite or shown without a class. |
suite-empty | A suite with no test cases. A run that collected no tests often still exits green. |
suite-name-missing | A suite without a name. |
time-thousands-separator | A time such as 1,234.567. The Jenkins junit-10.xsd pattern allows the commas, but a reader that parses the value as a plain number gets nothing. |
timestamp-invalid | A suite timestamp that is not an ISO 8601 date-time. |
case-outside-suite | A <testcase> directly under <testsuites>, which some readers skip. |
Notes point at gaps: a test case without a time (duration charts and shard balancing count it as zero), a failure with no message, disabled used where readers look for skipped, and elements that are not part of JUnit XML.
Retry elements that Maven Surefire writes, <flakyFailure>, <flakyError>, <rerunFailure> and <rerunError>, are known and not flagged.
Why the counts matter most
Tools read the same file in two ways. Some take the counts from the <testsuite> attributes, which is cheap; others count the <testcase> elements. When a script merges files, a writer crashes, or a retry plugin appends test cases without updating the totals, the two drift. The dashboard then says 2 failures while the test list shows 3, and a release gate that reads the attribute passes a failing build. The count-mismatch rule exists for that case.
Checking files in CI
This page checks a file you have in hand. To fail a build on a broken report, add a step after the tests that parses the file before it is uploaded; the JUnit XML analysis skill walks through why CI numbers disagree with the run and how to fix the writer.
Sources
Testland for teams
In developmentAutomated test reports, manual test runs and regression tracking in one workspace. A separate product, built on the same skills.
One email when it opens, nothing else. Privacy policy