testng-tests

Configures and runs TestNG - JVM testing framework with `@Test` priorities + groups + `dependsOnMethods`; `@DataProvider` for parametrized tests with method-level data sources; `testng.xml` suite definitions for grouping + parallelism config; listeners (`ITestListener`, `ISuiteListener`) for hooks; `ITestContext` for cross-test state; integrates with Maven Surefire / Gradle. Use when working with legacy TestNG codebases or needing TestNG-specific features (test method dependencies, suite-level XML config).

Install with skills.sh (any agent)

npx skills add testland/qa --skill testng-tests

View source

testng-tests

Overview

Per testng.org (opens in new window), TestNG (Test Next Generation) was the original JUnit-improvement project (~2004). Distinguishing features at the time:

Test method dependencies (dependsOnMethods / dependsOnGroups)
Test groups (logical grouping for selective runs)
Suite XML configuration (testng.xml, declarative test combinations)
DataProviders (method-source parametrization)

JUnit 5 has since adopted most of these via @ParameterizedTest, @MethodSource, @Nested, and @Disabled. New projects mostly default to JUnit 5; TestNG persists for legacy maintenance and teams preferring its specific patterns.

When to use

Maintaining a legacy TestNG codebase.
Test-method dependency requirements (rare; usually a smell, but legitimate for stage-gated integration tests).
Selenium-tradition projects (TestNG is common in the Selenium ecosystem).

For new code, prefer junit5-tests.

How to use

Add the TestNG dependency and switch the build's test task to the TestNG runner (see Install).
Write a test class with @Test methods - keep TestNG's assertEquals(actual, expected) argument order in mind (reversed from JUnit).
Parametrize repeated cases with a @DataProvider (see Worked example).
Run the suite (./gradlew test or mvn test) and reproduce a single failure by method name.
For the full lifecycle-annotation set, dependsOnMethods chains, groups, testng.xml suites / parallelism, listeners, and CI wiring, see references/annotations-suites-and-ci.md.

Install

build.gradle.kts:

dependencies {
    testImplementation("org.testng:testng:7.10.2")
}

tasks.test {
    useTestNG()
}

Maven:

<dependency>
    <groupId>org.testng</groupId>
    <artifactId>testng</artifactId>
    <version>7.10.2</version>
    <scope>test</scope>
</dependency>

Worked example

One TestNG test end to end - a plain @Test plus a @DataProvider-driven parametrized test against the same Calculator.

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
import static org.testng.Assert.assertEquals;

public class CalculatorTest {
    @Test
    public void addsTwoNumbers() {
        assertEquals(Calculator.add(1, 2), 3);
    }

    @DataProvider(name = "addCases")
    public Object[][] addCases() {
        return new Object[][] {
            {1, 2, 3},
            {0, 0, 0},
            {-1, 1, 0},
        };
    }

    @Test(dataProvider = "addCases")
    public void testAdd(int a, int b, int expected) {
        assertEquals(Calculator.add(a, b), expected);
    }
}

Note: TestNG's assertEquals signature is (actual, expected), reversed from JUnit. Easy source of bugs when migrating.

Run the suite:

./gradlew test    # or: mvn test

A @DataProvider method can also live in a separate class:

@Test(dataProvider = "addCases", dataProviderClass = TestData.class)
public void testAdd(int a, int b, int expected) { ... }

Anti-patterns

Anti-pattern	Why it fails	Fix
`assertEquals(expected, actual)` (JUnit order)	TestNG order is reversed; failure messages misleading	Use `assertEquals(actual, expected)` (Worked example)
Heavy `dependsOnMethods` chains	Test order coupling; one failure cascades	Independent tests + setUp methods
Skip groups + run all tests in CI	Long CI cycle; slow tests block fast	Use groups + selective runs (references)
Suite XML without team agreement	Hidden test grouping; confusing	Document suite intent or skip XML in favor of annotations
Mix TestNG + JUnit in same project	Two runners	Pick one

Limitations

TestNG vs JUnit ecosystem split - fewer integrations, less StackOverflow coverage.
assertEquals argument order is opposite of JUnit (migration source of bugs).
testng.xml configuration adds complexity; same can usually be done with annotations.
Less active development than JUnit 5.

References

tn-docs (opens in new window) - TestNG documentation
references/annotations-suites-and-ci.md - lifecycle annotations, dependencies, groups, testng.xml suites, listeners, CI.
junit5-tests, kotest-tests, spock-tests, scalatest - sister tools
test-code-conventions

TestNG annotations, dependencies, suites, listeners, and CI

View source

TestNG annotations, dependencies, suites, listeners, and CI

Deep reference for testng-tests SKILL.md. Consult when wiring the full lifecycle-annotation set, test-method dependencies, group-based selective runs, testng.xml suites with parallelism, listeners, or Maven / Gradle CI integration.

Lifecycle annotations

Per tn-docs (opens in new window):

public class TestLifecycle {
    @BeforeSuite  void beforeSuite()  { /* once before suite */ }
    @AfterSuite   void afterSuite()   { /* once after suite */ }
    @BeforeClass  void beforeClass()  { /* once before class */ }
    @AfterClass   void afterClass()   { /* once after class */ }
    @BeforeMethod void beforeMethod() { /* before each test */ }
    @AfterMethod  void afterMethod()  { /* after each test */ }
    @BeforeGroups void beforeGroups() { /* before tests in named group */ }
    @AfterGroups  void afterGroups()  { /* after tests in named group */ }

    @Test
    public void test1() { ... }
}

Priorities and dependencies

public class OrderedTests {
    @Test(priority = 1)
    public void firstTest() { ... }

    @Test(priority = 2)
    public void secondTest() { ... }

    @Test
    public void independentTest() { ... }
}

public class DependentTests {
    @Test
    public void createUser() { ... }

    @Test(dependsOnMethods = "createUser")
    public void updateUser() {
        // only runs if createUser passed
    }

    @Test(dependsOnMethods = "updateUser")
    public void deleteUser() { ... }
}

Dependencies are a smell in unit tests (each unit test should be independent). Legitimate for stage-gated integration suites (e.g., "create resource, modify, delete"). Use sparingly.

Groups and selective runs

@Test(groups = "fast")
public void fastTest1() { ... }

@Test(groups = {"slow", "integration"})
public void slowIntegration() { ... }

@Test(groups = "fast", dependsOnGroups = "init")
public void afterInit() { ... }

Run selectively:

mvn test -Dgroups=fast
# Or via testng.xml suite

testng.xml suite definitions

<!-- testng.xml -->
<suite name="MySuite" parallel="methods" thread-count="4">
    <test name="FastTests">
        <groups>
            <run>
                <include name="fast"/>
                <exclude name="integration"/>
            </run>
        </groups>
        <classes>
            <class name="com.example.CalculatorTest"/>
        </classes>
    </test>

    <test name="IntegrationTests">
        <groups>
            <run><include name="integration"/></run>
        </groups>
        <packages>
            <package name="com.example.integration"/>
        </packages>
    </test>
</suite>

Run via mvn test -Dsurefire.suiteXmlFiles=testng.xml.

Listeners (cross-test hooks)

public class CustomListener implements ITestListener {
    @Override
    public void onTestStart(ITestResult result) { ... }

    @Override
    public void onTestFailure(ITestResult result) {
        // capture screenshot, log additional context, etc.
    }
}

Apply per-class:

@Listeners(CustomListener.class)
public class MyTest { ... }

CI integration

- run: ./gradlew test
# Or with TestNG XML config:
- run: mvn test -Dsurefire.suiteXmlFiles=testng.xml

JaCoCo coverage works identically to JUnit setups.

References

tn-docs (opens in new window) - TestNG documentation

Related skills

assertj

Reference for AssertJ - the canonical JVM fluent-assertion library pairable with JUnit 5 / TestNG / Spock; covers the assertThat() entry point, collection matchers (contains, containsExactly, allSatisfy, extracting), exception assertions (assertThatThrownBy, catchThrowable), SoftAssertions for multi-failure collection, recursive comparison (usingRecursiveComparison) for deep equality, and domain-specific custom assertions via AbstractAssert. Use when writing JVM tests that need richer failure messages than built-in assertEquals, or when verifying complex object graphs, exception types, or collections.

junit5-tests

Configures and runs JUnit 5 (Jupiter) - modern JVM testing platform with annotations (`@Test` / `@ParameterizedTest` / `@RepeatedTest` / `@TestFactory`), lifecycle hooks (`@BeforeAll` / `@BeforeEach` / `@AfterEach` / `@AfterAll`), extension model (`@ExtendWith`), display names (`@DisplayName`), conditional execution (`@EnabledOnOs`, `@EnabledIf`), parallel execution config; integrates with Maven Surefire / Gradle test task / IntelliJ. Use when the user works with Java / Kotlin codebases needing the modern JVM standard.

kotest-tests

Configures and runs Kotest - Kotlin-native test framework with multiple specification styles (StringSpec, FunSpec, BehaviorSpec, DescribeSpec, ShouldSpec, FreeSpec, FeatureSpec, ExpectSpec, AnnotationSpec); rich matcher library; built-in property-based testing (alternative to jqwik); coroutines support; data-driven testing; isolation modes per-spec or per-test; integrates with Gradle JVM test task. Use when working with Kotlin and wanting Kotlin-idiomatic DSL over JUnit 5's annotation-driven approach. Matchers (shouldBe, shouldContain) are bundled with the runner and are not a drop-in replacement for a standalone JVM assertion library; for assertion-only use paired with JUnit 5 / TestNG / Spock see assertj.

scalatest

Configures and runs ScalaTest - Scala-native test framework with multiple specification styles (FlatSpec, FunSuite, WordSpec, FreeSpec, AsyncFlatSpec for async); Matchers DSL (`should equal`, `should contain`, `shouldBe a [Class]`); integrates with ScalaCheck for property-based testing; sbt + Maven + Gradle support; tagged-test selective execution. Use when working with Scala codebases.

spock-tests

Configures and runs Spock - Groovy-based JVM testing framework with given/when/then BDD blocks, where: data tables for parametrized tests, built-in mocking via Mock()/Stub()/Spy(), interaction-based testing (verify method calls in declarative DSL), implicit assertions in then: blocks. Use when working with Java/Kotlin codebases that benefit from Groovy DSL expressiveness, or maintaining existing Spock projects.