living-documentation-publisher

Overview

Per the Serenity BDD book (serenity-bdd/the-serenity-book, living-documentation.adoc (opens in new window)):

"Living documentation is generated by the automated test suite, and is therefore by definition it is always up-to-date."

The same source distinguishes living documentation from plain test reports: living documentation is authored before development starts, targets the whole team (BAs, product owners, stakeholders), and reads as a business-functionality narrative, not a pass/fail table.

This skill covers the full workflow: produce Cucumber JSON from a passing suite, feed it into a report renderer, apply tag-based section routing, and publish the HTML artefact to a Pages endpoint so stakeholders get a URL, not a zip file.

Two primary renderers are covered:

• multiple-cucumber-html-reporter (Node, any Cucumber-JS project) - see references/node-reporter.md.
• Serenity BDD aggregate report (JVM, Maven/Gradle projects using CucumberWithSerenity) - see references/serenity-reporter.md.

When to use

• The team already has Cucumber scenarios (see cucumber-testing) and stakeholders need to read acceptance-criteria status without opening a test runner.
• The living-documentation page must track only passing scenarios (not every scenario authored, including pending or skipped ones).
• CI should gate report publication so stakeholders never see a stale or red-run document.

If only engineers read the results, the JUnit XML feed to junit-xml-analysis (in the qa-test-reporting plugin) is sufficient.

How to use

1. Produce Cucumber JSON from a passing suite - add a json: formatter (Cucumber-JS) or let CucumberWithSerenity capture results (JVM). Step 1.
2. Exclude @wip / @pending scenarios so the document shows only accepted behaviour. Step 4.
3. Render the JSON to HTML - Node path via references/node-reporter.md, JVM path via references/serenity-reporter.md. Steps 2-3.
4. Tag scenarios by capability and sprint so stakeholders can navigate report sections. Step 4.
5. Gate publication on a green exit code so no red or stale run ships. Step 5.
6. Publish the HTML to GitHub/GitLab Pages in CI so stakeholders get a URL. Step 6, via references/ci-publish.md.

Worked example

A Cucumber-JS "Checkout Service" suite has 24 scenarios tagged by capability (@billing, @cart) and sprint (@sprint-12); 2 are still @wip.

1. CI runs npx cucumber-js features/ --tags "not @wip" --format json:reports/cucumber.json, producing JSON for the 22 passing scenarios.
2. node scripts/generate-report.js renders ./docs/living-documentation/ under the report name "Checkout Service - Living Documentation".
3. Because set -e precedes the generate step, a red run aborts before publishing.
4. peaceiris/actions-gh-pages@v4 pushes the folder to GitHub Pages.
5. Result: the product owner opens the Pages URL and sees 22 green acceptance criteria grouped under Billing and Cart, with the two @wip scenarios absent - an always-current, business-readable status page.

Step 1 - Produce Cucumber JSON output

The report renderers both consume Cucumber JSON. Configure the JSON formatter before wiring the renderer.

Cucumber-JS (in package.json):

{
  "scripts": {
    "test": "cucumber-js features/ --format json:reports/cucumber.json"
  }
}

For parallel shards, use a timestamped JSON filename to avoid overwrite - see references/node-reporter.md.

Cucumber-JVM with Serenity (in serenity.properties):

The CucumberWithSerenity runner captures results automatically; no explicit JSON formatter is needed. Serenity writes its own JSON artefacts to target/site/serenity/ as part of mvn verify (serenity-bdd/the-serenity-book, maven.adoc (opens in new window)).

Step 2 - Generate the HTML report (Node path)

For any Cucumber-JS project, render the JSON with multiple-cucumber-html-reporter: install it, add a scripts/generate-report.js that points jsonDir at ./reports/ and reportPath at ./docs/living-documentation/, then run npm test && node scripts/generate-report.js. Full install, script, and options table: references/node-reporter.md.

Step 3 - Generate the HTML report (JVM / Serenity path)

For Maven/Gradle projects running CucumberWithSerenity, bind the serenity-maven-plugin aggregate goal to post-integration-test and run mvn verify. The Requirements tab renders living documentation from the feature-directory hierarchy. Full plugin config, hierarchy labels, and readme.md enrichment: references/serenity-reporter.md.

Step 4 - Tag scenarios for report sections

Use Gherkin tags to categorise scenarios in both renderers.

In the feature file:

@billing @sprint-12
Scenario: Apply promo at checkout
  Given ...

Serenity tag filtering - run only tagged tests and limit the aggregate report to those requirements (serenity-bdd/the-serenity-book, filtering-reports.adoc (opens in new window)):

# Run and report on one sprint
mvn clean verify -Dcucumber.options="--tags=@sprint-12" \
  -Dtags=sprint-12

# Post-run report filtered to a tag
mvn serenity:aggregate -Dtags=sprint-12

Note: "requirements filtering only happens if you specify the tags option" (serenity-bdd/the-serenity-book, filtering-reports.adoc (opens in new window)).

Excluding pending/WIP from published docs:

# Cucumber-JS: exclude anything tagged @wip from the JSON
npx cucumber-js features/ \
  --tags "not @wip" \
  --format json:reports/cucumber.json

This keeps the living-documentation page free of scenarios that are not yet passing.

Step 5 - Only publish passing runs

Gate the report publication step on a green test exit code. In a shell script:

set -e
npm test                         # exits non-zero on any failure
node scripts/generate-report.js  # only reached if all tests pass

For Serenity, use serenity:check after verify (serenity-bdd/the-serenity-book, maven.adoc (opens in new window)):

mvn verify serenity:check        # fails the build if any scenario is red

This ensures the published artefact reflects only a fully-green run.

Step 6 - Publish to GitHub Pages (CI)

Publish the generated HTML to GitHub or GitLab Pages so stakeholders get a URL, not a zip file. Full GitHub Actions and GitLab Pages jobs (including the Serenity publish_dir override): references/ci-publish.md.

Anti-patterns

Limitations

• Serenity aggregate requires JVM test execution in the same Maven lifecycle. Running serenity:aggregate against JSON files from a Cucumber-JS run is not supported; use multiple-cucumber-html-reporter for Node projects.
• multiple-cucumber-html-reporter merges JSON by filename, not by run order. If parallel shards write JSON with identical names, results are silently overwritten. Use timestamped or shard-indexed filenames.
• Pages publish latency. GitHub Pages can take 60-120 seconds to reflect a push; stakeholders may see the previous version briefly after a CI run.
• Serenity requirements hierarchy is derived from directory layout. Renaming a feature directory breaks historical trend data in the report.

References

• ld (opens in new window) - Serenity BDD book, living-documentation.adoc: definition, Requirements tab, hierarchy, readme.md enrichment, evidence API.
• mv (opens in new window) - Serenity BDD book, maven.adoc: serenity-maven-plugin coordinates, aggregate goal, post-integration-test phase binding, mvn verify, serenity:check.
• fr (opens in new window) - Serenity BDD book, filtering-reports.adoc: -Dtags, -Dcucumber.options, requirements-filtering caveat.
• references/node-reporter.md - Node renderer install, generate-report.js, and the full options table.
• references/serenity-reporter.md - Serenity Maven plugin, aggregate goal, requirements hierarchy, readme.md enrichment.
• references/ci-publish.md - GitHub Actions and GitLab Pages publish jobs.
• cucumber-testing - upstream skill: produce Cucumber JSON with --format json:.
• junit-xml-analysis (in the qa-test-reporting plugin) - engineer-facing test result analysis (separate concern from stakeholder docs).
• github-actions-test-jobs (in the qa-ci-integration plugin) - CI job conventions for the publish step.