> For the complete documentation index, see [llms.txt](https://kerno.gitbook.io/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://kerno.gitbook.io/docs/guides/run-tests-in-ci.md). # How to Run your Baseline Tests in CI Learn how to replay your committed baseline tests against your application on every pull request, and read the result as a check. ### Introduction The baseline tests Kerno writes are files in your repository. Once they are committed, anything that can run them can run them, including your CI. This guide wires up the Kerno GitHub Action. On every pull request it replays the baseline tests you have committed against your running application and reports the result as a check, so behaviour that moved is caught where your team already reviews changes rather than on one developer's machine. It needs **no Kerno account, no API key and no agent**. The action pulls one public image and runs the tests already in your repository. Nothing calls a language model, which is also why it is fast and free to run. ### Prerequisites Before you begin, you will need: * Baseline tests committed under `/.kerno/scenarios`. See [How to Create Baseline Tests for your Endpoints](/docs/guides/capture-a-baseline.md). * A way to start your application in CI, and a URL it answers on. * A Linux runner with Docker. `ubuntu-latest` works as-is. {% hint style="info" %} Tests that read a database, mint tokens from a shared secret, or call a downstream service need values you would never commit. [Step 4](#step-4-tests-that-need-configuration) covers passing those in from secrets. {% endhint %} ### Step 1. Adding the workflow Create `.github/workflows/kerno.yaml`: ```yaml name: kerno on: pull_request jobs: kerno: runs-on: ubuntu-latest steps: - uses: actions/checkout@v5 # Your application. Start it however you already do — Kerno only connects to it. - run: docker compose up -d --wait - run: | for _ in $(seq 1 30); do curl -fsS -o /dev/null http://localhost:8080/health && exit 0 sleep 2 done echo "app did not become ready" >&2; exit 1 - uses: kernoio/kerno-check@v1 with: sut-url: http://localhost:8080 - uses: mikepenz/action-junit-report@v6 if: always() with: report_paths: kerno-junit.xml ``` Three things about that file are worth knowing: * **You start your application; Kerno connects to it.** The action never starts, builds or tears down your app. Whatever you already use, whether Compose, a service container, or `./gradlew bootRun &`, keep using it. * **Poll for readiness, not just for health.** `compose up --wait` waits for container health, which is not the same as your application serving requests. The loop above is worth copying. * **`localhost` is fine.** The tests execute inside a container, where `localhost` would mean the container itself, so the action rewrites a `localhost` or `127.0.0.1` URL to reach your runner. `@v1` follows every 1.x release. Pin `@v1.0.0` instead if you need a reference that never moves. ### Step 2. Reading the check The action writes a summary to the job page: ``` ⚠️ Kerno scenarios — my-service 30/47 passed — 17 skipped (never executed) against http://host.docker.internal:8080 ``` A run with nothing failed and nothing skipped is marked ✅; one with skips is marked ⚠️ so they are visible at a glance, and a failure is marked ❌. A failure is listed with the assertion that did not match and a diff of expected against actual, so you can see what moved without opening the logs. The JUnit report lands at `kerno-junit.xml`, which is what the reporter step turns into annotations on the changed lines. Each test comes back as one of the four verdicts described in [Baseline tests](/docs/core-concepts/scenarios-and-baselines.md#reading-test-results). JUnit has three states, so they map like this: | Kerno verdict | In the report | Fails the check | | --------------- | ------------- | --------------- | | Passed | passed | no | | Failed | failed | **yes** | | Blocked | skipped | no | | Not implemented | skipped | no | **Skipped tests never fail the check, and they are always counted separately.** A blocked test is a known state, meaning a dependency it needs is not configured, and a not-implemented one is reported honestly rather than counted as a pass. That is why the summary reads `30/47 passed — 17 skipped` rather than `47 passed`: a suite that asserts nothing should not look like coverage. ### Step 3. Choosing what runs By default the action discovers every `/.kerno/scenarios` tree in your repository and replays all of them, which is usually what a monorepo wants. To replay one application: ```yaml - uses: kernoio/kerno-check@v1 with: sut-url: http://localhost:8080 app-dir: services/orders ``` To replay a subset, filter on the path relative to the scenarios directory: ```yaml - uses: kernoio/kerno-check@v1 with: sut-url: http://localhost:8080 scenarios: endpoints/GET/** ``` `*` stays within a path segment and `**` crosses them. ### Step 4. Tests that need configuration Tests that read or seed a database, mint tokens from a shared secret, or call a downstream service need values that must not live in your repository. Name them in `forward-env` and supply them from secrets: ```yaml - uses: kernoio/kerno-check@v1 with: sut-url: http://localhost:8080 forward-env: | DATABASE_URL JWT_SECRET env: DATABASE_URL: ${{ secrets.DATABASE_URL }} JWT_SECRET: ${{ secrets.JWT_SECRET }} ``` Two properties worth relying on: * **Only the names you list are forwarded.** The rest of the runner's environment does not reach the container. * **A name with no value stops the run**, before the container starts, and says which variable is missing. That matters more than it sounds: a test that needed `DATABASE_URL` and did not get it fails at its database step while every HTTP assertion around it still passes, which reads as a partial success rather than a configuration mistake. ### Step 5. A monorepo with services on different ports `sut-url` gives every application the same address. When your services listen on different ports, map each one instead: ```yaml - uses: kernoio/kerno-check@v1 with: apps: | services/orders=http://localhost:8080 services/billing=http://localhost:8081 forward-env: | DATABASE_URL env: DATABASE_URL: ${{ secrets.DATABASE_URL }} ``` Each directory is the one containing `.kerno`, relative to the repository root, and each application is replayed against its own URL. `apps` replaces `sut-url`, and setting both is rejected rather than one silently winning. One report is written per application, so `report-path` is a **directory** in this mode, and the reporter step takes a glob: ```yaml - uses: kernoio/kerno-check@v1 with: apps: | services/orders=http://localhost:8080 services/billing=http://localhost:8081 report-path: kerno-reports - uses: mikepenz/action-junit-report@v6 if: always() with: report_paths: kerno-reports/*.xml ``` The counts are summed across every application, so one failing test in one service fails the check. ### What this does not do * **It does not create or update baseline tests.** That happens on a developer's machine, where the proposed tests can be reviewed before they are committed. CI replays what is already in the repository and nothing else. * **It does not start your application.** You start it and pass `sut-url`. Kerno connects to a system under test; it never manages one. * **It only forwards the environment variables you name.** Nothing else from the runner reaches your tests, and a name with no value stops the run rather than being quietly dropped. ### Reference **Inputs** | Input | Required | Default | | | ----------------- | ------------- | ------------------- | --------------------------------------------------------------------------------------------------- | | `sut-url` | unless `apps` | | Base URL of your running application. A `localhost` URL is rewritten so the container can reach it. | | `apps` | no | | One `=` per line, replaying each application against its own URL. Replaces `sut-url`. | | `forward-env` | no | | Environment variable names to pass to your tests, one per line, valued from the step's own `env:`. | | `app-dir` | no | *(repository root)* | Replay one application's tests. Unset discovers every `/.kerno/scenarios` tree. | | `scenarios` | no | *(all)* | Glob filter on the path relative to the scenarios directory. | | `image` | no | *(pinned digest)* | The runner image. Pinned by digest so a given version of the action always runs the same code. | | `report-path` | no | `kerno-junit.xml` | Where the JUnit report lands. A directory when `apps` is used. | | `fail-on-failure` | no | `true` | Set `false` to report without gating. | **Outputs**: `junit-path`, `total`, `passed`, `failed`, `skipped`. **Exit codes** | Code | | | ---- | --------------------------------------------------------------------------------------------------------- | | `0` | Nothing failed. Tests may have been skipped, so check the counts. | | `1` | At least one test failed. | | `2` | Configuration error: no tests found, or an unparseable or empty report. Never reported as a test failure. | | `3` | The runner could not start. A broken container must not look like a failing test. | {% hint style="info" %} The action gives the runner the `NET_ADMIN` capability so Kerno can intercept outbound HTTPS from your application. That is how a test can exercise a path that calls a third-party API without that API being reachable from CI. {% endhint %} ### Conclusion Your committed baseline tests now run on every pull request, and their exit code is your gate. Tests are still authored and reviewed locally, covered in [How to Review your Code Changes](/docs/guides/validate-code-changes.md), and CI is where the whole team finds out when behaviour moves.