> For the complete documentation index, see [llms.txt](https://kerno.gitbook.io/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://kerno.gitbook.io/docs/references/testing-modes.md).

# Testing modes

These dials set the ratio of test quality to speed you want, and how much access Kerno gets while doing it.

`kerno_endpoint_test` takes one required dial and several optional ones. The defaults are deliberately thorough, so you only need to reach for these when you want something faster, narrower, or more cautious.

{% hint style="info" %}
**Kerno stores none of these preferences.** If you settle on a set you like, your agent writes them into your own rules file so they are diffable, reviewable in a pull request, and travel with the repo. See [Saving your preferences](#saving-your-preferences).
{% endhint %}

### `type` — what the run does

Required. No default.

| Value      | What it does                                                                                                                             |
| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| `generate` | Analyzes the endpoint, plans scenarios, **pauses for your approval**, then implements and runs them. Use this the first time.            |
| `validate` | Runs the scenarios already on disk. No planning, no implementation. Use this after a code change to see whether behaviour moved.         |
| `update`   | Conservatively repairs existing scenarios after an intentional change. Keeps what still applies and edits only what the change requires. |

`validate` and `update` both need scenarios to exist. If none do, the run ends immediately with `needs_generate` and a message telling you to generate first.

**Choosing between `update` and `generate`.** Use `update` when your endpoint changed on purpose and the tests should follow: it defaults every existing scenario to "keep", makes surgical edits, and never deletes the happy path. Use `generate` when you want to start the endpoint's coverage over.

`generate` always pauses for plan approval. `update` only pauses if you set `interactive: true`.

### `effort` — how hard Kerno tries

Optional. Defaults to **`high`**.

| Value    | Runs per scenario                 | Adversarial review | What you get                                                                                                                                            |
| -------- | --------------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `low`    | 1                                 | no                 | The fastest, cheapest signal. The scenario is written and proven to pass once.                                                                          |
| `medium` | 2 consecutive, both must be clean | no                 | Proof the scenario is genuinely re-runnable, plus a second sample of randomized test data.                                                              |
| `high`   | 2 consecutive                     | yes                | Everything `medium` gives, plus an implement-critique-repair loop that rejects weakened, tautological, or swallowed assertions and incomplete coverage. |

The second run is not noise suppression. If a scenario passes once and fails the second time, that is a real finding: either its own setup and cleanup are not isolated, or the fresh random data hit an edge case the first sample missed. Both are worth knowing.

{% hint style="info" %}
`type: validate` runs existing scenarios rather than implementing new ones, so it performs a single run regardless of the `effort` you pass.
{% endhint %}

### `box_testing_strategy` — how much access Kerno gets

Optional. Defaults to **`white_box`**.

|                                              | `black_box`                | `white_box`                                                               |
| -------------------------------------------- | -------------------------- | ------------------------------------------------------------------------- |
| Datastore drivers available to scenario code | no                         | yes                                                                       |
| Database schema in the planning prompt       | suppressed                 | included                                                                  |
| Assertions about stored state                | not planned                | planned where useful                                                      |
| How preconditions and results are handled    | through your HTTP API only | through the API, falling back to the datastore where the API cannot do it |

**Black box** means Kerno only talks to your application over HTTP. Every precondition is established and every result verified through your own endpoints.

**White box** means Kerno may additionally read and write your datastore directly, for the cases your API cannot express: seeding a record that has no creation endpoint, or checking that a stored password is not the plaintext that was sent. This is what the datastore blocks in `kerno_save_config` enable. See [Environment Setup](/docs/core-concepts/environment-setup.md) for wiring them up.

Two things that surprise people:

* **Your datastore credentials still reach the sandbox under `black_box`.** The dial works by controlling planning and access. Drivers are withheld, the schema is suppressed, and preconditions and results run over HTTP. The environment still receives your credentials.
* **`black_box` does not hide your source code** from Kerno's planner. It constrains how scenarios interact with your running system, not what Kerno reads while writing them.

Configuring datastore access does not lock you into white box. Any individual run can opt out by passing `box_testing_strategy: black_box`.

{% hint style="warning" %}
**White box needs a schema it can derive from your source.** To read or write your datastore directly, Kerno first works out its shape from your source code, for example schema files, migrations, or ORM models. If it cannot derive one, any scenario that needs direct datastore access is marked blocked rather than run, and Kerno may ask you once to point it at a schema. Scenarios that only exercise HTTP are unaffected.
{% endhint %}

### `tags` — what kind of coverage

Optional. Defaults to `validation`.

| `tags`                      | Happy path | Functional coverage | Security scenarios |
| --------------------------- | ---------- | ------------------- | ------------------ |
| omitted or `["validation"]` | yes        | yes                 | no                 |
| `["security"]`              | yes        | **no**              | yes                |
| `["validation","security"]` | yes        | yes                 | yes                |

Note the middle row: `security` on its own **narrows** the run rather than adding to it. If you want your normal functional coverage plus security scenarios, pass both tags.

The happy path is always planned, whatever you pass.

See [Security testing](broken://pages/O6yzFP71JRsEJbcJIs0s) for what the `security` tag actually produces.

### `test_generation_context` — guidance for this endpoint

Optional free text describing what Kerno should know or do differently for this endpoint.

Its meaning depends on `type`:

* **`generate`** — complete guidance for planning. Passing a **different** value than last time discards the scenarios on disk and re-plans from scratch, so resend your previous guidance in full when you are adding to it.
* **`update`** — change intent: what changed and how the tests should adapt. It does not wipe existing scenarios.
* **`validate`** — ignored.

**Relationship to workspace config.** Durable rules belong in your Kerno config under `test-generation`, where they are version-controlled and apply automatically — see [Custom rules](/docs/core-concepts/custom-rules.md). The two are combined for each run, with this per-call argument taking precedence where they conflict.

{% hint style="warning" %}
Only the per-call argument triggers a re-plan. Editing `test-generation` rules in `.kerno/config.yaml` changes the guidance Kerno receives on the next `generate`, but existing scenarios are kept rather than re-planned. To force config changes through, either pass a changed `test_generation_context` or remove the endpoint's scenarios first.
{% endhint %}

### `interactive` — step through the run

Optional, defaults to `false`. When true, Kerno pauses before implementing each scenario to ask whether to proceed, skip, or take feedback, and pauses again if automatic retries are exhausted. For `type: update`, it also restores the plan-approval gate.

These pauses are gates, not failures. They surface through `kerno_feedback_pending`, not through `kerno_job`.

### `scenario_ids` — target specific scenarios

Optional. When provided, only the named scenarios are implemented or run; the rest are skipped. Omit it to target every scenario.

Note that this filters implementation and execution but not planning: a `generate` run still plans the full set. If none of the ids match, the call fails rather than quietly doing nothing.

### Saving your preferences

Kerno is stateless about your habits. It stores no testing preferences, and it does not learn them silently over time.

Instead, every endpoint-test response echoes a `resolved_intent` object showing what was used and which values had to be defaulted:

```json
"resolved_intent": {
  "effort": "high",
  "box_testing_strategy": "white_box",
  "tags": ["validation"],
  "defaulted": ["effort", "box_testing_strategy", "tags"]
}
```

If you have a standing preference, your coding agent writes it into your own rules file — `CLAUDE.md`, `.cursor/rules`, or `AGENTS.md` — as a pattern-matched table. That keeps it diffable, reviewable in a pull request, and travelling with the repository.

Ask your agent for `kerno_guide` with topic `rules_template` to get the table format. The rule is that the most specific endpoint pattern wins, and you should always keep one `*` fallback row describing what a bare call should resolve to for your codebase.