> For the complete documentation index, see [llms.txt](https://kerno.gitbook.io/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://kerno.gitbook.io/docs/references/testing-modes.md).

# Testing modes

Every dial on an entry point test: what to generate, how hard to try, and how much access Kerno gets.

These dials set the ratio of test quality to speed you want, and how much access Kerno gets while doing it.

`kerno_endpoint_test` takes one required dial and several optional ones. The defaults are deliberately thorough, so you only need to reach for these when you want something faster, narrower, or more cautious.

{% hint style="info" %}
**Kerno stores none of these preferences.** If you settle on a set you like, your agent writes them into your own rules file so they are diffable, reviewable in a pull request, and travel with the repo. See [Saving your preferences](#saving-your-preferences).
{% endhint %}

### `type`, what the run does

Required. No default.

| Value      | What it does                                                                                                                             |
| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| `generate` | Analyzes the entry point, plans scenarios, **pauses for your approval**, then implements and runs them. Use this the first time.         |
| `update`   | Conservatively repairs existing scenarios after an intentional change. Keeps what still applies and edits only what the change requires. |

`update` needs scenarios to exist. If none do, the run ends immediately with `needs_generate` and a message telling you to generate first.

To re-run the scenarios already on disk after a code change, use the separate `kerno_validate` tool. It runs existing scenarios only, with no planning and no implementation. See [Kerno MCP](/docs/references/kerno-mcp.md).

**Choosing between `update` and `generate`.** Use `update` when your entry point changed on purpose and the tests should follow: it defaults every existing scenario to "keep", makes surgical edits, and never deletes the happy path. Use `generate` when you want to start the entry point's coverage over.

`generate` always pauses for plan approval. `update` only pauses if you set `interactive: true`.

### `effort`, how hard Kerno tries

Optional. Defaults to **`high`**. An entry point your team marked critical in the Portal always runs at `high`, whatever you pass.

| Value    | Runs per scenario                 | Critique and repair | What you get                                                                                                                                            |
| -------- | --------------------------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `low`    | 1                                 | no                  | The fastest, cheapest signal. The scenario is written and proven to pass once.                                                                          |
| `medium` | 2 consecutive, both must be clean | no                  | Proof the scenario is genuinely re-runnable, plus a second sample of randomized test data.                                                              |
| `high`   | 2 consecutive                     | yes                 | Everything `medium` gives, plus an implement-critique-repair loop that rejects weakened, tautological, or swallowed assertions and incomplete coverage. |

The second run is not noise suppression. If a scenario passes once and fails the second time, that is a real finding: either its own setup and cleanup are not isolated, or the fresh random data hit an edge case the first sample missed. Both are worth knowing.

{% hint style="info" %}
`kerno_validate` re-runs the scenarios already on disk and takes no `effort` argument.
{% endhint %}

### `box_testing_strategy`, how much access Kerno gets

Optional. Defaults to **`white_box`**.

|                                              | `black_box`                | `white_box`                                                               |
| -------------------------------------------- | -------------------------- | ------------------------------------------------------------------------- |
| Datastore drivers available to scenario code | no                         | yes                                                                       |
| Database schema in the planning prompt       | suppressed                 | included                                                                  |
| Assertions about stored state                | not planned                | planned where useful                                                      |
| How preconditions and results are handled    | through your HTTP API only | through the API, falling back to the datastore where the API cannot do it |

**Black box** means Kerno only talks to your application over HTTP. Every precondition is established and every result verified through your own entry points.

**White box** means Kerno may additionally read and write your datastore directly, for the cases your API cannot express: seeding a record your API has no way to create, or checking that a stored password is not the plaintext that was sent. This is what the datastore blocks in `kerno_save_config` enable. See [Environment Setup](/docs/core-concepts/environment-setup.md) for wiring them up.

Two things that surprise people:

* **Your datastore credentials still reach the sandbox under `black_box`.** The dial works by controlling planning and access. Drivers are withheld, the schema is suppressed, and preconditions and results run over HTTP. The environment still receives your credentials.
* **`black_box` does not hide your source code** from Kerno's planner. It constrains how scenarios interact with your running system, not what Kerno reads while writing them.

Configuring datastore access does not lock you into white box. Any individual run can opt out by passing `box_testing_strategy: black_box`.

{% hint style="warning" %}
**White box needs a schema it can derive from your source.** To read or write your datastore directly, Kerno first works out its shape from your source code, for example schema files, migrations, or ORM models. If it cannot derive one, any scenario that needs direct datastore access is marked blocked rather than run, and Kerno may ask you once to point it at a schema. Scenarios that only exercise HTTP are unaffected.
{% endhint %}

### `tags`, what kind of coverage

Optional. Defaults to `validation`.

| `tags`                      | Happy path | Functional coverage | Security scenarios |
| --------------------------- | ---------- | ------------------- | ------------------ |
| omitted or `["validation"]` | yes        | yes                 | no                 |
| `["security"]`              | yes        | **no**              | yes                |
| `["validation","security"]` | yes        | yes                 | yes                |

Note the middle row: `security` on its own **narrows** the run rather than adding to it. If you want your normal functional coverage plus security scenarios, pass both tags.

The happy path is always planned, whatever you pass.

See [Security testing](/docs/references/security-testing.md) for what the `security` tag actually produces.

### `test_generation_context`, guidance for this entry point

Optional free text describing what Kerno should know or do differently for this entry point.

Its meaning depends on `type`:

* **`generate`.** Complete guidance for planning. Passing a **different** value than last time discards the scenarios on disk and re-plans from scratch, so resend your previous guidance in full when you are adding to it.
* **`update`.** Change intent: what changed and how the tests should adapt. It does not wipe existing scenarios.

**Relationship to workspace config.** Durable rules belong in your Kerno config under `test-generation`, where they are version-controlled and apply automatically. See [Custom rules](/docs/core-concepts/custom-rules.md). The two are combined for each run, with this per-call argument taking precedence where they conflict.

{% hint style="warning" %}
Only the per-call argument triggers a re-plan. Editing `test-generation` rules in `.kerno/config.yaml` changes the guidance Kerno receives on the next `generate`, but existing scenarios are kept rather than re-planned. To force config changes through, either pass a changed `test_generation_context` or remove the entry point's scenarios first.
{% endhint %}

### `interactive`, step through the run

Optional, defaults to `false`. When true, Kerno pauses before implementing each scenario to ask whether to proceed, skip, or take feedback, and pauses again if automatic retries are exhausted. For `type: update`, it also restores the plan-approval gate.

These pauses are gates that wait for your answer. `kerno_job` reports a paused run as `needs_user_feedback` with the open question, and `kerno_feedback_pending` lists it too.

### `scenario_ids`, target specific scenarios

Optional. When provided, only the named scenarios are implemented or run; the rest are skipped. Omit it to target every scenario.

Note that this filters implementation and execution but not planning: a `generate` run still plans the full set. If none of the ids match, the call fails rather than quietly doing nothing.

### Saving your preferences

Kerno is stateless about your habits. It stores no testing preferences, and it does not learn them silently over time.

Instead, every entry point test response echoes a `resolved_intent` object showing what was used and which values had to be defaulted:

```json
"resolved_intent": {
  "effort": "high",
  "box_testing_strategy": "white_box",
  "tags": ["validation"],
  "defaulted": ["effort", "box_testing_strategy", "tags"]
}
```

When anything was defaulted, it also carries a `save_to_rules_hint`. When a critical mark raised the effort, it carries `effort_forced_by: "critical_endpoint"`. `kerno_endpoint_test_many` reports only `effort_forced_by`, for each entry point.

If you have a standing preference, your coding agent writes it into your own rules file, `CLAUDE.md`, `.cursor/rules`, or `AGENTS.md`, as a pattern-matched table. That keeps it diffable, reviewable in a pull request, and travelling with the repository.

Ask your agent to use the `kerno-rules-template` skill to get the table format. The rule is that the most specific entry point pattern wins, and you should always keep one `*` fallback row describing what a bare call should resolve to for your codebase.