Testing

The platform's tests share one harness library, one discovery runner, and one web dashboard. Every test declares a small metadata header that says what it touches and where it may run; the runner reads that header to decide what to execute, and both the CLI and the dashboard consume the same result contract.

The two axes: tier and env

Every test declares a tier (blast radius — what it touches) and an env (where it is allowed to run). They are separate on purpose.

tier — drives which batch a test runs in:

The metadata header

Every runnable test (.php or a .sh gate) carries a header comment. It is read without executing the file, so the runner can enforce tiers before running anything:

/** @joinery-test
 * name: cloud_offload_engine
 * tier: safe            # safe | db | test-db | live
 * env: dev-only         # any | prod-verify | dev-only
 * needs: []             # e.g. [stripe-test-keys, macmini, mailgun, b2, rust, curl, chrome, test-cloud, agent-source]
 * covers: []            # optional repo-relative globs the suite reaches WITHOUT loading
 * timeout: 180          # optional wall-clock cap in seconds (default 180, max 1800; 5400 for a live suite)
 * parallel: true        # optional, safe tier only: runs beside other suites
 */

parallel: true puts a safe-tier suite in a pool that runs three at a time ahead of the serial batch. Declare it only for a suite that shares nothing written: no database writes (not even rolled-back ones), no site settings, no mail, no fixed file paths (temp names carry getmypid() or random bytes), no servers or ports, and no assertion on a machine-wide quantity such as free disk space. Two parts of that are enforced. The harness makes a parallel suite's database session read-only, so a write fails on the spot, and skips the shared mail and fixture cleanup passes. The runner fails any suite outside the safe tier that declares it. --serial and --only run parallel suites serially.

Set timeout: only when a suite genuinely needs longer than the 180-second default — a multi-minute build gate or a live third-party flow. The runner kills a test that exceeds its cap and marks it failed. Tests that explicitly declare a timeout beyond the web request window run from the CLI only (see the dashboard section).

A file that looks like a test (*_test.php, test_*.php, or a *_gate.sh) but has no header is listed as undeclared in the runner's summary — never silently skipped.

Running tests

From the command line:

php tests/run.php              # the safe tier
php tests/run.php db           # safe + db + test-db — the pre-deploy gate
php tests/run.php test-db      # only the test-database suites
php tests/run.php live         # only live tests (never implied)
php tests/run.php deploy       # does the deployed code run here — what upgrade.php runs
php tests/run.php --changed         # only the suites the edited files can reach
php tests/run.php db --changed      # the same narrowing, over the db gate
php tests/run.php db --changed=origin/main  # edited + everything different from a ref
php tests/run.php db --filter=api   # narrow by name or path substring
php tests/run.php db --serial       # disable the test-db lane overlap (fully serial)
php tests/run.php --only=tests/unit/dns_resolver_test.php  # one exact test by repo-relative path
php tests/run.php db --timeout=30   # override every test's declared wall-clock cap (seconds)
php tests/run.php --list       # list discovered tests, run nothing
php tests/run.php --json       # emit the aggregate JSON contract (for CI)
php tests/run.php db --fresh   # run every selected suite; borrow no earlier result
php tests/run.php db --no-wait # exit 3 at once if another run holds the gate
php tests/run.php --help       # the usage text; runs nothing

An option the runner does not declare is refused with the usage text and exit 2, never ignored: a misspelled --onyl= must not quietly run the whole safe tier in place of the one test that was meant. --only takes one path; a comma list matches nothing and the run says so.

safe, db and test-db are cumulative, so php tests/run.php db is the pre-deploy gate and covers the model CRUD suite. live runs only when named and never pulls in the others — it has real external effects, so it is always an explicit choice. Each test runs in its own subprocess, so a fatal in one file cannot take down the run. The runner exits non-zero if any test failed — it is the CI entry point.

When a run selects both test-db suites and anything else, the test-db suites run as one serial lane alongside the main batch, hiding their wall clock inside it. The overlap is safe because those suites write only to the copied test database — every write goes through DbConnector::set_test_mode(), and harness_finish() fails any test-db-tier suite whose process never entered test mode, so the isolation is enforced rather than assumed. The suites stay serial within the lane (they share the copy), lane results carry a [test-db] tag in the output, and a lane failure — including a crash of the lane worker — fails the gate. --serial forces the fully serial order, for debugging or as a fallback.

Every PHP suite shares one compile cache ({site root}/cache/tests/opcache), checked against each file's modification time on every include. Temp directories and the site's cache/ are excluded, because suites write PHP there and rewrite it within the same second. The deploy tier does not use it.

--changed: run what the change can reach

--changed narrows a run to the suites that can actually be affected by what is edited (git status: staged, unstaged, and untracked; --changed=<ref> adds everything different from the ref). It never changes what the full gate means — php tests/run.php db with no flags remains the complete pre-publish gate.

How a suite's reach is known. A PHP suite reaches code by loading it, so its result contract carries the repo files its process included (files in the JSON contract). Every run folds those into the coverage map at {site root}/cache/test_coverage_map.json, so the map is never older than the last time each suite ran, and a full run refreshes all of it. Files a suite reaches without loading them — a manifest it reads, a script it runs, the code a shell gate builds — are declared in its header as covers: globs (** crosses directories, * stays within one segment); a shell gate's covers: is its whole declaration.

Finer reach: the functions a suite ran. Where the PHP extension xhprof is loaded (on dev: sudo apt install php8.3-xhprof; nothing installs it elsewhere, and a box without it is simply file level), the runner starts each PHP suite with tests/lib/function_trace.php prepended (-d auto_prepend_file), which turns on xhprof's function trace before the suite's first line. The contract then also carries functions: every repository function and method the process ran. The runner stores each distinct list once, under {site root}/cache/test_fnsets/<id>.json; a result and a coverage map entry carry only its id (functions_set). The trace costs a small suite about 5-10% and a long one nothing measurable.

Selection rules. A suite in the requested batch runs when any of these holds, and the summary names which:

• the map has no entry for it — unknown means run, never skip;
• its own file changed, or a fixture in its area (tests/X/fixtures/ selects the suites under tests/X/; tests/fixtures/ selects everything);
• a changed file is in its recorded files or matches its covers;
• it drives the web server (needs: [dev-web]) and a core directory (includes/, data/, logic/, views/, adm/, api/, ajax/, theme/, serve.php) or its own plugin changed — Apache-side reach is not in the process's include list.

An edit inside function bodies reaches only the suites that ran them. When a changed PHP file differs only inside the bodies of its functions and methods, a suite that loaded it is selected only if its function set holds one of the changed functions, and the reason says which (`runs sessioncontrol::check_permission (public_html/includes/SessionControl.php)`). Every other kind of change selects every suite that loaded the file, as above: anything outside a body (top-level code, class headers, constants, properties, attributes, closures at file scope), a global function added, removed or renamed, a method added or removed (code probes for optional methods by name — method_exists($profile, 'backupRow') — so a method appearing or disappearing can change a suite that never ran one of that name), and a method's signature changed (an override that no longer matches breaks its class at load time). A suite with no function set — untraced, or its set pruned — is file level. Names compare in lower case without their namespace; a trait's methods, and those of an anonymous class at file scope, compare by method name alone, since the trace reports them under the class that uses them. A closure belongs to the function whose body holds it.

A change to tests/run.php, tests/lib/harness.php, tests/lib/coverage.php, tests/lib/discovery.php, tests/lib/gate.php or tests/lib/function_trace.php runs the whole batch: the harness changing invalidates every recorded reach, and no suite loads the runner, the selection, the discovery or the gate, so reach cannot see them. Every other file under tests/lib/ — a fixture library, the HTTP or logic helpers — is loaded by the suites that use it and recorded in their reach, so it selects only those. A stale or missing map can therefore only widen a run, never wrongly narrow it.

What the output says. The summary states how many changed files selected how many suites and why each ran; the aggregate JSON carries the same as selection. Changed code files that no suite in the estate reaches are listed as uncovered — a coverage gap named at the moment someone is looking (markdown, specs/ and docs/ are never listed). A --changed run that selects nothing exits 0: a docs-only change is legitimately green.

--changed refuses the live and deploy tiers (real effects and deployed nodes are never silently narrowed), refuses --only (which already names its test), and refuses to run without git and a repository.

One run at a time: the gate

Several agents share one box, and each runs the gate before checking in. A gate is a three-wide pool plus the test-database lane plus whatever a suite compiles, so two of them at once is more than four cores carry: the box stops answering, and the overlap is also where most flaky reds come from (two runs sharing one database and one tree). So the box runs one at a time.

A run selecting more than ten suites takes an exclusive lock on {site root}/cache/tests/gate.lock before its first suite and holds it until its last result is in. A second such run prints who holds the gate and how far along it is, then a progress line every thirty seconds, and starts the moment the lock frees:

Gate held by public-html-41 (pid 3408152): db --changed, 212/711 suites done, 0 failed, held 4m10s. Waiting (340 suites to run, more than the cutoff of 10)...
  still waiting (0m30s): db --changed, 260/711 suites done, 0 failed, held 4m40s
Gate acquired after 6m02s.

The kernel owns the lock, so a killed runner frees it at once and nothing needs cleaning up. gate.holder.json beside the lock names the holder — pid, tier, arguments, and the Claude session it runs under, resolved to the name the work launcher gave it — and is believed only while the lock is held. --no-wait exits with code 3 instead of waiting, for a caller that would rather come back later.

What does not queue:

• A run of ten suites or fewer — a --only, a narrow --filter, a --changed that reaches little. It runs at once, beside whatever holds the gate, so the working loop never waits behind a full run. The cutoff is GATE_QUEUE_CUTOFF in tests/lib/gate.php.
• The deploy tier, which runs once on a node after an upgrade swap with a rollback hanging on the result.
• Dashboard runs, which are one suite per request and so always below the cutoff. The dashboard shows the holder line while a gate runs.

A test of the runner spawns runners while the real gate is held: JOINERY_GATE_DIR relocates the lock and the run records, and JOINERY_GATE_CUTOFF lowers the cutoff so a one-suite run exercises the wait. Both are read only when the debug setting is on.

Run records and reuse

Every run leaves a record under {site root}/cache/tests/runs/: {run_id}.json written at the start (tier, arguments, holder, the tree id) and {run_id}.jsonl, one result per line as it lands (which is what a waiting runner reads for the holder's progress). The tree id is the git tree of the working tree as it stood when the run started — every tracked and untracked, non-ignored file under the repository root — made with git add -A into a temporary copy of the index and git write-tree, so the shared index and every prepared-commit index stay untouched. Records are kept for 48 hours.

Each result also says whether its reach is complete: the files the suite's process loaded (the same list the coverage map records) plus its covers globs are a complete statement of what the result depends on. That is true of a PHP suite that reported its files and of a shell gate, whose covers is its whole declaration. It is false for a crash or timeout, for a suite with needs: [dev-web] (Apache-side reach is not in its include list), and for live, prod-verify and deploy suites, which are never reused.

A run then borrows a suite's result from a record instead of running it when all of these hold:

1. the recorded reach is complete and the result is a pass. A fail is never borrowed: a red suite is re-proven on the current tree, which is also what lets a rerun alone tell a flaky suite from a broken one. A skip, crash or timeout is never borrowed either;
2. nothing in that reach differs between the record's tree and this run's — one git diff-tree per record answers it for every suite at once, and the decision is the --changed selection applied to that diff, so the two can never disagree about what a change reaches (comment-only and function-level PHP differences included);
3. the record's debug environment matches.

  PASS booking_flow                 61/61       8104ms  plugins/bookings/tests/booking_flow_test.php  [reused: 2026-10-14T18:40Z by public-html-41 (pid 3408152)]
...
Reused: 379 of 711 results from 6 earlier runs (20261014T185421Z-3408152, 20261014T192432Z-3718766, ...); 332 run now
Not borrowed: 332: loads public_html/plugins/server_manager/data/managed_nodes_class.php
Run record: 20261014T192632Z-3720986 (held the gate, waited 1m26s)

The Not borrowed line says what blocked reuse, most suites first, in the words the --changed selection uses. The example is a real run: one data class edited by another session between the two runs, and every suite that boots the model layer loads every data class, so a data class change invalidates most of the estate — for reuse exactly as for --changed.

A run that waited for the gate decides reuse again once it holds it, so everything the previous holder completed is on offer. A borrowed result does not touch the coverage map — the map already holds that reach from the run that produced it — and a record holds only what its own run ran, so a borrow always points at the run that produced the result.

A result's function set travels with it: a borrowed result is spared by an edit inside function bodies exactly as --changed would spare the suite. Sets the coverage map does not name are pruned with the records, after 48 hours; a record whose set is gone is file level.

What reuse does not know: it knows what a suite loaded and ran, not what the database held, what a setting was, or what another process wrote while the suite ran. That is the assumption --changed already makes when it skips a suite, and a result that was wrong for one of those reasons stays wrong when borrowed. --fresh looks at no record, and *the pre-publish gate is php tests/run.php db --fresh: a release proves the tree it ships, on the box, now. Reuse is for the working loop and the pre-checkin gate, where the question is "did my change break something".

deploy is not a development tier

safe, db and test-db all run in a checkout, and their tests are entitled to assert things about one: that every first-party plugin is present, that the components manifest matches the repository, that maintenance_scripts has the layout the installer expects. That is what env: any has always meant in practice — safe on any development environment.

A deployed site is not a checkout. It carries the plugins it uses and no others, keeps its own themes, and has no repository around it. Eleven safe suites fail on a production node for reasons that say nothing about the release, so safe must never be pointed at one — upgrade.php did that once and rolled back a perfectly good deploy on getjoinery.

deploy is the tier that may run there. It pulls in nothing and nothing pulls it in. Three rules for anything added to it:

• Assume no repository — not the plugin set, not the theme set, not a git checkout, not a sibling directory.
• Reads only. It runs on production, after a swap, with a rollback on the result.
• An unreachable dependency is a SKIP, not a failure. Reverting a working release because a socket would not open is the worse error.

A live gate never runs unless it is named, so it is entitled to assume that somebody who named it meant to run it. Several therefore need configuration — credentials, a host, a rig — and the rule for all of them is the same: a missing rig is a failure that names what to set, not a skip. A gate that passed because it was never configured is the silent success these gates exist to catch.

needs: is different, and is enforced by the runner rather than the gate: an absent toolchain or an unreachable machine is a reported SKIP with its reason. The distinction is between "this box cannot run it" (skip) and "this box could run it and nobody said against what" (fail).

tests/functional/sync/sync_soak_gate.sh is the current example. It wants JD_SOAK_SERVER, JD_SOAK_ACCOUNT and JD_SOAK_PASSWORD, refuses a server that looks like dev, builds the workspace, links two devices, runs a real storm-and-settle cycle against two real daemons, and checks that the actors actually worked and that a daemon was actually killed before it believes the six assertions that follow. See Drive Sync § The soak environment.

Any single test also runs on its own and prints a human summary, or emits the contract with --json:

php tests/unit/dns_resolver_test.php
php tests/unit/dns_resolver_test.php --json

The dashboard

/tests/ (superadmin, permission 10) lists every discovered test grouped by tier with its env and needs, plus per-tier "Run all" controls, and names the run holding the gate while one is running. Runs execute through the tests_run API action, which spawns the runner with --json; page JS calls /api/v1 with the browser-session credential. On production (debug off) dev-only tests render locked; live and prod-verify tests require a confirm naming the side effect before they run.

"Run all" runs a tier's tests one request at a time, client-side, rendering each result as it lands — so no single request spans a whole tier and results are never lost to a proxy timeout. It runs exactly the tests the section shows (the CLI keeps cumulative safe+db semantics; the dashboard does not). A test that declares a timeout: beyond the web request window renders a disabled CLI button — run it via php tests/run.php.

Writing a test

Require the shared harness, declare the header, build sections of assertions, and finish. The harness bootstraps PathHelper, Globalvars, SessionControl, DbConnector, and LibraryFunctions for you.

<?php
/** @joinery-test
 * name: my_feature
 * tier: db
 * env: dev-only
 * needs: []
 */
require_once(__DIR__ . '/../lib/harness.php');   // path is relative to your test
harness_boot();

section('Creation');
$user = make_user('MyFeature');                  // auto-cleaned at finish
check($user->key > 0, 'user was created');
ok('email is set', $user->get('usr_email') !== ''); // label-first alias

section('Behaviour');
harness_set_setting_mem('some_flag', '1');       // in-memory only, auto-restored
check(do_the_thing() === 'expected', 'thing does the thing');

harness_finish();                                 // MUST be last — prints/emits + exits

The assertion surface:

• check($condition, $label, $detail = '') — condition-first.
• ok($label, $condition, $detail = '') — label-first alias (same recorder).
• section($title) — group the checks that follow.
• harness_skip($label, $detail = '') — a skipped check (e.g. an unmet needs). The runner's line for the test shows it, 125/125 (1 skipped), so a proof that did not run is visible without reading the totals.

• Refused. harness_boot() and tests/run.php mark every test process with JOINERY_TEST_RUN and the suite's name in JOINERY_TEST_SUITE, and install two git hooks in the real repository. reference-transaction refuses a marked process moving a branch, a tag or HEAD (a fetch's remote-tracking refs pass); pre-push refuses its push. Each refusal is logged with the suite's name in .git/joinery-test-refusals.log. Nothing else is affected: an unmarked process, and every throwaway repository, has no such hook to meet.
• Compared. What a hook cannot see is snapshotted when a suite boots and compared in harness_finish(): .git/config and config.worktree, the shared index (it stays empty; commits are prepared in private indexes), and the refusal log. The runner does the same around its whole pass, which covers the shell gates. A change fails the suite (the check the real repository is as this test found it) or the run (an entry named repo_guard), saying what changed and what to put back. A branch that moved while the hooks stood is someone's commit and is only noted (Repository: in the summary); with no hook standing it fails.

• At teardown — NULL-alias mail received since the run booted, addressed to the test store alias or to a harness_fixture_email() address carrying this run's token. It runs last, because delivery is asynchronous and every other teardown step is time the relay gets to hand over what the run sent.
• At boot — the same shapes, matching any fixture address, but only mail older than a 600-second floor. This is the pass that actually empties the box: teardown provably cannot finish the job, because a message the relay hands over after teardown is already too late, and one was measured doing exactly that. What a previous run sent has certainly landed by the time the next run starts.

• get_setting_raw($name) / set_setting_raw($name, $value) — persisted DB settings.
• harness_set_setting_mem($key, $value) — override a setting in the Globalvars in-memory cache only, restored at teardown. Never persisted. A blank value does not override.* Globalvars::get_setting() treats an empty in-memory value as a cache miss and reads the database instead, so a setting cannot be switched off by blanking it — the live value silently wins and the test exercises the wrong branch. Use '0' where the setting is a flag; where the setting is free text whose emptiness means "off" (anti_spam_answer), read the live value and satisfy the check rather than trying to disable it.
• harness_test_mode() — switch to the copied test database and close it at teardown; use with tier test-db.

Talking to the site over HTTP

tests/lib/http.php is the one HTTP client for tests — require it instead of the harness (it pulls the harness in) whenever a test makes real requests. There is no per-suite curl to copy, and nothing to configure: the base URL comes from the webDir setting, and requests to that site are automatically pinned to the machine's own origin IP so they bypass Cloudflare and keep a stable REMOTE_ADDR. Every request carries X-Joinery-Test: 1, so an error it causes is recorded but never reported to the upgrade source automatically (see Error Handling § Automatic reports).

require_once(__DIR__ . '/../lib/http.php');
harness_boot();

$r = harness_request('POST', '/api/v1/action/foo', array(
    'headers' => harness_csrf_header($token),   // or key_headers(...) for API keys
    'body'    => array('a' => 1),               // arrays default to a JSON body
));
check($r['status'] === 200, 'foo ran', $r['raw']);

harness_request($method, $url, $opts) returns status, body/raw, json, headers (array), header_string, content_type, redirect_url, and error. $url is a path against the base URL, or a full absolute URL (signed links). The options cover the whole surface: encode (json|form|raw|multipart), files for multipart uploads, jar for a cookie session, cookies for extra cookies, follow for redirects, accept (defaults to application/json, pass null to omit), timeout, insecure, and pin to force origin pinning on/off.

For a browser (cookie) session rather than an API key:

• harness_jar_new($prefix) — a cookie jar file, deleted at teardown.
• harness_web_login($jar, $email, $password) — form-login and return the CSRF token from the resulting page, or null if login failed.
• harness_jar_csrf($jar) — the joinery_api_csrf cookie (present for anyone, anonymous included); harness_meta_csrf($html) reads the joinery-api-csrf meta tag instead (rendered only into a logged-in page). The two are not interchangeable — pick the one the endpoint expects.
• harness_csrf_header($token) — the X-Joinery-Csrf header a browser-session call must send.
• harness_put_chunk($path, $key_headers, $content_range, $body) — a raw-body chunk PUT for the upload transport.

Calling a logic function directly

Admin and page logic is normally reached through process_logic(), which exits on the post-save redirect — fatal to a test. tests/lib/logic.php calls the logic function itself instead and hands back the LogicResult:

require_once(__DIR__ . '/../lib/logic.php');

$result = harness_call_logic('logic/foo_logic.php', 'foo_logic',
    array('action' => 'add', 'name' => 'X'), 'POST');
check(!$result->error, 'save succeeded', (string)$result->error);

harness_call_logic($path, $fn, $data, $method) simulates the request method (REQUEST_METHOD, $_POST/$_GET/$_REQUEST) and restores the superglobals afterwards — $method matters, since form saves are gated on a POST while GET actions (e.g. new_version) must not be sent as a form save. It does not throw on failure, so a test can assert on the result; harness_call_logic_ok() is the variant that throws, for fixture-setup steps where a failure should stop the run.

The call runs as a web request for the model check: every save it makes is asked against the session and refused below the model's level, as on the site (Permissions). Outside a logic call, any write a suite makes while a simulated person is signed in is checked the same way (harness_boot() sets Access::$check_cli_persons); fixtures made with no one signed in are not. A fixture a suite builds as one of its users goes inside SystemBase::server_initiated_write() when no person's rule should apply to it.

What a test file asks of a model directly ($user->save(), make_user(), User::CreateNew(), Conversation::add_message()) is a fixture and is not checked, whoever is signed in. So a suite proves a flow passes the check only when it drives the flow through its logic (harness_call_logic()) or through a service that is not a model; and a visitor's flow is checked only through harness_call_logic(), since with no one signed in a CLI write is exempt.

The result contract

A declared .php test must emit this contract. If it doesn't — it crashed, or was never converted to the harness — the runner marks it failed regardless of its exit code (a script that prints "Failed" but exits 0 must never read as green). Only .sh gates are exit-code-only; that is their contract.

Every test emits the same JSON contract on a --json run (prefixed on the CLI by the sentinel @@JOINERY_TEST_RESULT@@ so the runner can locate it even after error output). The runner and the dashboard consume only this:

{
  "name": "cloud_offload_engine",
  "tier": "safe",
  "stats": { "total": 12, "passed": 12, "failed": 0, "skipped": 0 },
  "sections": [
    { "title": "Creation", "checks": [ { "label": "…", "passed": true, "detail": "" } ] }
  ],
  "duration_ms": 8
}

The test database

test-db tests run against a separate database so CRUD is isolated from live data. Rebuild it from /admin/admin_test_database.

It carries the schema, not the content. A rebuild takes the full structure of the live database — every table, column, index and foreign key — and seeds only the handful of tables the site cannot boot without (TestDatabaseHelper::referenceTables(): settings, the plugin registry, admin menus, the deletion rules permanent_delete() reads, and small reference data like zone names and email templates). The seeded tables are as live had them at the rebuild, so a suite that pins a new deletion rule or setting rebuilds the copy after update_database has written it. Every other table arrives empty with its constraints intact. Against a 1.3 GB live database the copy is about 16 MB, nearly all of which is the floor cost of ~200 empty tables and their indexes.

Nothing should be written that expects to find real rows there. The model suites create their own — see How the model suite satisfies foreign keys.

The runner keeps the copy current. A copy goes stale the moment update_database adds a table or column, and a stale copy fails every test-db suite with "column does not exist" noise. So when a run is about to execute test-db suites on a dev environment and the schema comparison says the copy is behind, the runner rebuilds it (structure copy, seconds, atomic rename) before the lane starts and says so in the output — and in the aggregate JSON as test_db_refresh. A failed rebuild is reported and the suites then fail with the real drift errors.

A full copy, content included, is a second button on the same page. It exists to reproduce a problem against real data; no test needs it, and it duplicates mail, sealed content and member records onto the same disk.

Rebuilding is a development action: the page refuses it when the debug setting is off, and the menu entry is hidden there.

Two suites drive the model estate against the copy; if either reports schema errors, rebuild first:

• tests/models/models_test.php — every data model's generic CRUD, validation and constraint behaviour, one check per model class.
• tests/models/multi_models_test.php — every collection class, one check per model that has one. For each it verifies that the loaded collection matches the equivalent direct SQL, that each declared filter narrows the result set, and that ordering and pagination agree with ORDER BY / LIMIT / OFFSET.

The copy does not receive update_database — that runs against the live database only. After any schema change, rebuild from /admin/admin_test_database so the copy matches; a test database that lags live validates models against a schema production does not have. Foreign key constraints in particular are materialized on the live database by update_database, so a stale copy silently loses that coverage. Rebuilding is cheap, so there is no reason to put it off.

dbname_test must name a database no site uses. The rebuild drops its target, and a rebuild refuses outright when that name turns out to be some other site's live database — which is possible, because the installer can provision a separate site named {site}_test alongside the main one.

Both test-db suites declare needs: [test-db]. The runner probes the connection rather than trusting dbname_test, so a checkout with no copy provisioned skips them with a named reason instead of failing the gate.

How the model suite satisfies foreign keys

ModelTester resolves every foreign-key-ish field to a real row. The declared 'foreign_key' field spec (target table + column — the same declaration DatabaseUpdater materializes as a constraint) is authoritative; without one, the naming convention applies: an FK column is the child prefix followed by the parent model's primary key (abv_abx_ab_test_id → AbTest). For each resolved reference the tester creates a fresh parent row through the target model (recursively, so the parent's own references are satisfied too) and removes it before the model's test finishes. Fresh parents keep results independent of which ids exist in the database and keep child-delete cascades away from real rows. Fields that resolve to no model (polymorphic ids like entity_id) get plain integers, which is only valid because they reference no single table.

Per-model test fixtures

A model whose validity rules a generic generator cannot infer — cross-field business rules, enum values enforced in code — declares them:

public static $test_fixture = array(
    'values'       => array('mem_scope' => self::SCOPE_SHARED), // pinned field values
    'update_field' => 'mem_content', // safe field for the update probe
);

values override generated data in every dataset the tester builds (and pinned fields are exempted from null-value probing — the pin exists because other values are invalid). update_field names the field the CRUD update step modifies, for models whose first spec-order field is enum-validated. generate lists optional fields the tester fills anyway, for a rule that spans fields: User lists usr_email, since an account needs an email or a username and neither is required alone. Declare a fixture only when the field specs genuinely cannot express the rule.

Plugin tests

A plugin's tests live in plugins/{plugin}/tests/ and carry the same header. They are discovered and listed alongside core tests (the dashboard includes them), and are reachable directly at /plugins/{plugin}/tests/{page} (superadmin). Follow the same harness conventions; the relative path to the harness from a plugin test is __DIR__ . '/../../../tests/lib/harness.php'.