Add search self-tests (hash consistency, QSearch sanity) and a native EPD parser (`epd` command + ParseEpdRecord), fixing a silent-corruption bug in convert_epd.pl that broke every Black-to-move EPD conversion.

testsearch.c: TestSearchHashConsistency exercises the hash table through real search (repeat-with-warm-table and shallow-then-deep agreement) rather than synthetic store/probe calls. TestQSearchSanity checks termination and a node-count ceiling; its own header documents that the TEST=1 self-test build doesn't define PERF_COUNTERS, so it checks whole-search node count rather than the qsearch-only counter. A planned null-move zugzwang test was dropped in favor of tests/zugzwang.epd (best-move outcome, not an internal counter) after the three classic literature positions turned out not to structurally reach this engine's verification-search code path on this build.

fen.c/script.c/command.c: new `epd <file>` command reads raw EPD directly via ParseEpdRecord, reusing ScriptCommand's existing suite-reporting machinery (factored into _ProcessSuiteLine/_PrintSuiteReport). Replaces the need for tests/convert_epd.pl, which had a real bug: its bm/am regex's [^b]+ filler breaks on any Black-to-move record (side-to-move field is literally "b"), and unscoped $fen/$id/$solution globals meant a failed match silently duplicated the previous record instead of erroring.

tests/ep_to_epd.pl converts the existing setboard/solution/avoid/id/go suites back to real EPD; cross-checked ecm_ringers.ep_ via `script` against ecm_ringers.epd via `epd` -- byte-identical node counts and solve stats. Added .epd siblings for every genuine tactical suite in tests/.

Added src/TESTING_PLAN.md: full audit of test*.c coverage gaps (dead SEE oracle, unwired eval/recognizer tests, zero hash-table correctness coverage, thin search self-tests) with a prioritized plan for closing them.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

1 files changed, 253 insertions, 0 deletions

diff --git a/src/TESTING_PLAN.md b/src/TESTING_PLAN.md
new file mode 100644
index 0000000..abf35e3
--- /dev/null
+++ b/src/TESTING_PLAN.md
@@ -0,0 +1,253 @@
+# Testing plan: closing coverage gaps in test*.c
+
+Written 2026-09-16 after an audit of every `src/test*.c` file and how
+(or whether) each is wired into `main.c`'s self-test sequence. Goal
+per Scott: not more test code for its own sake -- find the
+years-old bugs that asserts have been missing. Priorities: search,
+eval, generate (not exclusively). Perf testing is also explicitly
+in scope.
+
+## Dead/disconnected infrastructure (fix first -- near-zero cost)
+
+Several tests already exist but don't run. Recovering this coverage is
+strictly cheaper than writing anything new, so do this before anything
+else below.
+
+1. **SEE has zero active correctness testing.** `testsee.c`'s own
+   header comment admits it: `DebugSEE`/`DebugSEERecursive` (a
+   brute-force make/unmake exchange-walk oracle meant to check `SEE()`
+   against) is wrapped in `#ifdef TEST_BROKEN` -- a define nothing
+   sets. `testsee.c`'s only *called* functions (`TestGetAttacks`,
+   `TestIsAttackedBB`) test attack generation, not SEE. Given SEE is
+   what `GetMoveSee`/extensions/QSearch pruning all trust (and is the
+   exact code reworked in commit c0ac3d7), this is the single
+   highest-value fix here. Get `TEST_BROKEN` -> live (`TEST`), fix
+   whatever bitrot surfaces, and confirm it's called from `main.c`.
+2. **`TestEvalWithKnownPositions`** (`testeval.c`, ~20 hand-curated
+   "side X should be better here" positions from chess literature) is
+   defined but **never called from anywhere** -- not `main.c`, not
+   `command.c`. Wire it into the self-test sequence.
+3. **`TestRecognExhaustiveKNKP`** (`testrecogn.c`, exhaustive KNKP
+   endgame-recognizer enumeration) is defined and prototyped in
+   `chess.h` but referenced only in a comment -- never invoked. Only
+   the smaller table-driven `TestRecogn` runs. Wire it in.
+4. **The hash table has no correctness self-test at all.**
+   `testhash.c` contains `SanityCheckHashEntry` (an entry-shape
+   validator) and `AnalyzeFullHashTable` -- but the latter is a
+   post-search *diagnostic dump* called from `root.c`, not a test, and
+   it's never invoked from the `TEST` self-test sequence. See "New
+   test: hash correctness" below for what should replace this gap, not
+   just a wiring fix.
+5. **`TestEvalWithSymmetry`** only runs a *single* random symmetric
+   position per invocation, and only when a human types `eval` at the
+   prompt in an `EVAL_DUMP` build (`command.c:2203`) -- not part of the
+   automated suite, never loops. See "New test: eval symmetry fuzz"
+   below.
+
+## New tests, by area
+
+### Search (thinnest area relative to its bug surface)
+
+- `TestSearch` (`testsearch.c`) runs 20 random positions at fixed
+  **depth 2** only, checking solely "returned move is legal" and
+  "didn't blow the time limit."
+  - **DONE -- hash-consistency test** (`TestSearchHashConsistency`,
+    testsearch.c): searches a position to depth D twice in a row with
+    a warm hash table (scores must match), and separately confirms a
+    depth-D2 search seeded by a shallower depth-D1 warm-up agrees with
+    a from-scratch depth-D2 search. Wired into the `TEST=1` self-test
+    sequence (main.c).
+  - **DONE (partially) -- QSearch sanity test**
+    (`TestQSearchSanity`, testsearch.c): checks search termination
+    (no `INVALID_SCORE`/garbage) and a whole-search node-count
+    ceiling. Caveat found and documented in the function's own header
+    comment: the self-test suite builds with plain `gmake TEST=1`,
+    which doesn't define `PERF_COUNTERS` -- every `sCounters` field
+    (including `u64QNodeCount`, the qsearch-only counter this was
+    originally meant to check) is wrapped in `chess.h`'s `INC()` macro,
+    a no-op without that define. Confirmed live (`sd 3; go` on a
+    `TEST`-only binary reports "0 qnodes" regardless of real qsearch
+    activity). The test currently checks
+    `ctx->sCounters.tree.u64TotalNodeCount` instead (root.c's
+    unconditional per-node `++`, always live) -- a real but strictly
+    weaker signal, since it can't distinguish "qsearch exploded" from
+    "full-width search exploded." Building a `PERF_COUNTERS`-enabled
+    `TEST` profile (or adding an always-live qsearch-only counter) is
+    the natural follow-up to close this for real. The hand-auditable-
+    tree idea (`Trace()` every qsearch node on a small position) is
+    still worth doing but wasn't implemented this session.
+  - **NOT DONE -- determinism test** (same position/depth searched
+    twice, PV must match): explicitly descoped -- Scott's call is that
+    a stable-PV requirement makes the test break every time move
+    ordering changes for unrelated reasons, which is just noise, not a
+    real regression signal. A same-*score* (not same-PV) check is
+    already covered by the hash-consistency test above.
+  - **NOT DONE -- MP-vs-single-thread agreement test**: still open.
+    `split.c` has no dedicated correctness test.
+  - **DONE (as an EPD suite, not a C test) -- null-move/zugzwang
+    soundness**: `tests/zugzwang.epd` (5 positions, run via the new
+    `epd` command -- see below) checks actual best-move outcome
+    instead of an internal counter. An earlier attempt asserted
+    against `ctx->sCounters.extension.uZugzwang` (the verification-
+    search hit counter) using the 3 classic literature zugzwang
+    positions, but none of them structurally reach that code path on
+    this build: 2 have too little material for
+    `WeShouldTryNullmovePruning`'s own `> 2` non-pawn-count gate to
+    attempt null-move pruning at all, and the 3rd (despite landing in
+    the eligible window) gets resolved directly off this box's local
+    Syzygy tablebase before search ever touches null-move logic --
+    confirmed live via the `TABLEBASE MOVE` trace line. Removed in
+    favor of the EPD suite, which doesn't care about internal
+    mechanism, only the final answer. See `testsearch.c`'s comment
+    where `TestNullMoveZugzwang` used to be for the full writeup.
+
+**New infrastructure that came out of chasing the above (search.c
+touches none of this, but it's now how any future EPD-based search
+test should be added):**
+
+- **`epd <file>` engine command** (`EpdCommand`, script.c) parses and
+  runs a raw EPD suite file directly -- no perl preprocessing step
+  first. Backed by `ParseEpdRecord` (fen.c), real per-record EPD
+  parsing (4 position fields, optional FEN-style halfmove/fullmove
+  counters, then `;`-separated `bm`/`am`/`id` opcodes).
+- **Found and fixed while building this**: `tests/convert_epd.pl` (the
+  old perl-based EPD-to-script converter) had a real, silent-
+  corruption bug -- its `bm`/`am` regex used `[^b]+` as the filler
+  between the board and the opcode, which breaks on *every*
+  Black-to-move record (the side-to-move field is literally the
+  character `b`), and since `$fen`/`$id`/`$solution` were unscoped
+  globals rather than per-line locals, a failed match silently carried
+  the *previous* record's values forward instead of erroring.
+  Confirmed live: feeding it a 5-position batch with one Black-to-move
+  entry produced a byte-for-byte duplicate of the previous record for
+  that one line, no diagnostic at all. The script is left in place
+  (marked superseded in its own header) in case something still shells
+  out to it, but new suites should use `epd` instead.
+  Cross-checked the new parser against the old pipeline on
+  `ecm_ringers`: `script` (old `.ep_` format) and `epd` (new
+  `tests/ecm_ringers.epd`, produced by `ep_to_epd.pl`, see next item)
+  produced byte-identical node counts and solve stats.
+- **`tests/ep_to_epd.pl`**: one-off reverse converter, turns the
+  existing `setboard`/`solution`/`avoid`/`id`/`go` script-format
+  suites back into real EPD now that `epd` can read it directly. Run
+  against every genuine tactical suite in `tests/` (not `bench.ep_` or
+  `trial.ep_`, which use `sd`/`st`/`easy`/`quit` and aren't EPD
+  underneath) -- produced `.epd` siblings for `arasan`,
+  `ecm_confident_quick`, `ecm_confident_sample2`, `ecm_confident`,
+  `ecm_hard_quick`, `ecm_hard`, `ecm_late`, `ecm_quick`, `ecm_ringers`,
+  `ecm-gcp`, `ecm-hard`, `ecm`, `ecm016_only`, `LCT2`, `mes400`,
+  `newiq`, `quick`, `swiss`, `wac`. `ecm.epd` came out to 879 records,
+  matching the `606/879`/`622/879` solve-count baselines already
+  quoted elsewhere in CLAUDE.md.
+
+### Eval
+
+- Automate `TestEvalWithSymmetry` into a real fuzz loop (N random
+  symmetric positions, assert `Eval() == 0` for all, as part of the
+  standard self-test run) instead of the current single-shot
+  manual-only invocation.
+- **Adversarial/degenerate-position term-bounds test**: 0-pawn
+  positions, 9-queen positions, fully-open-board positions -- extend
+  `GenerateRandomLegalPosition` or hand-build cases, assert no eval
+  term produces an out-of-range score (reuse the existing
+  `EvalTrace`/`EVAL_DUMP` term-tracing machinery for this).
+- Audit for any **incrementally-maintained eval state** (pawn hash,
+  king-safety cache) and add a from-scratch-recompute-vs-incremental
+  consistency check if any exists -- classic "works until the hash
+  collides" bug class.
+- **Eval speed regression test**: no benchmark currently exists for
+  eval itself despite it being the named NPS bottleneck vs. Crafty
+  (see CLAUDE.md). Same interleaved-comparison shape as the existing
+  `TestGenerate*Speed` benchmarks, broken down by game phase.
+
+### Generate (best-covered area, but gaps remain)
+
+- Perft (`TestMoveGenerator`) only checks to **ply 4** even though
+  `uLeaves[]` already has reference values to ply 6/7 sitting unused --
+  extend to a `--perft-deep` slow path, and add perft-**divide**
+  (per-root-move leaf breakdown) to localize failures to a specific
+  move instead of just tripping a panic.
+- Add perft positions specifically chosen to stress **en passant,
+  underpromotion, and castling-through-check** individually (the
+  current 5 FENs give general coverage but weren't chosen for this).
+- `TestGenerateAllMovesSpeed` and the per-piece-type speed tests are
+  **pure microbenchmarks with no correctness assertion** -- they print
+  cycles and never diff the move *sets* produced by the mailbox vs.
+  bitboard generators being compared. Add a sorted-move-list-equality
+  assertion alongside each (both generators already run back-to-back
+  in the same loop, so this is nearly free) -- otherwise a bitboard
+  generator silently missing a move class just prints a suspiciously
+  fast number and passes.
+- `GetMoveSee`'s new non-pawn-capture SEE path (c0ac3d7) has no
+  dedicated test yet -- covered by the SEE oracle fix above once live.
+
+### Move/Position plumbing
+
+- Reasonably solid already (`TestMakeUnmakeMove`, `TestExposesCheck`,
+  `TestIsAttacked`). Add a **long-game fuzz test**: play N random games
+  of ~200 plies each (not just 1-ply-at-a-time exercises), asserting
+  `VerifyPositionConsistency` and `ComputeSig`-from-scratch agreement
+  with the incrementally-maintained signature at every ply. Confirm
+  `VerifyPositionConsistency` is actually compiled live in the
+  `TEST`/`DEBUG` build, not just present in source.
+
+### Hash (currently zero correctness coverage)
+
+New test needed, not just a wiring fix:
+- Store/probe roundtrip correctness.
+- Replacement-scheme behavior under forced collisions (fill table past
+  capacity with distinct signatures, confirm replacement policy
+  behaves as designed, not just "doesn't crash").
+- `ADJUST_MATE_IN_N` ply-distance adjustment on `HASH_FLAG_EXACT`
+  entries specifically -- the exact mechanism CLAUDE.md flags as
+  audited-by-hand but not test-covered after this session's `ASSERT`
+  removals in `QSearch()`.
+
+### Recognizer
+
+- Wire in `TestRecognExhaustiveKNKP` (see dead-infra list above).
+  Consider the same exhaustive-enumeration treatment for any other
+  small recognized endgame class in `recogn.c` (KPK, KRK) if one
+  exists without a test.
+
+## Perf testing
+
+Current coverage is narrow: five isolated per-piece-type
+mailbox-vs-bitboard microbenchmarks (`TestGenerate*Speed`), nothing for
+search, eval, or hash.
+
+- **Eval NPS/cycles-per-call benchmark**, same interleaved-comparison
+  shape as the existing generate benchmarks, by game phase -- eval is
+  the named bottleneck (CLAUDE.md's 1.5M vs 7.5M nps gap vs. Crafty)
+  and has no dedicated speed test today.
+- **Whole-search NPS benchmark**: `bench.c` already exists for this --
+  confirm whether it's part of the automated `TEST` self-test pass or
+  purely a manual `bench` command; if manual-only, wire a
+  fixed-node-budget version into the automated sequence so a
+  regression surfaces without someone remembering to run it by hand.
+- **Hash-probe-under-load benchmark**: cycles/probe at low occupancy
+  vs. near-full table -- replacement-scheme cost can regress silently
+  today with nothing to measure it.
+- **Stored baselines with pass/fail thresholds**, not print-only
+  output: today every perf test prints cycles/call and leaves
+  interpretation to a human reading the log. Store a last-known-good
+  baseline (same idea as `head_reference/` for solve-counts) and fail
+  loud if a call site regresses past some tolerance.
+
+## Suggested priority order
+
+1. Un-bitrot the three dead tests (SEE oracle,
+   `TestEvalWithKnownPositions`, `TestRecognExhaustiveKNKP`) --
+   near-zero cost, immediate coverage recovery.
+2. Hash-table correctness test (store/probe/replace/mate-adjust) --
+   currently the single biggest correctness blind spot.
+3. Search determinism + MP-vs-single-thread agreement test -- targets
+   the exact class of bug most likely to hide in threading/hash/
+   extension interactions that per-function unit tests can't see.
+4. Extend perft to deeper plies + divide output; add move-set-equality
+   assertions to the existing generate speed benchmarks (turns
+   dead-weight microbenchmarks into correctness tests for free).
+5. Eval: automate the symmetry fuzz test (loop N, not 1) and add
+   adversarial/degenerate-position term-bounds checks.
+6. Round out perf testing per the list above, with stored baselines
+   and pass/fail thresholds instead of print-only output.