summaryrefslogtreecommitdiff
path: root/src
diff options
context:
space:
mode:
authorScott Gasch <[email protected]>2026-09-16 00:08:27 -0700
committerScott Gasch <[email protected]>2026-09-16 00:08:27 -0700
commitb91c8453f3fe21c8ccd3568923d8e39141c9c945 (patch)
treeb908fea282ecd4075744590d3dfc88ffa903d943 /src
parentc0ac3d7845f60c7f6f2c3bfc306ceb0e4ac045ba (diff)
Add search self-tests (hash consistency, QSearch sanity) and a native EPD parser (`epd` command + ParseEpdRecord), fixing a silent-corruption bug in convert_epd.pl that broke every Black-to-move EPD conversion.
testsearch.c: TestSearchHashConsistency exercises the hash table through real search (repeat-with-warm-table and shallow-then-deep agreement) rather than synthetic store/probe calls. TestQSearchSanity checks termination and a node-count ceiling; its own header documents that the TEST=1 self-test build doesn't define PERF_COUNTERS, so it checks whole-search node count rather than the qsearch-only counter. A planned null-move zugzwang test was dropped in favor of tests/zugzwang.epd (best-move outcome, not an internal counter) after the three classic literature positions turned out not to structurally reach this engine's verification-search code path on this build. fen.c/script.c/command.c: new `epd <file>` command reads raw EPD directly via ParseEpdRecord, reusing ScriptCommand's existing suite-reporting machinery (factored into _ProcessSuiteLine/_PrintSuiteReport). Replaces the need for tests/convert_epd.pl, which had a real bug: its bm/am regex's [^b]+ filler breaks on any Black-to-move record (side-to-move field is literally "b"), and unscoped $fen/$id/$solution globals meant a failed match silently duplicated the previous record instead of erroring. tests/ep_to_epd.pl converts the existing setboard/solution/avoid/id/go suites back to real EPD; cross-checked ecm_ringers.ep_ via `script` against ecm_ringers.epd via `epd` -- byte-identical node counts and solve stats. Added .epd siblings for every genuine tactical suite in tests/. Added src/TESTING_PLAN.md: full audit of test*.c coverage gaps (dead SEE oracle, unwired eval/recognizer tests, zero hash-table correctness coverage, thin search self-tests) with a prioritized plan for closing them. Co-Authored-By: Claude Sonnet 5 <[email protected]>
Diffstat (limited to 'src')
-rw-r--r--src/TESTING_PLAN.md253
-rwxr-xr-xsrc/chess.h16
-rw-r--r--src/command.c6
-rw-r--r--src/fen.c216
-rw-r--r--src/main.c2
-rw-r--r--src/script.c378
-rw-r--r--src/testsearch.c298
7 files changed, 1053 insertions, 116 deletions
diff --git a/src/TESTING_PLAN.md b/src/TESTING_PLAN.md
new file mode 100644
index 0000000..abf35e3
--- /dev/null
+++ b/src/TESTING_PLAN.md
@@ -0,0 +1,253 @@
+# Testing plan: closing coverage gaps in test*.c
+
+Written 2026-09-16 after an audit of every `src/test*.c` file and how
+(or whether) each is wired into `main.c`'s self-test sequence. Goal
+per Scott: not more test code for its own sake -- find the
+years-old bugs that asserts have been missing. Priorities: search,
+eval, generate (not exclusively). Perf testing is also explicitly
+in scope.
+
+## Dead/disconnected infrastructure (fix first -- near-zero cost)
+
+Several tests already exist but don't run. Recovering this coverage is
+strictly cheaper than writing anything new, so do this before anything
+else below.
+
+1. **SEE has zero active correctness testing.** `testsee.c`'s own
+ header comment admits it: `DebugSEE`/`DebugSEERecursive` (a
+ brute-force make/unmake exchange-walk oracle meant to check `SEE()`
+ against) is wrapped in `#ifdef TEST_BROKEN` -- a define nothing
+ sets. `testsee.c`'s only *called* functions (`TestGetAttacks`,
+ `TestIsAttackedBB`) test attack generation, not SEE. Given SEE is
+ what `GetMoveSee`/extensions/QSearch pruning all trust (and is the
+ exact code reworked in commit c0ac3d7), this is the single
+ highest-value fix here. Get `TEST_BROKEN` -> live (`TEST`), fix
+ whatever bitrot surfaces, and confirm it's called from `main.c`.
+2. **`TestEvalWithKnownPositions`** (`testeval.c`, ~20 hand-curated
+ "side X should be better here" positions from chess literature) is
+ defined but **never called from anywhere** -- not `main.c`, not
+ `command.c`. Wire it into the self-test sequence.
+3. **`TestRecognExhaustiveKNKP`** (`testrecogn.c`, exhaustive KNKP
+ endgame-recognizer enumeration) is defined and prototyped in
+ `chess.h` but referenced only in a comment -- never invoked. Only
+ the smaller table-driven `TestRecogn` runs. Wire it in.
+4. **The hash table has no correctness self-test at all.**
+ `testhash.c` contains `SanityCheckHashEntry` (an entry-shape
+ validator) and `AnalyzeFullHashTable` -- but the latter is a
+ post-search *diagnostic dump* called from `root.c`, not a test, and
+ it's never invoked from the `TEST` self-test sequence. See "New
+ test: hash correctness" below for what should replace this gap, not
+ just a wiring fix.
+5. **`TestEvalWithSymmetry`** only runs a *single* random symmetric
+ position per invocation, and only when a human types `eval` at the
+ prompt in an `EVAL_DUMP` build (`command.c:2203`) -- not part of the
+ automated suite, never loops. See "New test: eval symmetry fuzz"
+ below.
+
+## New tests, by area
+
+### Search (thinnest area relative to its bug surface)
+
+- `TestSearch` (`testsearch.c`) runs 20 random positions at fixed
+ **depth 2** only, checking solely "returned move is legal" and
+ "didn't blow the time limit."
+ - **DONE -- hash-consistency test** (`TestSearchHashConsistency`,
+ testsearch.c): searches a position to depth D twice in a row with
+ a warm hash table (scores must match), and separately confirms a
+ depth-D2 search seeded by a shallower depth-D1 warm-up agrees with
+ a from-scratch depth-D2 search. Wired into the `TEST=1` self-test
+ sequence (main.c).
+ - **DONE (partially) -- QSearch sanity test**
+ (`TestQSearchSanity`, testsearch.c): checks search termination
+ (no `INVALID_SCORE`/garbage) and a whole-search node-count
+ ceiling. Caveat found and documented in the function's own header
+ comment: the self-test suite builds with plain `gmake TEST=1`,
+ which doesn't define `PERF_COUNTERS` -- every `sCounters` field
+ (including `u64QNodeCount`, the qsearch-only counter this was
+ originally meant to check) is wrapped in `chess.h`'s `INC()` macro,
+ a no-op without that define. Confirmed live (`sd 3; go` on a
+ `TEST`-only binary reports "0 qnodes" regardless of real qsearch
+ activity). The test currently checks
+ `ctx->sCounters.tree.u64TotalNodeCount` instead (root.c's
+ unconditional per-node `++`, always live) -- a real but strictly
+ weaker signal, since it can't distinguish "qsearch exploded" from
+ "full-width search exploded." Building a `PERF_COUNTERS`-enabled
+ `TEST` profile (or adding an always-live qsearch-only counter) is
+ the natural follow-up to close this for real. The hand-auditable-
+ tree idea (`Trace()` every qsearch node on a small position) is
+ still worth doing but wasn't implemented this session.
+ - **NOT DONE -- determinism test** (same position/depth searched
+ twice, PV must match): explicitly descoped -- Scott's call is that
+ a stable-PV requirement makes the test break every time move
+ ordering changes for unrelated reasons, which is just noise, not a
+ real regression signal. A same-*score* (not same-PV) check is
+ already covered by the hash-consistency test above.
+ - **NOT DONE -- MP-vs-single-thread agreement test**: still open.
+ `split.c` has no dedicated correctness test.
+ - **DONE (as an EPD suite, not a C test) -- null-move/zugzwang
+ soundness**: `tests/zugzwang.epd` (5 positions, run via the new
+ `epd` command -- see below) checks actual best-move outcome
+ instead of an internal counter. An earlier attempt asserted
+ against `ctx->sCounters.extension.uZugzwang` (the verification-
+ search hit counter) using the 3 classic literature zugzwang
+ positions, but none of them structurally reach that code path on
+ this build: 2 have too little material for
+ `WeShouldTryNullmovePruning`'s own `> 2` non-pawn-count gate to
+ attempt null-move pruning at all, and the 3rd (despite landing in
+ the eligible window) gets resolved directly off this box's local
+ Syzygy tablebase before search ever touches null-move logic --
+ confirmed live via the `TABLEBASE MOVE` trace line. Removed in
+ favor of the EPD suite, which doesn't care about internal
+ mechanism, only the final answer. See `testsearch.c`'s comment
+ where `TestNullMoveZugzwang` used to be for the full writeup.
+
+**New infrastructure that came out of chasing the above (search.c
+touches none of this, but it's now how any future EPD-based search
+test should be added):**
+
+- **`epd <file>` engine command** (`EpdCommand`, script.c) parses and
+ runs a raw EPD suite file directly -- no perl preprocessing step
+ first. Backed by `ParseEpdRecord` (fen.c), real per-record EPD
+ parsing (4 position fields, optional FEN-style halfmove/fullmove
+ counters, then `;`-separated `bm`/`am`/`id` opcodes).
+- **Found and fixed while building this**: `tests/convert_epd.pl` (the
+ old perl-based EPD-to-script converter) had a real, silent-
+ corruption bug -- its `bm`/`am` regex used `[^b]+` as the filler
+ between the board and the opcode, which breaks on *every*
+ Black-to-move record (the side-to-move field is literally the
+ character `b`), and since `$fen`/`$id`/`$solution` were unscoped
+ globals rather than per-line locals, a failed match silently carried
+ the *previous* record's values forward instead of erroring.
+ Confirmed live: feeding it a 5-position batch with one Black-to-move
+ entry produced a byte-for-byte duplicate of the previous record for
+ that one line, no diagnostic at all. The script is left in place
+ (marked superseded in its own header) in case something still shells
+ out to it, but new suites should use `epd` instead.
+ Cross-checked the new parser against the old pipeline on
+ `ecm_ringers`: `script` (old `.ep_` format) and `epd` (new
+ `tests/ecm_ringers.epd`, produced by `ep_to_epd.pl`, see next item)
+ produced byte-identical node counts and solve stats.
+- **`tests/ep_to_epd.pl`**: one-off reverse converter, turns the
+ existing `setboard`/`solution`/`avoid`/`id`/`go` script-format
+ suites back into real EPD now that `epd` can read it directly. Run
+ against every genuine tactical suite in `tests/` (not `bench.ep_` or
+ `trial.ep_`, which use `sd`/`st`/`easy`/`quit` and aren't EPD
+ underneath) -- produced `.epd` siblings for `arasan`,
+ `ecm_confident_quick`, `ecm_confident_sample2`, `ecm_confident`,
+ `ecm_hard_quick`, `ecm_hard`, `ecm_late`, `ecm_quick`, `ecm_ringers`,
+ `ecm-gcp`, `ecm-hard`, `ecm`, `ecm016_only`, `LCT2`, `mes400`,
+ `newiq`, `quick`, `swiss`, `wac`. `ecm.epd` came out to 879 records,
+ matching the `606/879`/`622/879` solve-count baselines already
+ quoted elsewhere in CLAUDE.md.
+
+### Eval
+
+- Automate `TestEvalWithSymmetry` into a real fuzz loop (N random
+ symmetric positions, assert `Eval() == 0` for all, as part of the
+ standard self-test run) instead of the current single-shot
+ manual-only invocation.
+- **Adversarial/degenerate-position term-bounds test**: 0-pawn
+ positions, 9-queen positions, fully-open-board positions -- extend
+ `GenerateRandomLegalPosition` or hand-build cases, assert no eval
+ term produces an out-of-range score (reuse the existing
+ `EvalTrace`/`EVAL_DUMP` term-tracing machinery for this).
+- Audit for any **incrementally-maintained eval state** (pawn hash,
+ king-safety cache) and add a from-scratch-recompute-vs-incremental
+ consistency check if any exists -- classic "works until the hash
+ collides" bug class.
+- **Eval speed regression test**: no benchmark currently exists for
+ eval itself despite it being the named NPS bottleneck vs. Crafty
+ (see CLAUDE.md). Same interleaved-comparison shape as the existing
+ `TestGenerate*Speed` benchmarks, broken down by game phase.
+
+### Generate (best-covered area, but gaps remain)
+
+- Perft (`TestMoveGenerator`) only checks to **ply 4** even though
+ `uLeaves[]` already has reference values to ply 6/7 sitting unused --
+ extend to a `--perft-deep` slow path, and add perft-**divide**
+ (per-root-move leaf breakdown) to localize failures to a specific
+ move instead of just tripping a panic.
+- Add perft positions specifically chosen to stress **en passant,
+ underpromotion, and castling-through-check** individually (the
+ current 5 FENs give general coverage but weren't chosen for this).
+- `TestGenerateAllMovesSpeed` and the per-piece-type speed tests are
+ **pure microbenchmarks with no correctness assertion** -- they print
+ cycles and never diff the move *sets* produced by the mailbox vs.
+ bitboard generators being compared. Add a sorted-move-list-equality
+ assertion alongside each (both generators already run back-to-back
+ in the same loop, so this is nearly free) -- otherwise a bitboard
+ generator silently missing a move class just prints a suspiciously
+ fast number and passes.
+- `GetMoveSee`'s new non-pawn-capture SEE path (c0ac3d7) has no
+ dedicated test yet -- covered by the SEE oracle fix above once live.
+
+### Move/Position plumbing
+
+- Reasonably solid already (`TestMakeUnmakeMove`, `TestExposesCheck`,
+ `TestIsAttacked`). Add a **long-game fuzz test**: play N random games
+ of ~200 plies each (not just 1-ply-at-a-time exercises), asserting
+ `VerifyPositionConsistency` and `ComputeSig`-from-scratch agreement
+ with the incrementally-maintained signature at every ply. Confirm
+ `VerifyPositionConsistency` is actually compiled live in the
+ `TEST`/`DEBUG` build, not just present in source.
+
+### Hash (currently zero correctness coverage)
+
+New test needed, not just a wiring fix:
+- Store/probe roundtrip correctness.
+- Replacement-scheme behavior under forced collisions (fill table past
+ capacity with distinct signatures, confirm replacement policy
+ behaves as designed, not just "doesn't crash").
+- `ADJUST_MATE_IN_N` ply-distance adjustment on `HASH_FLAG_EXACT`
+ entries specifically -- the exact mechanism CLAUDE.md flags as
+ audited-by-hand but not test-covered after this session's `ASSERT`
+ removals in `QSearch()`.
+
+### Recognizer
+
+- Wire in `TestRecognExhaustiveKNKP` (see dead-infra list above).
+ Consider the same exhaustive-enumeration treatment for any other
+ small recognized endgame class in `recogn.c` (KPK, KRK) if one
+ exists without a test.
+
+## Perf testing
+
+Current coverage is narrow: five isolated per-piece-type
+mailbox-vs-bitboard microbenchmarks (`TestGenerate*Speed`), nothing for
+search, eval, or hash.
+
+- **Eval NPS/cycles-per-call benchmark**, same interleaved-comparison
+ shape as the existing generate benchmarks, by game phase -- eval is
+ the named bottleneck (CLAUDE.md's 1.5M vs 7.5M nps gap vs. Crafty)
+ and has no dedicated speed test today.
+- **Whole-search NPS benchmark**: `bench.c` already exists for this --
+ confirm whether it's part of the automated `TEST` self-test pass or
+ purely a manual `bench` command; if manual-only, wire a
+ fixed-node-budget version into the automated sequence so a
+ regression surfaces without someone remembering to run it by hand.
+- **Hash-probe-under-load benchmark**: cycles/probe at low occupancy
+ vs. near-full table -- replacement-scheme cost can regress silently
+ today with nothing to measure it.
+- **Stored baselines with pass/fail thresholds**, not print-only
+ output: today every perf test prints cycles/call and leaves
+ interpretation to a human reading the log. Store a last-known-good
+ baseline (same idea as `head_reference/` for solve-counts) and fail
+ loud if a call site regresses past some tolerance.
+
+## Suggested priority order
+
+1. Un-bitrot the three dead tests (SEE oracle,
+ `TestEvalWithKnownPositions`, `TestRecognExhaustiveKNKP`) --
+ near-zero cost, immediate coverage recovery.
+2. Hash-table correctness test (store/probe/replace/mate-adjust) --
+ currently the single biggest correctness blind spot.
+3. Search determinism + MP-vs-single-thread agreement test -- targets
+ the exact class of bug most likely to hide in threading/hash/
+ extension interactions that per-function unit tests can't see.
+4. Extend perft to deeper plies + divide output; add move-set-equality
+ assertions to the existing generate speed benchmarks (turns
+ dead-weight microbenchmarks into correctness tests for free).
+5. Eval: automate the symmetry fuzz test (loop N, not 1) and add
+ adversarial/degenerate-position term-bounds checks.
+6. Round out perf testing per the list above, with stored baselines
+ and pass/fail thresholds instead of print-only output.
diff --git a/src/chess.h b/src/chess.h
index 16cdca9..38fdb3f 100755
--- a/src/chess.h
+++ b/src/chess.h
@@ -2162,6 +2162,15 @@ PositionToFen(POSITION *p);
FLAG
FenToPosition(POSITION *p, char *szFen);
+FLAG
+ParseEpdRecord(char *szLineIn,
+ char *szFen, ULONG cbFen,
+ char *szId, ULONG cbId,
+ char szMoves[][SMALL_STRING_LEN_CHAR],
+ ULONG uMaxMoves,
+ ULONG *puNumMoves,
+ FLAG *pfIsAvoid);
+
//
// testfen.c
//
@@ -3051,6 +3060,7 @@ IsLegalDrawByRepetition(void);
// script.c
//
COMMAND(ScriptCommand);
+COMMAND(EpdCommand);
COMMAND(SolutionCommand);
@@ -3303,6 +3313,12 @@ SanityCheckMoves(IN SEARCHER_THREAD_CONTEXT *ctx,
FLAG
TestSearch(void);
+FLAG
+TestSearchHashConsistency(void);
+
+FLAG
+TestQSearchSanity(void);
+
//
// see.c
//
diff --git a/src/command.c b/src/command.c
index 0662333..5087aee 100644
--- a/src/command.c
+++ b/src/command.c
@@ -2720,6 +2720,12 @@ COMMAND_PARSER_ENTRY g_ParserTable[] =
FALSE,
TRUE,
"Inform the engine about a game result" },
+ { "epd",
+ EpdCommand,
+ FALSE,
+ FALSE,
+ FALSE,
+ "Run a raw EPD test suite file directly (no conversion needed)" },
{ "script",
ScriptCommand,
FALSE,
diff --git a/src/fen.c b/src/fen.c
index 33810e6..60ff4dc 100644
--- a/src/fen.c
+++ b/src/fen.c
@@ -781,3 +781,219 @@ Return value:
SystemFreeMemory(szCapturedFen);
return(TRUE);
}
+
+
+FLAG
+ParseEpdRecord(IN char *szLineIn,
+ OUT char *szFen, IN ULONG cbFen,
+ OUT char *szId, IN ULONG cbId,
+ OUT char szMoves[][SMALL_STRING_LEN_CHAR],
+ IN ULONG uMaxMoves,
+ OUT ULONG *puNumMoves,
+ OUT FLAG *pfIsAvoid)
+/**
+
+Routine description:
+
+ Real per-record EPD parsing: pull the board/side-to-move/castling/
+ en-passant fields plus the "bm"/"am" and "id" opcodes out of one
+ raw EPD line, e.g.:
+
+ 8/8/1p1r1k2/p1pPN1p1/P3KnP1/1P6/8/3R4 b - - 0 1 bm Nxd5; id "zugzwang.005";
+
+ Written to replace tests/convert_epd.pl's single-regex approach
+ ("everything between the board and bm is [^b]+"), which silently
+ breaks on exactly this kind of line: side-to-move "b" contains the
+ character the regex's filler class explicitly excludes, so the
+ match fails outright, and because that script's $fen/$id/$solution
+ were plain globals rather than per-line locals, a failed match
+ silently carried the *previous* line's values forward instead of
+ erroring. This function fails closed instead: a line it can't
+ parse returns FALSE and touches none of the OUT parameters, so a
+ caller looping over a file can never mistake stale data left over
+ from the previous line for a real result.
+
+ Tolerates the two EPD-ish flavors seen in real suites: strict EPD
+ (exactly 4 position fields, then opcodes) and the more common
+ FEN-plus-opcodes hybrid (4 position fields, then halfmove/fullmove
+ counters, then opcodes) -- an opcode name is never purely numeric,
+ so the two are unambiguous to tell apart by just skipping leading
+ all-digit tokens before looking for "bm"/"am"/"id".
+
+Parameters:
+
+ IN char *szLineIn : one raw EPD line (not modified)
+ OUT char *szFen : receives "<board> <stm> <castle> <ep>"
+ IN ULONG cbFen : size of szFen
+ OUT char *szId : receives the id string (unquoted), or "" if none
+ IN ULONG cbId : size of szId
+ OUT char szMoves[][] : receives each bm/am move token, in order
+ IN ULONG uMaxMoves : capacity of szMoves (row count)
+ OUT ULONG *puNumMoves : number of moves written to szMoves
+ OUT FLAG *pfIsAvoid : TRUE if the opcode was "am", FALSE if "bm"
+
+Return value:
+
+ FLAG : TRUE if a fen and at least one bm/am move were found and
+ fit in the caller's buffers; FALSE otherwise (all OUT
+ parameters left untouched)
+
+**/
+{
+ char szWork[SMALL_STRING_LEN_CHAR];
+ char *szTokens[4];
+ char *p;
+ char *szOpcodeSection;
+ char *szChunk;
+ char *szSavePtr1, *szSavePtr2;
+ ULONG u;
+ ULONG uNumMoves = 0;
+ FLAG fIsAvoid = FALSE;
+ char szLocalId[SMALL_STRING_LEN_CHAR];
+ char szLocalFen[SMALL_STRING_LEN_CHAR];
+
+ szLocalId[0] = '\0';
+
+ if ((NULL == szLineIn) || ('\0' == szLineIn[0]))
+ {
+ return(FALSE);
+ }
+ strncpy(szWork, szLineIn, ARRAY_LENGTH(szWork) - 1);
+ szWork[ARRAY_LENGTH(szWork) - 1] = '\0';
+ szWork[strcspn(szWork, "\r\n")] = '\0';
+
+ p = szWork;
+ while (isspace((unsigned char)*p)) p++;
+ if (('\0' == *p) || ('#' == *p))
+ {
+ return(FALSE);
+ }
+
+ // First 4 whitespace-delimited tokens are always the position.
+ for (u = 0; u < 4; u++)
+ {
+ while (isspace((unsigned char)*p)) p++;
+ if ('\0' == *p)
+ {
+ return(FALSE);
+ }
+ szTokens[u] = p;
+ while (*p && !isspace((unsigned char)*p)) p++;
+ if (*p)
+ {
+ *p = '\0';
+ p++;
+ }
+ }
+ if (snprintf(szLocalFen, ARRAY_LENGTH(szLocalFen), "%s %s %s %s",
+ szTokens[0], szTokens[1], szTokens[2], szTokens[3]) < 0)
+ {
+ return(FALSE);
+ }
+
+ // Skip FEN-style halfmove/fullmove counters ahead of the first
+ // real opcode -- an opcode name is never purely numeric.
+ for (;;)
+ {
+ char *q;
+ while (isspace((unsigned char)*p)) p++;
+ q = p;
+ if ('\0' == *q)
+ {
+ break;
+ }
+ while (isdigit((unsigned char)*q)) q++;
+ if ((q == p) || (*q && !isspace((unsigned char)*q) && (';' != *q)))
+ {
+ // Not a bare numeric token -- this is the opcode section.
+ break;
+ }
+ if ('\0' == *q)
+ {
+ p = q;
+ break;
+ }
+ p = q;
+ }
+ szOpcodeSection = p;
+
+ // Now split what's left on ';' into individual opcode chunks.
+ for (szChunk = strtok_r(szOpcodeSection, ";", &szSavePtr1);
+ NULL != szChunk;
+ szChunk = strtok_r(NULL, ";", &szSavePtr1))
+ {
+ char *szTrimmed = szChunk;
+ char *szTail;
+
+ while (isspace((unsigned char)*szTrimmed)) szTrimmed++;
+ szTail = szTrimmed + strlen(szTrimmed);
+ while ((szTail > szTrimmed) && isspace((unsigned char)szTail[-1]))
+ {
+ szTail--;
+ *szTail = '\0';
+ }
+ if ('\0' == *szTrimmed)
+ {
+ continue;
+ }
+
+ if (((0 == STRNCMPI(szTrimmed, "bm", 2)) &&
+ isspace((unsigned char)szTrimmed[2])) ||
+ ((0 == STRNCMPI(szTrimmed, "am", 2)) &&
+ isspace((unsigned char)szTrimmed[2])))
+ {
+ char *szMoveTok;
+
+ fIsAvoid = (0 == STRNCMPI(szTrimmed, "am", 2));
+ szMoveTok = strtok_r(szTrimmed + 2, " \t", &szSavePtr2);
+ while (NULL != szMoveTok)
+ {
+ if (uNumMoves < uMaxMoves)
+ {
+ strncpy(szMoves[uNumMoves], szMoveTok,
+ SMALL_STRING_LEN_CHAR - 1);
+ szMoves[uNumMoves][SMALL_STRING_LEN_CHAR - 1] = '\0';
+ uNumMoves++;
+ }
+ szMoveTok = strtok_r(NULL, " \t", &szSavePtr2);
+ }
+ }
+ else if ((0 == STRNCMPI(szTrimmed, "id", 2)) &&
+ isspace((unsigned char)szTrimmed[2]))
+ {
+ char *szIdVal = szTrimmed + 2;
+ while (isspace((unsigned char)*szIdVal)) szIdVal++;
+ if ('"' == *szIdVal)
+ {
+ char *szEndQuote;
+ szIdVal++;
+ szEndQuote = strchr(szIdVal, '"');
+ if (NULL != szEndQuote)
+ {
+ *szEndQuote = '\0';
+ }
+ }
+ strncpy(szLocalId, szIdVal, ARRAY_LENGTH(szLocalId) - 1);
+ szLocalId[ARRAY_LENGTH(szLocalId) - 1] = '\0';
+ }
+ // Other opcodes (acd, ce, dm, c0, ...) intentionally ignored.
+ }
+
+ if (0 == uNumMoves)
+ {
+ return(FALSE);
+ }
+ if (strlen(szLocalFen) >= cbFen)
+ {
+ return(FALSE);
+ }
+ strcpy(szFen, szLocalFen);
+ if (strlen(szLocalId) >= cbId)
+ {
+ szLocalId[cbId - 1] = '\0';
+ }
+ strcpy(szId, szLocalId);
+ *puNumMoves = uNumMoves;
+ *pfIsAvoid = fIsAvoid;
+ return(TRUE);
+}
diff --git a/src/main.c b/src/main.c
index 351e848..f61d840 100644
--- a/src/main.c
+++ b/src/main.c
@@ -619,6 +619,8 @@ Return value:
TestIsAttacked();
TestMakeUnmakeMove();
TestSearch();
+ TestSearchHashConsistency();
+ TestQSearchSanity();
return(TRUE);
}
diff --git a/src/script.c b/src/script.c
index 0bc09e4..e8d6f9a 100644
--- a/src/script.c
+++ b/src/script.c
@@ -349,6 +349,160 @@ Return value:
+//
+// Shared by ScriptCommand and EpdCommand: process one already-read
+// line exactly the way a script file's line is processed -- "go"
+// triggers a search plus counter accumulation, anything else is
+// dispatched through the normal command parser. Factored out so
+// EpdCommand can feed it lines synthesized from a raw EPD record
+// (setboard/id/solution/avoid/go) instead of lines read verbatim from
+// a file, without duplicating this logic or its counter bookkeeping.
+//
+static void
+_ProcessSuiteLine(char *szLine, POSITION *pos)
+{
+ Trace("SCRIPT> %s\n", szLine);
+ if (!STRNCMPI(szLine, "go", 2))
+ {
+ (void)Think(pos);
+ g_SuiteCounters.u64TotalNodeCount +=
+ g_Options.u64NodesSearched;
+ g_SuiteCounters.u64TotalBetaCutoffs +=
+ g_Options.u64BetaCutoffs;
+ g_SuiteCounters.u64TotalBetaCutoffsOnFirstMove +=
+ g_Options.u64BetaCutoffsOnFirstMove;
+ g_SuiteCounters.u64TotalCounterMoveTries +=
+ g_Options.u64CounterMoveTries;
+ g_SuiteCounters.u64TotalCounterMoveHits +=
+ g_Options.u64CounterMoveHits;
+ g_SuiteCounters.u64TotalLeftoverTries +=
+ g_Options.u64LeftoverTries;
+ g_SuiteCounters.u64TotalLeftoverAlpha +=
+ g_Options.u64LeftoverAlpha;
+ g_SuiteCounters.u64TotalLeftoverFH +=
+ g_Options.u64LeftoverFH;
+ {
+ ULONG v;
+ for (v = 0; v < CM_EVIDENCE_BUCKETS; v++)
+ {
+ g_SuiteCounters.u64TotalCMEvidenceTries[v] +=
+ g_Options.u64CMEvidenceTries[v];
+ g_SuiteCounters.u64TotalCMEvidenceFH[v] +=
+ g_Options.u64CMEvidenceFH[v];
+ }
+ }
+ }
+ else
+ {
+ PushNewInput(szLine);
+ if (FALSE == g_fExitProgram)
+ {
+ ParseUserInput(FALSE);
+ }
+ }
+}
+
+
+//
+// Shared by ScriptCommand and EpdCommand: the end-of-suite banner and
+// statistics report. Verbatim extraction of what used to be
+// ScriptCommand's own tail -- see that function's history for the
+// meaning of each field.
+//
+static void
+_PrintSuiteReport(double dSuiteStart)
+{
+ ULONG u, v, uMax;
+ double dMult = (double)g_Options.uMyIncrement / (double)SUITE_NUM_HISTOGRAM;
+
+ Banner();
+ Trace("\n"
+ "TEST SCRIPT execution complete. Final statistics:\n"
+ "--------------------------------------------------\n"
+ " correct solutions : %u\n"
+ " incorrect solutions : %u\n"
+ " total problems : %u\n"
+ " total nodecount : %"
+ COMPILER_LONGLONG_UNSIGNED_FORMAT "\n"
+ " avg. search speed : %6.1f nps\n"
+ " avg. solution time : %3.1f sec\n"
+ " avg. search depth : %4.1f ply\n"
+ " 1st move beta cut : %5.2f percent\n"
+ " avg. eff. branching : %5.3f\n"
+ " counter move hit %% : %5.2f percent (%"
+ COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n"
+ " leftover : %5.2f%% alpha, %5.2f%% FH (%"
+ COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n"
+ " script time : %6.1f sec\n\n",
+ g_SuiteCounters.uCorrect,
+ g_SuiteCounters.uIncorrect,
+ g_SuiteCounters.uTotal,
+ g_SuiteCounters.u64TotalNodeCount,
+ ((double)g_SuiteCounters.u64TotalNodeCount /
+ (SystemTimeStamp() - dSuiteStart)),
+ ((g_SuiteCounters.uCorrect > 0) ?
+ (g_SuiteCounters.dSigmaSolutionTime /
+ (double)g_SuiteCounters.uCorrect) : 0.0),
+ ((double)g_SuiteCounters.uSigmaDepth /
+ (double)g_SuiteCounters.uTotal),
+ (100.0 * (double)g_SuiteCounters.u64TotalBetaCutoffsOnFirstMove /
+ ((double)g_SuiteCounters.u64TotalBetaCutoffs + 1.0)),
+ (g_SuiteCounters.dSigmaEBF /
+ ((double)g_SuiteCounters.uEBFCount + 1.0)),
+ (100.0 * (double)g_SuiteCounters.u64TotalCounterMoveHits /
+ ((double)g_SuiteCounters.u64TotalCounterMoveTries + 1.0)),
+ g_SuiteCounters.u64TotalCounterMoveTries,
+ (100.0 * (double)g_SuiteCounters.u64TotalLeftoverAlpha /
+ ((double)g_SuiteCounters.u64TotalLeftoverTries + 1.0)),
+ (100.0 * (double)g_SuiteCounters.u64TotalLeftoverFH /
+ ((double)g_SuiteCounters.u64TotalLeftoverTries + 1.0)),
+ g_SuiteCounters.u64TotalLeftoverTries,
+ (SystemTimeStamp() - dSuiteStart));
+
+ // Countermove evidence calibration -- FH% per evidence bucket,
+ // an ongoing check that COUNTERMOVE_EVIDENCE_THRESHOLD (chess.h)
+ // is still well-calibrated as the engine/data evolve.
+ {
+ static const ULONG uCMEvidenceFloors[CM_EVIDENCE_BUCKETS] =
+ { 0, 1, 100, 1000, 10000, 100000, 1000000 };
+ Trace("\ncountermove evidence calibration:\n");
+ for (v = 0; v < CM_EVIDENCE_BUCKETS; v++)
+ {
+ Trace(" evidence >= %8lu : %5.2f%% FH (%"
+ COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n",
+ uCMEvidenceFloors[v],
+ (100.0 * (double)g_SuiteCounters.u64TotalCMEvidenceFH[v] /
+ ((double)g_SuiteCounters.u64TotalCMEvidenceTries[v] + 1.0)),
+ g_SuiteCounters.u64TotalCMEvidenceTries[v]);
+ }
+ }
+
+ // Histogram stuff
+ if (g_SuiteCounters.uTotal > 0) {
+ uMax = g_SuiteCounters.uHistogram[0];
+ for (u = 1; u < SUITE_NUM_HISTOGRAM; u++)
+ {
+ if (g_SuiteCounters.uHistogram[u] > uMax)
+ {
+ uMax = g_SuiteCounters.uHistogram[u];
+ }
+ }
+ for (u = 0; u < SUITE_NUM_HISTOGRAM; u++)
+ {
+ Trace("%4.1f .. %4.1f: ",
+ (double)u * dMult, (double)(u + 1) * dMult);
+ for (v = 0;
+ (uMax > 0) && (v < 50 * g_SuiteCounters.uHistogram[u] / uMax);
+ v++)
+ {
+ Trace("*");
+ }
+ Trace("\n");
+ }
+ }
+}
+
+
COMMAND(ScriptCommand)
/**
@@ -389,8 +543,6 @@ Return value:
FILE *p;
static CHAR szLine[SMALL_STRING_LEN_CHAR];
double dSuiteStart;
- ULONG u, v, uMax;
- double dMult;
if (argc < 2)
{
@@ -402,143 +554,137 @@ Return value:
Trace("Error (file doesn't exist): %s\n", argv[1]);
return;
}
-
+
p = fopen(argv[1], "rb");
if (NULL != p)
{
- // Doing the script...
dSuiteStart = SystemTimeStamp();
memset(&(g_SuiteCounters), 0, sizeof(g_SuiteCounters));
while(fgets(szLine, ARRAY_LENGTH(szLine), p))
{
- Trace("SCRIPT> %s\n", szLine);
- if (!STRNCMPI(szLine, "go", 2))
- {
- (void)Think(pos);
- g_SuiteCounters.u64TotalNodeCount +=
- g_Options.u64NodesSearched;
- g_SuiteCounters.u64TotalBetaCutoffs +=
- g_Options.u64BetaCutoffs;
- g_SuiteCounters.u64TotalBetaCutoffsOnFirstMove +=
- g_Options.u64BetaCutoffsOnFirstMove;
- g_SuiteCounters.u64TotalCounterMoveTries +=
- g_Options.u64CounterMoveTries;
- g_SuiteCounters.u64TotalCounterMoveHits +=
- g_Options.u64CounterMoveHits;
- g_SuiteCounters.u64TotalLeftoverTries +=
- g_Options.u64LeftoverTries;
- g_SuiteCounters.u64TotalLeftoverAlpha +=
- g_Options.u64LeftoverAlpha;
- g_SuiteCounters.u64TotalLeftoverFH +=
- g_Options.u64LeftoverFH;
- for (v = 0; v < CM_EVIDENCE_BUCKETS; v++)
- {
- g_SuiteCounters.u64TotalCMEvidenceTries[v] +=
- g_Options.u64CMEvidenceTries[v];
- g_SuiteCounters.u64TotalCMEvidenceFH[v] +=
- g_Options.u64CMEvidenceFH[v];
- }
- }
- else
- {
- PushNewInput(szLine);
- if (TRUE == g_fExitProgram) break;
- ParseUserInput(FALSE);
- }
+ _ProcessSuiteLine(szLine, pos);
+ if (TRUE == g_fExitProgram) break;
}
fclose(p);
+ _PrintSuiteReport(dSuiteStart);
+ }
+}
- // Banner and end results
- dMult = (double)g_Options.uMyIncrement / (double)SUITE_NUM_HISTOGRAM;
- Banner();
- Trace("\n"
- "TEST SCRIPT execution complete. Final statistics:\n"
- "--------------------------------------------------\n"
- " correct solutions : %u\n"
- " incorrect solutions : %u\n"
- " total problems : %u\n"
- " total nodecount : %"
- COMPILER_LONGLONG_UNSIGNED_FORMAT "\n"
- " avg. search speed : %6.1f nps\n"
- " avg. solution time : %3.1f sec\n"
- " avg. search depth : %4.1f ply\n"
- " 1st move beta cut : %5.2f percent\n"
- " avg. eff. branching : %5.3f\n"
- " counter move hit %% : %5.2f percent (%"
- COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n"
- " leftover : %5.2f%% alpha, %5.2f%% FH (%"
- COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n"
- " script time : %6.1f sec\n\n",
- g_SuiteCounters.uCorrect,
- g_SuiteCounters.uIncorrect,
- g_SuiteCounters.uTotal,
- g_SuiteCounters.u64TotalNodeCount,
- ((double)g_SuiteCounters.u64TotalNodeCount /
- (SystemTimeStamp() - dSuiteStart)),
- ((g_SuiteCounters.uCorrect > 0) ?
- (g_SuiteCounters.dSigmaSolutionTime /
- (double)g_SuiteCounters.uCorrect) : 0.0),
- ((double)g_SuiteCounters.uSigmaDepth /
- (double)g_SuiteCounters.uTotal),
- (100.0 * (double)g_SuiteCounters.u64TotalBetaCutoffsOnFirstMove /
- ((double)g_SuiteCounters.u64TotalBetaCutoffs + 1.0)),
- (g_SuiteCounters.dSigmaEBF /
- ((double)g_SuiteCounters.uEBFCount + 1.0)),
- (100.0 * (double)g_SuiteCounters.u64TotalCounterMoveHits /
- ((double)g_SuiteCounters.u64TotalCounterMoveTries + 1.0)),
- g_SuiteCounters.u64TotalCounterMoveTries,
- (100.0 * (double)g_SuiteCounters.u64TotalLeftoverAlpha /
- ((double)g_SuiteCounters.u64TotalLeftoverTries + 1.0)),
- (100.0 * (double)g_SuiteCounters.u64TotalLeftoverFH /
- ((double)g_SuiteCounters.u64TotalLeftoverTries + 1.0)),
- g_SuiteCounters.u64TotalLeftoverTries,
- (SystemTimeStamp() - dSuiteStart));
- // Countermove evidence calibration -- FH% per evidence bucket,
- // an ongoing check that COUNTERMOVE_EVIDENCE_THRESHOLD (chess.h)
- // is still well-calibrated as the engine/data evolve.
+COMMAND(EpdCommand)
+/**
+
+Routine description:
+
+ Runs a raw EPD test suite file directly -- no perl preprocessing
+ into a setboard/solution/go script first. Historically the only
+ way to run a suite from real-world EPD (the "board stm castle ep
+ [halfmove fullmove] bm <move>; id \"...\"; ..." format most public
+ tactical suites ship in) was tests/convert_epd.pl, which had a
+ real, silent-corruption bug: its single "everything between the
+ board and bm is [^b]+" regex breaks on any Black-to-move record
+ (the side-to-move field is literally the character "b"), and
+ because $fen/$id/$solution were unscoped globals rather than
+ per-line locals, a failed match silently carried the *previous*
+ record's board and solution forward instead of erroring --
+ confirmed live by feeding it a real 5-position batch, where the
+ one Black-to-move line came out as a byte-for-byte duplicate of
+ the line before it. This command replaces that whole external
+ preprocessing step with real per-record EPD parsing (see
+ ParseEpdRecord, fen.c) feeding the exact same
+ _ProcessSuiteLine/_PrintSuiteReport machinery ScriptCommand uses,
+ so a raw EPD file can just be handed to the engine directly.
+
+ Usage:
+
+ epd <required filename>
+
+Parameters:
+
+ The COMMAND macro hides four arguments from the input parser:
+
+ CHAR *szInput : the full line of input
+ ULONG argc : number of argument chunks
+ CHAR *argv[] : array of ptrs to each argument chunk
+ POSITION *pos : a POSITION pointer to operate on
+
+Return value:
+
+ void
+
+**/
+{
+ FILE *p;
+ static CHAR szLine[SMALL_STRING_LEN_CHAR];
+ static CHAR szSynth[SMALL_STRING_LEN_CHAR];
+ CHAR szFen[SMALL_STRING_LEN_CHAR];
+ CHAR szId[SMALL_STRING_LEN_CHAR];
+ CHAR szMoves[SUITE_MAX_SOLUTIONS + SUITE_MAX_AVOIDS][SMALL_STRING_LEN_CHAR];
+ ULONG uNumMoves;
+ FLAG fIsAvoid;
+ double dSuiteStart;
+ ULONG u;
+ ULONG uLineNum = 0;
+
+ if (argc < 2)
+ {
+ Trace("Error (Usage: epd <filename>): %s\n", argv[0]);
+ return;
+ }
+ if (FALSE == SystemDoesFileExist(argv[1]))
+ {
+ Trace("Error (file doesn't exist): %s\n", argv[1]);
+ return;
+ }
+
+ p = fopen(argv[1], "rb");
+ if (NULL != p)
+ {
+ dSuiteStart = SystemTimeStamp();
+ memset(&(g_SuiteCounters), 0, sizeof(g_SuiteCounters));
+ while(fgets(szLine, ARRAY_LENGTH(szLine), p))
{
- static const ULONG uCMEvidenceFloors[CM_EVIDENCE_BUCKETS] =
- { 0, 1, 100, 1000, 10000, 100000, 1000000 };
- Trace("\ncountermove evidence calibration:\n");
- for (v = 0; v < CM_EVIDENCE_BUCKETS; v++)
+ uLineNum++;
+ if (FALSE == ParseEpdRecord(szLine, szFen, ARRAY_LENGTH(szFen),
+ szId, ARRAY_LENGTH(szId),
+ szMoves, ARRAY_LENGTH(szMoves),
+ &uNumMoves, &fIsAvoid))
{
- Trace(" evidence >= %8lu : %5.2f%% FH (%"
- COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n",
- uCMEvidenceFloors[v],
- (100.0 * (double)g_SuiteCounters.u64TotalCMEvidenceFH[v] /
- ((double)g_SuiteCounters.u64TotalCMEvidenceTries[v] + 1.0)),
- g_SuiteCounters.u64TotalCMEvidenceTries[v]);
+ if (szLine[0] && (szLine[0] != '\n') && (szLine[0] != '#'))
+ {
+ Trace("epd: skipping unparseable line %lu: %s",
+ uLineNum, szLine);
+ }
+ continue;
}
- }
- // Histogram stuff
- if (g_SuiteCounters.uTotal > 0) {
- uMax = g_SuiteCounters.uHistogram[0];
- for (u = 1; u < SUITE_NUM_HISTOGRAM; u++)
+ snprintf(szSynth, ARRAY_LENGTH(szSynth), "setboard %s\n", szFen);
+ _ProcessSuiteLine(szSynth, pos);
+ if (TRUE == g_fExitProgram) break;
+
+ if (szId[0])
{
- if (g_SuiteCounters.uHistogram[u] > uMax)
- {
- uMax = g_SuiteCounters.uHistogram[u];
- }
+ snprintf(szSynth, ARRAY_LENGTH(szSynth), "id %s\n", szId);
+ _ProcessSuiteLine(szSynth, pos);
}
- for (u = 0; u < SUITE_NUM_HISTOGRAM; u++)
+
+ for (u = 0; u < uNumMoves; u++)
{
- Trace("%4.1f .. %4.1f: ",
- (double)u * dMult, (double)(u + 1) * dMult);
- for (v = 0;
- (uMax > 0) && (v < 50 * g_SuiteCounters.uHistogram[u] / uMax);
- v++)
- {
- Trace("*");
- }
- Trace("\n");
+ snprintf(szSynth, ARRAY_LENGTH(szSynth), "%s %s\n",
+ fIsAvoid ? "avoid" : "solution", szMoves[u]);
+ _ProcessSuiteLine(szSynth, pos);
}
+
+ _ProcessSuiteLine("go\n", pos);
+ if (TRUE == g_fExitProgram) break;
}
+ fclose(p);
+ _PrintSuiteReport(dSuiteStart);
}
}
-FLAG
+
+FLAG
WeAreRunningASuite(void)
/**
diff --git a/src/testsearch.c b/src/testsearch.c
index 3d560ac..6b43f64 100644
--- a/src/testsearch.c
+++ b/src/testsearch.c
@@ -131,4 +131,302 @@ TestSearch(void)
SystemFreeMemory(ctx);
return(fRet);
}
+
+
+//
+// Shared setup for the hash-consistency and zugzwang tests below: run
+// one fixed-depth search on a fresh SEARCHER_THREAD_CONTEXT-relative
+// position (ctx is caller-owned and reused across calls, matching
+// TestSearch's own pattern -- SystemAllocateMemory'ing a fresh 75MB
+// context per call would be wasteful for tests that need many runs
+// over the same handful of positions) and return the root score.
+// Does NOT touch the hash table itself (no Clear/Dirty call) -- that
+// decision is the caller's, since the whole point of the hash tests
+// below is controlling exactly what's in the table before each call.
+//
+static SCORE
+_FixedDepthSearchScore(SEARCHER_THREAD_CONTEXT *ctx,
+ POSITION *pos,
+ ULONG uMaxDepth)
+{
+ GAME_RESULT result;
+ FLAG fPost = g_Options.fShouldPost;
+
+ g_Options.fShouldPost = FALSE;
+ InitializeSearcherContext(pos, ctx);
+ g_MoveTimer.bvFlags = 0;
+ g_Options.fPondering = FALSE;
+ g_Options.fThinking = TRUE;
+ g_Options.fSuccessfulPonder = FALSE;
+ MaintainDynamicMoveOrdering();
+ SetMoveTimerForTestingSearch();
+ g_Options.uMaxDepth = uMaxDepth;
+#if (PERF_COUNTERS && MP)
+ ClearHelperThreadIdleness();
+#endif
+ result = Iterate(ctx);
+ g_Options.fShouldPost = fPost;
+ if (RESULT_IN_PROGRESS != result.eResult)
+ {
+ // Mate/stalemate/draw at the root -- no root score was set by
+ // this iteration (there's no move to score). Not a bug, just
+ // not a comparable data point for the callers below.
+ return(INVALID_SCORE);
+ }
+ return(ctx->iRootScore);
+}
+
+
+//
+// Hash-table correctness, exercised through real search rather than
+// synthetic store/probe calls -- the actual bug class this is meant
+// to catch (stale entries surviving a generation boundary, a bound
+// getting misapplied, ADJUST_MATE_IN_N misfiring on an entry that
+// shouldn't get it) only shows up when a search's outcome actually
+// depends on what it reads back out of the table, which a
+// hand-constructed store()-then-probe() unit test can't exercise --
+// see TESTING_PLAN.md's "hash correctness" section for the case for a
+// synthetic store/probe test too; this is the complementary
+// behavioral half, not a replacement for it.
+//
+// Two invariants, neither of which cares about search internals
+// (move ordering, PV, node count) changing across future search work
+// -- only that the table is *transparent*, i.e. its presence changes
+// performance, never the answer:
+//
+// 1. Searching the same position to the same depth twice in a row
+// (second time with the table already populated by the first)
+// must produce the same score both times.
+// 2. Searching to depth D2 with the table pre-populated by a
+// shallower depth-D1 search of the same position must produce
+// the same score as searching directly to D2 from a freshly
+// cleared table. A depth-D1 entry read back during the depth-D2
+// search is only ever a bound, never trusted as exact at a
+// higher depth than it was stored at -- if it were, this would
+// catch it.
+//
+FLAG
+TestSearchHashConsistency(void)
+{
+ static const char *rgszFen[] =
+ {
+ "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1",
+ "r1bq1rk1/pp2bppp/2n1pn2/2pp4/3P4/2NBPN2/PP3PPP/R1BQ1RK1 w - - 0 1",
+ "r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 1",
+ "8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - 0 1",
+ };
+ const ULONG uD1 = 4;
+ const ULONG uD2 = 6;
+ SEARCHER_THREAD_CONTEXT *ctx;
+ POSITION pos;
+ SCORE iFirst, iSecond, iShallow, iDirect;
+ ULONG u;
+ FLAG fRet = TRUE;
+
+ Trace("Testing hash table consistency through search...\n");
+ ctx = SystemAllocateMemory(sizeof(SEARCHER_THREAD_CONTEXT));
+ ASSERT(ctx);
+
+ for (u = 0; u < ARRAY_LENGTH(rgszFen); u++)
+ {
+ if (FALSE == FenToPosition(&pos, (char *)rgszFen[u]))
+ {
+ UtilPanic(INCONSISTENT_STATE,
+ NULL, NULL, NULL, NULL,
+ __FILE__, __LINE__);
+ }
+
+ // Invariant 1: repeat-with-warm-table stability.
+ ClearHashTable();
+ DirtyHashTable();
+ iFirst = _FixedDepthSearchScore(ctx, &pos, uD1);
+ DirtyHashTable();
+ iSecond = _FixedDepthSearchScore(ctx, &pos, uD1);
+ if ((INVALID_SCORE != iFirst) &&
+ (INVALID_SCORE != iSecond) &&
+ (iFirst != iSecond))
+ {
+ Trace("TestSearchHashConsistency: position %lu: same-depth "
+ "rerun with warm hash table diverged (%d vs %d) -- "
+ "FEN: %s\n", u, iFirst, iSecond, rgszFen[u]);
+ fRet = FALSE;
+ }
+
+ // Invariant 2: a shallower pre-population must not corrupt a
+ // deeper search's answer.
+ ClearHashTable();
+ DirtyHashTable();
+ iShallow = _FixedDepthSearchScore(ctx, &pos, uD1);
+ DirtyHashTable();
+ iSecond = _FixedDepthSearchScore(ctx, &pos, uD2);
+
+ ClearHashTable();
+ DirtyHashTable();
+ iDirect = _FixedDepthSearchScore(ctx, &pos, uD2);
+
+ if ((INVALID_SCORE != iSecond) &&
+ (INVALID_SCORE != iDirect) &&
+ (iSecond != iDirect))
+ {
+ Trace("TestSearchHashConsistency: position %lu: depth-%lu "
+ "search seeded by a depth-%lu warm-up (%d) disagreed "
+ "with a from-scratch depth-%lu search (%d) -- FEN: "
+ "%s\n", u, uD2, uD1, iSecond, uD2, iDirect,
+ rgszFen[u]);
+ fRet = FALSE;
+ }
+ (void)iShallow;
+ }
+
+ if (FALSE == fRet)
+ {
+ UtilPanic(TESTCASE_FAILURE,
+ NULL, "TestSearchHashConsistency", NULL, NULL,
+ __FILE__, __LINE__);
+ }
+ ClearHashTable();
+ SystemFreeMemory(ctx);
+ return(fRet);
+}
+
+
+// Null-move soundness is intentionally NOT tested here as a C unit
+// test. An earlier version of this file asserted against
+// ctx->sCounters.extension.uZugzwang (searchsup.c's verification-
+// search counter) using the three classic "verified null-move
+// pruning" zugzwang positions -- but on this codebase's own material
+// gates (WeShouldTryNullmovePruning's `> 2` non-pawn-count guard,
+// TryNullmovePruning's `< 4` verification window) and this box's
+// Syzygy coverage, none of the three ever actually reached the
+// mechanism: two have too little material for null-move pruning to
+// be attempted at all, and the third gets resolved directly off the
+// local tablebase before search ever touches null-move logic. That
+// made the internal-counter approach fragile and environment-
+// dependent in a way a real best-move suite isn't. See
+// tests/zugzwang.epd (run with the `epd` command, e.g. `epd
+// tests/zugzwang.epd`) instead -- five known zugzwang positions
+// checked by best-move outcome, which doesn't care why the engine
+// gets it right, just whether it does.
+//
+
+//
+// QSearch is meant to stay small and shallow (search.c's own
+// ASSERT(pf->uQsearchNodes < 20000) at its only call site is a
+// no-op in a release build -- this makes a version of that bound a
+// real, always-checked test instead of a DEBUG-only assertion).
+//
+// Note this checks ctx->sCounters.tree.u64TotalNodeCount (whole
+// search, root.c's own unconditional per-node ++), not
+// u64QNodeCount specifically -- that counter, and every other field
+// in sCounters, is wrapped in the INC() macro (chess.h), which
+// compiles to nothing unless PERF_COUNTERS is defined, and this
+// self-test suite is built with plain `gmake TEST=1` (see CLAUDE.md),
+// which doesn't define it (confirmed live: a real `sd 3; go` on this
+// TEST-only binary reports "0 qnodes" no matter how much qsearch
+// activity actually happened). A whole-search bound is a strictly
+// weaker signal than a qsearch-only one -- it wouldn't distinguish
+// "qsearch exploded" from "full-width search exploded" -- but it's
+// the honest thing this build can actually check; rebuilding with
+// PERF_COUNTERS added would be needed to test u64QNodeCount for real.
+//
+// "Small" is also exactly what makes qsearch trees hand-auditable in
+// principle: a person could Trace() every node visited on one of the
+// positions below and read the resulting tree start to finish against
+// the board, unlike a full alpha-beta tree at real search depth. No
+// such tracing is wired up here (that would need a new instrumented
+// build define, not written this session) -- noting it as a
+// worthwhile follow-up, not implying it already exists.
+//
+FLAG
+TestQSearchSanity(void)
+{
+ static const char *rgszFen[] =
+ {
+ // Quiet position -- qsearch should do almost nothing (stand
+ // pat immediately, no captures available).
+ "4k3/8/8/8/8/8/8/4K3 w - - 0 1",
+ // A short, bounded capture sequence on one square.
+ "4k3/8/8/3p4/2P5/8/8/4K3 w - - 0 1",
+ // A deeper capturing melee -- the kind of position qsearch
+ // exists for.
+ "r1bq1rk1/pp2bppp/2n1pn2/2pp4/3P4/2NBPN2/PP3PPP/R1BQ1RK1 w - - 0 1",
+ "r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 1",
+ };
+ const ULONG uDepth = 3;
+ SEARCHER_THREAD_CONTEXT *ctx;
+ POSITION pos;
+ SCORE iScore;
+ ULONG u;
+ FLAG fRet = TRUE;
+
+ Trace("Testing QSearch sanity (node-count bound, termination)...\n");
+ ctx = SystemAllocateMemory(sizeof(SEARCHER_THREAD_CONTEXT));
+ ASSERT(ctx);
+
+ for (u = 0; u < ARRAY_LENGTH(rgszFen); u++)
+ {
+ if (FALSE == FenToPosition(&pos, (char *)rgszFen[u]))
+ {
+ UtilPanic(INCONSISTENT_STATE,
+ NULL, NULL, NULL, NULL,
+ __FILE__, __LINE__);
+ }
+ // uMaxDepth == 3 plies -- enough that the last full-width ply
+ // (uDepth == 0 in Search()'s ONE_PLY units) actually falls
+ // below Search()'s THREE_QUARTERS_PLY qsearch-entry threshold
+ // and real qsearch nodes get visited (measured live: depth 1
+ // resolved every position here without qsearch ever firing --
+ // g_uQNodeCount stayed 0 -- since a two-king or short-capture
+ // position this shallow settles via CommonSearchInit's own
+ // early-outs before the qsearch dispatch is ever reached).
+ // Still small enough to trace by hand per this function's own
+ // header comment.
+ ClearHashTable();
+ DirtyHashTable();
+ iScore = _FixedDepthSearchScore(ctx, &pos, uDepth);
+
+ if ((INVALID_SCORE != iScore) &&
+ !IS_VALID_SCORE(iScore))
+ {
+ Trace("TestQSearchSanity: position %lu returned an "
+ "invalid score %d -- FEN: %s\n", u, iScore,
+ rgszFen[u]);
+ fRet = FALSE;
+ }
+ Trace("TestQSearchSanity: position %lu total nodes this "
+ "search: %llu\n", u,
+ ctx->sCounters.tree.u64TotalNodeCount);
+
+ // Not the same invariant as search.c's per-entry
+ // ASSERT(pf->uQsearchNodes < 20000) (that resets per qsearch
+ // entry point and counts qsearch nodes only; this is a
+ // whole-search cumulative total, per this function's header
+ // comment on why), but a real, always-checked (not DEBUG-only)
+ // ceiling on "a shallow fixed-depth search on a small position
+ // stays small" -- a qsearch that's stopped terminating cleanly
+ // (e.g. a check-extension/repetition interaction letting
+ // QPLIES_OF_NON_CAPTURE_CHECKS get bypassed) will blow through
+ // this by orders of magnitude, not marginally, regardless of
+ // it being mixed in with full-width nodes here.
+ if (ctx->sCounters.tree.u64TotalNodeCount > 500000)
+ {
+ Trace("TestQSearchSanity: position %lu total node count "
+ "%llu is wildly outside the expected 'small' range "
+ "for depth %lu -- FEN: %s\n", u,
+ ctx->sCounters.tree.u64TotalNodeCount, uDepth,
+ rgszFen[u]);
+ fRet = FALSE;
+ }
+ }
+
+ if (FALSE == fRet)
+ {
+ UtilPanic(TESTCASE_FAILURE,
+ NULL, "TestQSearchSanity", NULL, NULL,
+ __FILE__, __LINE__);
+ }
+ ClearHashTable();
+ SystemFreeMemory(ctx);
+ return(fRet);
+}
#endif