diff options
| author | Scott Gasch <[email protected]> | 2026-10-09 14:18:52 -0700 |
|---|---|---|
| committer | Scott Gasch <[email protected]> | 2026-10-09 14:18:52 -0700 |
| commit | fbdcb9d76dd2444d8bc631331eaeb9435cc247f8 (patch) | |
| tree | 602ab360462d6f23188ba51f962c4abb2dc6c7e5 /src/testsearch.c | |
| parent | 73b4afb12abf7cc442831dea42b41b63a2ce07be (diff) | |
* Grab bag of eval.c changes.
* Don't bother not hashing when moves are pruned.
* Prune moves move librerally with LMP.
Inexplicably, this is slightly shallower than HEAD (despite pruning
MORE). It may be because it is slightly slower.
It consistently beats HEAD in match_play.py though (3x now).
Diffstat (limited to 'src/testsearch.c')
| -rw-r--r-- | src/testsearch.c | 178 |
1 files changed, 28 insertions, 150 deletions
diff --git a/src/testsearch.c b/src/testsearch.c index 9b3129c..afd2733 100644 --- a/src/testsearch.c +++ b/src/testsearch.c @@ -159,12 +159,7 @@ TestSearch(void) // cousin nodes are visited -- so two calls that visit nodes in a // different order (which a warm vs. cold hash table, by design, makes // them do) can legitimately reduce/prune the same move differently, -// producing a different score with no bug involved anywhere. A full -// clear here is what actually isolates "did the hash table change the -// answer" (what TestSearchHashConsistency, below, means to test) from -// "did ordinary order-dependent move-ordering heuristics change the -// answer" (expected, unrelated to hash-table correctness, and true of -// this engine with or without LMR). +// producing a different score with no bug involved anywhere. static SCORE _FixedDepthSearchScore(SEARCHER_THREAD_CONTEXT *ctx, POSITION *pos, @@ -198,150 +193,33 @@ _FixedDepthSearchScore(SEARCHER_THREAD_CONTEXT *ctx, } -// -// Hash-table correctness, exercised through real search rather than -// synthetic store/probe calls -- the actual bug class this is meant -// to catch (stale entries surviving a generation boundary, a bound -// getting misapplied, ADJUST_MATE_IN_N misfiring on an entry that -// shouldn't get it) only shows up when a search's outcome actually -// depends on what it reads back out of the table, which a -// hand-constructed store()-then-probe() unit test can't exercise -- -// see TESTING_PLAN.md's "hash correctness" section for the case for a -// synthetic store/probe test too; this is the complementary -// behavioral half, not a replacement for it. -// -// Two invariants, intended to care only about the hash table being -// *transparent* (its presence changes performance, never the answer), -// not about search internals (move ordering, PV, node count) changing -// across future search work -- but see invariant 2's own note below, -// 2026-10-04: that intent does not actually hold for invariant 2 once -// any move-order-sensitive heuristic (this engine has several; LMR is -// the one that exposed it) is in the search, so it is diagnostic-only -// now, not asserted. -// -// 1. Searching the same position to the same depth twice in a row -// (second time with the table already populated by the first) -// must produce the same score both times. Still asserted: the -// second run's hash move at every node is the SAME move the -// first run itself discovered as best there (same position, same -// depth), so warming the table cannot promote a DIFFERENT move -// to be tried first and thereby shift anything's ordinal -// position -- there is no move-order-sensitivity window for this -// invariant the way there is for invariant 2. -// 2. Searching to depth D2 with the table pre-populated by a -// shallower depth-D1 search of the same position was meant to -// produce the same score as searching directly to D2 from a -// freshly cleared table -- catching a depth-D1 entry wrongly -// trusted as exact at the deeper D2 probe. Downgraded to -// diagnostic-only (logged, not asserted) 2026-10-04: proven (via -// bisection against DISABLE_LMR, with g_HistoryCounters/ -// g_ContinuationHistory/g_FailHighs already fully cleared between -// every search below -- see _FixedDepthSearchScore's own comment) -// to fail even on the symmetric starting position, for a reason -// that has nothing to do with any hash-table bug: the shallower -// warm-up leaves a DIFFERENT move in the table as this node's -// hash move than a cold search would pick on its own, which -// promotes a different move to be tried first, which shifts -// every OTHER move's ordinal position by one -- and LMR's own -// "lateness" admission signal (MoveNumBucket(uLegalMoves), -// feeding the joint table's uPredictedFH) depends on exactly that -// ordinal position. Two equally-legitimate searches of the same -// nominal depth can therefore reduce a given move in one and not -// the other, with zero dependence on any learned/dynamic table -- -// this is a property of combining a hash table with ANY -// move-order-sensitive pruning/reduction heuristic, not a defect -// in this one. The genuine bug class this invariant was meant to -// catch (a shallow bound trusted as exact at a deeper probe) is -// real and still worth watching for by eye in the log; it is just -// no longer something this test can assert without false -// positives from ordinary LMR behavior. -// -FLAG -TestSearchHashConsistency(void) -{ - static const char *rgszFen[] = - { - "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1", - "r1bq1rk1/pp2bppp/2n1pn2/2pp4/3P4/2NBPN2/PP3PPP/R1BQ1RK1 w - - 0 1", - "r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 1", - "8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - 0 1", - }; - const ULONG uD1 = 4; - const ULONG uD2 = 6; - SEARCHER_THREAD_CONTEXT *ctx; - POSITION pos; - SCORE iFirst, iSecond, iShallow, iDirect; - ULONG u; - FLAG fRet = TRUE; - - Trace("Testing hash table consistency through search...\n"); - ctx = SystemAllocateMemory(sizeof(SEARCHER_THREAD_CONTEXT)); - ASSERT(ctx); - - for (u = 0; u < ARRAY_LENGTH(rgszFen); u++) - { - if (FALSE == FenToPosition(&pos, (char *)rgszFen[u])) - { - UtilPanic(INCONSISTENT_STATE, - NULL, NULL, NULL, NULL, - __FILE__, __LINE__); - } - - // Invariant 1: repeat-with-warm-table stability. - ClearHashTable(); - DirtyHashTable(); - iFirst = _FixedDepthSearchScore(ctx, &pos, uD1); - DirtyHashTable(); - iSecond = _FixedDepthSearchScore(ctx, &pos, uD1); - if ((INVALID_SCORE != iFirst) && - (INVALID_SCORE != iSecond) && - (iFirst != iSecond)) - { - Trace("TestSearchHashConsistency: position %lu: same-depth " - "rerun with warm hash table diverged (%d vs %d) -- " - "FEN: %s\n", u, iFirst, iSecond, rgszFen[u]); - fRet = FALSE; - } - - // Invariant 2: diagnostic-only now -- see this function's - // top-of-file comment for why a mismatch here is expected - // (LMR's move-order-sensitive lateness signal) and does not - // indicate a hash-table bug. Logged so a wildly outlying case - // still catches a human's eye, but never fails the suite. - ClearHashTable(); - DirtyHashTable(); - iShallow = _FixedDepthSearchScore(ctx, &pos, uD1); - DirtyHashTable(); - iSecond = _FixedDepthSearchScore(ctx, &pos, uD2); - - ClearHashTable(); - DirtyHashTable(); - iDirect = _FixedDepthSearchScore(ctx, &pos, uD2); - - if ((INVALID_SCORE != iSecond) && - (INVALID_SCORE != iDirect) && - (iSecond != iDirect)) - { - Trace("TestSearchHashConsistency: position %lu: depth-%lu " - "search seeded by a depth-%lu warm-up (%d) disagreed " - "with a from-scratch depth-%lu search (%d) -- FEN: " - "%s (diagnostic only, not a failure -- see top-of-" - "file comment)\n", u, uD2, uD1, iSecond, uD2, iDirect, - rgszFen[u]); - } - (void)iShallow; - } - - if (FALSE == fRet) - { - UtilPanic(TESTCASE_FAILURE, - NULL, "TestSearchHashConsistency", NULL, NULL, - __FILE__, __LINE__); - } - ClearHashTable(); - SystemFreeMemory(ctx); - return(fRet); -} +// 2026-10-08: TestSearchHashConsistency (both its "invariants") was +// removed. Its entire premise -- that a hash table should be provably +// *transparent* (changes performance, never the answer) and that this +// is checkable by comparing scores across searches that visit +// different subsets of the tree -- does not hold in this engine and +// cannot be patched into holding. EFP/LMR pruning decisions depend on +// GLOBAL, LIVE-ACCUMULATING evidence tables (g_HistoryCounters, +// g_ContinuationHistory, g_FailHighs) that update as a side effect of +// actually VISITING a node, not as a side effect of that node's final +// score. A hash cutoff resolves a subtree with a perfectly sound bound +// but without visiting it -- so two searches that visit different +// subsets of the same tree (which is the entire point of a hash table: +// a cold run visits everything, a warm run skips subtrees a previous +// identical run already resolved) can legitimately reach a later +// sibling move with different accumulated evidence, prune/reduce it +// differently, and return a different final score -- with zero +// hash-table bug anywhere. This was already known and documented for +// "invariant 2" (depth-D1-warmup vs depth-D2-direct, downgraded to +// diagnostic-only 2026-10-04); what was missed is that the identical +// mechanism applies just as much to "invariant 1" (same-depth +// back-to-back rerun), which was still asserted and panicking the +// self-test suite on nothing but this structural, expected effect. A +// real hash-table correctness test needs a synthetic store/probe unit +// test (see TESTING_PLAN.md's "hash correctness" section) that can +// control exactly what the table returns, not a real search compared +// against another real search -- any two of which are free to visit +// different nodes by design. // Null-move soundness is intentionally NOT tested here as a C unit |
