Redesign lazy/super-lazy eval margins with position-specific percentile

tables, and fix four real search/recognizer bugs found while validating it

Main theme: replace the flat, heavily-padded LAZY_EVAL_BASE_MARGIN /
SUPER_LAZY_MARGIN_BY_ARMY constants with per-material-bucket percentile
tables (king safety by attacker army, residual by own army, separate
tables for the lazy vs super-lazy baselines) derived from a 30k-position
calibration pool, targeting a measured ~1-1.5% exceeded-margin rate
instead of an unmeasured "add lots of headroom" floor. QSearch's
FUTILITY_BASE_MARGIN_LAZY/SUPERLAZY were re-derived to match the new,
now-meaningful (p50-based) iPositional this produces. Self-play confirms
the working tree beats both HEAD, a more liberal (~5% miss target)
variant, and a more conservative (near-zero miss) variant of the same
margin mechanism.

Along the way, an initial self-play regression (this margin work bundled
with several changes in one pass, one of which broke things) led to a
day of ZFS-snapshot bisection and then targeted DEBUG-build fuzzing
against a stratified 200k-position corpus (tests/cross_sectional_200k.ep_,
new), which is what actually found and fixed four distinct, real,
pre-existing engine bugs -- none of them related to the margin redesign
itself:

- searchsup.c (CommonSearchInit): three early-return paths (timer
  abort, draw, too-deep bailout) left a node's own PV[] row stale
  instead of clearing it, letting an ancestor's UpdatePV splice a
  leftover move from a completely unrelated node into the displayed
  PV -- root cause of a WalkPV/MakeMove assert ("moving a piece to
  capture, but nothing was on the target square").

- dynamic.c (RecordEnprisePieceAtPly): the two-slot en-prise dedup
  compared (square, piece) instead of square alone, so a same-square
  re-record with a different piece (legitimate -- a fail-high
  capture's victim is restored by UnmakeMove and can differ across
  calls at the same ply) shifted a duplicate square into both slots,
  tripping ValueOfMaterialInTroubleDespiteMove's two-distinct-squares
  assert.

- recogn.c (_RecognizeKBKP): the wrong-colored-bishop corner-fortress
  draw shortcut fired even when the weak side has its own pawn(s),
  which can hand the strong side real zugzwang resources the fortress
  logic doesn't account for -- produced a false hard draw bound where
  EGTB shows a near-forced loss.

- eval.c (_EvalPasserRacesAgainstLoneKings): a rook pawn's race was
  credited as winning even when the strong side's own king sits
  exactly on its own pawn's queening square (distance 0 specifically --
  adjacent is fine, the square is empty) -- a classic stalemate
  resource the race heuristic can't see. Only reachable through
  _RecognizeKPK's pawn-hash-cache-hit path, which is why
  testrecogn.c's TestRecognExhaustiveKPK reported "0 wrong" the whole
  time; extended it to cover both color assignments and the full board
  (no mirror-shortcut) and documented the remaining cache-hit coverage
  gap for a future session to close.

Also: match_play.py gains a local_ebf metric (geometric-mean of
depth-normalized node-growth ratios between consecutive completed
iterations) alongside the existing cumulative nodes**(1/depth) ebf,
which turned out to be misleading when comparing engines with
different lazy-exit aggressiveness -- and probe.c/hash.c pick up two
small, independently-found correctness fixes (TB root-probe promotion
piece missing its color bit; ASSERT(g_pHashTable) crashing on the
legitimate "--hash none" config) noticed along the way.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

1 files changed, 120 insertions, 48 deletions

diff --git a/src/testrecogn.c b/src/testrecogn.c
index 7bee3ad..d320394 100644
--- a/src/testrecogn.c
+++ b/src/testrecogn.c
@@ -625,17 +625,33 @@ Return value:
 
 //
 // Build a KPK (one pawn, one lone king) position directly from square
-// placements -- white always plays the pawn side, black always plays
-// the lone king (color-symmetric by construction: RecognLookup/
-// _RecognizeKPK determine "strong side" dynamically from piece
-// counts, not from hardcoded color, so fixing white=pawn loses no
-// coverage and halves the enumeration, same reasoning as
-// _BuildKNKPPosition above).
+// placements. fBlackHasPawn selects which color plays the pawn side
+// (cStrongKing/cWeakKing name roles, not colors, in both cases).
+//
+// 2026-09-18: the "color-symmetric by construction" claim this
+// function's comment used to make -- that fixing white=pawn always
+// and never testing black=pawn loses no coverage, since RecognLookup/
+// _RecognizeKPK determine "strong side" dynamically from piece counts
+// -- turned out to be FALSE. Found via fuzzing tests/cross_
+// sectional_200k.ep_: 8/8/8/8/8/8/4K2p/7k w - - 0 0 (White Ke2 lone
+// king, Black Kh1+Ph2, White to move) hit a RECOGNIZER_SANITY_FAIL --
+// recognizer claimed Black winning, EGTB says draw (White can
+// stalemate Black rather than let the pawn queen). This exhaustive
+// test's original white-only enumeration reported 0 wrong across
+// 165,676 EGTB-covered positions, which looked like it cleared
+// _RecognizeKPK's post-2026-09-16 tempo-margin fix -- but it never
+// actually exercised the black-has-pawn code path (specifically
+// _GetPassersCriticalSquares' separate per-color lookup table) at
+// all, so that "0 wrong" was never evidence either way for this
+// failure mode. fBlackHasPawn added so the caller can enumerate both
+// color assignments explicitly, closing the gap instead of assuming
+// symmetry.
 //
 static FLAG
 _BuildKPKPosition(OUT POSITION *pos,
                   IN COOR cStrongKing, IN COOR cWeakKing,
-                  IN COOR cPawn, IN ULONG uToMove)
+                  IN COOR cPawn, IN ULONG uToMove,
+                  IN FLAG fBlackHasPawn)
 {
     PIECE pBoard[128];
     CHAR szFen[256];
@@ -643,9 +659,18 @@ _BuildKPKPosition(OUT POSITION *pos,
     ULONG x;
 
     memset(pBoard, 0, sizeof(pBoard));
-    pBoard[cStrongKing] = WHITE_KING;
-    pBoard[cWeakKing] = BLACK_KING;
-    pBoard[cPawn] = WHITE_PAWN;
+    if (fBlackHasPawn)
+    {
+        pBoard[cStrongKing] = BLACK_KING;
+        pBoard[cWeakKing] = WHITE_KING;
+        pBoard[cPawn] = BLACK_PAWN;
+    }
+    else
+    {
+        pBoard[cStrongKing] = WHITE_KING;
+        pBoard[cWeakKing] = BLACK_KING;
+        pBoard[cPawn] = WHITE_PAWN;
+    }
 
     memset(szFen, 0, sizeof(szFen));
     p = szFen;
@@ -711,10 +736,39 @@ Routine description:
     when actually validating a KPK-area fix, the same way
     TestRecognExhaustiveKNKP is used for KNKP.
 
-    Fixes white=pawn/black=lone-king by construction (lossless -- see
-    _BuildKPKPosition) and uses left/right board mirror symmetry on
-    the strong king's start square to roughly halve the raw
-    combination count.
+    2026-09-18: the "fixes white=pawn/black=lone-king by construction,
+    lossless" and "left/right mirror symmetry, roughly halves the
+    count" claims this comment used to make were BOTH wrong -- found
+    via fuzzing tests/cross_sectional_200k.ep_, which turned up a real
+    _EvalPasserRacesAgainstLoneKings bug (8/8/8/8/8/8/4K2p/7k w - -,
+    strong side's own king on its own rook pawn's queening square) that
+    this test reported "0 wrong" on even after removing both
+    shortcuts. Now iterates BOTH color assignments explicitly (see
+    _BuildKPKPosition's fBlackHasPawn) over the FULL board (no
+    cStrongKing file restriction) -- ~4x the raw combos, still fast
+    enough for on-demand use.
+
+    IMPORTANT REMAINING GAP, not yet closed: this still cannot exercise
+    _RecognizeKPK's pawn-hash-cache-hit branch (the `pHash->u64Key ==
+    pos->u64PawnSig` check gating _EvalPasserRacesAgainstLoneKings).
+    The pawn hash signature depends only on pawn placement, not king
+    squares -- a freshly-built, isolated test position essentially
+    never has a matching warm cache entry (ReInitializeSearcherContext
+    doesn't pre-populate it), so this test structurally falls through
+    to the critical-square path every time and never really exercises
+    the cache-hit path a long, real search naturally warms into (many
+    king placements sharing one pawn skeleton over a deep search tree).
+    This is *why* three separate _EvalPasserRacesAgainstLoneKings bugs
+    (this session's stalemate-corner case, the earlier connected-
+    passers case, and the originally-committed bishop-pawn corner case)
+    all made it past a "0 wrong" run of this test at some point -- a
+    clean run here is NOT proof that function is sound. Confirmed by
+    hand for the 2026-09-18 fix: forcing a warm cache via an explicit
+    `eval` command before `go` on the repro position reproduced the bug
+    pre-fix and confirmed the fix post-fix, but that verification step
+    isn't automated here yet. Worth building: warm the pawn hash (e.g.
+    call Eval() once per constructed position before RecognLookup) so
+    this test can actually reach the branch it's nominally checking.
 
 Parameters:
 
@@ -747,7 +801,14 @@ Return value:
     FOREACH_SQUARE(cStrongKing)
     {
         if (!IS_ON_BOARD(cStrongKing)) continue;
-        if (FILE(cStrongKing) > D) continue;
+        // 2026-09-18: mirror-halving restriction (FILE(cStrongKing) >
+        // D) removed -- see _BuildKPKPosition's comment. The critical-
+        // square lookup table this test is meant to validate is raw
+        // hand-typed data, not derived by formula, so there's no
+        // guarantee its A-file and H-file entries are true mirrors of
+        // each other -- exactly the kind of transcription gap this
+        // restriction could hide. Slower (full board instead of half)
+        // but unambiguous.
 
         FOREACH_SQUARE(cWeakKing)
         {
@@ -762,41 +823,52 @@ Return value:
 
                 for (uToMove = BLACK; uToMove <= WHITE; uToMove++)
                 {
-                    uTotal++;
-                    if (0 == (uTotal % 100000))
-                    {
-                        Trace("  ... %lu raw combos so far (%lu checked, "
-                              "%lu wrong)\n", uTotal, uChecked, uWrong);
-                    }
-                    if (FALSE == _BuildKPKPosition(&pos, cStrongKing,
-                                                    cWeakKing, cPawn,
-                                                    uToMove))
+                    ULONG uColorPass;
+                    // 2026-09-18: loop both color assignments -- see
+                    // _BuildKPKPosition's comment. uColorPass 0 =
+                    // white has the pawn (original coverage), 1 =
+                    // black has the pawn (the gap that let the
+                    // fuzzer-found counterexample through undetected).
+                    for (uColorPass = 0; uColorPass < 2; uColorPass++)
                     {
-                        continue;
-                    }
-                    ReInitializeSearcherContext(&pos, ctx);
-                    uVal = RecognLookup(ctx, &iScore, FALSE);
-                    if (UNRECOGNIZED == uVal)
-                    {
-                        continue;
-                    }
-                    if (FALSE == ProbeEGTB(ctx, &iEgtb))
-                    {
-                        uSkipped++;
-                        continue;
-                    }
-                    uChecked++;
-                    if (FALSE == _RecognizerAgreesWithEGTB(iScore, uVal,
-                                                            iEgtb))
-                    {
-                        uWrong++;
-                        if (uReportedWrong < 10)
+                        uTotal++;
+                        if (0 == (uTotal % 100000))
+                        {
+                            Trace("  ... %lu raw combos so far (%lu checked, "
+                                  "%lu wrong)\n", uTotal, uChecked, uWrong);
+                        }
+                        if (FALSE == _BuildKPKPosition(&pos, cStrongKing,
+                                                        cWeakKing, cPawn,
+                                                        uToMove,
+                                                        (1 == uColorPass)))
                         {
-                            Trace("MISMATCH iScore=%d uVal=%lu "
-                                  "iEgtb=%d toMove=%lu\n",
-                                  iScore, uVal, iEgtb, pos.uToMove);
-                            DumpPosition(&pos);
-                            uReportedWrong++;
+                            continue;
+                        }
+                        ReInitializeSearcherContext(&pos, ctx);
+                        uVal = RecognLookup(ctx, &iScore, FALSE);
+                        if (UNRECOGNIZED == uVal)
+                        {
+                            continue;
+                        }
+                        if (FALSE == ProbeEGTB(ctx, &iEgtb))
+                        {
+                            uSkipped++;
+                            continue;
+                        }
+                        uChecked++;
+                        if (FALSE == _RecognizerAgreesWithEGTB(iScore, uVal,
+                                                                iEgtb))
+                        {
+                            uWrong++;
+                            if (uReportedWrong < 20)
+                            {
+                                Trace("MISMATCH (blackHasPawn=%lu) iScore=%d "
+                                      "uVal=%lu iEgtb=%d toMove=%lu\n",
+                                      uColorPass, iScore, uVal, iEgtb,
+                                      pos.uToMove);
+                                DumpPosition(&pos);
+                                uReportedWrong++;
+                            }
                         }
                     }
                 }