Redesign lazy/super-lazy eval margins with position-specific percentile

tables, and fix four real search/recognizer bugs found while validating it

Main theme: replace the flat, heavily-padded LAZY_EVAL_BASE_MARGIN /
SUPER_LAZY_MARGIN_BY_ARMY constants with per-material-bucket percentile
tables (king safety by attacker army, residual by own army, separate
tables for the lazy vs super-lazy baselines) derived from a 30k-position
calibration pool, targeting a measured ~1-1.5% exceeded-margin rate
instead of an unmeasured "add lots of headroom" floor. QSearch's
FUTILITY_BASE_MARGIN_LAZY/SUPERLAZY were re-derived to match the new,
now-meaningful (p50-based) iPositional this produces. Self-play confirms
the working tree beats both HEAD, a more liberal (~5% miss target)
variant, and a more conservative (near-zero miss) variant of the same
margin mechanism.

Along the way, an initial self-play regression (this margin work bundled
with several changes in one pass, one of which broke things) led to a
day of ZFS-snapshot bisection and then targeted DEBUG-build fuzzing
against a stratified 200k-position corpus (tests/cross_sectional_200k.ep_,
new), which is what actually found and fixed four distinct, real,
pre-existing engine bugs -- none of them related to the margin redesign
itself:

- searchsup.c (CommonSearchInit): three early-return paths (timer
  abort, draw, too-deep bailout) left a node's own PV[] row stale
  instead of clearing it, letting an ancestor's UpdatePV splice a
  leftover move from a completely unrelated node into the displayed
  PV -- root cause of a WalkPV/MakeMove assert ("moving a piece to
  capture, but nothing was on the target square").

- dynamic.c (RecordEnprisePieceAtPly): the two-slot en-prise dedup
  compared (square, piece) instead of square alone, so a same-square
  re-record with a different piece (legitimate -- a fail-high
  capture's victim is restored by UnmakeMove and can differ across
  calls at the same ply) shifted a duplicate square into both slots,
  tripping ValueOfMaterialInTroubleDespiteMove's two-distinct-squares
  assert.

- recogn.c (_RecognizeKBKP): the wrong-colored-bishop corner-fortress
  draw shortcut fired even when the weak side has its own pawn(s),
  which can hand the strong side real zugzwang resources the fortress
  logic doesn't account for -- produced a false hard draw bound where
  EGTB shows a near-forced loss.

- eval.c (_EvalPasserRacesAgainstLoneKings): a rook pawn's race was
  credited as winning even when the strong side's own king sits
  exactly on its own pawn's queening square (distance 0 specifically --
  adjacent is fine, the square is empty) -- a classic stalemate
  resource the race heuristic can't see. Only reachable through
  _RecognizeKPK's pawn-hash-cache-hit path, which is why
  testrecogn.c's TestRecognExhaustiveKPK reported "0 wrong" the whole
  time; extended it to cover both color assignments and the full board
  (no mirror-shortcut) and documented the remaining cache-hit coverage
  gap for a future session to close.

Also: match_play.py gains a local_ebf metric (geometric-mean of
depth-normalized node-growth ratios between consecutive completed
iterations) alongside the existing cumulative nodes**(1/depth) ebf,
which turned out to be misleading when comparing engines with
different lazy-exit aggressiveness -- and probe.c/hash.c pick up two
small, independently-found correctness fixes (TB root-probe promotion
piece missing its color bit; ASSERT(g_pHashTable) crashing on the
legitimate "--hash none" config) noticed along the way.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

1 files changed, 67 insertions, 6 deletions

diff --git a/src/chess.h b/src/chess.h
index 175ad5d..e585506 100755
--- a/src/chess.h
+++ b/src/chess.h
@@ -1102,9 +1102,13 @@ typedef struct _COUNTERS
         UINT64 u64CyclesEvalSuperLazy;
 
         // 2026-09-09: EvalLookForDrawishSituations runs after
-        // u64CyclesEvalPostLazyMisc's window closes, only on the
-        // full-eval exit -- previously fell into "unaccounted" in
-        // root.c's eval breakdown report.
+        // u64CyclesEvalPostLazyMisc's window closes -- previously fell
+        // into "unaccounted" in root.c's eval breakdown report.
+        // 2026-09-17: no longer full-eval-only -- also called from the
+        // regular-lazy pre-exit check (eval.c), so this counter now
+        // accumulates both call sites' cost. The two calls' real
+        // per-position-only work (DRAWISH_FACTS) only happens once
+        // per Eval() call regardless -- see _EnsureDrawishFactsComputed.
         UINT64 u64CyclesEvalDrawish;
 
         // Per-exit-path *total call cost*, i.e. the full entry-to-exit
@@ -1303,6 +1307,18 @@ typedef struct _PLY_INFO
     SCORE iBeta;
 #endif
     SCORE iEval;
+    // 2026-09-17: per-color iPositional from the SAME Eval() call that
+    // set iEval above -- previously discarded (Search() called Eval()
+    // with piPositional==NULL). Stored per-ply, right next to iEval,
+    // so any code walking the ply stack (up or down) can read both the
+    // score and its positional-bias estimate for a given node together
+    // -- e.g. a cleaner cross-ply "improving" signal that corrects for
+    // a tier switch (super-lazy vs full) between the two Eval() calls
+    // being compared, instead of just diffing raw iEval values. Not
+    // yet consumed that way anywhere; currently only the search.c EFP
+    // gate (VALUE_KNIGHT/VALUE_ROOK margins) reads this, for the
+    // node-it-was-computed-at only.
+    SCORE iPositional[2];
     INT iExtensionAmount;
     FLAG fInCheck;
     FLAG fInQsearch;
@@ -2901,6 +2917,9 @@ InsertHeadList(IN DLIST_ENTRY *pListHead,
 void
 ParseUserInput(FLAG fSearching);
 
+void
+ExecuteCommandLineDirectly(CHAR *szLine, POSITION *pos);
+
 FLAG
 InitializeCommandSystem(void);
 
@@ -3180,8 +3199,39 @@ IsDraw(SEARCHER_THREAD_CONTEXT *ctx);
 // exit -- can reference the matching tier's constant directly instead
 // of a stale flat value.
 #define FUTILITY_BASE_MARGIN_FULL      (450)
-#define FUTILITY_BASE_MARGIN_LAZY      (225)
-#define FUTILITY_BASE_MARGIN_SUPERLAZY (275)
+// 2026-09-17: LAZY raised 225->350, SUPERLAZY lowered 275->175, after
+// eval.c's lazy/super-lazy iPositional stopped being flat 0 and became
+// a real, signed p50-based typical-value estimate (see eval.c's
+// iKingP50ByAttackerArmyOnly/iResidualP50ByOwnArmy). Re-measured via
+// `calibrate qsearchfutility` (CALIBRATE_QSEARCH_FUTILITY build,
+// tests/twic_sample.ep_, 1500 real-game positions, sd 6) at the OLD
+// margins first, to see what the iPositional change alone had done:
+// generic capture/promo gate, surprise rate (wrongly-rejected-move
+// rate) by distance short of the futility bar, summed over piPositional
+// buckets:
+//   source        margin(old)  dist0-25  25-50  50-100 100-200 200-400  400+     n
+//   full          450 (n/a)      1.91%   1.72%   1.80%   1.30%   0.46%  0.11%  135,031
+//   regular lazy  225            6.22%   2.89%   2.07%   1.80%   1.58%  0.16%  1,024,889
+//   super lazy    275            1.23%   0.98%   0.90%   0.60%   0.24%  0.12%  1,813,965
+// Regular lazy got MORE dangerous than its original 2026-09-08
+// derivation (was 1.26% max at dist0-25; now 6.22%) -- the cross-tab
+// by piPositional bucket at dist0-25 showed why: piPositional<0 alone
+// carries an 8.9% surprise rate (n=66,040) vs. 1.9%/1.4% for the >=0
+// buckets -- piPositional is genuinely predictive (validates the p50
+// approach) but 225 wasn't wide enough for the negative population.
+// Super lazy got safer (was 7.96% at dist0-25; now 1.23%), real
+// headroom to tighten for speed. New values chosen by reading where
+// each tier's current worst bucket would land after the shift (not
+// guessed): LAZY +125 moves today's risky dist0-25 population into
+// territory today's data shows is ~1.8% (matching full-eval's own risk
+// level); SUPERLAZY -100 moves today's safe dist100-200 bucket (0.60%)
+// into the new near-boundary zone, still comfortably safe. FULL is
+// untouched -- its iPositional computation (full-eval exit) wasn't
+// changed this session, and its numbers above are consistent with the
+// original 2026-09-08 derivation. Re-derive with `calibrate
+// qsearchfutility` if iPositional's construction changes again.
+#define FUTILITY_BASE_MARGIN_LAZY      (350)
+#define FUTILITY_BASE_MARGIN_SUPERLAZY (175)
 
 // Measured: disabling this entirely (see lmr_testing/RESULTS.md) is a
 // clear net loss across ringers/confident_quick/hard_quick, so IID itself
@@ -3570,9 +3620,20 @@ DumpBaseMarginCalibration(void);
 
 #ifdef CALIBRATE_MARGIN_SAFETY
 void
-DumpMarginSafetyCalibration(void);
+DumpMarginSafetyCalibration(CHAR *szHistogramPathOverride);
 #endif
 
+#ifdef DUMP_RESIDUAL_SAMPLES
+void
+DumpResidualAndKingSafetyHistograms(void);
+#endif
+
+// Always-on (not gated behind a CALIBRATE_* build flag) -- see
+// eval.c's g_u64SuperLazyEvalsByBucket/g_u64LazyEvalsByBucket/
+// g_u64FullEvalsByBucket and the five INC-site call pairs in Eval().
+void
+DumpEvalExitRateByBucket(void);
+
 #ifdef CALIBRATE_QSEARCH_FUTILITY
 // Which futility gate in _ShouldWeConsiderThisMove rejected a move --
 // see search.c's _QFutRecordSample. The recapture gate (used to be