Redesign lazy/super-lazy eval margins with position-specific percentile

tables, and fix four real search/recognizer bugs found while validating it

Main theme: replace the flat, heavily-padded LAZY_EVAL_BASE_MARGIN /
SUPER_LAZY_MARGIN_BY_ARMY constants with per-material-bucket percentile
tables (king safety by attacker army, residual by own army, separate
tables for the lazy vs super-lazy baselines) derived from a 30k-position
calibration pool, targeting a measured ~1-1.5% exceeded-margin rate
instead of an unmeasured "add lots of headroom" floor. QSearch's
FUTILITY_BASE_MARGIN_LAZY/SUPERLAZY were re-derived to match the new,
now-meaningful (p50-based) iPositional this produces. Self-play confirms
the working tree beats both HEAD, a more liberal (~5% miss target)
variant, and a more conservative (near-zero miss) variant of the same
margin mechanism.

Along the way, an initial self-play regression (this margin work bundled
with several changes in one pass, one of which broke things) led to a
day of ZFS-snapshot bisection and then targeted DEBUG-build fuzzing
against a stratified 200k-position corpus (tests/cross_sectional_200k.ep_,
new), which is what actually found and fixed four distinct, real,
pre-existing engine bugs -- none of them related to the margin redesign
itself:

- searchsup.c (CommonSearchInit): three early-return paths (timer
  abort, draw, too-deep bailout) left a node's own PV[] row stale
  instead of clearing it, letting an ancestor's UpdatePV splice a
  leftover move from a completely unrelated node into the displayed
  PV -- root cause of a WalkPV/MakeMove assert ("moving a piece to
  capture, but nothing was on the target square").

- dynamic.c (RecordEnprisePieceAtPly): the two-slot en-prise dedup
  compared (square, piece) instead of square alone, so a same-square
  re-record with a different piece (legitimate -- a fail-high
  capture's victim is restored by UnmakeMove and can differ across
  calls at the same ply) shifted a duplicate square into both slots,
  tripping ValueOfMaterialInTroubleDespiteMove's two-distinct-squares
  assert.

- recogn.c (_RecognizeKBKP): the wrong-colored-bishop corner-fortress
  draw shortcut fired even when the weak side has its own pawn(s),
  which can hand the strong side real zugzwang resources the fortress
  logic doesn't account for -- produced a false hard draw bound where
  EGTB shows a near-forced loss.

- eval.c (_EvalPasserRacesAgainstLoneKings): a rook pawn's race was
  credited as winning even when the strong side's own king sits
  exactly on its own pawn's queening square (distance 0 specifically --
  adjacent is fine, the square is empty) -- a classic stalemate
  resource the race heuristic can't see. Only reachable through
  _RecognizeKPK's pawn-hash-cache-hit path, which is why
  testrecogn.c's TestRecognExhaustiveKPK reported "0 wrong" the whole
  time; extended it to cover both color assignments and the full board
  (no mirror-shortcut) and documented the remaining cache-hit coverage
  gap for a future session to close.

Also: match_play.py gains a local_ebf metric (geometric-mean of
depth-normalized node-growth ratios between consecutive completed
iterations) alongside the existing cumulative nodes**(1/depth) ebf,
which turned out to be misleading when comparing engines with
different lazy-exit aggressiveness -- and probe.c/hash.c pick up two
small, independently-found correctness fixes (TB root-probe promotion
piece missing its color bit; ASSERT(g_pHashTable) crashing on the
legitimate "--hash none" config) noticed along the way.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

1 files changed, 120 insertions, 0 deletions

diff --git a/src/TODO b/src/TODO
index ae02475..dce08d2 100644
--- a/src/TODO
+++ b/src/TODO
@@ -5,3 +5,123 @@ Think about futility pruning when a >= +NMATE
 Stick eval / king score in ply info and use it to trigger extensions
 
 r2r4/p4pk1/2p3p1/1p1nPR2/5p1Q/2N5/PPPq4/1K4R1 w - - 0 0 -- check extension crazy
+
+2026-09-17: search.c's extended futility pruning (VALUE_KNIGHT/VALUE_ROOK
+gate, PREPARE_TO_TRY_MOVES) now folds in iImprovement + iPositional[side]
+(the same per-tier p50 typical-value estimate QSearch's own futility gate
+uses), but still uses one flat margin regardless of which Eval() exit tier
+(PLY_INFO.uLastPositionalSource) produced iEval/iPositional at that node.
+iPositional corrects for a tier's mean bias, not its spread -- a coarse
+(super-lazy) read could carry more variance/tail risk than a fine one even
+after mean-centering. Rather than a categorical "never prune from
+super-lazy" rule (rejected as too blunt/magic-number-shaped -- iPositional
+already replaces trust-by-category with trust-by-measured-bias), the
+principled fix is a real EFP_BASE_MARGIN_BY_SOURCE[tier], calibrated the
+same way as QSearch's FUTILITY_BASE_MARGIN_BY_SOURCE (CALIBRATE_QSEARCH_
+FUTILITY-style harness: search the would-be-pruned move anyway, bucket
+surprise rate by distance-short-of-bar AND uLastPositionalSource, check
+whether the tiers actually need different margins here too). Not yet
+built -- deferred behind self-play validation of today's iPositional/
+futility-margin changes landing first.
+
+2026-09-17: self-calibrating lazy/super-lazy margins -- instead of (or on
+top of) the static per-bucket percentile tables, maintain a small mutable
+per-(tier, material-or-army-bucket) delta, adjusted at runtime: at a low
+sample rate (~1%), let a node that would have taken the cheap exit run to
+full eval anyway, compare, and nudge the delta -- up fast on a real miss
+(swing exceeded the margin), down slowly otherwise (fast-up/slow-down,
+not symmetric AIMD). Motivated by this session's CALIBRATE_MARGIN_SAFETY
+digging: exceeded-margin nodes aren't spread uniformly across the tree,
+they cluster heavily inside a handful of "spicy" root games' subtrees
+(see the recurring N1Q5/N1B5/N1R5/... and 5rk1/1b2Q2p/... FEN families
+that kept reappearing across every guess this session, all clearly the
+same search tree a few plies apart). A single fixed percentile can't
+respond to "we're currently deep in a dangerous subtree right now"; a
+fast-reacting runtime term could -- widen in response to a real miss
+right when the search is actually inside the risky region, then decay
+back out once it's past. Two real costs before building this: (1) it
+makes eval score depend on search visitation order/history, adding to
+the non-determinism MP already introduces and complicating the kind of
+fixed-margin A/B comparison work this session did throughout; (2) it's
+its own new tuning problem (step-up size, decay rate) layered on top of
+the static-table one -- don't start building this until the question of
+whether the current 30k-position calibration pool is even large/diverse
+enough (see cross_sectional_1/2.ep_ variance check, tests/) is settled,
+so it's clear which change is causing which effect. Explicitly rejected
+alternative: hand-carving structural exceptions for specific recurring
+motifs (e.g. "queen has infiltrated near the enemy king") -- doesn't
+generalize, and is exactly the kind of special-casing that rots.
+
+2026-09-18: STILL AN OPEN QUESTION, not decided either way. Scott's
+worry re-examining this: reaction speed. At a ~1-in-10k sample rate, an
+adaptive term probably can't react fast enough inside a spicy subtree to
+matter -- by the time enough samples land there to trip the widen, the
+search has likely already made every unsound cutoff in that region
+using the too-narrow margin, so the wrong bound is already baked into
+that iteration's result before the term ever moves. Raising the sample
+rate to compensate directly fights the entire reason the lazy tiers
+exist (paying full-eval cost specifically to make the cheap tiers
+marginally safer). Separately, the complexity cost compounds with
+something already proven expensive this session: guess #1-11 (eval.c)
+showed changing one bucket's *static* margin reshapes the whole search
+tree in ways that confound attribution (see those comments' "cross-
+bucket confound" notes) -- a runtime-adaptive term makes that confound
+permanent and load-bearing every game, rather than a one-time tuning
+headache, and two engines with identical DNA could diverge on eval
+*decisions* (not just timing) run to run. Current lean: shelve it, since
+the mechanism can only close the barn door after the horse for the
+specific node that triggered it, against a permanent tax on
+determinism/reasoning-about-Eval(), for a gain the static percentile
+tables already get within ~1% of via the boring approach. Not acted on
+-- left open in case reaction-speed concerns turn out to be wrong in
+practice (e.g. if "spicy subtree" actually means tens of thousands of
+nodes wide, not a handful, the sample-rate math changes).
+
+2026-09-18: build two eval.c lazy-margin variants and compare all three
+(current/baseline, "messy but fast", "super conservative") head to head.
+"Messy but fast": tune iResidualP2/98ByOwnArmy + iSuperLazyResidualP2/98-
+ByOwnArmy to accept ~5% exceeded rate against tests/cross_sectional_200k.ep_
+(search under each root position, not just eval in isolation -- want
+real search-tree exceeded rate, not a static-position read). "Super
+conservative": tune the same tables so the 200k set hardly ever misses.
+Then: (1) self-play match_play.py between baseline and each variant, and
+between the two variants directly; (2) full tests/ecm.ep_ run (not a
+curated subset) under each of the three binaries, same sd, compare
+solve counts. Interesting because it's a direct speed-vs-soundness
+tradeoff measurement on the exact margin mechanism this session spent a
+day mistakenly blaming for the 2026-09-17 regression (see recent git
+log/session notes) -- the real bug was searchsup.c's CommonSearchInit
+PV staleness, not the margins, so this is now safe to actually explore
+properly with a real experiment instead of blind percentile-guessing.
+  Cheaper derivation than 11 rounds of guess-rebuild-rerun (see eval.c's
+  guess #1-11 history comments): PersistMarginSafetyHistogramsToFile
+  (calibrate marginsafety, CALIBRATE_MARGIN_SAFETY build) persists the
+  raw per-bucket SWING distribution (real full-eval score minus the
+  lazy/super-lazy estimate), not just a boolean exceeded count -- so
+  one calibration run against the 200k pool lets a script sweep any
+  candidate margin M against the persisted histogram and read "fraction
+  of swings > M" directly, giving the M for ~5% exceeded and the M for
+  ~0% exceeded from the SAME run. eval_tune/lazy_eval_margins/
+  aggregate_marginsafety_shards.py already sums bins across shards;
+  needs a small "exceeded rate at threshold M" query added alongside
+  hist_percentile.py's percentile query. Caveat (documented repeatedly
+  in eval.c's guess history): the recorded swing distribution is itself
+  shaped by whatever margin was actually compiled in during that
+  calibration run -- a different final M reshapes the search tree,
+  which changes which nodes even reach the swing-recording code -- so
+  this gives a very good first estimate for both variants, not a
+  substitute for the real match_play/ecm validation once each variant
+  is actually built.
+
+2026-09-18: sharded ASSERT-fuzzing harness -- the technique that found
+the CommonSearchInit PV-staleness bug (DEBUG binary + gdb, batch mode,
+`script <shard>.ep_` at a shallow sd, let it run until either an
+ASSERT/SIGSEGV or clean exit) generalizes well and should be run
+routinely, not just once ad hoc. Shard tests/cross_sectional_200k.ep_
+(or any large stratified pool) into N pieces and run N concurrent
+DEBUG-build _typhoon processes under gdb, one per shard, each logging
+to its own file; investigate any hit (assert message + gdb backtrace)
+the same way this session did. Needs a full core budget (N cores) so
+it collides with self-play match runs -- schedule it for whenever the
+box isn't already saturated by match_play.py workers, not concurrently
+with them.