Redesign lazy/super-lazy eval margins with position-specific percentile

tables, and fix four real search/recognizer bugs found while validating it

Main theme: replace the flat, heavily-padded LAZY_EVAL_BASE_MARGIN /
SUPER_LAZY_MARGIN_BY_ARMY constants with per-material-bucket percentile
tables (king safety by attacker army, residual by own army, separate
tables for the lazy vs super-lazy baselines) derived from a 30k-position
calibration pool, targeting a measured ~1-1.5% exceeded-margin rate
instead of an unmeasured "add lots of headroom" floor. QSearch's
FUTILITY_BASE_MARGIN_LAZY/SUPERLAZY were re-derived to match the new,
now-meaningful (p50-based) iPositional this produces. Self-play confirms
the working tree beats both HEAD, a more liberal (~5% miss target)
variant, and a more conservative (near-zero miss) variant of the same
margin mechanism.

Along the way, an initial self-play regression (this margin work bundled
with several changes in one pass, one of which broke things) led to a
day of ZFS-snapshot bisection and then targeted DEBUG-build fuzzing
against a stratified 200k-position corpus (tests/cross_sectional_200k.ep_,
new), which is what actually found and fixed four distinct, real,
pre-existing engine bugs -- none of them related to the margin redesign
itself:

- searchsup.c (CommonSearchInit): three early-return paths (timer
  abort, draw, too-deep bailout) left a node's own PV[] row stale
  instead of clearing it, letting an ancestor's UpdatePV splice a
  leftover move from a completely unrelated node into the displayed
  PV -- root cause of a WalkPV/MakeMove assert ("moving a piece to
  capture, but nothing was on the target square").

- dynamic.c (RecordEnprisePieceAtPly): the two-slot en-prise dedup
  compared (square, piece) instead of square alone, so a same-square
  re-record with a different piece (legitimate -- a fail-high
  capture's victim is restored by UnmakeMove and can differ across
  calls at the same ply) shifted a duplicate square into both slots,
  tripping ValueOfMaterialInTroubleDespiteMove's two-distinct-squares
  assert.

- recogn.c (_RecognizeKBKP): the wrong-colored-bishop corner-fortress
  draw shortcut fired even when the weak side has its own pawn(s),
  which can hand the strong side real zugzwang resources the fortress
  logic doesn't account for -- produced a false hard draw bound where
  EGTB shows a near-forced loss.

- eval.c (_EvalPasserRacesAgainstLoneKings): a rook pawn's race was
  credited as winning even when the strong side's own king sits
  exactly on its own pawn's queening square (distance 0 specifically --
  adjacent is fine, the square is empty) -- a classic stalemate
  resource the race heuristic can't see. Only reachable through
  _RecognizeKPK's pawn-hash-cache-hit path, which is why
  testrecogn.c's TestRecognExhaustiveKPK reported "0 wrong" the whole
  time; extended it to cover both color assignments and the full board
  (no mirror-shortcut) and documented the remaining cache-hit coverage
  gap for a future session to close.

Also: match_play.py gains a local_ebf metric (geometric-mean of
depth-normalized node-growth ratios between consecutive completed
iterations) alongside the existing cumulative nodes**(1/depth) ebf,
which turned out to be misleading when comparing engines with
different lazy-exit aggressiveness -- and probe.c/hash.c pick up two
small, independently-found correctness fixes (TB root-probe promotion
piece missing its color bit; ASSERT(g_pHashTable) crashing on the
legitimate "--hash none" config) noticed along the way.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

1 files changed, 194 insertions, 0 deletions

diff --git a/src/eval_tune/lazy_eval_margins/README.md b/src/eval_tune/lazy_eval_margins/README.md
new file mode 100644
index 0000000..239bb26
--- /dev/null
+++ b/src/eval_tune/lazy_eval_margins/README.md
@@ -0,0 +1,194 @@
+# Lazy/super-lazy eval margin redesign (2026-09-17)
+
+## What changed and why
+
+Original goal: speed up `Eval()` (eval.c) by letting more nodes exit
+through the cheap "super lazy" (material-only, runs before `_EvalPawns`)
+and "lazy" (material + pawns + passers + bad-trade + bishop-pairs) tiers
+instead of paying for full eval, motivated by comparing against Crafty's
+much higher NPS.
+
+The old margin construction (still visible via `git show HEAD:src/eval.c`
+at commit `4a7ec9a`) was a patchwork: a flat `LAZY_EVAL_BASE_MARGIN` (75)
+seed, widened by a `CountKingSafetyDefects`-indexed king term (`_EvalKing`'s
+own slow bitboard scan, called *again* just to estimate a margin) plus a
+flat residual constant, then `MAX()`'d against a separately-measured
+flat floor per material bucket. Super-lazy was a single flat
+`SUPER_LAZY_MARGIN_BY_ARMY[8]` lookup, no per-position information at all.
+Both had real problems: `CountKingSafetyDefects` is slow despite being
+bitboard-based (defeats the point of being in the *cheap* path), and the
+`MAX()`-against-a-flat-floor pattern silently overrode the
+position-specific estimate in most cases (found and fixed twice this
+session, once per tier).
+
+## The new construction
+
+For **both** tiers, at every node, cheap **O(1) table lookups only** (no
+`CountKingSafetyDefects`, no attack-table generation):
+
+```
+margin_a = MAX0( (king_p_hi[my_bucket]  - king_p_lo[opp_bucket])
+                + (resid_p_hi[opp_bucket] - resid_p_lo[my_bucket]) )
+margin_b = MAX0( (king_p_hi[opp_bucket] - king_p_lo[my_bucket])
+                + (resid_p_hi[my_bucket]  - resid_p_lo[opp_bucket]) )
+```
+
+- `margin_a` gates the fail-high check (`score - margin_a > beta`):
+  soundness requires assuming the most pessimistic-for-me,
+  optimistic-for-opponent outcome for each term.
+- `margin_b` gates the fail-low check (`score + margin_b < alpha`):
+  the mirror image.
+- **King term**: bucketed by the **attacker's** own army scaler
+  (`MINU(7, armyScaler/4)`, 8 buckets, 0-31 scaler range) -- danger to a
+  king is a function of the attacker's material, not the king-owner's.
+  Shared between both tiers (`iKingP2ByAttackerArmyOnly` /
+  `iKingP98ByAttackerArmyOnly` in eval.c).
+- **Residual term** (mobility/trapped/passers-not-yet-counted/etc.):
+  bucketed by **each side's own** army scaler. Two separate tables
+  because the two tiers have different baselines to estimate *from*:
+  - `iResidualP2/98ByOwnArmy` -- regular lazy's baseline already
+    includes pawns/passers/bad-trade/bishop-pairs, so this residual
+    only needs to cover what's left (mobility, trapped pieces, etc).
+  - `iSuperLazyResidualP2/98ByOwnArmy` -- super-lazy's baseline is pure
+    material, so this residual is much wider (includes everything
+    regular lazy's own residual excludes).
+
+`_hi`/`_lo` are currently p98/p2 as shipped, but genuinely **per-bucket
+tunable** -- see "How to keep tuning" below. The king table has very
+little spread above p90 in every bucket (already checked against the
+full histogram) and was deliberately left alone throughout tuning;
+all of today's percentile-adjustment work only ever touched the two
+residual tables.
+
+`iPositional[side]` (the value fed back to QSearch's futility pruning,
+`search.c`'s `FUTILITY_BASE_MARGIN_BY_SOURCE[tier] + iPositional[side]`)
+is now a **separate**, deliberately different statistic: a **p50**
+(median, not tail-bound) typical-value estimate of that side's own
+missing terms, matching what full-eval's own `iPositional` actually is
+(a real point estimate, not a safety margin). See `iKingP50By...` /
+`iResidualP50By...` / `iSuperLazyResidualP50By...` in eval.c. Don't
+confuse these with the p98/p2 margin tables -- different statistic,
+different purpose (bias correction vs. soundness bound).
+
+## Where the histogram data lives
+
+`eval.c`'s `_PersistRawHistogramsToFile()` (called automatically every
+time a `DUMP_RESIDUAL_SAMPLES` build runs `calibrate residual`) writes
+**every raw bin** of all three histograms (residual, sl_residual, king)
+to `/var/tmp/typhoon/residual_raw_histograms.txt` -- not just the
+handful of percentile columns that happen to be printed to the console.
+This directory (`src/eval_tune/lazy_eval_margins/`) has copies from the
+2026-09-17 "guess #2" 30k-position calibration run
+(`residual_raw_histograms_guess2_30k.txt`) plus the reader script
+(`hist_percentile.py`) -- **copy the fresh file here after any future
+30k run you want to keep**, since `/var/tmp` is scratch and could be
+cleared.
+
+To read *any* percentile out of a saved histogram, without re-running
+anything:
+
+```sh
+python3 hist_percentile.py residual_raw_histograms_guess2_30k.txt          # default spread
+python3 hist_percentile.py residual_raw_histograms_guess2_30k.txt 0.93 0.965  # exact percentiles you want
+```
+
+## How to keep tuning
+
+1. **Build**: `gmake clean && gmake -j5 GENETIC=1 PERF_COUNTERS=1 MP=1 SIXTYFOUR=1 DUMP_RESIDUAL_SAMPLES=1 CALIBRATE_MARGIN_SAFETY=1`
+2. **Run the 30k stratified pool** (both commands in one invocation --
+   `calibrate residual` persists the histogram file, `calibrate
+   marginsafety` prints the real exceeded rate per **material** bucket
+   for both tiers):
+   ```sh
+   ./typhoon --cpus 1 --hash 256m --logfile /var/tmp/typhoon/run.log \
+     --batch --command "force; book name /nonexistent.book.bin; sd 6; \
+     script ../tests/eval_calibration_sample_strat30k.ep_; \
+     calibrate residual; calibrate marginsafety" \
+     > /var/tmp/typhoon/run.stdout 2>&1
+   ```
+   Takes roughly 20-25 minutes. Run it in the background
+   (`run_in_background`/`&`), don't block on it.
+3. **Read the exceeded rate per material bucket** from the
+   `calibrate marginsafety` output (`exceeded / n`, printed separately
+   for "Regular lazy" and "Super lazy", by **material** bucket --
+   combined army scaler / 8, a *different* bucketing than the
+   percentile tables' per-side scaler/4 buckets -- see the caveat
+   below).
+4. **Copy the fresh `/var/tmp/typhoon/residual_raw_histograms.txt`**
+   into this directory before it can get lost.
+5. **Pick new percentiles per per-side bucket** using
+   `hist_percentile.py` against the fresh file, and edit
+   `iResidualP2/98ByOwnArmy` / `iSuperLazyResidualP2/98ByOwnArmy` in
+   eval.c directly (keep each bucket's low/high pair **symmetric** --
+   e.g. p95/p5 together, never p98 mixed with p10 in the same bucket).
+6. **Rebuild and re-run step 2** to verify the real effect.
+
+### The two gotchas that cost real time this session
+
+- **The 500-position "quick check" sample is unreliable for this
+  specific measurement.** It's fine for cheap direction-of-travel
+  checks on most things, but exceeded-rate is a rare-event percentage,
+  and 500 positions (not even 500 independent draws -- a single game's
+  search tree can dominate a material bucket's node count) can be off
+  by 2-5x from the true 30k-pool rate. One bucket (super-lazy material
+  bucket 1) got tightened based on a 500-position reading of 0.035%,
+  then the real 30k number came back at 1.13% -- a dangerous-direction
+  miss. **Always confirm with the full 30k pool before trusting a
+  number**, especially anything under ~1%.
+- **Changing one bucket's margin changes measured behavior in OTHER
+  buckets too**, even ones whose own table entries never moved. This
+  isn't a bug -- `Eval()`'s exit decisions are real search behavior, so
+  widening/narrowing any margin reshapes the actual search tree
+  (different alpha/beta bounds propagate differently), which changes
+  which exact nodes land in *every* material bucket, not just the one
+  you touched. Material bucket 3's exceeded rate moved from 0.21% to
+  0.94% to 0.65% to 1.44% across different runs despite its own table
+  entries never changing once. Don't chase a bucket that "moved" if you
+  didn't touch its inputs -- check what else changed first.
+- **Material buckets and per-side buckets are different bucketing
+  schemes.** `calibrate marginsafety` reports by **material bucket**
+  (`_MaterialBucket`: combined army scaler for both sides, /8). The
+  percentile tables you actually edit are indexed by **per-side**
+  bucket (`armyScaler/4`, one bucket per side, looked up twice -- once
+  as "my bucket", once as "opponent's bucket"). A single material
+  bucket (e.g. combined scaler 24-31) is populated by many different
+  per-side bucket combinations (3+4, 2+5, 1+6, 0+7...), so there's no
+  clean 1:1 way to "fix material bucket 3" by editing "per-side bucket
+  3" alone -- expect diffuse, imprecise control, not a dial.
+
+## On the "insane" 2% idea
+
+Structurally straightforward: pick a looser matched percentile pair
+(e.g. p90/p10, or go outside the currently-recorded columns using
+`hist_percentile.py` against a fresh histogram) for whichever buckets
+you want at ~2%, following the same edit-rebuild-reverify loop above.
+Two things worth checking once you have it:
+
+1. **NPS gain isn't guaranteed to scale linearly with the exceeded-rate
+   relaxation** -- this session found the NPS win from the whole
+   redesign was much smaller than expected (see the `git log`/session
+   history around `EvalLookForDrawishSituations` getting added to both
+   cheap tiers, and `fBareKingEitherSide`'s two unconditional
+   `TotalNonPawnCount` calls on every node -- both are new fixed costs
+   that eat into any margin-driven exit-rate gain). Worth re-checking
+   the `Eval exit breakdown` / new `calibrate exitrate` numbers
+   (below) alongside NPS, not just NPS alone.
+2. **`FUTILITY_BASE_MARGIN_LAZY`/`SUPERLAZY`** (search.c/chess.h) were
+   calibrated against *this* `iPositional` distribution
+   (`CALIBRATE_QSEARCH_FUTILITY`, see chess.h's derivation comment near
+   those two `#define`s) -- if the eval-exit margins move a lot, especially
+   in a direction that changes how often/how confidently each tier
+   exits, it's worth re-running that calibration too rather than
+   assuming the existing 350/175 values still hold.
+
+## New tooling added this session (always available, no build flag)
+
+- **`calibrate exitrate`** -- prints eval exit rate (% super lazy / %
+  lazy / % full) **by material bucket**, not just the flat aggregate
+  "Eval exit breakdown" line every search already prints. Complements
+  the exceeded-rate data: a bucket with low exit rate AND low exceeded
+  rate has real untapped speed headroom; a bucket already near its
+  target exceeded rate with a healthy exit rate is well-calibrated.
+  Implemented in eval.c (`g_u64SuperLazyEvalsByBucket` /
+  `g_u64LazyEvalsByBucket` / `g_u64FullEvalsByBucket`,
+  `DumpEvalExitRateByBucket`) and wired into command.c.