# Lazy/super-lazy eval margin redesign (2026-09-17) ## What changed and why Original goal: speed up `Eval()` (eval.c) by letting more nodes exit through the cheap "super lazy" (material-only, runs before `_EvalPawns`) and "lazy" (material + pawns + passers + bad-trade + bishop-pairs) tiers instead of paying for full eval, motivated by comparing against Crafty's much higher NPS. The old margin construction (still visible via `git show HEAD:src/eval.c` at commit `4a7ec9a`) was a patchwork: a flat `LAZY_EVAL_BASE_MARGIN` (75) seed, widened by a `CountKingSafetyDefects`-indexed king term (`_EvalKing`'s own slow bitboard scan, called *again* just to estimate a margin) plus a flat residual constant, then `MAX()`'d against a separately-measured flat floor per material bucket. Super-lazy was a single flat `SUPER_LAZY_MARGIN_BY_ARMY[8]` lookup, no per-position information at all. Both had real problems: `CountKingSafetyDefects` is slow despite being bitboard-based (defeats the point of being in the *cheap* path), and the `MAX()`-against-a-flat-floor pattern silently overrode the position-specific estimate in most cases (found and fixed twice this session, once per tier). ## The new construction For **both** tiers, at every node, cheap **O(1) table lookups only** (no `CountKingSafetyDefects`, no attack-table generation): ``` margin_a = MAX0( (king_p_hi[my_bucket] - king_p_lo[opp_bucket]) + (resid_p_hi[opp_bucket] - resid_p_lo[my_bucket]) ) margin_b = MAX0( (king_p_hi[opp_bucket] - king_p_lo[my_bucket]) + (resid_p_hi[my_bucket] - resid_p_lo[opp_bucket]) ) ``` - `margin_a` gates the fail-high check (`score - margin_a > beta`): soundness requires assuming the most pessimistic-for-me, optimistic-for-opponent outcome for each term. - `margin_b` gates the fail-low check (`score + margin_b < alpha`): the mirror image. - **King term**: bucketed by the **attacker's** own army scaler (`MINU(7, armyScaler/4)`, 8 buckets, 0-31 scaler range) -- danger to a king is a function of the attacker's material, not the king-owner's. Shared between both tiers (`iKingP2ByAttackerArmyOnly` / `iKingP98ByAttackerArmyOnly` in eval.c). - **Residual term** (mobility/trapped/passers-not-yet-counted/etc.): bucketed by **each side's own** army scaler. Two separate tables because the two tiers have different baselines to estimate *from*: - `iResidualP2/98ByOwnArmy` -- regular lazy's baseline already includes pawns/passers/bad-trade/bishop-pairs, so this residual only needs to cover what's left (mobility, trapped pieces, etc). - `iSuperLazyResidualP2/98ByOwnArmy` -- super-lazy's baseline is pure material, so this residual is much wider (includes everything regular lazy's own residual excludes). `_hi`/`_lo` are currently p98/p2 as shipped, but genuinely **per-bucket tunable** -- see "How to keep tuning" below. The king table has very little spread above p90 in every bucket (already checked against the full histogram) and was deliberately left alone throughout tuning; all of today's percentile-adjustment work only ever touched the two residual tables. `iPositional[side]` (the value fed back to QSearch's futility pruning, `search.c`'s `FUTILITY_BASE_MARGIN_BY_SOURCE[tier] + iPositional[side]`) is now a **separate**, deliberately different statistic: a **p50** (median, not tail-bound) typical-value estimate of that side's own missing terms, matching what full-eval's own `iPositional` actually is (a real point estimate, not a safety margin). See `iKingP50By...` / `iResidualP50By...` / `iSuperLazyResidualP50By...` in eval.c. Don't confuse these with the p98/p2 margin tables -- different statistic, different purpose (bias correction vs. soundness bound). ## Where the histogram data lives `eval.c`'s `_PersistRawHistogramsToFile()` (called automatically every time a `DUMP_RESIDUAL_SAMPLES` build runs `calibrate residual`) writes **every raw bin** of all three histograms (residual, sl_residual, king) to `/var/tmp/typhoon/residual_raw_histograms.txt` -- not just the handful of percentile columns that happen to be printed to the console. This directory (`src/eval_tune/lazy_eval_margins/`) has copies from the 2026-09-17 "guess #2" 30k-position calibration run (`residual_raw_histograms_guess2_30k.txt`) plus the reader script (`hist_percentile.py`) -- **copy the fresh file here after any future 30k run you want to keep**, since `/var/tmp` is scratch and could be cleared. To read *any* percentile out of a saved histogram, without re-running anything: ```sh python3 hist_percentile.py residual_raw_histograms_guess2_30k.txt # default spread python3 hist_percentile.py residual_raw_histograms_guess2_30k.txt 0.93 0.965 # exact percentiles you want ``` ## How to keep tuning 1. **Build**: `gmake clean && gmake -j5 GENETIC=1 PERF_COUNTERS=1 MP=1 SIXTYFOUR=1 DUMP_RESIDUAL_SAMPLES=1 CALIBRATE_MARGIN_SAFETY=1` 2. **Run the 30k stratified pool** (both commands in one invocation -- `calibrate residual` persists the histogram file, `calibrate marginsafety` prints the real exceeded rate per **material** bucket for both tiers): ```sh ./typhoon --cpus 1 --hash 256m --logfile /var/tmp/typhoon/run.log \ --batch --command "force; book name /nonexistent.book.bin; sd 6; \ script ../tests/eval_calibration_sample_strat30k.ep_; \ calibrate residual; calibrate marginsafety" \ > /var/tmp/typhoon/run.stdout 2>&1 ``` Takes roughly 20-25 minutes. Run it in the background (`run_in_background`/`&`), don't block on it. 3. **Read the exceeded rate per material bucket** from the `calibrate marginsafety` output (`exceeded / n`, printed separately for "Regular lazy" and "Super lazy", by **material** bucket -- combined army scaler / 8, a *different* bucketing than the percentile tables' per-side scaler/4 buckets -- see the caveat below). 4. **Copy the fresh `/var/tmp/typhoon/residual_raw_histograms.txt`** into this directory before it can get lost. 5. **Pick new percentiles per per-side bucket** using `hist_percentile.py` against the fresh file, and edit `iResidualP2/98ByOwnArmy` / `iSuperLazyResidualP2/98ByOwnArmy` in eval.c directly (keep each bucket's low/high pair **symmetric** -- e.g. p95/p5 together, never p98 mixed with p10 in the same bucket). 6. **Rebuild and re-run step 2** to verify the real effect. ### The two gotchas that cost real time this session - **The 500-position "quick check" sample is unreliable for this specific measurement.** It's fine for cheap direction-of-travel checks on most things, but exceeded-rate is a rare-event percentage, and 500 positions (not even 500 independent draws -- a single game's search tree can dominate a material bucket's node count) can be off by 2-5x from the true 30k-pool rate. One bucket (super-lazy material bucket 1) got tightened based on a 500-position reading of 0.035%, then the real 30k number came back at 1.13% -- a dangerous-direction miss. **Always confirm with the full 30k pool before trusting a number**, especially anything under ~1%. - **Changing one bucket's margin changes measured behavior in OTHER buckets too**, even ones whose own table entries never moved. This isn't a bug -- `Eval()`'s exit decisions are real search behavior, so widening/narrowing any margin reshapes the actual search tree (different alpha/beta bounds propagate differently), which changes which exact nodes land in *every* material bucket, not just the one you touched. Material bucket 3's exceeded rate moved from 0.21% to 0.94% to 0.65% to 1.44% across different runs despite its own table entries never changing once. Don't chase a bucket that "moved" if you didn't touch its inputs -- check what else changed first. - **Material buckets and per-side buckets are different bucketing schemes.** `calibrate marginsafety` reports by **material bucket** (`_MaterialBucket`: combined army scaler for both sides, /8). The percentile tables you actually edit are indexed by **per-side** bucket (`armyScaler/4`, one bucket per side, looked up twice -- once as "my bucket", once as "opponent's bucket"). A single material bucket (e.g. combined scaler 24-31) is populated by many different per-side bucket combinations (3+4, 2+5, 1+6, 0+7...), so there's no clean 1:1 way to "fix material bucket 3" by editing "per-side bucket 3" alone -- expect diffuse, imprecise control, not a dial. ## On the "insane" 2% idea Structurally straightforward: pick a looser matched percentile pair (e.g. p90/p10, or go outside the currently-recorded columns using `hist_percentile.py` against a fresh histogram) for whichever buckets you want at ~2%, following the same edit-rebuild-reverify loop above. Two things worth checking once you have it: 1. **NPS gain isn't guaranteed to scale linearly with the exceeded-rate relaxation** -- this session found the NPS win from the whole redesign was much smaller than expected (see the `git log`/session history around `EvalLookForDrawishSituations` getting added to both cheap tiers, and `fBareKingEitherSide`'s two unconditional `TotalNonPawnCount` calls on every node -- both are new fixed costs that eat into any margin-driven exit-rate gain). Worth re-checking the `Eval exit breakdown` / new `calibrate exitrate` numbers (below) alongside NPS, not just NPS alone. 2. **`FUTILITY_BASE_MARGIN_LAZY`/`SUPERLAZY`** (search.c/chess.h) were calibrated against *this* `iPositional` distribution (`CALIBRATE_QSEARCH_FUTILITY`, see chess.h's derivation comment near those two `#define`s) -- if the eval-exit margins move a lot, especially in a direction that changes how often/how confidently each tier exits, it's worth re-running that calibration too rather than assuming the existing 350/175 values still hold. ## New tooling added this session (always available, no build flag) - **`calibrate exitrate`** -- prints eval exit rate (% super lazy / % lazy / % full) **by material bucket**, not just the flat aggregate "Eval exit breakdown" line every search already prints. Complements the exceeded-rate data: a bucket with low exit rate AND low exceeded rate has real untapped speed headroom; a bucket already near its target exceeded rate with a healthy exit rate is well-calibrated. Implemented in eval.c (`g_u64SuperLazyEvalsByBucket` / `g_u64LazyEvalsByBucket` / `g_u64FullEvalsByBucket`, `DumpEvalExitRateByBucket`) and wired into command.c.