Redesign lazy/super-lazy eval margins with position-specific percentile

tables, and fix four real search/recognizer bugs found while validating it

Main theme: replace the flat, heavily-padded LAZY_EVAL_BASE_MARGIN /
SUPER_LAZY_MARGIN_BY_ARMY constants with per-material-bucket percentile
tables (king safety by attacker army, residual by own army, separate
tables for the lazy vs super-lazy baselines) derived from a 30k-position
calibration pool, targeting a measured ~1-1.5% exceeded-margin rate
instead of an unmeasured "add lots of headroom" floor. QSearch's
FUTILITY_BASE_MARGIN_LAZY/SUPERLAZY were re-derived to match the new,
now-meaningful (p50-based) iPositional this produces. Self-play confirms
the working tree beats both HEAD, a more liberal (~5% miss target)
variant, and a more conservative (near-zero miss) variant of the same
margin mechanism.

Along the way, an initial self-play regression (this margin work bundled
with several changes in one pass, one of which broke things) led to a
day of ZFS-snapshot bisection and then targeted DEBUG-build fuzzing
against a stratified 200k-position corpus (tests/cross_sectional_200k.ep_,
new), which is what actually found and fixed four distinct, real,
pre-existing engine bugs -- none of them related to the margin redesign
itself:

- searchsup.c (CommonSearchInit): three early-return paths (timer
  abort, draw, too-deep bailout) left a node's own PV[] row stale
  instead of clearing it, letting an ancestor's UpdatePV splice a
  leftover move from a completely unrelated node into the displayed
  PV -- root cause of a WalkPV/MakeMove assert ("moving a piece to
  capture, but nothing was on the target square").

- dynamic.c (RecordEnprisePieceAtPly): the two-slot en-prise dedup
  compared (square, piece) instead of square alone, so a same-square
  re-record with a different piece (legitimate -- a fail-high
  capture's victim is restored by UnmakeMove and can differ across
  calls at the same ply) shifted a duplicate square into both slots,
  tripping ValueOfMaterialInTroubleDespiteMove's two-distinct-squares
  assert.

- recogn.c (_RecognizeKBKP): the wrong-colored-bishop corner-fortress
  draw shortcut fired even when the weak side has its own pawn(s),
  which can hand the strong side real zugzwang resources the fortress
  logic doesn't account for -- produced a false hard draw bound where
  EGTB shows a near-forced loss.

- eval.c (_EvalPasserRacesAgainstLoneKings): a rook pawn's race was
  credited as winning even when the strong side's own king sits
  exactly on its own pawn's queening square (distance 0 specifically --
  adjacent is fine, the square is empty) -- a classic stalemate
  resource the race heuristic can't see. Only reachable through
  _RecognizeKPK's pawn-hash-cache-hit path, which is why
  testrecogn.c's TestRecognExhaustiveKPK reported "0 wrong" the whole
  time; extended it to cover both color assignments and the full board
  (no mirror-shortcut) and documented the remaining cache-hit coverage
  gap for a future session to close.

Also: match_play.py gains a local_ebf metric (geometric-mean of
depth-normalized node-growth ratios between consecutive completed
iterations) alongside the existing cumulative nodes**(1/depth) ebf,
which turned out to be misleading when comparing engines with
different lazy-exit aggressiveness -- and probe.c/hash.c pick up two
small, independently-found correctness fixes (TB root-probe promotion
piece missing its color bit; ASSERT(g_pHashTable) crashing on the
legitimate "--hash none" config) noticed along the way.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

1 files changed, 90 insertions, 0 deletions

diff --git a/src/eval_tune/build_curated_texel_pool.py b/src/eval_tune/build_curated_texel_pool.py
new file mode 100644
index 0000000..86f61b8
--- /dev/null
+++ b/src/eval_tune/build_curated_texel_pool.py
@@ -0,0 +1,90 @@
+#!/usr/bin/env python3
+"""Turn a match_controls.py-matched king-safety CSV (mate/control pairs,
+fen retained -- see extract_king_features.py's 2026-09-15 fen-retention
+fix) into a Texel-ready (fen, result) pool: runs the same qsearch-
+agreement quiet filter used for the generic TWIC pool (this data was
+NEVER filtered for quietness at mining time by design -- see mine_king_
+mates.py's is_quiet_position, added 2026-09-15 -- so this step matters
+here at least as much as it does for TWIC, arguably more, since "N plies
+before a real mate" specifically targets the moment before a real attack
+lands, exactly where a quiet-by-SEE-but-not-by-search position is most
+likely).
+
+Result-label framing (a real modeling choice, not a literal game
+result): label==1 rows are "this color's king got mated in this game
+from here" -- treated as a loss (0.0) for that color. label==0 rows are
+the matched sound/control side -- treated as a win (1.0) for that color.
+Output is always from WHITE's POV (matching eval_white_pov_batch's own
+convention), so a row is flipped if mated_color_or_sampled_color is
+black. This is an approximation (these aren't actual final game
+results, just "survived this specific king-safety episode or not") --
+flagged explicitly rather than silently treated as equivalent to a real
+Texel outcome label.
+"""
+import argparse
+import csv
+import sys
+
+from tune_eval_dna import Engine, filter_by_qsearch_agreement
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("matched_csv")
+    ap.add_argument("out_tsv")
+    ap.add_argument("--engine", default="../typhoon")
+    ap.add_argument("--qsearch-max-diff", type=int, default=30)
+    ap.add_argument("--dna-scratch-dir", default="/tmp/typhoon_curated_pool")
+    args = ap.parse_args()
+
+    with open(args.matched_csv, newline="") as f:
+        rows = list(csv.DictReader(f))
+    print(f"Loaded {len(rows)} matched rows from {args.matched_csv}",
+          file=sys.stderr)
+
+    fen_to_row = {}
+    for r in rows:
+        fen_to_row[r["fen"]] = r
+    if len(fen_to_row) != len(rows):
+        print(f"WARNING: {len(rows) - len(fen_to_row)} duplicate FENs "
+              f"collapsed -- matched.csv shouldn't have dupes normally, "
+              f"investigate if this is large.", file=sys.stderr)
+
+    # (fen, result) pairs for the quiet-filter -- result value here is a
+    # placeholder (unused by filter_by_qsearch_agreement, which only
+    # reads/returns fens), real result labels are attached after.
+    placeholder_positions = [(fen, 0.0) for fen in fen_to_row]
+
+    with Engine(args.engine, args.dna_scratch_dir) as engine:
+        kept = filter_by_qsearch_agreement(
+            engine, placeholder_positions, max_diff_cp=args.qsearch_max_diff
+        )
+
+    n_flipped = 0
+    out_rows = []
+    for fen, _ in kept:
+        r = fen_to_row[fen]
+        label = int(r["label"])
+        color = r["mated_color_or_sampled_color"]
+        # label==1 (mate) -> loss (0.0) for that color; label==0
+        # (control/sound) -> win (1.0) for that color.
+        result_for_color = 0.0 if label == 1 else 1.0
+        if color == "black":
+            result_white_pov = 1.0 - result_for_color
+            n_flipped += 1
+        else:
+            result_white_pov = result_for_color
+        out_rows.append((fen, result_white_pov))
+
+    with open(args.out_tsv, "w") as f:
+        for fen, result in out_rows:
+            f.write(f"{fen}\t{result}\n")
+
+    print(f"Quiet-filter kept {len(kept)}/{len(fen_to_row)} "
+          f"({len(kept)/len(fen_to_row):.1%})", file=sys.stderr)
+    print(f"Wrote {len(out_rows)} (fen, result) rows to {args.out_tsv} "
+          f"({n_flipped} POV-flipped from black)", file=sys.stderr)
+
+
+if __name__ == "__main__":
+    main()