summaryrefslogtreecommitdiff
path: root/src
diff options
context:
space:
mode:
authorScott Gasch <[email protected]>2026-09-18 09:43:17 -0700
committerScott Gasch <[email protected]>2026-09-18 09:43:17 -0700
commitf698c02de213d0ed8968fb7cca5f70df51fa0763 (patch)
tree080d6443fbe62d71de092f9acde61da387dff408 /src
parent4a7ec9ac91939957fb3d3e5564fc24a385b422ff (diff)
Redesign lazy/super-lazy eval margins with position-specific percentile
tables, and fix four real search/recognizer bugs found while validating it Main theme: replace the flat, heavily-padded LAZY_EVAL_BASE_MARGIN / SUPER_LAZY_MARGIN_BY_ARMY constants with per-material-bucket percentile tables (king safety by attacker army, residual by own army, separate tables for the lazy vs super-lazy baselines) derived from a 30k-position calibration pool, targeting a measured ~1-1.5% exceeded-margin rate instead of an unmeasured "add lots of headroom" floor. QSearch's FUTILITY_BASE_MARGIN_LAZY/SUPERLAZY were re-derived to match the new, now-meaningful (p50-based) iPositional this produces. Self-play confirms the working tree beats both HEAD, a more liberal (~5% miss target) variant, and a more conservative (near-zero miss) variant of the same margin mechanism. Along the way, an initial self-play regression (this margin work bundled with several changes in one pass, one of which broke things) led to a day of ZFS-snapshot bisection and then targeted DEBUG-build fuzzing against a stratified 200k-position corpus (tests/cross_sectional_200k.ep_, new), which is what actually found and fixed four distinct, real, pre-existing engine bugs -- none of them related to the margin redesign itself: - searchsup.c (CommonSearchInit): three early-return paths (timer abort, draw, too-deep bailout) left a node's own PV[] row stale instead of clearing it, letting an ancestor's UpdatePV splice a leftover move from a completely unrelated node into the displayed PV -- root cause of a WalkPV/MakeMove assert ("moving a piece to capture, but nothing was on the target square"). - dynamic.c (RecordEnprisePieceAtPly): the two-slot en-prise dedup compared (square, piece) instead of square alone, so a same-square re-record with a different piece (legitimate -- a fail-high capture's victim is restored by UnmakeMove and can differ across calls at the same ply) shifted a duplicate square into both slots, tripping ValueOfMaterialInTroubleDespiteMove's two-distinct-squares assert. - recogn.c (_RecognizeKBKP): the wrong-colored-bishop corner-fortress draw shortcut fired even when the weak side has its own pawn(s), which can hand the strong side real zugzwang resources the fortress logic doesn't account for -- produced a false hard draw bound where EGTB shows a near-forced loss. - eval.c (_EvalPasserRacesAgainstLoneKings): a rook pawn's race was credited as winning even when the strong side's own king sits exactly on its own pawn's queening square (distance 0 specifically -- adjacent is fine, the square is empty) -- a classic stalemate resource the race heuristic can't see. Only reachable through _RecognizeKPK's pawn-hash-cache-hit path, which is why testrecogn.c's TestRecognExhaustiveKPK reported "0 wrong" the whole time; extended it to cover both color assignments and the full board (no mirror-shortcut) and documented the remaining cache-hit coverage gap for a future session to close. Also: match_play.py gains a local_ebf metric (geometric-mean of depth-normalized node-growth ratios between consecutive completed iterations) alongside the existing cumulative nodes**(1/depth) ebf, which turned out to be misleading when comparing engines with different lazy-exit aggressiveness -- and probe.c/hash.c pick up two small, independently-found correctness fixes (TB root-probe promotion piece missing its color bit; ASSERT(g_pHashTable) crashing on the legitimate "--hash none" config) noticed along the way. Co-Authored-By: Claude Sonnet 5 <[email protected]>
Diffstat (limited to 'src')
-rw-r--r--src/GNUmakefile3
-rw-r--r--src/TODO120
-rwxr-xr-xsrc/chess.h73
-rw-r--r--src/command.c201
-rw-r--r--src/dynamic.c19
-rw-r--r--src/eval.c2320
-rw-r--r--src/eval_tune/build_curated_texel_pool.py90
-rw-r--r--src/eval_tune/compare_king_safety_crafty.py112
-rw-r--r--src/eval_tune/fit_king_safety_logistic_material.py147
-rw-r--r--src/eval_tune/fit_king_safety_ridge.py202
-rw-r--r--src/eval_tune/lazy_eval_margins/README.md194
-rw-r--r--src/eval_tune/lazy_eval_margins/aggregate_marginsafety_shards.py92
-rw-r--r--src/eval_tune/lazy_eval_margins/hist_percentile.py86
-rw-r--r--src/eval_tune/lazy_eval_margins/residual_raw_histograms_guess2_30k.txt26
-rwxr-xr-xsrc/eval_tune/match_play.py102
-rw-r--r--src/eval_tune/sample_eval_calibration_positions.py182
-rwxr-xr-xsrc/eval_tune/test_vs_head.sh4
-rw-r--r--src/gamelist.c35
-rw-r--r--src/hash.c11
-rw-r--r--src/main.c12
-rw-r--r--src/probe.c26
-rw-r--r--src/recogn.c148
-rw-r--r--src/root.c56
-rw-r--r--src/script.c6
-rw-r--r--src/search.c42
-rw-r--r--src/searchsup.c39
-rw-r--r--src/testrecogn.c168
-rw-r--r--src/util.c13
28 files changed, 3785 insertions, 744 deletions
diff --git a/src/GNUmakefile b/src/GNUmakefile
index e6d24d8..e28649d 100644
--- a/src/GNUmakefile
+++ b/src/GNUmakefile
@@ -162,6 +162,9 @@ endif
ifdef CALIBRATE_MARGIN_SAFETY
PROFILE += -DCALIBRATE_MARGIN_SAFETY
endif
+ifdef DUMP_RESIDUAL_SAMPLES
+PROFILE += -DDUMP_RESIDUAL_SAMPLES
+endif
ifdef DIAG_NO_QSEARCH_FUTILITY
PROFILE += -DDIAG_NO_QSEARCH_FUTILITY
endif
diff --git a/src/TODO b/src/TODO
index ae02475..dce08d2 100644
--- a/src/TODO
+++ b/src/TODO
@@ -5,3 +5,123 @@ Think about futility pruning when a >= +NMATE
Stick eval / king score in ply info and use it to trigger extensions
r2r4/p4pk1/2p3p1/1p1nPR2/5p1Q/2N5/PPPq4/1K4R1 w - - 0 0 -- check extension crazy
+
+2026-09-17: search.c's extended futility pruning (VALUE_KNIGHT/VALUE_ROOK
+gate, PREPARE_TO_TRY_MOVES) now folds in iImprovement + iPositional[side]
+(the same per-tier p50 typical-value estimate QSearch's own futility gate
+uses), but still uses one flat margin regardless of which Eval() exit tier
+(PLY_INFO.uLastPositionalSource) produced iEval/iPositional at that node.
+iPositional corrects for a tier's mean bias, not its spread -- a coarse
+(super-lazy) read could carry more variance/tail risk than a fine one even
+after mean-centering. Rather than a categorical "never prune from
+super-lazy" rule (rejected as too blunt/magic-number-shaped -- iPositional
+already replaces trust-by-category with trust-by-measured-bias), the
+principled fix is a real EFP_BASE_MARGIN_BY_SOURCE[tier], calibrated the
+same way as QSearch's FUTILITY_BASE_MARGIN_BY_SOURCE (CALIBRATE_QSEARCH_
+FUTILITY-style harness: search the would-be-pruned move anyway, bucket
+surprise rate by distance-short-of-bar AND uLastPositionalSource, check
+whether the tiers actually need different margins here too). Not yet
+built -- deferred behind self-play validation of today's iPositional/
+futility-margin changes landing first.
+
+2026-09-17: self-calibrating lazy/super-lazy margins -- instead of (or on
+top of) the static per-bucket percentile tables, maintain a small mutable
+per-(tier, material-or-army-bucket) delta, adjusted at runtime: at a low
+sample rate (~1%), let a node that would have taken the cheap exit run to
+full eval anyway, compare, and nudge the delta -- up fast on a real miss
+(swing exceeded the margin), down slowly otherwise (fast-up/slow-down,
+not symmetric AIMD). Motivated by this session's CALIBRATE_MARGIN_SAFETY
+digging: exceeded-margin nodes aren't spread uniformly across the tree,
+they cluster heavily inside a handful of "spicy" root games' subtrees
+(see the recurring N1Q5/N1B5/N1R5/... and 5rk1/1b2Q2p/... FEN families
+that kept reappearing across every guess this session, all clearly the
+same search tree a few plies apart). A single fixed percentile can't
+respond to "we're currently deep in a dangerous subtree right now"; a
+fast-reacting runtime term could -- widen in response to a real miss
+right when the search is actually inside the risky region, then decay
+back out once it's past. Two real costs before building this: (1) it
+makes eval score depend on search visitation order/history, adding to
+the non-determinism MP already introduces and complicating the kind of
+fixed-margin A/B comparison work this session did throughout; (2) it's
+its own new tuning problem (step-up size, decay rate) layered on top of
+the static-table one -- don't start building this until the question of
+whether the current 30k-position calibration pool is even large/diverse
+enough (see cross_sectional_1/2.ep_ variance check, tests/) is settled,
+so it's clear which change is causing which effect. Explicitly rejected
+alternative: hand-carving structural exceptions for specific recurring
+motifs (e.g. "queen has infiltrated near the enemy king") -- doesn't
+generalize, and is exactly the kind of special-casing that rots.
+
+2026-09-18: STILL AN OPEN QUESTION, not decided either way. Scott's
+worry re-examining this: reaction speed. At a ~1-in-10k sample rate, an
+adaptive term probably can't react fast enough inside a spicy subtree to
+matter -- by the time enough samples land there to trip the widen, the
+search has likely already made every unsound cutoff in that region
+using the too-narrow margin, so the wrong bound is already baked into
+that iteration's result before the term ever moves. Raising the sample
+rate to compensate directly fights the entire reason the lazy tiers
+exist (paying full-eval cost specifically to make the cheap tiers
+marginally safer). Separately, the complexity cost compounds with
+something already proven expensive this session: guess #1-11 (eval.c)
+showed changing one bucket's *static* margin reshapes the whole search
+tree in ways that confound attribution (see those comments' "cross-
+bucket confound" notes) -- a runtime-adaptive term makes that confound
+permanent and load-bearing every game, rather than a one-time tuning
+headache, and two engines with identical DNA could diverge on eval
+*decisions* (not just timing) run to run. Current lean: shelve it, since
+the mechanism can only close the barn door after the horse for the
+specific node that triggered it, against a permanent tax on
+determinism/reasoning-about-Eval(), for a gain the static percentile
+tables already get within ~1% of via the boring approach. Not acted on
+-- left open in case reaction-speed concerns turn out to be wrong in
+practice (e.g. if "spicy subtree" actually means tens of thousands of
+nodes wide, not a handful, the sample-rate math changes).
+
+2026-09-18: build two eval.c lazy-margin variants and compare all three
+(current/baseline, "messy but fast", "super conservative") head to head.
+"Messy but fast": tune iResidualP2/98ByOwnArmy + iSuperLazyResidualP2/98-
+ByOwnArmy to accept ~5% exceeded rate against tests/cross_sectional_200k.ep_
+(search under each root position, not just eval in isolation -- want
+real search-tree exceeded rate, not a static-position read). "Super
+conservative": tune the same tables so the 200k set hardly ever misses.
+Then: (1) self-play match_play.py between baseline and each variant, and
+between the two variants directly; (2) full tests/ecm.ep_ run (not a
+curated subset) under each of the three binaries, same sd, compare
+solve counts. Interesting because it's a direct speed-vs-soundness
+tradeoff measurement on the exact margin mechanism this session spent a
+day mistakenly blaming for the 2026-09-17 regression (see recent git
+log/session notes) -- the real bug was searchsup.c's CommonSearchInit
+PV staleness, not the margins, so this is now safe to actually explore
+properly with a real experiment instead of blind percentile-guessing.
+ Cheaper derivation than 11 rounds of guess-rebuild-rerun (see eval.c's
+ guess #1-11 history comments): PersistMarginSafetyHistogramsToFile
+ (calibrate marginsafety, CALIBRATE_MARGIN_SAFETY build) persists the
+ raw per-bucket SWING distribution (real full-eval score minus the
+ lazy/super-lazy estimate), not just a boolean exceeded count -- so
+ one calibration run against the 200k pool lets a script sweep any
+ candidate margin M against the persisted histogram and read "fraction
+ of swings > M" directly, giving the M for ~5% exceeded and the M for
+ ~0% exceeded from the SAME run. eval_tune/lazy_eval_margins/
+ aggregate_marginsafety_shards.py already sums bins across shards;
+ needs a small "exceeded rate at threshold M" query added alongside
+ hist_percentile.py's percentile query. Caveat (documented repeatedly
+ in eval.c's guess history): the recorded swing distribution is itself
+ shaped by whatever margin was actually compiled in during that
+ calibration run -- a different final M reshapes the search tree,
+ which changes which nodes even reach the swing-recording code -- so
+ this gives a very good first estimate for both variants, not a
+ substitute for the real match_play/ecm validation once each variant
+ is actually built.
+
+2026-09-18: sharded ASSERT-fuzzing harness -- the technique that found
+the CommonSearchInit PV-staleness bug (DEBUG binary + gdb, batch mode,
+`script <shard>.ep_` at a shallow sd, let it run until either an
+ASSERT/SIGSEGV or clean exit) generalizes well and should be run
+routinely, not just once ad hoc. Shard tests/cross_sectional_200k.ep_
+(or any large stratified pool) into N pieces and run N concurrent
+DEBUG-build _typhoon processes under gdb, one per shard, each logging
+to its own file; investigate any hit (assert message + gdb backtrace)
+the same way this session did. Needs a full core budget (N cores) so
+it collides with self-play match runs -- schedule it for whenever the
+box isn't already saturated by match_play.py workers, not concurrently
+with them.
diff --git a/src/chess.h b/src/chess.h
index 175ad5d..e585506 100755
--- a/src/chess.h
+++ b/src/chess.h
@@ -1102,9 +1102,13 @@ typedef struct _COUNTERS
UINT64 u64CyclesEvalSuperLazy;
// 2026-09-09: EvalLookForDrawishSituations runs after
- // u64CyclesEvalPostLazyMisc's window closes, only on the
- // full-eval exit -- previously fell into "unaccounted" in
- // root.c's eval breakdown report.
+ // u64CyclesEvalPostLazyMisc's window closes -- previously fell
+ // into "unaccounted" in root.c's eval breakdown report.
+ // 2026-09-17: no longer full-eval-only -- also called from the
+ // regular-lazy pre-exit check (eval.c), so this counter now
+ // accumulates both call sites' cost. The two calls' real
+ // per-position-only work (DRAWISH_FACTS) only happens once
+ // per Eval() call regardless -- see _EnsureDrawishFactsComputed.
UINT64 u64CyclesEvalDrawish;
// Per-exit-path *total call cost*, i.e. the full entry-to-exit
@@ -1303,6 +1307,18 @@ typedef struct _PLY_INFO
SCORE iBeta;
#endif
SCORE iEval;
+ // 2026-09-17: per-color iPositional from the SAME Eval() call that
+ // set iEval above -- previously discarded (Search() called Eval()
+ // with piPositional==NULL). Stored per-ply, right next to iEval,
+ // so any code walking the ply stack (up or down) can read both the
+ // score and its positional-bias estimate for a given node together
+ // -- e.g. a cleaner cross-ply "improving" signal that corrects for
+ // a tier switch (super-lazy vs full) between the two Eval() calls
+ // being compared, instead of just diffing raw iEval values. Not
+ // yet consumed that way anywhere; currently only the search.c EFP
+ // gate (VALUE_KNIGHT/VALUE_ROOK margins) reads this, for the
+ // node-it-was-computed-at only.
+ SCORE iPositional[2];
INT iExtensionAmount;
FLAG fInCheck;
FLAG fInQsearch;
@@ -2901,6 +2917,9 @@ InsertHeadList(IN DLIST_ENTRY *pListHead,
void
ParseUserInput(FLAG fSearching);
+void
+ExecuteCommandLineDirectly(CHAR *szLine, POSITION *pos);
+
FLAG
InitializeCommandSystem(void);
@@ -3180,8 +3199,39 @@ IsDraw(SEARCHER_THREAD_CONTEXT *ctx);
// exit -- can reference the matching tier's constant directly instead
// of a stale flat value.
#define FUTILITY_BASE_MARGIN_FULL (450)
-#define FUTILITY_BASE_MARGIN_LAZY (225)
-#define FUTILITY_BASE_MARGIN_SUPERLAZY (275)
+// 2026-09-17: LAZY raised 225->350, SUPERLAZY lowered 275->175, after
+// eval.c's lazy/super-lazy iPositional stopped being flat 0 and became
+// a real, signed p50-based typical-value estimate (see eval.c's
+// iKingP50ByAttackerArmyOnly/iResidualP50ByOwnArmy). Re-measured via
+// `calibrate qsearchfutility` (CALIBRATE_QSEARCH_FUTILITY build,
+// tests/twic_sample.ep_, 1500 real-game positions, sd 6) at the OLD
+// margins first, to see what the iPositional change alone had done:
+// generic capture/promo gate, surprise rate (wrongly-rejected-move
+// rate) by distance short of the futility bar, summed over piPositional
+// buckets:
+// source margin(old) dist0-25 25-50 50-100 100-200 200-400 400+ n
+// full 450 (n/a) 1.91% 1.72% 1.80% 1.30% 0.46% 0.11% 135,031
+// regular lazy 225 6.22% 2.89% 2.07% 1.80% 1.58% 0.16% 1,024,889
+// super lazy 275 1.23% 0.98% 0.90% 0.60% 0.24% 0.12% 1,813,965
+// Regular lazy got MORE dangerous than its original 2026-09-08
+// derivation (was 1.26% max at dist0-25; now 6.22%) -- the cross-tab
+// by piPositional bucket at dist0-25 showed why: piPositional<0 alone
+// carries an 8.9% surprise rate (n=66,040) vs. 1.9%/1.4% for the >=0
+// buckets -- piPositional is genuinely predictive (validates the p50
+// approach) but 225 wasn't wide enough for the negative population.
+// Super lazy got safer (was 7.96% at dist0-25; now 1.23%), real
+// headroom to tighten for speed. New values chosen by reading where
+// each tier's current worst bucket would land after the shift (not
+// guessed): LAZY +125 moves today's risky dist0-25 population into
+// territory today's data shows is ~1.8% (matching full-eval's own risk
+// level); SUPERLAZY -100 moves today's safe dist100-200 bucket (0.60%)
+// into the new near-boundary zone, still comfortably safe. FULL is
+// untouched -- its iPositional computation (full-eval exit) wasn't
+// changed this session, and its numbers above are consistent with the
+// original 2026-09-08 derivation. Re-derive with `calibrate
+// qsearchfutility` if iPositional's construction changes again.
+#define FUTILITY_BASE_MARGIN_LAZY (350)
+#define FUTILITY_BASE_MARGIN_SUPERLAZY (175)
// Measured: disabling this entirely (see lmr_testing/RESULTS.md) is a
// clear net loss across ringers/confident_quick/hard_quick, so IID itself
@@ -3570,9 +3620,20 @@ DumpBaseMarginCalibration(void);
#ifdef CALIBRATE_MARGIN_SAFETY
void
-DumpMarginSafetyCalibration(void);
+DumpMarginSafetyCalibration(CHAR *szHistogramPathOverride);
#endif
+#ifdef DUMP_RESIDUAL_SAMPLES
+void
+DumpResidualAndKingSafetyHistograms(void);
+#endif
+
+// Always-on (not gated behind a CALIBRATE_* build flag) -- see
+// eval.c's g_u64SuperLazyEvalsByBucket/g_u64LazyEvalsByBucket/
+// g_u64FullEvalsByBucket and the five INC-site call pairs in Eval().
+void
+DumpEvalExitRateByBucket(void);
+
#ifdef CALIBRATE_QSEARCH_FUTILITY
// Which futility gate in _ShouldWeConsiderThisMove rejected a move --
// see search.c's _QFutRecordSample. The recapture gate (used to be
diff --git a/src/command.c b/src/command.c
index 5087aee..b201c31 100644
--- a/src/command.c
+++ b/src/command.c
@@ -910,12 +910,26 @@ Return value:
if (argc >= 2 && !STRCMPI(argv[1], "marginsafety"))
{
#ifdef CALIBRATE_MARGIN_SAFETY
- DumpMarginSafetyCalibration();
+ // Optional 3rd argument: output path for the persisted raw
+ // swing histogram (default /var/tmp/typhoon/marginsafety_raw_
+ // histograms.txt) -- needed so N shards of a sharded/parallel
+ // calibration run each write their own file instead of
+ // clobbering one shared path.
+ DumpMarginSafetyCalibration((argc >= 3) ? argv[2] : NULL);
#else
Trace("This binary was not built with CALIBRATE_MARGIN_SAFETY.\n");
#endif
return;
}
+ if (argc >= 2 && !STRCMPI(argv[1], "residual"))
+ {
+#ifdef DUMP_RESIDUAL_SAMPLES
+ DumpResidualAndKingSafetyHistograms();
+#else
+ Trace("This binary was not built with DUMP_RESIDUAL_SAMPLES.\n");
+#endif
+ return;
+ }
if (argc >= 2 && !STRCMPI(argv[1], "qsearchfutility"))
{
#ifdef CALIBRATE_QSEARCH_FUTILITY
@@ -925,11 +939,20 @@ Return value:
#endif
return;
}
+ if (argc >= 2 && !STRCMPI(argv[1], "exitrate"))
+ {
+ // Always-on, no build flag needed -- see eval.c's
+ // g_u64SuperLazyEvalsByBucket/g_u64LazyEvalsByBucket/
+ // g_u64FullEvalsByBucket.
+ DumpEvalExitRateByBucket();
+ return;
+ }
#ifdef CALIBRATE_POSITIONAL
if ((argc < 2) || STRCMPI(argv[1], "dump"))
{
Trace("Usage: calibrate dump | calibrate basemargin | "
- "calibrate marginsafety | calibrate qsearchfutility\n");
+ "calibrate marginsafety | calibrate qsearchfutility | "
+ "calibrate exitrate\n");
return;
}
DumpPositionalCalibration();
@@ -2882,7 +2905,179 @@ COMMAND(HelpCommand)
}
-void
+void
+ExecuteCommandLineDirectly(CHAR *szLine, POSITION *pos)
+/**
+
+Routine description:
+
+ Parse and execute exactly one command/move line against pos, right
+ now, without touching the global input event queue (g_InputEventList).
+
+ This is the not-currently-searching half of ParseUserInput's dispatch
+ logic, factored out so a caller that already has a line of text in
+ hand -- and knows for certain nothing is searching -- can execute it
+ directly instead of round-tripping through PushNewInput()+
+ ParseUserInput().
+
+ That round-trip is what ScriptCommand's _ProcessSuiteLine used to do
+ for every non-"go" script line, and it was a real bug (found
+ 2026-09-16): PushNewInput() appends to the *tail* of the shared
+ global queue and ParseUserInput() pops from the *head*, so it only
+ executes the line just pushed if the queue was empty at that moment.
+ It wasn't guaranteed to be -- PushNewInput() itself splits any
+ caller-supplied string on unquoted ';' into multiple queued entries
+ up front (see PushNewInput, input.c), so anything chained after
+ "script <file>" in the same --command string (e.g. "script foo.ep_;
+ calibrate marginsafety") was already sitting in the queue, ahead of
+ the script's own first line, before the script even started running.
+ The result: the very first script line's push-then-pop popped that
+ *pre-existing* queued command instead (executing it far too early,
+ with no data yet), and every line after that ran one line behind
+ forever (each push+pop executing the *previous* iteration's pushed
+ line, never the one just pushed), because in steady state a FIFO
+ queue with one push and one pop per iteration always yields what was
+ already at the head. Net effect: every `go` searched the *previous*
+ script line's position, not its own, and the true final position was
+ applied but never searched at all. Using this function instead means
+ a script line is parsed and dispatched inline, with no shared queue
+ involved, so there's nothing left in the queue to race against no
+ matter what the caller queued before or after "script".
+
+Parameters:
+
+ CHAR *szLine : mutable; will be NUL-chopped at the first '\n' and
+ further split in place while tokenizing, same as ParseUserInput
+ does to its own copy of the input line.
+ POSITION *pos
+
+Return value:
+
+ void
+
+**/
+{
+ CHAR *pCapturedInput = NULL;
+ CHAR *p = szLine;
+ CHAR *q;
+ ULONG u;
+ CHAR *argv[MAX_ARGS];
+ ULONG argc = 0;
+ FLAG fInQuote = FALSE;
+ MOVE mv;
+
+ while((*p) && (isspace(*p))) p++;
+ if (((*p) && (*p == '#')) ||
+ ((*p) && (*(p+1)) && (*p == '/') && (*(p+1) == '/')))
+ {
+ return;
+ }
+
+ q = p;
+ while(*q)
+ {
+ if (*q == '\n')
+ {
+ *q = '\0';
+ break;
+ }
+ q++;
+ }
+
+ pCapturedInput = STRDUP(p);
+ memset(argv, 0, sizeof(argv));
+ do
+ {
+ while(*p && isspace(*p)) p++;
+ if (!*p) break;
+ argv[argc++] = p;
+ fInQuote = FALSE;
+ while (*p)
+ {
+ if (*p == '"')
+ {
+ fInQuote = !fInQuote;
+ }
+ else if (*p == '{')
+ {
+ fInQuote = TRUE;
+ }
+ else if (*p == '}')
+ {
+ fInQuote = FALSE;
+ }
+ else if (isspace(*p) && (FALSE == fInQuote))
+ {
+ break;
+ }
+ p++;
+ }
+ if (!*p) break;
+ *p = '\0';
+ p++;
+ }
+ while(argc < MAX_ARGS);
+ if (argc == 0)
+ {
+ SystemFreeMemory(pCapturedInput);
+ return;
+ }
+
+ mv.uMove = 0;
+ switch(LooksLikeMove(argv[0]))
+ {
+ case MOVE_SAN:
+ mv = ParseMoveSan(argv[0], pos);
+ break;
+ case MOVE_ICS:
+ mv = ParseMoveIcs(argv[0], pos);
+ break;
+ case NOT_MOVE:
+ for (u = 0; u < ARRAY_LENGTH(g_ParserTable); u++)
+ {
+ if (!STRCMPI(g_ParserTable[u].szOpcode, argv[0]))
+ {
+ (*g_ParserTable[u].pFunction)
+ (pCapturedInput, argc, argv, pos);
+ SystemFreeMemory(pCapturedInput);
+ return;
+ }
+ }
+ if ((TRUE == LooksLikeFen(pCapturedInput)) &&
+ (TRUE == SetRootPosition(pCapturedInput)))
+ {
+ SystemFreeMemory(pCapturedInput);
+ return;
+ }
+ break;
+#ifdef DEBUG
+ default:
+ UtilPanic(SHOULD_NOT_GET_HERE,
+ NULL, NULL, NULL, NULL,
+ __FILE__, __LINE__);
+ break;
+#endif
+ }
+
+ if (mv.uMove)
+ {
+ if (FALSE == OfficiallyMakeMove(mv, 0, FALSE))
+ {
+ UtilPanic(CANNOT_OFFICIALLY_MAKE_MOVE,
+ GetRootPosition(),
+ (void *)(size_t)mv.uMove,
+ NULL,
+ NULL,
+ __FILE__, __LINE__);
+ }
+ } else {
+ Trace("Unknown/illegal move/command: \"%s\"\n", pCapturedInput);
+ }
+ SystemFreeMemory(pCapturedInput);
+}
+
+
+void
ParseUserInput(FLAG fSearching)
/**
diff --git a/src/dynamic.c b/src/dynamic.c
index 9c68993..c64794a 100644
--- a/src/dynamic.c
+++ b/src/dynamic.c
@@ -471,14 +471,27 @@ Return value:
ASSERT(p && IS_VALID_PIECE(p) && !IS_PAWN(p));
ASSERT(GET_COLOR(p) == uExpectedMover);
- if ((ctx->cEnprise[uPly][0] != cSquare) ||
- (ctx->pEnprise[uPly][0] != p))
+ // 2026-09-18: dedup on SQUARE alone, not (square, piece) -- the
+ // two slots are meant to hold the last two DISTINCT en-prise
+ // squares (see ValueOfMaterialInTroubleDespiteMove's "two
+ // different pieces hanging" use, which ASSERTs the two recorded
+ // squares differ). A same-square re-record with a different
+ // recorded piece -- legitimate per this function's own doc
+ // comment (a fail-high capture's victim is restored to cSquare by
+ // UnmakeMove, and can differ across calls at the same ply) --
+ // used to compare piece identity too, so it shifted the old
+ // (cSquare, oldPiece) entry down into slot 1 instead of updating
+ // slot 0 in place, leaving BOTH slots holding the same square.
+ // Found via fuzzing tests/cross_sectional_200k.ep_ (DEBUG build,
+ // sd 3): tripped ValueOfMaterialInTroubleDespiteMove's
+ // ASSERT(c != c1) from a live QSearch call, not a display path.
+ if (ctx->cEnprise[uPly][0] != cSquare)
{
ctx->cEnprise[uPly][1] = ctx->cEnprise[uPly][0];
ctx->pEnprise[uPly][1] = ctx->pEnprise[uPly][0];
ctx->cEnprise[uPly][0] = cSquare;
- ctx->pEnprise[uPly][0] = p;
}
+ ctx->pEnprise[uPly][0] = p;
}
diff --git a/src/eval.c b/src/eval.c
index 730365e..7d224e3 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -352,9 +352,19 @@ static SCORE PASSER_BONUS_AS_MATERIAL_COMES_OFF[32] =
0, 0, 0, 0, 0, 0, 0, 0,
};
-SCORE RACER_WINS_RACE = +800;
+// 2026-09-17: was +800 -- found via fuzzing that _RecognizeKPK stacks
+// material balance and a flat 2*VALUE_PAWN on top of this before
+// returning it as a hard recognizer bound (RECOGN_LOWER/UPPER), so a
+// "winning" race claim could reach ~988 total even before considering
+// whether the claim itself was right. Crafty's equivalent term
+// (pawn_can_promote, evaluate.h) is a flat 525 and -- critically --
+// only ever an additive eval nudge feeding real alpha-beta search,
+// never a hard bound. Turned down to ~300/pawn regardless of that
+// separate soundness fix (see the losing-side-has-no-pawns guard in
+// _RecognizeKPK) since the two problems compound independently.
+SCORE RACER_WINS_RACE = +300;
-static SCORE UNDEVELOPED_MINORS_IN_OPENING[5] =
+static SCORE UNDEVELOPED_MINORS_IN_OPENING[5] =
{// 0 1 2 3 4
0, -6, -10, -16, -22
};
@@ -2964,38 +2974,131 @@ Return value:
#endif // CALIBRATE_POSITIONAL
-#if defined(CALIBRATE_BASE_MARGIN) || defined(CALIBRATE_MARGIN_SAFETY)
+// 2026-09-17: BASE_MARGIN_MAT_BUCKETS/WIDTH and _MaterialBucket() pulled
+// out of the CALIBRATE_BASE_MARGIN/CALIBRATE_MARGIN_SAFETY-only guard
+// below -- now also used unconditionally (see g_u64EvalExitByBucket in
+// Eval(), and root.c's per-bucket exit-rate report) so a normal build
+// can report exit rate by material bucket, not just the flat aggregate
+// "Eval exit breakdown" line -- needed to see whether the p98/p2 margin
+// redesign actually moved the exit rate in any given bucket, since the
+// aggregate rate can mask buckets moving in opposite directions.
+#define BASE_MARGIN_MAT_BUCKETS 8
+#define BASE_MARGIN_MAT_WIDTH 8
+
+static ULONG
+_MaterialBucket(IN POSITION *pos)
+/**
+
+Routine description:
+
+ Coarse combined-material bucket, 0 (bare kings) to
+ BASE_MARGIN_MAT_BUCKETS-1 (full army). Originally CALIBRATE_BASE_
+ MARGIN-only; now also used by the always-on per-bucket eval-exit-
+ rate counters.
+
+Parameters:
+
+ POSITION *pos
+
+Return value:
+
+ ULONG : 0..BASE_MARGIN_MAT_BUCKETS-1
+
+**/
+{
+ ULONG uCombined = pos->uArmyScaler[WHITE] + pos->uArmyScaler[BLACK];
+ return MINU(BASE_MARGIN_MAT_BUCKETS - 1, uCombined / BASE_MARGIN_MAT_WIDTH);
+}
+
+// Always-on per-material-bucket eval-exit counters -- separate from
+// (and simpler than) the CALIBRATE_MARGIN_SAFETY machinery below: just
+// "how many nodes exited via this tier, in this material bucket",
+// enough to compute exit RATE per bucket (this array divided by the
+// per-bucket sum across all three) without needing a special
+// calibration build. See the five INC-site call pairs in Eval() below
+// and root.c's report.
+static UINT64 g_u64SuperLazyEvalsByBucket[BASE_MARGIN_MAT_BUCKETS];
+static UINT64 g_u64LazyEvalsByBucket[BASE_MARGIN_MAT_BUCKETS];
+static UINT64 g_u64FullEvalsByBucket[BASE_MARGIN_MAT_BUCKETS];
+
+void
+DumpEvalExitRateByBucket(void)
+/**
+
+Routine description:
+
+ Print per-material-bucket eval exit rate (super lazy / lazy / full,
+ as a percentage of that bucket's own total) -- command.c's
+ "calibrate exitrate". Complements the flat aggregate "Eval exit
+ breakdown" line (root.c, printed at the end of every search) by
+ showing whether that aggregate is hiding buckets moving in opposite
+ directions.
+
+Parameters:
+
+ void
+
+Return value:
+
+ void
+
+**/
+{
+ ULONG m;
+
+ Trace("Eval exit rate by material bucket (combined army scaler):\n");
+ for (m = 0; m < BASE_MARGIN_MAT_BUCKETS; m++)
+ {
+ UINT64 u64SL = g_u64SuperLazyEvalsByBucket[m];
+ UINT64 u64L = g_u64LazyEvalsByBucket[m];
+ UINT64 u64F = g_u64FullEvalsByBucket[m];
+ UINT64 u64Total = u64SL + u64L + u64F;
+ if (0 == u64Total)
+ {
+ Trace(" bucket %u (scaler %u-%u): no samples\n",
+ m, m * BASE_MARGIN_MAT_WIDTH,
+ (m * BASE_MARGIN_MAT_WIDTH) + BASE_MARGIN_MAT_WIDTH - 1);
+ continue;
+ }
+ Trace(" bucket %u (scaler %2u-%2u): n=%12" COMPILER_LONGLONG_UNSIGNED_FORMAT
+ " super=%5.2f%% lazy=%5.2f%% full=%5.2f%%\n",
+ m, m * BASE_MARGIN_MAT_WIDTH,
+ (m * BASE_MARGIN_MAT_WIDTH) + BASE_MARGIN_MAT_WIDTH - 1,
+ u64Total,
+ 100.0 * (double)u64SL / (double)u64Total,
+ 100.0 * (double)u64L / (double)u64Total,
+ 100.0 * (double)u64F / (double)u64Total);
+ }
+}
+
+
+#ifdef CALIBRATE_BASE_MARGIN
//
// board_representation/EVAL.md section 9: is LAZY_EVAL_BASE_MARGIN (the
// flat, un-widened margin Eval() screens with before ever calling
-// EstimatePositionalScore) well-sized? Two distinct questions, tracked
-// separately, neither requiring the lazy-exit machinery to be disabled
+// EstimatePositionalScore) well-sized? Tracks: once the gate passes and
+// EstimatePositionalScore has widened the margin, does a real exit
+// actually happen, broken out by the worse (max) of the two sides'
+// cheap defect-index buckets? This answers "does the 'very dangerous'
+// bucket (10) ever actually exit, or does its wide margin effectively
+// rule it out". Doesn't require the lazy-exit machinery to be disabled
// (unlike the CALIBRATE_POSITIONAL king-safety work above) -- real
-// exits still happen normally here, so this runs at normal speed:
+// exits still happen normally here, so this runs at normal speed.
//
-// 1. Once the gate passes and EstimatePositionalScore has widened the
-// margin, does a real exit actually happen, broken out by the
-// worse (max) of the two sides' cheap defect-index buckets? This
-// answers "does the 'very dangerous' bucket (10) ever actually
-// exit, or does its wide margin effectively rule it out".
-// 2. When the gate comes back *false* (we skip straight to full eval,
-// no lazy attempt tried at all) -- would a real exit have been
-// available anyway, had we bothered to compute the widened margin?
-// A "yes" here is a false negative: LAZY_EVAL_BASE_MARGIN was
-// tight enough to force full eval on a node that genuinely could
-// have taken the cheap path.
+// 2026-09-17: this used to also track a false-negative rate (the gate
+// coming back false, then checking whether a real exit would've been
+// available anyway) -- that required a flat-gate-false branch, which
+// stopped existing once the flat pre-check itself was retired (see the
+// `if (TRUE)` removal in Eval() below); removed alongside it rather
+// than leaving permanently-zero counters.
//
// 2026-09-06: added a material dimension -- pos->uArmyScaler[WHITE] +
// pos->uArmyScaler[BLACK] (0-62), bucketed into 8 coarse bands, since
// the lazy/full cost ratio measured directly (opening/middlegame vs a
// bare K+P endgame) varies ~7x by material, and LAZY_EVAL_BASE_MARGIN
// is currently one flat constant across all of it.
-#define BASE_MARGIN_MAT_BUCKETS 8
-#define BASE_MARGIN_MAT_WIDTH 8
static UINT64 g_uBaseMarginExitAttempted[BASE_MARGIN_MAT_BUCKETS][11];
static UINT64 g_uBaseMarginExitTaken[BASE_MARGIN_MAT_BUCKETS][11];
-static UINT64 g_uBaseMarginFalseNegative[BASE_MARGIN_MAT_BUCKETS];
-static UINT64 g_uBaseMarginCorrectlyFull[BASE_MARGIN_MAT_BUCKETS];
static ULONG
_MaxKingDangerBucket(IN SEARCHER_THREAD_CONTEXT *ctx,
@@ -3034,30 +3137,6 @@ Return value:
}
-static ULONG
-_MaterialBucket(IN POSITION *pos)
-/**
-
-Routine description:
-
- CALIBRATE_BASE_MARGIN instrumentation only: coarse combined-material
- bucket, 0 (bare kings) to BASE_MARGIN_MAT_BUCKETS-1 (full army).
-
-Parameters:
-
- POSITION *pos
-
-Return value:
-
- ULONG : 0..BASE_MARGIN_MAT_BUCKETS-1
-
-**/
-{
- ULONG uCombined = pos->uArmyScaler[WHITE] + pos->uArmyScaler[BLACK];
- return MINU(BASE_MARGIN_MAT_BUCKETS - 1, uCombined / BASE_MARGIN_MAT_WIDTH);
-}
-
-
void
DumpBaseMarginCalibration(void)
/**
@@ -3066,9 +3145,7 @@ Routine description:
Print LAZY_EVAL_BASE_MARGIN calibration stats gathered so far this
run: per-(material bucket, king-danger bucket) exit-attempted/
- exit-taken counts (command.c's "calibrate basemargin"), and the
- false-negative rate of the flat pre-EstimatePositionalScore gate,
- broken out by material bucket.
+ exit-taken counts (command.c's "calibrate basemargin").
Parameters:
@@ -3081,7 +3158,6 @@ Return value:
**/
{
ULONG m, u;
- UINT64 u64FN, u64CF;
for (m = 0; m < BASE_MARGIN_MAT_BUCKETS; m++)
{
@@ -3103,14 +3179,6 @@ Return value:
(100.0 * (double)g_uBaseMarginExitTaken[m][u] /
(double)g_uBaseMarginExitAttempted[m][u]));
}
- u64FN = g_uBaseMarginFalseNegative[m];
- u64CF = g_uBaseMarginCorrectlyFull[m];
- Trace(" false negatives: %" COMPILER_LONGLONG_UNSIGNED_FORMAT
- " correctly full: %" COMPILER_LONGLONG_UNSIGNED_FORMAT
- " rate: %.4f%%\n",
- u64FN, u64CF,
- (u64FN + u64CF > 0) ?
- (100.0 * (double)u64FN / (double)(u64FN + u64CF)) : 0.0);
}
}
#endif // CALIBRATE_BASE_MARGIN
@@ -3133,11 +3201,29 @@ Return value:
static UINT64 g_uMarginSwingMax[BASE_MARGIN_MAT_BUCKETS];
static UINT64 g_uMarginSwingCount[BASE_MARGIN_MAT_BUCKETS];
static UINT64 g_uMarginSwingSum[BASE_MARGIN_MAT_BUCKETS];
+// 2026-09-17: max/avg alone hide the shape of the swing distribution --
+// a bucket could have a huge max from one outlier and a small avg from
+// a long thin tail of near-zero swings, or a genuinely fat middle.
+// Histogram (same fixed-width-bin technique as RESID_HIST_* above,
+// swings are always >= 0 so no negative-offset needed) plus a sum-of-
+// squares accumulator lets DumpMarginSafetyCalibration report p10/p50/
+// p90 and stdev per bucket, not just max/avg.
+#define SWING_HIST_BIN_WIDTH 10
+#define SWING_HIST_BINS 300 // covers [0, 3000) cp; observed max so far ~1300
+static UINT64 g_uMarginSwingHist[BASE_MARGIN_MAT_BUCKETS][SWING_HIST_BINS];
+static UINT64 g_uMarginSwingSumSq[BASE_MARGIN_MAT_BUCKETS];
// Genuine soundness check: did the swing ever exceed the *actual*
// (already-widened) margin that was used to justify this specific
// exit? Unlike the raw max above (which just says "how far things
// moved"), this directly answers "was any real exit actually unsound".
static UINT64 g_uMarginExceeded[BASE_MARGIN_MAT_BUCKETS];
+// 2026-09-17: how many exceeded-margin FENs have already been printed
+// for this (tier, material bucket) -- capped at MARGIN_EXCEEDED_FEN_
+// SAMPLE_CAP so a run with hundreds of thousands of exceeded events
+// (a real possibility while tuning loose margins) doesn't flood the
+// log; just enough examples per bucket to eyeball by hand.
+#define MARGIN_EXCEEDED_FEN_SAMPLE_CAP 5
+static UINT64 g_uMarginExceededFenSamples[BASE_MARGIN_MAT_BUCKETS];
// Same four counters, kept in a fully separate set for the super-lazy
// exit (eval.c's material-only check before the regular lazy gate).
@@ -3148,27 +3234,56 @@ static UINT64 g_uMarginExceeded[BASE_MARGIN_MAT_BUCKETS];
static UINT64 g_uSuperLazyMarginSwingMax[BASE_MARGIN_MAT_BUCKETS];
static UINT64 g_uSuperLazyMarginSwingCount[BASE_MARGIN_MAT_BUCKETS];
static UINT64 g_uSuperLazyMarginSwingSum[BASE_MARGIN_MAT_BUCKETS];
+static UINT64 g_uSuperLazyMarginSwingHist[BASE_MARGIN_MAT_BUCKETS][SWING_HIST_BINS];
+static UINT64 g_uSuperLazyMarginSwingSumSq[BASE_MARGIN_MAT_BUCKETS];
static UINT64 g_uSuperLazyMarginExceeded[BASE_MARGIN_MAT_BUCKETS];
+static UINT64 g_uSuperLazyMarginExceededFenSamples[BASE_MARGIN_MAT_BUCKETS];
static void
_RecordMarginSafetySwingInto(IN OUT UINT64 *puMax,
IN OUT UINT64 *puCount,
IN OUT UINT64 *puSum,
+ IN OUT UINT64 *puSumSq,
+ IN OUT UINT64 puHist[][SWING_HIST_BINS],
IN OUT UINT64 *puExceeded,
+ IN OUT UINT64 *puFenSamples,
+ IN CHAR *szTierLabel,
IN POSITION *pos,
IN SCORE iSwing,
IN SCORE iActualMarginUsed)
{
ULONG uMatBucket = _MaterialBucket(pos);
+ ULONG uHistBin = MINU(SWING_HIST_BINS - 1,
+ (ULONG)iSwing / SWING_HIST_BIN_WIDTH);
if (iSwing > iActualMarginUsed)
{
puExceeded[uMatBucket]++;
+ // 2026-09-17: capped FEN sampler, a few examples per (tier,
+ // bucket) to eyeball by hand -- generalized from a 2026-09-16
+ // one-off (super-lazy bucket 0 alone) that first found a real
+ // eval bug this way ("in/out of action"). Cheap: PositionToFen
+ // only runs MARGIN_EXCEEDED_FEN_SAMPLE_CAP times per bucket for
+ // the life of the process, not once per exceeded event.
+ if (puFenSamples[uMatBucket] < MARGIN_EXCEEDED_FEN_SAMPLE_CAP)
+ {
+ CHAR *pszOutlierFen = PositionToFen(pos);
+ if (NULL != pszOutlierFen)
+ {
+ puFenSamples[uMatBucket]++;
+ Trace("MARGINSAFETY_EXCEEDED tier=%s bucket=%u swing=%d margin=%d fen=%s\n",
+ szTierLabel, uMatBucket, (int)iSwing,
+ (int)iActualMarginUsed, pszOutlierFen);
+ SystemFreeMemory(pszOutlierFen);
+ }
+ }
}
ASSERT(iSwing >= 0);
puCount[uMatBucket]++;
puSum[uMatBucket] += (UINT64)iSwing;
+ puSumSq[uMatBucket] += (UINT64)iSwing * (UINT64)iSwing;
+ puHist[uMatBucket][uHistBin]++;
if ((UINT64)iSwing > puMax[uMatBucket])
{
puMax[uMatBucket] = (UINT64)iSwing;
@@ -3203,7 +3318,9 @@ Return value:
**/
{
_RecordMarginSafetySwingInto(g_uMarginSwingMax, g_uMarginSwingCount,
- g_uMarginSwingSum, g_uMarginExceeded,
+ g_uMarginSwingSum, g_uMarginSwingSumSq,
+ g_uMarginSwingHist, g_uMarginExceeded,
+ g_uMarginExceededFenSamples, "lazy",
pos, iSwing, iActualMarginUsed);
}
@@ -3235,24 +3352,151 @@ Return value:
_RecordMarginSafetySwingInto(g_uSuperLazyMarginSwingMax,
g_uSuperLazyMarginSwingCount,
g_uSuperLazyMarginSwingSum,
+ g_uSuperLazyMarginSwingSumSq,
+ g_uSuperLazyMarginSwingHist,
g_uSuperLazyMarginExceeded,
+ g_uSuperLazyMarginExceededFenSamples,
+ "superlazy",
pos, iSwing, iActualMarginUsed);
}
+// Plain Newton's-method sqrt, no <math.h> -- chess.h's own VALUE_KING
+// #defines INFINITY as a board-eval sentinel, which collides with
+// <math.h>'s own INFINITY macro, so libm's sqrt() isn't available
+// here without one shadowing the other. Variance is always small
+// (swings are bounded well under 2^31), so a fixed handful of Newton
+// iterations from a decent seed converges to full double precision.
+static double
+_Sqrt(IN double d)
+{
+ double dGuess;
+ int i;
+
+ if (d <= 0.0)
+ {
+ return 0.0;
+ }
+ dGuess = d;
+ for (i = 0; i < 40; i++)
+ {
+ dGuess = 0.5 * (dGuess + (d / dGuess));
+ }
+ return dGuess;
+}
+
+
+// 2026-09-17: same idea as _PersistRawHistogramsToFile above, for the
+// swing histograms -- lets multiple sharded runs (e.g. N chunks of a
+// large position pool run in parallel on separate cores) be combined
+// after the fact into one true aggregate: sum count/exceeded/sum/
+// sumSq/histogram bins bucket-by-bucket across shard files, then
+// recompute mean/stdev/percentiles from the combined totals. Doing
+// this from raw counts is required for correctness -- percentiles
+// (p10/p50/p90) can't be validly averaged across shards, only derived
+// fresh from the combined histogram.
+static void
+_PersistMarginSwingHistogramToFile(IN FILE *f, IN CHAR *szTierLabel,
+ IN UINT64 *puCount, IN UINT64 *puExceeded,
+ IN UINT64 *puSum, IN UINT64 *puSumSq,
+ IN UINT64 *puMax,
+ IN UINT64 puHist[][SWING_HIST_BINS])
+{
+ ULONG b, i;
+
+ for (b = 0; b < BASE_MARGIN_MAT_BUCKETS; b++)
+ {
+ fprintf(f, "%s %u %" COMPILER_LONGLONG_UNSIGNED_FORMAT " %"
+ COMPILER_LONGLONG_UNSIGNED_FORMAT " %"
+ COMPILER_LONGLONG_UNSIGNED_FORMAT " %"
+ COMPILER_LONGLONG_UNSIGNED_FORMAT " %"
+ COMPILER_LONGLONG_UNSIGNED_FORMAT,
+ szTierLabel, b, puCount[b], puExceeded[b], puSum[b],
+ puSumSq[b], puMax[b]);
+ for (i = 0; i < SWING_HIST_BINS; i++)
+ {
+ fprintf(f, " %llu", (unsigned long long)puHist[b][i]);
+ }
+ fprintf(f, "\n");
+ }
+}
+
+
+void
+PersistMarginSafetyHistogramsToFile(IN CHAR *szPathOverride)
+/**
+
+Routine description:
+
+ Write both tiers' swing histograms (plus count/exceeded/sum/sumSq/
+ max per bucket) to a scratch file, in a format a small external
+ script can sum bucket-by-bucket across multiple shard runs and
+ recompute exact combined statistics from. See the comment on
+ _PersistMarginSwingHistogramToFile above.
+
+Parameters:
+
+ CHAR *szPathOverride : output path, or NULL to use the default
+ (/var/tmp/typhoon/marginsafety_raw_histograms.txt) -- pass a
+ distinct path per shard when running N copies of a large
+ position pool in parallel, since they'd otherwise all clobber
+ the same default file.
+
+Return value:
+
+ void
+
+**/
+{
+ FILE *f;
+ CHAR *szPath = (NULL != szPathOverride) ? szPathOverride :
+ "/var/tmp/typhoon/marginsafety_raw_histograms.txt";
+
+ f = fopen(szPath, "w");
+ if (NULL == f)
+ {
+ Trace("PersistMarginSafetyHistogramsToFile: could not open output file %s.\n",
+ szPath);
+ return;
+ }
+ fprintf(f, "# SWING_HIST_BINS=%d SWING_HIST_BIN_WIDTH=%d SWING_HIST_MIN=0\n",
+ SWING_HIST_BINS, SWING_HIST_BIN_WIDTH);
+ fprintf(f, "# format: <tier> <bucket> <count> <exceeded> <sum> <sumSq> "
+ "<max> <bin0> <bin1> ... <binN-1>\n");
+ _PersistMarginSwingHistogramToFile(f, "lazy", g_uMarginSwingCount,
+ g_uMarginExceeded, g_uMarginSwingSum,
+ g_uMarginSwingSumSq, g_uMarginSwingMax,
+ g_uMarginSwingHist);
+ _PersistMarginSwingHistogramToFile(f, "superlazy",
+ g_uSuperLazyMarginSwingCount,
+ g_uSuperLazyMarginExceeded,
+ g_uSuperLazyMarginSwingSum,
+ g_uSuperLazyMarginSwingSumSq,
+ g_uSuperLazyMarginSwingMax,
+ g_uSuperLazyMarginSwingHist);
+ fclose(f);
+ Trace("Raw margin-safety histograms written to %s\n", szPath);
+}
+
+
static void
_DumpMarginSafetyCalibrationInto(IN CHAR *szLabel,
IN UINT64 *puMax,
IN UINT64 *puCount,
IN UINT64 *puSum,
+ IN UINT64 *puSumSq,
IN UINT64 *puExceeded)
{
ULONG m;
- Trace("%s -- max/avg |real - lazy| swing among exits taken, "
+ Trace("%s -- |real - lazy| swing among exits taken, "
"by material bucket:\n", szLabel);
for (m = 0; m < BASE_MARGIN_MAT_BUCKETS; m++)
{
+ double dMean;
+ double dVariance;
+ double dStdev;
+
if (0 == puCount[m])
{
Trace(" material bucket %u (scaler %u-%u): no exits taken\n",
@@ -3260,25 +3504,42 @@ _DumpMarginSafetyCalibrationInto(IN CHAR *szLabel,
(m * BASE_MARGIN_MAT_WIDTH) + BASE_MARGIN_MAT_WIDTH - 1);
continue;
}
+ dMean = (double)puSum[m] / (double)puCount[m];
+ // E[X^2] - E[X]^2; clamp to 0 to guard against fp rounding
+ // producing a tiny negative just under zero.
+ dVariance = ((double)puSumSq[m] / (double)puCount[m]) -
+ (dMean * dMean);
+ if (dVariance < 0.0)
+ {
+ dVariance = 0.0;
+ }
+ dStdev = _Sqrt(dVariance);
+ // 2026-09-17: no longer printing p10/p50/p90 here -- percentiles
+ // don't combine validly across sharded runs (see
+ // PersistMarginSafetyHistogramsToFile), so a single shard's
+ // in-process percentile read stopped being the useful number
+ // once the workflow moved to persisting raw histogram bins and
+ // computing exact combined percentiles externally instead.
Trace(" material bucket %u (scaler %u-%u): max=%"
- COMPILER_LONGLONG_UNSIGNED_FORMAT " avg=%.1f (n=%"
+ COMPILER_LONGLONG_UNSIGNED_FORMAT " avg=%.1f stdev=%.1f "
+ "(n=%"
COMPILER_LONGLONG_UNSIGNED_FORMAT ") exceeded=%"
COMPILER_LONGLONG_UNSIGNED_FORMAT "\n",
m, m * BASE_MARGIN_MAT_WIDTH,
(m * BASE_MARGIN_MAT_WIDTH) + BASE_MARGIN_MAT_WIDTH - 1,
- puMax[m], (double)puSum[m] / (double)puCount[m],
+ puMax[m], dMean, dStdev,
puCount[m], puExceeded[m]);
}
}
void
-DumpMarginSafetyCalibration(void)
+DumpMarginSafetyCalibration(IN CHAR *szHistogramPathOverride)
/**
Routine description:
- Print, per material bucket, the max and average |real - lazy| swing
+ Print, per material bucket, the max/avg/stdev |real - lazy| swing
observed among nodes that actually took a lazy exit this run
(command.c's "calibrate marginsafety") -- once for the regular lazy
exit (LAZY_EVAL_BASE_MARGIN / EstimatePositionalScore) and once for
@@ -3287,11 +3548,14 @@ Routine description:
two together would hide whichever one is actually unsound. Either
section's max is the number that answers "how large would that
margin need to be, at that material level, before an exit there
- could have been unsound."
+ could have been unsound." Also persists the raw swing histograms
+ to file (see PersistMarginSafetyHistogramsToFile) so percentiles
+ can be computed exactly, including across multiple sharded runs.
Parameters:
- void
+ CHAR *szHistogramPathOverride : forwarded to
+ PersistMarginSafetyHistogramsToFile -- NULL for the default path.
Return value:
@@ -3301,16 +3565,314 @@ Return value:
{
_DumpMarginSafetyCalibrationInto("Regular lazy margin safety",
g_uMarginSwingMax, g_uMarginSwingCount,
- g_uMarginSwingSum, g_uMarginExceeded);
+ g_uMarginSwingSum, g_uMarginSwingSumSq,
+ g_uMarginExceeded);
_DumpMarginSafetyCalibrationInto("Super lazy margin safety",
g_uSuperLazyMarginSwingMax,
g_uSuperLazyMarginSwingCount,
g_uSuperLazyMarginSwingSum,
+ g_uSuperLazyMarginSwingSumSq,
g_uSuperLazyMarginExceeded);
+ PersistMarginSafetyHistogramsToFile(szHistogramPathOverride);
}
#endif // CALIBRATE_MARGIN_SAFETY
+// 2026-09-16/17: shared, file-scope tables for the margin_a/margin_b
+// (super-lazy, in Eval() below) and margin_a'/margin_b'
+// (EstimatePositionalScore, regular lazy, right after this) margin
+// constructions. King safety is ARMY-ONLY for both tiers (no
+// CountKingSafetyDefects call from either -- a joint attacker-army x
+// defect-count table was considered for regular lazy specifically, but
+// dropped: CountKingSafetyDefects is slow despite being bitboard-based,
+// not worth paying for if the simpler table is already good enough;
+// it stays in the codebase for its existing, unrelated search
+// extension/reduction-gate use). Residual is NOT shared between tiers,
+// despite living together here -- iResidualP2/98ByOwnArmy is scoped
+// to regular lazy's own baseline (after bishop pairs: excludes pawns/
+// passers/bad-trade/bishop-pairs, since regular lazy already has all
+// of that by the time it asks); iSuperLazyResidualP2/98ByOwnArmy is
+// scoped to super-lazy's own baseline (pure material: INCLUDES pawns/
+// passers/bad-trade/bishop-pairs, since super-lazy has none of it yet).
+// Using the regular-lazy-scoped table for super-lazy was a real bug in
+// the first version of this redesign -- it silently excluded exactly
+// the terms super-lazy needed most and got quietly papered over with a
+// MAX() against the old flat floor, which defeated the whole point of
+// building a position-specific estimate. Fixed by measuring the right
+// thing instead of re-adding a workaround.
+//
+// Speed/accuracy knob: measured exceeded rate (fraction of lazy/
+// super-lazy exits where the real full-eval score actually would have
+// landed back across alpha/beta -- i.e. this lazy exit was WRONG) by
+// percentile choice, both from the same 30k-position stratified pool,
+// same `sd 6` conditions, via CALIBRATE_MARGIN_SAFETY's `calibrate
+// marginsafety`. Bucket numbering here is the MATERIAL bucket
+// (_MaterialBucket: combined non-pawn army scaler for BOTH sides,
+// divided by 8 -- 0-7) -- NOT the same bucketing as the per-side
+// army-scaler/4 buckets the percentile tables below are keyed by.
+// Buckets 0-1 never take a regular-lazy exit at all (see
+// LAZY_EVAL_MIN_MATERIAL) so have no regular-lazy row.
+//
+// material bucket: 2 3 4 5 6 7
+// (combined scaler) (16-23) (24-31) (32-39) (40-47) (48-55) (56-63)
+// ----------------------- ------- ------- ------- ------- ------- -------
+// regular lazy, p90/p10: 0.97% 0.48% 0.72% 1.61% 1.54% 0.72%
+// regular lazy, p98/p2: 0.046% 0.013% 0.010% 0.029% 0.032% 0.041%
+//
+// material bucket: 1 2 3 4 5 6 7
+// (combined scaler) (8-15) (16-23) (24-31) (32-39) (40-47) (48-55) (56-63)
+// ----------------------- ------- ------- ------- ------- ------- ------- -------
+// super lazy, p90/p10: 9.43% 5.08% 3.73% 1.32% 1.14% 1.99% 0.54%
+// super lazy, p98/p2: 1.08% 0.349% 0.209% 0.045% 0.063% 0.069% 0.018%
+//
+// (Super-lazy has no material-bucket-0 row: excluded entirely, see the
+// bucket-0 comment at its exclusion gate below.)
+//
+// 2026-09-17 re-recalibration: switched from p90/p10 to p98/p2 given
+// the above -- p90/p10 exceeded on a non-trivial fraction of exits
+// (regular lazy: 0.5-1.6% across buckets; super-lazy: 0.5-9.4%, worst
+// in the low-material buckets), p98/p2 brings every bucket in both
+// tiers under 1.1%, an order of magnitude tighter. If p98/p2 still
+// isn't tight enough once self-play validates it, p99/p1 is the next
+// lever -- re-derive via `calibrate residual` against the same 30k
+// pool, no new position sample needed.
+// All four tables below are FINAL, derived from the same 30k-position
+// stratified pool (~104M-240M samples/bucket for residual/king,
+// buckets keyed by scaler/4). Full percentile spread recorded so a
+// future re-tightening (e.g. p99/p1) can read values straight off
+// these charts without re-running the 30k pool -- only the two
+// _P2By.../_P98By... lines actually feed the margin construction
+// below; min/p1/p5/p10/p50/p90/p95/p99/max are kept for reference.
+//
+// Residual by OWN army-scaler bucket (regular-lazy baseline: excludes
+// pawns/passers/bad-trade/bishop-pairs):
+// bucket 0 (scaler 0-3): n=104187433 min=-144 p1=-30 p2=-10 p5=-10 p10=-10 p50=10 p90=110 p95=130 p98=190 p99=230 max=767
+// bucket 1 (scaler 4-7): n=125821895 min=-166 p1=-10 p2=-10 p5=-10 p10=10 p50=50 p90=150 p95=210 p98=250 p99=290 max=817
+// bucket 2 (scaler 8-11): n=239860406 min=-174 p1=-30 p2=-30 p5=-10 p10=-10 p50=30 p90=130 p95=170 p98=230 p99=270 max=761
+// bucket 3 (scaler 12-15): n=190456720 min=-209 p1=-50 p2=-30 p5=-30 p10=-10 p50=30 p90=110 p95=130 p98=190 p99=230 max=708
+// bucket 4 (scaler 16-19): n=216350902 min=-220 p1=-70 p2=-70 p5=-50 p10=-30 p50=30 p90=90 p95=110 p98=150 p99=170 max=735
+// bucket 5 (scaler 20-23): n=189558383 min=-225 p1=-70 p2=-50 p5=-30 p10=-30 p50=30 p90=70 p95=90 p98=130 p99=150 max=728
+// bucket 6 (scaler 24-27): n=135722645 min=-231 p1=-90 p2=-70 p5=-50 p10=-30 p50=10 p90=70 p95=90 p98=110 p99=130 max=760
+// bucket 7 (scaler 28-31): n=164358044 min=-256 p1=-110 p2=-110 p5=-90 p10=-90 p50=-30 p90=30 p95=50 p98=70 p99=90 max=796
+// 2026-09-17 BEST-GUESS #2: guess #1 (p95/p5 uniformly) measured
+// 0.002-0.270% on a 500-position sample (buckets 2-7) -- still under
+// the ~0.4% target, especially buckets 4-5 (0.002% each). Per direct
+// instruction, loosened buckets 2-5 one more step to p90/p10 (values
+// read off the chart above, no new run); buckets 6-7 already looked
+// close to target at p95/p5 (0.270%/0.152%) and are left unchanged.
+// Buckets 0-1 never take a regular-lazy exit at all (LAZY_EVAL_MIN_
+// MATERIAL) so their values are moot either way. Caveat: the 500-
+// position sample is noisy (small event counts in low buckets, and
+// not truly 500 independent draws -- a single game's tree can dominate
+// a material bucket's node population), so this is still a guess
+// pending the next full 30k-position `calibrate marginsafety` run.
+// 2026-09-17 BEST-GUESS #3: real 30k-pool re-check of guess #2 (not
+// the noisy 500-position sample) measured bucket2=0.385% (right on
+// target) but buckets 3-7 all 0.048-0.099%, still well under the
+// 0.4-0.5% target -- loosened further to p85/p15 (exact values from
+// the persisted raw histogram, /var/tmp/typhoon/residual_raw_
+// histograms.txt -- see hist_percentile.py). Bucket 2 left unchanged
+// (already at target).
+// 2026-09-17 BEST-GUESS #4: buckets 0-2 left at guess #3 (0.06-0.39%
+// measured, buckets 0-1 take no lazy exits so moot). Buckets 3-7
+// measured 0.048-0.099%, well under the ~0.4-0.5% target -- loosened
+// one more notch (values read off the persisted 30k histogram via
+// hist_percentile.py, no new sample needed): bucket 3 p85/p15 ->
+// p80/p20, bucket 4 p85/p15 -> p80(tie)/p20, bucket 5 p85/p15 ->
+// p85(tie)/p20, bucket 6 p85/p15 -> p75/p25, bucket 7 p90/p20-ish ->
+// p75/p25 (P98 only, P2 already at its floor). Re-verify with a full
+// `calibrate marginsafety` pass -- not expected to land exactly on
+// target given cross-bucket confounds (see guess #3 comment above).
+// 2026-09-17 BEST-GUESS #5: buckets 3-5 loosened one more notch
+// (p80/p20-ish -> p70/p30, values from the persisted 30k histogram).
+// Bucket 0-2, 6-7 unchanged pending their own re-check (guess #4's
+// 30k run showed bucket 2 already over target without being touched,
+// and buckets 6-7 already over target from guess #4's loosening --
+// see command history).
+// 2026-09-17 BEST-GUESS #6: targeting ~0.66% in every bucket. Bucket 2
+// was drifting up untouched (0.72% guess #4 -> 1.04% guess #5) --
+// tightened p90/p10-ish -> p80/p20. Bucket 5 was under target (0.39%)
+// -- loosened a tad, p70/p30 -> p90/p30 (P98 side only).
+// 2026-09-17 BEST-GUESS #7: targeting ~0.66% in every bucket.
+// Bucket 2 (1.62%, way over) -- guess #6's "tighten" actually
+// narrowed the margin by mistake (tighten = lower exceeded rate =
+// WIDEN the percentile spread, not narrow it); corrected direction,
+// widened past guess #5's own (-10,130) to (-30,190) i.e. p97/p3.
+// Buckets 3/4/6 (0.71-0.74%, slightly over) widened a little (P98
+// +20 each). Bucket 5 (0.30%, under) narrowed back (P98 70->50,
+// reverting guess #6's accidental widen). Bucket 7 (0.47%, under)
+// narrowed a little (P2 -70->-50).
+// 2026-09-17 BEST-GUESS #8: buckets 3-5 came in well under target
+// (0.27-0.39% vs ~0.66%) -- narrowed further for speed (guess #5's
+// bucket 5 narrow was too small a step, barely moved anything;
+// bigger step this time). Bucket 3: p75/p30 -> p60/p40. Bucket 4:
+// P98 only, p75/p40 -> p55/p40 (P2 unchanged, bin ties at that end).
+// Bucket 5: p65/p35-ish -> p55/p45, moved both sides this time.
+// Buckets 0-2/6-7 untouched pending their own re-check.
+// 2026-09-18: restored to the guess-#11 values (11 rounds of
+// calibration against the 30k-position pool and the 200k-position
+// cross-sectional run, see the guess #1-11 history above) after
+// briefly suspecting -- and ruling out, via the 16:15 ZFS snapshot
+// still showing a winning match_play series with guesses #9/#10
+// already applied -- that this table's loosening was the source of
+// the 2026-09-17 self-play regression. The real cause was util.c's
+// WalkPV/MakeUserMove change (reverted separately).
+static const SCORE iResidualPessimisticByOwnArmy[8] =
+{
+ -10, -10, -10, 10, -10, 30, -10, -50,
+};
+static const SCORE iResidualOptimisticByOwnArmy[8] =
+{
+ 130, 210, 170, 50, 70, 30, 50, -10,
+};
+// Super-lazy-scoped residual by OWN army-scaler bucket (material-only
+// baseline: includes pawns/passers/bad-trade/bishop-pairs):
+// bucket 0 (scaler 0-3): n=104187433 min=-363 p1=-110 p2=-90 p5=-50 p10=-30 p50=30 p90=210 p95=290 p98=410 p99=530 max=2383
+// bucket 1 (scaler 4-7): n=125821895 min=-356 p1=-70 p2=-50 p5=-30 p10=-10 p50=90 p90=290 p95=370 p98=490 p99=830 max=2456
+// bucket 2 (scaler 8-11): n=239860406 min=-341 p1=-70 p2=-50 p5=-30 p10=-10 p50=70 p90=250 p95=330 p98=430 p99=530 max=2065
+// bucket 3 (scaler 12-15): n=190456720 min=-324 p1=-70 p2=-50 p5=-30 p10=-10 p50=70 p90=230 p95=290 p98=390 p99=450 max=2183
+// bucket 4 (scaler 16-19): n=216350902 min=-343 p1=-90 p2=-70 p5=-50 p10=-30 p50=50 p90=190 p95=250 p98=310 p99=370 max=1946
+// bucket 5 (scaler 20-23): n=189558383 min=-299 p1=-70 p2=-50 p5=-30 p10=-10 p50=70 p90=170 p95=230 p98=290 p99=330 max=1959
+// bucket 6 (scaler 24-27): n=135722645 min=-322 p1=-90 p2=-70 p5=-50 p10=-30 p50=50 p90=150 p95=190 p98=250 p99=290 max=1921
+// bucket 7 (scaler 28-31): n=164358044 min=-304 p1=-90 p2=-90 p5=-70 p10=-50 p50=10 p90=90 p95=130 p98=170 p99=210 max=2039
+// 2026-09-17 BEST-GUESS #3: real 30k-pool re-check of guess #2 revealed
+// the 500-position sample had misled guess #2 in the dangerous
+// direction on two buckets: material bucket 1 (reverted to p98/p2
+// based on a 500-position reading of 0.035%) actually measured 1.13%
+// at full scale, and material bucket 3 (table never touched at all)
+// measured 0.94% -- a real cross-bucket confound (changing OTHER
+// buckets' margins shifts real search behavior/tree shape everywhere,
+// so even an untouched bucket's measured exceeded rate moves). Per-
+// side buckets 0-3 tightened to p99/p1 (exact values from the
+// persisted raw histogram) to address both -- material buckets 1 and 3
+// are populated by a diffuse mix of per-side bucket combinations, not
+// a clean 1:1 mapping, so this can't precisely target either bucket
+// alone, only reduce risk broadly in the low-army region feeding both.
+// Buckets 4-6 measured 0.22-0.30%, still under target -- loosened
+// further to p85/p15. Bucket 7 (0.15%) and the king table (negligible
+// spread above p90 in every bucket, confirmed against the persisted
+// histogram) left unchanged. Real search-tree-shape changes are
+// expected from this (not just a measurement artifact) -- re-verify
+// with a full `calibrate marginsafety` pass, best-effort, not
+// expected to land exactly on target for every bucket given the
+// diffuse material-bucket-to-per-side-bucket mapping.
+// 2026-09-17 BEST-GUESS #4: bucket 1 was the worst outlier at 1.13%
+// measured (target ~0.4-0.5%) -- tightened one more notch past p99/p1,
+// to p99.5/p0.5 (values from the persisted 30k histogram). Buckets 0
+// and 2-3 left at guess #3 (bucket 3 measured 0.94%, also over target,
+// but a diffuse material-bucket confound with bucket 1 -- see guess #3
+// comment above -- so left alone pending its own re-check rather than
+// guessing a second simultaneous change). Buckets 4-7 measured
+// 0.15-0.30%, well under target -- loosened one more notch: buckets
+// 4-6 p85/p15 -> p80/p20, bucket 7 p90/p10 -> p85/p15.
+// 2026-09-17 BEST-GUESS #5: bucket 3 was over target (0.94% guess #3,
+// 1.23% guess #4, moving the wrong direction despite never being
+// touched -- a cross-bucket confound from bucket 1's changes) --
+// tightened past p99/p1 to p99.5/p0.5, same lever used for bucket 1.
+// Bucket 7 was still under target (0.15%/0.29%) -- loosened one more
+// notch, p85/p15 -> p75/p25 (P98 side only; P2 already at p85's floor).
+// 2026-09-17 BEST-GUESS #6: targeting ~0.66% in every bucket. Bucket
+// 1 (0.458%, under target) was at p99.5/p0.5 -- backed off a tiny bit
+// to p99/p1. Bucket 3 (1.11%, worst outlier, already at p99.5/p0.5)
+// has no looser-percentile lever left in this histogram's resolution
+// -- padded a fixed +120/-20 beyond p99.5/p0.5 instead. Bucket 6
+// (0.96%, over target, was p80/p20) tightened to p90/p10. Bucket 7
+// (0.36%, under target, was p75/p25) loosened a tad, P2 side only
+// (p25 -> p30-ish).
+// 2026-09-17 BEST-GUESS #7: targeting ~0.66% in every bucket.
+// Bucket 1 (0.49%, slightly under) narrowed a little (P2 -70->-50).
+// Buckets 2/4/6 (0.68/0.61/0.71%) already close enough -- left
+// alone. Bucket 3 (1.09%, still worst, already past p99.5) widened
+// further via a bigger pad (-110,650) -> (-150,850). Buckets 5/7
+// (0.54%/0.56%, slightly under) narrowed a little (P98 -20 each).
+// 2026-09-17 BEST-GUESS #9: same 200k/sd3 cross-sectional finding as
+// the regular-lazy table above, new target 1.5%. Bucket 2 (2.99%,
+// worst of all despite never being touched before -- already sitting
+// at p99/p0.5, percentile table exhausted) padded further, same lever
+// used for bucket 3 previously. Bucket 3 (1.95%, least-bad of these
+// three but still over, already heavily padded) padded a bit more.
+// Bucket 6 (2.19%) widened p90/p10 -> p96/p6.
+// 2026-09-17 BEST-GUESS #10: target ~1-1.5%. Bucket 1 (0.34%, well
+// under) loosened, p99/p2-ish -> ~p98/p4. Bucket 2 (3.45%, worst --
+// got WORSE after widening in guess #9 despite already sitting past
+// p99.5) padded further again regardless; this bucket has resisted
+// two widening passes in a row, likely the same cross-bucket-combo
+// issue diagnosed for bucket 3 earlier (material bucket != per-side
+// bucket, diffuse mapping) rather than something this table alone can
+// fix -- flagging for a real root-cause dig rather than more blind
+// widening if this doesn't help either. Buckets 3-7 left alone.
+// 2026-09-17 BEST-GUESS #11: bucket 2 (2.97%, still worst after three
+// widening rounds in a row -- see guess #10's note, likely a
+// cross-bucket-combo issue this table alone can't fully fix) padded
+// further again. Rest left alone.
+// 2026-09-18: restored to the guess-#11 values (see the matching note
+// on iResidualPessimisticByOwnArmy above) after ruling out this
+// table's loosening as the cause of the 2026-09-17 self-play
+// regression -- the 16:15 ZFS snapshot already had guesses #9/#10
+// applied and was still winning. The real cause was util.c's WalkPV/
+// MakeUserMove change (reverted separately).
+static const SCORE iSuperLazyResidualPessimisticByOwnArmy[8] =
+{
+ -110, -30, -190, -170, 10, 10, -50, -10,
+};
+static const SCORE iSuperLazyResidualOptimisticByOwnArmy[8] =
+{
+ 530, 490, 1050, 950, 150, 110, 190, 40,
+};
+// King safety by ATTACKER army-scaler bucket:
+// bucket 0 (scaler 0-3): n=104187433 min=-68 p1=-10 p2=10 p5=10 p10=10 p50=10 p90=30 p95=30 p98=50 p99=50 max=165
+// bucket 1 (scaler 4-7): n=125821895 min=0 p1=10 p2=10 p5=10 p10=10 p50=10 p90=30 p95=30 p98=50 p99=50 max=162
+// bucket 2 (scaler 8-11): n=239860406 min=-89 p1=-30 p2=-30 p5=-30 p10=-10 p50=10 p90=10 p95=10 p98=30 p99=30 max=162
+// bucket 3 (scaler 12-15): n=190456720 min=-176 p1=-70 p2=-70 p5=-50 p10=-50 p50=-10 p90=-10 p95=10 p98=10 p99=10 max=137
+// bucket 4 (scaler 16-19): n=216350902 min=-307 p1=-190 p2=-170 p5=-130 p10=-110 p50=-30 p90=-10 p95=-10 p98=-10 p99=10 max=129
+// bucket 5 (scaler 20-23): n=189558383 min=-354 p1=-250 p2=-230 p5=-190 p10=-130 p50=-50 p90=-30 p95=-10 p98=-10 p99=-10 max=45
+// bucket 6 (scaler 24-27): n=135722645 min=-401 p1=-270 p2=-250 p5=-170 p10=-150 p50=-70 p90=-30 p95=-10 p98=-10 p99=-10 max=13
+// bucket 7 (scaler 28-31): n=164358044 min=-394 p1=-190 p2=-170 p5=-130 p10=-90 p50=-50 p90=-10 p95=-10 p98=-10 p99=10 max=6
+static const SCORE iKingP2ByAttackerArmy[8] =
+{
+ 10, 10, -30, -70, -170, -230, -250, -170,
+};
+static const SCORE iKingP98ByAttackerArmy[8] =
+{
+ 50, 50, 30, 10, -10, -10, -10, -10,
+};
+
+// 2026-09-17: p50 tables, same 30k-position pool, same bucketing as
+// their P2/P98 siblings above -- but used for a different purpose:
+// a signed, unbounded (well, table-bounded) *typical-value estimate*
+// of that side's own missing positional terms, fed to QSearch's
+// futility margin as iPositional[side] on a lazy/super-lazy exit
+// (matches full-eval's own iPositional convention exactly -- see
+// Eval()'s "Adjust dynamic positional component" block below, which
+// computes a side's *own* real positional score minus its material,
+// not a relative-to-opponent quantity). This is deliberately NOT the
+// p98/p2 pessimistic/optimistic pairing margin_a/margin_b use --
+// p50 is the right statistic for a bias correction, p98/p2 is the
+// right statistic for a soundness bound. iPositional[color] =
+// iKingP50ByAttackerArmyOnly[opponent's army bucket] (opponent is the
+// attacker of this side's own king) + iResidualP50ByOwnArmy[this
+// side's own army bucket] (or the super-lazy-scoped sibling, for the
+// super-lazy exit).
+// residual p50 (regular-lazy baseline), by OWN army bucket:
+// 10, 50, 30, 30, 30, 30, 10, -30
+// super-lazy-scoped residual p50 (material-only baseline), by OWN army bucket:
+// 30, 90, 70, 70, 50, 70, 50, 10
+// king safety p50, by ATTACKER army bucket:
+// 10, 10, 10, -10, -30, -50, -70, -50
+static const SCORE iResidualP50ByOwnArmy[8] =
+{
+ 10, 50, 30, 30, 30, 30, 10, -30,
+};
+static const SCORE iSuperLazyResidualP50ByOwnArmy[8] =
+{
+ 30, 90, 70, 70, 50, 70, 50, 10,
+};
+static const SCORE iKingP50ByAttackerArmy[8] =
+{
+ 10, 10, 10, -10, -30, -50, -70, -50,
+};
+
+
static void
EstimatePositionalScore(IN SEARCHER_THREAD_CONTEXT *ctx,
IN POSITION *pos,
@@ -3325,16 +3887,15 @@ Routine description:
the alpha/beta margins to account for the positional terms that
have not been computed yet at this point in Eval() -- king safety
plus a flat residual covering mobility, passers, and everything
- else that genuinely requires attack-generation to know exactly.
+ else that genuinely requires full evaluation to find out.
Both terms below are calibrated from measured data (CALIBRATE_
- POSITIONAL instrumentation, ~1.6M full-eval samples over an ECM
- slice), not guessed: for each value, p90 of the *actual* score
- swing between this point in Eval() and full-eval completion, so
- the resulting margin is wrong (too narrow) on at most ~10% of
- calls, in either bucket. See conversation history for the
- percentile tables -- if re-tuning, regenerate them, don't hand-
- edit these numbers.
+ POSITIONAL instrumentation, 200M full-eval samples: for each
+ value, p90 of the *actual* score swing between this point in
+ Eval() and full-eval completion, so the resulting margin is wrong
+ (too narrow) on at most ~10% of calls, in either bucket. See
+ conversation history for the percentile tables -- if re-tuning,
+ regenerate them, don't hand- edit these numbers.
Note: king safety and "everything else" turned out to be close to
independent of piece count and of each other, so summing their
@@ -3366,125 +3927,103 @@ Return value:
**/
{
- // p90 of the real, unscaled |KING_SAFETY_BY_COUNTER| magnitude
- // _EvalKing computes, indexed per-side by MINU(10,
- // CountKingSafetyDefects(side)).
+ // 2026-09-16 redesign: replaces the old symmetric "sum both sides'
+ // unscaled p90 magnitude, rescale by attacker army at runtime"
+ // construction entirely -- that mechanism pooled the p90 fit
+ // across every material level for a given CountKingSafetyDefects
+ // bucket, then linearly rescaled by attacker army at use time; that
+ // rescale assumed a linear relationship between "how much raw
+ // danger a defect-count implies" and "how much material the
+ // attacker actually has" that the real terms (each individually
+ // gated on specific attacking pieces existing) don't obey -- a
+ // structural mismatch, not just a stale fit (see conversation
+ // history for the full derivation).
//
- // STALE as of 2026-09-06 (board_representation/EVAL.md section 9):
- // Refit 2026-09-15 against _EvalKing's linear-SCORE rewrite (king_
- // safety_data/HANDOFF.md's 2026-09-15 addendum), replacing the
- // pre-rewrite table below (which was itself already flagged stale --
- // fit against a since-changed bucket index, see git history).
- // _RecordPositionalCalibration's own bucket index was ALSO fixed at
- // the same time (dropped _GetFileStormDefects, which had silently
- // drifted out of sync with this function's own CountKingSafetyDefects-
- // only lookup back on 2026-09-06) -- this refit is against the
- // corrected, now-matching index, not a second layer of the same
- // mismatch. CountKingSafetyDefects itself is unchanged (its backing
- // table, KING_INITIAL_COUNTER_BY_LOCATION, was deliberately split
- // off from _EvalKing's own copy -- see KING_EVAL_INITIAL_LOCATION --
- // specifically so this recalibration never needs to touch it again).
+ // New approach: two direct, jointly-collected percentile tables --
+ // king safety bucketed by the ATTACKER's own army scaler (danger
+ // to a king is a function of the attacker's material, not the
+ // king-owner's), and residual (mobility/trapped/misc, with
+ // material/pawns/passers/bad-trade/bishop-pairs/king safety already
+ // excluded) bucketed by each side's OWN army scaler (residual is
+ // relative/congestion-driven, not attacker-driven, unlike king
+ // safety -- bucketing it by one side's own army alone, ignoring the
+ // enemy, was tried and came back noisy/non-monotonic; the
+ // differential does not, but this per-side percentile construction
+ // sidesteps needing the differential directly by keying each side's
+ // OWN contribution off its OWN army only, then combining per-side
+ // via the asymmetric construction below).
//
- // Sample: 5305 positions (5005 real TWIC games >=2400 Elo, quiet
- // plies, plus 300 ECM positions mixed in for tactical-position
- // coverage -- TWIC chosen as the primary source since it's
- // representative of ordinary cross-sectional positions, unlike
- // ECM's attack-the-king selection bias), sd4 each, 31.8M real
- // Eval() calls / 17.1M samples through the super-lazy gate. Bucket
- // 10 had zero samples in this run (the new terms apparently rarely
- // if ever co-occur with CountKingSafetyDefects>=10) -- extrapolated
- // from the bucket 8->9 trend (+25/step) rather than left stale;
- // re-run with a larger/more adversarial sample if this bucket ever
- // matters in practice.
+ // The margin construction itself mirrors this file's existing
+ // QSearch futility precedent (iPositional is returned per-color,
+ // not as a single "estimate minus full eval" blob, because a
+ // combined delta lets the two sides' real swings partially cancel
+ // and hides the structure that actually matters) -- and is the
+ // same operation as futility pruning, one stack frame up: for the
+ // fail-high check (score - margin_a > beta), soundness requires
+ // assuming the most PESSIMISTIC-for-me, OPTIMISTIC-for-my-opponent
+ // outcome for each term; the fail-low check (score + margin_b <
+ // alpha) requires the mirror. For every term, uniformly (no sign-
+ // flipping needed regardless of whether the raw term is usually
+ // positive or negative):
//
- // search.c's ">2" and searchsup.c's ">1" fixed-threshold extension
- // gates still read CountKingSafetyDefects' raw, unscaled return
- // value directly -- untouched by this refit (that function's own
- // behavior didn't change), so no separate recalibration needed there.
+ // margin_a = [opponent's p90(term) - my p10(term)], summed over terms
+ // margin_b = [my p90(term) - opponent's p10(term)], summed over terms
//
- static const SCORE iKingSwingP90[11] = {
- 30, 65, 90, 120, 155, 190, 220, 255, 265, 290, 315
- };
- // p90 of |mobility + passers + everything else combined|, measured
- // directly (no useful correlation found with piece count).
- static const SCORE iResidualP90 = 154;
- // Per-side lookup-then-scale, deliberately mirroring _EvalKing's
- // own shape exactly (look the raw counter up in a table first,
- // *then* scale that resulting estimate by the attacker's army,
- // rather than scaling the raw counter before the lookup) -- by
- // direct instruction, so this stays as symmetric as possible with
- // the real term it's estimating. CountKingSafetyDefects itself is
- // untouched by this: it keeps returning a small, unscaled count,
- // used as-is by its other two callers.
- SCORE iWhiteKingSwing;
- SCORE iBlackKingSwing;
- SCORE iKingTerm;
- ULONG uWhiteDefects, uBlackDefects;
-#ifdef EVAL_TIME
- UINT64 uSubTimer;
-#endif
-
-#ifdef EVAL_TIME
- uSubTimer = SystemReadTimeStampCounter();
-#endif
- uWhiteDefects = CountKingSafetyDefects(pos, WHITE);
- uBlackDefects = CountKingSafetyDefects(pos, BLACK);
-#ifdef EVAL_TIME
- ctx->sCounters.tree.u64CyclesEvalCountKingSafetyDefects +=
- (SystemReadTimeStampCounter() - uSubTimer);
-#endif
- iWhiteKingSwing = iKingSwingP90[MINU(10, uWhiteDefects)];
- iBlackKingSwing = iKingSwingP90[MINU(10, uBlackDefects)];
-
- iWhiteKingSwing = (iWhiteKingSwing *
- (SCORE)REDUCED_MATERIAL_DOWN_SCALER[pos->uArmyScaler[BLACK]]) / 8;
- iBlackKingSwing = (iBlackKingSwing *
- (SCORE)REDUCED_MATERIAL_DOWN_SCALER[pos->uArmyScaler[WHITE]]) / 8;
- iKingTerm = iWhiteKingSwing + iBlackKingSwing;
+ // King safety is cross-referenced (my king's term uses the
+ // OPPONENT's army bucket, since they're the attacker; the
+ // opponent's king term uses MY army bucket) -- residual is not
+ // (each side's residual is keyed by that side's own army).
+ //
+ // PROVISIONAL tables below: from a 200-position stratified
+ // (material-balanced) sample, ~8.76M full-eval samples -- adequate
+ // for validating the construction and getting real numbers in
+ // place, but due for a refresh from the much larger 30k-position
+ // stratified histogram run once it lands (board_representation/
+ // EVAL.md section 9 / 2026-09-16 super-lazy speed investigation).
+ // Re-derive via `calibrate residual` (DUMP_RESIDUAL_SAMPLES build).
+ // Uses the file-scope army-only king table (iKingP2/98ByAttacker-
+ // ArmyOnly) and residual table (iResidualP2/98ByOwnArmy), shared
+ // with super-lazy's identical construction below in Eval(). A
+ // joint (attacker-army x defect-count) king table was considered
+ // (CountKingSafetyDefects would supersede the army-only estimate
+ // once we're already committed to the more expensive lazy-eval
+ // path) but deliberately dropped -- CountKingSafetyDefects is slow
+ // despite being bitboard-based, not worth paying for here if the
+ // simpler army-only table is already good enough. It stays in the
+ // codebase for its existing, unrelated use as a search extension/
+ // reduction gate (search.c/searchsup.c) -- just no longer called
+ // from eval's margin construction at all, either tier.
+ ULONG uColor = pos->uToMove;
+ ULONG xColor = FLIP(uColor);
+ ULONG uMyBucket = MINU(7, pos->uArmyScaler[uColor] / 4);
+ ULONG uOppBucket = MINU(7, pos->uArmyScaler[xColor] / 4);
+ SCORE iSelfPessimisticEnemyOptimisticMargin, iSelfOptimisticEnemyPessimisticMargin;
- *piAlphaMargin += iKingTerm + iResidualP90;
- *piBetaMargin += iKingTerm + iResidualP90;
+ iSelfPessimisticEnemyOptimisticMargin = MAX0((iKingP98ByAttackerArmy[uMyBucket] -
+ iKingP2ByAttackerArmy[uOppBucket]) +
+ (iResidualOptimisticByOwnArmy[uOppBucket] -
+ iResidualPessimisticByOwnArmy[uMyBucket]));
+ iSelfOptimisticEnemyPessimisticMargin = MAX0((iKingP98ByAttackerArmy[uOppBucket] -
+ iKingP2ByAttackerArmy[uMyBucket]) +
+ (iResidualOptimisticByOwnArmy[uMyBucket] -
+ iResidualPessimisticByOwnArmy[uOppBucket]));
- // 2026-09-08: the p90 estimate above, while within its own
- // documented "~10% wrong" design tolerance, still let the regular
- // lazy exit's real swing exceed the margin actually used 2.11-
- // 2.38% of the time at combined army scaler 16-31 (CALIBRATE_
- // MARGIN_SAFETY, 1500 real-game positions, tests/twic_sample.ep_,
- // sd 8) -- not because the swing itself is much bigger there (max
- // observed 855/779, comparable to every other bucket's 603-701),
- // but because REDUCED_MATERIAL_DOWN_SCALER shrinks the king term
- // faster than the swing actually shrinks at that material level.
- // Floor the combined margin at the measured max-per-bucket (with
- // ~25% headroom) rather than re-deriving the whole p90 shape --
- // this leaves the estimate above as the primary driver wherever
- // it's already wide enough (buckets 4-7, where it already clears
- // these floors) and only kicks in where it wasn't. Buckets 0-1
- // never take a regular lazy exit at all (see LAZY_EVAL_MIN_
- // MATERIAL) so were never measured; use bucket 2's floor there
- // defensively rather than leaving them unfloored. Provisional --
- // re-derive with `calibrate marginsafety` if this margin's shape
- // changes again.
- {
- // 2026-09-08 re-calibration: original values above measured
- // |real - lazy| swing symmetrically, conflating "swung toward
- // the boundary" (the only direction that can make an exit
- // unsound) with "swung further away" (harmless). Re-measured
- // with a directional split (CALIBRATE_MARGIN_SAFETY, same
- // 1500-position tests/twic_sample.ep_, sd 8): the true
- // dangerous-direction max is 30-80% smaller in every bucket
- // than the old symmetric max, and every bucket showed
- // exceeded=0 even before adding headroom. New values are
- // directional-max * ~1.15-1.3 (more headroom on the smaller
- // buckets, where sample size is thinner).
- static const SCORE iSwingFloorByArmy[8] =
- {
- 297, 297, 297, 286, 461, 453, 582, 600,
- };
- ULONG uMatBucket = MINU(
- 7, (pos->uArmyScaler[WHITE] + pos->uArmyScaler[BLACK]) / 8);
- *piAlphaMargin = MAX(*piAlphaMargin, iSwingFloorByArmy[uMatBucket]);
- *piBetaMargin = MAX(*piBetaMargin, iSwingFloorByArmy[uMatBucket]);
- }
+ // Caller's iBetaMargin gates "score - margin > beta" (fail-high,
+ // needs margin_a); iAlphaMargin gates "score + margin < alpha"
+ // (fail-low, needs margin_b).
+ //
+ // 2026-09-17: the flat iSwingFloorByArmy MAX() floor that used to
+ // sit here is GONE, for the same reason super-lazy's analogous
+ // floor was already dropped (see Eval()'s super-lazy margin
+ // construction, above) -- a MAX() against a much larger flat number
+ // just silently overrides the position-specific estimate in most
+ // cases, defeating the entire point of building it. margin_a/
+ // margin_b now stand alone. If CALIBRATE_MARGIN_SAFETY turns up a
+ // real, non-trivial exceeded rate once this is validated against
+ // the 30k pool, the fix is tightening the percentile choice (e.g.
+ // p95/p5 or p99/p1), not resurrecting a flat backstop.
+ *piAlphaMargin += iSelfOptimisticEnemyPessimisticMargin;
+ *piBetaMargin += iSelfPessimisticEnemyOptimisticMargin;
}
@@ -3671,75 +4210,74 @@ Return value:
// count, no x-ray -- simply excluded by the friendly-occupied
// mask, nothing else needed.
//
- {
- BITBOARD bbAttack = _BishopAttacksBB(c, pos->bbOccupied);
- BITBOARD bbEnemySame = pos->bbPieces[xColor][BISHOP] |
- pos->bbPieces[xColor][KNIGHT];
- BITBOARD bbEnemyGEContinue = pos->bbPieces[xColor][ROOK] |
- pos->bbPieces[xColor][QUEEN] |
- COOR_TO_BB(KingPosition(pos, xColor));
- BITBOARD bbFriendBQ = pos->bbPieces[uColor][BISHOP] |
- pos->bbPieces[uColor][QUEEN];
- BITBOARD bbUnsafeForMinor = pos->bbPawnAttacks[xColor];
- BITBOARD bbExclude = 0;
- BITBOARD bbSeen = 0;
- BITBOARD bbLayer = bbAttack;
- BITBOARD bbMobility = 0;
- BITBOARD bbXrayAccum = 0;
- BITBOARD bbFirstLayerMask = 0;
+ BITBOARD bbAttack = _BishopAttacksBB(c, pos->bbOccupied);
+ BITBOARD bbEnemySame = pos->bbPieces[xColor][BISHOP] |
+ pos->bbPieces[xColor][KNIGHT];
+ BITBOARD bbEnemyGEContinue = pos->bbPieces[xColor][ROOK] |
+ pos->bbPieces[xColor][QUEEN] |
+ COOR_TO_BB(KingPosition(pos, xColor));
+ BITBOARD bbFriendBQ = pos->bbPieces[uColor][BISHOP] |
+ pos->bbPieces[uColor][QUEEN];
+ BITBOARD bbUnsafeForMinor = pos->bbPawnAttacks[xColor];
+ BITBOARD bbExclude = 0;
+ BITBOARD bbSeen = 0;
+ BITBOARD bbLayer = bbAttack;
+ BITBOARD bbMobility = 0;
+ BITBOARD bbXrayAccum = 0;
+ BITBOARD bbFirstLayerMask = 0;
- pos->bbMinorAttacks[uColor] |= bbAttack;
+ pos->bbMinorAttacks[uColor] |= bbAttack;
- //
- // Unified chain walk over both mobility and x-ray population
- // in one pass -- same technique and same reason as rook's
- // identical rewrite (board_representation/EVAL.md section 9):
- // the old mailbox walk's BMOB_FRIEND_XRAY/BMOB_ENEMY_GREATER
- // cases don't just x-ray past their blocker for attack-bit
- // purposes, they keep walking (fStop=FALSE) and keep crediting
- // mobility for whatever lies beyond. The originally-committed
- // bitboard version (and its single-hop-then-chain-following
- // x-ray fix) only ever handled the near side for *mobility*,
- // an under-count caught the same way rook's was: a two-bishop
- // battery on one diagonal with open squares beyond (old
- // mailbox: 12/8 mobility for the two bishops; near-side-only
- // bitboard version: 11/5).
- //
- for (;;)
- {
- BITBOARD bbNew = bbLayer & ~bbSeen;
- BITBOARD bbBlockersHere = bbNew & pos->bbOccupied;
- BITBOARD bbEmptyHere = bbNew & ~pos->bbOccupied;
- BITBOARD bbFriendBQHere = bbBlockersHere & bbFriendBQ;
- BITBOARD bbEnemySameHere = bbBlockersHere & bbEnemySame;
- BITBOARD bbEnemyGEHere = bbBlockersHere & bbEnemyGEContinue;
- BITBOARD bbFriendPawnHere = bbBlockersHere & pos->bbPawns[uColor];
- BITBOARD bbEnemyPawnHere = bbBlockersHere &
- pos->bbPawns[xColor];
- BITBOARD bbTransientHere = bbFriendPawnHere & bbPc;
- BITBOARD bbContinueHere;
+ //
+ // Unified chain walk over both mobility and x-ray population
+ // in one pass -- same technique and same reason as rook's
+ // identical rewrite (board_representation/EVAL.md section 9):
+ // the old mailbox walk's BMOB_FRIEND_XRAY/BMOB_ENEMY_GREATER
+ // cases don't just x-ray past their blocker for attack-bit
+ // purposes, they keep walking (fStop=FALSE) and keep crediting
+ // mobility for whatever lies beyond. The originally-committed
+ // bitboard version (and its single-hop-then-chain-following
+ // x-ray fix) only ever handled the near side for *mobility*,
+ // an under-count caught the same way rook's was: a two-bishop
+ // battery on one diagonal with open squares beyond (old
+ // mailbox: 12/8 mobility for the two bishops; near-side-only
+ // bitboard version: 11/5).
+ //
+ for (;;)
+ {
+ BITBOARD bbNew = bbLayer & ~bbSeen;
+ BITBOARD bbBlockersHere = bbNew & pos->bbOccupied;
+ BITBOARD bbEmptyHere = bbNew & ~pos->bbOccupied;
+ BITBOARD bbFriendBQHere = bbBlockersHere & bbFriendBQ;
+ BITBOARD bbEnemySameHere = bbBlockersHere & bbEnemySame;
+ BITBOARD bbEnemyGEHere = bbBlockersHere & bbEnemyGEContinue;
+ BITBOARD bbFriendPawnHere = bbBlockersHere & pos->bbPawns[uColor];
+ BITBOARD bbEnemyPawnHere = bbBlockersHere &
+ pos->bbPawns[xColor];
+ BITBOARD bbTransientHere = bbFriendPawnHere & bbPc;
+ BITBOARD bbContinueHere;
- bbSeen |= bbNew;
- bbXrayAccum |= (bbNew & bbFirstLayerMask);
- bbFirstLayerMask = ~(BITBOARD)0;
+ bbSeen |= bbNew;
+ bbXrayAccum |= (bbNew & bbFirstLayerMask);
+ bbFirstLayerMask = ~(BITBOARD)0;
- bbMobility |= (bbEmptyHere & ~bbUnsafeForMinor) |
- bbEnemySameHere | bbEnemyGEHere |
- (bbEnemyPawnHere & ~bbUnsafeForMinor) |
- bbTransientHere;
+ bbMobility |= (bbEmptyHere & ~bbUnsafeForMinor) |
+ bbEnemySameHere | bbEnemyGEHere |
+ (bbEnemyPawnHere & ~bbUnsafeForMinor) |
+ bbTransientHere;
- bbContinueHere = bbFriendBQHere | bbEnemyGEHere;
- if (!bbContinueHere)
- {
- break;
- }
- bbExclude |= bbContinueHere;
- bbLayer = _BishopAttacksBB(c, pos->bbOccupied & ~bbExclude);
+ bbContinueHere = bbFriendBQHere | bbEnemyGEHere;
+ if (!bbContinueHere)
+ {
+ break;
}
- pos->bbMinorXrayAttacks[uColor] |= bbXrayAccum;
-
- uTotalMobility = CountBits(bbMobility);
+ bbExclude |= bbContinueHere;
+ bbLayer = _BishopAttacksBB(c, pos->bbOccupied & ~bbExclude);
}
+ pos->bbMinorXrayAttacks[uColor] |= bbXrayAccum;
+
+ uTotalMobility = CountBits(bbMobility);
+
ASSERT(uTotalMobility <= 13);
EVAL_TERM(uColor,
BISHOP,
@@ -4847,6 +5385,31 @@ Return value:
//
if ((u == 2) && (uColor == WHITE))
{
+ // 2026-09-16: this used to sum the raw per-pawn king-
+ // distance differential (each term in [-7,+7]) over EVERY
+ // pawn on the board, then multiply the whole sum by 12 --
+ // unbounded in pawn count, so a genuine K+P endgame with
+ // 10+ pawns still on the board could reach several hundred
+ // centipawns from this one soft "my king is a bit more
+ // active" term (confirmed directly: CALIBRATE_MARGIN_
+ // SAFETY-adjacent sampling turned up real +391/-384
+ // outliers traced to exactly this line). That's worth more
+ // than a whole extra rook or bishop would be anywhere else
+ // in this eval, for a term that only ever fires when
+ // NEITHER side has a single piece left besides pawns (u==2
+ // gate) -- clearly wrong on its face regardless of the
+ // exact right number. Kept the raw per-pawn sum (rather
+ // than normalizing to an average) deliberately -- more
+ // pawns still on the board is a real reason this term
+ // could matter more, not just noise to average away -- and
+ // instead shrunk the per-pawn weight from 12 to 2, so even
+ // an unrealistic worst case (16 pawns, every one at the
+ // max +/-7 distance differential) caps at 224 and realistic
+ // cases land around 40-80, versus the old 500-700+.
+ // PROVISIONAL: the "2" stops the runaway magnitude but
+ // isn't derived from any calibration. Needs the same self-
+ // play validation discipline as every other king-safety
+ // magnitude in this file before trusting the new ceiling.
i = 0;
bb = (pos->bbPawns[WHITE] | pos->bbPawns[BLACK]);
while(IS_ON_BOARD(cSquare = CoorFromBitBoardRank8ToRank1(&bb)))
@@ -4858,7 +5421,7 @@ Return value:
KING,
c,
iKingScore,
- i * 12,
+ i * 2,
"in/out of \"action\"");
}
}
@@ -4958,6 +5521,55 @@ Return value:
cQueen = FILE_RANK_TO_COOR(FILE(c), QUEENING_RANK[uColor]);
ASSERT(FILE(cQueen) == FILE(c));
+ // 2026-09-18: rook pawn, strong side's OWN king on
+ // the same file, on or adjacent to its own
+ // queening square -- i.e. the strong king is in
+ // front of its own pawn. Checked up front, before
+ // either race-credit computation below gets a
+ // chance to touch uRacerDist[uColor] for this
+ // pawn, and skips via `continue` (not by resetting
+ // uRacerDist[uColor], which would also wipe out a
+ // legitimate credit already earned by a different
+ // passer of the same color from an earlier loop
+ // iteration -- bb[uColor] can hold more than one
+ // passer). A friendly king blocking its own rook
+ // pawn's queening square is a classic drawing
+ // resource (the defender can potentially reach a
+ // stalemate), not a neutral or helpful thing --
+ // unlike a center pawn, where the king merely costs
+ // a tempo (handled below via the existing +1 case),
+ // a rook pawn's single queening square occupied by
+ // its own king has no safe way to step aside
+ // without delicate tempo/opposition analysis this
+ // cheap eval-time heuristic can't do (that's what
+ // recogn.c's _RecognizeKPK critical-square logic is
+ // for). Found via fuzzing tests/cross_sectional_
+ // 200k.ep_: two of this session's real recognizer/
+ // eval bugs both had the strong side's own king in
+ // front of its own rook pawn (8/8/8/8/8/8/4K2p/7k
+ // w - - -- White can stalemate Black rather than
+ // let Ph2 queen).
+ //
+ // Distance must be exactly 0 (king ON the queening
+ // square), not <=1 -- if the king is merely
+ // adjacent, the queening square itself is empty
+ // and the pawn walks in and promotes next move
+ // with nothing in the way; that's a genuine win,
+ // not a stalemate risk. Only the king literally
+ // occupying its own pawn's only queening square is
+ // the danger this guards against.
+ if (((A == FILE(c)) || (H == FILE(c))) &&
+ (FILE(c) == FILE(KingPosition(pos, uColor))) &&
+ (0 == DISTANCE(KingPosition(pos, uColor), cQueen)))
+ {
+ // Matches the unconditional assignment every
+ // other path through this loop body makes --
+ // this color's own passer was genuinely
+ // examined here, even though disqualified.
+ fDontCountMeOut[uColor] = TRUE;
+ continue;
+ }
+
uPawnDist = RANK_DISTANCE(cQueen, c);
if ((RANK(c) == JUMPING_RANK[uColor]) &&
(IS_EMPTY(pos->rgSquare[c + d1].pPiece) &&
@@ -4998,6 +5610,14 @@ Return value:
uRacerDist[uColor] = MINU(uRacerDist[uColor],
(uPawnDist + 1));
}
+ // else: rook pawn, own king on the same file
+ // but more than 1 square from the queening
+ // square -- the specific adjacent/on-queening-
+ // square danger this session found (see the
+ // up-front `continue` guard above, right after
+ // cQueen is computed) doesn't apply here; leave
+ // as-is (no extra credit, no suppression),
+ // same as this branch's original behavior.
}
fDontCountMeOut[uColor] = TRUE;
@@ -5344,8 +5964,14 @@ Return value:
ASSERT(IS_ON_BOARD(c));
if (_WhoControlsSquareFast(pos, c) == FLIP(uColor))
{
-#ifdef DEBUG
+ // Unconditional even outside DEBUG builds: EVAL_TERM
+ // below reads p unconditionally (its PIE argument
+ // feeds EvalTrace() in EVAL_TRACE builds), so an
+ // assignment gated behind #ifdef DEBUG alone left p
+ // uninitialized on read in a non-DEBUG/EVAL_TRACE
+ // build.
p = pos->rgSquare[c].pPiece;
+#ifdef DEBUG
ASSERT(p);
ASSERT(!IS_PAWN(p));
ASSERT(GET_COLOR(p) == uColor);
@@ -5480,6 +6106,8 @@ Return value:
ULONG uEnemy = FLIP(uSide);
INT iMajorDiff;
INT iMinorDiff;
+ ULONG uSideKnights;
+ ULONG uSideBishops;
//
// A pawn always gives some winning chances (it can always try to
@@ -5490,14 +6118,20 @@ Return value:
return(TRUE);
}
+ // uSide's knight/bishop counts are each read three times below
+ // (the lone-minor check, iMinorDiff, and the two-knights check) --
+ // compute them once and reuse rather than re-popcounting the same
+ // two bitboards repeatedly.
+ uSideKnights = FastCountBits(pos->bbPieces[uSide][KNIGHT]);
+ uSideBishops = FastCountBits(pos->bbPieces[uSide][BISHOP]);
+
//
// No pawns and the only piece besides the king is a single minor:
// never enough material to force mate (K+N or K+B vs anything is
// never a forced win on material alone).
//
if ((2 == TotalNonPawnCount(pos, uSide)) &&
- (1 == (FastCountBits(pos->bbPieces[uSide][KNIGHT]) +
- FastCountBits(pos->bbPieces[uSide][BISHOP]))))
+ (1 == (uSideKnights + uSideBishops)))
{
return(FALSE);
}
@@ -5516,8 +6150,7 @@ Return value:
{
iMinorDiff = (INT)(FastCountBits(pos->bbPieces[uEnemy][KNIGHT]) +
FastCountBits(pos->bbPieces[uEnemy][BISHOP])) -
- (INT)(FastCountBits(pos->bbPieces[uSide][KNIGHT]) +
- FastCountBits(pos->bbPieces[uSide][BISHOP]));
+ (INT)(uSideKnights + uSideBishops);
if (iMajorDiff == iMinorDiff)
{
return(FALSE);
@@ -5532,8 +6165,8 @@ Return value:
// excluded, since that has one bishop, not zero) -- both are
// genuine, if sometimes technique-heavy, forced wins.
//
- if ((2 == FastCountBits(pos->bbPieces[uSide][KNIGHT])) &&
- (0 == FastCountBits(pos->bbPieces[uSide][BISHOP])) &&
+ if ((2 == uSideKnights) &&
+ (0 == uSideBishops) &&
(0 == FastCountBits(pos->bbPieces[uSide][ROOK])) &&
(0 == FastCountBits(pos->bbPieces[uSide][QUEEN])) &&
(1 == TotalNonPawnCount(pos, uEnemy)) &&
@@ -5556,12 +6189,59 @@ InitEval(void)
}
+// 2026-09-17: DRAWISH_FACTS holds the three position-only inputs
+// EvalLookForDrawishSituations needs (_SideHasWinningChances for
+// both sides, and the BOOC opposite-bishop check) -- none of these
+// depend on the score at all, only on pos, which doesn't change
+// between Eval()'s two call sites (the regular-lazy pre-exit check
+// and the end of full eval). A node that attempts the lazy exit and
+// then falls through to full eval anyway used to pay for
+// _SideHasWinningChances (two calls) and the BOOC check twice, once
+// per site, for identical answers both times -- only the score-vs-
+// drawscore/50-move comparisons genuinely need to be re-run per call
+// site, since the score itself legitimately differs between the
+// early lazy-preview value and the final full-eval value.
+// _EnsureDrawishFactsComputed is a cheap (single-branch) fetch-or-
+// compute -- called at both sites so either call order (lazy-then-
+// full, or full alone when the lazy attempt is skipped entirely for
+// bare-king/low-material positions) computes the facts exactly once.
+typedef struct _DRAWISH_FACTS
+{
+ FLAG fComputed;
+ FLAG fSideToMoveHasWinningChances;
+ FLAG fOpponentHasWinningChances;
+ FLAG fBOOC;
+} DRAWISH_FACTS;
+
+static void
+_EnsureDrawishFactsComputed(IN POSITION *pos, IN ULONG uColor,
+ IN OUT DRAWISH_FACTS *pFacts)
+{
+ if (pFacts->fComputed)
+ {
+ return;
+ }
+ pFacts->fSideToMoveHasWinningChances = _SideHasWinningChances(pos, uColor);
+ pFacts->fOpponentHasWinningChances =
+ _SideHasWinningChances(pos, FLIP(uColor));
+ pFacts->fBOOC = (TotalNonPawnCount(pos, WHITE) == 2) &&
+ (TotalNonPawnCount(pos, BLACK) == 2) &&
+ (FastCountBits(pos->bbPieces[WHITE][BISHOP]) == 1) &&
+ (FastCountBits(pos->bbPieces[BLACK][BISHOP]) == 1) &&
+ (WhiteSqBishopCount(pos, WHITE) != WhiteSqBishopCount(pos, BLACK));
+ pFacts->fComputed = TRUE;
+}
+
+
static void
EvalLookForDrawishSituations(POSITION *pos,
- SCORE *piScoreForSideToMove)
+ SCORE *piScoreForSideToMove,
+ IN DRAWISH_FACTS *pFacts)
{
ULONG uColor = pos->uToMove;
+ ASSERT(pFacts->fComputed);
+
// If the side who's ahead by raw material/positional score can't
// actually force a win with what it has left on the board (see
// _SideHasWinningChances), squash the score hard towards draw --
@@ -5574,24 +6254,21 @@ EvalLookForDrawishSituations(POSITION *pos,
// defender with the checks _SideHasWinningChances gates on, so
// some residual signal survives.
if ((*piScoreForSideToMove > g_iDrawScore[uColor]) &&
- (FALSE == _SideHasWinningChances(pos, uColor)))
+ (FALSE == pFacts->fSideToMoveHasWinningChances))
{
*piScoreForSideToMove = g_iDrawScore[uColor] +
((*piScoreForSideToMove - g_iDrawScore[uColor]) / 16);
}
else if ((*piScoreForSideToMove < g_iDrawScore[uColor]) &&
- (FALSE == _SideHasWinningChances(pos, FLIP(uColor))))
+ (FALSE == pFacts->fOpponentHasWinningChances))
{
*piScoreForSideToMove = g_iDrawScore[uColor] +
((*piScoreForSideToMove - g_iDrawScore[uColor]) / 16);
}
// If this is a BOOC endgame, push towards draw score too.
- if ((TotalNonPawnCount(pos, WHITE) == 2) &&
- (TotalNonPawnCount(pos, BLACK) == 2) &&
- (FastCountBits(pos->bbPieces[WHITE][BISHOP]) == 1) &&
- (FastCountBits(pos->bbPieces[BLACK][BISHOP]) == 1) &&
- (WhiteSqBishopCount(pos, WHITE) != WhiteSqBishopCount(pos, BLACK))) {
+ if (pFacts->fBOOC)
+ {
*piScoreForSideToMove = g_iDrawScore[uColor] +
((*piScoreForSideToMove - g_iDrawScore[uColor]) / 4);
}
@@ -5615,6 +6292,249 @@ EvalLookForDrawishSituations(POSITION *pos,
}
+#ifdef DUMP_RESIDUAL_SAMPLES
+//
+// Histogram-based percentile capture for residual (bucketed by OWN
+// army scaler) and king safety (bucketed by ATTACKER's army scaler) --
+// see the two call sites inside Eval() below. Deliberately a histogram
+// (same idea as CALIBRATE_POSITIONAL's g_CalibrateHist), not a raw
+// per-sample Trace: a raw dump against the 30k-position calibration
+// pool would run into the billions of lines (a 200-position sample
+// alone already produced 8.76M), while this stays a fixed, tiny amount
+// of memory regardless of sample count and prints one summary via
+// `calibrate residual`.
+//
+#define RESID_HIST_BUCKETS 8
+#define RESID_HIST_BINS 90
+#define RESID_HIST_BIN_WIDTH 20
+#define RESID_HIST_MIN (-900)
+
+static UINT64 g_ResidualHist[RESID_HIST_BUCKETS][RESID_HIST_BINS];
+static UINT64 g_SuperLazyResidualHist[RESID_HIST_BUCKETS][RESID_HIST_BINS];
+static UINT64 g_KingSafetyHist[RESID_HIST_BUCKETS][RESID_HIST_BINS];
+static INT g_ResidualMin[RESID_HIST_BUCKETS];
+static INT g_ResidualMax[RESID_HIST_BUCKETS];
+static INT g_SuperLazyResidualMin[RESID_HIST_BUCKETS];
+static INT g_SuperLazyResidualMax[RESID_HIST_BUCKETS];
+static INT g_KingSafetyMin[RESID_HIST_BUCKETS];
+static INT g_KingSafetyMax[RESID_HIST_BUCKETS];
+static FLAG g_fResidHistInit = FALSE;
+static ULONG g_uResidualOutliersPrinted = 0;
+static ULONG g_uKingSafetyOutliersPrinted = 0;
+
+static void
+_ResidHistInitIfNeeded(void)
+{
+ ULONG b;
+ if (TRUE == g_fResidHistInit) return;
+ for (b = 0; b < RESID_HIST_BUCKETS; b++)
+ {
+ g_ResidualMin[b] = 999999;
+ g_ResidualMax[b] = -999999;
+ g_SuperLazyResidualMin[b] = 999999;
+ g_SuperLazyResidualMax[b] = -999999;
+ g_KingSafetyMin[b] = 999999;
+ g_KingSafetyMax[b] = -999999;
+ }
+ g_fResidHistInit = TRUE;
+}
+
+static void
+_RecordResidualSample(ULONG uOwnArmyBucket, INT iValue)
+{
+ INT idx;
+ _ResidHistInitIfNeeded();
+ idx = (iValue - RESID_HIST_MIN) / RESID_HIST_BIN_WIDTH;
+ if (idx < 0) idx = 0;
+ if (idx >= RESID_HIST_BINS) idx = RESID_HIST_BINS - 1;
+ g_ResidualHist[uOwnArmyBucket][idx]++;
+ if (iValue < g_ResidualMin[uOwnArmyBucket]) g_ResidualMin[uOwnArmyBucket] = iValue;
+ if (iValue > g_ResidualMax[uOwnArmyBucket]) g_ResidualMax[uOwnArmyBucket] = iValue;
+}
+
+static void
+_RecordSuperLazyResidualSample(ULONG uOwnArmyBucket, INT iValue)
+{
+ INT idx;
+ _ResidHistInitIfNeeded();
+ idx = (iValue - RESID_HIST_MIN) / RESID_HIST_BIN_WIDTH;
+ if (idx < 0) idx = 0;
+ if (idx >= RESID_HIST_BINS) idx = RESID_HIST_BINS - 1;
+ g_SuperLazyResidualHist[uOwnArmyBucket][idx]++;
+ if (iValue < g_SuperLazyResidualMin[uOwnArmyBucket]) g_SuperLazyResidualMin[uOwnArmyBucket] = iValue;
+ if (iValue > g_SuperLazyResidualMax[uOwnArmyBucket]) g_SuperLazyResidualMax[uOwnArmyBucket] = iValue;
+}
+
+static void
+_RecordKingSafetySample(ULONG uAttackerArmyBucket, INT iValue)
+{
+ INT idx;
+ _ResidHistInitIfNeeded();
+ idx = (iValue - RESID_HIST_MIN) / RESID_HIST_BIN_WIDTH;
+ if (idx < 0) idx = 0;
+ if (idx >= RESID_HIST_BINS) idx = RESID_HIST_BINS - 1;
+ g_KingSafetyHist[uAttackerArmyBucket][idx]++;
+ if (iValue < g_KingSafetyMin[uAttackerArmyBucket]) g_KingSafetyMin[uAttackerArmyBucket] = iValue;
+ if (iValue > g_KingSafetyMax[uAttackerArmyBucket]) g_KingSafetyMax[uAttackerArmyBucket] = iValue;
+}
+
+static INT
+_HistPercentile(UINT64 *puHist, UINT64 uTotal, double dP)
+{
+ UINT64 uTarget = (UINT64)(dP * (double)uTotal);
+ UINT64 uRunning = 0;
+ ULONG i;
+ for (i = 0; i < RESID_HIST_BINS; i++)
+ {
+ uRunning += puHist[i];
+ if (uRunning >= uTarget)
+ {
+ return RESID_HIST_MIN + (INT)(i * RESID_HIST_BIN_WIDTH) +
+ (RESID_HIST_BIN_WIDTH / 2);
+ }
+ }
+ return RESID_HIST_MIN + (RESID_HIST_BINS * RESID_HIST_BIN_WIDTH);
+}
+
+static void
+_PersistRawHistogramsToFile(void)
+/**
+
+Routine description:
+
+ 2026-09-17: write every raw bin of all three histograms to a fixed
+ scratch file, so any future percentile (not just the ones
+ DumpResidualAndKingSafetyHistograms happens to print) can be read
+ off after the fact -- no need to re-run the 30k pool just to try a
+ different percentile choice. Plain text, one line per (histogram,
+ bucket): the bucket's min, max, then all RESID_HIST_BINS raw counts
+ space-separated. A future script can reconstruct any percentile via
+ the same cumulative-sum logic _HistPercentile uses.
+
+Parameters:
+
+ void
+
+Return value:
+
+ void
+
+**/
+{
+ FILE *f;
+ ULONG b, i;
+
+ f = fopen("/var/tmp/typhoon/residual_raw_histograms.txt", "w");
+ if (NULL == f)
+ {
+ Trace("_PersistRawHistogramsToFile: could not open output file.\n");
+ return;
+ }
+ fprintf(f, "# RESID_HIST_BINS=%d RESID_HIST_BIN_WIDTH=%d RESID_HIST_MIN=%d\n",
+ RESID_HIST_BINS, RESID_HIST_BIN_WIDTH, RESID_HIST_MIN);
+ fprintf(f, "# format: <histogram> <bucket> <min> <max> <bin0> <bin1> ... <binN-1>\n");
+ for (b = 0; b < RESID_HIST_BUCKETS; b++)
+ {
+ fprintf(f, "residual %u %d %d", b, g_ResidualMin[b], g_ResidualMax[b]);
+ for (i = 0; i < RESID_HIST_BINS; i++)
+ {
+ fprintf(f, " %llu", (unsigned long long)g_ResidualHist[b][i]);
+ }
+ fprintf(f, "\n");
+ }
+ for (b = 0; b < RESID_HIST_BUCKETS; b++)
+ {
+ fprintf(f, "sl_residual %u %d %d", b, g_SuperLazyResidualMin[b],
+ g_SuperLazyResidualMax[b]);
+ for (i = 0; i < RESID_HIST_BINS; i++)
+ {
+ fprintf(f, " %llu", (unsigned long long)g_SuperLazyResidualHist[b][i]);
+ }
+ fprintf(f, "\n");
+ }
+ for (b = 0; b < RESID_HIST_BUCKETS; b++)
+ {
+ fprintf(f, "king %u %d %d", b, g_KingSafetyMin[b], g_KingSafetyMax[b]);
+ for (i = 0; i < RESID_HIST_BINS; i++)
+ {
+ fprintf(f, " %llu", (unsigned long long)g_KingSafetyHist[b][i]);
+ }
+ fprintf(f, "\n");
+ }
+ fclose(f);
+ Trace("Raw histograms written to /var/tmp/typhoon/residual_raw_histograms.txt\n");
+}
+
+void
+DumpResidualAndKingSafetyHistograms(void)
+{
+ ULONG b, i;
+ UINT64 uTotal;
+
+ _PersistRawHistogramsToFile();
+
+#define TRACE_HIST_ROW(hist, min, max) \
+ Trace(" bucket %u (scaler %u-%u): n=%llu min=%d p1=%d p2=%d " \
+ "p5=%d p10=%d p50=%d p90=%d p95=%d p98=%d p99=%d max=%d\n", \
+ b, b * 4, b * 4 + 3, (unsigned long long)uTotal, (min), \
+ _HistPercentile((hist), uTotal, 0.01), \
+ _HistPercentile((hist), uTotal, 0.02), \
+ _HistPercentile((hist), uTotal, 0.05), \
+ _HistPercentile((hist), uTotal, 0.10), \
+ _HistPercentile((hist), uTotal, 0.50), \
+ _HistPercentile((hist), uTotal, 0.90), \
+ _HistPercentile((hist), uTotal, 0.95), \
+ _HistPercentile((hist), uTotal, 0.98), \
+ _HistPercentile((hist), uTotal, 0.99), \
+ (max))
+
+ Trace("Residual by OWN army-scaler bucket:\n");
+ for (b = 0; b < RESID_HIST_BUCKETS; b++)
+ {
+ uTotal = 0;
+ for (i = 0; i < RESID_HIST_BINS; i++) uTotal += g_ResidualHist[b][i];
+ if (0 == uTotal)
+ {
+ Trace(" bucket %u: no samples\n", b);
+ continue;
+ }
+ TRACE_HIST_ROW(g_ResidualHist[b], g_ResidualMin[b], g_ResidualMax[b]);
+ }
+
+ Trace("Super-lazy-scoped residual (material baseline, includes "
+ "pawns/passers/bad-trade/bishop-pairs) by OWN army-scaler "
+ "bucket:\n");
+ for (b = 0; b < RESID_HIST_BUCKETS; b++)
+ {
+ uTotal = 0;
+ for (i = 0; i < RESID_HIST_BINS; i++) uTotal += g_SuperLazyResidualHist[b][i];
+ if (0 == uTotal)
+ {
+ Trace(" bucket %u: no samples\n", b);
+ continue;
+ }
+ TRACE_HIST_ROW(g_SuperLazyResidualHist[b], g_SuperLazyResidualMin[b],
+ g_SuperLazyResidualMax[b]);
+ }
+
+ Trace("King safety by ATTACKER army-scaler bucket:\n");
+ for (b = 0; b < RESID_HIST_BUCKETS; b++)
+ {
+ uTotal = 0;
+ for (i = 0; i < RESID_HIST_BINS; i++) uTotal += g_KingSafetyHist[b][i];
+ if (0 == uTotal)
+ {
+ Trace(" bucket %u: no samples\n", b);
+ continue;
+ }
+ TRACE_HIST_ROW(g_KingSafetyHist[b], g_KingSafetyMin[b],
+ g_KingSafetyMax[b]);
+ }
+#undef TRACE_HIST_ROW
+}
+#endif
+
+
SCORE
Eval(IN SEARCHER_THREAD_CONTEXT *ctx,
IN SCORE iAlpha,
@@ -5656,6 +6576,10 @@ Return value:
ULONG xColor = FLIP(uColor);
BITBOARD bb;
FLAG fDeferred;
+ // Computed at most once per Eval() call, on first need, by
+ // whichever of the two EvalLookForDrawishSituations call sites
+ // runs first -- see DRAWISH_FACTS's comment above.
+ DRAWISH_FACTS sDrawishFacts = {0};
#ifdef EVAL_TIME
UINT64 uTimer = SystemReadTimeStampCounter();
#endif
@@ -5670,12 +6594,20 @@ Return value:
FLAG fWouldHaveExited = FALSE;
SCORE iSavedLazyScore = 0;
SCORE iSavedLazyPositional = 0;
+ // 2026-09-17: iPositional is no longer flat/symmetric across colors
+ // (p50-based, keyed by each side's own army bucket) -- iSavedLazy-
+ // Positional above stays the margin-safety accounting value (still
+ // symmetric, unrelated to iPositional), these two are what actually
+ // gets restored into (*piPositional)[WHITE]/[BLACK] below.
+ SCORE iSavedLazyPositionalWhite = 0;
+ SCORE iSavedLazyPositionalBlack = 0;
// Same idea, kept separate from the pair above: the super-lazy
// exit (added 2026-09-07) is a much cheaper, much-larger-gap check
// than the regular lazy exit and was never hooked into this
// harness -- see RecordSuperLazyMarginSafetySwing below.
FLAG fWouldHaveExitedSuperLazy = FALSE;
SCORE iSavedSuperLazyScore = 0;
+ SCORE iSavedSuperLazyMarginUsed = 0;
#endif
ASSERT(IS_VALID_SCORE(iAlpha));
ASSERT(IS_VALID_SCORE(iBeta));
@@ -5692,7 +6624,11 @@ Return value:
(PawnMaterial(pos, xColor) + pos->uNonPawnMaterial[xColor]);
#ifdef EVAL_DUMP
EvalTraceClear();
- Trace("Material:\n%d\t\t%d\n", pos->iScore[WHITE], pos->iScore[BLACK]);
+ Trace("Material:\n%d\t\t%d\t\t(%d for %s)\n",
+ pos->iScore[WHITE], pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
// Initialize army scalers here (moved ahead of the super-lazy exit
@@ -5717,53 +6653,118 @@ Return value:
(SystemReadTimeStampCounter() - u64MaterialSetupTimer);
#endif
+#ifdef DUMP_RESIDUAL_SAMPLES
+ // Super-lazy's OWN baseline -- material only, before _EvalPawns
+ // even runs. Separate from iResidualBaselineWhite/Black below
+ // (captured after bishop pairs, i.e. regular-lazy's baseline):
+ // that measurement deliberately excludes pawns/passers/bad-trade/
+ // bishop-pairs, which is correct for regular lazy (which already
+ // has all of that by the time it asks) but wrong for super-lazy,
+ // which has NONE of it yet. This is what a super-lazy-scoped
+ // residual needs to be measured against instead.
+ SCORE iSuperLazyResidualBaselineWhite = pos->iScore[WHITE];
+ SCORE iSuperLazyResidualBaselineBlack = pos->iScore[BLACK];
+#endif
+
+ // 2026-09-16: shared "too dangerous to shortcut" gate for both lazy
+ // tiers -- exactly the condition search.c's own passer-race code
+ // (_EvalPasserRacesAgainstLoneKings' caller) uses to decide whether
+ // the volatile won-passer-race verdict can fire: one side down to
+ // literally just a king (TotalNonPawnCount counts the king itself
+ // as 1, so ==1 means no other non-pawn material at all). Regular
+ // lazy already had a blunter, broader guard for this material
+ // region (LAZY_EVAL_MIN_MATERIAL, a combined-army-scaler band) --
+ // but that band doesn't actually imply either side is bare-king
+ // (e.g. a heavy attacker vs. a bare king can have a large combined
+ // scaler despite one side having zero other material), so it
+ // wasn't sufficient on its own. Super-lazy had no such guard at
+ // all. Folding EvalLookForDrawishSituations into both tiers (above/
+ // below) closes the "insufficient winning chances"/BOOC/50-move
+ // gap; this closes the remaining one -- the passer-race verdict
+ // needs pHash->bbPasserLocations, which isn't available yet at
+ // super-lazy's point in Eval() (it runs before _EvalPawns), so
+ // that correction can't cheaply be folded in the same way and the
+ // honest answer here is to just not shortcut in this regime.
+ FLAG fBareKingEitherSide = (1 == TotalNonPawnCount(pos, WHITE)) ||
+ (1 == TotalNonPawnCount(pos, BLACK));
+
// Super-lazy exit point.
#ifdef LAZY_EVAL
- // 2026-09-08: was a single flat 625 for every material level, then
- // split by material bucket using a 100-position/sd8 calibration
- // (symmetric |swing|, ~15-20% headroom). Re-calibrated against a
- // larger 1500-position sample with a directional split (only a
- // swing *toward* the alpha/beta boundary can make an exit unsound;
- // a swing away is harmless, and the original symmetric measurement
- // conflated the two) -- true dangerous-direction max ran 15-50%
- // below the old symmetric max in most buckets, and every bucket
- // showed exceeded=0 even before headroom. Values below are
- // directional-max * ~1.15-1.3 (more headroom on buckets with
- // thinner samples, e.g. bucket 0's n=512); buckets 2 and 5 kept at
- // their prior value since it was already tighter than a fresh 15%
- // headroom would give. Table indexed the same way _MaterialBucket
- // buckets the calibration data (combined army scaler / 8, 8
- // buckets), so it can be re-derived directly from a "calibrate
- // marginsafety" run. Not a hard theoretical bound -- re-check
- // before trusting this at sd well beyond 8.
- static const SCORE SUPER_LAZY_MARGIN_BY_ARMY[8] =
- {
- 300, 1635, 1800, 1070, 946, 850, 734, 698,
- };
+ // Combined-army bucket, still needed by the bucket-0 exclusion
+ // below (a structural exclusion of pawn-dominated near-bare-army
+ // positions, unrelated to the margin construction itself).
ULONG uSuperLazyMatBucket = MINU(
7, (pos->uArmyScaler[WHITE] + pos->uArmyScaler[BLACK]) / 8);
- SCORE iSuperLazyMargin = SUPER_LAZY_MARGIN_BY_ARMY[uSuperLazyMatBucket];
+
+ // 2026-09-16: exclude material bucket 0 (combined non-pawn army
+ // scaler 0-7) from super-lazy entirely -- CALIBRATE_MARGIN_SAFETY
+ // against 30k stratified positions found a 6.8% exceeded rate
+ // there even after the bare-king exclusion above and the
+ // EvalLookForDrawishSituations fix, traced by hand to real,
+ // externally-corroborated (cross-checked against Crafty's own eval
+ // breakdown on the same position, within 7%) swings of 650-750+
+ // centipawns from knight-vs-bishop-with-several-pawns-per-side
+ // endgames. Root cause: this bucket is defined purely by non-pawn
+ // material, so it says nothing about how many pawns are on the
+ // board, and super-lazy runs before _EvalPawns entirely -- meaning
+ // it's not just imprecise about pawn structure here, it's totally
+ // blind to it, in exactly the material regime where pawns (passed-
+ // pawn recognition, king-supports-own-passer, _ReEvalPassers) can
+ // be nearly the entire evaluation. Not a bug to fix with a bigger
+ // constant -- a structural mismatch between what this bucket
+ // measures (piece count) and what actually drives the score
+ // (pawn count) in this exact material range. Bucket 1+ (scaler
+ // >=8) doesn't have this problem to nearly the same degree since
+ // there's enough non-pawn material for pieces to matter alongside
+ // pawns, not be dwarfed by them.
+ if ((FALSE == fBareKingEitherSide) && (uSuperLazyMatBucket > 0))
{
#ifdef EVAL_TIME
UINT64 uSuperLazyTimer = SystemReadTimeStampCounter();
#endif
iScoreForSideToMove = (pos->iScore[uColor] -pos->iScore[xColor]);
ASSERT(IS_VALID_SCORE(iScoreForSideToMove));
- if (iScoreForSideToMove + iSuperLazyMargin < iAlpha)
+
+ // We have a super-rough eval -- material only at this point.
+ // Compute two margins to check whether things look so good or
+ // bad that we need not refine our eval because we are sure to
+ // FH/FL anyway.
+ ULONG uSuperLazyMyBucket = MINU(7, pos->uArmyScaler[uColor] / 4);
+ ULONG uSuperLazyOppBucket = MINU(7, pos->uArmyScaler[xColor] / 4);
+ SCORE iSuperLazySelfPessEnemyOptMargin = MAX0(
+ (iKingP98ByAttackerArmy[uSuperLazyMyBucket] -
+ iKingP2ByAttackerArmy[uSuperLazyOppBucket]) +
+ (iSuperLazyResidualOptimisticByOwnArmy[uSuperLazyOppBucket] -
+ iSuperLazyResidualPessimisticByOwnArmy[uSuperLazyMyBucket]));
+ SCORE iSuperLazySelfOptEnemyPessMargin = MAX0(
+ (iKingP98ByAttackerArmy[uSuperLazyOppBucket] -
+ iKingP2ByAttackerArmy[uSuperLazyMyBucket]) +
+ (iSuperLazyResidualOptimisticByOwnArmy[uSuperLazyMyBucket] -
+ iSuperLazyResidualPessimisticByOwnArmy[uSuperLazyOppBucket]));
+
+ if (iScoreForSideToMove + iSuperLazySelfOptEnemyPessMargin < iAlpha)
{
ASSERT(iScoreForSideToMove < iAlpha);
+#ifdef CALIBRATE_MARGIN_SAFETY
+ iSavedSuperLazyMarginUsed = iSuperLazySelfOptEnemyPessMargin;
+#endif
INC(ctx->sCounters.tree.u64SuperLazyEvals);
+ g_u64SuperLazyEvalsByBucket[_MaterialBucket(pos)]++;
if (NULL != piPositional)
{
- // No positional credit here -- a super-lazy verdict is
- // already a confident material-only read that we're
- // well outside the window; handing back extra slack
- // just re-inflates the qsearch futility margin and
- // defeats the point of taking the cheap exit. Let
- // FUTILITY_BASE_MARGIN_BY_SOURCE[SUPERLAZY] alone
- // govern from here.
- (*piPositional)[WHITE] = 0;
- (*piPositional)[BLACK] = 0;
+ // 2026-09-17: no longer flat 0 -- same p50-based typical-
+ // value estimate as regular lazy, using the super-lazy-
+ // scoped residual table (material-only baseline). See
+ // iKingP50ByAttackerArmyOnly/iSuperLazyResidualP50By-
+ // OwnArmy (file-scope, above).
+ ULONG uWhiteBucket = MINU(7, pos->uArmyScaler[WHITE] / 4);
+ ULONG uBlackBucket = MINU(7, pos->uArmyScaler[BLACK] / 4);
+ (*piPositional)[WHITE] =
+ iKingP50ByAttackerArmy[uBlackBucket] +
+ iSuperLazyResidualP50ByOwnArmy[uWhiteBucket];
+ (*piPositional)[BLACK] =
+ iKingP50ByAttackerArmy[uWhiteBucket] +
+ iSuperLazyResidualP50ByOwnArmy[uBlackBucket];
ctx->sPlyInfo[ctx->uPly].uLastPositionalSource = EVAL_POSITIONAL_SOURCE_SUPERLAZY;
}
#ifdef EVAL_TIME
@@ -5790,14 +6791,26 @@ Return value:
goto end;
#endif
}
- if (iScoreForSideToMove - iSuperLazyMargin > iBeta)
+ if (iScoreForSideToMove - iSuperLazySelfPessEnemyOptMargin > iBeta)
{
ASSERT(iScoreForSideToMove > iBeta);
+#ifdef CALIBRATE_MARGIN_SAFETY
+ iSavedSuperLazyMarginUsed = iSuperLazySelfPessEnemyOptMargin;
+#endif
INC(ctx->sCounters.tree.u64SuperLazyEvals);
+ g_u64SuperLazyEvalsByBucket[_MaterialBucket(pos)]++;
if (NULL != piPositional)
{
- (*piPositional)[WHITE] = 0;
- (*piPositional)[BLACK] = 0;
+ // See the analogous alpha-side branch above -- same p50
+ // typical-value estimate.
+ ULONG uWhiteBucket = MINU(7, pos->uArmyScaler[WHITE] / 4);
+ ULONG uBlackBucket = MINU(7, pos->uArmyScaler[BLACK] / 4);
+ (*piPositional)[WHITE] =
+ iKingP50ByAttackerArmy[uBlackBucket] +
+ iSuperLazyResidualP50ByOwnArmy[uWhiteBucket];
+ (*piPositional)[BLACK] =
+ iKingP50ByAttackerArmy[uWhiteBucket] +
+ iSuperLazyResidualP50ByOwnArmy[uBlackBucket];
ctx->sPlyInfo[ctx->uPly].uLastPositionalSource = EVAL_POSITIONAL_SOURCE_SUPERLAZY;
}
#ifdef EVAL_TIME
@@ -5826,30 +6839,29 @@ Return value:
}
#endif
- //
+ // If we get here, we did not take the super lazy exit. This is
+ // either a lazy exit node or a full eval node.
+
// Pawn eval. Note: if fDeferred comes back as TRUE then we have
// neither cleared nor initialized the attack tables. This is
- // because we hope that we can get a lazy eval cutoff here and
+ // because we hope that we can still get a lazy eval exit here and
// save some time. BEFORE ANY CODE BELOW TOUCHES THE ATTACK
// TABLES IT NEEDS TO CLEAR/POPULATE THEM THOUGH!!!
- //
#ifdef EVAL_TIME
+ UINT64 u64PawnTimer = SystemReadTimeStampCounter();
+ pHash = _EvalPawns(ctx, &fDeferred);
{
- UINT64 u64PawnTimer = SystemReadTimeStampCounter();
- pHash = _EvalPawns(ctx, &fDeferred);
+ UINT64 u64Elapsed = SystemReadTimeStampCounter() - u64PawnTimer;
+ ctx->sCounters.tree.u64CyclesEvalPawns += u64Elapsed;
+ if (fDeferred)
{
- UINT64 u64Elapsed = SystemReadTimeStampCounter() - u64PawnTimer;
- ctx->sCounters.tree.u64CyclesEvalPawns += u64Elapsed;
- if (fDeferred)
- {
- ctx->sCounters.tree.u64CyclesEvalPawnsHit += u64Elapsed;
- ctx->sCounters.tree.u64CountEvalPawnsHit++;
- }
- else
- {
- ctx->sCounters.tree.u64CyclesEvalPawnsMiss += u64Elapsed;
- ctx->sCounters.tree.u64CountEvalPawnsMiss++;
- }
+ ctx->sCounters.tree.u64CyclesEvalPawnsHit += u64Elapsed;
+ ctx->sCounters.tree.u64CountEvalPawnsHit++;
+ }
+ else
+ {
+ ctx->sCounters.tree.u64CyclesEvalPawnsMiss += u64Elapsed;
+ ctx->sCounters.tree.u64CountEvalPawnsMiss++;
}
}
#else
@@ -5861,63 +6873,89 @@ Return value:
pos->iScore[BLACK] += pHash->iScore[BLACK];
ASSERT(IS_VALID_SCORE(pos->iScore[WHITE] - pos->iScore[BLACK]));
#ifdef EVAL_DUMP
- Trace("After pawns:\n%d\t\t%d\n", pos->iScore[WHITE], pos->iScore[BLACK]);
+ Trace("After pawns:\n%d\t\t%d\t\t(%d for %s)\n",
+ pos->iScore[WHITE], pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
+ // Look for won passers races. Note: This code cannot trust the
+ // state of the attack tables yet!
#ifdef EVAL_TIME
UINT64 u64PasserBadtradeBishopTimer = SystemReadTimeStampCounter();
#endif
- //
- // Look for won passers races. Note: This code cannot trust the
- // state of the attack tables yet!
- //
- if ((TotalNonPawnCount(pos, WHITE) == 1) ||
- (TotalNonPawnCount(pos, BLACK) == 1))
+ if (fBareKingEitherSide)
{
(void)_EvalPasserRacesAgainstLoneKings(pos, pHash);
#ifdef EVAL_DUMP
- Trace("After passer races:\n%d\t\t%d\n", pos->iScore[WHITE],
- pos->iScore[BLACK]);
+ Trace("After passer races:\n%d\t\t%d\t\t(%d for %s)\n",
+ pos->iScore[WHITE], pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
}
- //
- // Bad trade code. Right. Note: this code cannot trust the state
- // of the attack tables...
- //
+ // Bad trade code. Note: this code cannot trust the state of the
+ // attack tables...
if (pos->uNonPawnMaterial[WHITE] != pos->uNonPawnMaterial[BLACK])
{
_EvalBadTrades(pos);
#ifdef EVAL_DUMP
- Trace("After bad trades:\n%d\t\t%d\n", pos->iScore[WHITE],
- pos->iScore[BLACK]);
+ Trace("After bad trades:\n%d\t\t%d\t\t(%d for %s)\n",
+ pos->iScore[WHITE],
+ pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
}
- //
// Bishop pairs. Again... Note: this code cannot trust the state
// of the attack tables.
- //
_EvalBishopPairs(pos);
#ifdef EVAL_DUMP
- Trace("After bishop pair bonus:\n%d\t\t%d\n", pos->iScore[WHITE],
- pos->iScore[BLACK]);
+ Trace("After bishop pair bonus:\n%d\t\t%d\t\t(%d for %s)\n",
+ pos->iScore[WHITE],
+ pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
#ifdef EVAL_TIME
ctx->sCounters.tree.u64CyclesEvalPasserBadtradeBishop +=
(SystemReadTimeStampCounter() - u64PasserBadtradeBishopTimer);
#endif
+#ifdef DUMP_RESIDUAL_SAMPLES
+ // Snapshot right here: material + pawns/passers + bad-trade +
+ // bishop pairs, before king safety or mobility/misc positional
+ // terms run. Subtracting this (the same baseline regular lazy
+ // itself reads its score from) plus the real per-side king score
+ // (ctx->sPlyInfo[ctx->uPly].iKingScore[color], stored once
+ // _EvalKing runs for that side) from the final full-eval score
+ // isolates exactly what iResidualP98 is trying to estimate:
+ // mobility + trapped pieces + everything else, with passers
+ // already excluded. See the dump site below, near "Almost done".
+ SCORE iResidualBaselineWhite = pos->iScore[WHITE];
+ SCORE iResidualBaselineBlack = pos->iScore[BLACK];
+#endif
+
#ifdef LAZY_EVAL
- //
// Below LAZY_EVAL_MIN_MATERIAL combined army scaler, skip lazy eval
// entirely (see its chess.h comment) -- measured directly via
// CALIBRATE_MARGIN_SAFETY, the regular lazy exit's real swing
// exceeded its own assumed margin 20.6% of the time in near-bare-
// king endgames, far above the ~0.01-0.36% rate everywhere else.
- //
- if (pos->uArmyScaler[WHITE] + pos->uArmyScaler[BLACK] >=
- LAZY_EVAL_MIN_MATERIAL)
+ // fBareKingEitherSide (set above, before the super-lazy exit point)
+ // is a separate, more precise guard for the specific passer-race
+ // mechanism -- kept in addition to this band, not instead of it,
+ // since a heavy attacker vs. a bare king can still clear this
+ // combined-scaler threshold despite one side being bare-king.
+ if ((pos->uArmyScaler[WHITE] + pos->uArmyScaler[BLACK] >=
+ LAZY_EVAL_MIN_MATERIAL) &&
+ (FALSE == fBareKingEitherSide))
{
#ifdef EVAL_TIME
UINT64 uLazyDecisionTimer = SystemReadTimeStampCounter();
@@ -5930,179 +6968,201 @@ Return value:
// 2. Pawn structure bonuses/penalties (incl passers/candidates)
// 3. Passer races are detected already
// 4. "Bad trade" code has already run
- // 5. We've already detected unwinnable endgames
- // 6. Bishop pairs
+ // 5. Bishop pairs
//
- iScoreForSideToMove = (pos->iScore[pos->uToMove] -
- pos->iScore[FLIP(pos->uToMove)]);
+ iScoreForSideToMove = (pos->iScore[uColor] - pos->iScore[xColor]);
ASSERT(IS_VALID_SCORE(iScoreForSideToMove));
+ // 2026-09-16: item 5 used to claim "we've already detected
+ // unwinnable endgames" here, but nothing before this point
+ // actually did -- EvalLookForDrawishSituations (insufficient-
+ // winning-chances squash, BOOC, 50-move dampening) was only
+ // ever called once, at the very end of the FULL eval path
+ // (see its call site below), so a regular-lazy exit could
+ // return a raw material-ahead score (e.g. "+300, bishop up")
+ // in a position full eval would have squashed hard toward a
+ // draw (BOOC: /4; no forcing material: /16) -- a large swing
+ // with nothing to do with king safety or mobility. Call it
+ // here too so regular lazy sees the same correction full eval
+ // would apply. Cheap and safe to do this early: unlike
+ // king safety/mobility, none of EvalLookForDrawishSituations'
+ // three checks touch attack bitboards, only piece bitboards/
+ // counts and pos->uFifty, all already valid by this point.
+ // _EnsureDrawishFactsComputed is a no-op if the call site
+ // below (full eval, if this node falls through to it) already
+ // populated sDrawishFacts -- can't happen in this order since
+ // this is the first potential call site, but written this way
+ // so either call order is correct.
+ TIMED_EVAL_CALL(ctx, u64CyclesEvalDrawish,
+ (_EnsureDrawishFactsComputed(pos, uColor, &sDrawishFacts),
+ EvalLookForDrawishSituations(pos, &iScoreForSideToMove,
+ &sDrawishFacts)));
+
//
// Eval has not considered several potentially large terms:
//
// 1. Piece positional components (such as mobility, trapped)
- // 2. King safety penalties
+ // 2. King safety penalties (which can be large!)
// 3. Other miscellaneous bonuses/penalties
//
// We build two "margins" to account for these components of the
// score.
//
- iAlphaMargin = iBetaMargin = LAZY_EVAL_BASE_MARGIN;
-
- //
- // If (score + alpha_margin) is already > alpha -OR-
- // (score - beta_margin) is already < beta
- //
- // ...then we can stop thinking about lazy eval; the rest of the
- // computation only increases the margin so we know LE will fail
- // and can save some work here.
- //
+ // 2026-09-17: no flat base margin, no floor -- matches super-
+ // lazy's convention. margin_a'/margin_b' (built entirely by
+ // EstimatePositionalScore below) stand alone. If CALIBRATE_
+ // MARGIN_SAFETY turns up a real, non-trivial exceeded rate once
+ // this is validated against the 30k pool, the fix is tightening
+ // the percentile choice there (e.g. p95/p5 or p99/p1), not
+ // resurrecting a flat backstop that would just override the
+ // position-specific estimate again.
+ iAlphaMargin = iBetaMargin = 0;
#ifdef CALIBRATE_POSITIONAL
ctx->fCalibrateCandidate = FALSE;
g_uCalibrateGateChecked++;
#endif
- if ((iScoreForSideToMove + iAlphaMargin < iAlpha) ||
- (iScoreForSideToMove - iBetaMargin > iBeta))
- {
#ifdef CALIBRATE_POSITIONAL
- g_uCalibrateGateTrue++;
+ g_uCalibrateGateTrue++;
#endif
- //
- // Ok, we can't say for sure that we won't take a lazy exit
- // yet. So do the expensive part of lazy eval estimation and
- // widen the margin further for king safety issues and passed
- // pawns.
- //
+ //
+ // Ok, we can't say for sure that we won't take a lazy exit
+ // yet. So do the expensive part of lazy eval estimation and
+ // widen the margin further for king safety issues and passed
+ // pawns.
+ //
#ifdef CALIBRATE_POSITIONAL
- // This node is one where a real (uninstrumented) build would
- // actually call EstimatePositionalScore and consider a lazy
- // exit -- the population _RecordPositionalCalibration should
- // be sampling from, per _EvalKing's calls below.
- ctx->fCalibrateCandidate = TRUE;
+ // This node is one where a real (uninstrumented) build would
+ // actually call EstimatePositionalScore and consider a lazy
+ // exit -- the population _RecordPositionalCalibration should
+ // be sampling from, per _EvalKing's calls below.
+ ctx->fCalibrateCandidate = TRUE;
#endif
- EstimatePositionalScore(ctx, pos, pHash, &iAlphaMargin, &iBetaMargin);
+ EstimatePositionalScore(ctx, pos, pHash, &iAlphaMargin, &iBetaMargin);
#ifdef CALIBRATE_BASE_MARGIN
- {
- ULONG uBucket = _MaxKingDangerBucket(ctx, pos, pHash);
- ULONG uMatBucket = _MaterialBucket(pos);
- g_uBaseMarginExitAttempted[uMatBucket][uBucket]++;
- if ((iScoreForSideToMove + iAlphaMargin <= iAlpha) ||
- (iScoreForSideToMove - iBetaMargin >= iBeta))
- {
- g_uBaseMarginExitTaken[uMatBucket][uBucket]++;
- }
- }
+ ULONG uBucket = _MaxKingDangerBucket(ctx, pos, pHash);
+ ULONG uMatBucket = _MaterialBucket(pos);
+ g_uBaseMarginExitAttempted[uMatBucket][uBucket]++;
+ if ((iScoreForSideToMove + iAlphaMargin <= iAlpha) ||
+ (iScoreForSideToMove - iBetaMargin >= iBeta))
+ {
+ g_uBaseMarginExitTaken[uMatBucket][uBucket]++;
+ }
#endif
- //
- // CALIBRATE_POSITIONAL builds never take the lazy exit here --
- // the whole point of a calibration run is to compute the cheap
- // defect-count estimate *and* let Eval() continue on to
- // _EvalKing's real computation on every node, so the two can be
- // logged and compared. See _RecordPositionalCalibration below
- // (called from _EvalKing) and DumpPositionalCalibration
- // (command.c's "calibrate dump").
- //
+
+ // CALIBRATE_POSITIONAL builds never take the lazy exit here --
+ // the whole point of a calibration run is to compute the cheap
+ // defect-count estimate *and* let Eval() continue on to
+ // _EvalKing's real computation on every node, so the two can be
+ // logged and compared. See _RecordPositionalCalibration below
+ // (called from _EvalKing) and DumpPositionalCalibration
+ // (command.c's "calibrate dump").
#ifndef CALIBRATE_POSITIONAL
- if (iScoreForSideToMove + iAlphaMargin <= iAlpha)
+ if (iScoreForSideToMove + iAlphaMargin <= iAlpha)
+ {
+ INC(ctx->sCounters.tree.u64LazyEvals);
+ g_u64LazyEvalsByBucket[_MaterialBucket(pos)]++;
+ if (NULL != piPositional)
{
- INC(ctx->sCounters.tree.u64LazyEvals);
- if (NULL != piPositional)
- {
- (*piPositional)[WHITE] = iAlphaMargin;
- (*piPositional)[BLACK] = iAlphaMargin;
- ctx->sPlyInfo[ctx->uPly].uLastPositionalSource = EVAL_POSITIONAL_SOURCE_LAZY;
- }
+ // 2026-09-17: no longer flat 0 -- a p50-based typical-
+ // value estimate of this side's own missing positional
+ // terms (king safety + residual), matching full-eval's
+ // own iPositional convention (that side's own real
+ // positional score, not a relative-to-opponent
+ // quantity). See iKingP50ByAttackerArmyOnly/
+ // iResidualP50ByOwnArmy (file-scope, above) for the
+ // full percentile charts this was derived from.
+ ULONG uWhiteBucket = MINU(7, pos->uArmyScaler[WHITE] / 4);
+ ULONG uBlackBucket = MINU(7, pos->uArmyScaler[BLACK] / 4);
+ (*piPositional)[WHITE] =
+ iKingP50ByAttackerArmy[uBlackBucket] +
+ iResidualP50ByOwnArmy[uWhiteBucket];
+ (*piPositional)[BLACK] =
+ iKingP50ByAttackerArmy[uWhiteBucket] +
+ iResidualP50ByOwnArmy[uBlackBucket];
+ ctx->sPlyInfo[ctx->uPly].uLastPositionalSource = EVAL_POSITIONAL_SOURCE_LAZY;
+ }
#ifdef EVAL_TIME
- ctx->sCounters.tree.u64CyclesEvalPreLazy +=
- (SystemReadTimeStampCounter() - uTimer);
+ ctx->sCounters.tree.u64CyclesEvalPreLazy +=
+ (SystemReadTimeStampCounter() - uTimer);
#endif
#ifdef CALIBRATE_MARGIN_SAFETY
- // Don't exit yet -- let the real full eval run below so we
- // can measure the true swing, then restore these values
- // right before `end:` so the caller sees exactly what a
- // normal build would have returned.
- fWouldHaveExited = TRUE;
- iSavedLazyScore = iScoreForSideToMove;
- iSavedLazyPositional = iAlphaMargin;
+ // Don't exit yet -- let the real full eval run below so we
+ // can measure the true swing, then restore these values
+ // right before `end:` so the caller sees exactly what a
+ // normal build would have returned.
+ fWouldHaveExited = TRUE;
+ iSavedLazyScore = iScoreForSideToMove;
+ iSavedLazyPositional = iAlphaMargin;
+ iSavedLazyPositionalWhite =
+ iKingP50ByAttackerArmyOnly[MINU(7, pos->uArmyScaler[BLACK] / 4)] +
+ iResidualP50ByOwnArmy[MINU(7, pos->uArmyScaler[WHITE] / 4)];
+ iSavedLazyPositionalBlack =
+ iKingP50ByAttackerArmyOnly[MINU(7, pos->uArmyScaler[WHITE] / 4)] +
+ iResidualP50ByOwnArmy[MINU(7, pos->uArmyScaler[BLACK] / 4)];
#else
#ifdef EVAL_TIME
- ctx->sCounters.tree.u64CyclesEvalLazyDecision +=
- (SystemReadTimeStampCounter() - uLazyDecisionTimer);
- ctx->sCounters.tree.u64CyclesLazyExit +=
- (SystemReadTimeStampCounter() - uTimer);
- ctx->sCounters.tree.u64CyclesInEval +=
- (SystemReadTimeStampCounter() - uTimer);
+ ctx->sCounters.tree.u64CyclesEvalLazyDecision +=
+ (SystemReadTimeStampCounter() - uLazyDecisionTimer);
+ ctx->sCounters.tree.u64CyclesLazyExit +=
+ (SystemReadTimeStampCounter() - uTimer);
+ ctx->sCounters.tree.u64CyclesInEval +=
+ (SystemReadTimeStampCounter() - uTimer);
#endif
- goto end;
+ goto end;
#endif
- }
- else if (iScoreForSideToMove - iBetaMargin >= iBeta)
+ }
+ else if (iScoreForSideToMove - iBetaMargin >= iBeta)
+ {
+ INC(ctx->sCounters.tree.u64LazyEvals);
+ g_u64LazyEvalsByBucket[_MaterialBucket(pos)]++;
+ if (NULL != piPositional)
{
- INC(ctx->sCounters.tree.u64LazyEvals);
- if (NULL != piPositional)
- {
- (*piPositional)[WHITE] = iBetaMargin;
- (*piPositional)[BLACK] = iBetaMargin;
- ctx->sPlyInfo[ctx->uPly].uLastPositionalSource = EVAL_POSITIONAL_SOURCE_LAZY;
- }
+ // See the analogous fail-low branch above -- same p50
+ // typical-value estimate, not a flat 0 or the margin
+ // itself.
+ ULONG uWhiteBucket = MINU(7, pos->uArmyScaler[WHITE] / 4);
+ ULONG uBlackBucket = MINU(7, pos->uArmyScaler[BLACK] / 4);
+ (*piPositional)[WHITE] =
+ iKingP50ByAttackerArmy[uBlackBucket] +
+ iResidualP50ByOwnArmy[uWhiteBucket];
+ (*piPositional)[BLACK] =
+ iKingP50ByAttackerArmy[uWhiteBucket] +
+ iResidualP50ByOwnArmy[uBlackBucket];
+ ctx->sPlyInfo[ctx->uPly].uLastPositionalSource = EVAL_POSITIONAL_SOURCE_LAZY;
+ }
#ifdef EVAL_TIME
- ctx->sCounters.tree.u64CyclesEvalPreLazy +=
- (SystemReadTimeStampCounter() - uTimer);
+ ctx->sCounters.tree.u64CyclesEvalPreLazy +=
+ (SystemReadTimeStampCounter() - uTimer);
#endif
#ifdef CALIBRATE_MARGIN_SAFETY
- fWouldHaveExited = TRUE;
- iSavedLazyScore = iScoreForSideToMove;
- iSavedLazyPositional = iBetaMargin;
+ fWouldHaveExited = TRUE;
+ iSavedLazyScore = iScoreForSideToMove;
+ iSavedLazyPositional = iBetaMargin;
+ iSavedLazyPositionalWhite =
+ iKingP50ByAttackerArmyOnly[MINU(7, pos->uArmyScaler[BLACK] / 4)] +
+ iResidualP50ByOwnArmy[MINU(7, pos->uArmyScaler[WHITE] / 4)];
+ iSavedLazyPositionalBlack =
+ iKingP50ByAttackerArmyOnly[MINU(7, pos->uArmyScaler[WHITE] / 4)] +
+ iResidualP50ByOwnArmy[MINU(7, pos->uArmyScaler[BLACK] / 4)];
#else
#ifdef EVAL_TIME
- ctx->sCounters.tree.u64CyclesEvalLazyDecision +=
- (SystemReadTimeStampCounter() - uLazyDecisionTimer);
- ctx->sCounters.tree.u64CyclesLazyExit +=
- (SystemReadTimeStampCounter() - uTimer);
- ctx->sCounters.tree.u64CyclesInEval +=
- (SystemReadTimeStampCounter() - uTimer);
-#endif
- goto end;
-#endif
- }
-#endif // !CALIBRATE_POSITIONAL
-#ifdef EVAL_TIME
ctx->sCounters.tree.u64CyclesEvalLazyDecision +=
(SystemReadTimeStampCounter() - uLazyDecisionTimer);
+ ctx->sCounters.tree.u64CyclesLazyExit +=
+ (SystemReadTimeStampCounter() - uTimer);
+ ctx->sCounters.tree.u64CyclesInEval +=
+ (SystemReadTimeStampCounter() - uTimer);
+#endif
+ goto end;
#endif
}
-#ifdef CALIBRATE_BASE_MARGIN
- else
- {
- //
- // The flat gate said "don't even try" -- measure whether the
- // real, widened margin would have allowed an exit anyway had
- // we bothered to compute it. A "yes" here is a false negative
- // attributable to LAZY_EVAL_BASE_MARGIN being tighter than it
- // needs to be. This extra EstimatePositionalScore call is
- // measurement-only overhead, real exit behavior is unaffected
- // (this is the branch where a real build already skips lazy
- // eval entirely).
- //
- SCORE iMeasuredAlphaMargin = LAZY_EVAL_BASE_MARGIN;
- SCORE iMeasuredBetaMargin = LAZY_EVAL_BASE_MARGIN;
- ULONG uMatBucket = _MaterialBucket(pos);
-
- EstimatePositionalScore(ctx, pos, pHash, &iMeasuredAlphaMargin,
- &iMeasuredBetaMargin);
- if ((iScoreForSideToMove + iMeasuredAlphaMargin <= iAlpha) ||
- (iScoreForSideToMove - iMeasuredBetaMargin >= iBeta))
- {
- g_uBaseMarginFalseNegative[uMatBucket]++;
- }
- else
- {
- g_uBaseMarginCorrectlyFull[uMatBucket]++;
- }
- }
+#endif // !CALIBRATE_POSITIONAL
+#ifdef EVAL_TIME
+ ctx->sCounters.tree.u64CyclesEvalLazyDecision +=
+ (SystemReadTimeStampCounter() - uLazyDecisionTimer);
#endif
- } // LAZY_EVAL_MIN_MATERIAL
+ } // lazy eval
#endif
#ifdef EVAL_TIME
@@ -6110,67 +7170,30 @@ Return value:
(SystemReadTimeStampCounter() - uTimer);
#endif
- //
- // If we have to clear/populate the attack table, do it now that
- // we know we aren't taking a lazy exit.
- //
+ // If we get here we are doing a full eval.
+ ctx->sPlyInfo[ctx->uPly].u64FullEvalSig = (pos->u64NonPawnSig ^ pos->u64PawnSig);
+
+ // See if we have to clear/populate the pawn attack table.
if (TRUE == fDeferred)
{
TIMED_EVAL_CALL(ctx, u64CyclesEvalAttackTablePop,
- _PopulatePawnAttackBits(pos));
+ _PopulatePawnAttackBits(pos));
}
- //
// Pre-compute some common terms used in per-piece evals:
- //
pos->uNumTrapped[BLACK] = pos->uNumTrapped[WHITE] = 0;
pos->iReducedMaterialDownScaler[BLACK] =
pos->iReducedMaterialDownScaler[WHITE] = 0;
-
- //
- // This is a "position is closed|open" number.
- //
pos->uClosedScaler = (FastCountBits(pos->bbPawns[WHITE]) + FastCountBits(pos->bbPawns[BLACK])) -
(pHash->uNumUnmovedPawns[WHITE] + pHash->uNumUnmovedPawns[BLACK]) +
(pHash->uNumRammedPawns);
ASSERT(pos->uClosedScaler >= 0);
ASSERT(pos->uClosedScaler <= 32);
+ pos->uMinorsAtHome[BLACK] = pos->uMinorsAtHome[WHITE] = 0;
//
// Evaluate individual pieces.
//
- // board_representation/EVAL.md section 0/2: replaces the old
- // cNonPawns[color][1..N] walk + per-piece mailbox lookup + p&0x4/
- // IS_KNIGHT branch dispatch (a hard-to-predict branch per piece,
- // on top of a redundant mailbox read the callee already redoes
- // for its own ASSERT) with a direct per-type location-bitboard
- // walk. pos->bbPieces[color][PIECE_TYPE] already exists,
- // incrementally maintained by move.c, and was simply unused by
- // Eval() until now -- no new infrastructure, purely a consumer
- // change. Also retires the cDefer/uDefer "remember the rooks and
- // queens for later" bookkeeping entirely: since each type is now
- // its own direct bitboard walk, "evaluate rooks after minors" is
- // just "do the rook-bitboard walk after the minor-bitboard walks"
- // -- nothing to defer.
- //
- // board_representation/EVAL.md section 9: the real dependency
- // isn't "side to move first" -- it's "both colors' minors done
- // before any rook or queen, both rooks done before any queen,
- // everything above done before either king" (_EvalRook/_EvalQueen
- // read the enemy's pos->bbMinorAttacks; _EvalKing reads both
- // colors' bbMinorAttacks/bbQueenAttacks; _ReEvalPassers further
- // down needs the king scores). Interleaving ours-then-enemy within
- // each type (rather than both colors' minors, then both colors'
- // rooks, then both colors' queens as separate phases) still
- // satisfies that same ordering -- knight-vs-bishop order *within*
- // a color, and encounter order within a single type's own bitboard
- // walk, were never meaningful (cNonPawns' order is arbitrary
- // swap-with-last-on-removal, not stable) and stay that way --
- // EVAL_TERM's plain score accumulation doesn't care.
- //
- pos->uMinorsAtHome[BLACK] = pos->uMinorsAtHome[WHITE] = 0;
- uColor = pos->uToMove;
- xColor = FLIP(uColor);
// Knights.
bb = pos->bbPieces[uColor][KNIGHT];
@@ -6182,8 +7205,12 @@ Return value:
TIMED_EVAL_CALL(ctx, u64CyclesEvalKnight,
_EvalKnight(pos, c, pHash));
#ifdef EVAL_DUMP
- Trace("After our N at %s:\n%d\t\t%d\n",
- CoorToString(c), pos->iScore[WHITE], pos->iScore[BLACK]);
+ Trace("After our N at %s:\n%d\t\t%d\t\t(%d for %s)\n",
+ CoorToString(c),
+ pos->iScore[WHITE], pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
}
@@ -6196,10 +7223,14 @@ Return value:
bb &= (bb - 1);
c = BIT_NUMBER_TO_COOR(u);
TIMED_EVAL_CALL(ctx, u64CyclesEvalKnight,
- _EvalKnight(pos, c, pHash));
+ _EvalKnight(pos, c, pHash));
#ifdef EVAL_DUMP
- Trace("After enemy N at %s:\n%d\t\t%d\n",
- CoorToString(c), pos->iScore[WHITE], pos->iScore[BLACK]);
+ Trace("After enemy N at %s:\n%d\t\t%d\t\t(%d for %s)\n",
+ CoorToString(c),
+ pos->iScore[WHITE], pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
}
@@ -6211,10 +7242,14 @@ Return value:
bb &= (bb - 1);
c = BIT_NUMBER_TO_COOR(u);
TIMED_EVAL_CALL(ctx, u64CyclesEvalBishop,
- _EvalBishop(pos, c, pHash));
+ _EvalBishop(pos, c, pHash));
#ifdef EVAL_DUMP
- Trace("After our B at %s:\n%d\t\t%d\n",
- CoorToString(c), pos->iScore[WHITE], pos->iScore[BLACK]);
+ Trace("After our B at %s:\n%d\t\t%d\t\t(%d for %s)\n",
+ CoorToString(c),
+ pos->iScore[WHITE], pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
}
@@ -6228,8 +7263,12 @@ Return value:
TIMED_EVAL_CALL(ctx, u64CyclesEvalBishop,
_EvalBishop(pos, c, pHash));
#ifdef EVAL_DUMP
- Trace("After enemy B at %s:\n%d\t\t%d\n",
- CoorToString(c), pos->iScore[WHITE], pos->iScore[BLACK]);
+ Trace("After enemy B at %s:\n%d\t\t%d\t\t(%d for %s)\n",
+ CoorToString(c),
+ pos->iScore[WHITE], pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
}
@@ -6242,8 +7281,12 @@ Return value:
c = BIT_NUMBER_TO_COOR(u);
TIMED_EVAL_CALL(ctx, u64CyclesEvalRook, _EvalRook(pos, c, pHash));
#ifdef EVAL_DUMP
- Trace("After our R at %s:\n%d\t\t%d\n",
- CoorToString(c), pos->iScore[WHITE], pos->iScore[BLACK]);
+ Trace("After our R at %s:\n%d\t\t%d\t\t(%d for %s)\n",
+ CoorToString(c),
+ pos->iScore[WHITE], pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
}
@@ -6256,8 +7299,12 @@ Return value:
c = BIT_NUMBER_TO_COOR(u);
TIMED_EVAL_CALL(ctx, u64CyclesEvalRook, _EvalRook(pos, c, pHash));
#ifdef EVAL_DUMP
- Trace("After enemy R at %s:\n%d\t\t%d\n",
- CoorToString(c), pos->iScore[WHITE], pos->iScore[BLACK]);
+ Trace("After enemy R at %s:\n%d\t\t%d\t\t(%d for %s)\n",
+ CoorToString(c),
+ pos->iScore[WHITE], pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
}
@@ -6270,8 +7317,12 @@ Return value:
c = BIT_NUMBER_TO_COOR(u);
TIMED_EVAL_CALL(ctx, u64CyclesEvalQueen, _EvalQueen(pos, c, pHash));
#ifdef EVAL_DUMP
- Trace("After our Q at %s:\n%d\t\t%d\n",
- CoorToString(c), pos->iScore[WHITE], pos->iScore[BLACK]);
+ Trace("After our Q at %s:\n%d\t\t%d\t\t(%d for %s)\n",
+ CoorToString(c),
+ pos->iScore[WHITE], pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
}
@@ -6284,8 +7335,12 @@ Return value:
c = BIT_NUMBER_TO_COOR(u);
TIMED_EVAL_CALL(ctx, u64CyclesEvalQueen, _EvalQueen(pos, c, pHash));
#ifdef EVAL_DUMP
- Trace("After enemy Q at %s:\n%d\t\t%d\n",
- CoorToString(c), pos->iScore[WHITE], pos->iScore[BLACK]);
+ Trace("After enemy Q at %s:\n%d\t\t%d\t\t(%d for %s)\n",
+ CoorToString(c),
+ pos->iScore[WHITE], pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
}
@@ -6293,7 +7348,6 @@ Return value:
// once here rather than in _EvalKing itself -- it's called twice
// below (once per king) and used to recompute the identical
// TotalNonPawnCount(WHITE)+TotalNonPawnCount(BLACK) sum both times.
- {
ULONG uTotalNonPawnsBothSides =
TotalNonPawnCount(pos, WHITE) + TotalNonPawnCount(pos, BLACK);
@@ -6310,8 +7364,12 @@ Return value:
TIMED_EVAL_CALL(ctx, u64CyclesEvalKing,
_EvalKing(pos, c, pHash, ctx, uTotalNonPawnsBothSides));
#ifdef EVAL_DUMP
- Trace("After our K at %s:\n%d\t\t%d\n",
- CoorToString(c), pos->iScore[WHITE], pos->iScore[BLACK]);
+ Trace("After our K at %s:\n%d\t\t%d\t\t(%d for %s)\n",
+ CoorToString(c),
+ pos->iScore[WHITE], pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
c = KingPosition(pos, xColor);
@@ -6326,10 +7384,13 @@ Return value:
#endif
TIMED_EVAL_CALL(ctx, u64CyclesEvalKing,
_EvalKing(pos, c, pHash, ctx, uTotalNonPawnsBothSides));
- }
#ifdef EVAL_DUMP
- Trace("After enemy K at %s:\n%d\t\t%d\n",
- CoorToString(c), pos->iScore[WHITE], pos->iScore[BLACK]);
+ Trace("After enemy K at %s:\n%d\t\t%d\t\t(%d for %s)\n",
+ CoorToString(c),
+ pos->iScore[WHITE], pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
// Now that we have the whole attack table generated, think about
@@ -6343,71 +7404,171 @@ Return value:
{
_ReEvalPassers(pos, pHash);
#ifdef EVAL_DUMP
- Trace("After passers:\n%d\t\t%d\n", pos->iScore[WHITE],
- pos->iScore[BLACK]);
+ Trace("After re-eval passers:\n%d\t\t%d\t\t(%d for %s)\n",
+ pos->iScore[WHITE], pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
}
- //
// _EvalLookForDanger/_EvalTrappedPieces only ever *add* a hint
// when they find one; a full eval that finds nothing this time
// would otherwise leave a stale, unrelated hint from an earlier
// visit to this ply sitting here. Clear this ply's self-danger
// slots first so a clean full eval reliably means a clean slate.
- //
ctx->cEnprise[ctx->uPly][0] = ctx->cEnprise[ctx->uPly][1] = ILLEGAL_COOR;
ctx->cTrapped[ctx->uPly] = ILLEGAL_COOR;
_EvalLookForDanger(ctx);
_EvalTrappedPieces(ctx);
- // We are now committed to the full-eval path (no more lazy/
- // super-lazy exits below this point) -- stamp this ply's
- // PLY_INFO with the position we just fully evaluated so every
- // "REQUIRES SIG MATCH" field this call populates (iKingScore
- // below; cEnprise/cTrapped above; the eval-scoped bb*Attacks
- // tables) can be validated by a downstream reader with one cheap
- // compare against this position's live signature, instead of the
- // old per-field piece-identity re-check. See PLY_INFO's own
- // comment in chess.h for why this, not u64Sig, is the field to
- // compare against.
- ctx->sPlyInfo[ctx->uPly].u64FullEvalSig =
- (pos->u64NonPawnSig ^ pos->u64PawnSig);
-
// TODO: endgame-specific knowledge? e.g. B over N in an endgame
// with 2 pawn wings?
// Roll in the reduced material down scaler terms (aggregate).
ASSERT(pos->iReducedMaterialDownScaler[BLACK] > -200);
ASSERT(pos->iReducedMaterialDownScaler[BLACK] < +200);
- iAlphaMargin = (pos->iReducedMaterialDownScaler[BLACK] *
- (SCORE)REDUCED_MATERIAL_DOWN_SCALER[pos->uArmyScaler[BLACK]]) / 8;
- pos->iScore[BLACK] += iAlphaMargin;
+ pos->iScore[BLACK] += (
+ pos->iReducedMaterialDownScaler[BLACK] *
+ (SCORE)REDUCED_MATERIAL_DOWN_SCALER[pos->uArmyScaler[BLACK]]
+ ) / 8;
ASSERT(pos->iReducedMaterialDownScaler[WHITE] > -200);
ASSERT(pos->iReducedMaterialDownScaler[WHITE] < +200);
- iAlphaMargin = (pos->iReducedMaterialDownScaler[WHITE] *
- (SCORE)REDUCED_MATERIAL_DOWN_SCALER[pos->uArmyScaler[WHITE]]) / 8;
- pos->iScore[WHITE] += iAlphaMargin;
+ pos->iScore[WHITE] += (
+ pos->iReducedMaterialDownScaler[WHITE] *
+ (SCORE)REDUCED_MATERIAL_DOWN_SCALER[pos->uArmyScaler[WHITE]]
+ ) / 8;
#ifdef EVAL_TIME
ctx->sCounters.tree.u64CyclesEvalPostLazyMisc +=
(SystemReadTimeStampCounter() - u64PostLazyMiscTimer);
#endif
#ifdef EVAL_DUMP
- Trace("After all pieces:\n%d\t\t%d\n",
- pos->iScore[WHITE], pos->iScore[BLACK]);
+ Trace("After all pieces:\n%d\t\t%d\t\t(%d for %s)\n",
+ pos->iScore[WHITE], pos->iScore[BLACK],
+ (pos->iScore[uColor] - pos->iScore[xColor]),
+ COLOR_NAME(uColor)
+ );
#endif
// Almost done
- iScoreForSideToMove = (pos->iScore[pos->uToMove] -
- pos->iScore[FLIP(pos->uToMove)]);
+ iScoreForSideToMove = (pos->iScore[uColor] - pos->iScore[xColor]);
- // Drive drawish situations towards the draw score.
- TIMED_EVAL_CALL(ctx, u64CyclesEvalDrawish,
- EvalLookForDrawishSituations(pos, &iScoreForSideToMove));
+#ifdef DUMP_RESIDUAL_SAMPLES
+ // One CSV-ish line per full Eval() call: combined army scaler
+ // (total pieces proxy), signed army-scaler diff (piece-balance
+ // proxy), and each side's isolated residual (mobility/trapped/
+ // misc, with material/pawns/passers/bad-trade/bishop-pairs/king
+ // safety already subtracted out via the baseline captured after
+ // bishop pairs, above, and the real per-side king score). Sampled
+ // before EvalLookForDrawishSituations' squash below, since that
+ // squash is a separate, already-handled mechanism, not part of
+ // what iResidualP98 is meant to estimate.
+ {
+ SCORE iResidualWhite = pos->iScore[WHITE] - iResidualBaselineWhite -
+ ctx->sPlyInfo[ctx->uPly].iKingScore[WHITE];
+ SCORE iResidualBlack = pos->iScore[BLACK] - iResidualBaselineBlack -
+ ctx->sPlyInfo[ctx->uPly].iKingScore[BLACK];
- //
- // TODO: detect and discourage blocked positions?
- //
+ // Histogram, not a raw per-sample Trace -- at 30k-position
+ // scale a raw dump would run into the billions of lines (a
+ // 200-position sample alone produced 8.76M). Bucketed by OWN
+ // army scaler (color-blind: each side contributes its own
+ // row), same convention as the earlier ad hoc Python analysis.
+ _RecordResidualSample(MINU(7, pos->uArmyScaler[WHITE] / 4),
+ (INT)iResidualWhite);
+ _RecordResidualSample(MINU(7, pos->uArmyScaler[BLACK] / 4),
+ (INT)iResidualBlack);
+
+ // Super-lazy-scoped residual: same idea, but relative to
+ // super-lazy's own (material-only) baseline, so pawns/passers/
+ // bad-trade/bishop-pairs are included instead of excluded --
+ // this is what super-lazy's own margin_a/margin_b construction
+ // actually needs (2026-09-17).
+ {
+ SCORE iSLResidualWhite = pos->iScore[WHITE] -
+ iSuperLazyResidualBaselineWhite -
+ ctx->sPlyInfo[ctx->uPly].iKingScore[WHITE];
+ SCORE iSLResidualBlack = pos->iScore[BLACK] -
+ iSuperLazyResidualBaselineBlack -
+ ctx->sPlyInfo[ctx->uPly].iKingScore[BLACK];
+ _RecordSuperLazyResidualSample(MINU(7, pos->uArmyScaler[WHITE] / 4),
+ (INT)iSLResidualWhite);
+ _RecordSuperLazyResidualSample(MINU(7, pos->uArmyScaler[BLACK] / 4),
+ (INT)iSLResidualBlack);
+ }
+
+ // Outlier FEN capture -- thresholds picked comfortably beyond
+ // the p10/p90 range seen in every bucket of the 200-position
+ // smoke sample (worst bucket's p90/p10 topped out around
+ // +/-230), so this only fires for genuine tail events worth
+ // going back and looking at by hand, not routine variance.
+ // 2026-09-17: capped -- the 200-position sample this threshold
+ // was tuned against never came close to needing a cap, but the
+ // far larger/more-varied 30k stratified pool hits this on a
+ // real fraction of nodes and produced 2GB+ of Trace output
+ // before filling /tmp on the first attempt. The histogram data
+ // above is unaffected by this cap; only the (nice-to-have,
+ // not essential) worked-example FEN capture is limited.
+ if (((abs((int)iResidualWhite) > 250) || (abs((int)iResidualBlack) > 250)) &&
+ (g_uResidualOutliersPrinted < 200))
+ {
+ CHAR *pszOutlierFen = PositionToFen(pos);
+ if (NULL != pszOutlierFen)
+ {
+ Trace("RESIDUAL_OUTLIER_FEN resW=%d resB=%d fen=%s\n",
+ (int)iResidualWhite, (int)iResidualBlack, pszOutlierFen);
+ SystemFreeMemory(pszOutlierFen);
+ g_uResidualOutliersPrinted++;
+ }
+ }
+
+ // Same color-blind, per-side idea for king safety: each side's
+ // REAL, already-scaled king contribution (ctx->sPlyInfo[...]
+ // .iKingScore[color] -- scaled by REDUCED_MATERIAL_DOWN_SCALER
+ // inside _EvalKing already, so this is the true realized
+ // magnitude, not the pre-scale raw value the old iKingSwingP90
+ // table was fit against), paired with the ATTACKER's own army
+ // (the side actually threatening that king), not the king
+ // owner's. Bucketing directly by attacker army this way means
+ // no runtime rescale step is needed at all in the redesign --
+ // the bucketing itself carries the army dependence.
+ _RecordKingSafetySample(MINU(7, pos->uArmyScaler[BLACK] / 4),
+ (INT)ctx->sPlyInfo[ctx->uPly].iKingScore[WHITE]);
+ _RecordKingSafetySample(MINU(7, pos->uArmyScaler[WHITE] / 4),
+ (INT)ctx->sPlyInfo[ctx->uPly].iKingScore[BLACK]);
+
+ // Same idea as the residual outlier capture above -- threshold
+ // picked beyond the p10/p90 range seen in every king-safety
+ // bucket (worst bucket topped out around +/-160).
+ if (((abs((int)ctx->sPlyInfo[ctx->uPly].iKingScore[WHITE]) > 200) ||
+ (abs((int)ctx->sPlyInfo[ctx->uPly].iKingScore[BLACK]) > 200)) &&
+ (g_uKingSafetyOutliersPrinted < 200))
+ {
+ CHAR *pszOutlierFen = PositionToFen(pos);
+ if (NULL != pszOutlierFen)
+ {
+ Trace("KINGSAFETY_OUTLIER_FEN kingW=%d kingB=%d fen=%s\n",
+ (int)ctx->sPlyInfo[ctx->uPly].iKingScore[WHITE],
+ (int)ctx->sPlyInfo[ctx->uPly].iKingScore[BLACK],
+ pszOutlierFen);
+ SystemFreeMemory(pszOutlierFen);
+ g_uKingSafetyOutliersPrinted++;
+ }
+ }
+ }
+#endif
+
+ // Drive drawish situations towards the draw score. Usually a
+ // no-op fetch here (sDrawishFacts already computed by the
+ // regular-lazy attempt above) -- only does real work when that
+ // attempt was skipped entirely (bare-king/low-material nodes,
+ // see LAZY_EVAL_MIN_MATERIAL/fBareKingEitherSide above), in which
+ // case this is that node's only call site.
+ TIMED_EVAL_CALL(ctx, u64CyclesEvalDrawish,
+ (_EnsureDrawishFactsComputed(pos, uColor, &sDrawishFacts),
+ EvalLookForDrawishSituations(pos, &iScoreForSideToMove,
+ &sDrawishFacts)));
// Adjust dynamic positional component. Unlike the lazy-exit
// margins above (always >= 0 by construction), this can go
@@ -6459,12 +7620,18 @@ Return value:
(iSavedSuperLazyScore < iAlpha)
? MAX0(iScoreForSideToMove - iSavedSuperLazyScore)
: MAX0(iSavedSuperLazyScore - iScoreForSideToMove),
- iSuperLazyMargin);
+ iSavedSuperLazyMarginUsed);
iScoreForSideToMove = iSavedSuperLazyScore;
if (NULL != piPositional)
{
- (*piPositional)[WHITE] = 0;
- (*piPositional)[BLACK] = 0;
+ ULONG uWhiteBucket = MINU(7, pos->uArmyScaler[WHITE] / 4);
+ ULONG uBlackBucket = MINU(7, pos->uArmyScaler[BLACK] / 4);
+ (*piPositional)[WHITE] =
+ iKingP50ByAttackerArmyOnly[uBlackBucket] +
+ iSuperLazyResidualP50ByOwnArmy[uWhiteBucket];
+ (*piPositional)[BLACK] =
+ iKingP50ByAttackerArmyOnly[uWhiteBucket] +
+ iSuperLazyResidualP50ByOwnArmy[uBlackBucket];
// CALIBRATE_MARGIN_SAFETY builds: the FULL tag set above
// reflects the fallthrough that just ran for measurement
// purposes, not what a normal build actually returns here
@@ -6497,8 +7664,8 @@ Return value:
iScoreForSideToMove = iSavedLazyScore;
if (NULL != piPositional)
{
- (*piPositional)[WHITE] = iSavedLazyPositional;
- (*piPositional)[BLACK] = iSavedLazyPositional;
+ (*piPositional)[WHITE] = iSavedLazyPositionalWhite;
+ (*piPositional)[BLACK] = iSavedLazyPositionalBlack;
ctx->sPlyInfo[ctx->uPly].uLastPositionalSource = EVAL_POSITIONAL_SOURCE_LAZY;
}
}
@@ -6511,6 +7678,7 @@ Return value:
uTimer);
#endif
INC(ctx->sCounters.tree.u64FullEvals);
+ g_u64FullEvalsByBucket[_MaterialBucket(pos)]++;
end:
ASSERT(IS_VALID_SCORE(iScoreForSideToMove));
diff --git a/src/eval_tune/build_curated_texel_pool.py b/src/eval_tune/build_curated_texel_pool.py
new file mode 100644
index 0000000..86f61b8
--- /dev/null
+++ b/src/eval_tune/build_curated_texel_pool.py
@@ -0,0 +1,90 @@
+#!/usr/bin/env python3
+"""Turn a match_controls.py-matched king-safety CSV (mate/control pairs,
+fen retained -- see extract_king_features.py's 2026-09-15 fen-retention
+fix) into a Texel-ready (fen, result) pool: runs the same qsearch-
+agreement quiet filter used for the generic TWIC pool (this data was
+NEVER filtered for quietness at mining time by design -- see mine_king_
+mates.py's is_quiet_position, added 2026-09-15 -- so this step matters
+here at least as much as it does for TWIC, arguably more, since "N plies
+before a real mate" specifically targets the moment before a real attack
+lands, exactly where a quiet-by-SEE-but-not-by-search position is most
+likely).
+
+Result-label framing (a real modeling choice, not a literal game
+result): label==1 rows are "this color's king got mated in this game
+from here" -- treated as a loss (0.0) for that color. label==0 rows are
+the matched sound/control side -- treated as a win (1.0) for that color.
+Output is always from WHITE's POV (matching eval_white_pov_batch's own
+convention), so a row is flipped if mated_color_or_sampled_color is
+black. This is an approximation (these aren't actual final game
+results, just "survived this specific king-safety episode or not") --
+flagged explicitly rather than silently treated as equivalent to a real
+Texel outcome label.
+"""
+import argparse
+import csv
+import sys
+
+from tune_eval_dna import Engine, filter_by_qsearch_agreement
+
+
+def main():
+ ap = argparse.ArgumentParser()
+ ap.add_argument("matched_csv")
+ ap.add_argument("out_tsv")
+ ap.add_argument("--engine", default="../typhoon")
+ ap.add_argument("--qsearch-max-diff", type=int, default=30)
+ ap.add_argument("--dna-scratch-dir", default="/tmp/typhoon_curated_pool")
+ args = ap.parse_args()
+
+ with open(args.matched_csv, newline="") as f:
+ rows = list(csv.DictReader(f))
+ print(f"Loaded {len(rows)} matched rows from {args.matched_csv}",
+ file=sys.stderr)
+
+ fen_to_row = {}
+ for r in rows:
+ fen_to_row[r["fen"]] = r
+ if len(fen_to_row) != len(rows):
+ print(f"WARNING: {len(rows) - len(fen_to_row)} duplicate FENs "
+ f"collapsed -- matched.csv shouldn't have dupes normally, "
+ f"investigate if this is large.", file=sys.stderr)
+
+ # (fen, result) pairs for the quiet-filter -- result value here is a
+ # placeholder (unused by filter_by_qsearch_agreement, which only
+ # reads/returns fens), real result labels are attached after.
+ placeholder_positions = [(fen, 0.0) for fen in fen_to_row]
+
+ with Engine(args.engine, args.dna_scratch_dir) as engine:
+ kept = filter_by_qsearch_agreement(
+ engine, placeholder_positions, max_diff_cp=args.qsearch_max_diff
+ )
+
+ n_flipped = 0
+ out_rows = []
+ for fen, _ in kept:
+ r = fen_to_row[fen]
+ label = int(r["label"])
+ color = r["mated_color_or_sampled_color"]
+ # label==1 (mate) -> loss (0.0) for that color; label==0
+ # (control/sound) -> win (1.0) for that color.
+ result_for_color = 0.0 if label == 1 else 1.0
+ if color == "black":
+ result_white_pov = 1.0 - result_for_color
+ n_flipped += 1
+ else:
+ result_white_pov = result_for_color
+ out_rows.append((fen, result_white_pov))
+
+ with open(args.out_tsv, "w") as f:
+ for fen, result in out_rows:
+ f.write(f"{fen}\t{result}\n")
+
+ print(f"Quiet-filter kept {len(kept)}/{len(fen_to_row)} "
+ f"({len(kept)/len(fen_to_row):.1%})", file=sys.stderr)
+ print(f"Wrote {len(out_rows)} (fen, result) rows to {args.out_tsv} "
+ f"({n_flipped} POV-flipped from black)", file=sys.stderr)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/src/eval_tune/compare_king_safety_crafty.py b/src/eval_tune/compare_king_safety_crafty.py
new file mode 100644
index 0000000..a018fc7
--- /dev/null
+++ b/src/eval_tune/compare_king_safety_crafty.py
@@ -0,0 +1,112 @@
+#!/usr/bin/env python3
+"""Empirical magnitude sanity check: for the same ~500 curated near-mate
+"quiet" positions used to derive the king-safety ring-defect features
+(see mine_king_mates.py / matched_combined_full.csv), compare typhoon's
+own king-safety subscore (iSafety, dumped via KING_DEFECT_DUMP's new
+'realsafety' field -- see chess.h/eval.c/command.c 2026-09-15 additions)
+against Crafty's "kings" row from its `score` command breakdown, for the
+side whose king safety is actually being assessed in each row.
+
+Crafty is spawned fresh (one process per FEN, via `setboard <fen>;
+score; quit`) rather than batched in one long-lived process -- testing
+showed a second setboard+score call inside one Crafty session silently
+reused the FIRST position's cached score, not a real re-evaluation, so
+batching would have silently produced 500 copies of the same number.
+"""
+import csv
+import re
+import subprocess
+import sys
+
+CRAFTY_BIN = "/home/scott/crafty/crafty"
+CRAFTY_CWD = "/home/scott/crafty"
+TYPHOON_KINGDUMP = "/usr/home/scott/typhoon/src/typhoon_kingdump"
+IN_CSV = "/tmp/typhoon/near_mate_500.csv"
+OUT_CSV = "/tmp/typhoon/king_safety_crafty_compare.csv"
+
+KINGS_ROW_RE = re.compile(
+ r"kings\.+\s+([-\d.]+)\s+\|\s+([-\d.]+)\s+([-\d.]+)\s+([-\d.]+)\s+\|\s+"
+ r"([-\d.]+)\s+([-\d.]+)\s+([-\d.]+)\s+\|"
+)
+
+
+def crafty_kings_comp(fen, color):
+ """Returns Crafty's 'kings' row 'comp' column (in pawns) for `color`."""
+ script = f"setboard {fen}\nscore\nquit\n"
+ p = subprocess.run([CRAFTY_BIN], input=script, cwd=CRAFTY_CWD,
+ capture_output=True, text=True, timeout=30)
+ m = KINGS_ROW_RE.search(p.stdout)
+ if not m:
+ return None
+ white_comp, black_comp = float(m.group(1)), float(m.group(5))
+ return white_comp if color == "white" else black_comp
+
+
+def load_rows():
+ with open(IN_CSV, newline="") as f:
+ return list(csv.DictReader(f))
+
+
+def typhoon_realsafety_batch(rows):
+ """Batches all FENs through one typhoon_kingdump `script` call (this
+ engine's setboard/eval IS known to work correctly across repeated
+ calls in one session -- see extract_king_features.py's run_shard)."""
+ script_path = "/tmp/typhoon/compare_batch.script"
+ out_path = "/tmp/typhoon/compare_batch.out"
+ with open(script_path, "w") as f:
+ f.write("force\nnopost\n")
+ for r in rows:
+ f.write(f"setboard {r['fen']}\n")
+ f.write("evalkingdump\n")
+ cmd = [TYPHOON_KINGDUMP, "--batch", "--command", f"script {script_path}"]
+ with open(out_path, "w") as logf:
+ subprocess.run(cmd, stdout=logf, stderr=subprocess.STDOUT, timeout=300)
+ dumps = []
+ with open(out_path) as f:
+ for line in f:
+ if line.startswith("KINGDEFECTDUMP"):
+ color = re.search(r"color=(\S+)", line).group(1)
+ realsafety = int(re.search(r"realsafety=(-?\d+)", line).group(1))
+ dumps.append((color, realsafety))
+ if len(dumps) != 2 * len(rows):
+ raise RuntimeError(f"expected {2*len(rows)} dump lines, got {len(dumps)}")
+ out = []
+ for i, r in enumerate(rows):
+ black_color, black_rs = dumps[2 * i]
+ white_color, white_rs = dumps[2 * i + 1]
+ assert black_color == "black" and white_color == "white"
+ out.append(black_rs if r["mated_color_or_sampled_color"] == "black" else white_rs)
+ return out
+
+
+def main():
+ rows = load_rows()
+ print(f"Loaded {len(rows)} rows", file=sys.stderr)
+
+ typhoon_vals = typhoon_realsafety_batch(rows)
+ print(f"typhoon: got {len(typhoon_vals)} realsafety values", file=sys.stderr)
+
+ results = []
+ for i, r in enumerate(rows):
+ crafty_comp = crafty_kings_comp(r["fen"], r["mated_color_or_sampled_color"])
+ if crafty_comp is None:
+ continue
+ results.append({
+ "fen": r["fen"],
+ "color": r["mated_color_or_sampled_color"],
+ "typhoon_realsafety_cp": typhoon_vals[i],
+ "crafty_kings_comp_cp": round(crafty_comp * 100, 1),
+ })
+ if (i + 1) % 50 == 0:
+ print(f" ...{i+1}/{len(rows)} crafty positions done", file=sys.stderr)
+
+ with open(OUT_CSV, "w", newline="") as f:
+ w = csv.DictWriter(f, fieldnames=["fen", "color", "typhoon_realsafety_cp",
+ "crafty_kings_comp_cp"])
+ w.writeheader()
+ w.writerows(results)
+ print(f"Wrote {len(results)} rows to {OUT_CSV}", file=sys.stderr)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/src/eval_tune/fit_king_safety_logistic_material.py b/src/eval_tune/fit_king_safety_logistic_material.py
new file mode 100644
index 0000000..6f364f3
--- /dev/null
+++ b/src/eval_tune/fit_king_safety_logistic_material.py
@@ -0,0 +1,147 @@
+#!/usr/bin/env python3
+"""Logistic-regression calibration of king-safety ring-defect terms
+against material balance as an external, non-circular yardstick.
+
+Why material balance and not the engine's own iSafety: iSafety is
+mechanically a weighted sum of (a subset of) these same ring-defect
+features under the CURRENT baseline weights -- using it as a covariate
+regresses the model against itself, collinear by construction, not just
+"maybe wrong". Material balance is computed independently, straight from
+the FEN via python-chess with standard piece values -- it has nothing to
+do with king-ring geometry and is anchored to a genuinely external,
+centuries-old convention (pawn=100, knight/bishop=~320, rook=500,
+queen=900), the same scale eval.c's own PIECE_VALUE table already uses
+elsewhere in the engine, unrelated to the king-safety subsystem.
+
+Logistic (not linear) regression on the same curated mate/control label,
+material_balance included as one covariate: its fitted coefficient tells
+you how many logit-units one pawn (100cp) is worth in this exact
+dataset. Divide every king-safety feature's coefficient by that, times
+100, and you get that feature's danger contribution in real centipawns,
+calibrated against an external ruler -- no Texel, no single hand-picked
+anchor constant.
+"""
+import csv
+import sys
+import chess
+import numpy as np
+from sklearn.linear_model import LogisticRegression
+
+sys.path.insert(0, "/usr/home/scott/typhoon/src/eval_tune")
+from fit_king_safety_ridge import tier_slots
+
+IN_CSV = sys.argv[1] if len(sys.argv) > 1 else "/tmp/typhoon/matched_combined_full.csv"
+
+PIECE_VALUES = {
+ chess.PAWN: 1.0,
+ chess.KNIGHT: 3.2,
+ chess.BISHOP: 3.3,
+ chess.ROOK: 5.0,
+ chess.QUEEN: 9.0,
+ chess.KING: 0.0,
+}
+
+NONPAWN_PIECES = ("minor", "rook", "queen")
+
+
+def material_balance_pawns(fen, color):
+ """Material balance in pawns, from `color`'s point of view."""
+ board = chess.Board(fen)
+ total = {chess.WHITE: 0.0, chess.BLACK: 0.0}
+ for piece_type, value in PIECE_VALUES.items():
+ for c in (chess.WHITE, chess.BLACK):
+ total[c] += value * len(board.pieces(piece_type, c))
+ my_color = chess.WHITE if color == "white" else chess.BLACK
+ other = chess.BLACK if my_color == chess.WHITE else chess.WHITE
+ return total[my_color] - total[other]
+
+
+def load_rows(path):
+ with open(path, newline="") as f:
+ return list(csv.DictReader(f))
+
+
+def build_features(rows_in):
+ feat_rows = []
+ y = []
+ col_names = None
+ skipped = 0
+ for r in rows_in:
+ color = r["mated_color_or_sampled_color"]
+
+ def offboard(s):
+ return int(r.get(f"sq{s}_offboard", 0) or 0) != 0
+
+ feat = {}
+ for tier in ("front", "back", "side"):
+ slots = tier_slots(color, tier)
+ for p in NONPAWN_PIECES:
+ feat[f"{tier}_atk_{p}"] = sum(
+ int(r[f"sq{s}_atk_{p}"]) for s in slots if not offboard(s)
+ )
+ feat[f"{tier}_undef"] = sum(
+ int(r[f"sq{s}_undef"]) for s in slots if not offboard(s)
+ )
+ feat["ring_atk_pawn"] = 0
+ feat["ring_xray"] = 0
+ for tier in ("front", "back", "side"):
+ slots = tier_slots(color, tier)
+ feat["ring_atk_pawn"] += sum(
+ int(r[f"sq{s}_atk_pawn"]) for s in slots if not offboard(s)
+ )
+ feat["ring_xray"] += sum(
+ int(r[f"sq{s}_xray"]) for s in slots if not offboard(s)
+ )
+ feat["front_flight"] = sum(
+ int(r[f"sq{s}_flight"]) for s in tier_slots(color, "front")
+ if not offboard(s)
+ )
+ feat["back_flight"] = sum(
+ int(r[f"sq{s}_flight"]) for s in tier_slots(color, "back")
+ if not offboard(s)
+ )
+ try:
+ feat["material_balance"] = material_balance_pawns(r["fen"], color)
+ except Exception:
+ skipped += 1
+ continue
+ if col_names is None:
+ col_names = list(feat.keys())
+ feat_rows.append([feat[c] for c in col_names])
+ y.append(int(r["label"]))
+ if skipped:
+ print(f"skipped {skipped} rows with unparseable FENs", file=sys.stderr)
+ return np.array(feat_rows, dtype=float), np.array(y, dtype=int), col_names
+
+
+def main():
+ rows_in = load_rows(IN_CSV)
+ print(f"Loaded {len(rows_in)} rows from {IN_CSV}")
+ X, y, col_names = build_features(rows_in)
+ print(f"Built {X.shape[1]} features (incl. material_balance): {col_names}")
+
+ C = float(sys.argv[2]) if len(sys.argv) > 2 else 0.05
+ m = LogisticRegression(penalty="l1", solver="liblinear", C=C, max_iter=5000)
+ m.fit(X, y)
+ coef = m.coef_[0]
+
+ mat_idx = col_names.index("material_balance")
+ beta_material = coef[mat_idx]
+ print(f"\nC={C} intercept={m.intercept_[0]:.4f}")
+ print(f"beta_material (logit per pawn of material) = {beta_material:.5f}")
+ if abs(beta_material) < 1e-6:
+ print("material_balance coefficient ~0 -- can't calibrate, "
+ "try a larger C (less regularization).")
+ return
+
+ print(f"\n{'feature':20s} {'beta (logit/unit)':>18s} {'cp (calibrated)':>16s}")
+ for name, b in zip(col_names, coef):
+ if name == "material_balance":
+ continue
+ cp = 100.0 * b / beta_material
+ flag = "" if abs(b) > 1e-9 else " (zeroed by L1)"
+ print(f"{name:20s} {b:18.5f} {cp:16.2f}{flag}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/src/eval_tune/fit_king_safety_ridge.py b/src/eval_tune/fit_king_safety_ridge.py
new file mode 100644
index 0000000..072c972
--- /dev/null
+++ b/src/eval_tune/fit_king_safety_ridge.py
@@ -0,0 +1,202 @@
+#!/usr/bin/env python3
+"""Sanity-check alternative to Texel for the king-safety terms: fit Ridge
+and ElasticNet regression directly against the mate/control label (1=mated
+from this position, 0=survived-to-a-real-draw control), using the same
+matched_combined_small.csv the curated-Texel run consumed (see
+build_curated_texel_pool.py / eval_tune's 2026-09-15 king-safety session).
+
+Unlike Texel, this fits a real regularized linear model in one shot (no
+coordinate descent over a live engine), so it can't reproduce Texel's
+sign-flip pathology from correlated coordinate updates stepping on each
+other -- if Ridge/ElasticNet still flip signs on physically-obvious terms
+(e.g. an undefended ring square should never be a *bonus*), that is strong
+evidence the flips are a real property of the feature collinearity, not an
+artifact of Texel's optimizer.
+
+Features are built to mirror the actual _EvalKing ring-tier terms as
+closely as the KING_DEFECT_DUMP per-square dump allows: slots 0-2 and 5-7
+swap front/back meaning by color (see data.c's InitializeKingAttackTables
+iFrontDeltas/iBackDeltas -- white front = deltas -17/-16/-15, black front =
++15/+16/+17), slots 3-4 are always the side squares. Each row's dump was
+already collapsed to the mated/sampled color's own perspective (see
+extract_king_features.py), so slot->tier mapping only depends on that row's
+color column, not board color.
+"""
+import csv
+import sys
+import numpy as np
+from sklearn.linear_model import Ridge, ElasticNet
+from sklearn.model_selection import train_test_split
+from sklearn.preprocessing import StandardScaler
+
+IN_CSV = sys.argv[1] if len(sys.argv) > 1 else "/tmp/typhoon/matched_combined_small.csv"
+
+PIECE_TYPES = ("pawn", "minor", "rook", "queen")
+WHITE_FRONT_SLOTS = (0, 1, 2)
+WHITE_BACK_SLOTS = (5, 6, 7)
+SIDE_SLOTS = (3, 4)
+
+
+def tier_slots(color, tier):
+ if tier == "side":
+ return SIDE_SLOTS
+ is_front = (tier == "front")
+ use_white_front_slots = is_front == (color == "white")
+ return WHITE_FRONT_SLOTS if use_white_front_slots else WHITE_BACK_SLOTS
+
+
+def load_rows(path):
+ with open(path, newline="") as f:
+ return list(csv.DictReader(f))
+
+
+def build_features(rows_in):
+ feat_rows = []
+ y = []
+ col_names = None
+ for r in rows_in:
+ color = r["mated_color_or_sampled_color"]
+
+ def offboard(s):
+ return int(r.get(f"sq{s}_offboard", 0) or 0) != 0
+
+ feat = {}
+ for tier in ("front", "back", "side"):
+ slots = tier_slots(color, tier)
+ for p in PIECE_TYPES:
+ feat[f"{tier}_atk_{p}"] = sum(
+ int(r[f"sq{s}_atk_{p}"]) for s in slots if not offboard(s)
+ )
+ feat[f"{tier}_undef"] = sum(
+ int(r[f"sq{s}_undef"]) for s in slots if not offboard(s)
+ )
+ feat[f"{tier}_xray"] = sum(
+ int(r[f"sq{s}_xray"]) for s in slots if not offboard(s)
+ )
+ feat[f"{tier}_flight"] = sum(
+ int(r[f"sq{s}_flight"]) for s in slots if not offboard(s)
+ )
+ feat["queenproximity"] = int(r["queenproximity"])
+ feat["filestorm"] = int(r["filestorm"])
+ feat["filestormopenfile"] = int(r["filestormopenfile"])
+ feat["filestormpawn"] = int(r["filestormpawn"])
+ feat["castled"] = int(r["castled"])
+ feat["minorsathome"] = int(r["minorsathome"])
+ feat["misscastleshort"] = int(r["misscastleshort"])
+ feat["misscastlelong"] = int(r["misscastlelong"])
+ if col_names is None:
+ col_names = list(feat.keys())
+ feat_rows.append([feat[c] for c in col_names])
+ y.append(float(r["label"]))
+ return np.array(feat_rows, dtype=float), np.array(y, dtype=float), col_names
+
+
+def report(name, model, X_cols, coefs, intercept):
+ print(f"\n=== {name} ===")
+ print(f"intercept: {intercept:.4f}")
+ order = np.argsort(-np.abs(coefs))
+ for i in order:
+ if abs(coefs[i]) < 1e-6:
+ continue
+ print(f" {X_cols[i]:28s} {coefs[i]:+.4f}")
+
+
+SANITY_KEYS = ("front_undef", "back_undef", "side_undef",
+ "front_xray", "back_xray", "side_xray",
+ "queenproximity", "filestorm")
+
+
+def sign_flips(coefs, col_names):
+ n = 0
+ for key in SANITY_KEYS:
+ if key in col_names:
+ if coefs[col_names.index(key)] < -1e-9:
+ n += 1
+ return n
+
+
+def sweep(X_train, y_train, X_test, y_test, col_names):
+ print("\n=== Ridge alpha sweep ===")
+ print(f"{'alpha':>10} {'train MSE':>10} {'test MSE':>10} {'sign flips':>11}")
+ for alpha in (0.1, 1, 3, 10, 30, 100, 300, 1000):
+ m = Ridge(alpha=alpha).fit(X_train, y_train)
+ tr = np.mean((m.predict(X_train) - y_train) ** 2)
+ te = np.mean((m.predict(X_test) - y_test) ** 2)
+ flips = sign_flips(m.coef_, col_names)
+ print(f"{alpha:>10} {tr:>10.4f} {te:>10.4f} {flips:>11}")
+
+ print("\n=== ElasticNet alpha/l1_ratio sweep ===")
+ print(f"{'alpha':>8} {'l1_ratio':>9} {'train MSE':>10} {'test MSE':>10} "
+ f"{'nnz':>4} {'sign flips':>11}")
+ for alpha in (0.002, 0.005, 0.01, 0.02, 0.05, 0.1, 0.2):
+ for l1_ratio in (0.1, 0.3, 0.5, 0.7, 0.9, 1.0):
+ m = ElasticNet(alpha=alpha, l1_ratio=l1_ratio, max_iter=20000)
+ m.fit(X_train, y_train)
+ tr = np.mean((m.predict(X_train) - y_train) ** 2)
+ te = np.mean((m.predict(X_test) - y_test) ** 2)
+ nnz = int(np.sum(np.abs(m.coef_) > 1e-9))
+ flips = sign_flips(m.coef_, col_names)
+ print(f"{alpha:>8} {l1_ratio:>9} {tr:>10.4f} {te:>10.4f} "
+ f"{nnz:>4} {flips:>11}")
+
+
+def main():
+ rows_in = load_rows(IN_CSV)
+ print(f"Loaded {len(rows_in)} rows from {IN_CSV}")
+ X, y, col_names = build_features(rows_in)
+ print(f"Built {X.shape[1]} features")
+
+ X_train, X_test, y_train, y_test = train_test_split(
+ X, y, test_size=0.15, random_state=42
+ )
+ scaler = StandardScaler().fit(X_train)
+ X_train_s = scaler.transform(X_train)
+ X_test_s = scaler.transform(X_test)
+
+ if "--sweep" in sys.argv:
+ sweep(X_train_s, y_train, X_test_s, y_test, col_names)
+ return
+
+ ridge = Ridge(alpha=10.0)
+ ridge.fit(X_train_s, y_train)
+ ridge_train_err = np.mean((ridge.predict(X_train_s) - y_train) ** 2)
+ ridge_test_err = np.mean((ridge.predict(X_test_s) - y_test) ** 2)
+ print(f"\nRidge(alpha=10): train MSE={ridge_train_err:.4f} test MSE={ridge_test_err:.4f}")
+ report("Ridge coefficients (standardized X)", ridge, col_names, ridge.coef_, ridge.intercept_)
+
+ enet_alpha = float(sys.argv[sys.argv.index("--enet-alpha") + 1]) if "--enet-alpha" in sys.argv else 0.02
+ enet_l1_ratio = float(sys.argv[sys.argv.index("--enet-l1-ratio") + 1]) if "--enet-l1-ratio" in sys.argv else 0.5
+ enet = ElasticNet(alpha=enet_alpha, l1_ratio=enet_l1_ratio, max_iter=20000)
+ enet.fit(X_train_s, y_train)
+ enet_train_err = np.mean((enet.predict(X_train_s) - y_train) ** 2)
+ enet_test_err = np.mean((enet.predict(X_test_s) - y_test) ** 2)
+ print(f"\nElasticNet(alpha={enet_alpha}, l1_ratio={enet_l1_ratio}): train MSE={enet_train_err:.4f} test MSE={enet_test_err:.4f}")
+ report("ElasticNet coefficients (standardized X)", enet, col_names, enet.coef_, enet.intercept_)
+
+ # Sign sanity check: undefended-attack terms and any "flight" term for
+ # the front tier should be non-negative-danger (i.e. push toward
+ # label==1/mate, positive coefficient here since label=1 is mate) --
+ # a *negative* coefficient on front_undef/back_undef/side_undef would
+ # mean "more undefended attacked squares near the king make mate LESS
+ # likely", which is not physically sane.
+ print("\n=== Sign sanity check (positive = more mate-associated, as expected) ===")
+ for name, coefs in (("Ridge", ridge.coef_), ("ElasticNet", enet.coef_)):
+ cols = col_names
+ bad = []
+ for key in ("front_undef", "back_undef", "side_undef",
+ "front_xray", "back_xray", "side_xray",
+ "queenproximity", "filestorm"):
+ if key in cols:
+ c = coefs[cols.index(key)]
+ flag = "" if c >= 0 else " <-- SIGN FLIP (should be >= 0)"
+ print(f" [{name}] {key:16s} {c:+.4f}{flag}")
+ if c < 0:
+ bad.append(key)
+ if bad:
+ print(f" [{name}] {len(bad)} suspicious sign(s): {bad}")
+ else:
+ print(f" [{name}] all checked terms have the expected sign")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/src/eval_tune/lazy_eval_margins/README.md b/src/eval_tune/lazy_eval_margins/README.md
new file mode 100644
index 0000000..239bb26
--- /dev/null
+++ b/src/eval_tune/lazy_eval_margins/README.md
@@ -0,0 +1,194 @@
+# Lazy/super-lazy eval margin redesign (2026-09-17)
+
+## What changed and why
+
+Original goal: speed up `Eval()` (eval.c) by letting more nodes exit
+through the cheap "super lazy" (material-only, runs before `_EvalPawns`)
+and "lazy" (material + pawns + passers + bad-trade + bishop-pairs) tiers
+instead of paying for full eval, motivated by comparing against Crafty's
+much higher NPS.
+
+The old margin construction (still visible via `git show HEAD:src/eval.c`
+at commit `4a7ec9a`) was a patchwork: a flat `LAZY_EVAL_BASE_MARGIN` (75)
+seed, widened by a `CountKingSafetyDefects`-indexed king term (`_EvalKing`'s
+own slow bitboard scan, called *again* just to estimate a margin) plus a
+flat residual constant, then `MAX()`'d against a separately-measured
+flat floor per material bucket. Super-lazy was a single flat
+`SUPER_LAZY_MARGIN_BY_ARMY[8]` lookup, no per-position information at all.
+Both had real problems: `CountKingSafetyDefects` is slow despite being
+bitboard-based (defeats the point of being in the *cheap* path), and the
+`MAX()`-against-a-flat-floor pattern silently overrode the
+position-specific estimate in most cases (found and fixed twice this
+session, once per tier).
+
+## The new construction
+
+For **both** tiers, at every node, cheap **O(1) table lookups only** (no
+`CountKingSafetyDefects`, no attack-table generation):
+
+```
+margin_a = MAX0( (king_p_hi[my_bucket] - king_p_lo[opp_bucket])
+ + (resid_p_hi[opp_bucket] - resid_p_lo[my_bucket]) )
+margin_b = MAX0( (king_p_hi[opp_bucket] - king_p_lo[my_bucket])
+ + (resid_p_hi[my_bucket] - resid_p_lo[opp_bucket]) )
+```
+
+- `margin_a` gates the fail-high check (`score - margin_a > beta`):
+ soundness requires assuming the most pessimistic-for-me,
+ optimistic-for-opponent outcome for each term.
+- `margin_b` gates the fail-low check (`score + margin_b < alpha`):
+ the mirror image.
+- **King term**: bucketed by the **attacker's** own army scaler
+ (`MINU(7, armyScaler/4)`, 8 buckets, 0-31 scaler range) -- danger to a
+ king is a function of the attacker's material, not the king-owner's.
+ Shared between both tiers (`iKingP2ByAttackerArmyOnly` /
+ `iKingP98ByAttackerArmyOnly` in eval.c).
+- **Residual term** (mobility/trapped/passers-not-yet-counted/etc.):
+ bucketed by **each side's own** army scaler. Two separate tables
+ because the two tiers have different baselines to estimate *from*:
+ - `iResidualP2/98ByOwnArmy` -- regular lazy's baseline already
+ includes pawns/passers/bad-trade/bishop-pairs, so this residual
+ only needs to cover what's left (mobility, trapped pieces, etc).
+ - `iSuperLazyResidualP2/98ByOwnArmy` -- super-lazy's baseline is pure
+ material, so this residual is much wider (includes everything
+ regular lazy's own residual excludes).
+
+`_hi`/`_lo` are currently p98/p2 as shipped, but genuinely **per-bucket
+tunable** -- see "How to keep tuning" below. The king table has very
+little spread above p90 in every bucket (already checked against the
+full histogram) and was deliberately left alone throughout tuning;
+all of today's percentile-adjustment work only ever touched the two
+residual tables.
+
+`iPositional[side]` (the value fed back to QSearch's futility pruning,
+`search.c`'s `FUTILITY_BASE_MARGIN_BY_SOURCE[tier] + iPositional[side]`)
+is now a **separate**, deliberately different statistic: a **p50**
+(median, not tail-bound) typical-value estimate of that side's own
+missing terms, matching what full-eval's own `iPositional` actually is
+(a real point estimate, not a safety margin). See `iKingP50By...` /
+`iResidualP50By...` / `iSuperLazyResidualP50By...` in eval.c. Don't
+confuse these with the p98/p2 margin tables -- different statistic,
+different purpose (bias correction vs. soundness bound).
+
+## Where the histogram data lives
+
+`eval.c`'s `_PersistRawHistogramsToFile()` (called automatically every
+time a `DUMP_RESIDUAL_SAMPLES` build runs `calibrate residual`) writes
+**every raw bin** of all three histograms (residual, sl_residual, king)
+to `/var/tmp/typhoon/residual_raw_histograms.txt` -- not just the
+handful of percentile columns that happen to be printed to the console.
+This directory (`src/eval_tune/lazy_eval_margins/`) has copies from the
+2026-09-17 "guess #2" 30k-position calibration run
+(`residual_raw_histograms_guess2_30k.txt`) plus the reader script
+(`hist_percentile.py`) -- **copy the fresh file here after any future
+30k run you want to keep**, since `/var/tmp` is scratch and could be
+cleared.
+
+To read *any* percentile out of a saved histogram, without re-running
+anything:
+
+```sh
+python3 hist_percentile.py residual_raw_histograms_guess2_30k.txt # default spread
+python3 hist_percentile.py residual_raw_histograms_guess2_30k.txt 0.93 0.965 # exact percentiles you want
+```
+
+## How to keep tuning
+
+1. **Build**: `gmake clean && gmake -j5 GENETIC=1 PERF_COUNTERS=1 MP=1 SIXTYFOUR=1 DUMP_RESIDUAL_SAMPLES=1 CALIBRATE_MARGIN_SAFETY=1`
+2. **Run the 30k stratified pool** (both commands in one invocation --
+ `calibrate residual` persists the histogram file, `calibrate
+ marginsafety` prints the real exceeded rate per **material** bucket
+ for both tiers):
+ ```sh
+ ./typhoon --cpus 1 --hash 256m --logfile /var/tmp/typhoon/run.log \
+ --batch --command "force; book name /nonexistent.book.bin; sd 6; \
+ script ../tests/eval_calibration_sample_strat30k.ep_; \
+ calibrate residual; calibrate marginsafety" \
+ > /var/tmp/typhoon/run.stdout 2>&1
+ ```
+ Takes roughly 20-25 minutes. Run it in the background
+ (`run_in_background`/`&`), don't block on it.
+3. **Read the exceeded rate per material bucket** from the
+ `calibrate marginsafety` output (`exceeded / n`, printed separately
+ for "Regular lazy" and "Super lazy", by **material** bucket --
+ combined army scaler / 8, a *different* bucketing than the
+ percentile tables' per-side scaler/4 buckets -- see the caveat
+ below).
+4. **Copy the fresh `/var/tmp/typhoon/residual_raw_histograms.txt`**
+ into this directory before it can get lost.
+5. **Pick new percentiles per per-side bucket** using
+ `hist_percentile.py` against the fresh file, and edit
+ `iResidualP2/98ByOwnArmy` / `iSuperLazyResidualP2/98ByOwnArmy` in
+ eval.c directly (keep each bucket's low/high pair **symmetric** --
+ e.g. p95/p5 together, never p98 mixed with p10 in the same bucket).
+6. **Rebuild and re-run step 2** to verify the real effect.
+
+### The two gotchas that cost real time this session
+
+- **The 500-position "quick check" sample is unreliable for this
+ specific measurement.** It's fine for cheap direction-of-travel
+ checks on most things, but exceeded-rate is a rare-event percentage,
+ and 500 positions (not even 500 independent draws -- a single game's
+ search tree can dominate a material bucket's node count) can be off
+ by 2-5x from the true 30k-pool rate. One bucket (super-lazy material
+ bucket 1) got tightened based on a 500-position reading of 0.035%,
+ then the real 30k number came back at 1.13% -- a dangerous-direction
+ miss. **Always confirm with the full 30k pool before trusting a
+ number**, especially anything under ~1%.
+- **Changing one bucket's margin changes measured behavior in OTHER
+ buckets too**, even ones whose own table entries never moved. This
+ isn't a bug -- `Eval()`'s exit decisions are real search behavior, so
+ widening/narrowing any margin reshapes the actual search tree
+ (different alpha/beta bounds propagate differently), which changes
+ which exact nodes land in *every* material bucket, not just the one
+ you touched. Material bucket 3's exceeded rate moved from 0.21% to
+ 0.94% to 0.65% to 1.44% across different runs despite its own table
+ entries never changing once. Don't chase a bucket that "moved" if you
+ didn't touch its inputs -- check what else changed first.
+- **Material buckets and per-side buckets are different bucketing
+ schemes.** `calibrate marginsafety` reports by **material bucket**
+ (`_MaterialBucket`: combined army scaler for both sides, /8). The
+ percentile tables you actually edit are indexed by **per-side**
+ bucket (`armyScaler/4`, one bucket per side, looked up twice -- once
+ as "my bucket", once as "opponent's bucket"). A single material
+ bucket (e.g. combined scaler 24-31) is populated by many different
+ per-side bucket combinations (3+4, 2+5, 1+6, 0+7...), so there's no
+ clean 1:1 way to "fix material bucket 3" by editing "per-side bucket
+ 3" alone -- expect diffuse, imprecise control, not a dial.
+
+## On the "insane" 2% idea
+
+Structurally straightforward: pick a looser matched percentile pair
+(e.g. p90/p10, or go outside the currently-recorded columns using
+`hist_percentile.py` against a fresh histogram) for whichever buckets
+you want at ~2%, following the same edit-rebuild-reverify loop above.
+Two things worth checking once you have it:
+
+1. **NPS gain isn't guaranteed to scale linearly with the exceeded-rate
+ relaxation** -- this session found the NPS win from the whole
+ redesign was much smaller than expected (see the `git log`/session
+ history around `EvalLookForDrawishSituations` getting added to both
+ cheap tiers, and `fBareKingEitherSide`'s two unconditional
+ `TotalNonPawnCount` calls on every node -- both are new fixed costs
+ that eat into any margin-driven exit-rate gain). Worth re-checking
+ the `Eval exit breakdown` / new `calibrate exitrate` numbers
+ (below) alongside NPS, not just NPS alone.
+2. **`FUTILITY_BASE_MARGIN_LAZY`/`SUPERLAZY`** (search.c/chess.h) were
+ calibrated against *this* `iPositional` distribution
+ (`CALIBRATE_QSEARCH_FUTILITY`, see chess.h's derivation comment near
+ those two `#define`s) -- if the eval-exit margins move a lot, especially
+ in a direction that changes how often/how confidently each tier
+ exits, it's worth re-running that calibration too rather than
+ assuming the existing 350/175 values still hold.
+
+## New tooling added this session (always available, no build flag)
+
+- **`calibrate exitrate`** -- prints eval exit rate (% super lazy / %
+ lazy / % full) **by material bucket**, not just the flat aggregate
+ "Eval exit breakdown" line every search already prints. Complements
+ the exceeded-rate data: a bucket with low exit rate AND low exceeded
+ rate has real untapped speed headroom; a bucket already near its
+ target exceeded rate with a healthy exit rate is well-calibrated.
+ Implemented in eval.c (`g_u64SuperLazyEvalsByBucket` /
+ `g_u64LazyEvalsByBucket` / `g_u64FullEvalsByBucket`,
+ `DumpEvalExitRateByBucket`) and wired into command.c.
diff --git a/src/eval_tune/lazy_eval_margins/aggregate_marginsafety_shards.py b/src/eval_tune/lazy_eval_margins/aggregate_marginsafety_shards.py
new file mode 100644
index 0000000..69e5c8e
--- /dev/null
+++ b/src/eval_tune/lazy_eval_margins/aggregate_marginsafety_shards.py
@@ -0,0 +1,92 @@
+#!/usr/bin/env python3
+"""Combine N shards' persisted raw swing histograms (eval.c's
+PersistMarginSafetyHistogramsToFile, "calibrate marginsafety <path>")
+into one true aggregate: sum count/exceeded/sum/sumSq/histogram bins
+bucket-by-bucket across all shard files, then recompute mean/stdev/
+percentiles from the combined totals. Percentiles/means from
+individual shards can't be validly averaged -- this reads raw counts
+only and derives every statistic fresh from the combined totals.
+
+Usage:
+ python3 aggregate_marginsafety_shards.py <shard_file> [<shard_file> ...]
+"""
+import sys
+from collections import defaultdict
+
+BIN_WIDTH = 10
+
+
+def load(path):
+ """Yield (tier, bucket, count, exceeded, sum_, sumsq, max_, bins)."""
+ with open(path) as f:
+ for line in f:
+ if line.startswith('#') or not line.strip():
+ continue
+ parts = line.split()
+ tier, bucket = parts[0], int(parts[1])
+ count, exceeded, sum_, sumsq, max_ = map(int, parts[2:7])
+ bins = list(map(int, parts[7:]))
+ yield tier, bucket, count, exceeded, sum_, sumsq, max_, bins
+
+
+def percentile(bins, total, p):
+ target = p * total
+ running = 0
+ for i, c in enumerate(bins):
+ running += c
+ if running >= target:
+ return i * BIN_WIDTH + BIN_WIDTH // 2
+ return (len(bins) - 1) * BIN_WIDTH
+
+
+def main():
+ if len(sys.argv) < 2:
+ print(__doc__)
+ sys.exit(1)
+
+ agg = defaultdict(lambda: {"count": 0, "exceeded": 0, "sum": 0,
+ "sumsq": 0, "max": 0})
+ agg_bins = {}
+
+ for path in sys.argv[1:]:
+ for tier, bucket, count, exceeded, sum_, sumsq, max_, bins in load(path):
+ key = (tier, bucket)
+ a = agg[key]
+ a["count"] += count
+ a["exceeded"] += exceeded
+ a["sum"] += sum_
+ a["sumsq"] += sumsq
+ a["max"] = max(a["max"], max_)
+ if key not in agg_bins:
+ agg_bins[key] = bins[:]
+ else:
+ for i, c in enumerate(bins):
+ agg_bins[key][i] += c
+
+ for tier in ("lazy", "superlazy"):
+ print(f"--- {tier} ---")
+ for bucket in range(8):
+ key = (tier, bucket)
+ if key not in agg:
+ continue
+ a = agg[key]
+ n = a["count"]
+ if n == 0:
+ print(f" bucket {bucket}: no exits taken")
+ continue
+ mean = a["sum"] / n
+ variance = max(0.0, a["sumsq"] / n - mean * mean)
+ stdev = variance ** 0.5
+ pct = 100.0 * a["exceeded"] / n
+ bins = agg_bins[key]
+ p10 = percentile(bins, n, 0.10)
+ p50 = percentile(bins, n, 0.50)
+ p90 = percentile(bins, n, 0.90)
+ p99 = percentile(bins, n, 0.99)
+ print(f" bucket {bucket}: n={n} exceeded={a['exceeded']} "
+ f"({pct:.3f}%) max={a['max']} avg={mean:.1f} "
+ f"stdev={stdev:.1f} p10={p10} p50={p50} p90={p90} p99={p99}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/src/eval_tune/lazy_eval_margins/hist_percentile.py b/src/eval_tune/lazy_eval_margins/hist_percentile.py
new file mode 100644
index 0000000..bd90444
--- /dev/null
+++ b/src/eval_tune/lazy_eval_margins/hist_percentile.py
@@ -0,0 +1,86 @@
+#!/usr/bin/env python3
+"""
+Read any percentile out of the raw lazy/super-lazy-eval histograms
+persisted by eval.c's _PersistRawHistogramsToFile (called automatically
+every time a DUMP_RESIDUAL_SAMPLES build runs `calibrate residual`).
+
+Usage:
+ python3 hist_percentile.py <histograms_file> [percentiles...]
+
+ python3 hist_percentile.py residual_raw_histograms_guess2_30k.txt
+ python3 hist_percentile.py residual_raw_histograms_guess2_30k.txt 0.93 0.97 0.985
+
+With no percentile arguments, prints a default useful spread for every
+bucket of all three histograms (residual, sl_residual, king). With
+percentile arguments (0.0-1.0), prints just those, letting you probe
+any exact percentile the fixed comment-block charts in eval.c didn't
+happen to include -- no re-running the 30k pool required, as long as
+this file (or a fresher one) still exists.
+
+File format (one line per histogram/bucket):
+ <name> <bucket 0-7> <min> <max> <bin0-count> <bin1-count> ... <binN-1-count>
+Bin i covers [RESID_HIST_MIN + i*RESID_HIST_BIN_WIDTH,
+ RESID_HIST_MIN + (i+1)*RESID_HIST_BIN_WIDTH).
+These constants are echoed in the file's own header comment and also
+hardcoded below (must match eval.c's #defines -- they haven't changed
+since this was built, but check if eval.c's RESID_HIST_* macros ever
+do).
+"""
+import sys
+
+BIN_WIDTH = 20
+MIN_VAL = -900
+DEFAULT_PERCENTILES = [0.01, 0.02, 0.05, 0.10, 0.15, 0.50,
+ 0.85, 0.90, 0.95, 0.98, 0.99]
+
+
+def load(path):
+ hists = {}
+ with open(path) as f:
+ for line in f:
+ if line.startswith('#') or not line.strip():
+ continue
+ parts = line.split()
+ name, bucket = parts[0], int(parts[1])
+ mn, mx = int(parts[2]), int(parts[3])
+ bins = list(map(int, parts[4:]))
+ hists[(name, bucket)] = (mn, mx, bins)
+ return hists
+
+
+def percentile(bins, total, p):
+ target = p * total
+ running = 0
+ for i, c in enumerate(bins):
+ running += c
+ if running >= target:
+ return MIN_VAL + i * BIN_WIDTH + BIN_WIDTH // 2
+ return MIN_VAL + len(bins) * BIN_WIDTH
+
+
+def main():
+ if len(sys.argv) < 2:
+ print(__doc__)
+ sys.exit(1)
+ path = sys.argv[1]
+ percentiles = [float(x) for x in sys.argv[2:]] or DEFAULT_PERCENTILES
+
+ hists = load(path)
+ names = sorted(set(name for name, _ in hists.keys()))
+ for name in names:
+ print(f"--- {name} ---")
+ for b in range(8):
+ if (name, b) not in hists:
+ continue
+ mn, mx, bins = hists[(name, b)]
+ total = sum(bins)
+ if total == 0:
+ print(f" bucket {b}: no samples")
+ continue
+ vals = " ".join(f"p{p*100:g}={percentile(bins, total, p)}"
+ for p in percentiles)
+ print(f" bucket {b}: n={total} min={mn} {vals} max={mx}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/src/eval_tune/lazy_eval_margins/residual_raw_histograms_guess2_30k.txt b/src/eval_tune/lazy_eval_margins/residual_raw_histograms_guess2_30k.txt
new file mode 100644
index 0000000..2b986fd
--- /dev/null
+++ b/src/eval_tune/lazy_eval_margins/residual_raw_histograms_guess2_30k.txt
@@ -0,0 +1,26 @@
+# RESID_HIST_BINS=90 RESID_HIST_BIN_WIDTH=20 RESID_HIST_MIN=-900
+# format: <histogram> <bucket> <min> <max> <bin0> <bin1> ... <binN-1>
+residual 0 -144 767 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 15 164 1363 26504 174479 870508 9266911 49119730 10195507 9392713 9036132 4277890 3487761 2369088 1537274 999863 733265 510947 325044 214567 159847 115810 75069 48198 53969 37634 17605 11800 13104 10928 6699 4315 1992 702 497 596 343 115 133 43 5 6 7 3 0 5 0 0 0 0 0 0
+residual 1 -156 817 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 13 44 510 2519 27653 88357 885128 6171719 17661445 21810645 17631977 17341084 11832862 7552148 6791430 4867074 3212050 2748617 1753842 1210171 1050324 619975 429316 296650 188513 141398 94497 77357 66118 48128 33631 28220 22011 16774 6674 4299 2332 2356 728 258 129 118 114 65 105 178 90 12 6 0 0 0 0
+residual 2 -183 767 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 39 298 1221 7993 62272 318787 1469893 6092481 18509685 37928483 55472424 40013831 25427921 16592110 10713766 7338910 5431623 3411498 2200485 1762575 1326601 917085 652040 541064 448084 316076 207264 141942 100658 72110 46937 32947 29716 21672 11431 8430 4380 2749 3811 4795 2881 1284 371 81 46 21 14 4 0 0 0 0 0 0
+residual 3 -210 708 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 11 102 778 7487 39369 185878 803334 2696551 7792351 17127020 31285590 43472163 35858955 18973011 11014510 7015793 4092753 2517064 1766213 1251219 892795 649846 465365 307463 220300 150647 108576 70034 54041 44466 24664 16116 8143 3820 2821 2969 2367 898 456 277 314 294 49 4 0 2 0 0 0 0 0 0 0 0 0
+residual 4 -225 735 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 26 355 2543 11604 72812 290938 1156430 4063826 9580273 17459580 27571067 37693079 41349983 32425962 19576659 10348483 5596569 3020560 1769703 1054860 663900 408572 251828 160650 111173 70030 48591 34664 28081 18251 10057 6322 4553 3129 2430 2085 1623 1246 648 227 136 107 37 7 11 3 1 3 0 0 0 0 0 0 0 0
+residual 5 -225 728 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 41 259 1047 6877 35991 215344 954842 2475083 5520028 11787046 23343313 36563338 41059148 31171892 17495367 8653512 4388935 2295381 1170946 613945 327609 182686 94836 58512 33370 22357 14125 7054 4429 2678 1824 1213 693 540 262 225 210 92 45 40 20 14 0 2 3 6 0 1 0 0 0 0 0 0 0 0
+residual 6 -224 760 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 25 193 1633 11244 59390 297180 1126668 3332277 7944759 14947141 23012354 27606104 25076655 16782590 8821595 4172757 2073401 1032648 497320 231717 113521 48847 25571 13957 9194 5013 3337 2048 763 658 376 369 399 182 156 158 100 54 30 56 35 30 11 5 0 2 0 0 0 1 0 0 0 0 0 0
+residual 7 -256 796 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 77 596 5373 33876 182971 1124882 5159235 14160661 23154137 28288490 27723692 24273561 18434696 11921382 6003346 2701510 1202658 564453 271480 120240 46627 25543 12741 6227 4692 3975 3596 2684 2263 2613 2155 1532 1027 1047 1954 2956 4217 2730 2126 967 332 208 116 73 24 31 16 8 4 8 1 2 0 0 0 0 0
+sl_residual 0 -363 2383 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 93 484 640 153 944 6091 20107 51368 43682 61452 73815 450532 1070740 1185017 1377715 1882895 6976318 18555405 16351680 6913999 5535807 6075056 5768506 5203691 4766197 3607623 2891038 2201502 1963000 1754691 1432133 1186087 971353 821438 589362 472728 426170 328019 262892 212348 175548 135083 114599 93897 72212 54737 41617 37979 29664 24002 15597 12435 12510 8346 6653 5274 4019 4598 3301 2277 2236 2950 746847
+sl_residual 1 -350 2456 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 5 12 209 154 212 1274 12139 17551 47595 38725 102721 275278 539485 924729 1517142 3059707 6915457 10963773 11809407 12944977 11134953 8917513 6063037 6607549 6690745 6033403 4709770 3464565 3120172 3067544 2518393 2152774 1875512 1409464 1171932 1008477 819500 684053 565223 450984 357881 297513 245308 193249 151350 115055 97969 79972 58946 51025 47163 37878 31393 29576 22418 16450 11622 7403 6610 4681 3334 1216751
+sl_residual 2 -341 2226 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 20 208 396 602 1065 5506 13385 39278 61658 95967 234519 537941 1151958 2189188 4202147 8406507 16076015 23816608 26868181 22293821 17876248 15042703 14417036 14403728 13616478 11421449 8195783 5804731 4559679 4050663 3497138 3025192 2548203 2088973 1766790 1506917 1243835 1005108 812946 678053 562124 499853 407287 313262 244722 202921 178275 143852 116515 86324 76908 71566 46813 32523 27196 25513 20425 15831 11815 9744 8976 991720
+sl_residual 3 -311 2183 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 21 61 154 542 1617 6306 18634 57796 137250 340158 813627 1900843 3798631 7183830 12657185 18955192 21399463 18622591 15039544 13069881 12002181 11690756 11082693 9079555 6607407 4844608 3632648 2812063 2286639 1952744 1628981 1325755 1088511 902859 716353 593121 507472 394467 286453 226564 184610 158212 129615 86332 75782 62737 49833 35072 29724 26078 19926 9245 7706 5500 4922 3131 3186 2083 1118 366910
+sl_residual 4 -332 1946 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 5 21 80 211 622 3216 7794 32108 85027 225564 575581 1288954 3162189 6503748 11524536 18055696 23379479 24854534 22254347 18468757 15612518 13595561 12248658 10461509 8232521 5973240 4332984 3091915 2436182 1930237 1434378 1046871 880613 710435 546475 437304 349561 246922 199709 151603 109361 79442 59223 45107 34391 31970 22142 15522 9573 9427 8249 7055 5312 2786 2100 2159 1781 1156 769 677 83810
+sl_residual 5 -255 1959 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 57 329 4118 5157 14547 53072 161291 460439 1366767 3711631 8379895 15114309 21009055 22617644 20719402 17623487 15520726 14438771 12605962 9666895 7026907 4848987 3389235 2392977 1724454 1361809 1038860 801316 631963 463280 347886 252687 203077 147415 112983 79834 51836 36878 26805 24492 15956 9257 7965 7546 4340 2921 2696 1937 1309 1570 558 619 572 317 501 410 9468
+sl_residual 6 -322 1921 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 10 71 75 533 2537 11871 45926 131486 411320 1022733 2455401 5155972 9050806 13534299 16064085 16289462 15246951 13506364 11485469 9336040 7270302 5223679 3423875 2232509 1548254 1085937 748196 537179 397751 283388 198894 143374 115014 84269 62144 41887 31983 20610 16322 8934 6864 4099 2740 1912 1078 897 984 861 495 349 232 220 168 174 1281 152 167 187 3754
+sl_residual 7 -290 2039 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 31 121 586 1390 5878 24253 78844 278500 840574 2421193 6465722 13171899 20355153 24186909 23376822 20790625 16475246 12347211 8814924 5940613 3741540 2250945 1341224 835775 536221 375668 245355 159741 114766 70869 52534 35541 27512 20952 14552 8525 6352 4366 3373 1674 1291 964 1139 1439 1256 1046 786 2036 1771 1237 1465 545 193 198 343 553 394 248 18929
+king 0 -68 181 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 69 13211 207523 934645 86648386 13071969 1298059 789094 81956 47100 5587 1482 69 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
+king 1 0 161 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 108064710 13984126 1444075 1033373 138118 49190 4019 2051 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
+king 2 -89 162 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1722 114112 2065803 15570929 85638651 125730930 7266574 838747 361815 45770 15105 279 350 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
+king 3 -176 137 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1661 19108 103023 459746 1201670 4330223 13600383 37139535 114485792 17300124 199194 64499 16526 4483 884 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
+king 4 -307 129 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 407 4859 24035 132051 369749 1106700 1918977 3315303 3740907 4954043 7287145 13988322 25447798 39536019 49741182 59441529 3861220 2098 608 489 235 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
+king 5 -354 45 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5827 43950 125055 504347 1037805 1783337 2511730 2350938 1689828 2132156 5231247 11219252 16784636 15952530 32037510 44817856 36448686 12878906 949585 0 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
+king 6 -401 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 5868 21230 53145 247683 406957 618270 890299 1239517 639988 477285 1587414 3082228 6139626 10016755 5146310 20237958 24080439 40947637 12534859 7979863 899195 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
+king 7 -394 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2363 7209 17894 78444 122463 206455 294717 391409 200453 202923 629244 1181690 2913486 5469013 3799599 16806699 19832913 50537919 22693260 38337669 1733993 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
diff --git a/src/eval_tune/match_play.py b/src/eval_tune/match_play.py
index e2e089a..d8a8e7b 100755
--- a/src/eval_tune/match_play.py
+++ b/src/eval_tune/match_play.py
@@ -31,8 +31,8 @@ Usage:
Prints:
CANDIDATE_SCORE=<0..1> GAMES=<n> WINS=<a> DRAWS=<b> LOSSES=<c> ELO=<+/-x.x>
LOWER95=<x> [SPRT=... LLR=... BOUNDS=... ELO0=... ELO1=...]
- CANDIDATE_STATS: nps=... ebf=... first_move_beta=...% nodes=... plies=...
- HEAD_STATS: nps=... ebf=... first_move_beta=...% nodes=... plies=...
+ CANDIDATE_STATS: nps=... ebf=... local_ebf=... first_move_beta=...% nodes=... plies=...
+ HEAD_STATS: nps=... ebf=... local_ebf=... first_move_beta=...% nodes=... plies=...
for the orchestrator to parse. Speed/tree-shape stats (EngineStats) are
pulled from the same per-move PostMoveSearchReport block every "go"
already prints, not a separate benchmark -- see EngineStats' docstring.
@@ -69,6 +69,21 @@ _SEARCHED_RE = re.compile(
_DEPTH_RE = re.compile(r"^tellothers d(\d+),")
_FIRST_MOVE_BETA_RE = re.compile(
r"^First move beta cutoff rate was\s+([\d.]+) percent\.")
+# 2026-09-18: per-iteration line UtilPrintPV (util.c) prints once per
+# completed depth during a move's search (both under-xboard "%2u %6d
+# %5u %-12llu %s" and human "%2u[+-]? %5s %-9s %-11llu %s" formats --
+# see util.c's UtilPrintPV; engines here run under-xboard since Engine
+# sends "xboard" at startup, per _STATS_TRAILING_LINES' comment above,
+# but this regex tolerates either since it only cares about the depth
+# and node-count fields, not the score/time/PV formatting in between).
+# u64TotalNodeCount is a running total for the WHOLE search (not reset
+# per iteration), so consecutive lines' node counts are monotonic
+# non-decreasing including across an aspiration-window fail-low/high
+# re-search at the SAME depth (root.c re-enters Search() with a wider
+# window at the same g_uIterateDepth, printing another line before
+# depth actually advances) -- see the EBF-from-consecutive-iterations
+# derivation in EngineStats.add_ply's caller for how that's handled.
+_ITER_RE = re.compile(r"^\s*(\d+)[+-]?\s+\S+\s+\S+\s+(\d+)\s")
# Reading PostMoveSearchReport's trailing block used to stop either at a
# bounded line count or at gamelist.c's "_white(move N)_"/"_black(move N)_"
# prompt -- both unreliable: the block's line count depends on whether the
@@ -283,6 +298,21 @@ def play_one_game(white_engine_path, black_engine_path, scratch_dir, fen,
# way around -- keep reading a bounded number of further
# lines to pick it up before moving on to the next ply.
ply_nodes = ply_time = ply_depth = ply_fmb = None
+ # 2026-09-18: local (per-iteration) branching factor, an
+ # alternative to the cumulative nodes**(1/depth) EBF below
+ # that isn't sensitive to which integer depth a move's
+ # search happens to finish on -- see _ITER_RE's comment and
+ # EngineStats' docstring for the full rationale. Track the
+ # most recently seen (depth, nodes) pair; each time depth
+ # actually advances (not just a same-depth aspiration
+ # re-search line), record nodes_now/nodes_prev as one
+ # branching-factor sample for this ply, geometric-mean them
+ # together at the end (arithmetic mean of ratios would let
+ # one huge outlier iteration dominate; geometric mean is
+ # the correct way to average ratios/growth-factors).
+ last_iter_depth = last_iter_nodes = None
+ iter_log_ratio_sum = 0.0
+ iter_ratio_count = 0
for _ in range(_STATS_TRAILING_LINES):
line = _readline_or_none(mover.stdout, _STATS_IDLE_TIMEOUT_SEC)
if line is None:
@@ -309,8 +339,27 @@ def play_one_game(white_engine_path, black_engine_path, scratch_dir, fen,
if fm:
ply_fmb = float(fm.group(1))
continue
+ im = _ITER_RE.match(line)
+ if im:
+ iter_depth = int(im.group(1))
+ iter_nodes = int(im.group(2))
+ if (last_iter_depth is not None and
+ last_iter_nodes is not None and
+ iter_depth != last_iter_depth and
+ iter_nodes > last_iter_nodes and
+ iter_depth > last_iter_depth):
+ ratio = iter_nodes / last_iter_nodes
+ depth_delta = iter_depth - last_iter_depth
+ iter_log_ratio_sum += math.log(ratio) / depth_delta
+ iter_ratio_count += 1
+ last_iter_depth = iter_depth
+ last_iter_nodes = iter_nodes
+ continue
if ply_nodes is not None and ply_time is not None:
- mover_stats.add_ply(ply_nodes, ply_time, ply_depth)
+ ply_local_ebf = (math.exp(iter_log_ratio_sum / iter_ratio_count)
+ if iter_ratio_count else None)
+ mover_stats.add_ply(ply_nodes, ply_time, ply_depth,
+ ply_local_ebf)
if ply_fmb is not None:
if white_to_move:
white_last_fmb = ply_fmb
@@ -376,7 +425,33 @@ class EngineStats:
(nodes**(1/depth) per move, see head_reference notes on why this can
diverge from first-move-beta-cutoff-rate when pruning/aspiration-
window re-searches inflate node counts independent of true tree
- shape), then averaged across all moves an engine played. First-move
+ shape), then averaged across all moves an engine played.
+
+ 2026-09-18: that cumulative nodes**(1/depth) EBF turned out to be
+ misleading when comparing two engines with different lazy-eval-exit
+ aggressiveness -- one engine measured LOWER nps and HIGHER avg_depth
+ than the other while ALSO reporting HIGHER ebf, which is not
+ reconcilable if ebf is really "nodes needed per unit depth" (lower
+ nps + higher depth implies fewer total nodes were needed to get
+ there, which should mean lower ebf, not higher). Root cause: this
+ formula uses the SEARCH-WIDE cumulative node count raised to
+ 1/(the single integer depth reported at the end), so it's extremely
+ sensitive to exactly which whole ply a move's search happens to
+ finish on within its time budget -- a difference in time-management
+ timing that pushes a move to complete one more (or one fewer) full
+ iteration can swing this number a lot with no real change in tree
+ efficiency, and this effect doesn't reliably average out because
+ node-count magnitude varies hugely across game phases/positions.
+ `local_ebf` (see add_ply) fixes this by computing depth-normalized
+ branching ratios from CONSECUTIVE completed-iteration node counts
+ within a single move's search (nodes_now/nodes_prev, depth-delta-
+ normalized, geometric-meaned across a move's iterations, then
+ arithmetic-meaned across moves same as avg_ebf) -- a true local
+ branching-factor measurement, insensitive to which absolute depth
+ the search happened to stop at. Kept the old ebf/avg_ebf fields
+ alongside rather than replacing them, since the cumulative version
+ is still what script.c's own suite runs report and there's value in
+ being able to cross-check the two against each other. First-move
beta cutoff rate is read as a running total inside a single engine
process's lifetime (root.c never resets those counters mid-process),
and since play_one_game spawns a fresh process per game, the value
@@ -391,12 +466,14 @@ class EngineStats:
self.plies = 0
self.ebf_sum = 0.0
self.ebf_count = 0
+ self.local_ebf_sum = 0.0
+ self.local_ebf_count = 0
self._fmb_weighted_sum = 0.0
self._fmb_weight = 0
self.depth_sum = 0
self.depth_count = 0
- def add_ply(self, nodes, elapsed, depth):
+ def add_ply(self, nodes, elapsed, depth, local_ebf=None):
self.total_nodes += nodes
self.total_time += elapsed
self.plies += 1
@@ -406,6 +483,9 @@ class EngineStats:
if depth and depth > 0 and nodes > 0:
self.ebf_sum += nodes ** (1.0 / depth)
self.ebf_count += 1
+ if local_ebf is not None:
+ self.local_ebf_sum += local_ebf
+ self.local_ebf_count += 1
def add_game_first_move_beta(self, pct, plies_in_game):
if pct is not None and plies_in_game > 0:
@@ -418,6 +498,8 @@ class EngineStats:
self.plies += other.plies
self.ebf_sum += other.ebf_sum
self.ebf_count += other.ebf_count
+ self.local_ebf_sum += other.local_ebf_sum
+ self.local_ebf_count += other.local_ebf_count
self._fmb_weighted_sum += other._fmb_weighted_sum
self._fmb_weight += other._fmb_weight
self.depth_sum += other.depth_sum
@@ -432,6 +514,11 @@ class EngineStats:
return self.ebf_sum / self.ebf_count if self.ebf_count else 0.0
@property
+ def avg_local_ebf(self):
+ return (self.local_ebf_sum / self.local_ebf_count
+ if self.local_ebf_count else 0.0)
+
+ @property
def avg_first_move_beta(self):
return (self._fmb_weighted_sum / self._fmb_weight
if self._fmb_weight else 0.0)
@@ -442,6 +529,7 @@ class EngineStats:
def summary(self):
return (f"nps={self.nps:.0f} ebf={self.avg_ebf:.3f} "
+ f"local_ebf={self.avg_local_ebf:.3f} "
f"depth={self.avg_depth:.2f} "
f"first_move_beta={self.avg_first_move_beta:.1f}% "
f"nodes={self.total_nodes} plies={self.plies}")
@@ -718,7 +806,9 @@ def main():
f"head={head_stats.nps:.0f})",
file=sys.stderr)
print(f" avg ebf: candidate={candidate_stats.avg_ebf:.3f} "
- f"head={head_stats.avg_ebf:.3f}",
+ f"head={head_stats.avg_ebf:.3f} "
+ f"(local: candidate={candidate_stats.avg_local_ebf:.3f} "
+ f"head={head_stats.avg_local_ebf:.3f})",
file=sys.stderr)
if sprt is not None and sprt_decision is None:
diff --git a/src/eval_tune/sample_eval_calibration_positions.py b/src/eval_tune/sample_eval_calibration_positions.py
new file mode 100644
index 0000000..6bf42d2
--- /dev/null
+++ b/src/eval_tune/sample_eval_calibration_positions.py
@@ -0,0 +1,182 @@
+#!/usr/bin/env python3
+"""Sample random (fen) positions from one or more PGN pools, at random
+plies anywhere in the game (not filtered for quiet/tactical), for the
+margin-safety recalibration described in board_representation/EVAL.md
+section 9. Unlike tune_eval_dna.py's quiet-position pools, this
+deliberately does NOT filter on SEE/hanging-material/qsearch-agreement --
+Eval() runs at every node of the real search tree, quiet or not, so the
+calibration sample should look like "all chess positions", not just the
+quiet subset used for Texel tuning.
+
+Uses the same random-byte-offset-seek trick as match_play.py's
+sample_openings() to avoid a full linear scan of multi-GB pools.
+
+Output: one FEN per source game (one random ply from openings-excluded
+range), written as alternating "setboard <fen>" / "go" lines matching
+tests/twic_sample.ep_'s format, suitable for a `script <file>` run
+against a CALIBRATE_MARGIN_SAFETY build.
+"""
+import argparse
+import hashlib
+import io
+import random
+import sys
+from pathlib import Path
+
+import chess
+import chess.pgn
+
+
+def iter_random_game_texts(pgn_path, rng, max_attempts):
+ """Yield raw PGN text blocks for randomly-located games in pgn_path,
+ same seek-scan-extract approach as match_play.py's sample_openings."""
+ st = Path(pgn_path).stat()
+ file_size = st.st_size
+ attempts = 0
+ with open(pgn_path, "rb") as fb:
+ while attempts < max_attempts:
+ attempts += 1
+ offset = rng.randint(0, max(file_size - 1, 0))
+ fb.seek(offset)
+ fb.readline()
+ start = fb.tell()
+ line = fb.readline()
+ while line and not line.startswith(b"[Event "):
+ start = fb.tell()
+ line = fb.readline()
+ if not line:
+ continue
+ fb.seek(start)
+ fb.readline()
+ end = fb.tell()
+ line = fb.readline()
+ while line and not line.startswith(b"[Event "):
+ end = fb.tell()
+ line = fb.readline()
+ fb.seek(start)
+ raw = fb.read(end - start).decode("utf-8", errors="replace")
+ yield raw
+
+
+PIECE_VALUES = {chess.KNIGHT: 3, chess.BISHOP: 3, chess.ROOK: 5, chess.QUEEN: 9}
+
+
+def material_bucket(board):
+ """Same bucketing as eval.c's _MaterialBucket: combined non-pawn
+ army value (pawn units) / 8, capped at 7 (0 = bare kings, 7 = full
+ army)."""
+ total = 0
+ for pt, v in PIECE_VALUES.items():
+ total += v * len(board.pieces(pt, chess.WHITE))
+ total += v * len(board.pieces(pt, chess.BLACK))
+ return min(7, total // 8)
+
+
+def sample_positions(pgn_paths, n, min_ply, seed, cache_dir, stratify=False):
+ key_parts = []
+ for p in pgn_paths:
+ st = Path(p).stat()
+ key_parts.append(f"{p}|{st.st_mtime}|{st.st_size}")
+ key = hashlib.sha256(
+ f"{'|'.join(key_parts)}|{n}|{min_ply}|{seed}|{stratify}".encode()
+ ).hexdigest()[:16]
+ cache_path = Path(cache_dir) / f"eval_calib_{key}.fens"
+ if cache_path.exists():
+ fens = cache_path.read_text().splitlines()
+ print(f" loaded {len(fens)} positions from cache {cache_path}",
+ file=sys.stderr)
+ return fens
+
+ rng = random.Random(seed)
+ fens = []
+ # Stratified mode needs many more discarded candidates (most draws
+ # land in the naturally-common middlegame buckets, not the rare
+ # near-endgame ones we're specifically trying to fill), so give it
+ # a much larger attempt budget than the unstratified path.
+ max_attempts = (n * 200 + 2000) if stratify else (n * 30 + 500)
+ attempts = 0
+ # Round-robin across pools so a large pool (twic_filtered) doesn't
+ # drown out a smaller one (any single lichess_elite month) purely by
+ # byte-offset odds; weight by requested count via cycling.
+ pool_iters = [iter_random_game_texts(p, rng, max_attempts) for p in pgn_paths]
+ idx = 0
+
+ # Unstratified: just take the first n candidates. Stratified: fill
+ # 8 equal-size buckets (by combined material) up to quota each,
+ # discarding candidates for buckets that are already full -- this
+ # is what actually fixes cross-sectional coverage (uniform-random-
+ # ply sampling is heavily skewed toward the middlegame/opening
+ # buckets simply because games spend more plies there than grinding
+ # out endgames; more positions alone doesn't fix that skew, only
+ # explicit per-bucket quotas do).
+ num_buckets = 8
+ quota = -(-n // num_buckets) if stratify else 0 # ceil division
+ bucket_counts = [0] * num_buckets
+
+ while len(fens) < n and attempts < max_attempts:
+ raw = next(pool_iters[idx % len(pool_iters)], None)
+ idx += 1
+ attempts += 1
+ if raw is None:
+ continue
+ game = chess.pgn.read_game(io.StringIO(raw))
+ if game is None:
+ continue
+ moves = list(game.mainline_moves())
+ if len(moves) < min_ply + 1:
+ continue
+ # Pick one random ply strictly after min_ply and strictly before
+ # the last move (avoid checkmate/stalemate terminal boards, which
+ # `go` can't search meaningfully).
+ target_ply = rng.randint(min_ply, len(moves) - 1)
+ board = game.board()
+ for i, mv in enumerate(moves):
+ if i == target_ply:
+ break
+ board.push(mv)
+ if board.is_game_over():
+ continue
+ if stratify:
+ b = material_bucket(board)
+ if bucket_counts[b] >= quota:
+ continue
+ bucket_counts[b] += 1
+ fens.append(board.fen())
+ if len(fens) % 2000 == 0:
+ print(f" sampled {len(fens)}/{n}...", file=sys.stderr)
+
+ if stratify:
+ print(f" final bucket counts: {bucket_counts}", file=sys.stderr)
+
+ Path(cache_dir).mkdir(parents=True, exist_ok=True)
+ cache_path.write_text("\n".join(fens) + "\n")
+ print(f" wrote {len(fens)} positions to cache {cache_path}", file=sys.stderr)
+ return fens
+
+
+def main():
+ ap = argparse.ArgumentParser()
+ ap.add_argument("pgn_paths", nargs="+", help="one or more PGN pool files")
+ ap.add_argument("--n", type=int, default=20000)
+ ap.add_argument("--min-ply", type=int, default=4,
+ help="skip book-ish very early plies")
+ ap.add_argument("--seed", type=int, default=0)
+ ap.add_argument("--cache-dir", default="eval_tune/position_cache")
+ ap.add_argument("--out", required=True, help="output .ep_ file")
+ ap.add_argument("--stratify", action="store_true",
+ help="fill 8 equal-size combined-material buckets "
+ "up to quota each, instead of taking positions "
+ "in natural (middlegame-skewed) draw order")
+ args = ap.parse_args()
+
+ fens = sample_positions(args.pgn_paths, args.n, args.min_ply, args.seed,
+ args.cache_dir, stratify=args.stratify)
+ with open(args.out, "w") as f:
+ for fen in fens:
+ f.write(f"setboard {fen}\n")
+ f.write("go\n")
+ print(f"Wrote {len(fens)} positions ({2 * len(fens)} lines) to {args.out}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/src/eval_tune/test_vs_head.sh b/src/eval_tune/test_vs_head.sh
index 29c050f..e937389 100755
--- a/src/eval_tune/test_vs_head.sh
+++ b/src/eval_tune/test_vs_head.sh
@@ -36,6 +36,10 @@ LABEL="${3:-match}"
if [ -n "$2" ]; then
CANDIDATE="$2"
else
+ cd ..
+ gmake clean
+ gmake release MP=1 PERF_COUNTERS=1
+ cd eval_tune
CANDIDATE=/tmp/typhoon/typhoon_match_play_candidate
cp -f ../typhoon ${CANDIDATE}
fi
diff --git a/src/gamelist.c b/src/gamelist.c
index 6adf151..3af04a8 100644
--- a/src/gamelist.c
+++ b/src/gamelist.c
@@ -289,15 +289,34 @@ Return value:
**/
{
-#ifdef DEBUG
- static POSITION p;
-
- memcpy(&p, &g_RootPosition, sizeof(POSITION));
-
- return(&p);
-#else
+ // 2026-09-16: used to hand DEBUG builds a pointer into a single
+ // shared `static POSITION p`, freshly overwritten from
+ // g_RootPosition on every call -- presumably meant as a defensive
+ // "give callers a copy, not the real thing," but since it's the
+ // SAME static buffer every call, any caller that stashes the
+ // returned pointer and expects it to stay stable across a later
+ // SetRootPosition() call (e.g. ScriptCommand/_ProcessSuiteLine,
+ // which captures GetRootPosition() once at the top of a script run
+ // and reuses that pointer across every setboard/go line in the
+ // file) silently gets a pointer that can be overwritten out from
+ // under it by any OTHER GetRootPosition() call anywhere in the
+ // program in the meantime -- confirmed as the root cause of a real,
+ // deterministic bug: a scripted "setboard <FEN>; go" pair would
+ // update the real g_RootPosition correctly but Think() would still
+ // see the position from BEFORE the setboard, because its captured
+ // `pos` pointer pointed at this shared buffer, snapshotted before
+ // the setboard ran and never refreshed. Confirmed via split.c: a
+ // split created deep in the (wrongly-rooted) search captured its
+ // OWN root snapshot via a LATER, independent GetRootPosition() call
+ // that correctly saw the real (already-updated) position, producing
+ // a split whose recorded move path and recorded root position
+ // belonged to two different positions entirely. DEBUG-only bug --
+ // release builds always used this same direct-pointer path and were
+ // never affected. No actual defensive value survives removing the
+ // indirection: matching the release build's behavior exactly (a
+ // stable pointer to the one real g_RootPosition, always current) is
+ // strictly more correct, not less safe.
return(&g_RootPosition);
-#endif
}
void
diff --git a/src/hash.c b/src/hash.c
index c5e5f85..803376d 100644
--- a/src/hash.c
+++ b/src/hash.c
@@ -834,7 +834,16 @@ Return value:
ASSERT(IS_VALID_SCORE(iBeta));
ASSERT(iAlpha < iBeta);
ASSERT(pHashMove->uMove == 0);
- ASSERT(NULL != g_pHashTable);
+ // 2026-09-17: NULL here is not a bug -- it's the legitimate "--hash
+ // none" state (InitializeHashSystem deliberately skips allocation
+ // when uNumHashTableEntries == 0, since IS_A_POWER_OF_2(0) is true
+ // by that macro's bitwise definition). This ASSERT used to fire
+ // unconditionally, making "--hash none" crash on the very first
+ // probe despite being a supported init-time configuration.
+ if (NULL == g_pHashTable)
+ {
+ return(NULL);
+ }
INC(ctx->sCounters.hash.u64Probes);
//
diff --git a/src/main.c b/src/main.c
index f61d840..5423805 100644
--- a/src/main.c
+++ b/src/main.c
@@ -454,7 +454,17 @@ Return value:
**/
{
{
- unsigned int uSeed = (unsigned int)time(0);
+ // 2026-09-16 diagnostic: allow forcing a fixed srand seed via
+ // TYPHOON_SEED for reproducing a specific run bit-for-bit
+ // (there's no --seed command-line flag) -- e.g. re-running the
+ // same debug_smoke_test.sh repro file to check whether a
+ // failure is deterministic or a genuine SMP timing race.
+ // TEMPORARY: revert once the split.c investigation is done,
+ // this isn't meant to be a permanent CLI feature.
+ const char *pszForcedSeed = getenv("TYPHOON_SEED");
+ unsigned int uSeed = pszForcedSeed ?
+ (unsigned int)strtoul(pszForcedSeed, NULL, 10) :
+ (unsigned int)time(0);
Trace("srand seed: %u\n", uSeed);
srand(uSeed);
}
diff --git a/src/probe.c b/src/probe.c
index 5054142..2bddf99 100644
--- a/src/probe.c
+++ b/src/probe.c
@@ -333,17 +333,29 @@ Return value:
static PIECE
-_TbPromotesToPiece(IN unsigned uPromotes)
+_TbPromotesToPiece(IN unsigned uPromotes, IN ULONG uSide)
/**
Routine description:
Map a TB_PROMOTES_* value from a Fathom root probe result to our
- own PIECE constant.
+ own PIECE constant. Must return a *colored* piece (BLACK_QUEEN|
+ uSide, matching san.c's "pPromoted = BLACK_QUEEN | pos->uToMove"
+ convention -- WHITE=1/BLACK=0 ORed onto the black-piece base value
+ selects the right color bit) since that's what move generation
+ actually stamps into mv.pPromoted. The previous version returned
+ the bare, colorless QUEEN/ROOK/BISHOP/KNIGHT constants (5/3/2/4 --
+ type-only values, never equal to any real generated move's
+ pPromoted), so ProbeEGTBRoot's move-list lookup below could never
+ match a promotion move for either side and always fell through to
+ ASSERT(FALSE) -- 100% reproducible whenever Fathom's suggested
+ root DTZ move happened to be a promotion, rare enough to need a
+ 200k-position stress run to surface.
Parameters:
unsigned uPromotes
+ ULONG uSide : WHITE or BLACK -- the side making the promoting move
Return value:
@@ -353,10 +365,10 @@ Return value:
{
switch (uPromotes)
{
- case TB_PROMOTES_QUEEN: return(QUEEN);
- case TB_PROMOTES_ROOK: return(ROOK);
- case TB_PROMOTES_BISHOP: return(BISHOP);
- case TB_PROMOTES_KNIGHT: return(KNIGHT);
+ case TB_PROMOTES_QUEEN: return(BLACK_QUEEN | uSide);
+ case TB_PROMOTES_ROOK: return(BLACK_ROOK | uSide);
+ case TB_PROMOTES_BISHOP: return(BLACK_BISHOP | uSide);
+ case TB_PROMOTES_KNIGHT: return(BLACK_KNIGHT | uSide);
default: return(0);
}
}
@@ -464,7 +476,7 @@ Return value:
cFrom = FROM64(TB_GET_FROM(uResult));
cTo = FROM64(TB_GET_TO(uResult));
- pPromotes = _TbPromotesToPiece(TB_GET_PROMOTES(uResult));
+ pPromotes = _TbPromotesToPiece(TB_GET_PROMOTES(uResult), pos->uToMove);
for (u = ctx->sMoveStack.uBegin[ctx->uPly];
u < ctx->sMoveStack.uEnd[ctx->uPly];
diff --git a/src/recogn.c b/src/recogn.c
index 07ee0d5..c573f8f 100644
--- a/src/recogn.c
+++ b/src/recogn.c
@@ -780,72 +780,37 @@ Return value:
{
//
// Neither side has a lone king. This is either KBKP+ or
- // KBP+KP+.
+ // KBP+KP+ -- i.e. the WEAK side has at least one pawn of its
+ // own, not just a bare king.
//
- if (FastCountBits(pos->bbPawns[uStrong]) > 0)
- {
- //
- // Strong side can maybe take an adjacent pawn and survive the
- // bad bishop.
- //
- if (uStrong == pos->uToMove)
- {
- return(UNRECOGNIZED);
- }
-
- //
- // Make sure the strong side has the right color bishop
- // for his pawns.
- //
- if (uStrong == WHITE)
- {
- if (!(bb & ~BBFILE[A]) &&
- (WhiteSqBishopCount(pos, WHITE) == 0) &&
- (DISTANCE(cWeakKing, A8) <= 1))
- {
- goto at_best_draw_for_strong;
- }
-
- if (!(bb & ~BBFILE[H]) &&
- (WhiteSqBishopCount(pos, WHITE) ==
- FastCountBits(pos->bbPieces[WHITE][BISHOP])) &&
- (DISTANCE(cWeakKing, H8) <= 1))
- {
- goto at_best_draw_for_strong;
- }
- }
- else
- {
- if (!(bb & ~BBFILE[A]) &&
- (WhiteSqBishopCount(pos, BLACK) ==
- FastCountBits(pos->bbPieces[BLACK][BISHOP])) &&
- (DISTANCE(cWeakKing, A1) <= 1))
- {
- goto at_best_draw_for_strong;
- }
-
- if (!(bb & ~BBFILE[H]) &&
- (WhiteSqBishopCount(pos, BLACK) == 0) &&
- (DISTANCE(cWeakKing, H1) <= 1))
- {
- goto at_best_draw_for_strong;
- }
- }
- return(UNRECOGNIZED);
- }
- else
- {
- //
- // Strong side has no pawns... there's a mate-in-1 here if
- // the weak king is stuck in the corner.
- //
- if ((FastCountBits(pos->bbPieces[uStrong][BISHOP]) > 1) ||
- (CORNER_DISTANCE(cWeakKing) == 0))
- {
- return(UNRECOGNIZED);
- }
- goto at_best_draw_for_strong;
- }
+ // 2026-09-18: this used to still apply the "wrong-colored-
+ // bishop rook-pawn corner fortress" shortcut here (same
+ // bishop-color/pawn-file/king-distance checks as the bare-
+ // weak-king KBPK branch below, goto at_best_draw_for_strong)
+ // -- but that shortcut's soundness depends on the weak side
+ // having nothing but a king to shelter with; it was never
+ // re-derived for a weak side that has its OWN pawn(s), which
+ // can hand the strong side real zugzwang/breakthrough
+ // resources the fortress technique doesn't account for.
+ // Found via fuzzing tests/cross_sectional_200k.ep_ (DEBUG
+ // build, RECOGNIZER_SANITY_FAIL): 6k1/6p1/7p/7P/5K2/8/4B3/8 b
+ // - - 0 0 (White Kf4/Be2/Ph5, Black Kg8/Pg7/Ph6) hit this
+ // exact shortcut (light-squared bishop, h-file pawn, weak
+ // king adjacent to h8) and returned a hard RECOGN_LOWER draw
+ // bound, but EGTB says it's a near-forced loss for Black --
+ // the extra g7 pawn (irrelevant to the fortress logic as
+ // written) apparently gives White exactly this kind of
+ // resource. Rather than guess a formally-correct boundary
+ // (e.g. "at most one weak-side pawn is still safe") without
+ // EGTB-backed proof, conservatively defer this whole case to
+ // UNRECOGNIZED -- same choice this session's earlier KPK fix
+ // made for its own unsound shortcut. Slower (falls through to
+ // real search/EGTB) but never wrong. The bare-weak-king KBPK
+ // branch below is untouched -- its own shortcuts already
+ // require TotalNonPawnCount(weak side) == 1, a genuinely bare
+ // king, which this counterexample doesn't undermine.
+ //
+ return(UNRECOGNIZED);
}
else
{
@@ -1037,11 +1002,32 @@ Return value:
if (TRUE == _EvalPasserRacesAgainstLoneKings(pos, pHash))
{
//
- // Someone wins.
+ // Someone wins -- but only trust this as a hard bound when
+ // the "losing" side has no pawns of its own left. 2026-09-17:
+ // fuzzing found _EvalPasserRacesAgainstLoneKings only checks
+ // whether a pawn already flagged "passed" outruns the enemy
+ // king -- it has no way to see a *connected*-pawn threat
+ // where neither pawn is individually passed (each still
+ // contested on an adjacent file) but together they still
+ // produce a passer and a real race once one of them
+ // advances. A real case: 8/2K5/6pk/2P2p2/8/6P1/8/8 b (white
+ // Pc5 "wins the race" against the far-away black king per
+ // this function) is actually a draw per Syzygy -- black's
+ // f5/g6 duo (blocked from passed-pawn status by white's own
+ // g3 pawn) produces enough real counterplay to hold. Crafty's
+ // EvaluatePassedPawnRaces (evaluate.c) has this identical
+ // blind spot -- same "only scan already-passed pawns" gate
+ // -- but it's safe there only because it's a soft eval
+ // nudge feeding real alpha-beta search, never a hard,
+ // search-terminating bound the way RECOGN_LOWER/UPPER is
+ // here. Restricting to "loser has zero pawns" is the one
+ // case this heuristic can't be undermined by a duo like
+ // this, since there's no pawn left to produce one.
//
ASSERT((pos->iScore[BLACK] == 0) || (pos->iScore[WHITE] == 0));
ASSERT((pos->iScore[BLACK] != 0) || (pos->iScore[WHITE] != 0));
- if (pos->iScore[BLACK] > 0)
+ if ((pos->iScore[BLACK] > 0) &&
+ (0 == FastCountBits(pos->bbPawns[WHITE])))
{
ASSERT(pos->iScore[WHITE] == 0);
*piScore = (pos->iMaterialBalance[BLACK] + 2 * VALUE_PAWN +
@@ -1054,9 +1040,9 @@ Return value:
}
return(RECOGN_LOWER);
}
- else
+ else if ((pos->iScore[WHITE] > 0) &&
+ (0 == FastCountBits(pos->bbPawns[BLACK])))
{
- ASSERT(pos->iScore[WHITE] > 0);
*piScore = (pos->iMaterialBalance[WHITE] + 2 * VALUE_PAWN +
pos->iScore[WHITE]);
ASSERT(IS_VALID_SCORE(*piScore));
@@ -1067,6 +1053,11 @@ Return value:
}
return(RECOGN_LOWER);
}
+ // Else: the "losing" side still has pawns of its own --
+ // don't trust the race verdict as a hard bound. Fall
+ // through to the UNRECOGNIZED/sophisticated-analysis logic
+ // below, same as if _EvalPasserRacesAgainstLoneKings had
+ // found no winner at all.
}
}
@@ -1391,6 +1382,25 @@ Return value:
{
*piScore = iScore;
ASSERT((-NMATE < iScore) && (iScore < +NMATE));
+#ifdef DEBUG
+ if (!_SanityCheckRecognizers(ctx, iScore, uVal))
+ {
+ CHAR *pszDiagFen = PositionToFen(pos);
+ SCORE iRawEgtb = -12345;
+ FLAG fEgtbOk = ProbeEGTB(ctx, &iRawEgtb);
+ Trace("RECOGNIZER_SANITY_FAIL uSigW=%u uSigB=%u uVal=%u "
+ "iScore=%d egtbOk=%u egtbRawScore=%d (>0=win for "
+ "side to move, 0=draw, <0=loss for side to move) "
+ "fen=%s\n",
+ uSig[WHITE], uSig[BLACK], uVal, (int)iScore,
+ fEgtbOk, (int)iRawEgtb,
+ (NULL != pszDiagFen) ? pszDiagFen : "?");
+ if (NULL != pszDiagFen)
+ {
+ SystemFreeMemory(pszDiagFen);
+ }
+ }
+#endif
ASSERT(_SanityCheckRecognizers(ctx, iScore, uVal));
return(uVal);
}
diff --git a/src/root.c b/src/root.c
index 314395a..74d2327 100644
--- a/src/root.c
+++ b/src/root.c
@@ -1768,6 +1768,62 @@ Return value:
MOVE mvBook;
CHAR *p;
+ //
+ // 2026-09-16: barrier against a real, confirmed race (split.c's
+ // per-split rendezvous only guarantees that *that split's own*
+ // recruited helpers have finished before its slot is recycled --
+ // nothing previously stitched that into a *global* guarantee that
+ // every helper thread anywhere is idle before a brand-new search
+ // begins reusing g_SplitInfo[] slots). Concretely reproduced: a
+ // helper thread recruited by one Think() call's search (e.g. an
+ // engine-startup analysis on the default position) can still be
+ // mid-flight when a *later*, unrelated Think() call starts,
+ // recycles the same slot index, and overwrites its mvPathToHere/
+ // sRootPosition -- the straggler helper then replays a stale move
+ // from the earlier search against the new search's root position
+ // (observed directly: split.c's helper-side SanityCheckMove failed
+ // replaying "Ng1-f3" -- an opening-book-shaped move from the
+ // default starting position -- against an unrelated mid-game FEN).
+ // g_uNumHelpersAvailable/g_uNumHelperThreads are always valid (0/0
+ // in non-MP or single-cpu builds), so this is unconditional and a
+ // no-op there. Deliberately unbounded, not a bounded best-effort
+ // retry like QuitCommand's shutdown wait (command.c) -- silently
+ // proceeding without this invariant is exactly the bug being
+ // fixed, so giving up early here would just reintroduce it; a
+ // warning fires if it's taking unexpectedly long so a genuine
+ // deadlock elsewhere is still observable rather than a silent hang.
+ // g_uNumHelpersAvailable/g_uNumHelperThreads only exist when split.c
+ // is actually compiled in (its whole body is #ifdef MP) -- guard
+ // this the same way, or a non-MP profile (e.g. TEST=1) fails to
+ // link against symbols that were never defined.
+#ifdef MP
+ if (g_uNumHelpersAvailable != g_uNumHelperThreads)
+ {
+ double dWaitStart = SystemTimeStamp();
+ FLAG fWarned = FALSE;
+
+ // Tight busy-spin, no sleep -- this should normally resolve in
+ // microseconds (same idiom as split.c's own per-split
+ // "wait for uNumThreadsHelping" loop), so sleeping even 1ms per
+ // iteration here would add real, gratuitous latency to every
+ // single move on the ordinary (non-racing) path.
+ while (g_uNumHelpersAvailable != g_uNumHelperThreads)
+ {
+ if ((FALSE == fWarned) &&
+ ((SystemTimeStamp() - dWaitStart) > 1.0))
+ {
+ Trace("Think(): still waiting for %lu/%lu helper "
+ "thread(s) to go idle before starting a new "
+ "search (%.1fs so far) -- possible stuck helper?\n",
+ (unsigned long)(g_uNumHelperThreads - g_uNumHelpersAvailable),
+ (unsigned long)g_uNumHelperThreads,
+ SystemTimeStamp() - dWaitStart);
+ fWarned = TRUE;
+ }
+ }
+ }
+#endif
+
InitializeSearcherContext(pos, &ctx);
g_MoveTimer.bvFlags = 0;
g_Options.fPondering = FALSE;
diff --git a/src/script.c b/src/script.c
index e8d6f9a..57145a6 100644
--- a/src/script.c
+++ b/src/script.c
@@ -394,11 +394,7 @@ _ProcessSuiteLine(char *szLine, POSITION *pos)
}
else
{
- PushNewInput(szLine);
- if (FALSE == g_fExitProgram)
- {
- ParseUserInput(FALSE);
- }
+ ExecuteCommandLineDirectly(szLine, pos);
}
}
diff --git a/src/search.c b/src/search.c
index 91e3b2b..cb82ea4 100644
--- a/src/search.c
+++ b/src/search.c
@@ -354,7 +354,7 @@ Search(IN SEARCHER_THREAD_CONTEXT *ctx,
}
// Maybe do nullmove pruning
- pi->iEval = iEval = Eval(ctx, iAlpha, iBeta, NULL);
+ pi->iEval = iEval = Eval(ctx, iAlpha, iBeta, &(pi->iPositional));
SCORE iImprovement = 0;
if (ctx->uPly > 1)
{
@@ -544,6 +544,25 @@ Search(IN SEARCHER_THREAD_CONTEXT *ctx,
if (iEval < iAlpha)
{
+ // 2026-09-17: tried folding + pi->iPositional[
+ // pos->uToMove] into both conditions below (same
+ // role as QSearch's FUTILITY_BASE_MARGIN_BY_
+ // SOURCE[tier] + iPositional[side]) -- rolled
+ // back, untested in isolation, after a bundled
+ // self-play run (also carrying the margin_a/
+ // margin_b percentile switch, the QSearch
+ // futility margin recalibration, the _EvalKing
+ // fix, and the drawish-in-both-lazy-tiers change)
+ // came back flat/slightly negative with no way
+ // to attribute the result to any one change. See
+ // TODO (EFP_BASE_MARGIN_BY_SOURCE entry) -- the
+ // idea itself isn't rejected, just never
+ // validated on its own. PLY_INFO.iPositional
+ // (chess.h) and the Eval() call populating it
+ // (just above, in this function) are left in
+ // place -- useful per-ply infrastructure
+ // regardless of whether this exact gate ends up
+ // using it.
if ((uDepth <= ONE_PLY + THREE_QUARTERS_PLY) &&
(iEval + VALUE_KNIGHT + iImprovement <= iAlpha))
{
@@ -1689,15 +1708,18 @@ _ShouldWeConsiderThisMove(IN SEARCHER_THREAD_CONTEXT *ctx,
// which Eval() exit tier produced iEval/rgiPositional this call.
// CALIBRATE_QSEARCH_FUTILITY data (1500 real-game positions,
// tests/twic_sample.ep_, sd 6) showed the three tiers need very
-// different margins to reach a similar surprise rate: full-eval
-// source was still failing 1.65-6.4% of diagnosed rejects even
-// hundreds of centipawns short of the bar, regular-lazy was already
-// close to safe (1.26% down to 0.43%), super-lazy was only risky very
-// close to the bar (7.96% at distance 0-25, 0.54% by 400+). Indexed
-// by PLY_INFO.uLastPositionalSource (set inside Eval() -- see
-// EVAL_POSITIONAL_SOURCE_* in chess.h). Provisional, derived from one
-// run at sd 6; re-derive with `calibrate qsearchfutility` if search
-// behavior affecting typical qsearch iEval/iFutility gaps changes.
+// different margins to reach a similar surprise rate. Indexed by
+// PLY_INFO.uLastPositionalSource (set inside Eval() -- see
+// EVAL_POSITIONAL_SOURCE_* in chess.h). Re-derive with `calibrate
+// qsearchfutility` if search behavior affecting typical qsearch
+// iEval/iFutility gaps changes.
+//
+// 2026-09-17 re-recalibration: LAZY and SUPERLAZY re-derived (same
+// harness/pool/depth) after eval.c's lazy/super-lazy iPositional
+// stopped being flat 0 and became a real, signed p50-based estimate --
+// see FUTILITY_BASE_MARGIN_LAZY/SUPERLAZY's own derivation comment in
+// chess.h for the full before/after surprise-rate table. FULL is
+// untouched (its iPositional wasn't part of that change).
//
// 2026-09-11: also reused (unmodified) by QSearchFromCheckNoStandPat's
// own stand-pat-safety-net prune below -- moved up here so both
diff --git a/src/searchsup.c b/src/searchsup.c
index 2ec001f..26856cd 100644
--- a/src/searchsup.c
+++ b/src/searchsup.c
@@ -1154,7 +1154,13 @@ Return value:
//
if (WE_SHOULD_STOP_SEARCHING)
{
+ // 2026-09-18: same staleness hazard as the IsDraw/too-deep
+ // paths below -- clear this ply's own PV[] slot too, not just
+ // mvBest. See the IsDraw branch's comment for the full
+ // mechanism (an ancestor's UpdatePV blindly copies this row
+ // forward assuming it's fresh).
pi->mvBest.uMove = 0;
+ pi->PV[ctx->uPly].uMove = 0;
*piScore = INVALID_SCORE;
goto end;
}
@@ -1191,10 +1197,33 @@ Return value:
pi->mvBest.uMove = 0;
INC(ctx->sCounters.tree.u64LeafCount);
*piScore = g_iDrawScore[ctx->sPosition.uToMove];
+ // 2026-09-18: pi->PV[ctx->uPly] has the exact same staleness
+ // hazard as pi->mvBest above, but was only ever cleared
+ // conditionally, via UpdatePV(ctx, DRAWMOVE) -- and only when
+ // the draw score actually falls in the open (alpha, beta)
+ // window. When it doesn't, this ply's PV[] row is left
+ // whatever an unrelated earlier node last wrote there.
+ // sPlyInfo[] rows are shared/reused across every node that
+ // ever occupies a given ply (see UpdatePV's own sentinel-case
+ // comment), so a later ancestor's UpdatePV can blindly copy
+ // that stale row forward via `this->PV[u] = next->PV[u]`,
+ // trusting its zero-terminator as proof of freshness --
+ // splicing a leftover move from a completely different
+ // subtree onto the real line. This is the root cause of a
+ // WalkPV/MakeMove assertion failure (move.c:881,
+ // "ASSERT(!IS_EMPTY(pCaptured))" -- a move that's real
+ // *somewhere* but whose capture target is empty on the
+ // actual board) reproduced against tests/cross_sectional_
+ // 200k.ep_. Clear unconditionally so a non-window draw score
+ // can't leave this row stale.
if ((*piAlpha < *piScore) && (*piScore < *piBeta))
{
UpdatePV(ctx, DRAWMOVE);
}
+ else
+ {
+ pi->PV[ctx->uPly].uMove = 0;
+ }
DTTRACE("<i bold=\"draw\"/>");
goto end;
}
@@ -1245,8 +1274,16 @@ Return value:
*piScore = Eval(ctx, *piAlpha, *piBeta, NULL);
pi->iEval = *piScore;
}
- // See the IsDraw branch above -- same staleness hazard, same fix.
+ // See the IsDraw branch above -- same staleness hazard, same
+ // fix -- this path returns a live, legitimate score (either a
+ // real Eval() call or *piAlpha) that absolutely can raise an
+ // ancestor's alpha, and is reached constantly (any node
+ // against the ply ceiling or qsearch-extend limit), making it
+ // the most likely real source of the reproduced WalkPV
+ // staleness bug (see the IsDraw branch's comment for the full
+ // mechanism and the move.c:881 assert it produces).
pi->mvBest.uMove = 0;
+ pi->PV[ctx->uPly].uMove = 0;
DTTRACE("<toodeep/>");
goto end;
}
diff --git a/src/testrecogn.c b/src/testrecogn.c
index 7bee3ad..d320394 100644
--- a/src/testrecogn.c
+++ b/src/testrecogn.c
@@ -625,17 +625,33 @@ Return value:
//
// Build a KPK (one pawn, one lone king) position directly from square
-// placements -- white always plays the pawn side, black always plays
-// the lone king (color-symmetric by construction: RecognLookup/
-// _RecognizeKPK determine "strong side" dynamically from piece
-// counts, not from hardcoded color, so fixing white=pawn loses no
-// coverage and halves the enumeration, same reasoning as
-// _BuildKNKPPosition above).
+// placements. fBlackHasPawn selects which color plays the pawn side
+// (cStrongKing/cWeakKing name roles, not colors, in both cases).
+//
+// 2026-09-18: the "color-symmetric by construction" claim this
+// function's comment used to make -- that fixing white=pawn always
+// and never testing black=pawn loses no coverage, since RecognLookup/
+// _RecognizeKPK determine "strong side" dynamically from piece counts
+// -- turned out to be FALSE. Found via fuzzing tests/cross_
+// sectional_200k.ep_: 8/8/8/8/8/8/4K2p/7k w - - 0 0 (White Ke2 lone
+// king, Black Kh1+Ph2, White to move) hit a RECOGNIZER_SANITY_FAIL --
+// recognizer claimed Black winning, EGTB says draw (White can
+// stalemate Black rather than let the pawn queen). This exhaustive
+// test's original white-only enumeration reported 0 wrong across
+// 165,676 EGTB-covered positions, which looked like it cleared
+// _RecognizeKPK's post-2026-09-16 tempo-margin fix -- but it never
+// actually exercised the black-has-pawn code path (specifically
+// _GetPassersCriticalSquares' separate per-color lookup table) at
+// all, so that "0 wrong" was never evidence either way for this
+// failure mode. fBlackHasPawn added so the caller can enumerate both
+// color assignments explicitly, closing the gap instead of assuming
+// symmetry.
//
static FLAG
_BuildKPKPosition(OUT POSITION *pos,
IN COOR cStrongKing, IN COOR cWeakKing,
- IN COOR cPawn, IN ULONG uToMove)
+ IN COOR cPawn, IN ULONG uToMove,
+ IN FLAG fBlackHasPawn)
{
PIECE pBoard[128];
CHAR szFen[256];
@@ -643,9 +659,18 @@ _BuildKPKPosition(OUT POSITION *pos,
ULONG x;
memset(pBoard, 0, sizeof(pBoard));
- pBoard[cStrongKing] = WHITE_KING;
- pBoard[cWeakKing] = BLACK_KING;
- pBoard[cPawn] = WHITE_PAWN;
+ if (fBlackHasPawn)
+ {
+ pBoard[cStrongKing] = BLACK_KING;
+ pBoard[cWeakKing] = WHITE_KING;
+ pBoard[cPawn] = BLACK_PAWN;
+ }
+ else
+ {
+ pBoard[cStrongKing] = WHITE_KING;
+ pBoard[cWeakKing] = BLACK_KING;
+ pBoard[cPawn] = WHITE_PAWN;
+ }
memset(szFen, 0, sizeof(szFen));
p = szFen;
@@ -711,10 +736,39 @@ Routine description:
when actually validating a KPK-area fix, the same way
TestRecognExhaustiveKNKP is used for KNKP.
- Fixes white=pawn/black=lone-king by construction (lossless -- see
- _BuildKPKPosition) and uses left/right board mirror symmetry on
- the strong king's start square to roughly halve the raw
- combination count.
+ 2026-09-18: the "fixes white=pawn/black=lone-king by construction,
+ lossless" and "left/right mirror symmetry, roughly halves the
+ count" claims this comment used to make were BOTH wrong -- found
+ via fuzzing tests/cross_sectional_200k.ep_, which turned up a real
+ _EvalPasserRacesAgainstLoneKings bug (8/8/8/8/8/8/4K2p/7k w - -,
+ strong side's own king on its own rook pawn's queening square) that
+ this test reported "0 wrong" on even after removing both
+ shortcuts. Now iterates BOTH color assignments explicitly (see
+ _BuildKPKPosition's fBlackHasPawn) over the FULL board (no
+ cStrongKing file restriction) -- ~4x the raw combos, still fast
+ enough for on-demand use.
+
+ IMPORTANT REMAINING GAP, not yet closed: this still cannot exercise
+ _RecognizeKPK's pawn-hash-cache-hit branch (the `pHash->u64Key ==
+ pos->u64PawnSig` check gating _EvalPasserRacesAgainstLoneKings).
+ The pawn hash signature depends only on pawn placement, not king
+ squares -- a freshly-built, isolated test position essentially
+ never has a matching warm cache entry (ReInitializeSearcherContext
+ doesn't pre-populate it), so this test structurally falls through
+ to the critical-square path every time and never really exercises
+ the cache-hit path a long, real search naturally warms into (many
+ king placements sharing one pawn skeleton over a deep search tree).
+ This is *why* three separate _EvalPasserRacesAgainstLoneKings bugs
+ (this session's stalemate-corner case, the earlier connected-
+ passers case, and the originally-committed bishop-pawn corner case)
+ all made it past a "0 wrong" run of this test at some point -- a
+ clean run here is NOT proof that function is sound. Confirmed by
+ hand for the 2026-09-18 fix: forcing a warm cache via an explicit
+ `eval` command before `go` on the repro position reproduced the bug
+ pre-fix and confirmed the fix post-fix, but that verification step
+ isn't automated here yet. Worth building: warm the pawn hash (e.g.
+ call Eval() once per constructed position before RecognLookup) so
+ this test can actually reach the branch it's nominally checking.
Parameters:
@@ -747,7 +801,14 @@ Return value:
FOREACH_SQUARE(cStrongKing)
{
if (!IS_ON_BOARD(cStrongKing)) continue;
- if (FILE(cStrongKing) > D) continue;
+ // 2026-09-18: mirror-halving restriction (FILE(cStrongKing) >
+ // D) removed -- see _BuildKPKPosition's comment. The critical-
+ // square lookup table this test is meant to validate is raw
+ // hand-typed data, not derived by formula, so there's no
+ // guarantee its A-file and H-file entries are true mirrors of
+ // each other -- exactly the kind of transcription gap this
+ // restriction could hide. Slower (full board instead of half)
+ // but unambiguous.
FOREACH_SQUARE(cWeakKing)
{
@@ -762,41 +823,52 @@ Return value:
for (uToMove = BLACK; uToMove <= WHITE; uToMove++)
{
- uTotal++;
- if (0 == (uTotal % 100000))
- {
- Trace(" ... %lu raw combos so far (%lu checked, "
- "%lu wrong)\n", uTotal, uChecked, uWrong);
- }
- if (FALSE == _BuildKPKPosition(&pos, cStrongKing,
- cWeakKing, cPawn,
- uToMove))
+ ULONG uColorPass;
+ // 2026-09-18: loop both color assignments -- see
+ // _BuildKPKPosition's comment. uColorPass 0 =
+ // white has the pawn (original coverage), 1 =
+ // black has the pawn (the gap that let the
+ // fuzzer-found counterexample through undetected).
+ for (uColorPass = 0; uColorPass < 2; uColorPass++)
{
- continue;
- }
- ReInitializeSearcherContext(&pos, ctx);
- uVal = RecognLookup(ctx, &iScore, FALSE);
- if (UNRECOGNIZED == uVal)
- {
- continue;
- }
- if (FALSE == ProbeEGTB(ctx, &iEgtb))
- {
- uSkipped++;
- continue;
- }
- uChecked++;
- if (FALSE == _RecognizerAgreesWithEGTB(iScore, uVal,
- iEgtb))
- {
- uWrong++;
- if (uReportedWrong < 10)
+ uTotal++;
+ if (0 == (uTotal % 100000))
+ {
+ Trace(" ... %lu raw combos so far (%lu checked, "
+ "%lu wrong)\n", uTotal, uChecked, uWrong);
+ }
+ if (FALSE == _BuildKPKPosition(&pos, cStrongKing,
+ cWeakKing, cPawn,
+ uToMove,
+ (1 == uColorPass)))
{
- Trace("MISMATCH iScore=%d uVal=%lu "
- "iEgtb=%d toMove=%lu\n",
- iScore, uVal, iEgtb, pos.uToMove);
- DumpPosition(&pos);
- uReportedWrong++;
+ continue;
+ }
+ ReInitializeSearcherContext(&pos, ctx);
+ uVal = RecognLookup(ctx, &iScore, FALSE);
+ if (UNRECOGNIZED == uVal)
+ {
+ continue;
+ }
+ if (FALSE == ProbeEGTB(ctx, &iEgtb))
+ {
+ uSkipped++;
+ continue;
+ }
+ uChecked++;
+ if (FALSE == _RecognizerAgreesWithEGTB(iScore, uVal,
+ iEgtb))
+ {
+ uWrong++;
+ if (uReportedWrong < 20)
+ {
+ Trace("MISMATCH (blackHasPawn=%lu) iScore=%d "
+ "uVal=%lu iEgtb=%d toMove=%lu\n",
+ uColorPass, iScore, uVal, iEgtb,
+ pos.uToMove);
+ DumpPosition(&pos);
+ uReportedWrong++;
+ }
}
}
}
diff --git a/src/util.c b/src/util.c
index a8ecc3c..deeb77d 100644
--- a/src/util.c
+++ b/src/util.c
@@ -1118,6 +1118,19 @@ Return value:
strncat(buf, " ", ARRAY_LENGTH(buf) - strlen(buf) - 1);
}
+ // 2026-09-18: reverted back to MakeMove -- the 09-17/18 switch
+ // to MakeUserMove (to tolerate a stale PV entry) called
+ // GenerateMoves() internally, which writes into
+ // ctx->sMoveStack at ctx->uPly. WalkPV runs on the live,
+ // actively-searching thread-0 context (called from PrintPV on
+ // every improved root score, not an offline diagnostic path),
+ // and was written assuming Make*Move only touches
+ // ctx->sPosition/ply bookkeeping -- clobbering the real
+ // search's move-stack slots mid-walk is a plausible source of
+ // the 2026-09-17 self-play regression this was bundled into.
+ // Trading back to the narrower stale-PV assert/crash risk
+ // MakeUserMove was meant to fix until that one can be
+ // addressed without touching sMoveStack.
if (FALSE == MakeMove(ctx, mv)) break;
u++;