Dynamic move-ordering/LMR session: honest FH tracking, decoupled admission/magnitude, bugfixes, tooling

Lands the 2026-09-28..10-03 move-ordering investigation (full history in
lmr_testing/*.md). Summary of what ships:

LMR/move-ordering redesign (searchsup.c, dynamic.c, data.c, chess.h):
- GetLMRReduction's admission decision (do we reduce at all) is now
  decoupled from magnitude (how much): LMR_STATIC_ADMIT_THOUSANDTHS=500
  gates admission via the 4D joint fail-high table (data.c) plus a
  countermove-match veto; magnitude is a flat Crafty/Ethereal-shaped
  log(depth)*log(moves) table (g_iLMRQuietReduction), which measured as
  good or better than the old per-tier joint-table payout while being
  structurally immune to the per-bucket miscalibration that sank the
  tiered approach. g_FailHighs honesty fix (real 0/0 init, not a fake
  prior) underlies the dynamic-danger signal.
- Retired NumLeftoverMovesToSelect's depth-indexed selection budget --
  evidence showed the leftover pool has real findable signal a bailout
  was discarding.
- ShouldLatePrune (overt late-move pruning) implemented but disabled at
  its call site -- measured net-negative at the tested threshold; kept
  for reference and as a base for a future hang-bucket-9-targeted retry.
- PRUNE_VERIFY, PRUNE_LIVE, DISABLE_LMR: opt-in diagnostic build flags
  (GNUmakefile), off in the standard profile.

Bug fixes found and fixed while exercising this work:
- JOINT_EXTRA_TIER_COUNT 4->6 (chess.h): fixed an out-of-bounds counter
  write reachable at remaining depth >= 14 (unreachable at sd10, routine
  in real search/self-play).
- Node-type instrumentation (search.c, root.c, searchsup.c): fixed four
  compounding defects that produced impossible percentages (e.g. 106.2%
  of PV nodes) -- wrong denominator, qsearch misclassified as main
  search, untyped early-exit nodes, and fail-low tests using the
  in-place-raised alpha instead of the initial one.
- FH-rate unit unification (search.c, searchsup.c, dynamic.c, chess.h):
  GetMoveFailHigh* returned per-mille while g_uJointFHThousandths is
  thousandths-of-a-percent, two different units sharing a "_THOUSANDTHS"
  suffix. Unified to one scale, renamed to "_PCT_THOUSANDTHS".
  RateBucket's band edges fixed to match.
- QSearch option B (search.c): waives the opponent-stand-pat requirement
  at NODE_PV only (sound there since the exact score propagates; not
  sound at null-window nodes). iExtensionAmount unit fix (was 0/2 into a
  ONE_PLY-scaled field). pi->iEval now published when Eval() runs.
- eval_tune/match_play.py local_ebf: was reporting 0.000 in every match
  since 2026-09-18 (regex only matched the trailing-stats parser, not
  the live move-scan loop; engines were also spawned with "nopost" so
  root.c never emitted the lines at all). True per-ply EBF is ~2.0, not
  the ~3.9 every prior match report implied.
- eval_tune/test_vs_head.sh: positional-arg shift ordering bug caused
  every no-extra-args run to silently rebuild ./typhoon with a
  non-standard profile and, separately, could feed a flag's value in as
  the candidate binary path.
- PERF_COUNTERS cleanup (eval.c, search.c): moved 3 leaked rdtsc sites to
  the existing EVAL_TIME guard, deleted two concluded one-off diagnostics
  (u64HeadStyle*, u64FHJointHist/RateBucket/AttemptsBucket) and their
  report plumbing, and re-gated a per-move hang-material estimate behind
  FH_TABLE_COLLECTION instead of running it unconditionally.

Infrastructure:
- match_play.py time-odds support (--odds/--st-head/--st-candidate),
  validated against the engine seat rather than color.
- Wilson-upper-bound re-bake of the joint table (data.c): cells store a
  confidence-aware bound instead of a raw point estimate, so a consumer
  can threshold on the value alone; monotone, so every consumer became
  strictly more conservative. Verified behavior-neutral for LMR.

Known-open items (not resolved here, see lmr_testing/GET_LMR_REDUCTION.md
section 10 and 2026-10-03_UNCOMMITTED_TRIAGE.md): the self-play gate on
this candidate is underpowered (+5 Elo, 95% CI roughly [-6, +17]); the
curated-suite solve-count deltas used earlier in the investigation turned
out to be noise-dominated cancellation when diffed per position, not the
clean signal they first appeared to be.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

1 files changed, 88 insertions, 27 deletions

diff --git a/src/dynamic.c b/src/dynamic.c
index 687fa74..2451fa6 100644
--- a/src/dynamic.c
+++ b/src/dynamic.c
@@ -176,14 +176,19 @@ Return value:
 
 **/
 {
-    ULONG u;
-
     memset(g_HistoryCounters, 0, sizeof(g_HistoryCounters));
     memset(g_ContinuationHistory, 0, sizeof(g_ContinuationHistory));
-    for (u = 0; u < FH_STATS_TABLE_SIZE; u++) 
-    {
-        g_FailHighs[u].uWholeThing = 0x00010001;
-    }
+    // 2026-10-02: was 0x00010001 (1 FH / 1 attempt, i.e. a faked 100%
+    // rate) so that GetMoveFailHighPercentage's callers had something
+    // non-zero to gate on even with no real evidence. That made the
+    // table dishonest -- a move's reported percentage didn't track its
+    // actual observed rate until enough real attempts diluted the
+    // fake prior back out. Zeroing it means GetMoveFailHighPercentage
+    // returns a genuine 0/0 until the first real observation; callers
+    // that need to withhold judgement on a cold move must check
+    // *puAttempts themselves rather than relying on an inflated
+    // starting percentage (see GetLMRReduction, searchsup.c).
+    memset(g_FailHighs, 0, sizeof(g_FailHighs));
 }
 
 
@@ -346,14 +351,14 @@ Routine description:
     generalizes across any branch where the opponent plays the same
     move again, unlike killers.
 
-    Depth-gated: a shallow fail-high (low uRemainingDepth, i.e. close
-    to the QSearch handoff) is much lower-confidence evidence than one
-    found with many plies still remaining, but unlike killers (which
-    are ply-scoped and self-correct locally), this table is global --
-    keyed only by move identity, with nothing to stop a later, shallow
-    write from clobbering an earlier, deep one just because it's more
-    recent.  Only overwrite when the new evidence is at least as deep
-    as what's already there.
+    2026-10-02: corrected -- this used to claim depth-gated writes
+    ("only overwrite when the new evidence is at least as deep as
+    what's already there"), but that was already stale: an A/B test
+    (see the inline comment below) measured depth-gating as *worse*
+    (lower hit rate, no EBF gain) than unconditional recency-based
+    overwrite, and the code was changed accordingly without this
+    routine description being updated to match. Always overwrites on
+    recency now; there is no depth gate.
 
 Parameters:
 
@@ -369,7 +374,6 @@ Return value:
 {
     MOVE mvLast;
     ULONG u;
-    ULONG uPly;
 
     ASSERT(!IS_CAPTURE_OR_PROMOTION(mv));
     ASSERT(mv.uMove);
@@ -405,15 +409,12 @@ Return value:
     // measured *worse* hit rate with no EBF gain -- this table is
     // global, not ply-scoped, so a frozen "deepest wins" entry can
     // go stale while later, more locally-relevant evidence gets
-    // rejected. Always overwrite (recency); track depth instead to
-    // scale the read-time bonus.
-    uPly = uRemainingDepth / ONE_PLY;
+    // rejected. Always overwrite (recency).
     if (!IS_SAME_MOVE(mv, ctx->mvCounter[u][0]))
     {
         ctx->mvCounter[u][1] = ctx->mvCounter[u][0];
         ctx->mvCounter[u][0] = mv;
     }
-    ctx->uCounterDepth[u] = (UCHAR)MINU(uPly, 255);
 }
 
 
@@ -994,25 +995,52 @@ Return value:
 
 
 ULONG
-GetMoveFailHighPercentage(IN MOVE mv, OUT ULONG *puAttempts)
+GetMoveFailHighPctThousandths(IN MOVE mv, OUT ULONG *puAttempts)
 /**
 
 Routine description:
 
-    Lookup a move in the fail high percentage history table and return
-    its approximate fail high percentage.
+    Lookup a move identity (see MOVE_TO_FH_INDEX -- (cFrom, cTo, pMoved),
+    NOT a specific position) in the fail-high history table and return
+    its observed fail-high rate in THOUSANDTHS OF A PERCENT (0-100000),
+    the same unit g_uJointFHThousandths (data.c) uses, so a caller can
+    compare this move's own dynamic rate directly against the static
+    table's bucket-specific prediction for the situation it's in right
+    now.
+
+    2026-10-03: this routine used to return per-mille (0-1000) while
+    claiming, in this very comment, to share the static table's unit --
+    which is thousandths of a PERCENT, 100x finer. Every caller was
+    individually correct (each compared against a constant calibrated
+    in per-mille), but the two families were impossible to tell apart by
+    name: ShouldLatePrune held LMP_DYNAMIC_MAX_THOUSANDTHS (1, meaning
+    0.1%) and LMP_STATIC_MAX_THOUSANDTHS (1, meaning 0.001%) side by
+    side. Converted to the static table's unit so the comparison this
+    comment always advertised is now literally valid, and every
+    threshold constant renamed to ..._PCT_THOUSANDTHS.
+    Returns a genuine 0/0 -- 0, *puAttempts 0 -- for a move
+    identity with no observations since the last ClearDynamicMoveOrdering;
+    callers that need to withhold judgement on a cold move must check
+    *puAttempts themselves, not assume a low return value means "safe."
+
+    Renamed from GetMoveFailHighPercentage (whole-percent resolution)
+    2026-10-02 -- whole-percent granularity can't distinguish 0% from
+    the sub-1%-of-the-time baseline rate (data.c:1209's "Overall
+    eligible FH%: 0.400%"), which matters once this is compared against
+    the joint table's own thousandths-resolution predictions rather
+    than a hand-picked whole-percent threshold.
 
 Parameters:
 
     MOVE mv
     ULONG *puAttempts : optional (may be NULL) -- receives the number of
-        observations the percentage is based on, so callers can weight
-        by sample confidence instead of trusting a percentage computed
-        from as few as one attempt.
+        observations the rate is based on, so callers can weight by
+        sample confidence instead of trusting a rate computed from as
+        few as one attempt.
 
 Return value:
 
-    ULONG
+    ULONG : fail-high rate in thousandths of a percent (0-100000)
 
 **/
 {
@@ -1026,11 +1054,44 @@ Return value:
     {
         return(0);
     }
-    n *= 100;
+    n *= 100000;
     return(n / d);
 }
 
 
+void
+GetMoveFailHighRawCounts(IN MOVE mv, OUT ULONG *puFailHighs, OUT ULONG *puAttempts)
+/**
+
+Routine description:
+
+    2026-10-02, one-off diagnostic support: raw (unrounded) FailHighs/
+    Attempts counts behind GetMoveFailHighPctThousandths, needed to
+    reconstruct HEAD's old (pre-honest-init) equivalent percentage
+    exactly -- (rawFH+1)*100/(rawAttempts+1) -- without the rounding
+    GetMoveFailHighPctThousandths' integer division already introduces.
+    Not meant for any other caller; remove once the HEAD-vs-candidate
+    reduction-rate comparison this supports is done.
+
+Parameters:
+
+    MOVE mv
+    ULONG *puFailHighs
+    ULONG *puAttempts
+
+Return value:
+
+    void
+
+**/
+{
+    ULONG u = MOVE_TO_FH_INDEX(mv);
+
+    *puFailHighs = g_FailHighs[u].u16FailHighs;
+    *puAttempts = g_FailHighs[u].u16Attempts;
+}
+
+
 void 
 CleanupDynamicMoveOrdering(void)
 /**