Dynamic move-ordering/LMR session: honest FH tracking, decoupled admission/magnitude, bugfixes, tooling

Lands the 2026-09-28..10-03 move-ordering investigation (full history in
lmr_testing/*.md). Summary of what ships:

LMR/move-ordering redesign (searchsup.c, dynamic.c, data.c, chess.h):
- GetLMRReduction's admission decision (do we reduce at all) is now
  decoupled from magnitude (how much): LMR_STATIC_ADMIT_THOUSANDTHS=500
  gates admission via the 4D joint fail-high table (data.c) plus a
  countermove-match veto; magnitude is a flat Crafty/Ethereal-shaped
  log(depth)*log(moves) table (g_iLMRQuietReduction), which measured as
  good or better than the old per-tier joint-table payout while being
  structurally immune to the per-bucket miscalibration that sank the
  tiered approach. g_FailHighs honesty fix (real 0/0 init, not a fake
  prior) underlies the dynamic-danger signal.
- Retired NumLeftoverMovesToSelect's depth-indexed selection budget --
  evidence showed the leftover pool has real findable signal a bailout
  was discarding.
- ShouldLatePrune (overt late-move pruning) implemented but disabled at
  its call site -- measured net-negative at the tested threshold; kept
  for reference and as a base for a future hang-bucket-9-targeted retry.
- PRUNE_VERIFY, PRUNE_LIVE, DISABLE_LMR: opt-in diagnostic build flags
  (GNUmakefile), off in the standard profile.

Bug fixes found and fixed while exercising this work:
- JOINT_EXTRA_TIER_COUNT 4->6 (chess.h): fixed an out-of-bounds counter
  write reachable at remaining depth >= 14 (unreachable at sd10, routine
  in real search/self-play).
- Node-type instrumentation (search.c, root.c, searchsup.c): fixed four
  compounding defects that produced impossible percentages (e.g. 106.2%
  of PV nodes) -- wrong denominator, qsearch misclassified as main
  search, untyped early-exit nodes, and fail-low tests using the
  in-place-raised alpha instead of the initial one.
- FH-rate unit unification (search.c, searchsup.c, dynamic.c, chess.h):
  GetMoveFailHigh* returned per-mille while g_uJointFHThousandths is
  thousandths-of-a-percent, two different units sharing a "_THOUSANDTHS"
  suffix. Unified to one scale, renamed to "_PCT_THOUSANDTHS".
  RateBucket's band edges fixed to match.
- QSearch option B (search.c): waives the opponent-stand-pat requirement
  at NODE_PV only (sound there since the exact score propagates; not
  sound at null-window nodes). iExtensionAmount unit fix (was 0/2 into a
  ONE_PLY-scaled field). pi->iEval now published when Eval() runs.
- eval_tune/match_play.py local_ebf: was reporting 0.000 in every match
  since 2026-09-18 (regex only matched the trailing-stats parser, not
  the live move-scan loop; engines were also spawned with "nopost" so
  root.c never emitted the lines at all). True per-ply EBF is ~2.0, not
  the ~3.9 every prior match report implied.
- eval_tune/test_vs_head.sh: positional-arg shift ordering bug caused
  every no-extra-args run to silently rebuild ./typhoon with a
  non-standard profile and, separately, could feed a flag's value in as
  the candidate binary path.
- PERF_COUNTERS cleanup (eval.c, search.c): moved 3 leaked rdtsc sites to
  the existing EVAL_TIME guard, deleted two concluded one-off diagnostics
  (u64HeadStyle*, u64FHJointHist/RateBucket/AttemptsBucket) and their
  report plumbing, and re-gated a per-move hang-material estimate behind
  FH_TABLE_COLLECTION instead of running it unconditionally.

Infrastructure:
- match_play.py time-odds support (--odds/--st-head/--st-candidate),
  validated against the engine seat rather than color.
- Wilson-upper-bound re-bake of the joint table (data.c): cells store a
  confidence-aware bound instead of a raw point estimate, so a consumer
  can threshold on the value alone; monotone, so every consumer became
  strictly more conservative. Verified behavior-neutral for LMR.

Known-open items (not resolved here, see lmr_testing/GET_LMR_REDUCTION.md
section 10 and 2026-10-03_UNCOMMITTED_TRIAGE.md): the self-play gate on
this candidate is underpowered (+5 Elo, 95% CI roughly [-6, +17]); the
curated-suite solve-count deltas used earlier in the investigation turned
out to be noise-dominated cancellation when diffed per position, not the
clean signal they first appeared to be.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

1 files changed, 104 insertions, 0 deletions

diff --git a/src/GNUmakefile b/src/GNUmakefile
index 1784551..b9b6d16 100644
--- a/src/GNUmakefile
+++ b/src/GNUmakefile
@@ -172,6 +172,110 @@ ifdef CALIBRATE_QSEARCH_FUTILITY
 PROFILE		+=	-DCALIBRATE_QSEARCH_FUTILITY
 endif
 endif # EVERYTHING
+# 2026-09-28: full-coverage (non-sampled) throwaway shadow re-search,
+# purely to measure the "silent miss" failure mode a reduced move's own
+# real re-search hook can't see: the extra-reduced score is < beta (no
+# real re-search triggered) but removing just the EXTRA plies (i.e. the
+# depth HEAD's own baseline -ONE_PLY reduction would have used) finds
+# a fail-high. Every extra-reduced move gets this check within a build
+# with this flag on -- doubles search cost for that subset, so
+# calibration-only, never part of a normal or self-play build. Opt in
+# explicitly, same as the other CALIBRATE_*/DUMP_* flags above.
+ifdef JOINT_EXTRA_VERIFY
+PROFILE		+=	-DJOINT_EXTRA_VERIFY
+endif
+# 2026-09-28: audit-only EFP disable, same technique this project used
+# during the joint-table collection run's self-censoring fix (force the
+# prune decision FALSE, keep computing/logging everything around it) --
+# isolates the joint-table-driven LMR's own behavior from EFP's
+# single-axis (gap/down only) margin-based pruning, which the
+# JOINT_EXTRA_VERIFY silent-miss data implicates as the dominant source
+# of the measured instability, not LMR's own reduction depth. Opt-in,
+# calibration-only.
+ifdef DISABLE_EFP
+PROFILE		+=	-DDISABLE_EFP
+endif
+# 2026-09-28: Trace() the FEN/move/scores whenever JOINT_EXTRA_VERIFY's
+# shadow search catches a silent miss -- for looking at real, concrete
+# failing cases by hand (the methodology that actually cracked the
+# hanging-material bug earlier this project, rather than continuing to
+# guess which other search mechanism might explain the EFP-tier-window
+# silent-miss spike). Requires JOINT_EXTRA_VERIFY. Calibration-only.
+ifdef SILENT_MISS_TRACE
+PROFILE		+=	-DSILENT_MISS_TRACE
+endif
+# 2026-09-29: alternate, maximally-simple extra-reduction policy for A/B
+# comparison against the axis-gated joint-table version -- skips the
+# dumb/gap/late table lookup entirely and just asks "is there enough
+# pre-reduction depth headroom to spare (>= 6 plies)?" If so, apply the
+# full extra reduction (R=3) uniformly, for ANY depth >= 6 (not just
+# exactly 6 -- 7, 8, 9, ... all get the same treatment). Tests whether
+# the axis-based table is pulling any real weight once a depth floor is
+# already in place, or whether depth alone does just as well.
+ifdef DEPTH_ONLY_EXTRA_REDUCTION
+PROFILE		+=	-DDEPTH_ONLY_EXTRA_REDUCTION
+endif
+# 2026-09-29: neutralize searchsup.c's null-move "history-reduced
+# bailout" branch (Tord Romstad idea) for our own shadow-verification
+# Search() call specifically, by temporarily zeroing our own
+# pi->iExtensionAmount around it -- see search.c's NEUTRALIZE_NULL_
+# BAILOUT comment. Correct in the real search path, that branch
+# fabricates a "please re-search me" placeholder score that our
+# bespoke, wrapper-less shadow call can't interpret and misreads as a
+# genuine fail-high. Build once WITHOUT this flag to measure how much
+# of our silent-miss data is this artifact (see
+# u64JointExtraVerifyNullBailoutSignature), once WITH it for the
+# corrected rate.
+# 2026-10-03: PRUNE_VERIFY -- shadow verification for the DEEP PRUNE gate
+# (ShouldLatePrune), the question JOINT_EXTRA_VERIFY structurally cannot
+# answer: "this move was about to be skipped ENTIRELY -- would searching
+# it have failed high?" AUDIT-ONLY by construction: the gate runs, its
+# verdict is recorded, and the move is then searched anyway, so the
+# measurement needs no bespoke shadow Search() call and has no
+# null-move-bailout exposure. Costs counter increments plus one extra
+# ShouldLatePrune evaluation per eligible move; never part of a normal
+# build. Pair with SILENT_MISS_TRACE to get a PRUNEMISS line (FEN, move,
+# alpha/beta, predicted FH, margin) for every miss found, for direct
+# manual investigation -- SPEC.md's agreed design requirement that any
+# wrong guess gets looked at and reasoned about, not just counted.
+# Requires PERF_COUNTERS.
+ifdef PRUNE_VERIFY
+PROFILE		+=	-DPRUNE_VERIFY
+endif
+
+# 2026-10-03: PRUNE_LIVE=<K> -- actually prune any gate-1-eligible move
+# whose 4D-table predicted FH% (Wilson upper bound, see data.c) is <= K
+# thousandths of a percent. Measurement vehicle for "what does pruning
+# N% of moves do to the branching factor", used with the C/r sweep in
+# /tmp/typhoon/cr_measure. NOT a shipping mechanism: it has none of the
+# agreed safety furniture (no dynamic per-move-identity veto, no
+# ShouldLatePrune move-count floor, no shadow verification) -- those are
+# what a real deep-prune gate needs. Measure with it, do not play with
+# it.
+# 2026-10-03: DISABLE_LMR -- GetLMRReduction returns 0 unconditionally,
+# so no move is ever reduced. Exists for ONE experiment: what is LMR
+# actually worth in this engine? The time-odds calibration established
+# that TIME-bought depth is ~95 Elo/ply here, but nothing has ever
+# measured whether REDUCTION-bought depth converts at that rate. Two
+# hypotheses fit "deeper but no stronger": (1) missed refutations, i.e.
+# the reduction is too aggressive and its errors cancel the gain --
+# remedy is to constrain it; or (2) nominal depth under reduction is a
+# debased currency that never carried a time-ply's worth of information
+# -- in which case constraining it cannot help, since less reduction
+# buys less depth at the same bad rate. Removing LMR entirely and
+# pricing the loss against the 95 Elo/ply yardstick separates them.
+# Measurement only; never a shipping configuration.
+ifdef DISABLE_LMR
+PROFILE		+=	-DDISABLE_LMR
+endif
+
+ifdef PRUNE_LIVE
+PROFILE		+=	-DPRUNE_LIVE=$(PRUNE_LIVE)
+endif
+
+ifdef NEUTRALIZE_NULL_BAILOUT
+PROFILE		+=	-DNEUTRALIZE_NULL_BAILOUT
+endif
 
 # Short commit hash (plus a -dirty suffix if the working tree has
 # uncommitted changes) baked into the binary so a --logfile trace can