Dynamic move-ordering/LMR session: honest FH tracking, decoupled admission/magnitude, bugfixes, tooling

Lands the 2026-09-28..10-03 move-ordering investigation (full history in
lmr_testing/*.md). Summary of what ships:

LMR/move-ordering redesign (searchsup.c, dynamic.c, data.c, chess.h):
- GetLMRReduction's admission decision (do we reduce at all) is now
  decoupled from magnitude (how much): LMR_STATIC_ADMIT_THOUSANDTHS=500
  gates admission via the 4D joint fail-high table (data.c) plus a
  countermove-match veto; magnitude is a flat Crafty/Ethereal-shaped
  log(depth)*log(moves) table (g_iLMRQuietReduction), which measured as
  good or better than the old per-tier joint-table payout while being
  structurally immune to the per-bucket miscalibration that sank the
  tiered approach. g_FailHighs honesty fix (real 0/0 init, not a fake
  prior) underlies the dynamic-danger signal.
- Retired NumLeftoverMovesToSelect's depth-indexed selection budget --
  evidence showed the leftover pool has real findable signal a bailout
  was discarding.
- ShouldLatePrune (overt late-move pruning) implemented but disabled at
  its call site -- measured net-negative at the tested threshold; kept
  for reference and as a base for a future hang-bucket-9-targeted retry.
- PRUNE_VERIFY, PRUNE_LIVE, DISABLE_LMR: opt-in diagnostic build flags
  (GNUmakefile), off in the standard profile.

Bug fixes found and fixed while exercising this work:
- JOINT_EXTRA_TIER_COUNT 4->6 (chess.h): fixed an out-of-bounds counter
  write reachable at remaining depth >= 14 (unreachable at sd10, routine
  in real search/self-play).
- Node-type instrumentation (search.c, root.c, searchsup.c): fixed four
  compounding defects that produced impossible percentages (e.g. 106.2%
  of PV nodes) -- wrong denominator, qsearch misclassified as main
  search, untyped early-exit nodes, and fail-low tests using the
  in-place-raised alpha instead of the initial one.
- FH-rate unit unification (search.c, searchsup.c, dynamic.c, chess.h):
  GetMoveFailHigh* returned per-mille while g_uJointFHThousandths is
  thousandths-of-a-percent, two different units sharing a "_THOUSANDTHS"
  suffix. Unified to one scale, renamed to "_PCT_THOUSANDTHS".
  RateBucket's band edges fixed to match.
- QSearch option B (search.c): waives the opponent-stand-pat requirement
  at NODE_PV only (sound there since the exact score propagates; not
  sound at null-window nodes). iExtensionAmount unit fix (was 0/2 into a
  ONE_PLY-scaled field). pi->iEval now published when Eval() runs.
- eval_tune/match_play.py local_ebf: was reporting 0.000 in every match
  since 2026-09-18 (regex only matched the trailing-stats parser, not
  the live move-scan loop; engines were also spawned with "nopost" so
  root.c never emitted the lines at all). True per-ply EBF is ~2.0, not
  the ~3.9 every prior match report implied.
- eval_tune/test_vs_head.sh: positional-arg shift ordering bug caused
  every no-extra-args run to silently rebuild ./typhoon with a
  non-standard profile and, separately, could feed a flag's value in as
  the candidate binary path.
- PERF_COUNTERS cleanup (eval.c, search.c): moved 3 leaked rdtsc sites to
  the existing EVAL_TIME guard, deleted two concluded one-off diagnostics
  (u64HeadStyle*, u64FHJointHist/RateBucket/AttemptsBucket) and their
  report plumbing, and re-gated a per-move hang-material estimate behind
  FH_TABLE_COLLECTION instead of running it unconditionally.

Infrastructure:
- match_play.py time-odds support (--odds/--st-head/--st-candidate),
  validated against the engine seat rather than color.
- Wilson-upper-bound re-bake of the joint table (data.c): cells store a
  confidence-aware bound instead of a raw point estimate, so a consumer
  can threshold on the value alone; monotone, so every consumer became
  strictly more conservative. Verified behavior-neutral for LMR.

Known-open items (not resolved here, see lmr_testing/GET_LMR_REDUCTION.md
section 10 and 2026-10-03_UNCOMMITTED_TRIAGE.md): the self-play gate on
this candidate is underpowered (+5 Elo, 95% CI roughly [-6, +17]); the
curated-suite solve-count deltas used earlier in the investigation turned
out to be noise-dominated cancellation when diffed per position, not the
clean signal they first appeared to be.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

1 files changed, 1543 insertions, 0 deletions

diff --git a/src/lmr_testing/SPEC.md b/src/lmr_testing/SPEC.md
new file mode 100644
index 0000000..55e6c5c
--- /dev/null
+++ b/src/lmr_testing/SPEC.md
@@ -0,0 +1,1543 @@
+# EXECUTIVE SUMMARY:
+
+We are implementing late move reduction / late move pruning in a chess
+engine based on a "simple idea": measure the propensity of a given
+move at a given node in the search tree (search.c/split.c) to fail
+high using (at least) four signals:
+
+- The move's "lateness", i.e. the order it was generated by the move
+  generator code (generate.c).  Because this code generates moves in
+  order with the best moves first, the only moves left at the end are
+  either quiet moves or losing captures, usually.  Not very
+  interesting ones, usually.  To be extra certain that the late moves
+  we are looking to reduce/prune are uninteresting, we also prohibit
+  checking moves, escapes from check, moves on PV nodes, moves with
+  extensions, killer moves, and, in general, do not prune when any of
+  several signals indicate that this "uninteresting quiet move" might
+  be more than meets the eye.
+- The move's "dumbness", i.e. does it seem to hang material?  A move
+  of our queen next to an enemy pawn might be a brilliant sacrifice
+  but most of the time it's a blunder.
+- The situation at the node (position) at which the move was
+  generated, i.e. is the side to move "winning" or are things "going
+  badly"?  Since our search is an alpha..beta search, the gap between
+  a static evaluation of the current node (iEval) and alpha (iAlpha)
+  indicates "how far the side to move would have to improve the score
+  at the current position in order for our move to matter (reach the
+  PV and/or fail high).  The bigger this gap, the less likely a move
+  is to fail high because to do so requires a stronger move.
+- The move's remaining depth.  This one is counter-intuitive but if
+  you do the work to compute the probability of a move to fail high
+  then you find that the deeper we search the move tree under that
+  move, the *less* likely the move is to fail high.  This is a really
+  interesting phenomenon -- it indicates that scores (and therefore
+  PVs) returned by shallow searches are very noisy but once you search
+  3-4-5-8-12 ply under the move, the score usually stabilizes and
+  actually diminishes the deeper you look.
+
+We call these four signals "axes" and have computed a probability table
+where, given a score for each of these signals, you can look up the
+computed probability that the move (and node) exhibiting these signals
+will fail high.  This is the core of our approach.
+
+# GOAL:
+
+The goal is to reduce the size of the chess search tree that we search
+(and therefore to reduce our effective branching factor) by reducing
+the depth we search the subset of moves we believe will not fail high
+or outright pruning them (i.e. not searching under them at all).  Each
+node saved (via reduction or pruning) is re-allocated to cause the
+engine to be able to search deeper on (hopefully) more interesting and
+meaningful nodes/moves elsewhere in the tree.
+
+# DANGER:
+
+When we reduce the search under a move we introduce error in the score
+returned.  It can bite us in two ways:
+- The move can fail high even though we did not expect it to.  If a
+  move fails high with a reduced depth search, search.c re-searches
+  it at full depth to confirm the fail high so this is not fatal; it
+  just wastes some time and double searches the tree under the move
+  in question.
+- The move can fail low even though if we would have searched it with
+  full depth it would have failed high.  This is a fatal miss; we have
+  lost information here.  To help defend against this, we have built a
+  version of the engine that, when it selects a move to reduce, it
+  then proceeds to run the search at full depth anyway and pays
+  attention to how many times a reduced move would have ended up
+  failing high.  This is very rare but not impossible.  It is, by far
+  the most dangerous thing about reducing moves.
+
+All in all, though, reducing the search tree under a weak,
+uninteresting move is less dangerous than the alternative: pruning the
+move completely.  Pruning means never searching under the move at all
+-- skipping it completely.  Therefore, the threshold to reduce a move
+is much less than the threshold needed to prune a move outright.
+When we are making irrecoverable decisions, we must be conservative!
+
+# Four-axis joint FH% table -- next steps, 2026-09-28
+
+See `lmr_testing/2026-09-28_four_axis_joint_table_plan.md` for the full
+plan (axis definitions, table shape/encoding, collection/bake recipe,
+consumption design). This section used to duplicate that entire file
+verbatim; removed 2026-09-30 (alongside the late-bucket fix below) so
+future edits only need to touch one copy.
+
+**Correction, 2026-09-30, carried over from the plan doc**: the table's
+`late` axis shipped-bucket mapping had an off-by-one. Raw `late`
+(`uLegalMoves` at the point it's sampled) can never be less than 2 -- the
+collection/consultation gate (`fPruneReduceEligible`, `search.c`) requires
+`uLegalMoves > 1`, and `_MoveNumBucket`'s own 1-based-index comment
+already ruled out 0. The old bucketing (`raw - 1`, real floor assumed to
+be 1) left shipped bucket 0 permanently unreachable -- a full
+`depth x dumb x gap` slice (1,452 of 21,780 shipped cells) that read the
+`0xFFFF` sentinel forever, not from insufficient collection but because
+no sample could ever land there. Fixed by shifting `raw - 2` instead (real
+floor is 2): raw 2-13 -> shipped 0-11, 14-21 -> 12, 22-31 -> 13, 32+ -> 14.
+Applied in `search.c`'s `_ShipLateBucket`, `dashboard.py`'s
+`ship_late_bucket`, and `joint_table_collection/bake_joint.py` (newly
+written -- imports `dashboard.py`'s bucket functions rather than
+redefining them). `data.c`'s `g_uJointFHThousandths` was rebaked from the
+2026-09-30 collection run with the fix in place: real coverage is
+18,775/21,780 (86.2%) populated, not the 17,444/21,780 (80.1%) the old
+bucketing would have reported on the same data. No data was discarded --
+this was a bucketing/reporting bug, not a population/gate mismatch (a
+separate concern raised and ruled out the same day, see below).
+
+# FH%-by-(depth, move, gap) table, EFP==LMP unification, and LMR as a
+# fallback -- brain dump, 2026-09-26
+
+This is a different, unrelated project from the `README.md`/
+`searchsup_GetLMRReduction.c`/`generate_counter_move_block.c` files
+already in this directory (that's the 2026-08-27 graded-LMR-curve +
+counter-move sweep, reverted to a clean baseline and stashed for
+possible later reuse). This write-up is about a same-week but
+independent effort: building a real, measured `P(fail-high)` surface
+and using it to drive EFP, and eventually LMR, instead of hand-picked
+constants or a hand-picked reduction curve.
+
+## The core idea
+
+EFP is a pruning method currently in use in the engine; we are
+considering amending it or replacing it.
+
+Historically, EFP (extended futility pruning) was Ernst Heinz's
+two-tier design: only active within ~2.75 ply of the qsearch horizon,
+gated by a hand-picked material margin (`VALUE_KNIGHT` at <=1.75 ply,
+`VALUE_ROOK` at <=2.75 ply), with a per-move `ComputeMoveScore(mv) <
+uFutilityMargin` test on top. LMP (late move pruning) didn't exist in
+this codebase at all; Crafty-style engines that have it usually key it
+on move number/depth alone, blind to how far below alpha the position
+actually looks.
+
+The thesis this session worked from: **EFP and LMP are answering the
+same underlying question** -- "is it safe to stop searching from here"
+-- just entering from different angles (EFP: "this move's static score
+already looks bad, gap-driven"; LMP: "we've tried enough moves,
+move-count-driven"). Once you have real data on `P(fail-high | depth,
+move, gap-to-alpha)`, there's no reason to maintain them as two
+separate hand-tuned mechanisms with two separate blind spots. One
+table, consulted from either direction, replaces both. See the
+conversation this file's SessionId corresponds to for the full
+back-and-forth that arrived at this; the short version is EFP was found
+to be nearly move-number-blind (a move at position 3 and a move at
+position 25 with the *same* gap get treated identically by the old
+margin test, despite real data showing a 6-12x difference in FH% between
+them at the same depth/gap), and that's what motivated building the
+joint table instead of patching EFP's old formula.
+
+## The data: what it is, exactly
+
+`f(remaining_depth, move_number, gap_to_alpha) -> P(fail-high)`,
+where:
+
+- **remaining_depth**: `uDepth / ONE_PLY`, whole plies, NOT capped at
+  the old EFP's 2-3 ply window -- deliberately extended, because the
+  data shows depth 1-2 is the *riskiest* part of the curve, not the
+  safest (FH% falls ~5x from depth 0-1 to depth 9-12, confirmed
+  independently holding move number fixed, so it's a real depth effect
+  not a move-mix artifact). EFP's old activation window covered exactly
+  the highest-risk region and left the safer, deeper region entirely
+  unpruned.
+- **move_number**: 1-based ordinal position in the move list. Also
+  measured as move-*fraction* (fraction of the move list exhausted,
+  20 buckets of 5%) as a parallel table, since "move 15 of 18" and
+  "move 15 of 54" are different signals and raw move count doesn't
+  capture that. The fraction table exists but the raw-move-number one
+  is what's actually wired into the live lookup right now.
+- **gap_to_alpha**: `iAlpha - iEval`, the NODE's static eval deficit
+  below alpha, in centipawns, bucketed 50cp wide (`FH_GAP_BUCKET_WIDTH`),
+  from -200cp up through +1400cp+ (`FH_GAP_BUCKETS = 32`, chess.h).
+  Deliberately the cheap, already-computed node-level gap (`iEval` is
+  computed for every node regardless of EFP) rather than a per-move gap
+  built on `ComputeMoveScore` -- the costlier per-move version was
+  considered and explicitly deferred, since computing it unconditionally
+  for every leftover move (to avoid the exact censoring bias described
+  below) would add real per-move cost everywhere, not just within EFP's
+  old narrow activation window.
+
+Of the three, **gap dominates by far** -- a ~28,000x swing in FH% from
+the worst to best gap bucket (measured on real data: 28.3% at -200..
+-150cp down to 0.001% at +1300..+1350cp), versus roughly 5-12x from
+depth or move number alone. But move number and depth both carry real,
+independent signal on top of gap -- confirmed by slicing FH% at a FIXED
+gap bucket and watching it still vary 4-12x across move number, and
+still vary ~5x across depth. All three matter; none alone is sufficient.
+
+## The population the data is measured over: "gate-1"
+
+This is the single most important methodological point, and got fixed
+late after an expensive lesson (see "the first real attempt" below).
+**The table must be trained on exactly the population that will
+actually reach it in real search, not on "all leftover moves."**
+
+Note: a "leftover move" is not a capture, killer or good countermove.
+They are the quiet moves and losing captures.  They are the tier of
+moves produced by generator.c with a score < GOOD_MOVE.
+
+If you train an FH-rate table on a broader population that includes
+checks, killer-adjacent moves, threat-node members, and normal
+captures -- categories that a real EFP/LMR gate would ALREADY exempt
+via cheap, separately-validated exemptions -- the resulting table is a
+systematic OVERESTIMATE of risk for the population that actually
+reaches it, because those excluded categories are (by design, since
+they earned their exemption from evidence) higher-FH-rate populations
+diluting the measurement.
+
+The "gate-1" eligibility checklist, applied both when COLLECTING the
+data and when CONSULTING the table live, is:
+
+- `fIsLeftoverMove` (sub-`GOOD_MOVE` -- not a winning/even capture,
+  killer, killer-mate, or sufficiently-evidenced countermove)
+- `!IS_CHECKING_MOVE(mv)`
+- `!IS_ESCAPING_CHECK(mv)`
+- `iExtend <= 0` (no extension already applied)
+- `!fThreat` (this node's own null-move probe didn't raise a threat
+  signal)
+- `(!IS_CAPTURE_OR_PROMOTION(mv)) || (iMoveSee < 0)` -- captures/
+  promotions excluded UNLESS losing (SEE < 0), which are admitted. This
+  was a deliberate widening from the old EFP's blanket capture
+  exemption: a losing capture is character-wise much closer to a bad
+  leftover move than a good one, and `iMoveSee` is already computed for
+  exactly this class of move at shallow depth (`uDepth < THREE_PLY`),
+  so admitting it costs nothing extra. Caveat: `iMoveSee` defaults to 0
+  (not a real SEE value, treated as "non-losing") for captures outside
+  that depth range where `GetMoveSee` never actually ran -- a
+  non-issue for the current depth-1-9 beachhead, but worth remembering
+  if the table's depth range is ever pushed further out without also
+  extending where SEE gets computed.
+
+The first data-collection pass (before this fix) trained on the
+unrestricted leftover population and showed real cells with FH% up to
+20-30% at aggressive-looking gap/move combos -- looked alarming, until
+the `fThisMoveEFPCandidate` audit mechanism (below) showed the ACTUAL
+population reaching a table lookup in real search had real FH% around
+0.1-0.15% at a "5.0% predicted" cutoff, a 30-50x gap entirely explained
+by the population mismatch, not a flaw in the gap/depth/move signal
+itself. Re-collecting on the gate-1-restricted population removed this
+bias; the retrained table's own audit numbers then matched its
+predictions almost exactly (within ~1-2%), confirmed by comparing real
+outcomes against the volume-weighted average PREDICTED value for
+exactly the flagged population -- NOT against the raw threshold used to
+flag it, which is a different, much more permissive number and an easy
+place to fool yourself (a threshold is a ceiling on what gets flagged,
+not the typical value of what actually does).
+
+## How the data is collected
+
+Three parallel counter tables live in `chess.h`'s `COUNTERS.tree`
+(mirrored into `g_Options` for suite-level reporting), all gated behind
+`#ifdef FH_TABLE_COLLECTION` (its own opt-in `gmake` flag, NOT part of
+the standard `PERF_COUNTERS` profile -- this instrumentation measurably
+costs NPS when active on every gate-1-eligible move, and there's no
+reason to pay that cost outside a dedicated collection run):
+
+- `u64LeftoverTriesByDepthMove` / `u64LeftoverFHByDepthMove` -- raw
+  move-number x depth, `[65][64]`.
+- `u64LeftoverTriesByDepthMoveFrac` / `...FHByDepthMoveFrac` -- move
+  FRACTION x depth, `[65][20]`. Watch out: `MOVE_COUNT(ctx, ctx->uPly)`
+  can be 0 when the hash move is tried before this ply's moves are
+  generated (a distinct search stage) -- skip the frac increment
+  entirely in that case, don't clamp/floor it. An earlier version used
+  a `MAXU(1, ...)` floor instead of skipping, which silently dumped
+  every such move into the "95-100% exhausted" bucket regardless of its
+  real position, corrupting it with a huge, spuriously-low-FH%
+  population -- caught by a smoke test where that one bucket had
+  20-100x the sample count of its neighbors. **Always smoke-test a new
+  counter design on a tiny run before trusting a big collection pass.**
+- `u64LeftoverTriesByDepthMoveGap` / `...FHByDepthMoveGap` -- the full
+  3-way table, `[65][64][32]`. This one's cell count is ~65x the other
+  two combined, and printing it in full (even just the compact
+  `FHGAPRAW` reassembly line, not a human-readable duplicate) produces
+  tens of thousands of lines per checkpoint on a real collection run --
+  budget log volume accordingly (this box's `/tmp` is a 30GB tmpfs;
+  don't let it fill during an unattended run).
+
+Reassembly across shards: each shard's raw output is greppable for a
+`FHRAW`/`FHFRACRAW`/`FHGAPRAW` line format
+(`TAG depth move [gap_bucket] fh_count tries`), one line per populated
+cell, printed at the end of a `script` run via `_PrintSuiteReport`
+(`script.c`). **Periodic checkpointing matters for anything that runs
+long enough to risk a timeout**: `ScriptCommand`'s position loop
+re-prints the full (cumulative, not delta) report every
+`CHECKPOINT_INTERVAL` (currently 5) positions, not just once at the
+true end -- found the hard way, the first version of this only printed
+once at the end and a shard that missed its `timeout` budget by 0.05
+seconds lost 100% of its data. **When reassembling from checkpointed
+output, take the LAST checkpoint block in a shard's log, never sum
+every occurrence** -- each checkpoint is a full cumulative snapshot,
+not a delta; summing multiple checkpoints double- (or 18x-) counts.
+
+A second, unrelated gotcha in the checkpoint counter itself: it used to
+key off `g_SuiteCounters.uTotal`, which is gated by `WeAreRunningASuite()`
+-- true only when the current position has a recorded `solution`/`avoid`
+move. Plain `setboard`/`go` position pools (like
+`cross_sectional_200k.ep_`, no EPD solution annotations) never
+increment `uTotal` at all, so the checkpoint silently never fired on
+that file format despite the code looking correct. Fixed with a
+separate, unconditionally-incrementing `uPositionsProcessed` counter.
+**If a checkpoint/report mechanism silently does nothing on a specific
+file format, suspect a hidden dependency on EPD solution metadata.**
+
+Position source used so far: `tests/cross_sectional_200k.ep_` -- per
+`TESTING_PLAN.md`, this pool's actual documented purpose is
+`fuzz_test.sh`'s crash-fuzzing corpus, not curated for FH-rate
+calibration specifically. No generation script for it exists anywhere
+in the repo (checked); its contents look like ordinary, plausible
+real-game positions on inspection, but that's "looks fine," not
+"verified representative" -- worth keeping in mind, and see "the
+tactical-position gap" below for why this specific pool's composition
+turned out to matter a lot.
+
+## The upcoming large sharded run
+
+Plan: run this collection sharded across many single-threaded
+(`--cpus 1` -- `split.c`'s parallel helpers are NOT wired into these
+counters, deliberately, so multi-threaded runs would silently under-
+count) processes, at real depth (the whole point of collecting deep-
+remaining-depth cells is that they're thin unless the root search goes
+substantially PAST the depth being measured -- a node with remaining
+depth R sits at ply ~(root_depth - R) from the root, and node count
+grows with ply-from-root, so `sd 6` can never produce a single
+remaining-depth-10 sample; you need `sd 16+` to make that shell
+non-trivially populated), for on the order of a week, against a large
+position sample.
+
+**Composition of that sample matters and was the subject of a real,
+expensive lesson this session (see "the tactical-position gap" below):
+mostly real-game cross-sectional positions, with a MODEST tactical
+fraction mixed in -- not a puzzle-heavy pool.** Scott's explicit framing:
+"we want an engine that plays chess, not solves tricky puzzles." A
+pool dominated by tactical puzzles would teach the table that leftover
+moves in general are riskier than they really are (puzzles are
+adversarially selected to have a non-obvious best move), producing a
+table that under-prunes in the ordinary quiet middlegame positions
+that make up most of a real game. Something like a 80/20 or 90/10
+real-game/tactical blend is a reasonable starting point to test, not a
+locked-in ratio.
+
+Practical constraints learned from the smaller collection runs already
+done:
+- Per-position cost at real depth varies enormously (34s to 300+s
+  observed at `sd 14-16` on different positions, single-threaded) --
+  don't budget wall-clock by assuming uniform per-position cost.
+  Feeding each shard a generous position POOL (more than it's expected
+  to finish) and bounding by wall-clock `timeout` rather than position
+  count means the run naturally uses whatever time is available without
+  needing to guess position count in advance.
+- The `/tmp` tmpfs on this box is shared and has been observed at 99%
+  full from OTHER processes mid-session (not our own files) -- don't
+  assume a `du`-based capacity check at the start of a run is still
+  valid hours later.
+- A stray, unrelated process (an interactive `xboard -fcp ./typhoon
+  -scp crafty` game, apparently left running from earlier work) has
+  shown up sharing the box's cores during a collection run. Worth a
+  `ps aux | grep typhoon` sanity check before assuming all available
+  cores are actually free.
+
+## What the data is FOR: EFP == LMP, LMR as a lower-risk fallback
+
+Once gate-1 is passed, the joint table gives a real `P(fail-high)`
+estimate. Two consumers, deliberately sequenced -- prune first (since
+it was the smaller, more contained change), reduction later:
+
+**EFP/LMP (unified)**: skip the move outright if predicted FH% is below
+some `epsilon_prune`. Irrecoverable, so `epsilon_prune` needs to be very
+conservative and, critically, should be picked from a real measured
+precision/aggressiveness curve (sweep several candidate thresholds
+through the SAME audit mechanism used to validate the table, plot real
+FH% and candidate volume against each), not guessed. Runtime-tunable
+via `set EFPCandidateThresholdTenths <tenths-of-a-percent>` specifically
+so this sweep doesn't require a rebuild per value.
+
+**LMR (future work, not yet touched this session beyond discussion)**:
+the SAME table, but as a reduction-amount driver instead of a binary
+skip -- `reduction_plies ~ f(-ln(predicted FH%))`, monotonically more
+aggressive the safer the cell looks. The key asymmetry that justifies a
+MORE permissive threshold for reduction than for pruning: a reduction
+is recoverable in (at least) the direction we can detect -- the
+existing "research deeper if history pruning failed" path already
+re-verifies a reduced move whose shallow search unexpectedly raises
+alpha. What it does NOT catch is the other direction: a reduced search
+that fails low, silently accepted, with nobody ever checking whether
+the FULL-depth search would have found a raise there instead. Built an
+audit for exactly this (`LMR_AUDIT` build flag, its own opt-in profile
+since it costs a real extra `Search()` call per audited candidate, not
+just a counter increment) -- for a move that got reduced and then
+failed low at the reduced depth, re-run the SAME scout window at full
+(reduction-undone) depth and check if THAT raises alpha. First real
+numbers (small sample): miss rate 4-10% at remaining depth 2, dropping
+under 1% at depth 3-4, then creeping back up to 3-7% at depth 5-6 -- a
+non-monotonic, "high-low-high" shape that wasn't the expected smooth
+decay and hasn't been investigated further yet.
+
+Both consumers, plus per-move veto signals (below), sit on top of the
+same gate-1-gated table lookup -- this is the concrete sense in which
+EFP and LMP "are the same thing now."
+
+## Per-move veto signals: two tried, both hit cliffs, no smooth middle ground
+
+The idea: once the table flags a move as a pruning candidate, consult a
+cheaper-or-not-much-more-expensive per-move signal as a final veto,
+since the table only knows the aggregate risk for this (depth, move,
+gap) CELL, not anything about this specific move's own tactical merit.
+Both signals tried this session are audit-only-first, real-pruning-
+second, and both turned out to be effectively binary rather than a
+smooth dial:
+
+1. **`ComputeMoveScore` veto, Heinz-style depth-scaled margin.**
+   Generalized the old two-tier `VALUE_KNIGHT`(1.75ply)/`VALUE_ROOK`
+   (2.75ply) activation margin to a linear scale across the whole
+   depth-1-9 range (`uEFPVetoMarginBase + (depth-1) *
+   uEFPVetoMarginPerPly`, both runtime-tunable). Swept the margin from
+   0 up to 300/200: below ~(200-300, 100-150) nothing gets rescued at
+   all; at (300, 200) every one of 5 known-loss positions gets rescued,
+   but node count on `ecm_ringers` comes back to within 0.6% of the
+   ORIGINAL unpruned baseline -- i.e. that setting is functionally "never
+   prune," not a real middle ground. **Retired this veto** in favor of
+   (2) below; not worth resurrecting without a fundamentally different
+   margin shape (this was linear-in-depth; something with real curvature
+   might behave differently, untested).
+
+2. **`GetMoveFailHighPercentage` veto** (per-move-IDENTITY history,
+   `dynamic.c`'s `g_FailHighs` table, keyed by `mv.uMove & 0xFFFFF` --
+   near move-exact, not just piece+destination). Exempt a flagged
+   candidate from pruning if it has real, evidenced history of failing
+   high: `(uFHAttempts >= MinSamples) && (uFHPct > Threshold)`.
+   **Critical, easy-to-miss fact about this table's lifecycle**: every
+   cell resets to `0x00010001` (1 attempt, 1 fail-high = a 100% FH-rate
+   PRIOR), not zero, on `ClearDynamicMoveOrdering()` -- which fires on
+   `setboard`/`new` (`PreGameReset`'s callers), i.e. **once per position
+   in a suite run, but only once per GAME in real play, never between
+   moves.** At the most permissive setting tested (`MinSamples=1,
+   Threshold=0`), the veto protects EVERY move by default until real
+   accumulated failures (only `_RecordMoveFailure` ever touches these
+   cells during a normal search; it increments the attempts denominator
+   without touching the fail-high numerator, diluting the percentage
+   toward 0 -- reaching an integer-truncated 0% takes on the order of
+   100+ observed failures of that exact move-identity within the
+   current search) prove a specific move safe. This is NOT "smart
+   triage using real evidence" so much as "protect nearly everything
+   by default, until proven otherwise" -- confirmed by measuring the
+   actual prune rate among flagged candidates: **70-80% of everything
+   the table flags gets vetoed and fully searched anyway**, at every
+   depth checked. That's a real, quantified NPS cost for zero node
+   savings on the majority of "candidate" evaluations.
+   Swept `(MinSamples, Threshold)` away from the extreme (1,0) setting
+   -- same cliff pattern as (1): every tested combo away from (1,0)
+   lost 4-5 of 5 known-loss rescues while only partially reducing the
+   waste rate; none beat "just tighten the table threshold with no veto
+   at all" (which alone recovers exactly 1 of 5, `ECM.272`, the one
+   case that was ever a pure threshold-calibration problem rather than
+   a population/data problem).
+
+**Open question, not yet resolved**: does this veto's real-game behavior
+degrade over the COURSE of a single game, given the table never resets
+mid-game? Live self-play match data shows the candidate's average-
+search-depth advantage over the unmodified baseline shrinking as the
+match's game count grows (within one fixed-configuration match:
++1.09 ply at 30 games, +0.57 at 60, +0.49 at 300) -- consistent with,
+though not yet proven to be caused by, staleness accumulating within
+individual games as they get longer. Needs a direct test: instrument
+depth-gain (or the candidate/actually-pruned ratio) as a function of
+MOVE NUMBER WITHIN a game, not just averaged across whole games, using
+actual game logs from a running match rather than inferring it from the
+aggregate trend.
+
+## `sd` vs `sn` vs real games -- and getting this backwards cost real time
+
+**`sd N` (fixed depth) is structurally the WRONG test for anything that
+trades tree size for depth.** At equal depth, a leaner-tree change can
+only ever cost solves relative to baseline -- it never gets credited
+for the depth it's supposed to buy, because the test held depth fixed
+instead of letting the efficiency convert into extra plies. This
+session ran an expensive round of `sd10` suite comparisons that showed
+what looked like a severe regression (14 net lost solutions across two
+curated suites) before catching that the test itself was the problem,
+not (only) the mechanism: re-run at `sn 5000000` (fixed node budget,
+the fair comparison, since it lets the efficiency actually convert into
+depth the way it's supposed to), the SAME positions and SAME build
+showed net +6 solved across all three curated suites with deeper
+average search depth everywhere. **Any test of a pruning/reduction
+change needs to specify which of these three it's using and why**:
+`sd` for "does this change the tree SHAPE at fixed effort" (the right
+question for judging LMR/EFP's aggressiveness in isolation), `sn` for
+"does this deliver more accuracy at fixed COST" (the right question
+for judging whether a leaner-tree change is worth it), real self-play
+games for "does this actually win more, accounting for everything
+neither synthetic test can capture" (time-control-driven node budgets
+that vary naturally by position, and the compounding effect of
+thousands of small decisions across a whole game rather than a handful
+of curated positions).
+
+Related, and worth its own callout: **curated ECM suites have real,
+documented noise** (see `tests/ECM.md`'s own "flaky/garbage" catalog --
+oscillating PVs, near-0.00 scores, single-line coin-flips). A ±1-2
+solve swing on a 90-position suite, or any result on the 11-position
+`ecm_ringers`, is not strong evidence on its own. Chasing a specific
+named position's recovery (the 5-position known-loss set used
+throughout this session's veto sweeps) can actively mislead: the
+0.5%-threshold configuration that achieved a PERFECT 5/5 recovery on
+that named set actually performed WORSE in full-suite aggregate (net
++2) than the 1.0%-threshold configuration it was being compared against
+(net +6) -- optimizing against a small, hand-picked set pulled toward a
+setting that was worse on the broader, still-imperfect-but-larger
+sample. The named positions are useful as a fast smoke test for "did
+this obviously break something we already know about," not as an
+optimization target.
+
+## The tactical-position gap: why this matters more than it first looked
+
+The first live-pruning attempt (table threshold 1.0%, no veto) showed a
+real, `ECM.md`-confirmed regression: legitimate tactics lost, including
+`ECM.272` -- a position independently, manually re-verified from scratch
+this same session (traced the exact sacrifice line, confirmed our own
+engine solves it cleanly and stably from depth 7, confirmed by an
+independent Crafty search converging to the same move) as a genuine,
+non-flaky tactic. Root cause, confirmed by a threshold sweep specific to
+the 5 known-loss positions: tightening the table threshold all the way
+to 0.1% (10x tighter than the default that failed) rescued exactly 1 of
+5 (`ECM.272`); the other 4 stayed wrong regardless. **That's the tell
+that it's a population/training-data problem, not a threshold-
+calibration problem** -- if tightening 10x doesn't move 4 of 5 cases at
+all, the table's own predicted value for whatever cell is involved is
+wrong, not just permissively thresholded. `cross_sectional_200k.ep_`
+(real games only) apparently underrepresents the "quiet-looking but
+critical" move shape tactical positions are specifically curated to
+contain, enough that the table confidently mispredicts it. This is the
+direct motivation for mixing a modest tactical fraction into the next,
+larger collection pass (see above) -- not to make the engine solve
+puzzles better, but because the training population needs to actually
+contain examples of the shape it's being asked to judge.
+
+## Status as of this write-up
+
+- Real pruning is OFF pending resolution of the open questions above
+  (in-game staleness, and the not-yet-attempted ECM-augmented
+  retraining). What's live right now (via `EFPCandidateThresholdTenths`/
+  `EFPFHVetoMinSamples`/`EFPFHVetoThreshold`, all runtime-settable) is
+  the most conservative configuration tested (0.1% table threshold,
+  `(1, 0)` veto), currently running as `candidate` in an ongoing SPRT-
+  style self-play match against unmodified `head` -- last checkpoint:
+  300 games, score 0.473, llr -0.50 (inconclusive, hasn't crossed either
+  bound; trending toward parity after an unfavorable dip at 60 games
+  that looks in hindsight like small-sample noise, consistent with the
+  "ECM suites are noisy, don't over-read small samples" lesson above).
+- `FH_TABLE_COLLECTION` build flag exists and works; a real, week-long,
+  larger, tactically-augmented collection run is the planned next major
+  step, informed by everything above.
+- The `ComputeMoveScore` veto is retired (dead end, see above); the
+  `GetMoveFailHighPercentage` veto's mid-game staleness question is
+  open and needs direct instrumentation, not just inference from match
+  trends.
+- LMR (reduction) work has not actually started yet beyond the
+  `LMR_AUDIT` miss-rate measurement infrastructure and this discussion
+  of how the risk threshold should differ from pruning's -- next in
+  line once the pruning side's open questions are resolved.
+
+---
+
+# Update, 2026-09-27/28: table-driven EFP goes fully live, two real
+# bugs found and fixed, self-play verdict is "roughly even"
+
+Continuation of the above, same overall project. This section covers
+what changed from "status as of 2026-09-26" (real pruning OFF,
+conservative audit-only config) through making it genuinely live,
+finding and fixing two real bugs via self-play (not suites), and the
+honest current verdict.
+
+## Old EFP fully replaced, not unioned
+
+Early in this stretch, discovered the old margin-based EFP block and
+the new table-driven block were BOTH live simultaneously (unioned via
+a shared `fThisMoveEFPPruned` flag) -- an artifact of iterating on the
+veto experiment without reconciling the two blocks. Fixed: old EFP
+wrapped in `if (FALSE && ...)`, permanently disabled; the table-driven
+mechanism is now tested as a genuine replacement, not an addition.
+
+Also added, matching old EFP's own behavior that the table-driven
+block had been missing: explicit `NODE_PV != pi->eNodeType` exclusion,
+and the losing-capture admission switched from a depth-gated local
+`iMoveSee` (only computed for `uDepth < THREE_PLY`, silently
+defaulting to "non-losing" outside that range) to the cached
+`MVF_SEE_KNOWN`/`MVF_SEE_NONNEGATIVE` move-stack flags `generate.c`
+already sets unconditionally at generation time for every capture/
+promotion -- correct at every depth, not just near the horizon.
+
+**Also discovered `ComputeMoveScore` is a no-op for quiet/leftover
+moves** under `MOVE_SCORING_NORMAL` (always returns 0) -- meaning old
+EFP's real selectivity, historically, came ONLY from its coarse
+depth/margin gate plus the killer/FH-history exemptions, never from
+any actual per-move tactical read. Worth remembering next time
+something is attributed to "EFP's margin test was smart about this
+move" -- it wasn't; it was smart about depth alone.
+
+## Depth-scaled threshold, not a flat one
+
+A flat FH% threshold applied uniformly across the whole prune-eligible
+depth range is structurally wrong: old EFP only ever operated within
+~1-2.75 ply of the horizon, where a wrong prune costs little (a small,
+bounded subtree). This mechanism's point is to extend pruning deeper,
+where a wrong prune is far more expensive (an exponentially larger
+subtree). `_EFPThresholdForDepth()` (search.c) interpolates
+GEOMETRICALLY (not linearly -- the two endpoints span orders of
+magnitude) between `uEFPThresholdShallowHundredths` (depth 1,
+permissive -- old EFP's own real gate tolerated FH% up to the tens of
+percent at 1-2 ply and still won) and `uEFPThresholdDeepHundredths`
+(at `uEFPMaxPruneDepth`, near the table's measurement floor).
+
+## `MoveLooksDumb`: from boolean, to magnitude bucket, to the joint table
+
+A new 4th signal, alongside depth/move/gap: does this quiet move
+statically look like it hangs material? Went through several rounds:
+
+1. **First attempt**: three OR'd sub-checks (a "reversal" move, a
+   pawn-attack, an ignored-hanging/trapped-piece read). Isolation
+   testing found only the pawn-attack idea behaves as intuition
+   predicts. The other two were retired for real, understood reasons,
+   not just "didn't help": the reversal check had a genuine indexing
+   bug (compared against the WRONG ply's move, `uPly-2` instead of
+   `uPly-3`) that masked a deeper conceptual problem once fixed --
+   reversal moves measure BACKWARDS (higher FH%, not lower), because a
+   reversal responds to the opponent's own intervening move and is
+   often a genuine correct reaction, not a wasted tempo. The
+   hanging/trapped-piece check also measured backwards: `cEnprise`/
+   `cTrapped` are written by the same parent `Eval()` call that
+   produces `iEval` (the gap axis's own input) -- a real confound with
+   gap, not independent signal.
+2. **Second attempt** (kept, still live): "does this move hand
+   material away for no reason" -- unconditionally for a strictly
+   cheaper attacker (pawn attacks any non-pawn; minor attacks a rook;
+   minor or rook attacks a queen), and conditionally for a
+   queen-vs-queen "undefended" case discussed and then REMOVED below.
+   Tried and retired: an equal-value-attacker-AND-zero-defenders test
+   for minors/rooks -- structurally broken, because
+   `pos->bbMinorAttacks[uMover]`/`bbRookAttacks[uMover]` (computed
+   PRE-move) already includes the mover's own pre-move reach to its
+   destination square (that's what makes the path legal), so "zero
+   defenders" was almost never true with multiple minors/rooks per
+   side and no cheap way to tell "the piece that just left" from "a
+   different piece of the same type still defending." The QUEEN case
+   sidesteps this because there's normally exactly one queen per side
+   (gated on `FastCountBits(bbPieces[uMover][QUEEN]) == 1`).
+3. **Boolean -> magnitude bucket**: "dropping a bishop to a pawn" and
+   "dropping a queen to a pawn" were both just `TRUE` before, throwing
+   away real information (a ~28,000x FH% swing across gap buckets
+   alone says magnitude matters). `MoveLooksDumbLossCp()` now returns
+   an approximate centipawn loss; `MoveLooksDumbLossBucket()`
+   coarsens it into `DUMB_LOSS_BUCKETS` (chess.h, =6) piece-shaped
+   tiers.
+4. **A real UnmakeMove-ordering bug**, caught before it shipped: the
+   FH-side counter increment originally called `MoveLooksDumbLossCp`
+   a SECOND time, after `UnmakeMove` had already restored the board
+   and flipped side-to-move -- silently classifying a different
+   position. Produced an impossible >100%-FH cell in testing. Fixed by
+   computing the dumb bucket ONCE (before `UnmakeMove`) and reusing
+   the stored value on both the tries-side and FH-side counters.
+5. **Joint (4D) table**: built `(depth, move, gap, dumb)` jointly, not
+   just as two separate marginal cuts, to test whether "dumb"
+   compounds with "gap" rather than being redundant. Confirmed real
+   compounding (e.g. depth 3, gap bucket ~4: dumb=0 -> 0.948% FH,
+   dumb=4 (post-coarsening numbering) -> 0.008% FH, a ~120x reduction
+   at 3.2M samples) -- this justified keeping dumb as a real 4th axis
+   instead of folding it into gap.
+
+### Queen-vs-queen removed outright (2026-09-27, this session's tail)
+
+The queen-vs-queen-undefended sub-check (originally bucket 5, ~975cp
+loss) was the one case that never behaved like the rest. Direct
+analysis of the 45-minute joint-run data: marginal FH% by dumb bucket
+was 0.057%/0.072%/0.098%/0.010% for buckets 1-4, but **0.457% for
+bucket 5 -- higher than "not dumb" (0.335%) baseline**, and this
+persisted (0.0147% vs. 0.0195% baseline, still elevated relative to
+bucket 4's 0.0045%) even after controlling for the gap axis. Root
+cause suspected but NOT confirmed: `_WhoAttacksSquareBB` (see.c) has
+no pin/legality awareness, so a pinned enemy queen still "shows" as
+attacking. Rather than ship a signal nobody understands, **the
+queen-vs-queen check was deleted from `MoveLooksDumbLossCp` entirely**
+(search.c), not just excluded downstream -- "losing a rook to a pawn
+is never a good idea" is a claim we can defend; "queen attacked by
+queen looks safe to prune" turned out not to be, for reasons still
+unexplained. `MoveLooksDumbLossBucket()` still has 5 tiers (0=not
+dumb, 1-4 = increasing loss magnitude: ~200/~400-475/~675/~875cp);
+bucket 5 (975cp) is simply unreachable now. `DUMB_LOSS_BUCKETS` stays
+at 6 in chess.h so the baked tables (sized `[...][6]`) don't need
+reshaping -- the top slot is just always zero.
+
+**A bug caught during this edit, worth remembering**: the first pass
+at simplifying `MoveLooksDumbLossBucket()` collapsed the 675cp and
+875cp tiers into one bucket, based on an incorrect assumption that
+removing the queen-vs-queen check also removed the 875cp value. It
+didn't -- 875cp (a queen attacked by a pawn) is still produced by the
+generic, unconditional pawn-attack branch at the top of the function,
+which was never touched. Caught by rereading the actual reachable
+value set before trusting the "simplification."
+
+## Real pruning: Tier 1 / Tier 2, then unified into one mechanism
+
+Two designs were tried, in order:
+
+**Tier 1 + Tier 2 (retired)**: Tier 1 = percentage-curve pruning,
+capped at `uEFPMaxPruneDepth`. Tier 2 = `JointMoveNeverFailsHigh()`
+(data.c) -- prune unconditionally at ANY remaining depth when the 4D
+joint table shows a confirmed zero fail-high over a large-enough
+sample (rule-of-three statistical confidence), EXCLUDING dumb buckets
+0 (not dumb -- this table says nothing about ordinary moves) and 5
+(queen-vs-queen, even before it was removed from the signal entirely).
+Audited before trusting: found real misses at depth 3-4 even at
+`MinSamples=10000` -- one exact miss traced to a genuine, deliberate
+queen sacrifice (`4rrk1/6p1/p7/1p1Np3/1bq1P3/7Q/PPP5/1K1R2R1 w - - 3 0`,
+`Qh6`) near the enemy king, not a bug. Added `Tier2MinDepth=5` after
+confirming clean audits at depth >=5 across all three curated suites.
+
+**Unified (current, live)**: per explicit direction ("I think we
+should not have tier1 and tier2 ... prune everywhere again but use a
+graduated threshold" -- then, when the deep end of the percentage
+curve flatlined at its representable floor, "I don't want 0.01%. I
+want 0 FHs ever seen with >5k positions, >10k positions, etc"):
+
+```
+uCandDepth = uDepth / ONE_PLY
+if (uCandDepth < g_Options.uEFPMaxPruneDepth):
+    prune if PredictedFHPercentHundredths(...) < _EFPThresholdForDepth(uCandDepth)
+else:
+    prune if ConfirmedZeroFailHigh(uCandDepth, uMoveNum, gap, dumbBucket)
+```
+
+`ConfirmedZeroFailHigh()` (data.c) requires a literal zero observed
+fail-high over a sample count that DOUBLES each ply past
+`uEFPMaxPruneDepth` (base `g_Options.uZeroConfBaseSamples`, default
+5000 at the boundary depth, 10000 the next ply, 20000 the next, ...).
+**As of 2026-09-27 this reads the same 4D joint table Tier 2 used**
+(`g_uJointTries`/`g_uJointFH`), not a separate gap-only marginal --
+see "the self-play regression and its real cause" below for why that
+change was necessary.
+
+## The self-play regression and its real cause
+
+First live self-play run of the unified design (30 games): **-8=21+1,
+score=0.383, llr=-0.42** trending firmly toward reject, with `ebf`
+WORSE for candidate (3.590 vs head 3.522) and candidate searching
+SHALLOWER (12.87 vs 12.92 avg depth) at similar nps -- exactly
+backwards from what a working pruning mechanism should show. This was
+"clearly a bug, not just an aggressive tradeoff" territory, matching
+Scott's own read: "I think there's a bug in the pruning impl."
+
+An advisor consultation (mid-investigation) proposed a structural
+theory -- pruning removes move-ordering/TT information that's
+load-bearing elsewhere in the tree, and this cost scales with how deep
+the prune fires, which old EFP never paid because it stayed within
+1-2.75 ply of the horizon. That's a real, general risk (see "on
+reduction vs. pruning" below), but a direct code audit found something
+more concrete and immediately actionable: **`ConfirmedZeroFailHigh`
+was reading `g_uGapRawTries`/`g_uGapRawFH` -- the (depth, move, gap)
+marginal SUMMED OVER THE DUMB AXIS.** A cell could read "zero FH in
+5000+ tries" purely because ordinary, non-dumb moves dominated that
+cell's volume, while a rare queen-hanging move sharing the exact same
+(depth, move, gap) cell had never actually been safely tested on its
+own. Fixed by switching `ConfirmedZeroFailHigh` onto the 4D joint
+table (adding a `uDumbBucket` parameter, same table
+`JointMoveNeverFailsHigh` already used), refusing to trust a zero read
+for the queen-vs-queen bucket outright (matching Tier 2's own
+caution, at the time this bucket still existed).
+
+**Result, same match restarted after the fix (60 games total, 30 new
+under the fixed binary)**: ebf flipped from "candidate worse" to
+EXACTLY TIED (3.619 = 3.619), and average depth flipped from
+"candidate shallower" to "candidate deeper" (13.05 vs 12.81) -- the
+signature of a real fix, not noise. `g_uGapRawTries`/`FH` and the
+`GAPRAW_*` macros are now fully dead code, left in place (unused) at
+the time of this write-up but candidates for deletion.
+
+**The `g_uGapRawTries`/`g_uGapRawFH` tables were later deleted
+entirely** (2026-09-27, same session) once the gap-bucket rework below
+made keeping them around pointless.
+
+## Quantifying "how much statistical confidence is actually needed"
+
+A real question raised directly: given a per-cell "confirmed zero"
+read only bounds the true rate via rule-of-three (~3/N for N samples),
+and this gate fires many times per real game, how much does that
+matter? Worked from real counters, not guesses:
+
+- Real `sd10` counters (post-fix): ~303 actual deep-zone (depth >=6)
+  prunes per 31.3M nodes searched, i.e. ~1 per 103,000 nodes.
+- Scaled to a real blitz budget (1.5M nps x 350s ~= 500M nodes/game):
+  **~4,800 deep-zone prunes per game.**
+- Rule-of-three bound at the current sample bar (5000 at depth 6,
+  doubling deeper): expected WORST-CASE misses per game ~= 2 (not
+  thousands -- an earlier back-of-envelope estimate using a wrong,
+  much larger node-count assumption was corrected in-session).
+- To hit a "1 unlucky game in 1000" target at that per-game
+  application volume: `required_N ~= 3 * applications_per_game /
+  target_misses_per_game ~= 3 * 4800 / 0.001 ~= 14.4 million` samples
+  in the specific cell being trusted, not 5,000-40,000.
+- **Brute-force collection to reach that is not viable**: the
+  45-minute run put ~5,000 samples into a representative deep cell:
+  getting to 15M is ~3000x more volume, i.e. ~94 days wall-clock even
+  running all 16 shards this box supports in parallel (already at the
+  ceiling -- more shards doesn't help past core count). Confirmed and
+  accepted as a dead end in-session.
+
+**2026-10-01 correction: the "1 unlucky game in 1000" target itself was
+never validated, and the whole framing here was too conservative.**
+Scott's direct pushback: nobody decided 1/1000 was the right number --
+it was picked, not derived -- and 1/100 or even 1/20 is reasonable. Two
+things compound this:
+
+1. **Not every prune mistake costs the game.** A silent miss (we prune
+   a line that would have fail-highed) only matters if that line
+   actually would have changed the game's outcome -- a strict subset of
+   "a miss occurred." The rule-of-three math above bounds the rate of
+   MISSES, not the rate of GAMES ACTUALLY LOST to a miss, so even the
+   loosened budgets below are still conservative relative to the thing
+   that actually matters (Elo).
+2. **The deeper problem: this section was optimizing the wrong
+   quantity.** It treats "minimize miss probability until some arbitrary
+   confidence bar" as the goal in itself. The actual goal is net playing
+   strength: aggregate nodes saved by pruning/reducing get reinvested as
+   extra search depth elsewhere, and depth finds tactics -- that's a real
+   benefit that has to be weighed against the real (but apparently
+   small, see "the self-play regression" and the LMR self-play match
+   above) cost of occasional misses. A `g_uJointFHThousandths`-driven
+   gate exists precisely so this tradeoff can be made *intelligently*
+   (deeper/leaner where the data says it's safe-ish) rather than via a
+   single global, hand-picked confidence bar -- but picking that bar by
+   "make misses astronomically rare" short-circuits the whole exercise
+   and defaults back to maximal conservatism regardless of what the
+   table actually knows. The open comparison against Crafty (much
+   leaner trees, routinely 7-8 plies deeper at equal node budgets) is
+   the concrete signal that this project has been on the over-cautious
+   side of that tradeoff.
+
+Recomputed the same formula at looser budgets, same 4,800
+applications/game assumption:
+
+```
+1/1000 games -> N ~= 14.4M   (0 deep-zone zero-FH cells clear this, as of this run)
+1/100  games -> N ~= 1.44M   (0 deep-zone zero-FH cells clear this -- largest is 1.34M, close)
+1/20   games -> N ~= 288K    (16 deep-zone zero-FH cells clear this TODAY, no more collection needed)
+```
+
+**Decided direction, not yet executed**: stop treating the
+rule-of-three sample bar as the primary lever at all. Keep it as a
+floor against outright garbage (near-zero-sample cells), but the real
+experiment going forward is to try materially more aggressive
+settings on both mechanisms -- the 1/20-budget prune bar (288K, usable
+immediately) and reduce tiers picked to target real extra depth (aim
+for whole plies, not the +0.3 ply the first p75/p25/p5 self-play run
+produced) -- and read the result by playing strength and search depth
+in self-play, not by how rare a miss is in isolation. The right
+long-term framing: **the actual metric to tune against is net Elo
+(or a cheap proxy -- depth/nps/ebf movement under a fixed node budget),
+with the FH% table used to bias WHERE cuts happen, not to justify an
+arbitrarily tiny global miss rate.**
+- **What DOES help, cheaply**: coarsening the bucket grid concentrates
+  the same collected volume into fewer, denser cells -- see next
+  section. This buys real headroom without any extra collection time.
+
+## Open gap, 2026-10-01: compounding errors under reinvested depth is unmeasured and unmeasurable by anything built so far
+
+Every statistic gathered for this project so far -- raw FH%, the
+`JOINT_EXTRA_VERIFY` silent-miss rate, the rule-of-three sample bound
+above -- is a SINGLE-DECISION statistic: it characterizes the chance
+that one gate, at one node, misjudges. None of it says anything about
+what happens once that misjudgment is embedded in a continuation that
+is then searched DEEPER, specifically because the nodes saved by
+pruning/reducing elsewhere get reinvested as exactly that extra depth.
+
+Scott's framing: we have no information about what happens if we prune
+wrong ~5x/game and reduce wrong such that a reduced-depth search
+fails low where a full-depth search would have failed high, AND in
+doing so we bought ourselves ~2 extra plies per move on average --
+because searching those 2 extra plies means the SAME gate fires again,
+2 more times, deeper under the same kind of position, with more real
+tree beneath each repetition.
+
+**Why this could be qualitatively worse than the per-decision rates
+suggest, not just "the same risk, more often":** if the 4-axis model's
+misjudgments were i.i.d. noise, compounding wouldn't be very dangerous
+-- a short chain of independently low-probability errors is still
+low-probability, same as any noise process. But this project has
+already found the model has CORRELATED, structured blind spots, not
+just noise: the queen-vs-queen dumb-bucket (removed outright, see
+above -- "safe" reads that weren't, for reasons still not fully
+understood) and the null-move-bailout signature bug (silent-miss
+investigation, see above) were both systematic, not random. Reinvesting
+saved nodes as depth specifically UNDERNEATH the branches just judged
+"safe" means that if a blind spot like those exists, extra depth
+doesn't dilute it with more search -- it drills further into exactly
+the part of the tree most likely to share it.
+
+**This cannot be measured by instrumentation built so far, even in
+principle.** `JOINT_EXTRA_VERIFY`'s silent-miss rate and the rule-of-
+three bound both describe P(one gate misfires once); there is no way
+to decompose an aggregate self-play score back into "cost from
+compounding errors under reinvested depth" vs. "benefit from genuinely
+sound extra depth" -- they are inseparably confounded in that single
+number by construction. Per-node instrumentation was always going to
+be structurally blind to this, no matter how much more data is
+collected at the per-cell level.
+
+**Practical implication for the next (more aggressive) self-play run**:
+don't just read the final SPRT score. Pull the actual lost/drawn games
+under the aggressive settings and check whether losses cluster around
+a recognizable, repeated position type (a live blind spot, same shape
+as queen-vs-queen) versus being spread uniformly (ordinary noise at a
+higher-variance operating point). `SILENT_MISS_TRACE`'s existing
+FEN/move/score logging is most of the needed infrastructure already --
+it would need to stay attached through to which game a flagged miss
+occurred in (not just logged as an isolated per-node event) to make
+this check possible. Not yet built; flagging as the concrete next
+instrumentation gap before trusting a bolder self-play result as
+"just noise" either way.
+
+## Gap-bucket redesign: 32 uniform bins -> 6 piece-value-aligned bins
+
+Per direct instruction ("Why do we have 32 gap buckets?! I think they
+should be like: alpha-100, alpha-300, alpha-500, alpha-900,
+alpha-1500, worse than that"), `FH_GAP_BUCKETS` (chess.h) went from
+32 uniform 50cp-wide bins (`FH_GAP_BUCKET_MIN=-200`,
+`FH_GAP_BUCKET_WIDTH=50`) down to 6:
+
+```
+bucket 0 : gap <  100
+bucket 1 : 100 <= gap <  300
+bucket 2 : 300 <= gap <  500
+bucket 3 : 500 <= gap <  900
+bucket 4 : 900 <= gap < 1500
+bucket 5 : gap >= 1500
+```
+
+Rationale: the fine 50cp resolution was mostly wasted precision in the
+near-alpha region (highest FH%, 9%+ -- never going to pass any sane
+pruning threshold regardless of exactly which of ~6 fine bins it lands
+in) and starved precision in the tail, which is exactly where a
+pruning DECISION actually lives. A 32->6 collapse is a ~5.3x reduction
+in total cell count, i.e. a ~5.3x increase in effective samples per
+cell for the SAME collection run, for free.
+
+**`GapCpToBucket(iGapCp)` (data.c) is now the single source of truth**
+for this mapping -- previously the -200/50 arithmetic was duplicated
+inline at SIX separate call sites (two lookup functions, one
+collection increment... actually three lookup functions and three
+collection sites) with no shared helper, exactly the kind of
+duplication that drifts silently. Every lookup
+(`PredictedFHPercentHundredths`, `JointMoveNeverFailsHigh`,
+`ConfirmedZeroFailHigh`) and every collection increment (search.c, two
+sites) now calls this one function.
+
+**Both baked tables were rebuilt from scratch** using this new
+bucketing, sourced from the SAME 45-minute joint-run pickle
+(`reassembled_joint.pkl`'s `gap` marginal for the percentage table,
+`joint` dict for the 4D table) -- this also fixes a real, previously
+flagged inconsistency: the percentage table used to be baked from an
+EARLIER, smaller, differently-composed collection run
+(`reassembled_gate1.pkl`) than the joint table, meaning the two halves
+of the "one seamless mechanism" disagreed about what population they
+described. Now both come from one dataset.
+
+**Verified after rebake** (`sd10`, all three curated suites): solve
+counts essentially unchanged (one single-position flip on
+`ecm_confident_quick`, within normal suite noise), and candidate
+volume at depth 6-9 on that suite jumped from ~132/85/50/1 tries
+(before coarsening) to **4,177/4,603/3,150/170** (after) -- a ~30-60x
+increase in samples per surviving cell, exactly the intended effect,
+with real observed FH still 0.000% at every one of those depths.
+
+## Self-play verdict as of this write-up (2026-09-28)
+
+A fresh match (post queen-vs-queen removal + gap-bucket rebake)
+against `head_reference`, tracked over time:
+
+| games | score | llr | ebf (cand vs head) | avg depth (cand vs head) |
+|---|---|---|---|---|
+| 100 | 0.520 | +0.10 | 3.641 < 3.654 (better) | 13.21 > 12.93 (better) |
+| 170 | 0.521 | +0.16 | 3.679 < 3.685 (better) | 12.94 > 12.69 (better) |
+| 1000 | **0.479** | **-1.34** | 3.617 > 3.600 (worse) | 13.19 > 13.05 (still better) |
+
+**Honest read: the early (100-170 game) positive trend did not hold
+at 1000 games.** llr swung from +0.16 to -1.34 -- more than a third of
+the way to the H0=-2.94 reject boundary. Average search depth still
+favors candidate at every checkpoint (the mechanism genuinely is
+reaching deeper), but that extra depth is not converting into a score
+edge, and ebf has flipped to favor head at the larger sample. This is
+roughly consistent with nearly every measurement across this whole
+project once the concrete bugs (EFP/table union, dumb-axis pooling,
+queen-vs-queen) were fixed: **"even," not "clearly ahead."**
+
+This matches the advisor's structural argument from the mid-session
+consultation: pruning at depth ranges beyond old EFP's narrow 1-2.75
+ply removes move-ordering/TT information that's load-bearing
+elsewhere in the tree, and that cost doesn't fully disappear just
+because the individual prune decisions are well-calibrated. Fixing
+concrete bugs recovers lost ground (30-game catastrophe -> 1000-game
+parity) but does not, on its own, produce a clear win.
+
+**Decision point reached, not yet resolved**: either (a) let the
+current match run to a real statistical conclusion (llr crossing
+either bound) before drawing further conclusions from noisy
+checkpoints, or (b) accept "even" as pruning's ceiling at this depth
+range and pivot to reduction (see next section) instead of continuing
+to chase a clear win from harder pruning alone.
+
+## On reduction vs. pruning, as the likely next step
+
+Not yet implemented. The core argument for why reduction should not
+need anywhere near the ~15M-samples-per-cell bar pruning would need at
+a permissive threshold: **a reduction is self-correcting when wrong.**
+The standard PVS/LMR re-search machinery already re-verifies a reduced
+move whose shallow search unexpectedly raises alpha or fails high. A
+wrong prune is permanently lost; a wrong reduction costs one extra
+re-search. That changes the risk calculus completely and was always
+the second half of the original plan ("beat EFP at its own game and
+then use this table to layer on reduction with a less conservative
+threshold").
+
+Sketch discussed, not yet built: stack an additive `extra_reduction_
+plies` term onto whatever `GetLMRReduction()` already computes for a
+gate-1-eligible move, driven by `PredictedFHPercentHundredths`:
+
+```
+FH% >= 5%              : +0  (no extra reduction)
+1% <= FH% < 5%          : +1 ply
+0.1% <= FH% < 1%        : +2 plies
+0.01% <= FH% < 0.1%     : +3 plies
+confirmed zero, or
+  FH% < 0.01%           : +4 plies
+```
+
+Clamped so total depth never goes below some floor, same clamping
+discipline `ComputeMoveExtension`/`GetLMRReduction` already use.
+Reuse the SAME gate-1-eligible population and the SAME table this
+whole document is about -- no new data collection needed to start
+this, only new search.c wiring at the LMR call site.
+
+## How to regenerate the tables (reproducibility notes)
+
+Both baked tables (`g_uFHPercentHundredths` and `g_uJointTries`/
+`g_uJointFH`, all in data.c) are derived from `.pkl` files produced by
+reassembling a sharded collection run's raw `FHGAPRAW`/`FHJOINTRAW`
+output lines (see "how the data is collected" above for the format).
+**These `.pkl` files and the reassembly/codegen scripts live under
+`/tmp/typhoon/` (scratch, per this repo's `CLAUDE.md` policy) and are
+NOT checked into git** -- if they're gone, the tables can only be
+regenerated by re-running collection from scratch. As of this
+write-up they still exist at `/tmp/typhoon/fh_joint_run/
+reassembled_joint.pkl` (the current, live source for BOTH tables).
+
+To regenerate from an existing reassembled pickle after a bucket/table
+shape change (what was actually done for the 2026-09-27 gap-bucket
+rework):
+
+1. Load the pickle (`pickle.load`) -- top-level dict with `joint`
+   (key: `(depth, move_idx[0..15], old_gap[0..31], dumb[0..5])` ->
+   `[fh_count, tries]`, note **move_idx here is already 0-based,
+   pre-capped at collection time** -- do NOT subtract 1 again) and
+   `gap` (key: `(depth, move_raw[1..63], old_gap[0..31])` ->
+   `[fh_count, tries]`, note **this one IS 1-based raw move number**,
+   a genuinely different convention between the two dicts that cost a
+   real debugging cycle to catch -- verify which convention a given
+   dict uses before writing a regeneration script, don't assume they
+   match).
+2. Re-bucket the gap axis using whatever the CURRENT `GapCpToBucket`
+   boundaries are (old bucket `g`'s underlying range is
+   `[-200+50g, -200+50(g+1))`; map each old bucket to whichever new
+   bucket contains that range -- boundaries were deliberately chosen
+   as multiples of the old 50cp bin width so this mapping is exact
+   except for the old table's final overflow bin, which has no
+   defined upper edge and needs a documented judgment call).
+3. Sum `[fh, tries]` for every old cell that maps to the same new
+   `(depth, move, new_gap, dumb)` key.
+4. Emit C initializer text matching the existing array declarations
+   exactly (dimension order, `// move N` / `// depth N` comments for
+   readability), then splice it into data.c replacing the old array
+   body between its declaration line and matching closing `};` (brace-
+   depth-tracked, not a fixed line-count assumption -- table bodies
+   are tens of thousands of lines and shift size on every rebuild).
+5. Rebuild (`gmake clean && gmake -j5 GENETIC=1 PERF_COUNTERS=1 MP=1
+   SIXTYFOUR=1`) and re-run the `sd10` three-suite sanity check before
+   trusting the result -- solve counts should be stable (+/-1 on
+   `ecm_confident_quick` is normal suite noise) and the EFP candidate
+   audit block's real-FH% should still read ~0.000% at depth >=
+   `uEFPMaxPruneDepth`.
+
+To regenerate from a FRESH collection run instead of an existing
+pickle: re-run the same sharded, `--cpus 1`, `FH_TABLE_COLLECTION`-
+gated collection process described earlier in this document (16
+shards, wall-clock-bounded via `timeout`, mixed real-game/tactical
+position pool), reassemble each shard's LAST checkpoint block's
+`FHGAPRAW`/`FHJOINTRAW` lines into the same pickle shape, then follow
+steps 2-5 above.
+
+---
+
+# Update, 2026-09-28/29: phase 2 (graduated LMR), a real measurement
+# bug found and fixed, EFP's per-move check confirmed dead, and the
+# "deep prune" idea
+
+Picks up where the previous section left off (reduction as the
+likely next step after pruning's self-play verdict landed at "even").
+This is a companion, largely independent table build -- a fresh 4-axis
+`(depth, dumb, gap, late)` joint table baked from a new 60-90 minute
+16-shard `sd 16` collection run (see
+`lmr_testing/2026-09-28_four_axis_joint_table_plan.md` for the full
+collection/bake recipe), `g_uJointFHThousandths[12][11][11][15]` in
+`data.c`, thousandths-of-a-percent resolution, `0xFFFF` sentinel below
+a 500-tries-per-cell floor. Same underlying idea as the EFP work
+above, applied to reduction instead of pruning, and building its own
+independent measurement infrastructure along the way.
+
+## The mechanism: strictly additive over `GetLMRReduction`
+
+Deliberately the smallest first experiment: HEAD's existing flat
+`-ONE_PLY` reduction (`GetLMRReduction`, searchsup.c) is left
+completely unchanged and still decides whether a move gets reduced at
+all. The table is only ever consulted to add MORE plies on top of a
+move already reduced -- never less, never on a move `GetLMRReduction`
+wouldn't have touched anyway. `EFP is left untouched too (including
+its known bug, see below) -- this round is measuring the LMR
+augmentation in isolation, on purpose.
+
+Tier thresholds (tries-weighted percentiles of the table's predicted-
+FH% distribution, computed against the ACTUAL LMR-eligible population
+-- `uLegalMoves > 5`, real `uDepth > TWO_PLY`, dumb bucket 1 excluded
+as ~97% SEE-scored losing captures/underpromotions misfiled onto the
+quiet-move axis, same lesson as the EFP work's dumb-bucket-1 exclusion
+above):
+
+```
+FH% >= p75 (0.072%) : +0 (no extra reduction, baseline -1 ply only)
+FH% <  p75           : +1 ply
+FH% <  p25 (0.005%)  : +2 plies
+FH% <  p5  (0.001%)  : +3 plies (max)
+```
+
+`JOINT_EXTRA_MIN_REMAINING_DEPTH` floor: currently `ONE_PLY` (the
+original, most aggressive setting) after a round-trip described below
+-- briefly raised to `FOUR_PLY` out of caution, then reverted once the
+apparent danger was shown to be a measurement artifact, not a real
+safety problem.
+
+Main move loop (`search.c`) only, same as the EFP work -- not wired
+into `split.c`'s `HelpSearch`, for the same reason (per-node snapshot
+arrays with no path into a helper thread's own context; every
+comparison protocol in this project already runs `--cpus 1`, so this
+costs nothing in what can currently be validated).
+
+## The null-move-bailout measurement bug (the big one this round)
+
+Built `JOINT_EXTRA_VERIFY` (shadow-search verification: after a real
+reduced search fails to raise beta, re-run at
+`uNextDepth + tier*ONE_PLY` and check if THAT would have failed high --
+same idea as the EFP-side "audit before trusting" discipline
+throughout this whole project) and `SILENT_MISS_TRACE` (log FEN/
+scores/moves on every detected miss). First numbers were alarming: up
+to 1-11% silent-miss rates across various configurations -- a
+"reduced search fails low, hiding a fail-high the real search would
+have found" rate that would have meant this mechanism was unsafe.
+
+**Root cause, found via live gdb stepping through an actual flagged
+case, not guessed**: `TryNullmovePruning`'s (searchsup.c) intentional
+Tord Romstad idea --
+
+```c
+// This is an idea from Tord Romstad on ccc: if we are below a
+// history reduced node and we try a nullmove and the nullmove
+// fails low and the move that refuted the nullmove involves
+// the same piece moved in the reduced move at ply-1 then bail
+// out of the reduced depth search now and research at full
+// depth. ...
+if ((mv.cFrom == (pi-1)->mv.cTo) && ((pi-1)->iExtensionAmount < 0))
+{
+    ...
+    *piNullScore = iAlpha - 1;
+    return TRUE;
+}
+```
+
+deliberately FABRICATES a score of `iAlpha - 1` and returns `TRUE`
+instead of doing a real re-search, specifically because that
+fabricated value negates up to exactly `iBeta+1` one level up --
+correctly triggering the REAL search's own existing
+re-search-on-fail-high machinery. This is correct, intentional, and
+safe in the real search path. It is NOT safe inside our bespoke,
+one-off shadow-verification `Search()` call, which has no surrounding
+wrapper to catch that "please re-search me" signal -- our own
+verification code misread the fabricated fail-high as a genuine one.
+
+**Fix**: `NEUTRALIZE_NULL_BAILOUT` (GNUmakefile flag) -- temporarily
+zero `pi->iExtensionAmount` (save/restore) around just our own shadow
+`Search()` call, disabling only this one precondition for our own
+verification, leaving every other occurrence of the real mechanism
+(anywhere else in the tree, on any other move) completely unaffected.
+
+**Result**: true silent-miss rate, measured correctly, is ~0.00056%
+(~1 in 178,000) even at the most aggressive `ONE_PLY` floor -- the
+80-99.8% of "misses" the earlier numbers showed were this measurement
+artifact, not real danger. This is why the floor was reverted from
+`FOUR_PLY` back to `ONE_PLY`: the conservative retreat had been a
+reaction to contaminated data.
+
+**Lesson for next time a shadow/audit `Search()` call is built
+anywhere in this codebase**: any node-local mechanism that
+communicates results by fabricating a boundary score and relying on a
+CALLER a level up to interpret it correctly (this null-move bailout,
+and structurally anything similar) is a landmine for exactly this
+kind of one-off internal verification call. Check for this class of
+mechanism specifically before trusting a shadow-search-based audit's
+raw miss rate.
+
+## Genuine (post-fix) misses: two distinct classes, neither cheaply catchable
+
+Investigated the ~handful of genuine misses surviving the fix by hand,
+one at a time, using a fresh, independent `sd1` vs `sd8`-through-`sd12`
+comparison per position (NOT the `SILENT_MISS_TRACE` hash-chain
+`shadowRefutation` guess, which was separately confirmed unreliable --
+`GetPonderMove`'s hash-chain walk can read stale/unrelated entries
+since the hash table is global, not scoped to the specific
+verification call; one hand-traced "refutation" turned out to be a
+completely different line than what an independent `sd12` search
+actually found). Two real, distinct patterns found:
+
+1. **Narrow-window boundary noise, not a real miss.** One traced
+   example showed `realScore`/`shadowScore` swinging wildly
+   (`-500` to `+144`) at the trace's logged `alpha=143 beta=144` --
+   but independently re-searching the SAME position freely (`sd1`
+   through `sd12`, no artificial window) showed a stable, consistent
+   assessment the whole way (-3.5 to -3.75), no swing at all. The
+   trace's narrow inherited PVS window means `shadowScore >= beta`
+   only proves "clears this specific tight ancestor-relative
+   threshold," not "the position is actually good" -- a real but
+   mundane form of alpha-beta boundary noise, not a blind spot a
+   heuristic could detect or avoid.
+2. **Genuine horizon-effect quiet tactics.** A second traced example
+   (`3r1rk1/1pq1bppp/p7/n2R4/P1N3Q1/1P6/1B3PPP/R5K1 b - - 3 0`, after
+   White's `Qd4`) was real: `sd1` likes `Qd4` (+2.06), but `sd8`
+   through `sd12` all agree it loses to `Bb6` (a quiet, non-capturing
+   deflection attacking the queen -- `Qxb6 Qxd7` nets Black the
+   exchange). `Bb6` isn't a capture or check, so it's entirely outside
+   `sd1`'s horizon; this is inherent to any bounded qsearch
+   (captures/checks only), not a specific bug.
+
+**Tried and abandoned, per direct experimentation with
+`QSearchFromCheckNoStandPat`'s (search.c) check-extension gate**:
+loosened the `+2 uQsearchCheckExtension` conditions three times in a
+row (narrowed the inner OR-conditions, then removed the
+`CountKingSafetyDefects(pos, ...) > 4` outer gate entirely), rebuilding
+and re-running the SAME automated `sd1`/`sd8` batch comparison
+(224 unique genuine-miss positions) after each change. **All three
+variants showed no measurable difference from baseline** (median
+swing 1.35, mean ~2.71-2.73, same top-10 largest-swing list/PVs, ~20.5%
+sd1==sd8 move-agreement rate, every time) -- this specific lever
+doesn't touch the class of miss the genuine examples actually show
+(the `Bb6` example isn't a check, so a check-generation-budget change
+was never going to catch it). Reverted to the original condition set.
+Conclusion, by mutual agreement: this is a quirk inherent to low-depth
+search, not a systematic, cheaply-fixable blind spot -- punt on
+chasing it further.
+
+## EFP's `ComputeMoveScore` per-move check confirmed dead, quantitatively
+
+Separately (but directly relevant to "should the table eventually
+replace EFP"), traced `ComputeMoveScore`/`DecodeMoveValueEstimate`
+(searchsup.c) end to end and confirmed by code reading (not just
+inference) that EFP's per-move sanity check --
+
+```c
+(ComputeMoveScore(ctx, mv, (x - 1)) < uFutilityMargin)
+```
+
+-- is a tautology given the gates already passed at that point in the
+`if`: every quiet (non-capture/promotion) move decodes to exactly `0`
+under `MOVE_SCORING_NORMAL`/`MOVE_SCORING_QSEARCH_*`, and
+`IS_CAPTURE_OR_PROMOTION(mv)` is already excluded earlier in the same
+condition -- so this check can never evaluate to anything but `0 <
+uFutilityMargin`, which is already guaranteed true by the
+`(uFutilityMargin)` gate a few lines up. It contributes nothing; EFP's
+real (and only) per-move selectivity has always come from the
+killer-adjacency and `GetMoveFailHighPercentage` history exemptions
+below it, confirming and extending the earlier (2026-09-27) finding in
+this same doc that "EFP's margin test was never smart about the
+specific move."
+
+## Quantifying what EFP is blind to, using the new table
+
+Queried `g_uJointFHThousandths` directly over EFP's actual typical
+operating envelope (1-2 plies remaining, gap 300-500cp below alpha --
+the `VALUE_KNIGHT`-to-`VALUE_ROOK` margin -- leftover moves,
+move-number bucket ~6-30+):
+
+- Overall range across that envelope: ~0.000% to ~0.37%, median
+  ~0.009% (p10-p90: 0.001%-0.046%).
+- **The `dumb` axis -- EFP is completely blind to this -- swings FH%
+  15-30x within an otherwise-fixed cell** (e.g. depth=1, gap=4,
+  late=8: `dumb=0` -> 0.032%, `dumb=5` -> 0.002%, `dumb=9` -> 0.001%).
+- **The `gap` axis -- EFP's only real lever -- only swings FH% ~4x**
+  across its ENTIRE knight-to-rook operating range (gap=3 -> 0.068%,
+  gap=5 -> 0.017%).
+- The `late` axis (partially, structurally captured by EFP's
+  `uLegalMoves > 1` gate, but not gradedly) also swings ~4-5x
+  (late=5 -> 0.041%, late=14 -> 0.009%).
+
+**The axis EFP cannot see (dumb) has a bigger effect than the one axis
+it does look at (gap).** Direct, quantified evidence that the table
+carries real information EFP structurally cannot use, independent of
+the `ComputeMoveScore` bug above.
+
+## The "deep prune" idea: cut hopeless-looking branches much higher in the tree
+
+Scott's proposal, motivated by the above: rather than (or in addition
+to) EFP's high-volume, low-value pruning right at the horizon (1-2
+plies remaining, ~30-100 node subtrees, mostly qsearch), use the same
+table to prune a MUCH smaller number of branches much higher up (6-8
+plies remaining) where a real hopeless/late/dumb-material cell reads a
+genuinely evidence-backed (500+ tries per the shipped floor) exact
+`0.0000%` observed FH rate.
+
+**Order-of-magnitude check, from real measured numbers (self-play
+`ebf`~3.6, and an observed ~84% qsearch-node share in a real `sd8`
+trace)**: full-width-nodes-times-qsearch-multiplier estimate puts a
+1-2-ply subtree at ~30-100 nodes (matches EFP's actual footprint) vs.
+~20,000 (depth 6) to ~250,000 (depth 8) nodes for a deep subtree --
+several orders of magnitude more payoff per prune decision.
+
+**Confirmed the table actually has real, evidence-backed near-zero
+cells there**: at depth=6-8, gap=10 (>=901cp behind -- roughly a queen
+or more), late=14 (very late/leftover), dumb=6-9 (materially hanging),
+observed FH% reads exactly `0.0000%`, and because every non-sentinel
+cell already cleared the 500-tries collection floor, this is a real
+"watched 500+ times, never once failed high" statement, not a guess.
+The most extreme combination (`dumb=10` AND `gap=10` together) doesn't
+even have data yet -- too rare to have cleared the floor in the
+current (60-90 minute) collection run, which is itself part of the
+argument for a bigger rebuild (see next section).
+
+**This is explicitly the highest-conviction next direction discussed
+this session** -- higher potential payoff than either the LMR
+augmentation above or an EFP patch, but also acknowledged as the
+riskiest (an irreversible prune, firing deeper in the tree where a
+wrong call costs an exponentially bigger subtree than either EFP or
+this round's LMR work ever risked).
+
+## Agreed design requirements for the deep-prune mechanism, before it's built for real
+
+1. **One unified eligibility gate for ALL pruning/reduction
+   mechanisms** (EFP, this round's LMR augmentation, any future LMP,
+   and the deep prune) -- extend the existing `fPruneReduceEligible`
+   checklist (already shared infrastructure from the earlier EFP work)
+   to explicitly cover: no checks, no escaping check, leftover moves
+   only, not a killer (previous ply or this ply), no reason to extend
+   this node, no reason the LAST move by this side was extended
+   (EFP's existing gate has this piece already), `!fThreat`, not a
+   capture unless losing, `iAlpha < +NMATE`. Rationale, per Scott
+   directly: these preconditions define "unremarkable move," which is
+   also exactly the population the table itself was trained against --
+   using a different gate at consult time than at collection time is
+   the same population-mismatch mistake the EFP work already paid for
+   once (see "the population the data is measured over" above).
+
+2. **A rebuilt, larger, deeper table before trusting "confirmed
+   zero" for anything irreversible.** Current table: 60-90 minutes,
+   16 shards, `sd 16`. Planned rebuild: overnight or longer (8-12+
+   hours/shard), deeper (`sd 18-20`+, to populate depth 10-13 with
+   real signal instead of the coarsened overflow bucket), bigger/more
+   varied position pool specifically to get real sample volume into
+   the deep+hopeless+late cells that are currently unpopulated (e.g.
+   `dumb=10`+`gap=10` combined). Blocked for now: this box doesn't
+   have spare cores to run a 16-shard `--cpus 1` collection run
+   alongside the ongoing 12-worker self-play match (each `match_play.py`
+   worker spawns 2 single-threaded engine processes, so 12 workers is
+   already ~24 concurrent processes) -- waiting for the self-play LLR
+   to resolve (cross either SPRT bound) before starting this.
+
+3. **"Certain" means literally `0.0%` observed, at real sample
+   volume** -- not a low-but-nonzero tier, per Scott's explicit
+   framing. A follow-on, not-yet-designed refinement: how much sample
+   volume is actually enough given how many times per game this gate
+   would fire (see the earlier EFP section's "how much statistical
+   confidence is actually needed" analysis -- same rule-of-three
+   reasoning applies here and needs re-deriving for whatever
+   application rate the deep-prune gate turns out to have).
+
+4. **A prune-specific shadow-verification harness, built BEFORE any
+   real pruning, same discipline as `JOINT_EXTRA_VERIFY` above but
+   answering a different question**: not "would a deeper search of
+   this reduced move have failed high" but "would searching this move
+   to real depth at all have failed high, given we were about to skip
+   it entirely." Must apply the (now understood) `NEUTRALIZE_NULL_
+   BAILOUT` fix from day one, and must dump FEN + full details to a
+   log on every actual miss found, for direct manual investigation
+   (matching how the two genuine-miss classes above were understood) --
+   agreed explicitly: any wrong guess gets looked at and reasoned
+   about, not just counted.
+
+5. **A dynamic, per-move-identity component is needed and is
+   currently MISSING from this round's design** -- `GetLMRReduction`'s
+   own existing pattern (require the specific move's own
+   `GetMoveFailHighPercentage` history to independently agree, not
+   just the static table cell) was flagged mid-session as something
+   the deep-prune gate must also have: require both the static table
+   cell to read genuine evidence-backed zero AND this exact move's own
+   accumulated in-search history to show zero fail-highs over some
+   minimum tries, before ever pruning for real. Countermove-conditioned
+   dynamic veto was raised as a further, explicitly UNTESTED idea for
+   later, not part of the first attempt.
+
+## Self-play status as of this write-up (2026-09-29, ongoing)
+
+First live test of the LMR-only augmentation (EFP left untouched,
+including its known `ComputeMoveScore` bug) against `head_reference`,
+`match_play.py`, `--st 4`, 12 workers, SPRT (`elo0=0, elo1=5`).
+
+**Pruning/reduction heuristic settings live for this entire match**
+(`search.c` constants, unchanged for the whole run -- see "the mechanism:
+strictly additive over `GetLMRReduction`" and "tier thresholds" sections
+above for how these were originally derived):
+
+```
+JOINT_EXTRA_TIER1_THOUSANDTHS   = 72  // ~p75 of LMR-slice FH%: below this, +1 ply
+JOINT_EXTRA_TIER2_THOUSANDTHS   = 5   // ~p25: below this, +2 plies
+JOINT_EXTRA_TIER3_THOUSANDTHS   = 1   // ~p5: below this, +3 plies
+JOINT_EXTRA_MAX_PLIES           = 3
+JOINT_EXTRA_MIN_REMAINING_DEPTH = ONE_PLY   // most aggressive floor, post round-trip (see above)
+```
+
+i.e.: HEAD's own baseline `GetLMRReduction` (`-ONE_PLY`) is left untouched;
+on top of it, a gate-1-eligible move whose 4D joint-table predicted FH%
+falls under 0.072% gets +1 extra ply, under 0.005% gets +2, under 0.001%
+gets +3 (capped by `JOINT_EXTRA_MAX_PLIES` and by the
+`JOINT_EXTRA_MIN_REMAINING_DEPTH` floor so combined reduction never drops
+remaining depth below one ply). EFP was deliberately left untouched for
+this round (see below).
+
+| games | score | llr | avg depth (cand vs head) | ebf (cand vs head) | nps (cand vs head) |
+|---|---|---|---|---|---|
+| 110  | 0.532 | +0.16 | 12.90 > 12.77 | 3.561 < 3.664 | 1,625,612 > 1,570,311 |
+| 290  | 0.533 | +0.45 | 13.21 > 12.80 | 3.545 < 3.651 | 1,618,609 > 1,576,194 |
+| 730  | 0.510 | +0.28 | 13.30 > 12.99 | 3.546 < 3.622 | 1,621,970 > 1,589,133 |
+| 2212 (final, killed by hand) | 0.502 | -0.19 | 13.28 > 12.99 | 3.543 < 3.615 | 1,604,237 > 1,577,331 |
+
+Trending positive early (110-730 games) but the match was killed by hand
+at 2212 games once it settled back to dead-even: `SPRT=INCONCLUSIVE
+LLR=-0.189 BOUNDS=[-2.944,2.944] ELO=+1.4+/-14.5 LOWER95=0.4812`. At this
+sample size SPRT hadn't crossed only because `elo1=5` sits inside the ~12
+Elo-wide 95% confidence interval the match had already converged to --
+another 75 hours (the printed ETA) would have bought a label, not new
+information, so continuing to let it run was judged not worth it.
+
+**Caveat added 2026-10-01**: this entire match ran under the
+pre-unification gate (`uLegalMoves > 5`, the population the p75/p25/p5
+tier thresholds above were themselves derived against). EFP/LMR were
+unified onto `fPruneReduceEligible` (`uLegalMoves > 1`) afterward, which
+widens the live population these thresholds now classify without
+re-deriving or re-testing them. So this "even, no strength gain" verdict
+is evidence about the *original* p75/p25/p5 thresholds against the
+*narrower* pre-unification population -- it says nothing yet about
+their behavior (or the current hand-adjusted 150/5/1 thresholds, see
+`search.c`'s `JOINT_EXTRA_TIER*_THOUSANDTHS`) against the current,
+wider `uLegalMoves > 1` population. Needs its own self-play run before
+treating this verdict as settled for the current code.
+
+**Verdict, Scott's read, 2026-09-30**: the mechanism *works* in the narrow
+sense that it delivers real, consistent extra depth/speed (+0.3 ply,
+~+1.7% nps, leaner `ebf`) for no loss in measured playing strength -- but
+that 0.3 ply isn't enough on its own to show up as a score edge; the
+extra depth has to be large enough to let the engine see things HEAD
+doesn't. Two candidate next steps, not yet decided between:
+1. **Find more *sound* pruning/reduction opportunities** via the planned
+   larger/deeper table rebuild (below) -- current table has real
+   low-traffic cells (sparse or unpopulated deep+hopeless+late
+   combinations) that a bigger collection run should fill in, unlocking
+   additional reduction without cost.
+2. **Tune the existing curve to be a bit more aggressive as a starting
+   point.** Scott's specific framing: the current tier thresholds
+   (`p75`/`p25`/`p5` -> `+0`/`+1`/`+2`/`+3` plies, see above) aren't
+   hurting playing strength but also aren't buying enough depth/speed to
+   realize a gain -- worth pushing slightly harder on the SAME curve
+   shape before waiting on a full table rebuild.
+Leaning toward trying (2) as a cheap, fast experiment first (doesn't need
+the rebuild or its core budget) while (1)'s rebuild runs in parallel once
+the match's ~24 `match_play.py` engine processes are confirmed dead.
+
+**The ~3-4% nps and ~0.4-ply depth advantage are almost certainly the
+SAME underlying effect measured two ways, not two independent
+confirmations.** This is a fixed-wall-clock-per-move match (`--st 4`),
+so depth reached is a direct function of average cost per node. If
+extra reduction shifts the tree's node mix toward proportionally more
+(cheap) qsearch nodes and fewer (expensive) full-width nodes -- which
+the lower `ebf` for candidate at both checkpoints is directly
+consistent with -- that simultaneously raises the blended nodes/second
+figure AND lets iterative deepening reach one more ply in the same
+time budget. Confirmed as a real, structural effect (not process
+scheduling noise): stable over an hour-plus of real gameplay across
+~24 concurrent single-threaded engine processes on a shared,
+contended box, not a short or single-sample reading.
+
+## Build hygiene note (2026-09-29)
+
+Before this self-play run started, did a full audit + clean rebuild to
+confirm the shipped binary carries none of the investigation-only
+instrumentation: `JOINT_EXTRA_VERIFY`, `SILENT_MISS_TRACE`,
+`NEUTRALIZE_NULL_BAILOUT`, `DISABLE_EFP`, `DEPTH_ONLY_EXTRA_REDUCTION`
+are all `ifdef`-gated behind their own opt-in `GNUmakefile` flags, none
+part of the standard `GENETIC=1 PERF_COUNTERS=1 MP=1 SIXTYFOUR=1`
+profile; confirmed absent from a clean build's own
+`--command ""`-printed `Make profile used:` line. `QSearchFromCheckNo
+StandPat`'s three rounds of live manual experimentation (see above)
+were confirmed reverted to their original condition set via `git diff`.
+No stray debugger hooks (`BREAKPOINT`) remained in `search.c`. Modified
+files vs. HEAD for this whole effort: `GNUmakefile`, `chess.h`,
+`data.c`, `root.c`, `script.c`, `search.c`.
+
+## Status / open questions as of this write-up
+
+- LMR augmentation: live, in self-play, inconclusive but trending
+  positive (llr +0.45 at 290 games). No further code changes planned
+  until this resolves one way or the other -- can't run the planned
+  table rebuild concurrently with self-play on this box's core count.
+- EFP: left deliberately untouched (including the confirmed-dead
+  `ComputeMoveScore` check) for this round, on purpose, to keep the
+  LMR-only test interpretable. Two candidate next steps discussed and
+  NOT yet decided between: (a) a conservative "veto EFP's prune when
+  the table thinks this move is surprisingly dangerous" patch (small,
+  reversible, directly fixes the confirmed dead-check bug), or (b)
+  skip straight to the deep-prune mechanism and let IT eventually
+  subsume/retire EFP rather than patching EFP incrementally. Leaning
+  toward (b) as the real long-term direction, but agreed the
+  incremental-experiment discipline that's served this whole project
+  well should still apply -- not committing to skipping (a) outright.
+- Deep-prune mechanism: designed in discussion (this section), not yet
+  implemented. Blocked on: (1) the overnight/longer table rebuild
+  (itself blocked on self-play finishing, core-count constrained), (2)
+  building the prune-specific shadow-verification harness, (3) the
+  unified-gate refactor, (4) deciding on the dynamic per-move-identity
+  component's exact shape.
+- Open, explicitly undecided strategic question: does the table
+  eventually fully replace EFP (one mechanism, deep+shallow both
+  handled by table-driven decisions), or do EFP (shallow, high-volume,
+  patched) and a new deep-prune mechanism (deep, low-volume,
+  table-driven) coexist as two mechanisms answering the same
+  underlying question from different depths? Deferred until real
+  numbers exist from both the current self-play match and a future
+  deep-prune experiment.