summaryrefslogtreecommitdiff
path: root/src/lmr_testing
diff options
context:
space:
mode:
authorScott Gasch <[email protected]>2026-10-04 08:17:52 -0700
committerScott Gasch <[email protected]>2026-10-04 08:17:52 -0700
commitbbdf1bda8f410df6aa9be0a38e71b7db04982198 (patch)
treee8315247f700ee94875cd6d2eb5dd81aef71395d /src/lmr_testing
parent9b28080f8b3dca4588cb2aaaf748451f45e3c626 (diff)
Dynamic move-ordering/LMR session: honest FH tracking, decoupled admission/magnitude, bugfixes, tooling
Lands the 2026-09-28..10-03 move-ordering investigation (full history in lmr_testing/*.md). Summary of what ships: LMR/move-ordering redesign (searchsup.c, dynamic.c, data.c, chess.h): - GetLMRReduction's admission decision (do we reduce at all) is now decoupled from magnitude (how much): LMR_STATIC_ADMIT_THOUSANDTHS=500 gates admission via the 4D joint fail-high table (data.c) plus a countermove-match veto; magnitude is a flat Crafty/Ethereal-shaped log(depth)*log(moves) table (g_iLMRQuietReduction), which measured as good or better than the old per-tier joint-table payout while being structurally immune to the per-bucket miscalibration that sank the tiered approach. g_FailHighs honesty fix (real 0/0 init, not a fake prior) underlies the dynamic-danger signal. - Retired NumLeftoverMovesToSelect's depth-indexed selection budget -- evidence showed the leftover pool has real findable signal a bailout was discarding. - ShouldLatePrune (overt late-move pruning) implemented but disabled at its call site -- measured net-negative at the tested threshold; kept for reference and as a base for a future hang-bucket-9-targeted retry. - PRUNE_VERIFY, PRUNE_LIVE, DISABLE_LMR: opt-in diagnostic build flags (GNUmakefile), off in the standard profile. Bug fixes found and fixed while exercising this work: - JOINT_EXTRA_TIER_COUNT 4->6 (chess.h): fixed an out-of-bounds counter write reachable at remaining depth >= 14 (unreachable at sd10, routine in real search/self-play). - Node-type instrumentation (search.c, root.c, searchsup.c): fixed four compounding defects that produced impossible percentages (e.g. 106.2% of PV nodes) -- wrong denominator, qsearch misclassified as main search, untyped early-exit nodes, and fail-low tests using the in-place-raised alpha instead of the initial one. - FH-rate unit unification (search.c, searchsup.c, dynamic.c, chess.h): GetMoveFailHigh* returned per-mille while g_uJointFHThousandths is thousandths-of-a-percent, two different units sharing a "_THOUSANDTHS" suffix. Unified to one scale, renamed to "_PCT_THOUSANDTHS". RateBucket's band edges fixed to match. - QSearch option B (search.c): waives the opponent-stand-pat requirement at NODE_PV only (sound there since the exact score propagates; not sound at null-window nodes). iExtensionAmount unit fix (was 0/2 into a ONE_PLY-scaled field). pi->iEval now published when Eval() runs. - eval_tune/match_play.py local_ebf: was reporting 0.000 in every match since 2026-09-18 (regex only matched the trailing-stats parser, not the live move-scan loop; engines were also spawned with "nopost" so root.c never emitted the lines at all). True per-ply EBF is ~2.0, not the ~3.9 every prior match report implied. - eval_tune/test_vs_head.sh: positional-arg shift ordering bug caused every no-extra-args run to silently rebuild ./typhoon with a non-standard profile and, separately, could feed a flag's value in as the candidate binary path. - PERF_COUNTERS cleanup (eval.c, search.c): moved 3 leaked rdtsc sites to the existing EVAL_TIME guard, deleted two concluded one-off diagnostics (u64HeadStyle*, u64FHJointHist/RateBucket/AttemptsBucket) and their report plumbing, and re-gated a per-move hang-material estimate behind FH_TABLE_COLLECTION instead of running it unconditionally. Infrastructure: - match_play.py time-odds support (--odds/--st-head/--st-candidate), validated against the engine seat rather than color. - Wilson-upper-bound re-bake of the joint table (data.c): cells store a confidence-aware bound instead of a raw point estimate, so a consumer can threshold on the value alone; monotone, so every consumer became strictly more conservative. Verified behavior-neutral for LMR. Known-open items (not resolved here, see lmr_testing/GET_LMR_REDUCTION.md section 10 and 2026-10-03_UNCOMMITTED_TRIAGE.md): the self-play gate on this candidate is underpowered (+5 Elo, 95% CI roughly [-6, +17]); the curated-suite solve-count deltas used earlier in the investigation turned out to be noise-dominated cancellation when diffed per position, not the clean signal they first appeared to be. Co-Authored-By: Claude Sonnet 5 <[email protected]>
Diffstat (limited to 'src/lmr_testing')
-rw-r--r--src/lmr_testing/2026-09-28_four_axis_joint_table_plan.md238
-rw-r--r--src/lmr_testing/2026-10-03_LMR_REDESIGN.md172
-rw-r--r--src/lmr_testing/2026-10-03_UNCOMMITTED_TRIAGE.md137
-rw-r--r--src/lmr_testing/GET_LMR_REDUCTION.md1971
-rw-r--r--src/lmr_testing/SPEC.md1543
5 files changed, 4061 insertions, 0 deletions
diff --git a/src/lmr_testing/2026-09-28_four_axis_joint_table_plan.md b/src/lmr_testing/2026-09-28_four_axis_joint_table_plan.md
new file mode 100644
index 0000000..4a67561
--- /dev/null
+++ b/src/lmr_testing/2026-09-28_four_axis_joint_table_plan.md
@@ -0,0 +1,238 @@
+# Four-axis joint FH% table -- next steps, 2026-09-28
+
+Checkpoint: `git stash` (`stash@{0}`, message starting "four-axis (dumb/gap/
+late/depth) marginal calibration instrumentation...") holds the working
+instrumentation this plan builds on: `fPruneReduceEligible` (search.c), the
+`bbMinorAttacks`/`bbRookAttacks` snapshot fix, and four independently-wired
+marginal calibration axes (dumb, gap, late, depth), each confirmed clean at
+real volume on `tests/ecm_ringers.ep_`, `sd 14`, single position family. Pop
+that stash to resume from exactly this point.
+
+## Where each axis landed (marginal, single-position-family sample)
+
+All four measured against the *eligible* leftover baseline (0.31% FH,
+matching HEAD's own EFP/LMR eligibility checklist plus losing-capture
+admission via cached SEE sign, plus `iAlpha < +NMATE`):
+
+- **dumb** (material hung, `EstimateHangingMaterialCp`): every bucket below
+ baseline except 1-100cp (~3x baseline, traced to SEE-scored losing
+ captures + underpromotions -- a real SEE limitation, not a bug; plan is to
+ exclude this bucket from pruning/reduction eligibility, not fix SEE).
+- **gap** (`iAlpha - iEval`): clean, ~4 orders of magnitude, 3.80% (gap<=0)
+ down to ~0.00% (900cp+), strictly decreasing.
+- **late** (this move's ordinal position, `uLegalMoves`): clean, ~23x decline
+ move 2 (1.61%) to move 27 (0.07%), small uptick in the deep tail (raw-move-
+ number-vs-fraction-of-list issue, already known from earlier work).
+- **depth** (`uDepth / ONE_PLY` at this node, before this move's own
+ extension/reduction): clean, ~10-15x decline depth 0 (0.44%) to depth 9-13
+ (~0.00-0.06%). Hypothesis: shallow subtrees are noisier -- the opponent
+ hasn't had enough plies to find its real defense, so a "leftover" move
+ spuriously crosses beta more often near the horizon than deep, where the
+ subtree has room to converge to the move's true (usually bad) value. This
+ is in tension with the traditional "EFP is safest near the horizon"
+ argument, which is about *cost per mistake* (small subtree), not
+ *frequency of mistakes* -- our data says frequency is actually *highest*
+ right where the traditional argument calls it safest. The tradeoff is
+ real, not resolved by this data alone.
+
+## Why a joint table now, not a cheap combined estimate
+
+Discussed and rejected the "naive-Bayes combine the four marginals, skip
+the joint collection" shortcut as the first move: cheaper, but doesn't
+address the actual open question, which is what to do about the fact that
+*every* prior attempt in this project to turn a calibrated signal into a
+live pruning/reduction win has landed at "roughly even," never a clear win,
+regardless of how correctness-verified the signal is. That pattern won't be
+fixed by better precision; if it's real, it's more likely explained by
+information the search loses when a move gets skipped (history/TT/killer
+contributions later parts of the tree would have used) or by distribution
+shift (the table is trained under full search, deployed under a search that
+now prunes/reduces differently based on it). Building the joint table is
+still worth doing -- Scott wants the real thing, not an approximation, and
+a short test run is cheap -- but the *first live test* of it must be via
+**reduction, not pruning** (see "Consumption, phase 2" below), specifically
+because reduction is self-correcting (existing re-search-on-unexpected-
+raise machinery) in a way pruning structurally cannot be. If reduction
+*also* lands at "even," that is real evidence the ceiling is structural,
+not a signal-quality problem, and no further table refinement will fix it.
+
+## Table shape
+
+Same four axes as the marginal work, but joint, cache-shaped, and
+coarsened where the marginal curves already went flat:
+
+| axis | marginal buckets | shipped buckets | boundary notes |
+|-------|-----------------:|----------------:|-----------------|
+| depth | 25 | **12** | exact 0-9 (all real signal/volume lives here), then 10-13 grouped, 14+ overflow |
+| dumb | 11 | **11** | unchanged -- already tight, semantically meaningful (0=not dumb, 1-9=100cp bins, 10=901cp+) |
+| gap | 11 | **11** | unchanged -- already tight (0=gap<=0, 1-9=100cp bins, 10=901cp+) |
+| late | 33 | **15** | raw move 1 is never produced (`uMoveNum` is 1-based) and raw move 2 is the real floor (the `fPruneReduceEligible`/collection gate requires `uLegalMoves > 1`) -- exact moves 2-13 (biggest swings live here), then 14-21 / 22-31 grouped, 32+ overflow. See 2026-09-30 correction note below: the original bucketing shifted by only 1 (assuming raw 1 was the floor), leaving shipped bucket 0 permanently empty. |
+
+Total shipped cells: 12 x 11 x 11 x 15 = **21,780**.
+
+**Axis order in the C array: depth outermost.** Depth is the one axis fixed
+for an entire node's move loop; dumb/gap/late all vary move-to-move within
+that same loop. Declaring the array as `[SEARCH_DEPTH_BUCKETS][DUMB][GAP]
+[LATE]` means every lookup during one node's whole move loop stays inside
+one depth-slice: 11 x 11 x 15 = 1,815 cells. At 2 bytes/cell (see below)
+that slice is **~3.5KB** -- small enough to stay resident in L1 for the
+entire node, not just L2 for the whole table. First touch per node pays one
+cache-miss; everything else in that node is hot. The internal order of
+dumb/gap/late doesn't matter much for locality (all three vary per-move
+similarly) -- pick whatever's most readable; `[depth][dumb][gap][late]`
+matches the order we normally discuss them in.
+
+## Cell encoding: thousandths of a percent, not hundredths
+
+`USHORT`, storing `FH% * 1000` (so 1 == 0.001%, 1000 == 1.000%). Max
+representable ~65.535%, far more than needed (every real cell we've seen is
+under ~6%) -- the range headroom isn't the point, the *low-end* precision
+is: real cells run from ~0.00% up through maybe 2-4%, and confirmed-zero
+tests and dumb/gap tail buckets both live down in the 0.00-0.10% range
+where hundredths-of-a-percent (the old table's unit) was already too coarse
+to distinguish "genuinely near-zero" from "true zero." Thousandths gives
+10x finer resolution exactly where decisions get made. No special encoding
+needed for the high end -- Scott's "anything over ~0.39% is useless to us
+anyway" observation is true (pruning/reduction decisions never care whether
+a bad cell is 2% or 6%), but it doesn't require clamping or a different
+scale; a linear thousandths encoding already wastes no precision there,
+because we simply never read the top of the range for a decision. Leave it
+be.
+
+Sentinel: `0xFFFF` (`FH_TABLE_SENTINEL`-equivalent) means "below this
+run's sample floor, don't trust this cell yet" -- see below.
+
+## Sample floor, this run only
+
+This is a 90-minute preliminary collection, not the eventual continuously-
+updated table. For *this* pass, bake with a `MIN_SAMPLES` floor (propose
+500, tune after seeing real per-cell counts) -- any cell below the floor
+gets the sentinel instead of a real percentage. A consuming (reduction)
+lookup that hits a sentinel cell should fall back to the coarser gap-only
+or dumb-only marginal estimate rather than treat "insufficient data" as
+"confirmed safe." Once this table is run for real (a week, or continuously
+in the background, matching the original project's stated intent), the
+floor can loosen or come off entirely -- Scott's point that "we don't need
+to care whether 0.0% came from 10k or 10B tries" is correct *for a table
+that keeps accumulating*, not for a single 90-minute snapshot.
+
+## How to declare and populate it
+
+1. **New raw collection counters** (chess.h `COUNTERS.tree`, mirrored into
+ `g_Options` the same way the four marginal axes already are): a single
+ joint 4D array at *marginal* (uncoarsened) resolution --
+ `UINT64 u64JointTries[SEARCH_DEPTH_BUCKETS_RAW(25)][HANG_CP_BUCKETS(11)]
+ [GAP_CP_BUCKETS(11)][MOVE_NUM_BUCKETS_RAW(33)]` and the matching
+ `u64JointFH` -- 25*11*11*33 = 99,825 cells * 16 bytes (two UINT64) =
+ ~1.6MB per counter pair. This is collection-only, lives in
+ `SEARCHER_THREAD_CONTEXT` (already ~75MB), and is never shipped -- size
+ doesn't matter here, only in the final baked table. Collecting at full
+ marginal resolution (not pre-coarsened) preserves the option to try a
+ different final bucket shape later without re-collecting, matching how
+ the gap-bucket rework reused the same joint-run pickle by re-bucketing
+ at bake time instead of re-running collection.
+
+2. **Wiring in `search.c`**: reuse the exact same
+ `fPruneReduceEligible`-gated block the four marginal axes already sit
+ in. Compute all four bucket indices (`_HangCpBucket`, `_GapCpBucket`,
+ `_MoveNumBucket`, `_DepthBucket` -- already written, unchanged) and
+ increment the single joint cell `u64JointTries[depthBucket][dumbBucket]
+ [gapBucket][lateBucket]` once per eligible move, `u64JointFH[...]` once
+ per eligible fail-high, in addition to (not instead of) the four
+ existing marginal increments -- keep the marginals live during this run
+ as a cheap cross-check that the joint reduces to what we already
+ understand.
+
+3. **Reassembly format**: a new `FOURAXISRAW depth dumb gap late fh tries`
+ line, printed the same checkpoint-friendly way as the old
+ `FHGAPRAW`/`FHJOINTRAW` lines were (full cumulative dump every N
+ positions via the suite-report path, last-checkpoint-only on
+ reassembly, never summed across checkpoints).
+
+4. **Shard/collect**: 16 shards, `--cpus 1` each (matches every prior
+ collection run in this project -- split.c's parallel helpers don't feed
+ these counters), `sd 16` or deeper (need real depth to populate the
+ depth 8-13 buckets at all -- shallower runs structurally can't reach
+ them), 90 minutes wall-clock per shard via `timeout`, mixed real-game
+ (`cross_sectional_200k.ep_`) + modest tactical (`ecm.ep_`) position pool,
+ same ~90/10 blend as prior runs. Output to `/usr/local/tmp/typhoon/`
+ (persistent, not `/tmp` -- see the disk-full incident earlier this
+ project). **Do the same smoke-test discipline as every prior run**: 3-5
+ minute all-16-shards test first, confirm 0% pruned/no crashes/sane disk
+ growth, *then* commit to the full 90 minutes.
+
+5. **Bake**: python script reads each shard's last checkpoint block,
+ aggregates `FOURAXISRAW` lines into a dict keyed by the *raw* (25 x 11 x
+ 11 x 33) indices, re-buckets depth and late down to the shipped (12,
+ 15) shape (dumb and gap pass through unchanged), applies the sample
+ floor, converts surviving cells to thousandths-of-a-percent, emits a C
+ initializer for `g_uJointFHThousandths[12][11][11][15]` matching the
+ depth-outermost declaration above, and splices it into `data.c` the same
+ brace-depth-tracked way prior bakes did (never a fixed-line-count
+ assumption -- table bodies are huge and shift size on every rebuild).
+
+## Consumption, phase 2 (after the table exists -- not this pass)
+
+Replace the current flat one-ply LMR with a graduated additive term at the
+same call site, using the *already-computed* eligibility gate and bucket
+indices (no new per-move cost beyond the table lookup itself, since
+`fPruneReduceEligible` and all four bucket functions already run on every
+candidate move once this lands):
+
+```
+uPredictedFH = LookupJointTable(depthBucket, dumbBucket, gapBucket, lateBucket);
+extra_reduction_plies = f(-ln(uPredictedFH)) // more permissive than pruning
+ // ever was -- see the doc's
+ // own "on reduction vs
+ // pruning" section for why
+ // reduction can tolerate a
+ // much less conservative
+ // threshold (self-
+ // correcting via re-search,
+ // unlike a prune)
+```
+
+Exclude the dumb axis's 1-100cp bucket from ever triggering *extra*
+reduction beyond whatever the other three axes independently justify --
+that bucket is understood to be SEE's blind spot (structural undervaluing
+of certain sacrifices/underpromotions), not a real "this move is fine"
+signal, and treating it as "safe to reduce more" would be trusting exactly
+the population we already know fools the estimate.
+
+First real test: self-play against `head_reference` (current flat one-ply
+LMR), same protocol as every prior match in this project (LOWER95 >= 0.5
+gate, not raw score; fixed `sd`/`sn` matched to what's actually being
+tested). Watch specifically for whether this breaks the "always even"
+pattern -- if it doesn't, that's the signal to stop iterating on this
+approach rather than build a bigger/more precise table.
+
+## Correction, 2026-09-30: late-bucket-0 was permanently dead, now fixed
+
+Found while reviewing coverage on a fresh 16-shard collection run:
+`_ShipLateBucket`/`ship_late_bucket`'s `-1` shift (search.c, dashboard.py)
+assumed the real floor for raw `late` was 1 (`uMoveNum` is 1-based). It
+isn't -- the collection/consultation gate itself (`fPruneReduceEligible`,
+`search.c`, requires `uLegalMoves > 1`) means raw `late` is never less
+than 2. So shipped bucket 0 (meant for raw==1) could never receive a
+sample: it read `0xFFFF` (sentinel) in every baked table to date, not
+because of insufficient collection time but because the bucketing wasted
+a slot on an unreachable raw value. This alone accounted for 1,452 of the
+21,780 shipped cells (a full `depth x dumb x gap` slice) permanently
+reading "no data," inflating the apparent coverage gap by ~6.7 percentage
+points in every prior coverage report.
+
+**Fix**: shift by 2 instead of 1, anchored on the real floor (raw >= 2):
+raw 2-13 -> shipped 0-11 (singleton), 14-21 -> 12, 22-31 -> 13, 32+ -> 14.
+Same bucket widths as before, just correctly anchored. Applied in lockstep
+to `search.c`'s `_ShipLateBucket` (live lookup path), `dashboard.py`'s
+`ship_late_bucket` (coverage reporting), and the newly-written
+`joint_table_collection/bake_joint.py` (which imports `dashboard.py`'s
+version rather than redefining it, so it can't drift out of sync). The
+table in `data.c` was rebaked from the 2026-09-30 collection run with the
+corrected bucketing; real coverage came out to 18,775/21,780 (86.2%)
+populated, vs. 17,444/21,780 (80.1%) that the old (wrong) bucketing would
+have reported on the same data.
+
+No prior collection data needed to be discarded for this -- it's purely a
+bucketing/reporting bug, not a population/gate problem (a separate concern
+raised and ruled out the same day; see SPEC.md's corresponding note).
diff --git a/src/lmr_testing/2026-10-03_LMR_REDESIGN.md b/src/lmr_testing/2026-10-03_LMR_REDESIGN.md
new file mode 100644
index 0000000..68ed068
--- /dev/null
+++ b/src/lmr_testing/2026-10-03_LMR_REDESIGN.md
@@ -0,0 +1,172 @@
+# Re-imagining the baseline LMR, 2026-10-03
+
+Question: the dumb flat `-1 ply on every leftover move` is load-bearing to
+~40-50 Elo and +1.17 ply (`GET_LMR_REDUCTION.md` 10.12a). Five attempts to
+improve on it have all measured as noise. If we rebuilt the baseline from
+scratch with what we now know, what would it be?
+
+Not looking for a miracle. Looking for ~5 Elo and some sound depth.
+
+---
+
+## The reframing that matters
+
+**Every one of the five attempts pushed in the same direction: trade accuracy
+for depth.** Graded magnitude, wider admission, bigger cuts, LMP, deep
+pruning -- all of them reduce or remove more search in exchange for a higher
+root iteration number.
+
+We now have that direction's exchange rate: **~8 Elo per marginal ply**
+(10.12b). It is a bad rate, and it is bad for a structural reason -- a
+reduction is a *transfer*, not new resource. It lowers the depth of the
+refutation lines (where the move choice is actually decided) to raise the
+depth of the PV. Time-bought plies, which raise the whole distribution, are
+worth ~95 Elo/ply.
+
+**Nobody has ever tried the other direction: trade depth for accuracy.**
+
+That is the gap. If accuracy converts better than depth -- and there is no
+reason to assume it converts at 8 Elo/ply, because it is not buying plies at
+all -- then the remaining few Elo are on that side of the ledger, not this one.
+
+---
+
+## What the five attempts actually taught us
+
+| lesson | evidence |
+|---|---|
+| **Coverage beats magnitude.** Breadth of admission moves total tree size far more than depth-of-cut per move. | section 3 (HEAD-style reduces 76% of decisions and beat the "honest" 62-66% design on node count); section 6 |
+| **Magnitude is capped by the depth DISTRIBUTION, not the formula.** 76% of the LMR population sits at remaining depth buckets 2-3, where `iMaxExtra = depth - 3` permits zero extra plies -- and where the `log(d)*log(m)` curve prescribes ~1 ply anyway, i.e. exactly flat. The curve's whole contribution is **mean 0.164 extra plies ~= +0.05 root ply**. | 10.8, 10.12c |
+| **Learned per-situation grading loses to a flat formula.** Table-driven tiers vs flat `log(d)*log(m)`: 0.224 vs 0.244 mean extra plies -- a wash, and the tier version broke with a re-search-rate inversion when widened. | section 8, 10.8 |
+| **Per-move dynamic signals cannot identify the dangerous moves.** 11 of 23 verified low-band prune misses sit at own-FH-rate **exactly 0** -- the per-move history agreed with the static table and both were wrong. A veto catching the rest would refuse ~49,000 sound prunes per miss prevented. | 10.12h |
+| **`eNodeType` is uninformative here.** Full-width ALL nodes fail HIGH 69-71% of the time. The structural label is close to meaningless. | 10.12f, 10.12d |
+| **Move ordering is the real sophistication, and it is already good.** FMBC 90-95%; `r ~= 2.17`. By the time a move is in the leftover pool the generator has already triaged it -- the table is re-triaging an already-triaged population. | 10.11a, 10.12c |
+| **Only fail-HIGHS are verified.** The re-search path fires on `iScore >= iBeta`. A reduced search that fails low is accepted silently, forever. | search.c move loop |
+
+That last row is the one nobody has acted on.
+
+---
+
+## The redesign
+
+### A. Keep (this is where the 40 Elo lives)
+- **Broad admission.** ~93% of eligible decisions get reduced. Five attempts
+ to be cleverer about *which* moves all failed.
+- **One magnitude formula, `log(depth)*log(moves)`.** Already present
+ (`g_iLMRQuietReduction`, dynamic.c, built by `InitLMRTable`). Keep it as the
+ single source of magnitude. No table, no tiers.
+- **The one-full-width-ply floor.** `JOINT_EXTRA_MIN_REMAINING_DEPTH = ONE_PLY`
+ guarantees a reduced move still gets a real node before qsearch. Stockfish
+ keeps the identical invariant (`search.cpp:1370`, `std::max(1, ...)`) and
+ gets its leanness from separate explicit pruning. Twice falsified as a
+ target (10.9a).
+- **Killer-adjacency vetoes** (this ply's killers, ply-3's). Cheap, exact,
+ principled -- a specific move at a specific ply, not an aggregate rate.
+
+### B. ADD: verify fail-lows near the margin <-- the actual proposal
+The only unverified error direction, attacked with the search's own evidence
+rather than a static prior.
+
+Today, in `search.c`'s move loop:
+```c
+iScore = -Search(ctx, -iAlpha - 1, -iAlpha, uNextDepth); /* reduced */
+if ((iExtend < 0) && (iScore >= iBeta)) { ...re-search full depth... }
+```
+At a null window `iBeta == iAlpha + 1`, so that branch means "the reduced
+search beat alpha". A reduced search that returns `iScore <= iAlpha` is taken
+at face value. Proposed addition:
+
+```c
+else if ((iExtend < 0) && (iScore > iAlpha - LMR_NEAR_MISS_MARGIN))
+{
+ /* Reduced search failed low, but only just. The reduced search is the
+ weakest evidence in the tree and this is the one direction the
+ existing machinery never checks. Re-search at full depth. */
+ uNextDepth -= iExtend;
+ pi->iExtensionAmount = 0; /* defuse the null-move bailout */
+ iScore = -Search(ctx, -iAlpha - 1, -iAlpha, uNextDepth);
+}
+```
+
+Why this is different from everything tried:
+- **Sound by construction.** It only ever ADDS search. It cannot introduce a
+ wrong answer; the worst case is wasted nodes. Every previous attempt risked
+ correctness to buy depth.
+- **It trades depth for accuracy** -- the untried direction. It will cost
+ nominal plies (marginal plies are worth ~8 Elo, so that is a cheap thing to
+ spend) in exchange for catching refutations the reduction hid.
+- **Self-measuring, and falsifiable in one suite run.** Instrument
+ `near_miss_researches` and `near_miss_flips` (how many came back
+ `>= iBeta`). If the flip rate is ~0, the idea is dead immediately and
+ cost nothing. If it is a few percent, those are refutations currently being
+ missed, and the margin can be swept.
+- **It is targeted.** A move returning `alpha - 5` nearly made it; one
+ returning `alpha - 300` did not. Score proximity is real evidence, unlike
+ the cell averages the 4D table offers.
+
+Sweep `LMR_NEAR_MISS_MARGIN` over 0 / 10 / 25 / 50 / 100 cp. Measure flip rate
+and node cost per margin before any self-play.
+
+### C. DELETE (each is unmeasured or measured-as-noise, and each costs real
+### work per decision in a SHIPPING build)
+- **The 4D table admission gate.** `LMR_STATIC_ADMIT_PCT_THOUSANDTHS` vetoes
+ 509 of 21,780 cells -- 2.3% of cell space, 7.4% of traffic -- and the
+ threshold was chosen in section 6 on solve-count deltas 10.6 showed were
+ noise. Removing it also removes `EstimateHangingMaterialCp` plus four bucket
+ functions plus a table lookup *per eligible move*, which is the same class of
+ per-move cost the PERF_COUNTERS cleanup just went after. (Hold if EFP ends up
+ needing the table -- see 10.13 item 1.)
+- **`fDynamicDanger`'s extras gate.** Fires on 27.4% of decisions and only
+ zeroes the extra plies -- which 85% of reduced moves never get anyway. The
+ section 1 "honesty fix" that motivated the whole 2026-10-02 session is
+ therefore nearly inert in shipped behaviour.
+- **The bare countermove-match veto.** Unconditional, unmeasured, and
+ generate.c's own use of a countermove match is evidence-GATED precisely
+ because a bare match was shown not to carry signal.
+- **`JOINT_EXTRA_TIER1/2/3_PCT_THOUSANDTHS`** -- vestigial (TIER1 in comments
+ only, TIER3 only in the disabled LMP path).
+
+### D. CONSIDER: widen coverage past the leftover pool
+`fPruneReduceEligible` requires `fIsLeftoverMove` (sub-`GOOD_MOVE`). Crafty and
+Stockfish reduce **all** quiet moves past a move-count threshold, including ones
+the generator scored into a higher tier. Coverage is the single lever with
+replicated evidence behind it (section 3, section 6, 10.2), and this is the
+largest untapped slice.
+
+But: it buys marginal plies, so expect ~8 Elo/ply and bound expectations
+accordingly. Do it *after* B, because B makes additional coverage safer -- the
+verification catches the extra mistakes wider coverage would make.
+
+### E. DO NOT (all falsified; reasons in 10.13 item 8)
+Per-situation magnitude grading; `eNodeType` modulation; clamp removal /
+reducing into qsearch; dynamic per-move vetoes; chasing bigger cuts.
+
+---
+
+## What this adds up to
+
+A baseline LMR that is **simpler** than today's (no 4D table, no dynamic
+extras gate, no countermove veto, one magnitude formula) and has **one new
+mechanism** that is sound by construction and pushes in the only untried
+direction.
+
+Honest expectation: the deletions are roughly Elo-neutral and buy a little
+nps; the near-miss verification is the only part with real upside, and its
+upside is unknown rather than bounded at 8 Elo/ply, because it is not buying
+plies. If its flip rate comes back near zero it dies in one suite run, which
+is the cheapest possible failure mode.
+
+That is the whole pitch: stop buying depth at 8 Elo/ply, spend some of it back
+on the one error the search never checks.
+
+## Order of work
+1. Instrument the near-miss flip rate ONLY (no behaviour change): count how
+ many reduced fail-lows land within each margin band, and -- behind a
+ verify flag -- how many of those flip to `>= iBeta` at full depth. One
+ suite run answers whether B is worth building.
+2. If the flip rate is material, implement B and sweep the margin on node
+ count + `sn` suites, then self-play with `elo0=0 elo1=15`.
+3. Do the deletions in C as a separate commit, measured on nps (idle box,
+ round-robin arms) rather than on Elo -- they are a complexity/speed play,
+ not a strength play.
+4. D last, and only if B worked.
diff --git a/src/lmr_testing/2026-10-03_UNCOMMITTED_TRIAGE.md b/src/lmr_testing/2026-10-03_UNCOMMITTED_TRIAGE.md
new file mode 100644
index 0000000..f379e96
--- /dev/null
+++ b/src/lmr_testing/2026-10-03_UNCOMMITTED_TRIAGE.md
@@ -0,0 +1,137 @@
+# Uncommitted-change triage, 2026-10-03 EOD
+
+`git status` shows 15 modified files, **5,313 insertions / 194 deletions**.
+That spans several sessions, not just today (date markers in the added lines:
+33 from 2026-10-03, 24 from 10-02, 21 from 09-28, plus 09-29/09-30/10-01).
+This file sorts it into KEEP / DECIDE / DISCARD so a commit can be assembled
+without re-deriving the reasoning.
+
+Nothing here is committed. Evidence for every claim is in
+`GET_LMR_REDUCTION.md` section 10 and `SPEC.md`.
+
+---
+
+## KEEP -- unambiguous, no open questions
+
+### Bug fixes (all found by exercising code, all verified)
+
+| what | file | why it is solid |
+|---|---|---|
+| `JOINT_EXTRA_TIER_COUNT` 4 -> 6 | chess.h | `JOINT_EXTRA_MAX_PLIES` is 5, so `*puExtraTier` could be 4 or 5 and both consumers indexed 4-element arrays out of bounds -- into `u64JointExtraResearch[0]` and into `u64LMPPruned`. Unreachable at sd10 (needs remaining depth >= 14), live in any real search. Counters-only, but it corrupted the exact calibration diagnostic candidates get judged on. 10.4 |
+| Node-type surprise line: 4 defects | search.c, root.c, searchsup.c | (1) denominator was total nodes not per-type; (2) qsearch included, ~84% of the tree, so a stand-pat cutoff read as "ALL failed high"; (3) `CommonSearchInit` early exits reached the counters un-typed and scored against a stale `eNodeType` -- produced an impossible "106.2% of PV nodes"; (4) fail-low test used the IN-PLACE-RAISED `iAlpha`, so every successful alpha-raise counted as a fail-low (PV row read 99.1%). Now uses `iInitialAlpha`. 10.12f |
+| FH-rate unit unification | search.c, searchsup.c, dynamic.c, chess.h | `GetMoveFailHigh*` returned per-mille while `g_uJointFHThousandths` is thousandths-of-a-PERCENT -- 100x apart, same `_THOUSANDTHS` suffix. `ShouldLatePrune` held `LMP_DYNAMIC_MAX_THOUSANDTHS (1)` meaning 0.1% four lines from `LMP_STATIC_MAX_THOUSANDTHS (1)` meaning 0.001%. Every caller was individually correct; the naming made that luck. Converted the whole dynamic family to the table's unit, renamed to `..._PCT_THOUSANDTHS`. 10.8 |
+| `RateBucket` band edges x100 | searchsup.c | Collateral of the unit change: edges were per-mille, so every rate >= 0.1% collapsed into the top bucket and five middle bands read zero. Caught because the histogram printed only its first and last columns. |
+| `local_ebf` fix | eval_tune/match_play.py | Reported 0.000 in **every match since it was added (2026-09-18)**. Two causes: `_ITER_RE` was applied only in the trailing stats parser but the PV lines arrive during the search in the move-scan loop; and `Engine.__init__` sends `nopost`, so `root.c:248` never emitted them at all. Now parsed in both loops plus an opt-in `--post`. This is the metric that proved the published `ebf` (cumulative `nodes**(1/depth)`) has been wrong in every match report -- true per-ply ratio is **~2.0, not ~3.9**. 10.10c |
+| `test_vs_head.sh` shift ordering | eval_tune/test_vs_head.sh | Positionals were shifted BEFORE `$2` was read, so (a) every no-extra-args run took the rebuild branch and clobbered `./typhoon` with a non-standard profile (`gmake release MP=1 PERF_COUNTERS=1`, missing GENETIC/SIXTYFOUR), and (b) the first pass-through flag's VALUE became the candidate binary path -- producing a baffling `PermissionError` from Popen trying to execute a directory. Also `${VAR:-default}` -> `${VAR-default}` so an explicitly empty `SPRT_ARGS=""` is honored. |
+
+### PERF_COUNTERS cleanup (10.12g)
+The standard profile was carrying investigation debris that did real WORK, not
+counter increments.
+
+- **3 `rdtsc` sites moved to `EVAL_TIME`** (eval.c). Bracketed every
+ king-safety hash probe. An `EVAL_TIME` flag already existed for exactly this
+ and already guarded the other **32** rdtsc sites in the file -- these were
+ the only leaks. Report side in root.c was already EVAL_TIME-guarded.
+- **2 one-off diagnostics deleted outright**: `u64HeadStyle*` (called
+ `GetMoveFailHighRawCounts`) and `u64FHJointHist` (called
+ `GetMoveFailHighPctThousandths`). Both had comments saying to remove them
+ once their investigations concluded; both concluded, and both findings are
+ recorded (10.2's admission rates; section 4's bimodal dynamic rate). Removed
+ counters, `g_Options` mirrors, suite accumulation AND both report blocks --
+ a diagnostic printing zeros forever is worse than none. Binary -7,760 bytes.
+- **4D collection block re-gated to `FH_TABLE_COLLECTION`** (search.c). It
+ calls `EstimateHangingMaterialCp` per eligible move; CLAUDE.md already keeps
+ that flag out of the standard profile *because it costs nps*. Verified by
+ guard inspection: `search.c:1637` is now `PERF_COUNTERS > FH_TABLE_COLLECTION`.
+ It was also computing the hang bucket TWICE per move in any build with a
+ prune gate.
+
+**nps payoff NOT MEASURED** -- two attempts invalid (32-engine match on the
+box; one produced `perf_clean` faster than `noperf`, which is impossible).
+Re-measure idle, round-robin arms. The changes are justified on inspection
+regardless.
+
+### Infrastructure
+- **`match_play.py` time-odds support** (`--odds`, `--st-head`,
+ `--st-candidate`). Strict validation because a degraded odds match still
+ scores ~0.500 and looks healthy. Odds follow the ENGINE, not the seat --
+ verified live, since the harness swaps colours every other game, so keying on
+ White would silently balance the match. `TC=` stamped into the result line.
+ This is what produced the time-odds calibration.
+- **`PRUNE_VERIFY`** (GNUmakefile, search.c, chess.h, root.c, script.c).
+ Audit-only: evaluates the prune gate, records banded by predicted FH%, then
+ declines to prune and watches the move's own ordinary search. No shadow
+ `Search()` call, so no null-move-bailout exposure. Banding by the decision
+ variable answers "is `prune if pred <= K` sound" for every K in one run. This
+ is the instrument any future pruning work needs. 10.12h
+- **`DISABLE_LMR`** (GNUmakefile, searchsup.c). Diagnostic flag that produced
+ the control experiment. Keep the flag, never ship the configuration.
+
+### QSearch changes (10.12e) -- smoke-tested with asserts
+- **Option B**: `fIncludeChecks` waives the opponent-stand-pat requirement at
+ `NODE_PV`, keeping the material requirement. The suppression is sound for
+ null-window nodes (a fail-low there IS the parent's cutoff, so precision buys
+ nothing) but not for wide-window nodes, where the exact score propagates and
+ mate-vs-queen-loss is a different PV -- the forced-mate case the machinery
+ exists for. Reads `eNodeType`, never writes it.
+- **`iExtensionAmount` unit fix**: was written as 0 or 2 (qsearch check PLIES)
+ into a ONE_PLY-scaled field -- a latent 32x error, benign only because both
+ readers test sign.
+- **`pi->iEval` published** when the lazy `Eval()` actually runs.
+
+### Wilson re-bake (data.c, joint_table_collection/splice_joint.py) -- 10.11e
+Cells store the **Wilson upper bound** instead of the point estimate, so a
+cell's value encodes its own sample confidence and a consumer can threshold on
+the value ALONE. Under point estimates **8,248 of 21,780 cells read exactly 0**,
+mostly on little evidence, so any "prune where the table says 0" rule was far
+broader than the data supported; under Wilson, 136 do. Monotone (no cell's value
+is ever lowered), so every consumer became strictly more conservative.
+Verified behaviour-neutral for LMR: admission 92.79% vs 92.88% before, tier
+split and ringers solve count unchanged. `LMR_STATIC_ADMIT_PCT_THOUSANDTHS`
+stays at 500 -- it was briefly raised to 1000 on a CELL-count argument, which
+was wrong because cell counts are not traffic-weighted.
+
+---
+
+## DECIDE -- real, but with an open question
+
+| what | open question |
+|---|---|
+| **The 4D table itself** (data.c, 1,777 lines) + `joint_table_collection/` | Its only live shipping consumer is `LMR_STATIC_ADMIT_PCT_THOUSANDTHS`, which vetoes 2.3% of cell space / 7.4% of traffic -- and that threshold was chosen on solve-count deltas 10.6 showed were noise. Decisive cheap test: disable admission (`= 100000`) and run vs current; if neutral the table buys nothing in a shipping build. **But item 1 of 10.13 (give EFP the dumb axis) may give it a real consumer -- do that first.** |
+| **Graded LMR** (`LMR_STATIC_ADMIT`, flat-table magnitude, dynamic/countermove vetoes) | Worth ~8 Elo/ply marginal, i.e. a few Elo total, never cleanly self-play tested against flat -1 (10.12c). The `log(d)*log(m)` magnitude contributes mean **0.164 extra plies** because at remaining depth <= 2 the curve prescribes ~1 ply, which IS flat -1. Keep or simplify per the redesign in `2026-10-03_LMR_REDESIGN.md`. |
+| **`PRUNE_LIVE`** | Measured +0.88 ply, +7 Elo -- exactly its corrected ceiling. Keep the flag for measurement; do not ship the configuration. |
+| **`split.c` zero-snapshot passing** | `HelpSearch` passes all-zero attack snapshots to `GetLMRReduction`, so `EstimateHangingMaterialCp` degrades there. Documented and deliberate (every comparison protocol runs `--cpus 1`), but it means MP search reduces differently from single-threaded. Fine as long as that stays true. |
+
+---
+
+## DISCARD -- scratch that leaked into the tree
+
+Per CLAUDE.md's scratch policy these belong under `/tmp/typhoon/`, not `src/`:
+
+- `cdefault_games.pgn`, `chkdefault_games.pgn`, `codds_games.pgn`,
+ `dflt_games.pgn` -- from this session's `test_vs_head.sh` argument-construction
+ tests (GAMES=0/2 runs). **Delete.**
+- `kiwi_log.txt`, `eval_tune/kiwi_log.txt`, `eval_tune/eval_comp.txt`,
+ `blunders.pgn`, `players.dat`, `.xboardrc` -- untracked clutter from earlier
+ sessions; not mine to judge, but none look like deliberate artifacts.
+
+Deliberately-kept untracked directories (do NOT delete): `head_reference/`,
+`joint_table_collection/`, `nullmove_calib/`, `eval_tune/position_cache/`,
+`eval_tune/opening_cache/`.
+
+---
+
+## Suggested commit split
+
+1. **Bug fixes** -- OOB counter, node-type x4, unit unification, RateBucket,
+ `local_ebf`, `test_vs_head.sh` shift. Independently correct, no behaviour
+ bets. Run `./precommit_check.sh` first (search.c behaviour did change via
+ the QSearch work).
+2. **PERF_COUNTERS cleanup** -- after an idle-box nps measurement, so the
+ commit message can state what it bought.
+3. **Infrastructure** -- time-odds, `PRUNE_VERIFY`, `DISABLE_LMR`.
+4. **QSearch changes** -- option B et al. Smoke-tested; ideally also a short
+ self-play run, since option B is a real behaviour change never measured in
+ isolation.
+5. **Table + graded LMR + PRUNE_LIVE** -- hold until 10.13 items 1 and 5 are
+ resolved.
diff --git a/src/lmr_testing/GET_LMR_REDUCTION.md b/src/lmr_testing/GET_LMR_REDUCTION.md
new file mode 100644
index 0000000..5b40031
--- /dev/null
+++ b/src/lmr_testing/GET_LMR_REDUCTION.md
@@ -0,0 +1,1971 @@
+# GetLMRReduction overhaul -- session handoff (2026-10-02)
+
+**Note on filename**: this supersedes `lmr_testing/GET_LMR_REDUCION.md`
+(missing a T -- an earlier typo in this same session). That file's
+content is reproduced/updated below; don't treat the two as
+independent histories, the old one is just a mid-session snapshot.
+
+This session started by making `GetMoveFailHighPercentage`'s dynamic
+fail-high table honest, discovered that honesty alone broke LMR's
+node-savings, spent a long stretch diagnosing and redesigning the
+gate through several dead ends, and ended with a real, working
+redesign: self-play validation is pending as of this handoff (the
+user is launching it separately). This file is the full history.
+
+**Current code state** (verified by grep immediately before writing
+this): `searchsup.c`'s `GetLMRReduction` uses
+
+- `LMR_STATIC_ADMIT_THOUSANDTHS = 500` -- wide-open admission ("do we
+ reduce at all").
+- `JOINT_EXTRA_TIER1/2/3_THOUSANDTHS = 72/5/1` -- now vestigial for
+ the live magnitude decision (see section 8); TIER1 only appears in
+ comments, TIER3 is reused by `LMP_STATIC_MAX_THOUSANDTHS` for the
+ (currently disabled) LMP experiment.
+- `LMR_FH_BASELINE_THOUSANDTHS = 4` -- dynamic-danger baseline,
+ unchanged all session.
+- Magnitude ("how much extra") is now computed by
+ `_FlatTableExtraPlies(uRemainingDepth, uLegalMoves)`, which reads
+ `g_iLMRQuietReduction` (dynamic.c) -- a Crafty/Ethereal-shaped flat
+ `log(depth)*log(moves)` table that existed, fully built, completely
+ unread, since before this session started. This is the live,
+ working mechanism as of this handoff -- see section 8.
+- `JOINT_EXTRA_MAX_PLIES = 5` (cap on extra plies stacked on the base
+ `-ONE_PLY`).
+- LMP (overt late-move pruning, section 7) is implemented in
+ `ShouldLatePrune` (searchsup.c) but **disabled** at its call site
+ (`search.c:1228`, `if (FALSE && ...)`) -- measured net-negative, kept
+ for reference, not deleted.
+
+Build is clean (`gmake -j5 GENETIC=1 PERF_COUNTERS=1 MP=1 SIXTYFOUR=1`,
+no warnings) at every checkpoint described below.
+
+## 1. The honesty fix (where this started)
+
+`g_FailHighs` (dynamic.c) used to seed every move-identity to a fake
+`1 FH / 1 attempt` (100%) on `ClearDynamicMoveOrdering`, so
+`GetMoveFailHighPercentage` never returned a real 0% -- a move needed
+~9-10 *clean* observations before its decaying fake prior dropped
+below the old `<= 10%` gate. Fixed:
+
+- `ClearDynamicMoveOrdering` now zeroes the table for real (0/0).
+- Renamed to `GetMoveFailHighThousandths` (dynamic.c) -- returns the
+ rate in thousandths (0-1000), not whole percent, since the real
+ population baseline (data.c's comment: **0.400%** tries-weighted,
+ across the 500B-node joint-table collection run) needs sub-1%
+ resolution to mean anything.
+- Added `GetMoveFailHighRawCounts` (dynamic.c) -- raw FH/attempts, used
+ by diagnostics below.
+
+Also while in there: killed the stale doc-comment in `_NewCounterMove`
+(claimed depth-gated overwrites; code actually always overwrites on
+recency, confirmed by its own inline A/B-test comment -- just the
+routine description never got updated). Removed `uCounterDepth`
+(write-only, zero readers, the "read-time bonus" it was meant to feed
+was never built) and the dead `FIRST_COUNTER_MOVE`/`SECOND_COUNTER_MOVE`
+flag constants (chess.h).
+
+Added a countermove-table veto to `GetLMRReduction` itself: a bare
+match against `ctx->sPlyInfo[ctx->uPly-2].mv` (the move played
+immediately before `mv` -- **two** plies back from here, since `mv`
+has already been made by the time `GetLMRReduction` runs) withholds
+reduction, independent of generate.c's evidence-gated tier promotion.
+
+## 2. Consolidation: one function, not two files
+
+The joint-table-driven "extra reduction" logic used to live in
+search.c as post-processing on top of `GetLMRReduction`'s own baseline
+decision -- split across two files/two inconsistent dynamic checks
+purely to minimize the original diff against HEAD. Moved the whole
+decision into `GetLMRReduction` (searchsup.c):
+
+- `EstimateHangingMaterialCp`, `HangCpBucket`, `GapCpBucket`,
+ `MoveNumBucket`, `DepthBucket` are no longer file-static -- they're
+ declared in chess.h now because search.c's own joint-table
+ *collection* instrumentation (gated on `fPruneReduceEligible` alone,
+ independent of the LMR decision) still needs them.
+- `GetLMRReduction` gained three params: `bbMinorAttacksSnapshot[2]`,
+ `bbRookAttacksSnapshot[2]` (for `EstimateHangingMaterialCp`;
+ split.c's HelpSearch has no such snapshot, passes all-zero arrays,
+ which degrade safely), and `OUT ULONG *puExtraTier` (optional, feeds
+ search.c's re-search-rate calibration instrumentation).
+- search.c's move loop is now just one call + two counter increments.
+
+## 3. The surprise: depth went DOWN, not up
+
+First redesign attempt kept the *old* two-check shape (dynamic rate
+must be low AND bucket-specific, both gates, a bare 0.4%-baseline
+veto) but made it honest. Measured against `head_reference` (committed
+baseline, no joint-table mechanism at all -- its `GetLMRReduction`
+only ever applies a flat `-ONE_PLY`, old fake-prior-decay gate):
+
+- **sd10 node count went UP 2-6%** across all three curated suites --
+ the honest gate was *more* conservative than HEAD's old loose one.
+- A single position (ECM.233) pushed to sd20 showed candidate using
+ 3x HEAD's node count at the same depth (18 ply) -- undeniable, not
+ a suite-aggregate artifact.
+- A diagnostic (`u64HeadStyleEligible`/`u64HeadStyleWouldReduce`,
+ chess.h) replays HEAD's exact old gate formula --
+ `(rawFH+1)*100/(rawAttempts+1) <= 10` -- against the SAME live
+ trajectory as the real decision. Result: **HEAD-style reduces
+ ~76-77% of eligible decisions, always by exactly 1 ply.** The honest
+ design (0.4% baseline) only reduced 62-66%, but with some decisions
+ getting 2-4 plies. Average ply-saved-per-eligible-call was actually
+ *slightly higher* for the new design (0.80-0.88 vs HEAD's 0.73-0.76)
+ -- so the problem wasn't magnitude, it was **coverage**: breadth
+ (how many moves get touched at all) matters more to total tree size
+ than depth-of-cut-per-touch, and extra-tier magnitude carries a
+ re-search tax (2-3%/tier) that eats part of its own benefit. This
+ theme (coverage beats magnitude, magnitude has a tax) recurs
+ throughout the rest of the session -- see sections 8/9.
+- A pure threshold sweep on the EXTRA-tier ceiling alone (holding the
+ honest 0.4% baseline fixed) never recovered HEAD's node count, not
+ even at the old deliberate-extreme-experiment setting
+ (9000/8300/1000 thousandths) -- confirming magnitude-tuning on an
+ already-small qualified population is the wrong lever.
+- A sweep of the 0.4% *baseline* itself told a different story:
+ loosening it to 100 thousandths (10%, matching HEAD's old literal
+ ceiling) recovered coverage (75.6%) and **beat HEAD's node count by
+ 35%**. Disabling the countermove-match veto or the bucket-prediction
+ veto *individually* moved node count by only 1-10% -- noise,
+ basically. **The flat 0.4% baseline was almost the entire story**,
+ not the "smart" vetoes. This is the first sighting of the pattern
+ that eventually won: a simple, broad admission lever beats precise
+ per-decision judgment.
+
+## 4. Why the population shape matters: the FH histogram
+
+A joint histogram of `(uAttempts bucket) x (uFHThousandths bucket)` at
+the live decision point (`u64FHJointHist`, chess.h;
+`AttemptsBucket`/`RateBucket`, searchsup.c -- still live
+`PERF_COUNTERS`-only diagnostic code, not yet removed) showed, on
+confident_quick (18.7M eligible decisions):
+
+- **~71% sit at exactly 0% observed rate**, including a large,
+ well-evidenced chunk (3.29M with 100+ attempts, genuinely zero FH
+ across real volume, not a fluke).
+- **A real, separate, well-evidenced population sits durably at
+ 1-10%** -- most of that mass is in the `attempts=100+` row (713K at
+ 1-1.9%, 1.1M at 2-4.9%, 657K at 5-9.9%), i.e. **not** small-sample
+ noise. (A real, narrower small-n artifact *does* exist separately:
+ at `attempts <= 9`, rates between ~1% and ~10% are structurally
+ unreachable -- `1/9 ≈ 11.1%` is the smallest nonzero rate at 9
+ attempts -- but this is a distinct, smaller phenomenon from the
+ large well-evidenced 1-10% population above.)
+- **~7% are pure-cold** (`uAttempts == 0`) and, under the original
+ two-population design, got flatly refused any reduction regardless
+ of what the static joint table said. The static table needs NO game
+ history (keyed on depth/hang/gap/late, not move identity) -- gating
+ it behind a dynamic-history requirement threw away its whole point
+ for this slice.
+
+So the dynamic rate is **bimodal**, not unimodal around 0.4% -- the
+population average is a real summary statistic of a mixture, not
+representative of either mode.
+
+## 5. Static-first and three-population redesigns (SUPERSEDED)
+
+Two intermediate redesigns, both now superseded by section 8's simpler
+mechanism, but worth recording for what they established:
+
+- **Static-first**: flipped priority so the static bucket prediction
+ became primary ("should we reduce at all"), dynamic signals became
+ vetoes on top rather than a precondition. Established the
+ qualitative lesson (cold moves should defer to the static table, not
+ be refused outright) that survived into every later design.
+- **Three-population**: cold / warm+safe / warm+dangerous, with
+ warm+dangerous capped to base-only rather than fully vetoed. This
+ *also* established a real, lasting bug: `JOINT_EXTRA_TIER1_
+ THOUSANDTHS` was doing double duty as both the admission threshold
+ ("reduce at all") and the magnitude threshold ("+1 extra ply") --
+ decoupling these (section 6) is what unblocked real progress.
+
+**CONTAMINATION WARNING, still relevant if this file's old numbers
+ever resurface**: partway through this session,
+`JOINT_EXTRA_TIER1/2/3_THOUSANDTHS` got silently stuck at
+**9000/8300/1000** (an old deliberate-extreme-experiment setting) for
+a long stretch, because a `cp searchsup.c searchsup.c.orig_full`
+snapshot captured a sweep script's leftover state without checking it
+first. Every measurement taken during that window (the HeadStyle
+diagnostic, the FH histogram, the first static-first pass) was
+measured under that contamination. **Lesson for next time**: after ANY
+sed-based sweep script, explicitly grep the tier/baseline constants
+before trusting a `cp`'d file as a clean base for anything else.
+
+Despite the contamination, one result survived it, independently
+corroborated later: **depth gain from LMR changes is heavily damped**.
+Even at the extreme (accidentally-active) ceiling, sn5M average search
+depth only moved 0.1-0.3 ply vs HEAD. This matches a documented
+finding already in search.c's history (2026-10-01 comment): the same
+9000/8300/1000 setting, in an *overnight self-play run*, predicted
+~1.0 extra ply/decision but measured only **+0.29 actual root-depth
+ply** -- a ~0.3x damping ratio, attributed to EBF: iterative
+deepening's cost scales by the effective branching factor per ply, so
+affording one more full iteration needs saving roughly `(EBF-1)/EBF`
+(~70-75% for EBF 3.5-4) of the previous iteration's total cost.
+
+**Crucial later correction to this section's pessimism** (from the
+user, mid-session): a *different* overnight self-play run, using the
+old p5/p25/p75-derived "dodgy" thresholds, did NOT just match HEAD --
+it ran **0.9-1.0 ply deeper and was even-or-better in actual game
+strength**. This is the one piece of ground-truth (real game outcome,
+not an ECM proxy) evidence the whole session has, and it directly
+contradicts "tune LMR for modest percentage savings is the wrong order
+of magnitude lever" -- it says aggressive-even-if-locally-unsound
+reduction, tolerated because iterative deepening self-corrects
+individual mistakes on the next pass, is exactly the right lever. This
+reframing drove everything in sections 6-9.
+
+## 6. Decoupling admission from magnitude
+
+Advisor-reviewed fix, directly motivated by the three-population bug
+above and the "0.9-1.0 ply overnight" evidence: **introduced a
+separate constant, `LMR_STATIC_ADMIT_THOUSANDTHS`, used only for "do
+we reduce at all"**, independent of `JOINT_EXTRA_TIER1_THOUSANDTHS`
+("does this earn +1 extra ply"). Also fixed the sentinel (no static
+data) branch: previously required `uAttempts > 0` even when static had
+no opinion either, needlessly refusing cold+sentinel moves; changed to
+admit whenever not dynamically dangerous, since absence of data is not
+evidence of danger.
+
+Swept `LMR_STATIC_ADMIT_THOUSANDTHS` on sn5M (all three suites, vs
+`head_reference`):
+
+| setting | ringers depth/solve | confident_quick depth/solve | hard_quick depth/solve |
+|---|---|---|---|
+| HEAD | 11.6 / 6 | 12.4 / 78 | 11.4 / 19 |
+| 500 | 11.7 / 6 | 12.8 / 77 | 11.9 / 19 |
+| 1000 (no-op gate) | 11.9 / 4 | 12.8 / 76 | 12.1 / 18 |
+
+500 was flat-to-noise on solves (0/-1/0) while gaining real depth
+(+0.1/+0.4/+0.5 ply). 1000 was deeper still but **lost solves on every
+single suite simultaneously** (-2/-2/-1) -- a consistent, same-direction
+signal, not noise, meaning the extra depth from over-wide admission
+wasn't paying for itself. **500 was adopted as the working admission
+value**; it remains the current setting (section "Current code state"
+above).
+
+**Methodology note the user corrected mid-session**: sd10 (fixed
+depth) solve-count drops are expected and uninteresting (searching
+fewer nodes before the ceiling naturally solves fewer problems) --
+solve counts only matter as a signal at **sn5M (fixed node budget)**,
+where a drop means the extra depth isn't translating to correctness.
+Don't conflate the two contexts when reading solve deltas.
+
+## 7. Late move pruning (LMP) -- tried, net-negative, disabled not deleted
+
+Prompted by comparing against Crafty's actual source
+(`crafty-25.2/search.c:542,580`): Crafty's LMP is a hard `break` (skip
+ALL remaining moves at a node, not just one) gated purely on
+`depth < LMP_depth(16)` and `order > LMP[depth]` (a precomputed
+`3 + (i+.5)^1.9` table) -- no static table, no dynamic history, no
+per-move judgment at all. Implemented a conservative analog,
+`ShouldLatePrune` (searchsup.c): overt skip (not reduction) for the
+narrowest, most-certain tail only -- `uRemainingDepth <=
+LMP_MAX_REMAINING_DEPTH(6 plies)`, move count past a per-depth floor,
+AND both static (`JOINT_EXTRA_TIER3_THOUSANDTHS`, the strictest tier)
+and dynamic (well-evidenced, near-zero rate) signals agreeing it's
+ultra-safe, with killer/countermove exemptions carried over unchanged.
+Wired into search.c ahead of LMR/EFP so a pruned move skips both;
+reused `fThisMoveEFPPruned`/`fAnyMoveEFPPruned`'s existing hash-
+correctness plumbing rather than duplicating it.
+
+Measured on top of the admission=500 baseline, sn5M:
+
+| suite | admit=500 alone | +LMP |
+|---|---|---|
+| ringers | 11.7 / 6 | 11.7 / 6 (unchanged) |
+| confident_quick | 12.8 / 77 | 12.8 / **76** |
+| hard_quick | 11.9 / 19 | 11.9 / **18** |
+
+Depth/EBF barely moved (same as admit=500 alone), but solve count
+dropped one further on two of three suites at sn5M (the metric that
+matters per section 6's note) -- a real cost with no offsetting
+benefit. **Disabled at the call site (`search.c:1228`, `if (FALSE &&
+...)`), not deleted** -- the mechanism may be worth revisiting with
+different thresholds (the tested band was very narrow -- only
+~250K-moves-pruned out of ~23M total nodes on sd10 ringers, i.e. this
+barely touched anything, so "net-negative" here means "net-negative
+at this specific narrow threshold," not "pruning can't work." Crafty's
+own version is dramatically more aggressive (breaks on ALL remaining
+moves, not a per-move judgment call) -- this was flagged mid-session
+as "clearly stronger but way dodgier," and was deliberately not
+pursued further given the explicit instruction not to get crazy with
+pruning (irreversible, no re-search safety net, unlike LMR).
+
+**Known, unfixed issue in `ShouldLatePrune`'s depth gate** (found
+reviewing the Crafty comparison, documented in-code at
+`searchsup.c`'s `LMP_MAX_REMAINING_DEPTH` comment but not patched,
+since LMP is disabled anyway): the function only permits pruning near
+the leaf (`uRemainingDepth <= 6*ONE_PLY`), mirroring Crafty's
+`FP_depth`/`LMP_depth` near-leaf restriction. But our own joint table
+shows predicted FH% **declining** as remaining depth grows (checked
+directly, dumb=0/gap=0/late=0 slice: depth-bucket 0 → 17119
+thousandths, bucket 5 → 4033, bucket 11 → 617, a ~25x drop) -- so the
+ceiling permits pruning exactly where our own data says risk is
+*highest*, and forbids it where risk is *lowest*. Crafty isn't
+self-contradicting the same way: it has no static table at all, so
+the near-leaf restriction *is* its entire risk model (a mistake near
+the leaf is cheap to re-discover). We inherited the structural gate
+without inheriting its justification, stacked on a mechanism that
+already has a better, contradicting signal.
+
+The user's correct reframing: the *cost* of a wrong prune does
+legitimately grow with remaining depth, but that should tighten the
+**confidence bar**, not forbid the decision outright past a cutoff. The
+principled fix is in `lmr_testing/SPEC.md`'s "how much statistical
+confidence is actually needed" section: gate on the **static table's
+own per-cell sample count** via a rule-of-three bound (`N ≈ 3 *
+applications_per_game / target_misses_per_game`) -- not on remaining
+depth, not on the dynamic per-move-identity table. SPEC.md derives
+`N ≈ 14.4M` for a 1/1000-games miss target (no deep-zone zero-FH cell
+clears this) and, after Scott's pushback that 1/1000 was picked, not
+derived, a looser 1/100-games target gives `N ≈ 1.44M` (largest real
+cell: 1.34M, close) or 1/20-games gives `N ≈ 288K` (16 deep-zone
+zero-FH cells clear this today). **Blocker**: `g_uJointFHThousandths`
+(data.c) ships only the rate per cell, not the sample count behind it
+(cells need ≥500 tries to ship at all, but a 500-try cell and a
+1.34M-try cell are indistinguishable once shipped) -- applying this
+bound correctly needs a re-bake (`bake_joint.py` + `data.c`) adding a
+companion per-cell count array. Not done this session; a stand-in
+(depth-scaled dynamic-sample floor) was drafted and then deliberately
+reverted rather than leaving an uncalibrated heuristic in place --
+current code still has the original (acknowledged-wrong) depth
+ceiling, now with this critique documented alongside it.
+
+**Concrete lead for whoever re-bakes this**: querying the raw
+collection counts directly (not the shipped table) found **43 cells
+with a genuine 0.0% fail-high prediction (0 fail-highs across all
+tries) backed by ≥288,000 tries** -- clearing the 1/20-games bound
+above. These are heavily concentrated at **hang/dumb bucket 9**
+(`HangCpBucket`'s near-top bucket, `EstimateHangingMaterialCp`'s
+largest "this move hangs a lot of material for nothing" reading),
+essentially regardless of depth/gap/late bucket -- the single largest
+qualifying cell has **5.1M tries, zero fail-highs**. This isn't a
+sparse-sampling coincidence: it's the "dumb" axis doing exactly the
+job it was built for at its most extreme setting -- a move that
+obviously just hangs material for nothing is essentially never
+actually the best move at a node, so "it raises alpha" can't happen
+independent of depth/gap/lateness context. **This is the real
+population LMP should target once sample counts are shippable**: not
+"near the leaf" (the current, backwards ceiling), but "hang bucket 9,
+any depth" -- a structural, not positional, criterion.
+
+## 8. Magnitude-loosening attempts: the real dead end, and the actual fix
+
+With admission settled at 500, the next lever tried was MAGNITUDE --
+how many extra plies a reduced move gets. Two attempts:
+
+**Attempt A -- widen the bucket-tier thresholds.** Changed
+`72/5/1/cap-3` to `150/20/5/cap-4`. Rationale: Crafty's own LMR table,
+at the remaining-depth range our suites actually exercise (5-10
+plies), applies larger cuts (2-6 plies) than our tiers were allowing.
+**Result: sd10 ringers node count went UP, 23.0M -> 27.7M -- WORSE
+than even HEAD's 26.9M.** The re-search-rate diagnostic showed why:
+tier 1's research rate (4.13%) became *higher* than tier 0's (2.73%)
+-- exactly the "distribution-shift failure" the code's own calibration
+comment warns about (low-predicted-FH tiers should show a RISING
+research rate with tier, never an inversion). Widening tier 1's
+*admission* boundary let in a bucket of moves genuinely more dangerous
+than baseline, and the resulting re-search tax outweighed the savings.
+Reverted immediately.
+
+**Attempt B -- raise only the tier 2/3 *payout*, leave all admission
+boundaries (72/5/1) untouched.** Rationale: tiers 2/3 already showed
+LOWER research rates than tier 0 even before any change -- i.e.
+headroom, not a crack, unlike tier 1. Bumped tier2 payout 2->3, tier3
+3->5, cap to 5. **Result: sd10 node count barely moved (23.04M ->
+22.97M, 0.3%)** -- essentially null. Diagnosed (advisor-assisted)
+after the fact: **89% of all reduced moves land in tier 0 (no extra
+ply at all)**, 10.3% in tier 1, and only ~1.1% combined in tiers 2/3 --
+so tuning tiers 2/3's payout only ever touches the smallest sliver of
+the population; it mathematically cannot move the aggregate much
+either way. Separately, the new `cap=5` needed `remaining_plies >= 8`
+to even engage (`iMaxExtra` floor logic), which these suites' typical
+5-10-ply remaining depth rarely reaches -- a second reason this came
+back null.
+
+**The actual fix -- stop grading magnitude with the joint table at
+all.** The diagnosis above reframed the real question: 89% of the
+population (tier 0) was getting a flat, ungraded `-ONE_PLY` with no
+magnitude mechanism touching it whatsoever -- every experiment so far
+had been tuning the payout of an 11% minority. Separately, a grep
+turned up `g_iLMRQuietReduction[MAX_PLY_PER_SEARCH+1][LMR_TABLE_MAX_MOVES+1]`
+(dynamic.c, built by `InitLMRTable()`, called from `main.c`) -- a
+**Crafty/Ethereal-shaped flat `log(depth)*log(moves)` table, fully
+built at startup, read by nothing, anywhere in the codebase.** A prior
+session had built the simple mechanism and then gone and built the
+whole joint-table/dynamic-FH system instead, without ever wiring the
+simple one in. Sanity-checked its magnitudes against Crafty's actual
+table at the depth range that matters (5-10 plies remaining): d=5,m=8
+gives ~2.1 plies here vs. Crafty's 2.3; d=10,m=8 gives ~2.7 vs. 3.0 --
+same ballpark, same shape, and structurally immune to the "one bucket
+silently miscalibrated" failure mode that broke Attempt A, because
+it's not a lookup into learned per-situation fail-high predictions at
+all -- just depth and move order.
+
+Wired `_FlatTableExtraPlies(uRemainingDepth, uLegalMoves)` in as the
+magnitude source, **replacing** the old `_JointTableExtraPlies`
+tier-lookup (which is now dead code, deleted) -- admission logic
+(static/dynamic agreement, killer/countermove vetoes,
+`LMR_STATIC_ADMIT_THOUSANDTHS=500`) is completely unchanged; only "how
+much extra" changed. The table returns the FULL reduction in
+ONE_PLY-scaled ticks; subtracting the existing `ONE_PLY` base and
+flooring at 0 gives the extra-ply count, clamped by the existing
+`JOINT_EXTRA_MIN_REMAINING_DEPTH` floor logic as before.
+
+**Result -- the best of the whole session, every metric improved or
+flat simultaneously:**
+
+sd10 ringers node count: **19.56M**, vs. HEAD's 26.94M (-27.4%) and
+vs. the admit=500-alone baseline's 23.04M (-15.1% further). Solve
+count unchanged (3/11).
+
+sn5M, all three suites, vs. HEAD and vs. the previous best
+(admit=500 alone, no LMP, no magnitude change):
+
+| suite | HEAD | admit=500 alone | **flat-table magnitude** |
+|---|---|---|---|
+| ringers | 11.6 ply / 6 solved | 11.7 / 6 | **11.8 / 6** |
+| confident_quick | 12.4 / 78 | 12.8 / 77 | **12.8 / 77** |
+| hard_quick | 11.4 / 19 | 11.9 / 19 | **12.0 / 19** |
+
+Depth improved on every suite vs. both HEAD and the previous best
+(+0.2/+0.4/+0.6 ply vs. HEAD); hard_quick's solve count is perfectly
+flat (19/19) despite the largest depth gain of any candidate this
+session; no suite lost a solve relative to the admit=500 baseline.
+This is the first candidate all session where no metric had to be
+traded off against another.
+
+## 9. The pattern across the whole session
+
+Every lever that worked was a **simple, broad, structural** one
+(admission width, Crafty's flat depth x order table); every lever that
+failed or plateaued was a **precise, learned, per-situation** one
+(bucket-tier magnitude grading, narrow LMP thresholds). The common
+failure mode of the precise levers was a correlated miscalibration in
+one slice of the population (tier 1's admission boundary) that a flat
+mechanism structurally cannot have, because it has no slices to
+miscalibrate -- only a single smooth surface in (depth, move-order).
+Crafty's own source confirms this isn't a coincidence: neither its LMP
+nor its LMR table consults any static or dynamic per-move signal at
+all, and it reaches ply 19 at node counts where this engine was
+reaching ply 11-12 before this session. The dynamic/static per-move
+tables are still useful -- they're what's deciding *whether* to
+reduce at all (admission), where they've demonstrably worked -- the
+mistake was asking the same machinery to also grade *how much*.
+
+## Where we left off / next steps
+
+1. **Self-play validation is the next step, launched by the user
+ separately from this handoff.** Every metric used all session (sd10
+ node count, sn5M depth/EBF, sn5M solve count) is a proxy; the ECM
+ suites are small and documented-noisy (`../tests/ECM.md`); the only
+ ground-truth evidence all session is the overnight self-play run
+ referenced in section 5. **Use a genuine time-per-move or fixed-
+ node self-play gate, never `sd` (fixed depth defeats the entire
+ mechanism being tested -- confirmed in `typhoon_lmr_sd11_sn10m_
+ protocol` memory).** Check which existing script
+ (`eval_tune/match_play.py`, `.dna`-file-oriented, built for eval
+ tuning -- or `eval_tune/test_vs_head.sh`, currently modified in the
+ working tree) actually supports two *binaries* head-to-head, since
+ `match_play.py` as built takes two DNA files against one binary,
+ not two search-behavior variants.
+2. **Cross-check any solve-count deltas against `../tests/ECM.md`**
+ before trusting them -- gained/lost positions skew heavily
+ flaky/noise; the project's own doc tracks which ECM numbers are
+ known-flaky vs. known-legit.
+3. The current working config (admission=500, flat-table magnitude,
+ LMP disabled) has NOT been tried with LMP re-enabled using a
+ structurally different criterion than the narrow one tested in
+ section 7. Section 7's concrete lead (43 real cells, hang/dumb
+ bucket 9, up to 5.1M tries, genuine 0% FH -- a "this move obviously
+ hangs material" criterion, not a depth-near-leaf one) is the
+ specific next experiment: ship per-cell sample counts
+ (`bake_joint.py`/`data.c` re-bake), then gate `ShouldLatePrune` on
+ hang-bucket-9-or-similar + a rule-of-three-cleared sample count,
+ dropping the backwards depth ceiling entirely. Not attempted this
+ session (the re-bake is real work, and the explicit instruction was
+ to not get crazy with pruning without a principled basis -- this
+ gives one).
+4. Before running any further sed-based sweep, confirm the baseline
+ file is actually clean (section 5's contamination warning) --
+ explicitly grep the tier/baseline/admission constants, don't just
+ trust a `cp` from a few steps ago.
+5. The `u64FHJointHist`/`AttemptsBucket`/`RateBucket`/
+ `u64HeadStyleEligible`/`u64HeadStyleWouldReduce`/`u64LMPPruned`
+ diagnostics (chess.h, searchsup.c, root.c, script.c) are explicitly
+ one-off and still wired in under `#ifdef PERF_COUNTERS`. Harmless to
+ leave (no behavior change, diagnostic-only), but strip them out once
+ this investigation concludes.
+6. `JOINT_EXTRA_TIER1_THOUSANDTHS`/`JOINT_EXTRA_TIER2_THOUSANDTHS` are
+ now unused outside of comments/the disabled LMP path respectively --
+ candidates for removal once the flat-table magnitude approach is
+ confirmed by self-play, along with the now-dead `_JointTableExtraPlies`
+ function (already deleted) and its tier-sweep history in this doc,
+ which is kept here only as a record of why the simpler mechanism won.
+7. If self-play confirms this is a real strength gain, update
+ `head_reference/` (binary + logs + README) to make this the new
+ comparison baseline, per this codebase's own stated protocol
+ (`CLAUDE.md`'s "head_reference protocol" section).
+
+---
+
+# Update, 2026-10-03: admission-rate measurement, a counter-bounds bug, and
+# why this session's solve-count evidence does not support what it was used for
+
+Written while the self-play validation from "Where we left off" item 1 was
+running (`--st 4`, SPRT elo0=0/elo1=5, 12+ workers). At 1710 games:
+`-426 =831 +453`, score 0.508, llr +0.41, avg depth candidate 13.55 vs head
+12.90, ebf 3.464 vs 3.647, nps essentially tied (1.360M vs 1.365M).
+
+All measurements below were taken with the live candidate binary
+(`./typhoon`, confirmed `-DPERF_COUNTERS` and newer than every source file),
+`--cpus 1`, `--hash 256m`, three curated suites run serially.
+`eval_tune/match_play.py` confirmed to spawn engines with `--cpus 1`, so
+`split.c`'s degraded all-zero-snapshot `GetLMRReduction` path is not involved
+in the self-play numbers either.
+
+## 10.1. The match result is underpowered, not null
+
+Score 0.5079 over 1710 games with that draw rate gives an SD of the mean of
+0.0087, i.e. **+5.5 Elo with a 95% CI of roughly [-6, +17]**. The llr is
+drifting toward H1, not away. The honest statement is that this match spans
+everything from a small loss to a solid gain and cannot yet distinguish them
+-- not "the depth didn't translate," and not "it's fine."
+
+The extra depth is real rather than nominal: `ln(3.647)/ln(3.464) = 1.0414`,
+and `1.0414 x 12.90 = 13.43` against 13.55 measured. The ebf reduction fully
+accounts for the depth gained, with no residual.
+
+**That "no residual" is itself the interesting part.** A well-targeted
+reduction should beat the arithmetic, because the saved nodes go
+preferentially to lines that matter, so accuracy-at-fixed-nodes improves
+beyond what pure depth predicts. Getting exactly the arithmetic and nothing
+more is the signature of a near-uniform tax on the leftover pool rather than
+a reallocation. See section 10.7b.
+
+## 10.2. Admission rate: the candidate reduces far MORE than HEAD, not less
+
+Ran the still-wired `u64HeadStyleEligible`/`u64HeadStyleWouldReduce`
+diagnostic (section 3 (original)) at `sd 10`. Last cumulative checkpoint per suite:
+
+| suite | eligible calls | candidate: SOME reduction | HEAD-style: flat -1 | delta |
+|---|---|---|---|---|
+| ecm_ringers | 2,897,448 | 92.88% | 71.60% | +21.3 |
+| ecm_confident_quick | 15,345,704 | 94.47% | 70.37% | +24.1 |
+| ecm_hard_quick | 28,632,443 | 93.26% | 74.99% | +18.3 |
+
+The section 3 (original) under-coverage problem is comprehensively gone. Coverage is now
+~19-24 points WIDER than HEAD-style's.
+
+**Cause, checked rather than assumed.** The first hypothesis was that the
+smoothed re-bake widened admission, since `data.c`'s current header reads
+"Every cell is populated -- no 0xffff sentinels remain" with 5,241/21,780
+cells shrunk toward a coarser marginal. That hypothesis is wrong, and
+backwards: the sentinel branch in `GetLMRReduction` was the PERMISSIVE one
+(`fBaseEligible = (FALSE == fDynamicDanger)`, admit unless dynamically
+dangerous) while the value branch is CONDITIONAL
+(`uPredictedFH < LMR_STATIC_ADMIT_THOUSANDTHS`). Moving cells from sentinel
+to value can only TIGHTEN.
+
+Parsing `g_uJointFHThousandths` out of data.c directly: only **733 of 21,780
+cells (3.4%) read >= 500**, and 224 of those sit in dumb bucket 1 where
+`_JointTablePredictedFHThousandths` returns the sentinel regardless -- so only
+~509 cells (2.3% of the space) can actually veto. They concentrate in gap
+bucket 0 (480 of 733, i.e. near-alpha) and depth buckets 0-1. So
+`LMR_STATIC_ADMIT_THOUSANDTHS = 500` has ALWAYS been close to wide open;
+section 6 (original)'s 500-vs-1000 sweep was distinguishing a ~6% slice of calls, not a
+broad gate.
+
+What actually widened coverage from section 3 (original)'s 62-66% to today's 93-94% was
+**section 6 (original)'s decoupling**: `fDynamicDanger` was demoted from vetoing the base
+reduction to merely forcing `uExtraPlies = 0`. A code change, not a data
+change.
+
+**Corollary: section 1 (original)'s `g_FailHighs` honesty fix is now nearly inert.**
+`fDynamicDanger` no longer vetoes the base reduction (except for dumb-bucket-1
+moves, which take the sentinel branch), and it only zeroes the extra plies --
+which ~85% of reduced moves never receive anyway (section 3 (original) below). The lever
+that started that whole session has almost no remaining effect on shipped
+behavior. Worth knowing before investing further in that table's accuracy.
+
+## 10.3. Magnitude is still overwhelmingly base-only
+
+Tier share of all REDUCED moves, across the three suites: tier 0 ~85-86%,
+tier 1 ~11-13%, tier 2 ~2.2-2.4%, tier 3 ~0.0-0.01%. Section 8 (original)'s "89% of all
+reduced moves land in tier 0" survived the switch from `_JointTableExtraPlies`
+to `_FlatTableExtraPlies` essentially unchanged.
+
+So the +0.65 ply in self-play is bought almost entirely by BREADTH (93-94%
+admission at a flat -1 ply), not by depth-of-cut. The flat table replaced the
+fragile tier payouts without actually paying out much more than they did.
+
+## 10.4. BUG: `JOINT_EXTRA_MAX_PLIES` (5) overruns `JOINT_EXTRA_TIER_COUNT` (4)
+
+```
+chess.h:999 #define JOINT_EXTRA_TIER_COUNT (4) // valid indices 0..3
+chess.h:1143 UINT64 u64JointExtraTries[JOINT_EXTRA_TIER_COUNT];
+chess.h:1144 UINT64 u64JointExtraResearch[JOINT_EXTRA_TIER_COUNT];
+chess.h:1149 UINT64 u64LMPPruned;
+searchsup.c:514 #define JOINT_EXTRA_MAX_PLIES (5)
+searchsup.c:934-6 if (uExtraPlies > JOINT_EXTRA_MAX_PLIES) uExtraPlies = JOINT_EXTRA_MAX_PLIES;
+searchsup.c:942 *puExtraTier = uExtraPlies; // 0..5 escapes to the caller
+search.c:1279 INC(ctx->sCounters.tree.u64JointExtraTries[uExtraTier]);
+search.c:1489 INC(ctx->sCounters.tree.u64JointExtraResearch[uThisMoveJointExtraTier]);
+```
+
+With `uExtraTier == 4`: `search.c:1279` writes `u64JointExtraTries[4]`, which
+lands on `u64JointExtraResearch[0]`; `search.c:1489` writes
+`u64JointExtraResearch[4]`, which is past BOTH arrays and lands on
+`u64LMPPruned`. The only other clamp, `iMaxExtra` (searchsup.c:922), bounds
+against remaining depth, not against the array -- at remaining depth 14 it
+permits 11.
+
+**Why no suite run ever caught it.** `_FlatTableExtraPlies` returns
+`(total_ticks - ONE_PLY)/ONE_PLY` with
+`total = 0.7844 + log(d)*log(m)/2.4696`. Tier 4 needs `total >= 5.0`:
+
+| remaining depth | moves tried | total | tier |
+|---|---|---|---|
+| 10 | 63 (LMR_TABLE_MAX_MOVES) | 4.65 | 3 |
+| 14 | 31 | 4.45 | 3 |
+| 14 | 63 | 5.21 | **4** |
+| 20 | 63 | 5.81 | **4** |
+
+At `sd 10` remaining depth never exceeds 10, so tier 3 is the ceiling and
+every suite measurement in this document (section 8 (original)'s included) is clean. The
+live match averages 13.55 ply, so tier 4 fires there routinely.
+
+**Blast radius**: counters only. Nothing reads these to make a search
+decision, so the running match's RESULT IS VALID, and `root.c:1558-1561`'s
+memcpys use `sizeof` the 4-element destinations so it does not propagate into
+`g_Options`. What it corrupts is (a) tier 0's apparent re-search rate, i.e.
+precisely the calibration diagnostic the next candidate would be judged on,
+and (b) `u64LMPPruned`, which can read nonzero while LMP is disabled at
+`search.c:1228` -- a thoroughly confusing symptom for whoever hits it next.
+
+**Fix**: raise `JOINT_EXTRA_TIER_COUNT` to 6 (`script.c`'s report loop is
+already bounded by the constant, so reporting stays faithful). Clamping
+`JOINT_EXTRA_MAX_PLIES` to 3 instead would change search behavior and is
+therefore not a pure bug fix.
+
+## 10.5. The by-tier re-search-rate diagnostic no longer means what section 8 (original) used it for
+
+```
+ tier0 tier1 tier2 tier3
+ringers 2.63% 3.84% 3.55% 3.85% (77 tries)
+confident_quick 2.69% 2.94% 2.87% 3.98% (1,204 tries)
+hard_quick 2.60% 3.74% 3.30% 2.88% (1,768 tries)
+```
+
+Section 8 (original) rejected Attempt A because tier 1's research rate (4.13%) came in
+HIGHER than tier 0's (2.73%), calling it the "distribution-shift failure."
+That was a valid test WHEN tier was a predicted-FH% band
+(`_JointTableExtraPlies`): a rising rate meant the risk ordering had broken.
+
+Under `_FlatTableExtraPlies`, tier is pure MAGNITUDE. A bigger cut
+mechanically needs more re-searches regardless of how well-calibrated
+anything is, so the inversion carries no calibration signal at all. Note
+tier 2 < tier 1 on all three suites, which no FH-ordering story predicts but
+a "bigger cut costs more, noisily" story accommodates fine.
+
+To be useful again this diagnostic must be re-based on PREDICTED FH%
+independent of applied magnitude. As currently reported it should not be used
+to accept or reject a candidate. Aggregate re-search tax is ~2.6-2.8% of
+reduced moves, which is modest.
+
+## 10.6. The solve-count evidence in this document does not support what it was used for
+
+Ran `sn 5000000` on all three suites (the metric section 6 (original) established as the
+one that matters) against `head_reference/logs/sn5m_*.out`, then diffed
+PER POSITION instead of reading the aggregate:
+
+| suite | HEAD solved/depth | candidate solved/depth | positions FLIPPED |
+|---|---|---|---|
+| ecm_ringers | 6/11 @ 11.6 ply | 6/11 @ 11.8 ply | 4 of 11 (36%) |
+| ecm_confident_quick | 78/90 @ 12.4 ply | 77/90 @ 12.8 ply | 7 of 90 ( 8%) |
+| ecm_hard_quick | 19/90 @ 11.4 ply | 20/90 @ 12.0 ply | 15 of 90 (17%) |
+
+Depth reproduces section 6 (original)/8's admit=500 row closely (+0.2/+0.4/+0.6 ply) and
+is the trustworthy axis. The solve counts are not, and cross-referencing every
+flip against `../tests/ECM.md` shows the near-flat aggregates are
+CANCELLATION, not stability:
+
+- **ringers (net 0)**: gains `ECM.213` (catalogued flaky, +0.37 unconfirmed
+ repetition line) and `ECM.270` (catalogued BAD SOLUTION -- score exactly
+ 0.00, Crafty's own sd20 prefers Rxc5); losses `ECM.721` and `ECM.746`, both
+ catalogued CLEAN. **Net 0 is two real losses masked by two junk gains.**
+- **confident_quick (net -1)**: real gains `ECM.519` (clean, +7.33) and
+ `ECM.736` (Crafty confirms the same move across depths 24-28 while HEAD
+ wobbles); real loss `ECM.348` (NOT CATALOGUED); `ECM.357` gain and
+ `ECM.715`/`716`/`826` losses all catalogued flaky or explicit horizon
+ artifacts. **De-noised this suite is roughly +2/-1 -- the OPPOSITE sign
+ from its aggregate.**
+- **hard_quick (net +1)**: real losses `ECM.128` (clean, +2.66, no
+ oscillation) and `ECM.846` (explicitly RECLASSIFIED NON-flaky on 2026-09-11
+ from two-engine deep stable agreement); real gains `ECM.476`
+ (Crafty-confirmed legit) and `ECM.803` (Crafty finds mate in 10 from the
+ solution move). `ECM.866` is catalogued verbatim as "a genuinely coin-flip
+ position." `ECM.018`/`084`/`516`/`640`/`774` NOT CATALOGUED.
+
+**Uncatalogued positions that flipped and should be triaged into ECM.md**:
+`ECM.348`, `ECM.018`, `ECM.084`, `ECM.516`, `ECM.640`, `ECM.774`.
+
+### Implication: section 6 (original)'s admission decision may rest on noise
+
+`LMR_STATIC_ADMIT_THOUSANDTHS = 500` was chosen over 1000 because 1000 "lost
+solves on every single suite simultaneously (-2/-2/-1) -- a consistent,
+same-direction signal, not noise." But a single configuration churns 4/11,
+7/90 and 15/90 positions here, with each net produced by catalogued-junk
+cancelling catalogued-clean in BOTH directions. A -2/-2/-1 aggregate sits
+well inside that churn.
+
+Direction-consistency across three small aggregates was treated as evidence
+against noise. That inference does not hold when each aggregate is itself a
+near-cancellation of 8-36% churn.
+
+The same caveat applies to **section 7 (original)'s LMP rejection** (-1 on two suites)
+and **section 8 (original)'s "no suite lost a solve relative to the admit=500
+baseline."** None of those conclusions are refuted -- they are unsupported,
+which is different. 500 is not shown to be wrong; it is shown to rest on
+nothing.
+
+**Protocol change this implies**: never report a curated-suite solve delta as
+an aggregate again. Diff per position, classify each flip against ECM.md, and
+report "N real gains / M real losses / K junk / J uncatalogued." The script
+used here is three lines of `grep`/`sed`/`join` over the two `.out` files and
+should be checked in rather than re-derived.
+
+## 10.7. Proposed directions, in priority order
+
+**Superseded in part by 10.9** -- written before the magnitude question was
+reconciled (10.8). 10.7a and 10.7c still stand as written; 10.7b is absorbed
+into 10.9c; 10.7d is DONE; 10.7e still stands. Read 10.9 for the current
+direction list.
+
+### 10.7a. HEAD vs HEAD at 2:1 time odds -- the decisive unrun experiment
+
+Grepped `lmr_testing/*.md`, `../src/CLAUDE.md` and `../tests/ECM.md`: this
+has never been run or recorded. It is the highest-value measurement available
+and it depends on neither ECM nor any proxy.
+
+**CORRECTED 2026-10-03** -- an earlier version of this section had the
+ebf/time conversion INVERTED (it claimed +0.65 ply was "a 0.19x doubling").
+The correct arithmetic, at ebf 3.6:
+
+- One ply costs a factor of ebf in nodes, so one DOUBLING of time buys
+ `ln(2)/ln(3.6)` = **0.54 ply**, not ~1.6.
+- Conversely this candidate's +0.65 ply costs `3.6^0.65` = 2.30x the nodes,
+ i.e. it is worth about **1.20 doublings** of time-equivalent.
+
+That inversion mattered, because it made one of the two branches below look
+plausible when it is not. With the correct conversion, if the measured
+Elo-per-doubling is D, then a sound +0.65 ply "should" be worth
+`1.20 x D` Elo:
+
+| measured D (Elo/doubling) | expected Elo for +0.65 ply | vs. the ~+2 to +5 measured |
+|---|---|---|
+| 60 | ~72 | 15-30x shortfall |
+| 40 | ~48 | 10-20x shortfall |
+| 20 | ~24 | 5-12x shortfall |
+| ~3 | ~4 | matches |
+
+So for the candidate's measured result to be "exactly what depth is worth
+here", a doubling of time would have to be worth only ~2-4 Elo -- i.e.
+doubling the thinking time barely helps this engine at all. That is a strong,
+falsifiable claim, and the measurement settles it:
+
+- **D is small (< ~10 Elo/doubling)**: typhoon is severely eval-limited.
+ Depth is nearly worthless to it, no search-efficiency work can pay, and the
+ lever is eval/speed. Consistent with the known 1.5M-vs-7.5M nps gap vs
+ Crafty and with eval being the separately-identified bottleneck.
+- **D is normal (~20-70 Elo/doubling)**: then reduction-bought plies are
+ worth drastically less than time-bought plies of the same nominal size, and
+ the question becomes WHY. That is a search-QUALITY investigation (what is
+ being lost in the reduced tree), not a tuning exercise -- and it would
+ explain the whole project's "always slightly positive, never significant"
+ pattern in one stroke.
+
+Note which way the remaining bias runs: Elo-per-doubling generally SHRINKS
+with depth (diminishing returns), so a D measured at a short time control is
+an upper bound on D at a longer one. Using a short-TC D therefore
+over-estimates the expected Elo and over-states the shortfall. Record the
+time control the calibration was actually run at.
+
+### 10.7b. Direct the cuts rather than widening them further
+
+At 93-94% admission and 85% flat -1 ply, the mechanism is close to "reduce
+every late quiet move by one ply" -- a uniform tax. A uniform tax lowers ebf
+and buys depth, and it structurally CANNOT protect critical lines, because
+nothing in it distinguishes them. Section 1 (original)'s dynamic signal was demoted to
+extras-only, section 8 (original) removed the joint table from magnitude, and the
+admission gate it still owns vetoes 2.3% of the cell space: step by step, each
+individually justified, the mechanism converged on undirected.
+
+Three inputs for directing it are already in scope and unused:
+
+1. **`iImprovement`** is a declared parameter of `GetLMRReduction`
+ (searchsup.c:638) and is never read in the body. Not-improving nodes
+ conventionally reduce ~1 ply more.
+2. **`pi->eNodeType`** is computed for every node and is exact
+ (`NODE_PV` iff wide window, searchsup.c:2203). CUT nodes tolerate more
+ reduction than ALL nodes, which must search everything to prove a fail-low.
+ LMR ignores the distinction.
+3. **`g_HistoryCounters`/`g_ContinuationHistory`** feed move ordering but
+ reach reduction only as a binary FH-rate veto, never as a scale on R.
+
+Frame these as flat structural +/-1 adjustments on a single smooth surface --
+explicitly NOT the "precise learned per-situation" grading that failed in
+section 8 (original). They have no buckets to miscalibrate, which is why section 9 (original)'s own
+pattern predicts they behave differently from the tier payouts.
+
+### 10.7c. Re-examine the original section 9's conclusion
+
+Section 9 (original) concluded that simple broad structural levers work and precise
+learned per-situation levers fail. That is an accurate description of what
+moved NODE COUNTS. It may be the wrong inference about Elo: the broad lever is
+what produced the no-residual, flat-accuracy depth in section 1 (original), and all three
+of section 9 (original)'s supporting measurements (sd10 node count, sn5M solve count,
+by-tier research rate) are now either noise-level (section 6 (original)) or no longer
+meaningful (section 10.5). Node count is not Elo. This is the inference in this
+document I would trust least.
+
+### 10.7d. Fix the counter bug before the next calibration read -- DONE 2026-10-03
+
+Section 10.4. Fixed: `JOINT_EXTRA_TIER_COUNT` 4 -> 6 (chess.h:999), with the
+broken lockstep against `JOINT_EXTRA_MAX_PLIES` documented in place. Every
+user of the constant was already bounded by it (both counter structs, the
+`JOINT_EXTRA_VERIFY` arrays, all six `script.c` loops), so nothing else
+needed touching. Raised the count rather than clamping `MAX_PLIES` back to 3,
+which would have been a search-behavior change rather than a fix. Clean
+rebuild, no warnings, batch self-test passes.
+
+Note for anyone reading counters from a binary built BEFORE this fix --
+including the 1710-game match in 10.1 -- tier 0's re-search rate and
+`u64LMPPruned` are corrupted there. The game results are unaffected.
+
+### 10.7e. Re-derive the admission threshold, if 10.7a says depth is worth buying
+
+Nothing currently justifies 500 over any other value (section 10.6). If 10.7a shows
+depth converts, sweep it against a metric that survives scrutiny -- move
+agreement against a deep oracle over a few hundred positions gives far more
+samples per position than a binary solved/unsolved on 90, and measures
+"better decisions per node" directly.
+
+## 10.8. Why magnitude is shallow: fully reconciled, plus a unit landmine fixed
+
+Follow-up to 10.3 ("magnitude is still overwhelmingly base-only"), which
+reported the symptom without explaining it. Driven by a direct question:
+if we have a 4-axis table that judges P(fail-high), why fall back on a
+"more primitive" `log(depth)*log(moves)` formula to size the bet? We are
+at a blackjack table and should bet more when we have an edge.
+
+The answer turns out to be that the magnitude *function* was never the
+binding constraint, so swapping it changed almost nothing -- and the
+reason it isn't binding is a deliberate invariant, not an oversight.
+
+### The table is not signal-starved
+
+Reassembled the `FOURAXISRAW` collection cells from the three 10.2 `sd10`
+runs (87.8M eligible tries, 25,159 distinct raw cells) and looked up every
+cell's shipped prediction. Tries-weighted, over the admitted population:
+
+| predicted FH | share |
+|---|---|
+| < 0.001% | 3.49% |
+| 0.001-0.005% | 18.97% |
+| 0.005-0.072% | 48.07% |
+| 0.072-0.5% | 18.84% |
+| >= 0.5% (admission veto) | 10.36% |
+
+**70.5% of admitted moves read below 0.072%** (the old TIER1 boundary) and
+22.5% read below 0.005%. So "we rarely encounter moves the table thinks are
+safe" is false -- it thinks most of the population is safe.
+
+### The two magnitude sources are a wash
+
+Over LMR's actual population (see the correction below), mean extra plies:
+**4-axis table 0.224, flat log(d)*log(m) 0.244** -- the flat formula is
+marginally MORE generous, and they put the same ~62% of traffic in tier 0.
+Within the depth>=4 slice where extras are legal at all, 0.919 vs 1.002.
+They disagree per-cell on 36% of depth>=4 tries (21% where the table wants
+less, 15% where it wants more) at essentially identical means.
+
+So section 8 (original)'s swap did not meaningfully change how much we
+reduce. It changed WHICH moves get the extras, at the same aggregate. That
+also means section 8 (original)'s headline result -- ringers 23.04M ->
+19.56M nodes, attributed to this swap -- **is not explained by the change
+it was attributed to.** Still unexplained; flagging rather than guessing.
+
+### What actually binds: the one-full-width-ply invariant
+
+`Search()` drops into `QSearch` at `uDepth < THREE_QUARTERS_PLY`
+(search.c:462). A reduced move consumes TWO plies before any extra (the
+normal `- ONE_PLY` decrement plus the base `-ONE_PLY` carried in
+`iExtend`), so `JOINT_EXTRA_MIN_REMAINING_DEPTH = ONE_PLY` encodes exactly
+"a reduced move still gets at least one genuine full-width ply before the
+horizon." `searchsup.c`'s `iMaxExtra` then works out to
+**`remaining_plies - 3`**, and `uDepth > TWO_PLY` (search.c:1254) enforces
+the same invariant for the base ply (at remaining depth 3, base-only leaves
+exactly `ONE_PLY`). One invariant, two gates -- the design is coherent, and
+this floor is load-bearing rather than conservative padding.
+
+Consequence: **no extra ply is permissible below remaining depth-bucket 4,
+for any signal, from any source.** Depth mix of the LMR population:
+
+| depth bucket | share of LMR pop | iMaxExtra |
+|---|---|---|
+| 2 (uDepth 129-191) | 45.7% | 0 |
+| 3 | 30.5% | 0 |
+| 4 | 14.7% | 1 |
+| 5 | 5.8% | 2 |
+| 6 | 2.3% | 3 |
+| 7+ | 1.1% | 4+ |
+
+Only ~23.9% of the population can receive an extra ply at all. This is just
+the shape of an alpha-beta tree: node count grows with ply-from-root, so the
+mass is near the leaves, where there is no depth left to cut.
+
+### Reconciliation of the model against the engine
+
+10.3 left a 2.7x discrepancy between a first (wrong) model and the measured
+mean. Both causes found:
+
+1. **Wrong population floor (my error).** The first pass filtered at depth
+ bucket >= 3, but `uDepth > TWO_PLY` admits bucket 2 (`uDepth` 129-191),
+ where `iMaxExtra` is 0. Bucket 2 is 21.25% of the eligible population and
+ was excluded entirely. Note also that `FOURAXISRAW` collection is gated on
+ `fPruneReduceEligible` ALONE -- it does not apply LMR's own
+ `uDepth > TWO_PLY`, so the collection population is ~4x LMR's and must be
+ filtered before any magnitude projection. Easy trap; it is a different
+ population than the one `u64FHJointHist` counts.
+2. **The dynamic component**, as predicted. Read straight off
+ `u64FHJointHist` with no rebuild: `fDynamicDanger` (own rate > the 0.400%
+ baseline, with any samples at all) zeroes the extras on **27.4%** of LMR
+ decisions (27.0 / 25.1 / 28.6% across the three suites, 46.9M calls).
+ 62.1% are warm-with-rate-exactly-0 and 7.1% pure cold; both pass.
+
+| model | mean extra | x (1 - 0.274) | measured |
+|---|---|---|---|
+| bucket >= 3 (wrong floor) | 0.440 | 0.320 | 0.164 |
+| **bucket >= 2 (correct)** | **0.244** | **0.177** | **0.164** |
+
+Within 8%, and the residual has a known sign: `FOURAXISRAW` increments
+happen AFTER the EFP decision, so EFP-pruned moves get a
+`GetLMRReduction` call (counted in the 46.9M histogram) but no collection
+increment. EFP fires near the horizon, so the proxy is biased away from the
+shallowest depths, which inflates the model exactly as observed.
+
+**So the magnitude story is: the clamp is the dominant factor, the dynamic
+veto is the second, and the choice of magnitude function is third and
+nearly irrelevant.**
+
+### Unit landmine fixed (2026-10-03)
+
+`GetMoveFailHighThousandths` returned PER-MILLE (0-1000) while
+`g_uJointFHThousandths` is in thousandths of a PERCENT -- 100x finer. Every
+caller was individually correct, but the two families were indistinguishable
+by name, and `ShouldLatePrune` held both four lines apart:
+`LMP_DYNAMIC_MAX_THOUSANDTHS (1)` meaning 0.1% beside
+`LMP_STATIC_MAX_THOUSANDTHS (1)` meaning 0.001%. The function's own doc
+comment asserted it shared the static table's unit "so a caller can compare
+this move's own dynamic rate directly against the static table's
+prediction" -- which is how the trap got planted.
+
+Fixed by converting the whole dynamic family to the table's unit (not just
+the two constants originally flagged -- leaving EFP and LMP in per-mille
+would have relocated the trap):
+
+| constant | was | now |
+|---|---|---|
+| `GetMoveFailHighPctThousandths` (renamed) | 0-1000 | 0-100000 |
+| `LMR_FH_BASELINE_PCT_THOUSANDTHS` | 4 | 400 (0.400%) |
+| `LMR_STATIC_ADMIT_PCT_THOUSANDTHS` | 500 | 500 (unchanged) |
+| `EFP_FH_PRUNE_THRESHOLD_PCT_THOUSANDTHS` | 100 | 10000 (10%) |
+| `LMP_DYNAMIC_MAX_PCT_THOUSANDTHS` | 1 | 100 (0.100%) |
+| `LMP_STATIC_MAX_PCT_THOUSANDTHS` | 1 | unchanged |
+
+Documented in-code: the old `> 4` per-mille test could only fire once
+`fh/attempts >= 0.005`, so the SHIPPED gate behaved like 0.5%, not the
+0.400% it documented. 400 is faithful to the intent and a hair stricter than
+what shipped; the affected band needs >= 200 observations of a single move
+identity to express and is essentially unpopulated per `u64FHJointHist`.
+Intended behavior preserved; shipped behavior changed immeasurably.
+
+## 10.9. Future directions from 10.8
+
+Premise, per Scott: when the table says a move is safe we should be more
+willing to (1) reduce beyond the clamp, letting the move drop straight into
+qsearch, or (2) prune it outright. Both are right in kind; they differ by
+more than an order of magnitude in expected payoff, and the measurement
+says which is which.
+
+### 10.9a. Reduce into qsearch when the table certifies safety -- a SAFETY probe, not a depth lever
+
+Make `JOINT_EXTRA_MIN_REMAINING_DEPTH` table-conditional: `ONE_PLY`
+normally, 0 when predicted FH clears a strict bar. This is the FIRST place
+in this project where the table would do something the flat
+`log(d)*log(moves)` formula structurally cannot express -- a conditional
+floor. It is also strictly safer than pruning: the move still receives a
+qsearch score and still trips the existing fail-high re-search
+(search.c:1484), so it stays recoverable in the one direction we can detect.
+
+Measured payoff ceiling (mean extra plies, before/after the 27.4% dynamic
+veto):
+
+| variant | mean extra | after dyn veto |
+|---|---|---|
+| today (`MIN_REMAINING = ONE_PLY`) | 0.224 | 0.162 |
+| unconditional `MIN_REMAINING = 0` | 0.506 | 0.368 |
+| conditional, pred < 0.005% | 0.302 | 0.219 |
+| conditional, pred < 0.001% | 0.227 | 0.165 |
+
+**Then apply section 5 (original)'s own damping measurement** (1.0 extra
+ply/decision -> +0.29 measured root ply): the conditional-at-0.005% variant
+buys ~+0.057 extra ply/decision, i.e. roughly **+0.017 root ply.** The
+unconditional version reaches ~+0.06 root ply. Both are an order of
+magnitude below the +0.65 ply this candidate ALREADY achieves, and well
+below measurement noise.
+
+**So this cannot be justified as a strength experiment -- the arithmetic
+will not support it.** Its real value is as the cheapest available probe of
+whether the table's "safe" verdict survives a much harsher consequence: if
+pred<0.005% cells can lose their last full-width ply with no measurable
+damage, that is direct evidence the table is trustworthy at the extreme,
+which is the precondition 10.9b needs anyway. Run it at the strict
+threshold, judge it on silent-miss rate (reusing `JOINT_EXTRA_VERIFY` plus
+the `NEUTRALIZE_NULL_BAILOUT` fix, per section 8 (original)'s lesson), and
+do NOT expect or claim a depth gain.
+
+Risk class to watch specifically: the refutation that is itself a quiet move
+one ply deep. That is precisely the documented genuine-miss class 2 (the
+`Bb6` deflection, invisible to qsearch by construction), and it is the exact
+thing the discarded full-width ply existed to catch.
+
+### 10.9b. Prune outright -- where the order of magnitude actually is
+
+Reduction saves plies off a subtree; pruning removes the subtree. The
+savings scale completely differently, which is why this dominates 10.9a:
+
+- 1.88% of eligible tries sit at depth >= 4 with predicted FH < 0.005%
+ (0.87% at depth >= 5, 0.38% at depth >= 6).
+- By SPEC.md's own order-of-magnitude check a depth-6 subtree is ~20,000
+ nodes against 30-100 for a near-leaf node.
+- A 1.88% population of 100-1000x larger subtrees dominates the node budget
+ in a way no magnitude change can approach.
+
+The machinery already exists (`ShouldLatePrune`, searchsup.c, disabled at
+search.c:1228) and section 7 (original)'s concrete lead is still unused:
+**43 cells with a genuine 0.0% observed fail-high backed by >= 288,000
+tries, concentrated at hang/dumb bucket 9 essentially regardless of
+depth/gap/late, the largest with 5.1M tries and zero fail-highs.** The
+criterion is structural ("this move obviously hangs material for nothing"),
+not positional, and it replaces the current backwards near-leaf depth
+ceiling -- which section 7 (original) already documents as permitting
+pruning exactly where our own table says risk is HIGHEST.
+
+Prerequisites, unchanged from section 7 (original) and SPEC.md's agreed
+design requirements: ship per-cell sample counts (`bake_joint.py` +
+`data.c`) so a rule-of-three bound can gate on evidence rather than on
+remaining depth, and build the prune-specific shadow-verification harness
+BEFORE pruning for real.
+
+### 10.9c. Coverage into the populations currently excluded outright
+
+Neither 10.9a nor 10.9b touches anything outside the already-admitted
+leftover pool. Still untouched: the 10.4% that admission vetoes, every
+`NODE_PV` node, and every non-leftover (sub-`GOOD_MOVE`-failing) move --
+`fPruneReduceEligible` excludes the latter two structurally. Crafty and
+Ethereal both reduce all quiet moves past a move-count threshold, and
+"coverage beats magnitude" is the most replicated finding in this document
+(section 3 (original), section 6 (original), 10.2). This is the largest
+remaining lever by population size, and 10.8 is the explanation for WHY it
+is larger than magnitude: magnitude is capped by the depth distribution,
+coverage is not.
+
+Also in this family, and nearly free: the three unused inputs from 10.7b
+(`iImprovement`, a declared-but-unread parameter of `GetLMRReduction`;
+`pi->eNodeType`; history as a scale on R rather than a binary veto).
+
+### 10.9d. Standing caveat over all of the above
+
+10.7a (HEAD vs HEAD at 2:1 time odds) still gates every one of these. If a
+ply is worth ~25 Elo in this engine rather than the ~60-70 a healthy engine
+shows, then even 10.9b's node savings convert poorly and the real lever is
+eval/speed, not search. That measurement is cheaper than any direction here
+and decides whether to pursue them at all.
+
+### Two errors of mine recorded, because the reasoning is the useful part
+
+1. **"Relax `JOINT_EXTRA_MIN_REMAINING_DEPTH` to 0 as a cheap way to buy
+ magnitude"** -- withdrawn. It is not a magnitude tweak; it converts
+ reduction into a qsearch-only evaluation, i.e. stealth LMP, and the
+ invariant being relaxed is load-bearing (see 10.8). It survives only in
+ the much narrower, table-gated, safety-probe form of 10.9a.
+2. **Filtering the collection population at depth bucket >= 3** instead of
+ >= 2, which over-predicted mean magnitude by ~1.8x and briefly made the
+ clamp look like it hard-capped 56% of decisions rather than the correct
+ ~76% (buckets 2 and 3 combined). Corrected in 10.8; the lesson is that
+ `FOURAXISRAW` and `u64FHJointHist` count DIFFERENT populations and
+ neither is LMR's without explicit filtering.
+
+## 10.10. The time-odds calibration, and the Crafty gap decomposed (2026-10-03)
+
+Two measurements that between them reframe this whole project. Neither
+depends on ECM solve counts or any proxy.
+
+### 10.10a. A ply is worth ~95 Elo to this engine
+
+HEAD vs HEAD at 2:1 time odds (`--st 2 --odds 2`, same binary both sides,
+no SPRT -- measuring a quantity, not testing a hypothesis). Stopped by hand
+at 170 games once the interval was decisive:
+
+| games | score | Elo | depth (cand vs head) |
+|---|---|---|---|
+| 20 | 0.700 | +147 +/-109 | 11.93 vs 11.08 |
+| 70 | 0.650 | +108 +/- 58 | 12.37 vs 11.33 |
+| 120 | 0.646 | +105 +/- 45 | 12.33 vs 11.47 |
+| 170 | 0.621 | **+86 +/- 37** | 12.37 vs 11.47 |
+
+**Result: a doubling of time buys ~0.90 ply and ~+86 Elo, i.e. ~95 Elo per
+ply** at this time control on this box. The 95% interval [48, 123] excludes
+by a wide margin the "a ply is worth almost nothing here" hypothesis that
+10.7a entertained.
+
+**This is the yardstick the project never had.** Any future depth-buying
+change can be pre-registered against it: predicted ceiling = measured ply
+gain x ~95 Elo. A change that buys +0.3 ply has a ~28 Elo ceiling (so
+`elo0=0 elo1=15` will resolve it in hours); a change that buys +0.05 ply
+cannot clear any bar worth caring about and should not get a match at all.
+That single number retires the entire class of 2,000-game inconclusive runs
+this document is otherwise a history of.
+
+### 10.10b. Reduction-bought plies are worth ~1/11th of time-bought plies
+
+Side by side, same engine, same box:
+
+| source of depth | ply gain | Elo | Elo/ply |
+|---|---|---|---|
+| 2x time | +0.90 | +86 | **~95** |
+| LMR candidate (10.1) | +0.65 | ~+5 | **~8.5** |
+| the section 5 (original) overnight "dodgy thresholds" run | +0.9-1.0 | even-or-better | ~0-10 |
+
+Two independent LMR configurations convert at under 10 Elo/ply while time
+converts at ~95. **The conversion loss is structural to reduction, not a
+function of how hard it is pushed** -- the more aggressive config did not
+convert better. So "squeeze LMR until it reaches 1.0 ply" predicts roughly
++10 Elo, not +95, and at r~2.0 (below) reaching 1.0 ply means HALVING node
+count, against the ~27% savings achieved today.
+
+Leading explanation (Scott's, and it fits): nominal root depth advances
+honestly (PV nodes are never reduced), but the REFUTATION lines -- the late
+quiet moves at ALL nodes whose fail-low proofs justify the move choice --
+are searched shallower than before. Alpha-beta's answer is only as good as
+its cutoff proofs, and reduction weakens exactly those. Time-bought depth
+deepens everything proportionally; reduction-bought depth deepens the PV
+while thinning its own verification.
+
+### 10.10c. `local_ebf` was dead on arrival; the real branching factor is ~2.0, not ~3.9
+
+Three stacked bugs hid this, found while trying to answer "is our branching
+factor really 3.9?" (Scott's suspicion: it is ~2.0, and the harness's number
+is inflated by aspiration/root-PV-change re-search cost rather than real tree
+growth):
+
+1. `match_play.py`'s `ebf` is `nodes ** (1.0/depth)` on the CUMULATIVE node
+ count -- not a per-ply ratio. Its own docstring (2026-09-18) documents
+ this and says `local_ebf` was added to fix it.
+2. `local_ebf` -- the correct estimator (geometric mean of
+ `nodes_now/nodes_prev` across consecutive completed iterations) -- read
+ `0.000` in every match since it was added. `_ITER_RE` was applied only in
+ the trailing stats parser, but the per-iteration PV lines arrive DURING
+ the search, in the move-scan loop, which read and discarded them.
+3. Even after fixing (2), still 0.000: `Engine.__init__`
+ (`tune_eval_dna.py:120`) sends **`nopost`**, and `root.c:248` gates
+ per-iteration PV output on `g_Options.fShouldPost`, so the lines were
+ never emitted at all. Now opt-in via `match_play.py --post`.
+
+Measured once working, `sd 9`, two engines: **local_ebf 2.02 and 2.12**
+against the same runs' cumulative `ebf` of 4.12 and 4.14. Scott's estimate
+was right and the published ebf has been wrong in every match report in this
+project. Independently corroborated by the time-odds depth response: 2x time
+-> +0.90 ply implies a per-ply ratio of `2^(1/0.90)` ~= 2.1.
+
+Also found: **`--st` takes integer seconds** -- `--st 0.5` yields "Soft time
+limit 00:00.000", i.e. an effectively instant search. Any sub-second `--st`
+value in a past experiment produced garbage games.
+
+### 10.10d. The Crafty gap: 4.7 plies of branching + 2.7 plies of speed
+
+Direct measurement, 12 real-game positions from `tests/cross_sectional_1.ep_`,
+both engines at fixed depths 6-12, 64M hash, single-threaded, book off
+(`/tmp/typhoon/cr_measure/`). Node accounting verified comparable
+beforehand by code reading: Crafty increments per `MakeMove`
+(`search.c:452`, `quiesce.c:185/238/352`, `root.c:101` -- qsearch nodes ARE
+counted), typhoon once per `Search`/`QSearch` entry (`searchsup.c:2019`).
+
+Fitting `nodes(d) = C * r^d`, with `r` taken from consecutive-ratio
+geometric means (NOT from the regression -- `C` and `r` trade off badly over
+a 7-point depth range, visible as absurd `C` of 1,362-4,102 on Crafty's
+low-`r` positions):
+
+| | typhoon | Crafty |
+|---|---|---|
+| **r** (median) | **2.17** | **1.74** |
+| r range across 12 positions | 1.75-2.50 | 1.44-1.99 |
+
+Decomposition of the ~7-ply gap Scott observes in live games (Crafty at ~19
+when typhoon is at 12):
+
+| term | contribution |
+|---|---|
+| branching (r 2.17 vs 1.74 -> depth x 1.391) | **+4.7 ply** |
+| speed (1.43M vs 6.6M nps = 4.6x nodes) | **+2.7 ply** |
+| predicted Crafty depth at typhoon's 12 | **19.4** (observed ~19) |
+
+Two independently measured quantities reproduce the observation with no
+residual.
+
+**A hypothesis this KILLS, recorded because it was mine and it was wrong:**
+that the gap lives in `C` (unit-subtree cost) because typhoon's qsearch is
+too wide. typhoon's qsearch is indeed ~84% of its nodes against Crafty's
+~15% (same position, `sd 8`: typhoon 482,156 nodes / 406,205 qnodes; Crafty
+65,688 nodes / ~10K evals) -- but `C` fits LOWER for typhoon (~287) than
+Crafty (~448), and `C` sits inside the logarithm, so even a 20% `C`
+reduction is worth ~0.3 ply. The qsearch width difference is real and
+costs almost nothing in depth. The earlier arithmetic that made `C` look
+decisive assumed equal `r` and failed a plausibility check (it implied
+Crafty's depth-1 search costs 11 nodes, when it must visit ~35 children).
+
+So: **the branching term is the bigger half of the gap and the one that can
+be attacked.** Scott's instinct to work on search efficiency was right.
+
+### 10.10e. Crafty's lazy-eval rates, measured
+
+Settling a disagreement about whether Crafty's eval is cruder. Crafty has
+TWO lazy exits and the printed fields distinguish them (`iterate.c:624-630`,
+both percentages over denominator `evaluations + lazy_evaluations`):
+`lazy=` is the FIRST (material-only, `evaluate.c:60/64`), `full=` went
+through piece scoring, and the residual is the SECOND lazy exit
+(`evaluate.c:243`, skips piece scoring).
+
+12 positions, `sd 10`: material-only **28.8% mean** (median 30, range 0-45);
+second lazy exit **24.9%**; full eval **46.3%**. So **~54% of Crafty's eval
+calls skip the expensive piece scoring**, which supports Scott's 40-50%
+recollection; the material-only tier alone is ~29%, not the 20% claimed
+earlier in this session from a single position at a single depth (the same
+position reads 30% at depth 10 -- that number was a bad sample, and
+depth-dependent).
+
+typhoon has the matching three-way split (`u64SuperLazyEvals`/
+`u64LazyEvals`/`u64FullEvals` plus cycle counts, `root.c:508-542`) but
+behind `#ifdef LAZY_EVAL`, not in the standard profile -- one rebuild gets a
+like-for-like comparison.
+
+**Caveat on what this can support**: lazy exits fire when the score is
+already far outside the window, i.e. in lopsided positions where detail does
+not matter. These counters measure eval COST, not eval QUALITY where it
+counts. Neither side of the eval-quality argument is settled by them.
+
+### 10.10f. Exchange truncation: the mechanism, and three fixes with very different costs
+
+Scott's symptom: losing a pawn or knight "at the end of an exchange" against
+Crafty, prompting an idea to extend all captures fractionally.
+
+NOT the futility margin -- `iFutility = iAlpha -
+(FUTILITY_BASE_MARGIN_BY_SOURCE[pi->uLastPositionalSource] +
+rgiPositional[...]) - iEval` is indexed by eval's own positional source,
+i.e. genuinely eval-calibrated rather than hand-picked.
+
+The mechanism is the SEE-ordered early exit at the top of `QSearch`'s move
+loop: moves are SEE-sorted, and the first `iValue <= 0` does `goto end`,
+abandoning every remaining move at that node. So qsearch only ever explores
+captures SEE scores as non-losing, and **any exchange whose resolution
+requires passing through a SEE-negative intermediate capture is outside the
+horizon entirely.**
+
+That cutoff is sound only to the extent SEE is right, and SPEC.md already
+documents that it is not, specifically: `_WhoAttacksSquareBB` (see.c) has no
+pin or legality awareness -- the finding that made the queen-vs-queen dumb
+bucket read backwards and got that signal deleted. A pinned defender still
+counts as a recapturer, so SEE calls a winning capture losing and qsearch
+never tries it. Same root cause, different symptom.
+
+| fix | touches | cost |
+|---|---|---|
+| make SEE pin-aware (see.c) | `C` | ~0.3 ply for a 20% `C` rise; also improves every other SEE consumer (`fPruneReduceEligible`'s capture rule, EFP, `MVF_SEE_*`) |
+| admit SEE<0 captures at shallow qsearch depth | `C` | bounded, tunable |
+| extend all captures fractionally | **`r`** | exponential; at ~95 Elo/ply, expensive |
+
+Recommendation: fix the cause on the `C` side before spending `r`.
+
+### 10.10g. Next steps implied
+
+1. **Measure Delta-r directly for extension changes, do not infer it from
+ the counters.** `root.c:568-577` prints every extension source count, but
+ those count GRANTS, before `MINU(ONE_PLY, ...)` caps and before
+ `g_uExtensionReduction[]` decays -- so they cannot give the applied ply
+ inflation that actually moves `r`. The 12-position C/r sweep
+ (`/tmp/typhoon/cr_measure/run.sh`) is now the right instrument: build with
+ extensions trimmed toward Crafty's single SEE-gated 1-ply check and
+ re-run it. Crafty has exactly ONE extension (`search.c:476/483`, checks
+ only, `check_depth=1`, SEE-gated, and it disables reductions for that
+ move); typhoon has ten fractional per-move sources plus five node-level
+ ones. Whether typhoon's 3/4-ply-but-decayed check extension "washes out"
+ against Crafty's 1-ply-but-SEE-gated one is exactly the question a Delta-r
+ measurement answers and counter arithmetic does not.
+2. **Route matters more than destination.** Closing `r` from 2.17 to 1.74 is
+ worth 4.7 plies ~= 450 Elo at the 10.10a yardstick. But reduction-bought
+ plies convert at ~8.5 Elo/ply (10.10b), so the question is not "how do we
+ lower r" but "how does Crafty lower r SOUNDLY". Extension trimming is the
+ first candidate identified that is not "reduce harder": it declines to
+ spend effort rather than suppressing information.
+3. **Clamp removal is off the table.** Stockfish floors the reduced depth at
+ 1 ply (`search.cpp:1370`, `std::max(1, ...)`) -- the same invariant
+ typhoon has -- and achieves its leanness through separate explicit
+ pruning (Step 15, `search.cpp:1166`). Crafty has no such floor and does
+ not even verify a move reduced into qsearch (the full-depth re-search at
+ `search.c:957` is inside the `depth > 0` branch only). The strongest
+ engine keeps typhoon's invariant. Combined with 10.9a's arithmetic
+ (+0.017 root ply), two independent lines say drop this idea.
+4. The 10.9b hang-bucket-9 pruning lead is unaffected by any of the above
+ and remains the best-evidenced pruning target (3.3% of eligible tries,
+ observed FH 0.0112%, 44x safer than hang bucket 0).
+
+## 10.11. Where the branching gap actually comes from, and the prioritized backlog (2026-10-03)
+
+### 10.11a. Crafty's r advantage is move-list TRUNCATION, not reduction grading
+
+Settling the "should we grade reduction magnitude harder" question for good.
+Crafty's entire LMR adjustment, `search.c:579-581`:
+
+```c
+reduce = LMR[Min(depth, 31)][Min(order, 63)];
+if (reduce && (pv_node || extend))
+ reduce--;
+```
+
+One decrement. No history, no improving, no node-type, no per-move signal --
+and it reaches r = 1.74 against typhoon's 2.17. **So adjustments cannot be
+where their leanness comes from.** (Stockfish does have a dozen such
+adjustments; Crafty does not, and Crafty is still far leaner than typhoon.
+This retires the "typhoon's candidate is SF's base with no adjustments"
+theory floated earlier in this session, and matches Scott's report of five
+prior LMR attempts with assorted factor-mucking, none of which paid off.)
+
+What sits immediately ABOVE those lines in Crafty's move loop does the work:
+
+```c
+if (depth < FP_depth && !check && MaterialSTM(wtm) + FP_margin[depth] <= alpha && !pv_node) {
+ tree->moves_fpruned++; break; // abandons ALL remaining moves
+}
+if (order > LMP[depth] && depth < LMP_depth && !pv_node && !check && ...) {
+ tree->moves_mpruned++; break; // abandons ALL remaining moves
+}
+```
+
+Measured in one `sd 8` sample: `fp=10.0K mcp=10.3K` against `nodes=65688` --
+**~20,000 move-list truncations in a 66,000-node search.** typhoon has EFP (a
+per-move `continue`) and a disabled LMP, and **no truncation mechanism at
+all**. That is the structural difference behind the 4.7-ply branching term
+in 10.10d.
+
+### 10.11b. "We are holding the 4D table wrong" -- the specific sense
+
+The table is currently consulted for reduction ADMISSION at a threshold that
+vetoes **2.3% of the cell space** (733 of 21,780 cells read >= 500, and 224 of
+those are in the sentinel'd dumb bucket 1). Section 8 (original) removed it
+from magnitude; section 6 (original) demoted the dynamic half to extras-only.
+Step by step a surface with a ~28,000x dynamic range in predicted FH% was
+reduced to making a near-binary decision that fires on ~10% of traffic.
+
+Its dynamic range only earns anything where the decision's stakes are
+correspondingly asymmetric:
+
+- A **reduction** is recoverable and its payoff is a ply or two off one
+ subtree. Fine-grained risk estimates are wasted there -- and 10.8 measured
+ exactly that: table-driven magnitude 0.224 mean extra plies vs the flat
+ `log(d)*log(m)` table's 0.244. A wash.
+- A **prune** is irreversible and its payoff scales with SUBTREE SIZE
+ (~20,000 nodes at depth 6 vs 30-100 near the leaf). That is where
+ "0.0112% observed FH across 2.9M tries, 44x safer than baseline" is worth
+ having, and where Crafty operates blind on depth-and-movecount alone.
+
+**So: Crafty gets r = 1.74 by truncating move lists on a crude rule, and we
+have a measured FH surface they do not. Holding the table right means using
+it to decide WHERE TO TRUNCATE.**
+
+### 10.11c. Scott's dimension argument: a sound derivation of Crafty's `break`
+
+Of the four axes, only **dumbness** is genuinely move-specific:
+
+| axis | scope |
+|---|---|
+| gap to alpha | node-level constant |
+| remaining depth | node-level constant |
+| lateness (move number) | move-specific, but MONOTONE -- every later move is at least as late |
+| dumbness (hang cp) | move-specific, varies arbitrarily |
+
+Therefore: if the table **marginalized over dumbness** reads "never fails
+high" for this (depth, gap, late) cell, that read applies at least as
+strongly to every remaining move at the node, because two axes are fixed and
+the third only worsens. That justifies abandoning the rest of the move list
+-- Crafty's `break`, derived from our own data instead of assumed.
+
+**This does NOT contradict the 2026-09-27 dumb-axis-pooling bug**, and the
+distinction is subtle enough to be worth stating: that bug was
+`ConfirmedZeroFailHigh` using the dumb-SUMMED marginal to vouch for a
+*specific* move sitting in a rare, dangerous dumb bucket -- unsound, because
+the cell's zero was produced by ordinary moves dominating its volume while
+the rare queen-hanging move had never been tested on its own. The argument
+here runs the opposite direction: it uses the pooled read to vouch for the
+**dominant** population. Since dumb=0 is 82.3% of all tries, a pooled rate
+of ~0 mathematically requires the dumb=0 subpopulation to be ~0 too, so
+pooling is CONSERVATIVE for this purpose.
+
+Preferred implementation (Scott's): per-move `continue` rather than one
+`break`, so the move-specific dumbness veto still applies to the move in
+hand. Nearly the same node savings, none of the blanket assumption.
+
+### 10.11d. The near-leaf depth ceiling is backwards and should be deleted
+
+`ShouldLatePrune`'s `LMP_MAX_REMAINING_DEPTH <= 6*ONE_PLY` was inherited
+from Crafty, where a near-leaf restriction IS the whole risk model because
+they have no table. Our own table shows predicted FH% FALLING ~25x from
+depth bucket 0 to bucket 11, so the ceiling permits pruning where risk is
+highest and forbids it where risk is lowest (already documented in
+section 7 (original), never patched).
+
+The cost of a wrong prune does rise with remaining depth -- which is what a
+rule-of-three CONFIDENCE BAR is for, not a depth cutoff. Per Scott: pruning
+a move with depth 11 under it is fine *if* the static read says "no way" and
+the dynamic vetoes are silent; benefit rises exponentially with remaining
+depth while measured risk falls.
+
+### 10.11e. Per-cell sample counts are available NOW -- no re-collection
+
+`joint_table_collection/pickles/raw_2026-10-02.pkl` survives and holds
+per-cell `(fh, tries)` for all 20,075 populated cells; `grand_tries` =
+690,062,068,603 over 56,530 positions. Running SPEC.md's rule-of-three bars
+(`N ~= 3 * applications_per_game / target_misses_per_game`, ~4,800
+applications/game):
+
+| miss budget | required N | zero-FH cells clearing it |
+|---|---|---|
+| 1 game in 20 | 288,000 | **43** |
+| 1 game in 100 | 1,440,000 | **9** |
+| 1 game in 1000 | 14,400,000 | 0 |
+
+This reproduces section 7 (original)'s "43 cells" lead exactly and adds the
+1/100 tier. **The blocker named in section 7 -- that `g_uJointFHThousandths`
+ships only the rate, not the sample count behind it -- is a
+`splice_joint.py` + `data.c` change against an existing pickle, not a new
+collection run.**
+
+### 10.11f. Prioritized backlog
+
+**1. HIGHEST IMPACT -- deep pruning via the table (10.11b/c/d/e).**
+Rationale: Crafty's 4.7-ply branching advantage is demonstrably truncation
+(10.11a); typhoon has no truncation; the payoff is exponential in remaining
+depth; the data exists; 43 cells already clear a 1/20-games bar. Closing r
+from 2.17 toward ~1.9 is worth ~1.6 ply ~= ~150 Elo at the 10.10a yardstick
+-- an order of magnitude more than anything else on this list.
+Steps: (a) ship per-cell tries via `splice_joint.py`/`data.c`; (b) build the
+prune-specific shadow-verification harness FIRST, with the
+`NEUTRALIZE_NULL_BAILOUT` fix applied from day one (SPEC.md's agreed design
+requirement #4); (c) re-point `ShouldLatePrune` at a rule-of-three-cleared
+sample bar + dumb-marginalized monotonicity criterion, deleting the depth
+ceiling; (d) keep the dynamic per-move-identity veto (requirement #5).
+
+**2. CHEAP AND UNATTENDED -- HEAD vs HEAD with LMR disabled.**
+The control condition 25 years of this work has lacked. The 10.10a yardstick
+measures what TIME-bought depth is worth; nothing has ever measured whether
+HEAD's own flat -1 ply LMR converts. If removing it costs 0.5 ply and ~45
+Elo, reduction-bought depth does convert for HEAD and the candidate broke
+something specific. If it costs 0.5 ply and ~5 Elo, reduction-bought depth
+has never been worth anything in this engine and the answer is LESS
+reduction, not better reduction. One build, one match, runs while other work
+proceeds. Use `SPRT_ARGS="--sprt --elo0 0 --elo1 15"`.
+
+**3. DURABLE INFRASTRUCTURE -- the move-agreement test.**
+Replaces curated-suite solve counts (shown to be noise at these sample sizes
+in 10.6) with a metric that has a usable signal-to-noise ratio:
+ - take ~300 real-game positions (`tests/cross_sectional_1.ep_`);
+ - build an ORACLE best move per position -- ideally Crafty at high depth,
+ independently implemented and cheap for them;
+ - run each candidate at a fixed `sn` budget; record % agreement with oracle.
+Every position contributes a signal (not just the ~20% that are tactical
+puzzles with a known answer), there is no dependence on EPD solution
+correctness, and 300 positions gives a binomial SE of ~2.9%.
+Immediate use: at EQUAL node budget, does the LMR candidate pick the oracle's
+move less often than HEAD? Yes => its extra nominal depth costs real decision
+quality. No => the depth was simply never worth anything (a different problem
+with a different fix). Refinement: score both moves with the oracle and only
+count a disagreement when it prefers its own by a real margin.
+
+**4. SMALL BUT CLEAN -- drop the six negligible extension sources.**
+From a real `sd 15` dump: `zug` fired **9** times, `endg` 352, `mult+` 1,410,
+`recap` 10,339, `sing` 10,296, `q+` 35,427 -- against the check extension's
+1,483,277. These cannot affect anything at those volumes and need no gate;
+just confirm grant counts go to zero and node counts do not move. Weighted
+by ply amount the check extension is **~78% of all extension spend**; every
+other source combined is ~0.35 ply ~= a ~33 Elo ceiling, and individually
+3-9 Elo each -- unmeasurable by self-play (~25,000 games to separate 5 Elo
+from 0). So test the mid-tier (threat + bm + 1mv + !kmvs + pawn) as ONE
+bundle, not five matches.
+
+**5. SCREEN-ONLY -- check-extension magnitude variants.**
+Non-negotiable as a mechanism (Scott: prior attempts at SEE-gating caused
+sacrifice-blindness; "king moves out" variants exploded the branching
+factor). But the AMOUNT and gate are open: 3/4 -> 1/2 ply; material gate ->
+Crafty's SEE gate; 3/4 -> 1 full ply (Crafty's amount -- extensions are not
+automatically bad). ~4 minutes each on the C/r sweep
+(`/tmp/typhoon/cr_measure/run.sh`). Note the 3/4-ply SEE>=0 branch is
+700,479 grants = 2.1M of ~2.9M check quarter-plies, i.e. 57% of ALL
+extension spend -- that is the branch worth screening, not the 1/4-ply one.
+
+**6. DEAD -- do not revisit.**
+ - *LMR magnitude grading / per-factor adjustment*: five prior attempts,
+ plus 10.8 (table vs flat is a wash) plus 10.11a (Crafty has one
+ decrement and is leaner anyway).
+ - *Clamp removal / reducing into qsearch*: 10.9a arithmetic (+0.017 root
+ ply) plus Stockfish flooring reduced depth at 1 ply
+ (`search.cpp:1370`) while getting its leanness from separate explicit
+ pruning.
+ - *SEE<0 check extension removal*: measured Delta-r = -0.004 (median over 12
+ positions), a ~3 Elo ceiling. Also independently reported to cause
+ sacrifice-blindness in earlier work. The counter-based estimate that
+ sized this at 21% of extension spend was wrong by ~20x because
+ `uCheckByQuarterPly[1]` pools the SEE<0 and low-enemy-material branches
+ -- **screen with Delta-r, never size an extension from grant counts.**
+ - *Attacking qsearch width to close the Crafty gap*: 10.10d -- `C` fits
+ LOWER for typhoon (~287) than Crafty (~448), and `C` sits inside the
+ logarithm, so a 20% change is worth ~0.3 ply.
+
+## 10.12. The control experiment, the conversion-rate model, and why flat LMR is hard to beat (2026-10-03, later)
+
+### 10.12a. LMR vs no LMR -- the control condition 25 years of this work never had
+
+`DISABLE_LMR` (GNUmakefile) makes `GetLMRReduction` return 0 before consulting
+any signal. Two binaries from the SAME tree, differing only in that flag, run
+head-to-head with NO SPRT (measuring a quantity, not testing a hypothesis --
+same reasoning as the time-odds calibration in 10.10a). `noLMR` is the
+baseline, so a positive score means LMR helps.
+
+| games | score | Elo | depth (withLMR vs noLMR) |
+|---|---|---|---|
+| 50 | 0.550 | +35 +/- 69 | 13.75 vs 12.59 (+1.16) |
+| 90 | 0.550 | +35 +/- 51 | 13.96 vs 12.79 (+1.17) |
+| 110 | 0.559 | +41 +/- 46 | 13.72 vs 12.57 (+1.15) |
+| 150 | 0.553 | +37 +/- 40 | 13.59 vs 12.43 (+1.16) |
+| 180 | 0.575 | +53 +/- 36 | 13.51 vs 12.36 (+1.15) |
+| 210 | 0.569 | **+48 +/- 34** | 13.54 vs 12.35 (+1.19) |
+
+**LMR is worth roughly +40-50 Elo and buys ~1.17 ply.** Remarkably stable
+across six checkpoints. The depth delta is the largest and steadiest of any
+candidate in this document's history.
+
+This settles a question nobody had ever asked: LMR *does* earn its keep here.
+Scott's prior ("LMR has never earned its keep in this engine") was wrong, and
+so was the stronger form of the author's "reduction-bought depth is debased"
+thesis.
+
+### 10.12b. The conversion-rate model: diminishing returns, not a flat rate
+
+Pulling every Elo-per-ply measurement together:
+
+| mechanism | what it buys | measured rate |
+|---|---|---|
+| **LMR vs NO LMR** (the FIRST reduction) | +1.17 ply | **~40 Elo/ply** |
+| graded LMR *on top of* flat LMR | +0.65 ply | ~8 Elo/ply |
+| pruning (`PRUNE_LIVE=10`) *on top of* graded LMR | +0.88 ply | ~8 Elo/ply |
+| 2x time (NEW resource, not a transfer) | +0.90 ply | ~95 Elo/ply |
+
+That is a decreasing-marginal-returns curve, and it is a better model than the
+flat "~8 Elo/ply for all tree-shaping" claim made earlier in this session.
+
+**Why the earlier claim was wrong**: the graded-LMR and prune candidates were
+both measured against `head_reference`, WHICH ALREADY HAS LMR. Their +0.65 and
++0.88 ply were MARGINAL plies stacked on an already-reduced tree. 10.12a
+measures the FIRST ply of reduction, which is where the value is.
+
+**Practical calibration (Scott's framing, and it is the right one)**: there is
+not another 40 Elo available from LMR/LMP/pruning work. There is maybe 40-45
+Elo TOTAL in the whole reduction program, of which the dumb flat -1 ply
+already captures ~85%. Five failed attempts to improve on it do not mean it
+cannot be improved -- they mean expectations should be calibrated to single
+digits, and that single digits cannot be measured in affordable self-play
+(resolving +5 vs 0 needs ~10,000 games).
+
+**ALSO NOTE a yardstick error this corrects.** 10.10a divided the time-odds
+result (+86 Elo for +0.90 ply) and called it "~95 Elo per ply," then used that
+to compute ceilings all session. That division assumes depth is the ONLY
+channel through which extra nodes help. It is not: at fixed nodes, a ply bought
+by shrinking the tree is worth far less. So "95 Elo/ply" is the rate for
+TIME-bought plies specifically and must never be applied to tree-shaping
+changes. Every ceiling computed from it earlier in this document was inflated
+(e.g. `PRUNE_LIVE=10`'s ceiling was quoted as 33 Elo; corrected to ~7, and it
+measured +7 -- it hit its real ceiling exactly rather than underperforming).
+
+### 10.12c. Why a flat -1 ply is so hard to beat -- the actual mechanism
+
+The curve is already present (`_FlatTableExtraPlies` reads
+`g_iLMRQuietReduction`, dynamic.c's `log(depth)*log(moves)` table). The problem
+is WHERE it can act. Evaluating `0.7844 + log(d)*log(m)/2.4696`:
+
+| remaining depth | moves tried | formula | extra plies |
+|---|---|---|---|
+| 2 | 16 | 1.56 | **0** |
+| 3 | 16 | 2.02 | 1 |
+| 4 | 16 | 2.34 | 1 |
+| 10 | 20 | 3.58 | 2 |
+
+**At remaining depth <= 2 the curve prescribes ~1 ply -- which IS flat -1.**
+And 76% of the LMR population sits at depth buckets 2-3 (10.8). So flat is not
+a crude approximation of the curve; at the depths where the mass lives, the two
+AGREE.
+
+Measured consequence: the curve contributes mean **0.164 extra plies** over the
+LMR population, which through section 5 (original)'s damping (~1.0 extra
+ply/decision -> +0.29 root ply) is about **+0.05 root ply**. That is the
+curve's entire current contribution, and it explains the null results directly.
+
+The curve's remaining headroom is the ~11% of moves at remaining depth >= 4,
+further limited by `iMaxExtra = depth - 3` (the one-full-width-ply invariant).
+
+**Open and never cleanly tested**: curve-vs-flat in self-play. The 2212-game
+run (section "Self-play status", 2026-09-29) tested the joint-table TIER
+magnitude under the pre-unification `uLegalMoves > 5` gate, NOT section 8's
+`log(d)*log(m)` version, which was only ever validated on sd10 node counts and
+sn5M suites. An earlier draft of this section wrongly treated that null as
+having answered the question.
+
+### 10.12d. Retraction: eNodeType is too unreliable to modulate reduction
+
+10.7b and 10.9c recommended using `pi->eNodeType` to reduce more at CUT nodes
+than ALL nodes. **Withdrawn.** After fixing the node-type surprise counters
+(10.12f) the full-width rates read: ALL nodes fail HIGH 69-71% of the time,
+CUT nodes fail low 12-14%. An "ALL node" that fails high two-thirds of the time
+is not behaving like an ALL node -- the structural label (assigned purely from
+the parent's type) is close to uninformative here. Modulating reduction off a
+signal that unreliable adds complexity for noise.
+
+### 10.12e. QSearch / QSearchFromCheckNoStandPat review
+
+Reviewed for logic errors in the custom stand-pat and check-generation rules.
+
+**SHIPPED: option B -- waive the opponent-stand-pat requirement at PV nodes.**
+`fIncludeChecks`' first clause suppressed check generation whenever the
+opponent could stand pat at an ancestor. That suppression rests on a sound
+argument *for null-window nodes*: once their stand-pat has raised alpha, no
+line below that score can change what they return, so resolving mate-in-10 vs
+"drops a queen" is wasted work -- and even when their stand-pat did NOT raise
+alpha, the node is heading for a fail-low, which IS the parent's beta cutoff,
+so any score at or below alpha answers the question identically. It does NOT
+hold at a WIDE-WINDOW node, where the exact score propagates to the root and
+those two outcomes are a different PV and a different reported score -- which
+is the forced-mate-from-qsearch case this machinery exists for. Now:
+`(fCouldStandPat[FLIP(uToMove)] == FALSE) || (NODE_PV == pi->eNodeType)`,
+keeping the material requirement (an independent sanity check, not a
+suppression). Reads eNodeType, never writes it. Uses eNodeType rather than
+recomputing `(iBeta != iAlpha+1)` because iAlpha may have just been raised by
+the stand-pat, which would make a recomputed width test read narrow for a node
+that is genuinely PV.
+
+**SHIPPED: `iExtensionAmount` unit fix.** `QSearchFromCheckNoStandPat` wrote
+`uQsearchCheckExtension` (0 or 2, counting qsearch CHECK PLIES) into a field
+that is ONE_PLY-scaled everywhere else (root.c writes HALF_PLY, split.c writes
+iLMR, search.c writes iExtend) -- a latent 32x error. Benign as it stood (both
+readers only test sign) but the field is already a documented stale-state
+hazard. Now scaled; sign and all reader behavior unchanged.
+
+**SHIPPED: `pi->iEval` published** when `QSearchFromCheckNoStandPat`'s lazy
+`Eval()` actually runs. It used to stay purely local, so that ply slot kept a
+stale value. Still stale on the common in-check path where Eval is never called
+at all (no stand-pat is offered, so there is nothing to compute it for) --
+making that safe needs invalidation on entry, a separate change.
+
+**TRIED AND REVERTED THE SAME DAY: `iBestScore = MAX(iAlpha, iEval -
+iInTrouble)`.** The observation behind it was real -- the beta test lives only
+in the else-branch, so a node with `iEval >> iBeta` and hanging material
+returns a fail-low instead of cutting off. The fix was wrong twice over, and
+self-play caught it in 10 games (candidate depth collapsed to +0.05 ply vs
+head, from the +0.65 the same tree showed without it):
+
+ Work the signs. A parent searching this node passed window
+ `(-iBeta_p, -iAlpha_p)`, so `iAlpha` here == `-iBeta_p`. The parent fails high
+ iff `-V >= iBeta_p`, i.e. iff `V <= iAlpha`. So returning exactly `iAlpha` is
+ not a crude placeholder -- **it is precisely the value that hands the parent
+ its cutoff.** Returning MAX(...) instead reports a better-than-alpha value and
+ denies that cutoff (~0.45 ply of lost depth), AND asserts the node is worth at
+ least `iEval - iInTrouble` without searching anything -- a stand-pat credit at
+ a discount, exactly what the branch exists to refuse.
+
+`MIN(iAlpha, iEval - iInTrouble)` was then proposed and also rejected: it is
+depth-neutral (V <= iAlpha always, so the cutoff always fires) but it cannot
+produce the cutoff the original observation was about (MIN never exceeds
+iAlpha), and it asserts an unearned TIGHTER upper bound, which propagates as an
+inflated lower bound into the parent's `StoreLowerBound`. The original
+observation is still unfixed; a sound fix needs a bound the node has EARNED.
+Probably moot: a node with material in trouble almost always has a capture
+available, and captures sort above the `iValue <= 0` early exit, so the move
+loop usually recovers the cutoff by search.
+
+**Confirmed correct** (checked, no change needed): the qsearch-entry reset
+invariant (`search.c:573` and `command.c:739` clear `fCouldStandPat[]`,
+`uQsearchNodes`, `uQsearchDepth` and reset `uQsearchCheckDepth` before every
+drop into qsearch; the only other QSearch call site is
+`CALIBRATE_QSEARCH_FUTILITY`-only and already inside a qsearch state);
+`fCouldStandPat` scoping (each node saves/restores only its own side's flag and
+never writes the opponent's, so `fCouldStandPat[FLIP(uToMove)]` genuinely
+reflects the parent); paired `uQsearchDepth`/`uQsearchCheckDepth` increments
+with no `goto end` between any pair; and the `uLegalMoves > 0` guard on escape
+futility pruning, which correctly prevents a bogus mate score even when leading
+escapes are illegal.
+
+**The exchange-truncation mechanism**, for the record: it is NOT the futility
+margin (`iFutility = iAlpha - (FUTILITY_BASE_MARGIN_BY_SOURCE[
+pi->uLastPositionalSource] + rgiPositional[...]) - iEval` is genuinely
+eval-calibrated). It is the SEE-ordered early exit at the top of QSearch's move
+loop: moves are SEE-sorted and the first `iValue <= 0` does `goto end`,
+abandoning every remaining move. So qsearch only explores captures SEE scores
+as non-losing, and any exchange whose resolution needs a SEE-negative
+intermediate capture is outside the horizon. That cutoff is sound only to the
+extent SEE is right, and SPEC.md already documents `_WhoAttacksSquareBB`
+(see.c) having no pin/legality awareness. Three fixes with very different
+costs: make SEE pin-aware (`C`-side, ~0.3 ply for a 20% C rise, and improves
+every other SEE consumer); admit SEE<0 captures at shallow qsearch depth
+(`C`-side, bounded); extend all captures fractionally (`r`-side, exponential).
+Fix the cause on the C side before spending r.
+
+### 10.12f. Node-type surprise line: four defects
+
+Scott's question ("how can 47% of ALL nodes fail high?") was right, and chasing
+it found four separate defects in one Trace line:
+
+1. **Denominator was total nodes**, not nodes of that type.
+2. **Qsearch was included** (~84% of the tree). A qsearch node whose stand-pat
+ beats beta returns `iBestScore >= iBeta` immediately; typed ALL, that reads
+ as "ALL failed high" while being the most ordinary event in qsearch. Now
+ excluded from numerator (QSearch's copy MOVED to its own slots) and
+ denominator (`u64NodeTypeCount`, incremented in `Search()` only).
+3. **Stale node types.** Every early exit in `CommonSearchInit` returns TRUE
+ *before* `pi->eNodeType` is assigned, and the caller's `goto end` lands on
+ the counters -- which scored the node against whatever type the PREVIOUS
+ node at that ply left behind. That produced an impossible "106.2% of PV
+ nodes". Now guarded by `fNodeTypedThisVisit`.
+4. **The fail-low test used the in-place-raised `iAlpha`.** `search.c:2048`
+ does `iAlpha = iScore`, so a node that SUCCESSFULLY raised alpha ends with
+ `iAlpha == iBestScore` and `(iBestScore <= iAlpha)` reads true -- every
+ success counted as a fail-low. The PV row read 99.1%. Now uses
+ `iInitialAlpha`, which already existed for exactly this distinction
+ (`search.c:2105`).
+
+Also added, per Scott's QNODE suggestion applied to the counters rather than
+the enum: `u64QNodeTypeCount`/`u64QNodeSurprise`, plus a **PVS identity
+cross-check**. The identity: a CUT node failing low returns `score <= alpha_C`,
+so its parent (an ALL node) sees `-score_C >= beta_P` and fails HIGH, and that
+parent cannot have had an earlier child fail low (it would have cut off first)
+-- so #(CUT fails low) and #(ALL fails high) should run ~1:1. Counting qsearch
+separately brought the measured ratio from 3.6 to **1.86**. The residual means
+~46% of ALL-node fail-highs come from nodes resolved with NO child fail-low at
+all: qsearch stand-pat cutoffs, interior-node recognizer/EGTB fail-highs, and
+null-move cutoffs (including the Tord Romstad bailout, which fabricates a score
+and searches nothing). The NODE_TYPE enum and every behavioral consumer (all of
+which only test against NODE_PV) are untouched.
+
+### 10.12g. PERF_COUNTERS cleanup -- the standard profile was carrying investigation debris
+
+An audit for real WORK (not counter increments) inside `#ifdef PERF_COUNTERS`
+found five sites in three categories. All three were fixed.
+
+**Category 1 -- three `rdtsc` calls leaking into the standard build.**
+`eval.c`'s king-safety hash probe bracketed itself with
+`SystemReadTimeStampCounter()` on entry and exit, in one of the hottest paths
+in the engine; `rdtsc` is ~20-30 cycles and partially serializing. An
+`EVAL_TIME=1` flag already exists for exactly this ("make a version that counts
+cycles spent in eval") and already guards the other **32** rdtsc sites in that
+file -- these three were the only leaks. Moved to EVAL_TIME. (The report side
+in root.c was already EVAL_TIME-guarded, so nothing dangles.)
+
+**Category 2 -- two one-off diagnostics, deleted outright.** Both did per-
+decision lookups plus integer division at the hottest decision point in the
+search, and both had comments saying to remove them once their investigations
+concluded:
+ - `u64HeadStyleEligible`/`u64HeadStyleWouldReduce` (called
+ `GetMoveFailHighRawCounts`). Its investigation concluded -- it produced
+ 10.2's 92-94%-vs-70-75% admission finding.
+ - `u64FHJointHist` (called `GetMoveFailHighPctThousandths`). Also concluded --
+ it produced section 4 (original)'s bimodal-dynamic-rate finding.
+Removed the counters, the `g_Options` mirrors, the suite accumulation and BOTH
+report blocks -- a diagnostic that prints zeros forever is worse than none.
+`AttemptsBucket`/`RateBucket` went with them. Binary shrank 7,760 bytes.
+
+**Category 3 -- the 4D collection block, re-gated to FH_TABLE_COLLECTION.**
+`search.c`'s `u64JointTries`/`u64JointFH` collection calls
+`EstimateHangingMaterialCp` (real attack-bitboard work) per eligible move. It
+is COLLECTION instrumentation, and CLAUDE.md already says FH_TABLE_COLLECTION
+is deliberately kept out of the standard profile *because this measurably costs
+nps* -- this block had escaped that gating. Verified by guard inspection:
+`search.c:1637` is now `PERF_COUNTERS > FH_TABLE_COLLECTION`; the only other
+`EstimateHangingMaterialCp` call site is the prune predictor, correctly behind
+`PRUNE_VERIFY`/`PRUNE_LIVE`.
+**Bonus:** it was computing the hang bucket TWICE per move in any build with a
+prune gate (the gate needs the same value), which inflated the `PRUNE_LIVE`
+candidate's nps deficit (-6.0% vs head) and made that match mildly unfair to
+the candidate.
+
+**nps impact: NOT MEASURED.** Two attempts were made and both are worthless
+because the box was running a 32-engine self-play match. The second attempt
+produced `perf_clean` FASTER than `noperf`, which is impossible (the cleaned
+build still has every counter increment `noperf` lacks) -- proof that noise
+exceeded the effect. Also a methodology error worth not repeating: the script
+ran all of arm A, then all of B, then all of C, so a load drift masquerades as
+an effect. **Re-measure on an idle box with ROUND-ROBIN interleaved arms**,
+fixed node budget (`sn 8000000`), same position, 3+ runs each.
+
+### 10.12h. The deep-prune audit and miss verification
+
+`PRUNE_VERIFY` (GNUmakefile) -- audit-only shadow verification for the prune
+gate. It evaluates the table's prediction for every `fPruneReduceEligible` move,
+records it banded by predicted FH%, then **declines to prune** and watches
+whether the move fails high in its own ordinary search. No bespoke shadow
+`Search()` call is needed, which sidesteps the null-move-bailout fabrication
+that made `JOINT_EXTRA_VERIFY`'s first numbers 80-99.8% artifact. Banding by the
+decision variable means one run answers "is `prune if pred <= K` sound" for
+every candidate K at once.
+
+Pooled over the three curated suites (65.6M flagged decisions), measured FH%
+ran **3-19x BELOW predicted** in every prunable band -- the Wilson re-bake
+(10.10/10.11e) doing what it was for. But `cross_sectional` positions (250
+samples, the real-game distribution) came back **5-11x WORSE than the curated
+suites** in the low bands, with `measured > predicted`. The explanation is
+distributional and inverts SPEC.md's stated worry: in a tactical puzzle the
+action is in forcing moves, which gate-1 excludes by construction, so leftover
+quiet moves there really are the dregs; in an ordinary middlegame the best move
+is frequently a quiet one, landing squarely in the leftover pool. **Validating
+a prune gate on puzzles understates its risk for real games.**
+
+Verification of the 71 low-band (`pred <= 2`) cross_sectional misses against
+clean deeper searches: **27 confirmed real, 44 artifacts.** The margin column
+discriminates cleanly -- `margin ~= 0` means artifact (a score that only just
+clears a tight inherited PVS window), huge margin means a genuine forced mate.
+That inverts the author's initial guess, which had it backwards.
+
+One confirmed band-0 miss is worth keeping as a counter-example to the dumb
+axis: `6k1/p4p1p/3p2q1/2p1rP2/2P3p1/P7/1P1Qn1PK/3R1R2 w`, after `Qg6`. The
+queen only LOOKS hung -- White is losing too badly to take it (best line grabs
+the e2 knight; White is -14 at depth 19), so a move the table rates
+"obviously hangs material for nothing, never fails high" is actually winning.
+A structural mispredict that no amount of sampling conservatism catches, which
+is precisely why this harness had to exist before pruning went live.
+
+**Prune threshold measurements** (12-position C/r sweep, paired per-position
+deltas -- note the author initially reported difference-of-medians instead of
+median-of-differences, which flipped the sign on two of three arms):
+
+| threshold | share of eligible | median dr | ply @d12 | verified miss rate |
+|---|---|---|---|---|
+| `pred <= 2` | 4.3% | -0.0040 | +0.03 | ~1 in 481,000 |
+| `pred <= 5` | 16.6% | -0.0281 | +0.21 | ~1 in 360,000 |
+| `pred <= 10` | 30.6% | -0.0465 | +0.35 | ~1 in 121,000 |
+
+Band 6-10 alone carries 79% of the misses for 29% of the flagged volume, so
+`pred <= 5` keeps ~60% of the depth for ~16% of the risk. `PRUNE_LIVE=10`
+self-play: +0.88 ply, +7 Elo at 150 games (llr flat) -- which is exactly its
+corrected ~7 Elo ceiling (10.12b), so it performed as well as it possibly could
+and that is still not interesting.
+
+Also measured and worth recording: **a dynamic veto cannot save these prunes.**
+Cross-tabulating prunable-band candidates by (own-FH attempts x own-FH rate),
+11 of 23 low-band misses sit at **rate = 0** -- the per-move history agrees with
+the static table and both are wrong. A rule catching the 12 high-rate misses
+would refuse ~390,000 sound prunes to do it (~49,000 per miss prevented), the
+same failure mode that retired the `GetMoveFailHighPercentage` veto before.
+
+## 10.13. Open action items as of 2026-10-03 EOD
+
+Ordered by expected value. Items 1-2 are the only ones with a plausible
+double-digit Elo payoff.
+
+### 1. Give EFP the `dumb` axis -- the best remaining idea
+EFP is keyed on gap-to-alpha and blind to the other three. Measured over EFP's
+actual operating envelope (10.2): the **dumb axis swings FH% 15-30x** within an
+otherwise-fixed cell while **gap -- EFP's only lever -- swings ~4x** across its
+entire knight-to-rook range. EFP is reading the weakest signal and ignoring the
+strongest. And unlike every LMR/LMP experiment, this is an **accuracy** play
+rather than a depth purchase: EFP prunes the same volume either way, so it is
+not buying marginal plies at ~8 Elo/ply and nothing measured so far bounds its
+upside. It also finally gives the 4D table a consumer that uses its dynamic
+range. Caveat from section "the tactical-position gap": the first table-driven
+EFP attempt lost ECM.272 and four others and threshold-tightening rescued only
+one -- diagnosed as a training-population problem. The table has been
+re-collected and Wilson-re-baked since, so retry rather than assume.
+
+### 2. The nps gap
+Crafty measured at **5.5M nps** on a real 10-second search (an earlier 7.5M
+figure came from a 0.01s search where the timer is meaningless -- do not quote
+it) against typhoon's ~1.4M. Even discounting for Crafty counting illegal moves
+as nodes (`search.c:452` increments before the legality test), that is roughly
+4x. Time-bought depth converts at ~95 Elo/ply, so the speed term in the Crafty
+gap (~2.7 ply) is worth ~250 Elo -- the only lever on the table with that
+magnitude. Scott's profiling says abandoning the 0x88/bitboard hybrid buys
+maybe 500knps, which does not close it, and the mechanism for their remaining
+advantage is not understood.
+
+### 3. Re-measure the PERF_COUNTERS cleanup on an idle box
+Round-robin interleaved arms, `sn 8000000`, same position, 3+ runs each. Two
+prior attempts are worthless (see 10.12g). Arms are built:
+`/tmp/typhoon/candidates/typhoon_{perf,perf_clean,noperf}`.
+
+### 4. Curve-vs-flat LMR, cleanly
+Never tested in self-play (10.12c). One flag: current tree vs current tree with
+`_FlatTableExtraPlies` forced to 0. Expected ~1-3 Elo from the arithmetic, so
+self-play cannot resolve it -- measure **node count at fixed depth** instead,
+where 0.164 mean extra plies should be visible.
+
+### 5. Decide the fate of the 4D table
+Its only live shipping consumer is `LMR_STATIC_ADMIT_PCT_THOUSANDTHS`, which
+vetoes 509 of 21,780 cells (2.3% of cell space, 7.4% of traffic) -- and that
+threshold was chosen in section 6 (original) on solve-count deltas that 10.6
+showed were noise. **Cheap decisive test**: build with admission effectively
+disabled (`= 100000`) and run against current. If neutral, the table buys
+nothing in a shipping build. NOTE item 1 may make this moot by giving the table
+a real consumer -- do item 1 first.
+
+### 6. Triage six uncatalogued ECM positions into ../tests/ECM.md
+`ECM.348`, `ECM.018`, `ECM.084`, `ECM.516`, `ECM.640`, `ECM.774` all flipped in
+the 10.6 per-position diff and are not in the flaky/legit catalogue.
+
+### 7. Smaller / hygiene
+- **Strip the remaining one-off diagnostics**: `u64LMPPruned` and the
+ `JOINT_EXTRA_VERIFY` family are still wired under PERF_COUNTERS.
+- **Share the hang bucket** between the collection instrumentation and any
+ prune gate instead of computing `EstimateHangingMaterialCp` twice (10.12g).
+- **`pi->iEval` is still stale** on the common in-check qsearch path (Eval is
+ never called there). Only reader is `search.c:760`'s `iImprovement`, which
+ should never see a qsearch ply given the layering -- add an ASSERT recording
+ that invariant.
+- **`JOINT_EXTRA_TIER1/2/3_PCT_THOUSANDTHS`** are vestigial (TIER1 appears only
+ in comments, TIER3 only in the disabled LMP path).
+- **A sound fix for the `iEval >> iBeta` lost cutoff** in QSearch's
+ material-in-trouble branch is still open; both MAX and MIN forms were tried
+ and rejected (10.12e). Needs a bound the node has EARNED. Probably low value
+ -- the move loop usually recovers the cutoff by search.
+- **`ShouldLatePrune`'s depth ceiling is still backwards** (`<= 6*ONE_PLY`
+ permits pruning where the table says risk is highest). Moot while LMP is
+ disabled.
+
+### 8. Shelved deliberately
+- **`PRUNE_LIVE`/LMP**: ~7 Elo, hit its ceiling exactly, needs the table most.
+ Code stays behind its flag; do not ship.
+- **`DISABLE_LMR`**: diagnostic only. 10.12a says removing LMR costs ~40-50
+ Elo, so this is NOT a shipping candidate.
+- **Clamp removal / reducing into qsearch**: twice falsified (10.9a arithmetic;
+ Stockfish floors reduced depth at 1 ply while getting its leanness from
+ separate explicit pruning).
+- **LMR magnitude grading via the joint table**: 10.8 (table vs flat is a wash)
+ plus 10.11a (Crafty has ONE `reduce--` and is leaner anyway).
+- **`eNodeType` as an LMR signal**: 10.12d.
+- **SEE<0 check-extension removal**: measured dr = -0.004, ~3 Elo ceiling, and
+ independently reported to cause sacrifice-blindness.
diff --git a/src/lmr_testing/SPEC.md b/src/lmr_testing/SPEC.md
new file mode 100644
index 0000000..55e6c5c
--- /dev/null
+++ b/src/lmr_testing/SPEC.md
@@ -0,0 +1,1543 @@
+# EXECUTIVE SUMMARY:
+
+We are implementing late move reduction / late move pruning in a chess
+engine based on a "simple idea": measure the propensity of a given
+move at a given node in the search tree (search.c/split.c) to fail
+high using (at least) four signals:
+
+- The move's "lateness", i.e. the order it was generated by the move
+ generator code (generate.c). Because this code generates moves in
+ order with the best moves first, the only moves left at the end are
+ either quiet moves or losing captures, usually. Not very
+ interesting ones, usually. To be extra certain that the late moves
+ we are looking to reduce/prune are uninteresting, we also prohibit
+ checking moves, escapes from check, moves on PV nodes, moves with
+ extensions, killer moves, and, in general, do not prune when any of
+ several signals indicate that this "uninteresting quiet move" might
+ be more than meets the eye.
+- The move's "dumbness", i.e. does it seem to hang material? A move
+ of our queen next to an enemy pawn might be a brilliant sacrifice
+ but most of the time it's a blunder.
+- The situation at the node (position) at which the move was
+ generated, i.e. is the side to move "winning" or are things "going
+ badly"? Since our search is an alpha..beta search, the gap between
+ a static evaluation of the current node (iEval) and alpha (iAlpha)
+ indicates "how far the side to move would have to improve the score
+ at the current position in order for our move to matter (reach the
+ PV and/or fail high). The bigger this gap, the less likely a move
+ is to fail high because to do so requires a stronger move.
+- The move's remaining depth. This one is counter-intuitive but if
+ you do the work to compute the probability of a move to fail high
+ then you find that the deeper we search the move tree under that
+ move, the *less* likely the move is to fail high. This is a really
+ interesting phenomenon -- it indicates that scores (and therefore
+ PVs) returned by shallow searches are very noisy but once you search
+ 3-4-5-8-12 ply under the move, the score usually stabilizes and
+ actually diminishes the deeper you look.
+
+We call these four signals "axes" and have computed a probability table
+where, given a score for each of these signals, you can look up the
+computed probability that the move (and node) exhibiting these signals
+will fail high. This is the core of our approach.
+
+# GOAL:
+
+The goal is to reduce the size of the chess search tree that we search
+(and therefore to reduce our effective branching factor) by reducing
+the depth we search the subset of moves we believe will not fail high
+or outright pruning them (i.e. not searching under them at all). Each
+node saved (via reduction or pruning) is re-allocated to cause the
+engine to be able to search deeper on (hopefully) more interesting and
+meaningful nodes/moves elsewhere in the tree.
+
+# DANGER:
+
+When we reduce the search under a move we introduce error in the score
+returned. It can bite us in two ways:
+- The move can fail high even though we did not expect it to. If a
+ move fails high with a reduced depth search, search.c re-searches
+ it at full depth to confirm the fail high so this is not fatal; it
+ just wastes some time and double searches the tree under the move
+ in question.
+- The move can fail low even though if we would have searched it with
+ full depth it would have failed high. This is a fatal miss; we have
+ lost information here. To help defend against this, we have built a
+ version of the engine that, when it selects a move to reduce, it
+ then proceeds to run the search at full depth anyway and pays
+ attention to how many times a reduced move would have ended up
+ failing high. This is very rare but not impossible. It is, by far
+ the most dangerous thing about reducing moves.
+
+All in all, though, reducing the search tree under a weak,
+uninteresting move is less dangerous than the alternative: pruning the
+move completely. Pruning means never searching under the move at all
+-- skipping it completely. Therefore, the threshold to reduce a move
+is much less than the threshold needed to prune a move outright.
+When we are making irrecoverable decisions, we must be conservative!
+
+# Four-axis joint FH% table -- next steps, 2026-09-28
+
+See `lmr_testing/2026-09-28_four_axis_joint_table_plan.md` for the full
+plan (axis definitions, table shape/encoding, collection/bake recipe,
+consumption design). This section used to duplicate that entire file
+verbatim; removed 2026-09-30 (alongside the late-bucket fix below) so
+future edits only need to touch one copy.
+
+**Correction, 2026-09-30, carried over from the plan doc**: the table's
+`late` axis shipped-bucket mapping had an off-by-one. Raw `late`
+(`uLegalMoves` at the point it's sampled) can never be less than 2 -- the
+collection/consultation gate (`fPruneReduceEligible`, `search.c`) requires
+`uLegalMoves > 1`, and `_MoveNumBucket`'s own 1-based-index comment
+already ruled out 0. The old bucketing (`raw - 1`, real floor assumed to
+be 1) left shipped bucket 0 permanently unreachable -- a full
+`depth x dumb x gap` slice (1,452 of 21,780 shipped cells) that read the
+`0xFFFF` sentinel forever, not from insufficient collection but because
+no sample could ever land there. Fixed by shifting `raw - 2` instead (real
+floor is 2): raw 2-13 -> shipped 0-11, 14-21 -> 12, 22-31 -> 13, 32+ -> 14.
+Applied in `search.c`'s `_ShipLateBucket`, `dashboard.py`'s
+`ship_late_bucket`, and `joint_table_collection/bake_joint.py` (newly
+written -- imports `dashboard.py`'s bucket functions rather than
+redefining them). `data.c`'s `g_uJointFHThousandths` was rebaked from the
+2026-09-30 collection run with the fix in place: real coverage is
+18,775/21,780 (86.2%) populated, not the 17,444/21,780 (80.1%) the old
+bucketing would have reported on the same data. No data was discarded --
+this was a bucketing/reporting bug, not a population/gate mismatch (a
+separate concern raised and ruled out the same day, see below).
+
+# FH%-by-(depth, move, gap) table, EFP==LMP unification, and LMR as a
+# fallback -- brain dump, 2026-09-26
+
+This is a different, unrelated project from the `README.md`/
+`searchsup_GetLMRReduction.c`/`generate_counter_move_block.c` files
+already in this directory (that's the 2026-08-27 graded-LMR-curve +
+counter-move sweep, reverted to a clean baseline and stashed for
+possible later reuse). This write-up is about a same-week but
+independent effort: building a real, measured `P(fail-high)` surface
+and using it to drive EFP, and eventually LMR, instead of hand-picked
+constants or a hand-picked reduction curve.
+
+## The core idea
+
+EFP is a pruning method currently in use in the engine; we are
+considering amending it or replacing it.
+
+Historically, EFP (extended futility pruning) was Ernst Heinz's
+two-tier design: only active within ~2.75 ply of the qsearch horizon,
+gated by a hand-picked material margin (`VALUE_KNIGHT` at <=1.75 ply,
+`VALUE_ROOK` at <=2.75 ply), with a per-move `ComputeMoveScore(mv) <
+uFutilityMargin` test on top. LMP (late move pruning) didn't exist in
+this codebase at all; Crafty-style engines that have it usually key it
+on move number/depth alone, blind to how far below alpha the position
+actually looks.
+
+The thesis this session worked from: **EFP and LMP are answering the
+same underlying question** -- "is it safe to stop searching from here"
+-- just entering from different angles (EFP: "this move's static score
+already looks bad, gap-driven"; LMP: "we've tried enough moves,
+move-count-driven"). Once you have real data on `P(fail-high | depth,
+move, gap-to-alpha)`, there's no reason to maintain them as two
+separate hand-tuned mechanisms with two separate blind spots. One
+table, consulted from either direction, replaces both. See the
+conversation this file's SessionId corresponds to for the full
+back-and-forth that arrived at this; the short version is EFP was found
+to be nearly move-number-blind (a move at position 3 and a move at
+position 25 with the *same* gap get treated identically by the old
+margin test, despite real data showing a 6-12x difference in FH% between
+them at the same depth/gap), and that's what motivated building the
+joint table instead of patching EFP's old formula.
+
+## The data: what it is, exactly
+
+`f(remaining_depth, move_number, gap_to_alpha) -> P(fail-high)`,
+where:
+
+- **remaining_depth**: `uDepth / ONE_PLY`, whole plies, NOT capped at
+ the old EFP's 2-3 ply window -- deliberately extended, because the
+ data shows depth 1-2 is the *riskiest* part of the curve, not the
+ safest (FH% falls ~5x from depth 0-1 to depth 9-12, confirmed
+ independently holding move number fixed, so it's a real depth effect
+ not a move-mix artifact). EFP's old activation window covered exactly
+ the highest-risk region and left the safer, deeper region entirely
+ unpruned.
+- **move_number**: 1-based ordinal position in the move list. Also
+ measured as move-*fraction* (fraction of the move list exhausted,
+ 20 buckets of 5%) as a parallel table, since "move 15 of 18" and
+ "move 15 of 54" are different signals and raw move count doesn't
+ capture that. The fraction table exists but the raw-move-number one
+ is what's actually wired into the live lookup right now.
+- **gap_to_alpha**: `iAlpha - iEval`, the NODE's static eval deficit
+ below alpha, in centipawns, bucketed 50cp wide (`FH_GAP_BUCKET_WIDTH`),
+ from -200cp up through +1400cp+ (`FH_GAP_BUCKETS = 32`, chess.h).
+ Deliberately the cheap, already-computed node-level gap (`iEval` is
+ computed for every node regardless of EFP) rather than a per-move gap
+ built on `ComputeMoveScore` -- the costlier per-move version was
+ considered and explicitly deferred, since computing it unconditionally
+ for every leftover move (to avoid the exact censoring bias described
+ below) would add real per-move cost everywhere, not just within EFP's
+ old narrow activation window.
+
+Of the three, **gap dominates by far** -- a ~28,000x swing in FH% from
+the worst to best gap bucket (measured on real data: 28.3% at -200..
+-150cp down to 0.001% at +1300..+1350cp), versus roughly 5-12x from
+depth or move number alone. But move number and depth both carry real,
+independent signal on top of gap -- confirmed by slicing FH% at a FIXED
+gap bucket and watching it still vary 4-12x across move number, and
+still vary ~5x across depth. All three matter; none alone is sufficient.
+
+## The population the data is measured over: "gate-1"
+
+This is the single most important methodological point, and got fixed
+late after an expensive lesson (see "the first real attempt" below).
+**The table must be trained on exactly the population that will
+actually reach it in real search, not on "all leftover moves."**
+
+Note: a "leftover move" is not a capture, killer or good countermove.
+They are the quiet moves and losing captures. They are the tier of
+moves produced by generator.c with a score < GOOD_MOVE.
+
+If you train an FH-rate table on a broader population that includes
+checks, killer-adjacent moves, threat-node members, and normal
+captures -- categories that a real EFP/LMR gate would ALREADY exempt
+via cheap, separately-validated exemptions -- the resulting table is a
+systematic OVERESTIMATE of risk for the population that actually
+reaches it, because those excluded categories are (by design, since
+they earned their exemption from evidence) higher-FH-rate populations
+diluting the measurement.
+
+The "gate-1" eligibility checklist, applied both when COLLECTING the
+data and when CONSULTING the table live, is:
+
+- `fIsLeftoverMove` (sub-`GOOD_MOVE` -- not a winning/even capture,
+ killer, killer-mate, or sufficiently-evidenced countermove)
+- `!IS_CHECKING_MOVE(mv)`
+- `!IS_ESCAPING_CHECK(mv)`
+- `iExtend <= 0` (no extension already applied)
+- `!fThreat` (this node's own null-move probe didn't raise a threat
+ signal)
+- `(!IS_CAPTURE_OR_PROMOTION(mv)) || (iMoveSee < 0)` -- captures/
+ promotions excluded UNLESS losing (SEE < 0), which are admitted. This
+ was a deliberate widening from the old EFP's blanket capture
+ exemption: a losing capture is character-wise much closer to a bad
+ leftover move than a good one, and `iMoveSee` is already computed for
+ exactly this class of move at shallow depth (`uDepth < THREE_PLY`),
+ so admitting it costs nothing extra. Caveat: `iMoveSee` defaults to 0
+ (not a real SEE value, treated as "non-losing") for captures outside
+ that depth range where `GetMoveSee` never actually ran -- a
+ non-issue for the current depth-1-9 beachhead, but worth remembering
+ if the table's depth range is ever pushed further out without also
+ extending where SEE gets computed.
+
+The first data-collection pass (before this fix) trained on the
+unrestricted leftover population and showed real cells with FH% up to
+20-30% at aggressive-looking gap/move combos -- looked alarming, until
+the `fThisMoveEFPCandidate` audit mechanism (below) showed the ACTUAL
+population reaching a table lookup in real search had real FH% around
+0.1-0.15% at a "5.0% predicted" cutoff, a 30-50x gap entirely explained
+by the population mismatch, not a flaw in the gap/depth/move signal
+itself. Re-collecting on the gate-1-restricted population removed this
+bias; the retrained table's own audit numbers then matched its
+predictions almost exactly (within ~1-2%), confirmed by comparing real
+outcomes against the volume-weighted average PREDICTED value for
+exactly the flagged population -- NOT against the raw threshold used to
+flag it, which is a different, much more permissive number and an easy
+place to fool yourself (a threshold is a ceiling on what gets flagged,
+not the typical value of what actually does).
+
+## How the data is collected
+
+Three parallel counter tables live in `chess.h`'s `COUNTERS.tree`
+(mirrored into `g_Options` for suite-level reporting), all gated behind
+`#ifdef FH_TABLE_COLLECTION` (its own opt-in `gmake` flag, NOT part of
+the standard `PERF_COUNTERS` profile -- this instrumentation measurably
+costs NPS when active on every gate-1-eligible move, and there's no
+reason to pay that cost outside a dedicated collection run):
+
+- `u64LeftoverTriesByDepthMove` / `u64LeftoverFHByDepthMove` -- raw
+ move-number x depth, `[65][64]`.
+- `u64LeftoverTriesByDepthMoveFrac` / `...FHByDepthMoveFrac` -- move
+ FRACTION x depth, `[65][20]`. Watch out: `MOVE_COUNT(ctx, ctx->uPly)`
+ can be 0 when the hash move is tried before this ply's moves are
+ generated (a distinct search stage) -- skip the frac increment
+ entirely in that case, don't clamp/floor it. An earlier version used
+ a `MAXU(1, ...)` floor instead of skipping, which silently dumped
+ every such move into the "95-100% exhausted" bucket regardless of its
+ real position, corrupting it with a huge, spuriously-low-FH%
+ population -- caught by a smoke test where that one bucket had
+ 20-100x the sample count of its neighbors. **Always smoke-test a new
+ counter design on a tiny run before trusting a big collection pass.**
+- `u64LeftoverTriesByDepthMoveGap` / `...FHByDepthMoveGap` -- the full
+ 3-way table, `[65][64][32]`. This one's cell count is ~65x the other
+ two combined, and printing it in full (even just the compact
+ `FHGAPRAW` reassembly line, not a human-readable duplicate) produces
+ tens of thousands of lines per checkpoint on a real collection run --
+ budget log volume accordingly (this box's `/tmp` is a 30GB tmpfs;
+ don't let it fill during an unattended run).
+
+Reassembly across shards: each shard's raw output is greppable for a
+`FHRAW`/`FHFRACRAW`/`FHGAPRAW` line format
+(`TAG depth move [gap_bucket] fh_count tries`), one line per populated
+cell, printed at the end of a `script` run via `_PrintSuiteReport`
+(`script.c`). **Periodic checkpointing matters for anything that runs
+long enough to risk a timeout**: `ScriptCommand`'s position loop
+re-prints the full (cumulative, not delta) report every
+`CHECKPOINT_INTERVAL` (currently 5) positions, not just once at the
+true end -- found the hard way, the first version of this only printed
+once at the end and a shard that missed its `timeout` budget by 0.05
+seconds lost 100% of its data. **When reassembling from checkpointed
+output, take the LAST checkpoint block in a shard's log, never sum
+every occurrence** -- each checkpoint is a full cumulative snapshot,
+not a delta; summing multiple checkpoints double- (or 18x-) counts.
+
+A second, unrelated gotcha in the checkpoint counter itself: it used to
+key off `g_SuiteCounters.uTotal`, which is gated by `WeAreRunningASuite()`
+-- true only when the current position has a recorded `solution`/`avoid`
+move. Plain `setboard`/`go` position pools (like
+`cross_sectional_200k.ep_`, no EPD solution annotations) never
+increment `uTotal` at all, so the checkpoint silently never fired on
+that file format despite the code looking correct. Fixed with a
+separate, unconditionally-incrementing `uPositionsProcessed` counter.
+**If a checkpoint/report mechanism silently does nothing on a specific
+file format, suspect a hidden dependency on EPD solution metadata.**
+
+Position source used so far: `tests/cross_sectional_200k.ep_` -- per
+`TESTING_PLAN.md`, this pool's actual documented purpose is
+`fuzz_test.sh`'s crash-fuzzing corpus, not curated for FH-rate
+calibration specifically. No generation script for it exists anywhere
+in the repo (checked); its contents look like ordinary, plausible
+real-game positions on inspection, but that's "looks fine," not
+"verified representative" -- worth keeping in mind, and see "the
+tactical-position gap" below for why this specific pool's composition
+turned out to matter a lot.
+
+## The upcoming large sharded run
+
+Plan: run this collection sharded across many single-threaded
+(`--cpus 1` -- `split.c`'s parallel helpers are NOT wired into these
+counters, deliberately, so multi-threaded runs would silently under-
+count) processes, at real depth (the whole point of collecting deep-
+remaining-depth cells is that they're thin unless the root search goes
+substantially PAST the depth being measured -- a node with remaining
+depth R sits at ply ~(root_depth - R) from the root, and node count
+grows with ply-from-root, so `sd 6` can never produce a single
+remaining-depth-10 sample; you need `sd 16+` to make that shell
+non-trivially populated), for on the order of a week, against a large
+position sample.
+
+**Composition of that sample matters and was the subject of a real,
+expensive lesson this session (see "the tactical-position gap" below):
+mostly real-game cross-sectional positions, with a MODEST tactical
+fraction mixed in -- not a puzzle-heavy pool.** Scott's explicit framing:
+"we want an engine that plays chess, not solves tricky puzzles." A
+pool dominated by tactical puzzles would teach the table that leftover
+moves in general are riskier than they really are (puzzles are
+adversarially selected to have a non-obvious best move), producing a
+table that under-prunes in the ordinary quiet middlegame positions
+that make up most of a real game. Something like a 80/20 or 90/10
+real-game/tactical blend is a reasonable starting point to test, not a
+locked-in ratio.
+
+Practical constraints learned from the smaller collection runs already
+done:
+- Per-position cost at real depth varies enormously (34s to 300+s
+ observed at `sd 14-16` on different positions, single-threaded) --
+ don't budget wall-clock by assuming uniform per-position cost.
+ Feeding each shard a generous position POOL (more than it's expected
+ to finish) and bounding by wall-clock `timeout` rather than position
+ count means the run naturally uses whatever time is available without
+ needing to guess position count in advance.
+- The `/tmp` tmpfs on this box is shared and has been observed at 99%
+ full from OTHER processes mid-session (not our own files) -- don't
+ assume a `du`-based capacity check at the start of a run is still
+ valid hours later.
+- A stray, unrelated process (an interactive `xboard -fcp ./typhoon
+ -scp crafty` game, apparently left running from earlier work) has
+ shown up sharing the box's cores during a collection run. Worth a
+ `ps aux | grep typhoon` sanity check before assuming all available
+ cores are actually free.
+
+## What the data is FOR: EFP == LMP, LMR as a lower-risk fallback
+
+Once gate-1 is passed, the joint table gives a real `P(fail-high)`
+estimate. Two consumers, deliberately sequenced -- prune first (since
+it was the smaller, more contained change), reduction later:
+
+**EFP/LMP (unified)**: skip the move outright if predicted FH% is below
+some `epsilon_prune`. Irrecoverable, so `epsilon_prune` needs to be very
+conservative and, critically, should be picked from a real measured
+precision/aggressiveness curve (sweep several candidate thresholds
+through the SAME audit mechanism used to validate the table, plot real
+FH% and candidate volume against each), not guessed. Runtime-tunable
+via `set EFPCandidateThresholdTenths <tenths-of-a-percent>` specifically
+so this sweep doesn't require a rebuild per value.
+
+**LMR (future work, not yet touched this session beyond discussion)**:
+the SAME table, but as a reduction-amount driver instead of a binary
+skip -- `reduction_plies ~ f(-ln(predicted FH%))`, monotonically more
+aggressive the safer the cell looks. The key asymmetry that justifies a
+MORE permissive threshold for reduction than for pruning: a reduction
+is recoverable in (at least) the direction we can detect -- the
+existing "research deeper if history pruning failed" path already
+re-verifies a reduced move whose shallow search unexpectedly raises
+alpha. What it does NOT catch is the other direction: a reduced search
+that fails low, silently accepted, with nobody ever checking whether
+the FULL-depth search would have found a raise there instead. Built an
+audit for exactly this (`LMR_AUDIT` build flag, its own opt-in profile
+since it costs a real extra `Search()` call per audited candidate, not
+just a counter increment) -- for a move that got reduced and then
+failed low at the reduced depth, re-run the SAME scout window at full
+(reduction-undone) depth and check if THAT raises alpha. First real
+numbers (small sample): miss rate 4-10% at remaining depth 2, dropping
+under 1% at depth 3-4, then creeping back up to 3-7% at depth 5-6 -- a
+non-monotonic, "high-low-high" shape that wasn't the expected smooth
+decay and hasn't been investigated further yet.
+
+Both consumers, plus per-move veto signals (below), sit on top of the
+same gate-1-gated table lookup -- this is the concrete sense in which
+EFP and LMP "are the same thing now."
+
+## Per-move veto signals: two tried, both hit cliffs, no smooth middle ground
+
+The idea: once the table flags a move as a pruning candidate, consult a
+cheaper-or-not-much-more-expensive per-move signal as a final veto,
+since the table only knows the aggregate risk for this (depth, move,
+gap) CELL, not anything about this specific move's own tactical merit.
+Both signals tried this session are audit-only-first, real-pruning-
+second, and both turned out to be effectively binary rather than a
+smooth dial:
+
+1. **`ComputeMoveScore` veto, Heinz-style depth-scaled margin.**
+ Generalized the old two-tier `VALUE_KNIGHT`(1.75ply)/`VALUE_ROOK`
+ (2.75ply) activation margin to a linear scale across the whole
+ depth-1-9 range (`uEFPVetoMarginBase + (depth-1) *
+ uEFPVetoMarginPerPly`, both runtime-tunable). Swept the margin from
+ 0 up to 300/200: below ~(200-300, 100-150) nothing gets rescued at
+ all; at (300, 200) every one of 5 known-loss positions gets rescued,
+ but node count on `ecm_ringers` comes back to within 0.6% of the
+ ORIGINAL unpruned baseline -- i.e. that setting is functionally "never
+ prune," not a real middle ground. **Retired this veto** in favor of
+ (2) below; not worth resurrecting without a fundamentally different
+ margin shape (this was linear-in-depth; something with real curvature
+ might behave differently, untested).
+
+2. **`GetMoveFailHighPercentage` veto** (per-move-IDENTITY history,
+ `dynamic.c`'s `g_FailHighs` table, keyed by `mv.uMove & 0xFFFFF` --
+ near move-exact, not just piece+destination). Exempt a flagged
+ candidate from pruning if it has real, evidenced history of failing
+ high: `(uFHAttempts >= MinSamples) && (uFHPct > Threshold)`.
+ **Critical, easy-to-miss fact about this table's lifecycle**: every
+ cell resets to `0x00010001` (1 attempt, 1 fail-high = a 100% FH-rate
+ PRIOR), not zero, on `ClearDynamicMoveOrdering()` -- which fires on
+ `setboard`/`new` (`PreGameReset`'s callers), i.e. **once per position
+ in a suite run, but only once per GAME in real play, never between
+ moves.** At the most permissive setting tested (`MinSamples=1,
+ Threshold=0`), the veto protects EVERY move by default until real
+ accumulated failures (only `_RecordMoveFailure` ever touches these
+ cells during a normal search; it increments the attempts denominator
+ without touching the fail-high numerator, diluting the percentage
+ toward 0 -- reaching an integer-truncated 0% takes on the order of
+ 100+ observed failures of that exact move-identity within the
+ current search) prove a specific move safe. This is NOT "smart
+ triage using real evidence" so much as "protect nearly everything
+ by default, until proven otherwise" -- confirmed by measuring the
+ actual prune rate among flagged candidates: **70-80% of everything
+ the table flags gets vetoed and fully searched anyway**, at every
+ depth checked. That's a real, quantified NPS cost for zero node
+ savings on the majority of "candidate" evaluations.
+ Swept `(MinSamples, Threshold)` away from the extreme (1,0) setting
+ -- same cliff pattern as (1): every tested combo away from (1,0)
+ lost 4-5 of 5 known-loss rescues while only partially reducing the
+ waste rate; none beat "just tighten the table threshold with no veto
+ at all" (which alone recovers exactly 1 of 5, `ECM.272`, the one
+ case that was ever a pure threshold-calibration problem rather than
+ a population/data problem).
+
+**Open question, not yet resolved**: does this veto's real-game behavior
+degrade over the COURSE of a single game, given the table never resets
+mid-game? Live self-play match data shows the candidate's average-
+search-depth advantage over the unmodified baseline shrinking as the
+match's game count grows (within one fixed-configuration match:
++1.09 ply at 30 games, +0.57 at 60, +0.49 at 300) -- consistent with,
+though not yet proven to be caused by, staleness accumulating within
+individual games as they get longer. Needs a direct test: instrument
+depth-gain (or the candidate/actually-pruned ratio) as a function of
+MOVE NUMBER WITHIN a game, not just averaged across whole games, using
+actual game logs from a running match rather than inferring it from the
+aggregate trend.
+
+## `sd` vs `sn` vs real games -- and getting this backwards cost real time
+
+**`sd N` (fixed depth) is structurally the WRONG test for anything that
+trades tree size for depth.** At equal depth, a leaner-tree change can
+only ever cost solves relative to baseline -- it never gets credited
+for the depth it's supposed to buy, because the test held depth fixed
+instead of letting the efficiency convert into extra plies. This
+session ran an expensive round of `sd10` suite comparisons that showed
+what looked like a severe regression (14 net lost solutions across two
+curated suites) before catching that the test itself was the problem,
+not (only) the mechanism: re-run at `sn 5000000` (fixed node budget,
+the fair comparison, since it lets the efficiency actually convert into
+depth the way it's supposed to), the SAME positions and SAME build
+showed net +6 solved across all three curated suites with deeper
+average search depth everywhere. **Any test of a pruning/reduction
+change needs to specify which of these three it's using and why**:
+`sd` for "does this change the tree SHAPE at fixed effort" (the right
+question for judging LMR/EFP's aggressiveness in isolation), `sn` for
+"does this deliver more accuracy at fixed COST" (the right question
+for judging whether a leaner-tree change is worth it), real self-play
+games for "does this actually win more, accounting for everything
+neither synthetic test can capture" (time-control-driven node budgets
+that vary naturally by position, and the compounding effect of
+thousands of small decisions across a whole game rather than a handful
+of curated positions).
+
+Related, and worth its own callout: **curated ECM suites have real,
+documented noise** (see `tests/ECM.md`'s own "flaky/garbage" catalog --
+oscillating PVs, near-0.00 scores, single-line coin-flips). A ±1-2
+solve swing on a 90-position suite, or any result on the 11-position
+`ecm_ringers`, is not strong evidence on its own. Chasing a specific
+named position's recovery (the 5-position known-loss set used
+throughout this session's veto sweeps) can actively mislead: the
+0.5%-threshold configuration that achieved a PERFECT 5/5 recovery on
+that named set actually performed WORSE in full-suite aggregate (net
++2) than the 1.0%-threshold configuration it was being compared against
+(net +6) -- optimizing against a small, hand-picked set pulled toward a
+setting that was worse on the broader, still-imperfect-but-larger
+sample. The named positions are useful as a fast smoke test for "did
+this obviously break something we already know about," not as an
+optimization target.
+
+## The tactical-position gap: why this matters more than it first looked
+
+The first live-pruning attempt (table threshold 1.0%, no veto) showed a
+real, `ECM.md`-confirmed regression: legitimate tactics lost, including
+`ECM.272` -- a position independently, manually re-verified from scratch
+this same session (traced the exact sacrifice line, confirmed our own
+engine solves it cleanly and stably from depth 7, confirmed by an
+independent Crafty search converging to the same move) as a genuine,
+non-flaky tactic. Root cause, confirmed by a threshold sweep specific to
+the 5 known-loss positions: tightening the table threshold all the way
+to 0.1% (10x tighter than the default that failed) rescued exactly 1 of
+5 (`ECM.272`); the other 4 stayed wrong regardless. **That's the tell
+that it's a population/training-data problem, not a threshold-
+calibration problem** -- if tightening 10x doesn't move 4 of 5 cases at
+all, the table's own predicted value for whatever cell is involved is
+wrong, not just permissively thresholded. `cross_sectional_200k.ep_`
+(real games only) apparently underrepresents the "quiet-looking but
+critical" move shape tactical positions are specifically curated to
+contain, enough that the table confidently mispredicts it. This is the
+direct motivation for mixing a modest tactical fraction into the next,
+larger collection pass (see above) -- not to make the engine solve
+puzzles better, but because the training population needs to actually
+contain examples of the shape it's being asked to judge.
+
+## Status as of this write-up
+
+- Real pruning is OFF pending resolution of the open questions above
+ (in-game staleness, and the not-yet-attempted ECM-augmented
+ retraining). What's live right now (via `EFPCandidateThresholdTenths`/
+ `EFPFHVetoMinSamples`/`EFPFHVetoThreshold`, all runtime-settable) is
+ the most conservative configuration tested (0.1% table threshold,
+ `(1, 0)` veto), currently running as `candidate` in an ongoing SPRT-
+ style self-play match against unmodified `head` -- last checkpoint:
+ 300 games, score 0.473, llr -0.50 (inconclusive, hasn't crossed either
+ bound; trending toward parity after an unfavorable dip at 60 games
+ that looks in hindsight like small-sample noise, consistent with the
+ "ECM suites are noisy, don't over-read small samples" lesson above).
+- `FH_TABLE_COLLECTION` build flag exists and works; a real, week-long,
+ larger, tactically-augmented collection run is the planned next major
+ step, informed by everything above.
+- The `ComputeMoveScore` veto is retired (dead end, see above); the
+ `GetMoveFailHighPercentage` veto's mid-game staleness question is
+ open and needs direct instrumentation, not just inference from match
+ trends.
+- LMR (reduction) work has not actually started yet beyond the
+ `LMR_AUDIT` miss-rate measurement infrastructure and this discussion
+ of how the risk threshold should differ from pruning's -- next in
+ line once the pruning side's open questions are resolved.
+
+---
+
+# Update, 2026-09-27/28: table-driven EFP goes fully live, two real
+# bugs found and fixed, self-play verdict is "roughly even"
+
+Continuation of the above, same overall project. This section covers
+what changed from "status as of 2026-09-26" (real pruning OFF,
+conservative audit-only config) through making it genuinely live,
+finding and fixing two real bugs via self-play (not suites), and the
+honest current verdict.
+
+## Old EFP fully replaced, not unioned
+
+Early in this stretch, discovered the old margin-based EFP block and
+the new table-driven block were BOTH live simultaneously (unioned via
+a shared `fThisMoveEFPPruned` flag) -- an artifact of iterating on the
+veto experiment without reconciling the two blocks. Fixed: old EFP
+wrapped in `if (FALSE && ...)`, permanently disabled; the table-driven
+mechanism is now tested as a genuine replacement, not an addition.
+
+Also added, matching old EFP's own behavior that the table-driven
+block had been missing: explicit `NODE_PV != pi->eNodeType` exclusion,
+and the losing-capture admission switched from a depth-gated local
+`iMoveSee` (only computed for `uDepth < THREE_PLY`, silently
+defaulting to "non-losing" outside that range) to the cached
+`MVF_SEE_KNOWN`/`MVF_SEE_NONNEGATIVE` move-stack flags `generate.c`
+already sets unconditionally at generation time for every capture/
+promotion -- correct at every depth, not just near the horizon.
+
+**Also discovered `ComputeMoveScore` is a no-op for quiet/leftover
+moves** under `MOVE_SCORING_NORMAL` (always returns 0) -- meaning old
+EFP's real selectivity, historically, came ONLY from its coarse
+depth/margin gate plus the killer/FH-history exemptions, never from
+any actual per-move tactical read. Worth remembering next time
+something is attributed to "EFP's margin test was smart about this
+move" -- it wasn't; it was smart about depth alone.
+
+## Depth-scaled threshold, not a flat one
+
+A flat FH% threshold applied uniformly across the whole prune-eligible
+depth range is structurally wrong: old EFP only ever operated within
+~1-2.75 ply of the horizon, where a wrong prune costs little (a small,
+bounded subtree). This mechanism's point is to extend pruning deeper,
+where a wrong prune is far more expensive (an exponentially larger
+subtree). `_EFPThresholdForDepth()` (search.c) interpolates
+GEOMETRICALLY (not linearly -- the two endpoints span orders of
+magnitude) between `uEFPThresholdShallowHundredths` (depth 1,
+permissive -- old EFP's own real gate tolerated FH% up to the tens of
+percent at 1-2 ply and still won) and `uEFPThresholdDeepHundredths`
+(at `uEFPMaxPruneDepth`, near the table's measurement floor).
+
+## `MoveLooksDumb`: from boolean, to magnitude bucket, to the joint table
+
+A new 4th signal, alongside depth/move/gap: does this quiet move
+statically look like it hangs material? Went through several rounds:
+
+1. **First attempt**: three OR'd sub-checks (a "reversal" move, a
+ pawn-attack, an ignored-hanging/trapped-piece read). Isolation
+ testing found only the pawn-attack idea behaves as intuition
+ predicts. The other two were retired for real, understood reasons,
+ not just "didn't help": the reversal check had a genuine indexing
+ bug (compared against the WRONG ply's move, `uPly-2` instead of
+ `uPly-3`) that masked a deeper conceptual problem once fixed --
+ reversal moves measure BACKWARDS (higher FH%, not lower), because a
+ reversal responds to the opponent's own intervening move and is
+ often a genuine correct reaction, not a wasted tempo. The
+ hanging/trapped-piece check also measured backwards: `cEnprise`/
+ `cTrapped` are written by the same parent `Eval()` call that
+ produces `iEval` (the gap axis's own input) -- a real confound with
+ gap, not independent signal.
+2. **Second attempt** (kept, still live): "does this move hand
+ material away for no reason" -- unconditionally for a strictly
+ cheaper attacker (pawn attacks any non-pawn; minor attacks a rook;
+ minor or rook attacks a queen), and conditionally for a
+ queen-vs-queen "undefended" case discussed and then REMOVED below.
+ Tried and retired: an equal-value-attacker-AND-zero-defenders test
+ for minors/rooks -- structurally broken, because
+ `pos->bbMinorAttacks[uMover]`/`bbRookAttacks[uMover]` (computed
+ PRE-move) already includes the mover's own pre-move reach to its
+ destination square (that's what makes the path legal), so "zero
+ defenders" was almost never true with multiple minors/rooks per
+ side and no cheap way to tell "the piece that just left" from "a
+ different piece of the same type still defending." The QUEEN case
+ sidesteps this because there's normally exactly one queen per side
+ (gated on `FastCountBits(bbPieces[uMover][QUEEN]) == 1`).
+3. **Boolean -> magnitude bucket**: "dropping a bishop to a pawn" and
+ "dropping a queen to a pawn" were both just `TRUE` before, throwing
+ away real information (a ~28,000x FH% swing across gap buckets
+ alone says magnitude matters). `MoveLooksDumbLossCp()` now returns
+ an approximate centipawn loss; `MoveLooksDumbLossBucket()`
+ coarsens it into `DUMB_LOSS_BUCKETS` (chess.h, =6) piece-shaped
+ tiers.
+4. **A real UnmakeMove-ordering bug**, caught before it shipped: the
+ FH-side counter increment originally called `MoveLooksDumbLossCp`
+ a SECOND time, after `UnmakeMove` had already restored the board
+ and flipped side-to-move -- silently classifying a different
+ position. Produced an impossible >100%-FH cell in testing. Fixed by
+ computing the dumb bucket ONCE (before `UnmakeMove`) and reusing
+ the stored value on both the tries-side and FH-side counters.
+5. **Joint (4D) table**: built `(depth, move, gap, dumb)` jointly, not
+ just as two separate marginal cuts, to test whether "dumb"
+ compounds with "gap" rather than being redundant. Confirmed real
+ compounding (e.g. depth 3, gap bucket ~4: dumb=0 -> 0.948% FH,
+ dumb=4 (post-coarsening numbering) -> 0.008% FH, a ~120x reduction
+ at 3.2M samples) -- this justified keeping dumb as a real 4th axis
+ instead of folding it into gap.
+
+### Queen-vs-queen removed outright (2026-09-27, this session's tail)
+
+The queen-vs-queen-undefended sub-check (originally bucket 5, ~975cp
+loss) was the one case that never behaved like the rest. Direct
+analysis of the 45-minute joint-run data: marginal FH% by dumb bucket
+was 0.057%/0.072%/0.098%/0.010% for buckets 1-4, but **0.457% for
+bucket 5 -- higher than "not dumb" (0.335%) baseline**, and this
+persisted (0.0147% vs. 0.0195% baseline, still elevated relative to
+bucket 4's 0.0045%) even after controlling for the gap axis. Root
+cause suspected but NOT confirmed: `_WhoAttacksSquareBB` (see.c) has
+no pin/legality awareness, so a pinned enemy queen still "shows" as
+attacking. Rather than ship a signal nobody understands, **the
+queen-vs-queen check was deleted from `MoveLooksDumbLossCp` entirely**
+(search.c), not just excluded downstream -- "losing a rook to a pawn
+is never a good idea" is a claim we can defend; "queen attacked by
+queen looks safe to prune" turned out not to be, for reasons still
+unexplained. `MoveLooksDumbLossBucket()` still has 5 tiers (0=not
+dumb, 1-4 = increasing loss magnitude: ~200/~400-475/~675/~875cp);
+bucket 5 (975cp) is simply unreachable now. `DUMB_LOSS_BUCKETS` stays
+at 6 in chess.h so the baked tables (sized `[...][6]`) don't need
+reshaping -- the top slot is just always zero.
+
+**A bug caught during this edit, worth remembering**: the first pass
+at simplifying `MoveLooksDumbLossBucket()` collapsed the 675cp and
+875cp tiers into one bucket, based on an incorrect assumption that
+removing the queen-vs-queen check also removed the 875cp value. It
+didn't -- 875cp (a queen attacked by a pawn) is still produced by the
+generic, unconditional pawn-attack branch at the top of the function,
+which was never touched. Caught by rereading the actual reachable
+value set before trusting the "simplification."
+
+## Real pruning: Tier 1 / Tier 2, then unified into one mechanism
+
+Two designs were tried, in order:
+
+**Tier 1 + Tier 2 (retired)**: Tier 1 = percentage-curve pruning,
+capped at `uEFPMaxPruneDepth`. Tier 2 = `JointMoveNeverFailsHigh()`
+(data.c) -- prune unconditionally at ANY remaining depth when the 4D
+joint table shows a confirmed zero fail-high over a large-enough
+sample (rule-of-three statistical confidence), EXCLUDING dumb buckets
+0 (not dumb -- this table says nothing about ordinary moves) and 5
+(queen-vs-queen, even before it was removed from the signal entirely).
+Audited before trusting: found real misses at depth 3-4 even at
+`MinSamples=10000` -- one exact miss traced to a genuine, deliberate
+queen sacrifice (`4rrk1/6p1/p7/1p1Np3/1bq1P3/7Q/PPP5/1K1R2R1 w - - 3 0`,
+`Qh6`) near the enemy king, not a bug. Added `Tier2MinDepth=5` after
+confirming clean audits at depth >=5 across all three curated suites.
+
+**Unified (current, live)**: per explicit direction ("I think we
+should not have tier1 and tier2 ... prune everywhere again but use a
+graduated threshold" -- then, when the deep end of the percentage
+curve flatlined at its representable floor, "I don't want 0.01%. I
+want 0 FHs ever seen with >5k positions, >10k positions, etc"):
+
+```
+uCandDepth = uDepth / ONE_PLY
+if (uCandDepth < g_Options.uEFPMaxPruneDepth):
+ prune if PredictedFHPercentHundredths(...) < _EFPThresholdForDepth(uCandDepth)
+else:
+ prune if ConfirmedZeroFailHigh(uCandDepth, uMoveNum, gap, dumbBucket)
+```
+
+`ConfirmedZeroFailHigh()` (data.c) requires a literal zero observed
+fail-high over a sample count that DOUBLES each ply past
+`uEFPMaxPruneDepth` (base `g_Options.uZeroConfBaseSamples`, default
+5000 at the boundary depth, 10000 the next ply, 20000 the next, ...).
+**As of 2026-09-27 this reads the same 4D joint table Tier 2 used**
+(`g_uJointTries`/`g_uJointFH`), not a separate gap-only marginal --
+see "the self-play regression and its real cause" below for why that
+change was necessary.
+
+## The self-play regression and its real cause
+
+First live self-play run of the unified design (30 games): **-8=21+1,
+score=0.383, llr=-0.42** trending firmly toward reject, with `ebf`
+WORSE for candidate (3.590 vs head 3.522) and candidate searching
+SHALLOWER (12.87 vs 12.92 avg depth) at similar nps -- exactly
+backwards from what a working pruning mechanism should show. This was
+"clearly a bug, not just an aggressive tradeoff" territory, matching
+Scott's own read: "I think there's a bug in the pruning impl."
+
+An advisor consultation (mid-investigation) proposed a structural
+theory -- pruning removes move-ordering/TT information that's
+load-bearing elsewhere in the tree, and this cost scales with how deep
+the prune fires, which old EFP never paid because it stayed within
+1-2.75 ply of the horizon. That's a real, general risk (see "on
+reduction vs. pruning" below), but a direct code audit found something
+more concrete and immediately actionable: **`ConfirmedZeroFailHigh`
+was reading `g_uGapRawTries`/`g_uGapRawFH` -- the (depth, move, gap)
+marginal SUMMED OVER THE DUMB AXIS.** A cell could read "zero FH in
+5000+ tries" purely because ordinary, non-dumb moves dominated that
+cell's volume, while a rare queen-hanging move sharing the exact same
+(depth, move, gap) cell had never actually been safely tested on its
+own. Fixed by switching `ConfirmedZeroFailHigh` onto the 4D joint
+table (adding a `uDumbBucket` parameter, same table
+`JointMoveNeverFailsHigh` already used), refusing to trust a zero read
+for the queen-vs-queen bucket outright (matching Tier 2's own
+caution, at the time this bucket still existed).
+
+**Result, same match restarted after the fix (60 games total, 30 new
+under the fixed binary)**: ebf flipped from "candidate worse" to
+EXACTLY TIED (3.619 = 3.619), and average depth flipped from
+"candidate shallower" to "candidate deeper" (13.05 vs 12.81) -- the
+signature of a real fix, not noise. `g_uGapRawTries`/`FH` and the
+`GAPRAW_*` macros are now fully dead code, left in place (unused) at
+the time of this write-up but candidates for deletion.
+
+**The `g_uGapRawTries`/`g_uGapRawFH` tables were later deleted
+entirely** (2026-09-27, same session) once the gap-bucket rework below
+made keeping them around pointless.
+
+## Quantifying "how much statistical confidence is actually needed"
+
+A real question raised directly: given a per-cell "confirmed zero"
+read only bounds the true rate via rule-of-three (~3/N for N samples),
+and this gate fires many times per real game, how much does that
+matter? Worked from real counters, not guesses:
+
+- Real `sd10` counters (post-fix): ~303 actual deep-zone (depth >=6)
+ prunes per 31.3M nodes searched, i.e. ~1 per 103,000 nodes.
+- Scaled to a real blitz budget (1.5M nps x 350s ~= 500M nodes/game):
+ **~4,800 deep-zone prunes per game.**
+- Rule-of-three bound at the current sample bar (5000 at depth 6,
+ doubling deeper): expected WORST-CASE misses per game ~= 2 (not
+ thousands -- an earlier back-of-envelope estimate using a wrong,
+ much larger node-count assumption was corrected in-session).
+- To hit a "1 unlucky game in 1000" target at that per-game
+ application volume: `required_N ~= 3 * applications_per_game /
+ target_misses_per_game ~= 3 * 4800 / 0.001 ~= 14.4 million` samples
+ in the specific cell being trusted, not 5,000-40,000.
+- **Brute-force collection to reach that is not viable**: the
+ 45-minute run put ~5,000 samples into a representative deep cell:
+ getting to 15M is ~3000x more volume, i.e. ~94 days wall-clock even
+ running all 16 shards this box supports in parallel (already at the
+ ceiling -- more shards doesn't help past core count). Confirmed and
+ accepted as a dead end in-session.
+
+**2026-10-01 correction: the "1 unlucky game in 1000" target itself was
+never validated, and the whole framing here was too conservative.**
+Scott's direct pushback: nobody decided 1/1000 was the right number --
+it was picked, not derived -- and 1/100 or even 1/20 is reasonable. Two
+things compound this:
+
+1. **Not every prune mistake costs the game.** A silent miss (we prune
+ a line that would have fail-highed) only matters if that line
+ actually would have changed the game's outcome -- a strict subset of
+ "a miss occurred." The rule-of-three math above bounds the rate of
+ MISSES, not the rate of GAMES ACTUALLY LOST to a miss, so even the
+ loosened budgets below are still conservative relative to the thing
+ that actually matters (Elo).
+2. **The deeper problem: this section was optimizing the wrong
+ quantity.** It treats "minimize miss probability until some arbitrary
+ confidence bar" as the goal in itself. The actual goal is net playing
+ strength: aggregate nodes saved by pruning/reducing get reinvested as
+ extra search depth elsewhere, and depth finds tactics -- that's a real
+ benefit that has to be weighed against the real (but apparently
+ small, see "the self-play regression" and the LMR self-play match
+ above) cost of occasional misses. A `g_uJointFHThousandths`-driven
+ gate exists precisely so this tradeoff can be made *intelligently*
+ (deeper/leaner where the data says it's safe-ish) rather than via a
+ single global, hand-picked confidence bar -- but picking that bar by
+ "make misses astronomically rare" short-circuits the whole exercise
+ and defaults back to maximal conservatism regardless of what the
+ table actually knows. The open comparison against Crafty (much
+ leaner trees, routinely 7-8 plies deeper at equal node budgets) is
+ the concrete signal that this project has been on the over-cautious
+ side of that tradeoff.
+
+Recomputed the same formula at looser budgets, same 4,800
+applications/game assumption:
+
+```
+1/1000 games -> N ~= 14.4M (0 deep-zone zero-FH cells clear this, as of this run)
+1/100 games -> N ~= 1.44M (0 deep-zone zero-FH cells clear this -- largest is 1.34M, close)
+1/20 games -> N ~= 288K (16 deep-zone zero-FH cells clear this TODAY, no more collection needed)
+```
+
+**Decided direction, not yet executed**: stop treating the
+rule-of-three sample bar as the primary lever at all. Keep it as a
+floor against outright garbage (near-zero-sample cells), but the real
+experiment going forward is to try materially more aggressive
+settings on both mechanisms -- the 1/20-budget prune bar (288K, usable
+immediately) and reduce tiers picked to target real extra depth (aim
+for whole plies, not the +0.3 ply the first p75/p25/p5 self-play run
+produced) -- and read the result by playing strength and search depth
+in self-play, not by how rare a miss is in isolation. The right
+long-term framing: **the actual metric to tune against is net Elo
+(or a cheap proxy -- depth/nps/ebf movement under a fixed node budget),
+with the FH% table used to bias WHERE cuts happen, not to justify an
+arbitrarily tiny global miss rate.**
+- **What DOES help, cheaply**: coarsening the bucket grid concentrates
+ the same collected volume into fewer, denser cells -- see next
+ section. This buys real headroom without any extra collection time.
+
+## Open gap, 2026-10-01: compounding errors under reinvested depth is unmeasured and unmeasurable by anything built so far
+
+Every statistic gathered for this project so far -- raw FH%, the
+`JOINT_EXTRA_VERIFY` silent-miss rate, the rule-of-three sample bound
+above -- is a SINGLE-DECISION statistic: it characterizes the chance
+that one gate, at one node, misjudges. None of it says anything about
+what happens once that misjudgment is embedded in a continuation that
+is then searched DEEPER, specifically because the nodes saved by
+pruning/reducing elsewhere get reinvested as exactly that extra depth.
+
+Scott's framing: we have no information about what happens if we prune
+wrong ~5x/game and reduce wrong such that a reduced-depth search
+fails low where a full-depth search would have failed high, AND in
+doing so we bought ourselves ~2 extra plies per move on average --
+because searching those 2 extra plies means the SAME gate fires again,
+2 more times, deeper under the same kind of position, with more real
+tree beneath each repetition.
+
+**Why this could be qualitatively worse than the per-decision rates
+suggest, not just "the same risk, more often":** if the 4-axis model's
+misjudgments were i.i.d. noise, compounding wouldn't be very dangerous
+-- a short chain of independently low-probability errors is still
+low-probability, same as any noise process. But this project has
+already found the model has CORRELATED, structured blind spots, not
+just noise: the queen-vs-queen dumb-bucket (removed outright, see
+above -- "safe" reads that weren't, for reasons still not fully
+understood) and the null-move-bailout signature bug (silent-miss
+investigation, see above) were both systematic, not random. Reinvesting
+saved nodes as depth specifically UNDERNEATH the branches just judged
+"safe" means that if a blind spot like those exists, extra depth
+doesn't dilute it with more search -- it drills further into exactly
+the part of the tree most likely to share it.
+
+**This cannot be measured by instrumentation built so far, even in
+principle.** `JOINT_EXTRA_VERIFY`'s silent-miss rate and the rule-of-
+three bound both describe P(one gate misfires once); there is no way
+to decompose an aggregate self-play score back into "cost from
+compounding errors under reinvested depth" vs. "benefit from genuinely
+sound extra depth" -- they are inseparably confounded in that single
+number by construction. Per-node instrumentation was always going to
+be structurally blind to this, no matter how much more data is
+collected at the per-cell level.
+
+**Practical implication for the next (more aggressive) self-play run**:
+don't just read the final SPRT score. Pull the actual lost/drawn games
+under the aggressive settings and check whether losses cluster around
+a recognizable, repeated position type (a live blind spot, same shape
+as queen-vs-queen) versus being spread uniformly (ordinary noise at a
+higher-variance operating point). `SILENT_MISS_TRACE`'s existing
+FEN/move/score logging is most of the needed infrastructure already --
+it would need to stay attached through to which game a flagged miss
+occurred in (not just logged as an isolated per-node event) to make
+this check possible. Not yet built; flagging as the concrete next
+instrumentation gap before trusting a bolder self-play result as
+"just noise" either way.
+
+## Gap-bucket redesign: 32 uniform bins -> 6 piece-value-aligned bins
+
+Per direct instruction ("Why do we have 32 gap buckets?! I think they
+should be like: alpha-100, alpha-300, alpha-500, alpha-900,
+alpha-1500, worse than that"), `FH_GAP_BUCKETS` (chess.h) went from
+32 uniform 50cp-wide bins (`FH_GAP_BUCKET_MIN=-200`,
+`FH_GAP_BUCKET_WIDTH=50`) down to 6:
+
+```
+bucket 0 : gap < 100
+bucket 1 : 100 <= gap < 300
+bucket 2 : 300 <= gap < 500
+bucket 3 : 500 <= gap < 900
+bucket 4 : 900 <= gap < 1500
+bucket 5 : gap >= 1500
+```
+
+Rationale: the fine 50cp resolution was mostly wasted precision in the
+near-alpha region (highest FH%, 9%+ -- never going to pass any sane
+pruning threshold regardless of exactly which of ~6 fine bins it lands
+in) and starved precision in the tail, which is exactly where a
+pruning DECISION actually lives. A 32->6 collapse is a ~5.3x reduction
+in total cell count, i.e. a ~5.3x increase in effective samples per
+cell for the SAME collection run, for free.
+
+**`GapCpToBucket(iGapCp)` (data.c) is now the single source of truth**
+for this mapping -- previously the -200/50 arithmetic was duplicated
+inline at SIX separate call sites (two lookup functions, one
+collection increment... actually three lookup functions and three
+collection sites) with no shared helper, exactly the kind of
+duplication that drifts silently. Every lookup
+(`PredictedFHPercentHundredths`, `JointMoveNeverFailsHigh`,
+`ConfirmedZeroFailHigh`) and every collection increment (search.c, two
+sites) now calls this one function.
+
+**Both baked tables were rebuilt from scratch** using this new
+bucketing, sourced from the SAME 45-minute joint-run pickle
+(`reassembled_joint.pkl`'s `gap` marginal for the percentage table,
+`joint` dict for the 4D table) -- this also fixes a real, previously
+flagged inconsistency: the percentage table used to be baked from an
+EARLIER, smaller, differently-composed collection run
+(`reassembled_gate1.pkl`) than the joint table, meaning the two halves
+of the "one seamless mechanism" disagreed about what population they
+described. Now both come from one dataset.
+
+**Verified after rebake** (`sd10`, all three curated suites): solve
+counts essentially unchanged (one single-position flip on
+`ecm_confident_quick`, within normal suite noise), and candidate
+volume at depth 6-9 on that suite jumped from ~132/85/50/1 tries
+(before coarsening) to **4,177/4,603/3,150/170** (after) -- a ~30-60x
+increase in samples per surviving cell, exactly the intended effect,
+with real observed FH still 0.000% at every one of those depths.
+
+## Self-play verdict as of this write-up (2026-09-28)
+
+A fresh match (post queen-vs-queen removal + gap-bucket rebake)
+against `head_reference`, tracked over time:
+
+| games | score | llr | ebf (cand vs head) | avg depth (cand vs head) |
+|---|---|---|---|---|
+| 100 | 0.520 | +0.10 | 3.641 < 3.654 (better) | 13.21 > 12.93 (better) |
+| 170 | 0.521 | +0.16 | 3.679 < 3.685 (better) | 12.94 > 12.69 (better) |
+| 1000 | **0.479** | **-1.34** | 3.617 > 3.600 (worse) | 13.19 > 13.05 (still better) |
+
+**Honest read: the early (100-170 game) positive trend did not hold
+at 1000 games.** llr swung from +0.16 to -1.34 -- more than a third of
+the way to the H0=-2.94 reject boundary. Average search depth still
+favors candidate at every checkpoint (the mechanism genuinely is
+reaching deeper), but that extra depth is not converting into a score
+edge, and ebf has flipped to favor head at the larger sample. This is
+roughly consistent with nearly every measurement across this whole
+project once the concrete bugs (EFP/table union, dumb-axis pooling,
+queen-vs-queen) were fixed: **"even," not "clearly ahead."**
+
+This matches the advisor's structural argument from the mid-session
+consultation: pruning at depth ranges beyond old EFP's narrow 1-2.75
+ply removes move-ordering/TT information that's load-bearing
+elsewhere in the tree, and that cost doesn't fully disappear just
+because the individual prune decisions are well-calibrated. Fixing
+concrete bugs recovers lost ground (30-game catastrophe -> 1000-game
+parity) but does not, on its own, produce a clear win.
+
+**Decision point reached, not yet resolved**: either (a) let the
+current match run to a real statistical conclusion (llr crossing
+either bound) before drawing further conclusions from noisy
+checkpoints, or (b) accept "even" as pruning's ceiling at this depth
+range and pivot to reduction (see next section) instead of continuing
+to chase a clear win from harder pruning alone.
+
+## On reduction vs. pruning, as the likely next step
+
+Not yet implemented. The core argument for why reduction should not
+need anywhere near the ~15M-samples-per-cell bar pruning would need at
+a permissive threshold: **a reduction is self-correcting when wrong.**
+The standard PVS/LMR re-search machinery already re-verifies a reduced
+move whose shallow search unexpectedly raises alpha or fails high. A
+wrong prune is permanently lost; a wrong reduction costs one extra
+re-search. That changes the risk calculus completely and was always
+the second half of the original plan ("beat EFP at its own game and
+then use this table to layer on reduction with a less conservative
+threshold").
+
+Sketch discussed, not yet built: stack an additive `extra_reduction_
+plies` term onto whatever `GetLMRReduction()` already computes for a
+gate-1-eligible move, driven by `PredictedFHPercentHundredths`:
+
+```
+FH% >= 5% : +0 (no extra reduction)
+1% <= FH% < 5% : +1 ply
+0.1% <= FH% < 1% : +2 plies
+0.01% <= FH% < 0.1% : +3 plies
+confirmed zero, or
+ FH% < 0.01% : +4 plies
+```
+
+Clamped so total depth never goes below some floor, same clamping
+discipline `ComputeMoveExtension`/`GetLMRReduction` already use.
+Reuse the SAME gate-1-eligible population and the SAME table this
+whole document is about -- no new data collection needed to start
+this, only new search.c wiring at the LMR call site.
+
+## How to regenerate the tables (reproducibility notes)
+
+Both baked tables (`g_uFHPercentHundredths` and `g_uJointTries`/
+`g_uJointFH`, all in data.c) are derived from `.pkl` files produced by
+reassembling a sharded collection run's raw `FHGAPRAW`/`FHJOINTRAW`
+output lines (see "how the data is collected" above for the format).
+**These `.pkl` files and the reassembly/codegen scripts live under
+`/tmp/typhoon/` (scratch, per this repo's `CLAUDE.md` policy) and are
+NOT checked into git** -- if they're gone, the tables can only be
+regenerated by re-running collection from scratch. As of this
+write-up they still exist at `/tmp/typhoon/fh_joint_run/
+reassembled_joint.pkl` (the current, live source for BOTH tables).
+
+To regenerate from an existing reassembled pickle after a bucket/table
+shape change (what was actually done for the 2026-09-27 gap-bucket
+rework):
+
+1. Load the pickle (`pickle.load`) -- top-level dict with `joint`
+ (key: `(depth, move_idx[0..15], old_gap[0..31], dumb[0..5])` ->
+ `[fh_count, tries]`, note **move_idx here is already 0-based,
+ pre-capped at collection time** -- do NOT subtract 1 again) and
+ `gap` (key: `(depth, move_raw[1..63], old_gap[0..31])` ->
+ `[fh_count, tries]`, note **this one IS 1-based raw move number**,
+ a genuinely different convention between the two dicts that cost a
+ real debugging cycle to catch -- verify which convention a given
+ dict uses before writing a regeneration script, don't assume they
+ match).
+2. Re-bucket the gap axis using whatever the CURRENT `GapCpToBucket`
+ boundaries are (old bucket `g`'s underlying range is
+ `[-200+50g, -200+50(g+1))`; map each old bucket to whichever new
+ bucket contains that range -- boundaries were deliberately chosen
+ as multiples of the old 50cp bin width so this mapping is exact
+ except for the old table's final overflow bin, which has no
+ defined upper edge and needs a documented judgment call).
+3. Sum `[fh, tries]` for every old cell that maps to the same new
+ `(depth, move, new_gap, dumb)` key.
+4. Emit C initializer text matching the existing array declarations
+ exactly (dimension order, `// move N` / `// depth N` comments for
+ readability), then splice it into data.c replacing the old array
+ body between its declaration line and matching closing `};` (brace-
+ depth-tracked, not a fixed line-count assumption -- table bodies
+ are tens of thousands of lines and shift size on every rebuild).
+5. Rebuild (`gmake clean && gmake -j5 GENETIC=1 PERF_COUNTERS=1 MP=1
+ SIXTYFOUR=1`) and re-run the `sd10` three-suite sanity check before
+ trusting the result -- solve counts should be stable (+/-1 on
+ `ecm_confident_quick` is normal suite noise) and the EFP candidate
+ audit block's real-FH% should still read ~0.000% at depth >=
+ `uEFPMaxPruneDepth`.
+
+To regenerate from a FRESH collection run instead of an existing
+pickle: re-run the same sharded, `--cpus 1`, `FH_TABLE_COLLECTION`-
+gated collection process described earlier in this document (16
+shards, wall-clock-bounded via `timeout`, mixed real-game/tactical
+position pool), reassemble each shard's LAST checkpoint block's
+`FHGAPRAW`/`FHJOINTRAW` lines into the same pickle shape, then follow
+steps 2-5 above.
+
+---
+
+# Update, 2026-09-28/29: phase 2 (graduated LMR), a real measurement
+# bug found and fixed, EFP's per-move check confirmed dead, and the
+# "deep prune" idea
+
+Picks up where the previous section left off (reduction as the
+likely next step after pruning's self-play verdict landed at "even").
+This is a companion, largely independent table build -- a fresh 4-axis
+`(depth, dumb, gap, late)` joint table baked from a new 60-90 minute
+16-shard `sd 16` collection run (see
+`lmr_testing/2026-09-28_four_axis_joint_table_plan.md` for the full
+collection/bake recipe), `g_uJointFHThousandths[12][11][11][15]` in
+`data.c`, thousandths-of-a-percent resolution, `0xFFFF` sentinel below
+a 500-tries-per-cell floor. Same underlying idea as the EFP work
+above, applied to reduction instead of pruning, and building its own
+independent measurement infrastructure along the way.
+
+## The mechanism: strictly additive over `GetLMRReduction`
+
+Deliberately the smallest first experiment: HEAD's existing flat
+`-ONE_PLY` reduction (`GetLMRReduction`, searchsup.c) is left
+completely unchanged and still decides whether a move gets reduced at
+all. The table is only ever consulted to add MORE plies on top of a
+move already reduced -- never less, never on a move `GetLMRReduction`
+wouldn't have touched anyway. `EFP is left untouched too (including
+its known bug, see below) -- this round is measuring the LMR
+augmentation in isolation, on purpose.
+
+Tier thresholds (tries-weighted percentiles of the table's predicted-
+FH% distribution, computed against the ACTUAL LMR-eligible population
+-- `uLegalMoves > 5`, real `uDepth > TWO_PLY`, dumb bucket 1 excluded
+as ~97% SEE-scored losing captures/underpromotions misfiled onto the
+quiet-move axis, same lesson as the EFP work's dumb-bucket-1 exclusion
+above):
+
+```
+FH% >= p75 (0.072%) : +0 (no extra reduction, baseline -1 ply only)
+FH% < p75 : +1 ply
+FH% < p25 (0.005%) : +2 plies
+FH% < p5 (0.001%) : +3 plies (max)
+```
+
+`JOINT_EXTRA_MIN_REMAINING_DEPTH` floor: currently `ONE_PLY` (the
+original, most aggressive setting) after a round-trip described below
+-- briefly raised to `FOUR_PLY` out of caution, then reverted once the
+apparent danger was shown to be a measurement artifact, not a real
+safety problem.
+
+Main move loop (`search.c`) only, same as the EFP work -- not wired
+into `split.c`'s `HelpSearch`, for the same reason (per-node snapshot
+arrays with no path into a helper thread's own context; every
+comparison protocol in this project already runs `--cpus 1`, so this
+costs nothing in what can currently be validated).
+
+## The null-move-bailout measurement bug (the big one this round)
+
+Built `JOINT_EXTRA_VERIFY` (shadow-search verification: after a real
+reduced search fails to raise beta, re-run at
+`uNextDepth + tier*ONE_PLY` and check if THAT would have failed high --
+same idea as the EFP-side "audit before trusting" discipline
+throughout this whole project) and `SILENT_MISS_TRACE` (log FEN/
+scores/moves on every detected miss). First numbers were alarming: up
+to 1-11% silent-miss rates across various configurations -- a
+"reduced search fails low, hiding a fail-high the real search would
+have found" rate that would have meant this mechanism was unsafe.
+
+**Root cause, found via live gdb stepping through an actual flagged
+case, not guessed**: `TryNullmovePruning`'s (searchsup.c) intentional
+Tord Romstad idea --
+
+```c
+// This is an idea from Tord Romstad on ccc: if we are below a
+// history reduced node and we try a nullmove and the nullmove
+// fails low and the move that refuted the nullmove involves
+// the same piece moved in the reduced move at ply-1 then bail
+// out of the reduced depth search now and research at full
+// depth. ...
+if ((mv.cFrom == (pi-1)->mv.cTo) && ((pi-1)->iExtensionAmount < 0))
+{
+ ...
+ *piNullScore = iAlpha - 1;
+ return TRUE;
+}
+```
+
+deliberately FABRICATES a score of `iAlpha - 1` and returns `TRUE`
+instead of doing a real re-search, specifically because that
+fabricated value negates up to exactly `iBeta+1` one level up --
+correctly triggering the REAL search's own existing
+re-search-on-fail-high machinery. This is correct, intentional, and
+safe in the real search path. It is NOT safe inside our bespoke,
+one-off shadow-verification `Search()` call, which has no surrounding
+wrapper to catch that "please re-search me" signal -- our own
+verification code misread the fabricated fail-high as a genuine one.
+
+**Fix**: `NEUTRALIZE_NULL_BAILOUT` (GNUmakefile flag) -- temporarily
+zero `pi->iExtensionAmount` (save/restore) around just our own shadow
+`Search()` call, disabling only this one precondition for our own
+verification, leaving every other occurrence of the real mechanism
+(anywhere else in the tree, on any other move) completely unaffected.
+
+**Result**: true silent-miss rate, measured correctly, is ~0.00056%
+(~1 in 178,000) even at the most aggressive `ONE_PLY` floor -- the
+80-99.8% of "misses" the earlier numbers showed were this measurement
+artifact, not real danger. This is why the floor was reverted from
+`FOUR_PLY` back to `ONE_PLY`: the conservative retreat had been a
+reaction to contaminated data.
+
+**Lesson for next time a shadow/audit `Search()` call is built
+anywhere in this codebase**: any node-local mechanism that
+communicates results by fabricating a boundary score and relying on a
+CALLER a level up to interpret it correctly (this null-move bailout,
+and structurally anything similar) is a landmine for exactly this
+kind of one-off internal verification call. Check for this class of
+mechanism specifically before trusting a shadow-search-based audit's
+raw miss rate.
+
+## Genuine (post-fix) misses: two distinct classes, neither cheaply catchable
+
+Investigated the ~handful of genuine misses surviving the fix by hand,
+one at a time, using a fresh, independent `sd1` vs `sd8`-through-`sd12`
+comparison per position (NOT the `SILENT_MISS_TRACE` hash-chain
+`shadowRefutation` guess, which was separately confirmed unreliable --
+`GetPonderMove`'s hash-chain walk can read stale/unrelated entries
+since the hash table is global, not scoped to the specific
+verification call; one hand-traced "refutation" turned out to be a
+completely different line than what an independent `sd12` search
+actually found). Two real, distinct patterns found:
+
+1. **Narrow-window boundary noise, not a real miss.** One traced
+ example showed `realScore`/`shadowScore` swinging wildly
+ (`-500` to `+144`) at the trace's logged `alpha=143 beta=144` --
+ but independently re-searching the SAME position freely (`sd1`
+ through `sd12`, no artificial window) showed a stable, consistent
+ assessment the whole way (-3.5 to -3.75), no swing at all. The
+ trace's narrow inherited PVS window means `shadowScore >= beta`
+ only proves "clears this specific tight ancestor-relative
+ threshold," not "the position is actually good" -- a real but
+ mundane form of alpha-beta boundary noise, not a blind spot a
+ heuristic could detect or avoid.
+2. **Genuine horizon-effect quiet tactics.** A second traced example
+ (`3r1rk1/1pq1bppp/p7/n2R4/P1N3Q1/1P6/1B3PPP/R5K1 b - - 3 0`, after
+ White's `Qd4`) was real: `sd1` likes `Qd4` (+2.06), but `sd8`
+ through `sd12` all agree it loses to `Bb6` (a quiet, non-capturing
+ deflection attacking the queen -- `Qxb6 Qxd7` nets Black the
+ exchange). `Bb6` isn't a capture or check, so it's entirely outside
+ `sd1`'s horizon; this is inherent to any bounded qsearch
+ (captures/checks only), not a specific bug.
+
+**Tried and abandoned, per direct experimentation with
+`QSearchFromCheckNoStandPat`'s (search.c) check-extension gate**:
+loosened the `+2 uQsearchCheckExtension` conditions three times in a
+row (narrowed the inner OR-conditions, then removed the
+`CountKingSafetyDefects(pos, ...) > 4` outer gate entirely), rebuilding
+and re-running the SAME automated `sd1`/`sd8` batch comparison
+(224 unique genuine-miss positions) after each change. **All three
+variants showed no measurable difference from baseline** (median
+swing 1.35, mean ~2.71-2.73, same top-10 largest-swing list/PVs, ~20.5%
+sd1==sd8 move-agreement rate, every time) -- this specific lever
+doesn't touch the class of miss the genuine examples actually show
+(the `Bb6` example isn't a check, so a check-generation-budget change
+was never going to catch it). Reverted to the original condition set.
+Conclusion, by mutual agreement: this is a quirk inherent to low-depth
+search, not a systematic, cheaply-fixable blind spot -- punt on
+chasing it further.
+
+## EFP's `ComputeMoveScore` per-move check confirmed dead, quantitatively
+
+Separately (but directly relevant to "should the table eventually
+replace EFP"), traced `ComputeMoveScore`/`DecodeMoveValueEstimate`
+(searchsup.c) end to end and confirmed by code reading (not just
+inference) that EFP's per-move sanity check --
+
+```c
+(ComputeMoveScore(ctx, mv, (x - 1)) < uFutilityMargin)
+```
+
+-- is a tautology given the gates already passed at that point in the
+`if`: every quiet (non-capture/promotion) move decodes to exactly `0`
+under `MOVE_SCORING_NORMAL`/`MOVE_SCORING_QSEARCH_*`, and
+`IS_CAPTURE_OR_PROMOTION(mv)` is already excluded earlier in the same
+condition -- so this check can never evaluate to anything but `0 <
+uFutilityMargin`, which is already guaranteed true by the
+`(uFutilityMargin)` gate a few lines up. It contributes nothing; EFP's
+real (and only) per-move selectivity has always come from the
+killer-adjacency and `GetMoveFailHighPercentage` history exemptions
+below it, confirming and extending the earlier (2026-09-27) finding in
+this same doc that "EFP's margin test was never smart about the
+specific move."
+
+## Quantifying what EFP is blind to, using the new table
+
+Queried `g_uJointFHThousandths` directly over EFP's actual typical
+operating envelope (1-2 plies remaining, gap 300-500cp below alpha --
+the `VALUE_KNIGHT`-to-`VALUE_ROOK` margin -- leftover moves,
+move-number bucket ~6-30+):
+
+- Overall range across that envelope: ~0.000% to ~0.37%, median
+ ~0.009% (p10-p90: 0.001%-0.046%).
+- **The `dumb` axis -- EFP is completely blind to this -- swings FH%
+ 15-30x within an otherwise-fixed cell** (e.g. depth=1, gap=4,
+ late=8: `dumb=0` -> 0.032%, `dumb=5` -> 0.002%, `dumb=9` -> 0.001%).
+- **The `gap` axis -- EFP's only real lever -- only swings FH% ~4x**
+ across its ENTIRE knight-to-rook operating range (gap=3 -> 0.068%,
+ gap=5 -> 0.017%).
+- The `late` axis (partially, structurally captured by EFP's
+ `uLegalMoves > 1` gate, but not gradedly) also swings ~4-5x
+ (late=5 -> 0.041%, late=14 -> 0.009%).
+
+**The axis EFP cannot see (dumb) has a bigger effect than the one axis
+it does look at (gap).** Direct, quantified evidence that the table
+carries real information EFP structurally cannot use, independent of
+the `ComputeMoveScore` bug above.
+
+## The "deep prune" idea: cut hopeless-looking branches much higher in the tree
+
+Scott's proposal, motivated by the above: rather than (or in addition
+to) EFP's high-volume, low-value pruning right at the horizon (1-2
+plies remaining, ~30-100 node subtrees, mostly qsearch), use the same
+table to prune a MUCH smaller number of branches much higher up (6-8
+plies remaining) where a real hopeless/late/dumb-material cell reads a
+genuinely evidence-backed (500+ tries per the shipped floor) exact
+`0.0000%` observed FH rate.
+
+**Order-of-magnitude check, from real measured numbers (self-play
+`ebf`~3.6, and an observed ~84% qsearch-node share in a real `sd8`
+trace)**: full-width-nodes-times-qsearch-multiplier estimate puts a
+1-2-ply subtree at ~30-100 nodes (matches EFP's actual footprint) vs.
+~20,000 (depth 6) to ~250,000 (depth 8) nodes for a deep subtree --
+several orders of magnitude more payoff per prune decision.
+
+**Confirmed the table actually has real, evidence-backed near-zero
+cells there**: at depth=6-8, gap=10 (>=901cp behind -- roughly a queen
+or more), late=14 (very late/leftover), dumb=6-9 (materially hanging),
+observed FH% reads exactly `0.0000%`, and because every non-sentinel
+cell already cleared the 500-tries collection floor, this is a real
+"watched 500+ times, never once failed high" statement, not a guess.
+The most extreme combination (`dumb=10` AND `gap=10` together) doesn't
+even have data yet -- too rare to have cleared the floor in the
+current (60-90 minute) collection run, which is itself part of the
+argument for a bigger rebuild (see next section).
+
+**This is explicitly the highest-conviction next direction discussed
+this session** -- higher potential payoff than either the LMR
+augmentation above or an EFP patch, but also acknowledged as the
+riskiest (an irreversible prune, firing deeper in the tree where a
+wrong call costs an exponentially bigger subtree than either EFP or
+this round's LMR work ever risked).
+
+## Agreed design requirements for the deep-prune mechanism, before it's built for real
+
+1. **One unified eligibility gate for ALL pruning/reduction
+ mechanisms** (EFP, this round's LMR augmentation, any future LMP,
+ and the deep prune) -- extend the existing `fPruneReduceEligible`
+ checklist (already shared infrastructure from the earlier EFP work)
+ to explicitly cover: no checks, no escaping check, leftover moves
+ only, not a killer (previous ply or this ply), no reason to extend
+ this node, no reason the LAST move by this side was extended
+ (EFP's existing gate has this piece already), `!fThreat`, not a
+ capture unless losing, `iAlpha < +NMATE`. Rationale, per Scott
+ directly: these preconditions define "unremarkable move," which is
+ also exactly the population the table itself was trained against --
+ using a different gate at consult time than at collection time is
+ the same population-mismatch mistake the EFP work already paid for
+ once (see "the population the data is measured over" above).
+
+2. **A rebuilt, larger, deeper table before trusting "confirmed
+ zero" for anything irreversible.** Current table: 60-90 minutes,
+ 16 shards, `sd 16`. Planned rebuild: overnight or longer (8-12+
+ hours/shard), deeper (`sd 18-20`+, to populate depth 10-13 with
+ real signal instead of the coarsened overflow bucket), bigger/more
+ varied position pool specifically to get real sample volume into
+ the deep+hopeless+late cells that are currently unpopulated (e.g.
+ `dumb=10`+`gap=10` combined). Blocked for now: this box doesn't
+ have spare cores to run a 16-shard `--cpus 1` collection run
+ alongside the ongoing 12-worker self-play match (each `match_play.py`
+ worker spawns 2 single-threaded engine processes, so 12 workers is
+ already ~24 concurrent processes) -- waiting for the self-play LLR
+ to resolve (cross either SPRT bound) before starting this.
+
+3. **"Certain" means literally `0.0%` observed, at real sample
+ volume** -- not a low-but-nonzero tier, per Scott's explicit
+ framing. A follow-on, not-yet-designed refinement: how much sample
+ volume is actually enough given how many times per game this gate
+ would fire (see the earlier EFP section's "how much statistical
+ confidence is actually needed" analysis -- same rule-of-three
+ reasoning applies here and needs re-deriving for whatever
+ application rate the deep-prune gate turns out to have).
+
+4. **A prune-specific shadow-verification harness, built BEFORE any
+ real pruning, same discipline as `JOINT_EXTRA_VERIFY` above but
+ answering a different question**: not "would a deeper search of
+ this reduced move have failed high" but "would searching this move
+ to real depth at all have failed high, given we were about to skip
+ it entirely." Must apply the (now understood) `NEUTRALIZE_NULL_
+ BAILOUT` fix from day one, and must dump FEN + full details to a
+ log on every actual miss found, for direct manual investigation
+ (matching how the two genuine-miss classes above were understood) --
+ agreed explicitly: any wrong guess gets looked at and reasoned
+ about, not just counted.
+
+5. **A dynamic, per-move-identity component is needed and is
+ currently MISSING from this round's design** -- `GetLMRReduction`'s
+ own existing pattern (require the specific move's own
+ `GetMoveFailHighPercentage` history to independently agree, not
+ just the static table cell) was flagged mid-session as something
+ the deep-prune gate must also have: require both the static table
+ cell to read genuine evidence-backed zero AND this exact move's own
+ accumulated in-search history to show zero fail-highs over some
+ minimum tries, before ever pruning for real. Countermove-conditioned
+ dynamic veto was raised as a further, explicitly UNTESTED idea for
+ later, not part of the first attempt.
+
+## Self-play status as of this write-up (2026-09-29, ongoing)
+
+First live test of the LMR-only augmentation (EFP left untouched,
+including its known `ComputeMoveScore` bug) against `head_reference`,
+`match_play.py`, `--st 4`, 12 workers, SPRT (`elo0=0, elo1=5`).
+
+**Pruning/reduction heuristic settings live for this entire match**
+(`search.c` constants, unchanged for the whole run -- see "the mechanism:
+strictly additive over `GetLMRReduction`" and "tier thresholds" sections
+above for how these were originally derived):
+
+```
+JOINT_EXTRA_TIER1_THOUSANDTHS = 72 // ~p75 of LMR-slice FH%: below this, +1 ply
+JOINT_EXTRA_TIER2_THOUSANDTHS = 5 // ~p25: below this, +2 plies
+JOINT_EXTRA_TIER3_THOUSANDTHS = 1 // ~p5: below this, +3 plies
+JOINT_EXTRA_MAX_PLIES = 3
+JOINT_EXTRA_MIN_REMAINING_DEPTH = ONE_PLY // most aggressive floor, post round-trip (see above)
+```
+
+i.e.: HEAD's own baseline `GetLMRReduction` (`-ONE_PLY`) is left untouched;
+on top of it, a gate-1-eligible move whose 4D joint-table predicted FH%
+falls under 0.072% gets +1 extra ply, under 0.005% gets +2, under 0.001%
+gets +3 (capped by `JOINT_EXTRA_MAX_PLIES` and by the
+`JOINT_EXTRA_MIN_REMAINING_DEPTH` floor so combined reduction never drops
+remaining depth below one ply). EFP was deliberately left untouched for
+this round (see below).
+
+| games | score | llr | avg depth (cand vs head) | ebf (cand vs head) | nps (cand vs head) |
+|---|---|---|---|---|---|
+| 110 | 0.532 | +0.16 | 12.90 > 12.77 | 3.561 < 3.664 | 1,625,612 > 1,570,311 |
+| 290 | 0.533 | +0.45 | 13.21 > 12.80 | 3.545 < 3.651 | 1,618,609 > 1,576,194 |
+| 730 | 0.510 | +0.28 | 13.30 > 12.99 | 3.546 < 3.622 | 1,621,970 > 1,589,133 |
+| 2212 (final, killed by hand) | 0.502 | -0.19 | 13.28 > 12.99 | 3.543 < 3.615 | 1,604,237 > 1,577,331 |
+
+Trending positive early (110-730 games) but the match was killed by hand
+at 2212 games once it settled back to dead-even: `SPRT=INCONCLUSIVE
+LLR=-0.189 BOUNDS=[-2.944,2.944] ELO=+1.4+/-14.5 LOWER95=0.4812`. At this
+sample size SPRT hadn't crossed only because `elo1=5` sits inside the ~12
+Elo-wide 95% confidence interval the match had already converged to --
+another 75 hours (the printed ETA) would have bought a label, not new
+information, so continuing to let it run was judged not worth it.
+
+**Caveat added 2026-10-01**: this entire match ran under the
+pre-unification gate (`uLegalMoves > 5`, the population the p75/p25/p5
+tier thresholds above were themselves derived against). EFP/LMR were
+unified onto `fPruneReduceEligible` (`uLegalMoves > 1`) afterward, which
+widens the live population these thresholds now classify without
+re-deriving or re-testing them. So this "even, no strength gain" verdict
+is evidence about the *original* p75/p25/p5 thresholds against the
+*narrower* pre-unification population -- it says nothing yet about
+their behavior (or the current hand-adjusted 150/5/1 thresholds, see
+`search.c`'s `JOINT_EXTRA_TIER*_THOUSANDTHS`) against the current,
+wider `uLegalMoves > 1` population. Needs its own self-play run before
+treating this verdict as settled for the current code.
+
+**Verdict, Scott's read, 2026-09-30**: the mechanism *works* in the narrow
+sense that it delivers real, consistent extra depth/speed (+0.3 ply,
+~+1.7% nps, leaner `ebf`) for no loss in measured playing strength -- but
+that 0.3 ply isn't enough on its own to show up as a score edge; the
+extra depth has to be large enough to let the engine see things HEAD
+doesn't. Two candidate next steps, not yet decided between:
+1. **Find more *sound* pruning/reduction opportunities** via the planned
+ larger/deeper table rebuild (below) -- current table has real
+ low-traffic cells (sparse or unpopulated deep+hopeless+late
+ combinations) that a bigger collection run should fill in, unlocking
+ additional reduction without cost.
+2. **Tune the existing curve to be a bit more aggressive as a starting
+ point.** Scott's specific framing: the current tier thresholds
+ (`p75`/`p25`/`p5` -> `+0`/`+1`/`+2`/`+3` plies, see above) aren't
+ hurting playing strength but also aren't buying enough depth/speed to
+ realize a gain -- worth pushing slightly harder on the SAME curve
+ shape before waiting on a full table rebuild.
+Leaning toward trying (2) as a cheap, fast experiment first (doesn't need
+the rebuild or its core budget) while (1)'s rebuild runs in parallel once
+the match's ~24 `match_play.py` engine processes are confirmed dead.
+
+**The ~3-4% nps and ~0.4-ply depth advantage are almost certainly the
+SAME underlying effect measured two ways, not two independent
+confirmations.** This is a fixed-wall-clock-per-move match (`--st 4`),
+so depth reached is a direct function of average cost per node. If
+extra reduction shifts the tree's node mix toward proportionally more
+(cheap) qsearch nodes and fewer (expensive) full-width nodes -- which
+the lower `ebf` for candidate at both checkpoints is directly
+consistent with -- that simultaneously raises the blended nodes/second
+figure AND lets iterative deepening reach one more ply in the same
+time budget. Confirmed as a real, structural effect (not process
+scheduling noise): stable over an hour-plus of real gameplay across
+~24 concurrent single-threaded engine processes on a shared,
+contended box, not a short or single-sample reading.
+
+## Build hygiene note (2026-09-29)
+
+Before this self-play run started, did a full audit + clean rebuild to
+confirm the shipped binary carries none of the investigation-only
+instrumentation: `JOINT_EXTRA_VERIFY`, `SILENT_MISS_TRACE`,
+`NEUTRALIZE_NULL_BAILOUT`, `DISABLE_EFP`, `DEPTH_ONLY_EXTRA_REDUCTION`
+are all `ifdef`-gated behind their own opt-in `GNUmakefile` flags, none
+part of the standard `GENETIC=1 PERF_COUNTERS=1 MP=1 SIXTYFOUR=1`
+profile; confirmed absent from a clean build's own
+`--command ""`-printed `Make profile used:` line. `QSearchFromCheckNo
+StandPat`'s three rounds of live manual experimentation (see above)
+were confirmed reverted to their original condition set via `git diff`.
+No stray debugger hooks (`BREAKPOINT`) remained in `search.c`. Modified
+files vs. HEAD for this whole effort: `GNUmakefile`, `chess.h`,
+`data.c`, `root.c`, `script.c`, `search.c`.
+
+## Status / open questions as of this write-up
+
+- LMR augmentation: live, in self-play, inconclusive but trending
+ positive (llr +0.45 at 290 games). No further code changes planned
+ until this resolves one way or the other -- can't run the planned
+ table rebuild concurrently with self-play on this box's core count.
+- EFP: left deliberately untouched (including the confirmed-dead
+ `ComputeMoveScore` check) for this round, on purpose, to keep the
+ LMR-only test interpretable. Two candidate next steps discussed and
+ NOT yet decided between: (a) a conservative "veto EFP's prune when
+ the table thinks this move is surprisingly dangerous" patch (small,
+ reversible, directly fixes the confirmed dead-check bug), or (b)
+ skip straight to the deep-prune mechanism and let IT eventually
+ subsume/retire EFP rather than patching EFP incrementally. Leaning
+ toward (b) as the real long-term direction, but agreed the
+ incremental-experiment discipline that's served this whole project
+ well should still apply -- not committing to skipping (a) outright.
+- Deep-prune mechanism: designed in discussion (this section), not yet
+ implemented. Blocked on: (1) the overnight/longer table rebuild
+ (itself blocked on self-play finishing, core-count constrained), (2)
+ building the prune-specific shadow-verification harness, (3) the
+ unified-gate refactor, (4) deciding on the dynamic per-move-identity
+ component's exact shape.
+- Open, explicitly undecided strategic question: does the table
+ eventually fully replace EFP (one mechanism, deep+shallow both
+ handled by table-driven decisions), or do EFP (shallow, high-volume,
+ patched) and a new deep-prune mechanism (deep, low-volume,
+ table-driven) coexist as two mechanisms answering the same
+ underlying question from different depths? Deferred until real
+ numbers exist from both the current self-play match and a future
+ deep-prune experiment.