summaryrefslogtreecommitdiff
path: root/src/lmr_testing/GET_LMR_REDUCTION.md
diff options
context:
space:
mode:
authorScott Gasch <[email protected]>2026-10-04 08:17:52 -0700
committerScott Gasch <[email protected]>2026-10-04 08:17:52 -0700
commitbbdf1bda8f410df6aa9be0a38e71b7db04982198 (patch)
treee8315247f700ee94875cd6d2eb5dd81aef71395d /src/lmr_testing/GET_LMR_REDUCTION.md
parent9b28080f8b3dca4588cb2aaaf748451f45e3c626 (diff)
Dynamic move-ordering/LMR session: honest FH tracking, decoupled admission/magnitude, bugfixes, tooling
Lands the 2026-09-28..10-03 move-ordering investigation (full history in lmr_testing/*.md). Summary of what ships: LMR/move-ordering redesign (searchsup.c, dynamic.c, data.c, chess.h): - GetLMRReduction's admission decision (do we reduce at all) is now decoupled from magnitude (how much): LMR_STATIC_ADMIT_THOUSANDTHS=500 gates admission via the 4D joint fail-high table (data.c) plus a countermove-match veto; magnitude is a flat Crafty/Ethereal-shaped log(depth)*log(moves) table (g_iLMRQuietReduction), which measured as good or better than the old per-tier joint-table payout while being structurally immune to the per-bucket miscalibration that sank the tiered approach. g_FailHighs honesty fix (real 0/0 init, not a fake prior) underlies the dynamic-danger signal. - Retired NumLeftoverMovesToSelect's depth-indexed selection budget -- evidence showed the leftover pool has real findable signal a bailout was discarding. - ShouldLatePrune (overt late-move pruning) implemented but disabled at its call site -- measured net-negative at the tested threshold; kept for reference and as a base for a future hang-bucket-9-targeted retry. - PRUNE_VERIFY, PRUNE_LIVE, DISABLE_LMR: opt-in diagnostic build flags (GNUmakefile), off in the standard profile. Bug fixes found and fixed while exercising this work: - JOINT_EXTRA_TIER_COUNT 4->6 (chess.h): fixed an out-of-bounds counter write reachable at remaining depth >= 14 (unreachable at sd10, routine in real search/self-play). - Node-type instrumentation (search.c, root.c, searchsup.c): fixed four compounding defects that produced impossible percentages (e.g. 106.2% of PV nodes) -- wrong denominator, qsearch misclassified as main search, untyped early-exit nodes, and fail-low tests using the in-place-raised alpha instead of the initial one. - FH-rate unit unification (search.c, searchsup.c, dynamic.c, chess.h): GetMoveFailHigh* returned per-mille while g_uJointFHThousandths is thousandths-of-a-percent, two different units sharing a "_THOUSANDTHS" suffix. Unified to one scale, renamed to "_PCT_THOUSANDTHS". RateBucket's band edges fixed to match. - QSearch option B (search.c): waives the opponent-stand-pat requirement at NODE_PV only (sound there since the exact score propagates; not sound at null-window nodes). iExtensionAmount unit fix (was 0/2 into a ONE_PLY-scaled field). pi->iEval now published when Eval() runs. - eval_tune/match_play.py local_ebf: was reporting 0.000 in every match since 2026-09-18 (regex only matched the trailing-stats parser, not the live move-scan loop; engines were also spawned with "nopost" so root.c never emitted the lines at all). True per-ply EBF is ~2.0, not the ~3.9 every prior match report implied. - eval_tune/test_vs_head.sh: positional-arg shift ordering bug caused every no-extra-args run to silently rebuild ./typhoon with a non-standard profile and, separately, could feed a flag's value in as the candidate binary path. - PERF_COUNTERS cleanup (eval.c, search.c): moved 3 leaked rdtsc sites to the existing EVAL_TIME guard, deleted two concluded one-off diagnostics (u64HeadStyle*, u64FHJointHist/RateBucket/AttemptsBucket) and their report plumbing, and re-gated a per-move hang-material estimate behind FH_TABLE_COLLECTION instead of running it unconditionally. Infrastructure: - match_play.py time-odds support (--odds/--st-head/--st-candidate), validated against the engine seat rather than color. - Wilson-upper-bound re-bake of the joint table (data.c): cells store a confidence-aware bound instead of a raw point estimate, so a consumer can threshold on the value alone; monotone, so every consumer became strictly more conservative. Verified behavior-neutral for LMR. Known-open items (not resolved here, see lmr_testing/GET_LMR_REDUCTION.md section 10 and 2026-10-03_UNCOMMITTED_TRIAGE.md): the self-play gate on this candidate is underpowered (+5 Elo, 95% CI roughly [-6, +17]); the curated-suite solve-count deltas used earlier in the investigation turned out to be noise-dominated cancellation when diffed per position, not the clean signal they first appeared to be. Co-Authored-By: Claude Sonnet 5 <[email protected]>
Diffstat (limited to 'src/lmr_testing/GET_LMR_REDUCTION.md')
-rw-r--r--src/lmr_testing/GET_LMR_REDUCTION.md1971
1 files changed, 1971 insertions, 0 deletions
diff --git a/src/lmr_testing/GET_LMR_REDUCTION.md b/src/lmr_testing/GET_LMR_REDUCTION.md
new file mode 100644
index 0000000..5b40031
--- /dev/null
+++ b/src/lmr_testing/GET_LMR_REDUCTION.md
@@ -0,0 +1,1971 @@
+# GetLMRReduction overhaul -- session handoff (2026-10-02)
+
+**Note on filename**: this supersedes `lmr_testing/GET_LMR_REDUCION.md`
+(missing a T -- an earlier typo in this same session). That file's
+content is reproduced/updated below; don't treat the two as
+independent histories, the old one is just a mid-session snapshot.
+
+This session started by making `GetMoveFailHighPercentage`'s dynamic
+fail-high table honest, discovered that honesty alone broke LMR's
+node-savings, spent a long stretch diagnosing and redesigning the
+gate through several dead ends, and ended with a real, working
+redesign: self-play validation is pending as of this handoff (the
+user is launching it separately). This file is the full history.
+
+**Current code state** (verified by grep immediately before writing
+this): `searchsup.c`'s `GetLMRReduction` uses
+
+- `LMR_STATIC_ADMIT_THOUSANDTHS = 500` -- wide-open admission ("do we
+ reduce at all").
+- `JOINT_EXTRA_TIER1/2/3_THOUSANDTHS = 72/5/1` -- now vestigial for
+ the live magnitude decision (see section 8); TIER1 only appears in
+ comments, TIER3 is reused by `LMP_STATIC_MAX_THOUSANDTHS` for the
+ (currently disabled) LMP experiment.
+- `LMR_FH_BASELINE_THOUSANDTHS = 4` -- dynamic-danger baseline,
+ unchanged all session.
+- Magnitude ("how much extra") is now computed by
+ `_FlatTableExtraPlies(uRemainingDepth, uLegalMoves)`, which reads
+ `g_iLMRQuietReduction` (dynamic.c) -- a Crafty/Ethereal-shaped flat
+ `log(depth)*log(moves)` table that existed, fully built, completely
+ unread, since before this session started. This is the live,
+ working mechanism as of this handoff -- see section 8.
+- `JOINT_EXTRA_MAX_PLIES = 5` (cap on extra plies stacked on the base
+ `-ONE_PLY`).
+- LMP (overt late-move pruning, section 7) is implemented in
+ `ShouldLatePrune` (searchsup.c) but **disabled** at its call site
+ (`search.c:1228`, `if (FALSE && ...)`) -- measured net-negative, kept
+ for reference, not deleted.
+
+Build is clean (`gmake -j5 GENETIC=1 PERF_COUNTERS=1 MP=1 SIXTYFOUR=1`,
+no warnings) at every checkpoint described below.
+
+## 1. The honesty fix (where this started)
+
+`g_FailHighs` (dynamic.c) used to seed every move-identity to a fake
+`1 FH / 1 attempt` (100%) on `ClearDynamicMoveOrdering`, so
+`GetMoveFailHighPercentage` never returned a real 0% -- a move needed
+~9-10 *clean* observations before its decaying fake prior dropped
+below the old `<= 10%` gate. Fixed:
+
+- `ClearDynamicMoveOrdering` now zeroes the table for real (0/0).
+- Renamed to `GetMoveFailHighThousandths` (dynamic.c) -- returns the
+ rate in thousandths (0-1000), not whole percent, since the real
+ population baseline (data.c's comment: **0.400%** tries-weighted,
+ across the 500B-node joint-table collection run) needs sub-1%
+ resolution to mean anything.
+- Added `GetMoveFailHighRawCounts` (dynamic.c) -- raw FH/attempts, used
+ by diagnostics below.
+
+Also while in there: killed the stale doc-comment in `_NewCounterMove`
+(claimed depth-gated overwrites; code actually always overwrites on
+recency, confirmed by its own inline A/B-test comment -- just the
+routine description never got updated). Removed `uCounterDepth`
+(write-only, zero readers, the "read-time bonus" it was meant to feed
+was never built) and the dead `FIRST_COUNTER_MOVE`/`SECOND_COUNTER_MOVE`
+flag constants (chess.h).
+
+Added a countermove-table veto to `GetLMRReduction` itself: a bare
+match against `ctx->sPlyInfo[ctx->uPly-2].mv` (the move played
+immediately before `mv` -- **two** plies back from here, since `mv`
+has already been made by the time `GetLMRReduction` runs) withholds
+reduction, independent of generate.c's evidence-gated tier promotion.
+
+## 2. Consolidation: one function, not two files
+
+The joint-table-driven "extra reduction" logic used to live in
+search.c as post-processing on top of `GetLMRReduction`'s own baseline
+decision -- split across two files/two inconsistent dynamic checks
+purely to minimize the original diff against HEAD. Moved the whole
+decision into `GetLMRReduction` (searchsup.c):
+
+- `EstimateHangingMaterialCp`, `HangCpBucket`, `GapCpBucket`,
+ `MoveNumBucket`, `DepthBucket` are no longer file-static -- they're
+ declared in chess.h now because search.c's own joint-table
+ *collection* instrumentation (gated on `fPruneReduceEligible` alone,
+ independent of the LMR decision) still needs them.
+- `GetLMRReduction` gained three params: `bbMinorAttacksSnapshot[2]`,
+ `bbRookAttacksSnapshot[2]` (for `EstimateHangingMaterialCp`;
+ split.c's HelpSearch has no such snapshot, passes all-zero arrays,
+ which degrade safely), and `OUT ULONG *puExtraTier` (optional, feeds
+ search.c's re-search-rate calibration instrumentation).
+- search.c's move loop is now just one call + two counter increments.
+
+## 3. The surprise: depth went DOWN, not up
+
+First redesign attempt kept the *old* two-check shape (dynamic rate
+must be low AND bucket-specific, both gates, a bare 0.4%-baseline
+veto) but made it honest. Measured against `head_reference` (committed
+baseline, no joint-table mechanism at all -- its `GetLMRReduction`
+only ever applies a flat `-ONE_PLY`, old fake-prior-decay gate):
+
+- **sd10 node count went UP 2-6%** across all three curated suites --
+ the honest gate was *more* conservative than HEAD's old loose one.
+- A single position (ECM.233) pushed to sd20 showed candidate using
+ 3x HEAD's node count at the same depth (18 ply) -- undeniable, not
+ a suite-aggregate artifact.
+- A diagnostic (`u64HeadStyleEligible`/`u64HeadStyleWouldReduce`,
+ chess.h) replays HEAD's exact old gate formula --
+ `(rawFH+1)*100/(rawAttempts+1) <= 10` -- against the SAME live
+ trajectory as the real decision. Result: **HEAD-style reduces
+ ~76-77% of eligible decisions, always by exactly 1 ply.** The honest
+ design (0.4% baseline) only reduced 62-66%, but with some decisions
+ getting 2-4 plies. Average ply-saved-per-eligible-call was actually
+ *slightly higher* for the new design (0.80-0.88 vs HEAD's 0.73-0.76)
+ -- so the problem wasn't magnitude, it was **coverage**: breadth
+ (how many moves get touched at all) matters more to total tree size
+ than depth-of-cut-per-touch, and extra-tier magnitude carries a
+ re-search tax (2-3%/tier) that eats part of its own benefit. This
+ theme (coverage beats magnitude, magnitude has a tax) recurs
+ throughout the rest of the session -- see sections 8/9.
+- A pure threshold sweep on the EXTRA-tier ceiling alone (holding the
+ honest 0.4% baseline fixed) never recovered HEAD's node count, not
+ even at the old deliberate-extreme-experiment setting
+ (9000/8300/1000 thousandths) -- confirming magnitude-tuning on an
+ already-small qualified population is the wrong lever.
+- A sweep of the 0.4% *baseline* itself told a different story:
+ loosening it to 100 thousandths (10%, matching HEAD's old literal
+ ceiling) recovered coverage (75.6%) and **beat HEAD's node count by
+ 35%**. Disabling the countermove-match veto or the bucket-prediction
+ veto *individually* moved node count by only 1-10% -- noise,
+ basically. **The flat 0.4% baseline was almost the entire story**,
+ not the "smart" vetoes. This is the first sighting of the pattern
+ that eventually won: a simple, broad admission lever beats precise
+ per-decision judgment.
+
+## 4. Why the population shape matters: the FH histogram
+
+A joint histogram of `(uAttempts bucket) x (uFHThousandths bucket)` at
+the live decision point (`u64FHJointHist`, chess.h;
+`AttemptsBucket`/`RateBucket`, searchsup.c -- still live
+`PERF_COUNTERS`-only diagnostic code, not yet removed) showed, on
+confident_quick (18.7M eligible decisions):
+
+- **~71% sit at exactly 0% observed rate**, including a large,
+ well-evidenced chunk (3.29M with 100+ attempts, genuinely zero FH
+ across real volume, not a fluke).
+- **A real, separate, well-evidenced population sits durably at
+ 1-10%** -- most of that mass is in the `attempts=100+` row (713K at
+ 1-1.9%, 1.1M at 2-4.9%, 657K at 5-9.9%), i.e. **not** small-sample
+ noise. (A real, narrower small-n artifact *does* exist separately:
+ at `attempts <= 9`, rates between ~1% and ~10% are structurally
+ unreachable -- `1/9 ≈ 11.1%` is the smallest nonzero rate at 9
+ attempts -- but this is a distinct, smaller phenomenon from the
+ large well-evidenced 1-10% population above.)
+- **~7% are pure-cold** (`uAttempts == 0`) and, under the original
+ two-population design, got flatly refused any reduction regardless
+ of what the static joint table said. The static table needs NO game
+ history (keyed on depth/hang/gap/late, not move identity) -- gating
+ it behind a dynamic-history requirement threw away its whole point
+ for this slice.
+
+So the dynamic rate is **bimodal**, not unimodal around 0.4% -- the
+population average is a real summary statistic of a mixture, not
+representative of either mode.
+
+## 5. Static-first and three-population redesigns (SUPERSEDED)
+
+Two intermediate redesigns, both now superseded by section 8's simpler
+mechanism, but worth recording for what they established:
+
+- **Static-first**: flipped priority so the static bucket prediction
+ became primary ("should we reduce at all"), dynamic signals became
+ vetoes on top rather than a precondition. Established the
+ qualitative lesson (cold moves should defer to the static table, not
+ be refused outright) that survived into every later design.
+- **Three-population**: cold / warm+safe / warm+dangerous, with
+ warm+dangerous capped to base-only rather than fully vetoed. This
+ *also* established a real, lasting bug: `JOINT_EXTRA_TIER1_
+ THOUSANDTHS` was doing double duty as both the admission threshold
+ ("reduce at all") and the magnitude threshold ("+1 extra ply") --
+ decoupling these (section 6) is what unblocked real progress.
+
+**CONTAMINATION WARNING, still relevant if this file's old numbers
+ever resurface**: partway through this session,
+`JOINT_EXTRA_TIER1/2/3_THOUSANDTHS` got silently stuck at
+**9000/8300/1000** (an old deliberate-extreme-experiment setting) for
+a long stretch, because a `cp searchsup.c searchsup.c.orig_full`
+snapshot captured a sweep script's leftover state without checking it
+first. Every measurement taken during that window (the HeadStyle
+diagnostic, the FH histogram, the first static-first pass) was
+measured under that contamination. **Lesson for next time**: after ANY
+sed-based sweep script, explicitly grep the tier/baseline constants
+before trusting a `cp`'d file as a clean base for anything else.
+
+Despite the contamination, one result survived it, independently
+corroborated later: **depth gain from LMR changes is heavily damped**.
+Even at the extreme (accidentally-active) ceiling, sn5M average search
+depth only moved 0.1-0.3 ply vs HEAD. This matches a documented
+finding already in search.c's history (2026-10-01 comment): the same
+9000/8300/1000 setting, in an *overnight self-play run*, predicted
+~1.0 extra ply/decision but measured only **+0.29 actual root-depth
+ply** -- a ~0.3x damping ratio, attributed to EBF: iterative
+deepening's cost scales by the effective branching factor per ply, so
+affording one more full iteration needs saving roughly `(EBF-1)/EBF`
+(~70-75% for EBF 3.5-4) of the previous iteration's total cost.
+
+**Crucial later correction to this section's pessimism** (from the
+user, mid-session): a *different* overnight self-play run, using the
+old p5/p25/p75-derived "dodgy" thresholds, did NOT just match HEAD --
+it ran **0.9-1.0 ply deeper and was even-or-better in actual game
+strength**. This is the one piece of ground-truth (real game outcome,
+not an ECM proxy) evidence the whole session has, and it directly
+contradicts "tune LMR for modest percentage savings is the wrong order
+of magnitude lever" -- it says aggressive-even-if-locally-unsound
+reduction, tolerated because iterative deepening self-corrects
+individual mistakes on the next pass, is exactly the right lever. This
+reframing drove everything in sections 6-9.
+
+## 6. Decoupling admission from magnitude
+
+Advisor-reviewed fix, directly motivated by the three-population bug
+above and the "0.9-1.0 ply overnight" evidence: **introduced a
+separate constant, `LMR_STATIC_ADMIT_THOUSANDTHS`, used only for "do
+we reduce at all"**, independent of `JOINT_EXTRA_TIER1_THOUSANDTHS`
+("does this earn +1 extra ply"). Also fixed the sentinel (no static
+data) branch: previously required `uAttempts > 0` even when static had
+no opinion either, needlessly refusing cold+sentinel moves; changed to
+admit whenever not dynamically dangerous, since absence of data is not
+evidence of danger.
+
+Swept `LMR_STATIC_ADMIT_THOUSANDTHS` on sn5M (all three suites, vs
+`head_reference`):
+
+| setting | ringers depth/solve | confident_quick depth/solve | hard_quick depth/solve |
+|---|---|---|---|
+| HEAD | 11.6 / 6 | 12.4 / 78 | 11.4 / 19 |
+| 500 | 11.7 / 6 | 12.8 / 77 | 11.9 / 19 |
+| 1000 (no-op gate) | 11.9 / 4 | 12.8 / 76 | 12.1 / 18 |
+
+500 was flat-to-noise on solves (0/-1/0) while gaining real depth
+(+0.1/+0.4/+0.5 ply). 1000 was deeper still but **lost solves on every
+single suite simultaneously** (-2/-2/-1) -- a consistent, same-direction
+signal, not noise, meaning the extra depth from over-wide admission
+wasn't paying for itself. **500 was adopted as the working admission
+value**; it remains the current setting (section "Current code state"
+above).
+
+**Methodology note the user corrected mid-session**: sd10 (fixed
+depth) solve-count drops are expected and uninteresting (searching
+fewer nodes before the ceiling naturally solves fewer problems) --
+solve counts only matter as a signal at **sn5M (fixed node budget)**,
+where a drop means the extra depth isn't translating to correctness.
+Don't conflate the two contexts when reading solve deltas.
+
+## 7. Late move pruning (LMP) -- tried, net-negative, disabled not deleted
+
+Prompted by comparing against Crafty's actual source
+(`crafty-25.2/search.c:542,580`): Crafty's LMP is a hard `break` (skip
+ALL remaining moves at a node, not just one) gated purely on
+`depth < LMP_depth(16)` and `order > LMP[depth]` (a precomputed
+`3 + (i+.5)^1.9` table) -- no static table, no dynamic history, no
+per-move judgment at all. Implemented a conservative analog,
+`ShouldLatePrune` (searchsup.c): overt skip (not reduction) for the
+narrowest, most-certain tail only -- `uRemainingDepth <=
+LMP_MAX_REMAINING_DEPTH(6 plies)`, move count past a per-depth floor,
+AND both static (`JOINT_EXTRA_TIER3_THOUSANDTHS`, the strictest tier)
+and dynamic (well-evidenced, near-zero rate) signals agreeing it's
+ultra-safe, with killer/countermove exemptions carried over unchanged.
+Wired into search.c ahead of LMR/EFP so a pruned move skips both;
+reused `fThisMoveEFPPruned`/`fAnyMoveEFPPruned`'s existing hash-
+correctness plumbing rather than duplicating it.
+
+Measured on top of the admission=500 baseline, sn5M:
+
+| suite | admit=500 alone | +LMP |
+|---|---|---|
+| ringers | 11.7 / 6 | 11.7 / 6 (unchanged) |
+| confident_quick | 12.8 / 77 | 12.8 / **76** |
+| hard_quick | 11.9 / 19 | 11.9 / **18** |
+
+Depth/EBF barely moved (same as admit=500 alone), but solve count
+dropped one further on two of three suites at sn5M (the metric that
+matters per section 6's note) -- a real cost with no offsetting
+benefit. **Disabled at the call site (`search.c:1228`, `if (FALSE &&
+...)`), not deleted** -- the mechanism may be worth revisiting with
+different thresholds (the tested band was very narrow -- only
+~250K-moves-pruned out of ~23M total nodes on sd10 ringers, i.e. this
+barely touched anything, so "net-negative" here means "net-negative
+at this specific narrow threshold," not "pruning can't work." Crafty's
+own version is dramatically more aggressive (breaks on ALL remaining
+moves, not a per-move judgment call) -- this was flagged mid-session
+as "clearly stronger but way dodgier," and was deliberately not
+pursued further given the explicit instruction not to get crazy with
+pruning (irreversible, no re-search safety net, unlike LMR).
+
+**Known, unfixed issue in `ShouldLatePrune`'s depth gate** (found
+reviewing the Crafty comparison, documented in-code at
+`searchsup.c`'s `LMP_MAX_REMAINING_DEPTH` comment but not patched,
+since LMP is disabled anyway): the function only permits pruning near
+the leaf (`uRemainingDepth <= 6*ONE_PLY`), mirroring Crafty's
+`FP_depth`/`LMP_depth` near-leaf restriction. But our own joint table
+shows predicted FH% **declining** as remaining depth grows (checked
+directly, dumb=0/gap=0/late=0 slice: depth-bucket 0 → 17119
+thousandths, bucket 5 → 4033, bucket 11 → 617, a ~25x drop) -- so the
+ceiling permits pruning exactly where our own data says risk is
+*highest*, and forbids it where risk is *lowest*. Crafty isn't
+self-contradicting the same way: it has no static table at all, so
+the near-leaf restriction *is* its entire risk model (a mistake near
+the leaf is cheap to re-discover). We inherited the structural gate
+without inheriting its justification, stacked on a mechanism that
+already has a better, contradicting signal.
+
+The user's correct reframing: the *cost* of a wrong prune does
+legitimately grow with remaining depth, but that should tighten the
+**confidence bar**, not forbid the decision outright past a cutoff. The
+principled fix is in `lmr_testing/SPEC.md`'s "how much statistical
+confidence is actually needed" section: gate on the **static table's
+own per-cell sample count** via a rule-of-three bound (`N ≈ 3 *
+applications_per_game / target_misses_per_game`) -- not on remaining
+depth, not on the dynamic per-move-identity table. SPEC.md derives
+`N ≈ 14.4M` for a 1/1000-games miss target (no deep-zone zero-FH cell
+clears this) and, after Scott's pushback that 1/1000 was picked, not
+derived, a looser 1/100-games target gives `N ≈ 1.44M` (largest real
+cell: 1.34M, close) or 1/20-games gives `N ≈ 288K` (16 deep-zone
+zero-FH cells clear this today). **Blocker**: `g_uJointFHThousandths`
+(data.c) ships only the rate per cell, not the sample count behind it
+(cells need ≥500 tries to ship at all, but a 500-try cell and a
+1.34M-try cell are indistinguishable once shipped) -- applying this
+bound correctly needs a re-bake (`bake_joint.py` + `data.c`) adding a
+companion per-cell count array. Not done this session; a stand-in
+(depth-scaled dynamic-sample floor) was drafted and then deliberately
+reverted rather than leaving an uncalibrated heuristic in place --
+current code still has the original (acknowledged-wrong) depth
+ceiling, now with this critique documented alongside it.
+
+**Concrete lead for whoever re-bakes this**: querying the raw
+collection counts directly (not the shipped table) found **43 cells
+with a genuine 0.0% fail-high prediction (0 fail-highs across all
+tries) backed by ≥288,000 tries** -- clearing the 1/20-games bound
+above. These are heavily concentrated at **hang/dumb bucket 9**
+(`HangCpBucket`'s near-top bucket, `EstimateHangingMaterialCp`'s
+largest "this move hangs a lot of material for nothing" reading),
+essentially regardless of depth/gap/late bucket -- the single largest
+qualifying cell has **5.1M tries, zero fail-highs**. This isn't a
+sparse-sampling coincidence: it's the "dumb" axis doing exactly the
+job it was built for at its most extreme setting -- a move that
+obviously just hangs material for nothing is essentially never
+actually the best move at a node, so "it raises alpha" can't happen
+independent of depth/gap/lateness context. **This is the real
+population LMP should target once sample counts are shippable**: not
+"near the leaf" (the current, backwards ceiling), but "hang bucket 9,
+any depth" -- a structural, not positional, criterion.
+
+## 8. Magnitude-loosening attempts: the real dead end, and the actual fix
+
+With admission settled at 500, the next lever tried was MAGNITUDE --
+how many extra plies a reduced move gets. Two attempts:
+
+**Attempt A -- widen the bucket-tier thresholds.** Changed
+`72/5/1/cap-3` to `150/20/5/cap-4`. Rationale: Crafty's own LMR table,
+at the remaining-depth range our suites actually exercise (5-10
+plies), applies larger cuts (2-6 plies) than our tiers were allowing.
+**Result: sd10 ringers node count went UP, 23.0M -> 27.7M -- WORSE
+than even HEAD's 26.9M.** The re-search-rate diagnostic showed why:
+tier 1's research rate (4.13%) became *higher* than tier 0's (2.73%)
+-- exactly the "distribution-shift failure" the code's own calibration
+comment warns about (low-predicted-FH tiers should show a RISING
+research rate with tier, never an inversion). Widening tier 1's
+*admission* boundary let in a bucket of moves genuinely more dangerous
+than baseline, and the resulting re-search tax outweighed the savings.
+Reverted immediately.
+
+**Attempt B -- raise only the tier 2/3 *payout*, leave all admission
+boundaries (72/5/1) untouched.** Rationale: tiers 2/3 already showed
+LOWER research rates than tier 0 even before any change -- i.e.
+headroom, not a crack, unlike tier 1. Bumped tier2 payout 2->3, tier3
+3->5, cap to 5. **Result: sd10 node count barely moved (23.04M ->
+22.97M, 0.3%)** -- essentially null. Diagnosed (advisor-assisted)
+after the fact: **89% of all reduced moves land in tier 0 (no extra
+ply at all)**, 10.3% in tier 1, and only ~1.1% combined in tiers 2/3 --
+so tuning tiers 2/3's payout only ever touches the smallest sliver of
+the population; it mathematically cannot move the aggregate much
+either way. Separately, the new `cap=5` needed `remaining_plies >= 8`
+to even engage (`iMaxExtra` floor logic), which these suites' typical
+5-10-ply remaining depth rarely reaches -- a second reason this came
+back null.
+
+**The actual fix -- stop grading magnitude with the joint table at
+all.** The diagnosis above reframed the real question: 89% of the
+population (tier 0) was getting a flat, ungraded `-ONE_PLY` with no
+magnitude mechanism touching it whatsoever -- every experiment so far
+had been tuning the payout of an 11% minority. Separately, a grep
+turned up `g_iLMRQuietReduction[MAX_PLY_PER_SEARCH+1][LMR_TABLE_MAX_MOVES+1]`
+(dynamic.c, built by `InitLMRTable()`, called from `main.c`) -- a
+**Crafty/Ethereal-shaped flat `log(depth)*log(moves)` table, fully
+built at startup, read by nothing, anywhere in the codebase.** A prior
+session had built the simple mechanism and then gone and built the
+whole joint-table/dynamic-FH system instead, without ever wiring the
+simple one in. Sanity-checked its magnitudes against Crafty's actual
+table at the depth range that matters (5-10 plies remaining): d=5,m=8
+gives ~2.1 plies here vs. Crafty's 2.3; d=10,m=8 gives ~2.7 vs. 3.0 --
+same ballpark, same shape, and structurally immune to the "one bucket
+silently miscalibrated" failure mode that broke Attempt A, because
+it's not a lookup into learned per-situation fail-high predictions at
+all -- just depth and move order.
+
+Wired `_FlatTableExtraPlies(uRemainingDepth, uLegalMoves)` in as the
+magnitude source, **replacing** the old `_JointTableExtraPlies`
+tier-lookup (which is now dead code, deleted) -- admission logic
+(static/dynamic agreement, killer/countermove vetoes,
+`LMR_STATIC_ADMIT_THOUSANDTHS=500`) is completely unchanged; only "how
+much extra" changed. The table returns the FULL reduction in
+ONE_PLY-scaled ticks; subtracting the existing `ONE_PLY` base and
+flooring at 0 gives the extra-ply count, clamped by the existing
+`JOINT_EXTRA_MIN_REMAINING_DEPTH` floor logic as before.
+
+**Result -- the best of the whole session, every metric improved or
+flat simultaneously:**
+
+sd10 ringers node count: **19.56M**, vs. HEAD's 26.94M (-27.4%) and
+vs. the admit=500-alone baseline's 23.04M (-15.1% further). Solve
+count unchanged (3/11).
+
+sn5M, all three suites, vs. HEAD and vs. the previous best
+(admit=500 alone, no LMP, no magnitude change):
+
+| suite | HEAD | admit=500 alone | **flat-table magnitude** |
+|---|---|---|---|
+| ringers | 11.6 ply / 6 solved | 11.7 / 6 | **11.8 / 6** |
+| confident_quick | 12.4 / 78 | 12.8 / 77 | **12.8 / 77** |
+| hard_quick | 11.4 / 19 | 11.9 / 19 | **12.0 / 19** |
+
+Depth improved on every suite vs. both HEAD and the previous best
+(+0.2/+0.4/+0.6 ply vs. HEAD); hard_quick's solve count is perfectly
+flat (19/19) despite the largest depth gain of any candidate this
+session; no suite lost a solve relative to the admit=500 baseline.
+This is the first candidate all session where no metric had to be
+traded off against another.
+
+## 9. The pattern across the whole session
+
+Every lever that worked was a **simple, broad, structural** one
+(admission width, Crafty's flat depth x order table); every lever that
+failed or plateaued was a **precise, learned, per-situation** one
+(bucket-tier magnitude grading, narrow LMP thresholds). The common
+failure mode of the precise levers was a correlated miscalibration in
+one slice of the population (tier 1's admission boundary) that a flat
+mechanism structurally cannot have, because it has no slices to
+miscalibrate -- only a single smooth surface in (depth, move-order).
+Crafty's own source confirms this isn't a coincidence: neither its LMP
+nor its LMR table consults any static or dynamic per-move signal at
+all, and it reaches ply 19 at node counts where this engine was
+reaching ply 11-12 before this session. The dynamic/static per-move
+tables are still useful -- they're what's deciding *whether* to
+reduce at all (admission), where they've demonstrably worked -- the
+mistake was asking the same machinery to also grade *how much*.
+
+## Where we left off / next steps
+
+1. **Self-play validation is the next step, launched by the user
+ separately from this handoff.** Every metric used all session (sd10
+ node count, sn5M depth/EBF, sn5M solve count) is a proxy; the ECM
+ suites are small and documented-noisy (`../tests/ECM.md`); the only
+ ground-truth evidence all session is the overnight self-play run
+ referenced in section 5. **Use a genuine time-per-move or fixed-
+ node self-play gate, never `sd` (fixed depth defeats the entire
+ mechanism being tested -- confirmed in `typhoon_lmr_sd11_sn10m_
+ protocol` memory).** Check which existing script
+ (`eval_tune/match_play.py`, `.dna`-file-oriented, built for eval
+ tuning -- or `eval_tune/test_vs_head.sh`, currently modified in the
+ working tree) actually supports two *binaries* head-to-head, since
+ `match_play.py` as built takes two DNA files against one binary,
+ not two search-behavior variants.
+2. **Cross-check any solve-count deltas against `../tests/ECM.md`**
+ before trusting them -- gained/lost positions skew heavily
+ flaky/noise; the project's own doc tracks which ECM numbers are
+ known-flaky vs. known-legit.
+3. The current working config (admission=500, flat-table magnitude,
+ LMP disabled) has NOT been tried with LMP re-enabled using a
+ structurally different criterion than the narrow one tested in
+ section 7. Section 7's concrete lead (43 real cells, hang/dumb
+ bucket 9, up to 5.1M tries, genuine 0% FH -- a "this move obviously
+ hangs material" criterion, not a depth-near-leaf one) is the
+ specific next experiment: ship per-cell sample counts
+ (`bake_joint.py`/`data.c` re-bake), then gate `ShouldLatePrune` on
+ hang-bucket-9-or-similar + a rule-of-three-cleared sample count,
+ dropping the backwards depth ceiling entirely. Not attempted this
+ session (the re-bake is real work, and the explicit instruction was
+ to not get crazy with pruning without a principled basis -- this
+ gives one).
+4. Before running any further sed-based sweep, confirm the baseline
+ file is actually clean (section 5's contamination warning) --
+ explicitly grep the tier/baseline/admission constants, don't just
+ trust a `cp` from a few steps ago.
+5. The `u64FHJointHist`/`AttemptsBucket`/`RateBucket`/
+ `u64HeadStyleEligible`/`u64HeadStyleWouldReduce`/`u64LMPPruned`
+ diagnostics (chess.h, searchsup.c, root.c, script.c) are explicitly
+ one-off and still wired in under `#ifdef PERF_COUNTERS`. Harmless to
+ leave (no behavior change, diagnostic-only), but strip them out once
+ this investigation concludes.
+6. `JOINT_EXTRA_TIER1_THOUSANDTHS`/`JOINT_EXTRA_TIER2_THOUSANDTHS` are
+ now unused outside of comments/the disabled LMP path respectively --
+ candidates for removal once the flat-table magnitude approach is
+ confirmed by self-play, along with the now-dead `_JointTableExtraPlies`
+ function (already deleted) and its tier-sweep history in this doc,
+ which is kept here only as a record of why the simpler mechanism won.
+7. If self-play confirms this is a real strength gain, update
+ `head_reference/` (binary + logs + README) to make this the new
+ comparison baseline, per this codebase's own stated protocol
+ (`CLAUDE.md`'s "head_reference protocol" section).
+
+---
+
+# Update, 2026-10-03: admission-rate measurement, a counter-bounds bug, and
+# why this session's solve-count evidence does not support what it was used for
+
+Written while the self-play validation from "Where we left off" item 1 was
+running (`--st 4`, SPRT elo0=0/elo1=5, 12+ workers). At 1710 games:
+`-426 =831 +453`, score 0.508, llr +0.41, avg depth candidate 13.55 vs head
+12.90, ebf 3.464 vs 3.647, nps essentially tied (1.360M vs 1.365M).
+
+All measurements below were taken with the live candidate binary
+(`./typhoon`, confirmed `-DPERF_COUNTERS` and newer than every source file),
+`--cpus 1`, `--hash 256m`, three curated suites run serially.
+`eval_tune/match_play.py` confirmed to spawn engines with `--cpus 1`, so
+`split.c`'s degraded all-zero-snapshot `GetLMRReduction` path is not involved
+in the self-play numbers either.
+
+## 10.1. The match result is underpowered, not null
+
+Score 0.5079 over 1710 games with that draw rate gives an SD of the mean of
+0.0087, i.e. **+5.5 Elo with a 95% CI of roughly [-6, +17]**. The llr is
+drifting toward H1, not away. The honest statement is that this match spans
+everything from a small loss to a solid gain and cannot yet distinguish them
+-- not "the depth didn't translate," and not "it's fine."
+
+The extra depth is real rather than nominal: `ln(3.647)/ln(3.464) = 1.0414`,
+and `1.0414 x 12.90 = 13.43` against 13.55 measured. The ebf reduction fully
+accounts for the depth gained, with no residual.
+
+**That "no residual" is itself the interesting part.** A well-targeted
+reduction should beat the arithmetic, because the saved nodes go
+preferentially to lines that matter, so accuracy-at-fixed-nodes improves
+beyond what pure depth predicts. Getting exactly the arithmetic and nothing
+more is the signature of a near-uniform tax on the leftover pool rather than
+a reallocation. See section 10.7b.
+
+## 10.2. Admission rate: the candidate reduces far MORE than HEAD, not less
+
+Ran the still-wired `u64HeadStyleEligible`/`u64HeadStyleWouldReduce`
+diagnostic (section 3 (original)) at `sd 10`. Last cumulative checkpoint per suite:
+
+| suite | eligible calls | candidate: SOME reduction | HEAD-style: flat -1 | delta |
+|---|---|---|---|---|
+| ecm_ringers | 2,897,448 | 92.88% | 71.60% | +21.3 |
+| ecm_confident_quick | 15,345,704 | 94.47% | 70.37% | +24.1 |
+| ecm_hard_quick | 28,632,443 | 93.26% | 74.99% | +18.3 |
+
+The section 3 (original) under-coverage problem is comprehensively gone. Coverage is now
+~19-24 points WIDER than HEAD-style's.
+
+**Cause, checked rather than assumed.** The first hypothesis was that the
+smoothed re-bake widened admission, since `data.c`'s current header reads
+"Every cell is populated -- no 0xffff sentinels remain" with 5,241/21,780
+cells shrunk toward a coarser marginal. That hypothesis is wrong, and
+backwards: the sentinel branch in `GetLMRReduction` was the PERMISSIVE one
+(`fBaseEligible = (FALSE == fDynamicDanger)`, admit unless dynamically
+dangerous) while the value branch is CONDITIONAL
+(`uPredictedFH < LMR_STATIC_ADMIT_THOUSANDTHS`). Moving cells from sentinel
+to value can only TIGHTEN.
+
+Parsing `g_uJointFHThousandths` out of data.c directly: only **733 of 21,780
+cells (3.4%) read >= 500**, and 224 of those sit in dumb bucket 1 where
+`_JointTablePredictedFHThousandths` returns the sentinel regardless -- so only
+~509 cells (2.3% of the space) can actually veto. They concentrate in gap
+bucket 0 (480 of 733, i.e. near-alpha) and depth buckets 0-1. So
+`LMR_STATIC_ADMIT_THOUSANDTHS = 500` has ALWAYS been close to wide open;
+section 6 (original)'s 500-vs-1000 sweep was distinguishing a ~6% slice of calls, not a
+broad gate.
+
+What actually widened coverage from section 3 (original)'s 62-66% to today's 93-94% was
+**section 6 (original)'s decoupling**: `fDynamicDanger` was demoted from vetoing the base
+reduction to merely forcing `uExtraPlies = 0`. A code change, not a data
+change.
+
+**Corollary: section 1 (original)'s `g_FailHighs` honesty fix is now nearly inert.**
+`fDynamicDanger` no longer vetoes the base reduction (except for dumb-bucket-1
+moves, which take the sentinel branch), and it only zeroes the extra plies --
+which ~85% of reduced moves never receive anyway (section 3 (original) below). The lever
+that started that whole session has almost no remaining effect on shipped
+behavior. Worth knowing before investing further in that table's accuracy.
+
+## 10.3. Magnitude is still overwhelmingly base-only
+
+Tier share of all REDUCED moves, across the three suites: tier 0 ~85-86%,
+tier 1 ~11-13%, tier 2 ~2.2-2.4%, tier 3 ~0.0-0.01%. Section 8 (original)'s "89% of all
+reduced moves land in tier 0" survived the switch from `_JointTableExtraPlies`
+to `_FlatTableExtraPlies` essentially unchanged.
+
+So the +0.65 ply in self-play is bought almost entirely by BREADTH (93-94%
+admission at a flat -1 ply), not by depth-of-cut. The flat table replaced the
+fragile tier payouts without actually paying out much more than they did.
+
+## 10.4. BUG: `JOINT_EXTRA_MAX_PLIES` (5) overruns `JOINT_EXTRA_TIER_COUNT` (4)
+
+```
+chess.h:999 #define JOINT_EXTRA_TIER_COUNT (4) // valid indices 0..3
+chess.h:1143 UINT64 u64JointExtraTries[JOINT_EXTRA_TIER_COUNT];
+chess.h:1144 UINT64 u64JointExtraResearch[JOINT_EXTRA_TIER_COUNT];
+chess.h:1149 UINT64 u64LMPPruned;
+searchsup.c:514 #define JOINT_EXTRA_MAX_PLIES (5)
+searchsup.c:934-6 if (uExtraPlies > JOINT_EXTRA_MAX_PLIES) uExtraPlies = JOINT_EXTRA_MAX_PLIES;
+searchsup.c:942 *puExtraTier = uExtraPlies; // 0..5 escapes to the caller
+search.c:1279 INC(ctx->sCounters.tree.u64JointExtraTries[uExtraTier]);
+search.c:1489 INC(ctx->sCounters.tree.u64JointExtraResearch[uThisMoveJointExtraTier]);
+```
+
+With `uExtraTier == 4`: `search.c:1279` writes `u64JointExtraTries[4]`, which
+lands on `u64JointExtraResearch[0]`; `search.c:1489` writes
+`u64JointExtraResearch[4]`, which is past BOTH arrays and lands on
+`u64LMPPruned`. The only other clamp, `iMaxExtra` (searchsup.c:922), bounds
+against remaining depth, not against the array -- at remaining depth 14 it
+permits 11.
+
+**Why no suite run ever caught it.** `_FlatTableExtraPlies` returns
+`(total_ticks - ONE_PLY)/ONE_PLY` with
+`total = 0.7844 + log(d)*log(m)/2.4696`. Tier 4 needs `total >= 5.0`:
+
+| remaining depth | moves tried | total | tier |
+|---|---|---|---|
+| 10 | 63 (LMR_TABLE_MAX_MOVES) | 4.65 | 3 |
+| 14 | 31 | 4.45 | 3 |
+| 14 | 63 | 5.21 | **4** |
+| 20 | 63 | 5.81 | **4** |
+
+At `sd 10` remaining depth never exceeds 10, so tier 3 is the ceiling and
+every suite measurement in this document (section 8 (original)'s included) is clean. The
+live match averages 13.55 ply, so tier 4 fires there routinely.
+
+**Blast radius**: counters only. Nothing reads these to make a search
+decision, so the running match's RESULT IS VALID, and `root.c:1558-1561`'s
+memcpys use `sizeof` the 4-element destinations so it does not propagate into
+`g_Options`. What it corrupts is (a) tier 0's apparent re-search rate, i.e.
+precisely the calibration diagnostic the next candidate would be judged on,
+and (b) `u64LMPPruned`, which can read nonzero while LMP is disabled at
+`search.c:1228` -- a thoroughly confusing symptom for whoever hits it next.
+
+**Fix**: raise `JOINT_EXTRA_TIER_COUNT` to 6 (`script.c`'s report loop is
+already bounded by the constant, so reporting stays faithful). Clamping
+`JOINT_EXTRA_MAX_PLIES` to 3 instead would change search behavior and is
+therefore not a pure bug fix.
+
+## 10.5. The by-tier re-search-rate diagnostic no longer means what section 8 (original) used it for
+
+```
+ tier0 tier1 tier2 tier3
+ringers 2.63% 3.84% 3.55% 3.85% (77 tries)
+confident_quick 2.69% 2.94% 2.87% 3.98% (1,204 tries)
+hard_quick 2.60% 3.74% 3.30% 2.88% (1,768 tries)
+```
+
+Section 8 (original) rejected Attempt A because tier 1's research rate (4.13%) came in
+HIGHER than tier 0's (2.73%), calling it the "distribution-shift failure."
+That was a valid test WHEN tier was a predicted-FH% band
+(`_JointTableExtraPlies`): a rising rate meant the risk ordering had broken.
+
+Under `_FlatTableExtraPlies`, tier is pure MAGNITUDE. A bigger cut
+mechanically needs more re-searches regardless of how well-calibrated
+anything is, so the inversion carries no calibration signal at all. Note
+tier 2 < tier 1 on all three suites, which no FH-ordering story predicts but
+a "bigger cut costs more, noisily" story accommodates fine.
+
+To be useful again this diagnostic must be re-based on PREDICTED FH%
+independent of applied magnitude. As currently reported it should not be used
+to accept or reject a candidate. Aggregate re-search tax is ~2.6-2.8% of
+reduced moves, which is modest.
+
+## 10.6. The solve-count evidence in this document does not support what it was used for
+
+Ran `sn 5000000` on all three suites (the metric section 6 (original) established as the
+one that matters) against `head_reference/logs/sn5m_*.out`, then diffed
+PER POSITION instead of reading the aggregate:
+
+| suite | HEAD solved/depth | candidate solved/depth | positions FLIPPED |
+|---|---|---|---|
+| ecm_ringers | 6/11 @ 11.6 ply | 6/11 @ 11.8 ply | 4 of 11 (36%) |
+| ecm_confident_quick | 78/90 @ 12.4 ply | 77/90 @ 12.8 ply | 7 of 90 ( 8%) |
+| ecm_hard_quick | 19/90 @ 11.4 ply | 20/90 @ 12.0 ply | 15 of 90 (17%) |
+
+Depth reproduces section 6 (original)/8's admit=500 row closely (+0.2/+0.4/+0.6 ply) and
+is the trustworthy axis. The solve counts are not, and cross-referencing every
+flip against `../tests/ECM.md` shows the near-flat aggregates are
+CANCELLATION, not stability:
+
+- **ringers (net 0)**: gains `ECM.213` (catalogued flaky, +0.37 unconfirmed
+ repetition line) and `ECM.270` (catalogued BAD SOLUTION -- score exactly
+ 0.00, Crafty's own sd20 prefers Rxc5); losses `ECM.721` and `ECM.746`, both
+ catalogued CLEAN. **Net 0 is two real losses masked by two junk gains.**
+- **confident_quick (net -1)**: real gains `ECM.519` (clean, +7.33) and
+ `ECM.736` (Crafty confirms the same move across depths 24-28 while HEAD
+ wobbles); real loss `ECM.348` (NOT CATALOGUED); `ECM.357` gain and
+ `ECM.715`/`716`/`826` losses all catalogued flaky or explicit horizon
+ artifacts. **De-noised this suite is roughly +2/-1 -- the OPPOSITE sign
+ from its aggregate.**
+- **hard_quick (net +1)**: real losses `ECM.128` (clean, +2.66, no
+ oscillation) and `ECM.846` (explicitly RECLASSIFIED NON-flaky on 2026-09-11
+ from two-engine deep stable agreement); real gains `ECM.476`
+ (Crafty-confirmed legit) and `ECM.803` (Crafty finds mate in 10 from the
+ solution move). `ECM.866` is catalogued verbatim as "a genuinely coin-flip
+ position." `ECM.018`/`084`/`516`/`640`/`774` NOT CATALOGUED.
+
+**Uncatalogued positions that flipped and should be triaged into ECM.md**:
+`ECM.348`, `ECM.018`, `ECM.084`, `ECM.516`, `ECM.640`, `ECM.774`.
+
+### Implication: section 6 (original)'s admission decision may rest on noise
+
+`LMR_STATIC_ADMIT_THOUSANDTHS = 500` was chosen over 1000 because 1000 "lost
+solves on every single suite simultaneously (-2/-2/-1) -- a consistent,
+same-direction signal, not noise." But a single configuration churns 4/11,
+7/90 and 15/90 positions here, with each net produced by catalogued-junk
+cancelling catalogued-clean in BOTH directions. A -2/-2/-1 aggregate sits
+well inside that churn.
+
+Direction-consistency across three small aggregates was treated as evidence
+against noise. That inference does not hold when each aggregate is itself a
+near-cancellation of 8-36% churn.
+
+The same caveat applies to **section 7 (original)'s LMP rejection** (-1 on two suites)
+and **section 8 (original)'s "no suite lost a solve relative to the admit=500
+baseline."** None of those conclusions are refuted -- they are unsupported,
+which is different. 500 is not shown to be wrong; it is shown to rest on
+nothing.
+
+**Protocol change this implies**: never report a curated-suite solve delta as
+an aggregate again. Diff per position, classify each flip against ECM.md, and
+report "N real gains / M real losses / K junk / J uncatalogued." The script
+used here is three lines of `grep`/`sed`/`join` over the two `.out` files and
+should be checked in rather than re-derived.
+
+## 10.7. Proposed directions, in priority order
+
+**Superseded in part by 10.9** -- written before the magnitude question was
+reconciled (10.8). 10.7a and 10.7c still stand as written; 10.7b is absorbed
+into 10.9c; 10.7d is DONE; 10.7e still stands. Read 10.9 for the current
+direction list.
+
+### 10.7a. HEAD vs HEAD at 2:1 time odds -- the decisive unrun experiment
+
+Grepped `lmr_testing/*.md`, `../src/CLAUDE.md` and `../tests/ECM.md`: this
+has never been run or recorded. It is the highest-value measurement available
+and it depends on neither ECM nor any proxy.
+
+**CORRECTED 2026-10-03** -- an earlier version of this section had the
+ebf/time conversion INVERTED (it claimed +0.65 ply was "a 0.19x doubling").
+The correct arithmetic, at ebf 3.6:
+
+- One ply costs a factor of ebf in nodes, so one DOUBLING of time buys
+ `ln(2)/ln(3.6)` = **0.54 ply**, not ~1.6.
+- Conversely this candidate's +0.65 ply costs `3.6^0.65` = 2.30x the nodes,
+ i.e. it is worth about **1.20 doublings** of time-equivalent.
+
+That inversion mattered, because it made one of the two branches below look
+plausible when it is not. With the correct conversion, if the measured
+Elo-per-doubling is D, then a sound +0.65 ply "should" be worth
+`1.20 x D` Elo:
+
+| measured D (Elo/doubling) | expected Elo for +0.65 ply | vs. the ~+2 to +5 measured |
+|---|---|---|
+| 60 | ~72 | 15-30x shortfall |
+| 40 | ~48 | 10-20x shortfall |
+| 20 | ~24 | 5-12x shortfall |
+| ~3 | ~4 | matches |
+
+So for the candidate's measured result to be "exactly what depth is worth
+here", a doubling of time would have to be worth only ~2-4 Elo -- i.e.
+doubling the thinking time barely helps this engine at all. That is a strong,
+falsifiable claim, and the measurement settles it:
+
+- **D is small (< ~10 Elo/doubling)**: typhoon is severely eval-limited.
+ Depth is nearly worthless to it, no search-efficiency work can pay, and the
+ lever is eval/speed. Consistent with the known 1.5M-vs-7.5M nps gap vs
+ Crafty and with eval being the separately-identified bottleneck.
+- **D is normal (~20-70 Elo/doubling)**: then reduction-bought plies are
+ worth drastically less than time-bought plies of the same nominal size, and
+ the question becomes WHY. That is a search-QUALITY investigation (what is
+ being lost in the reduced tree), not a tuning exercise -- and it would
+ explain the whole project's "always slightly positive, never significant"
+ pattern in one stroke.
+
+Note which way the remaining bias runs: Elo-per-doubling generally SHRINKS
+with depth (diminishing returns), so a D measured at a short time control is
+an upper bound on D at a longer one. Using a short-TC D therefore
+over-estimates the expected Elo and over-states the shortfall. Record the
+time control the calibration was actually run at.
+
+### 10.7b. Direct the cuts rather than widening them further
+
+At 93-94% admission and 85% flat -1 ply, the mechanism is close to "reduce
+every late quiet move by one ply" -- a uniform tax. A uniform tax lowers ebf
+and buys depth, and it structurally CANNOT protect critical lines, because
+nothing in it distinguishes them. Section 1 (original)'s dynamic signal was demoted to
+extras-only, section 8 (original) removed the joint table from magnitude, and the
+admission gate it still owns vetoes 2.3% of the cell space: step by step, each
+individually justified, the mechanism converged on undirected.
+
+Three inputs for directing it are already in scope and unused:
+
+1. **`iImprovement`** is a declared parameter of `GetLMRReduction`
+ (searchsup.c:638) and is never read in the body. Not-improving nodes
+ conventionally reduce ~1 ply more.
+2. **`pi->eNodeType`** is computed for every node and is exact
+ (`NODE_PV` iff wide window, searchsup.c:2203). CUT nodes tolerate more
+ reduction than ALL nodes, which must search everything to prove a fail-low.
+ LMR ignores the distinction.
+3. **`g_HistoryCounters`/`g_ContinuationHistory`** feed move ordering but
+ reach reduction only as a binary FH-rate veto, never as a scale on R.
+
+Frame these as flat structural +/-1 adjustments on a single smooth surface --
+explicitly NOT the "precise learned per-situation" grading that failed in
+section 8 (original). They have no buckets to miscalibrate, which is why section 9 (original)'s own
+pattern predicts they behave differently from the tier payouts.
+
+### 10.7c. Re-examine the original section 9's conclusion
+
+Section 9 (original) concluded that simple broad structural levers work and precise
+learned per-situation levers fail. That is an accurate description of what
+moved NODE COUNTS. It may be the wrong inference about Elo: the broad lever is
+what produced the no-residual, flat-accuracy depth in section 1 (original), and all three
+of section 9 (original)'s supporting measurements (sd10 node count, sn5M solve count,
+by-tier research rate) are now either noise-level (section 6 (original)) or no longer
+meaningful (section 10.5). Node count is not Elo. This is the inference in this
+document I would trust least.
+
+### 10.7d. Fix the counter bug before the next calibration read -- DONE 2026-10-03
+
+Section 10.4. Fixed: `JOINT_EXTRA_TIER_COUNT` 4 -> 6 (chess.h:999), with the
+broken lockstep against `JOINT_EXTRA_MAX_PLIES` documented in place. Every
+user of the constant was already bounded by it (both counter structs, the
+`JOINT_EXTRA_VERIFY` arrays, all six `script.c` loops), so nothing else
+needed touching. Raised the count rather than clamping `MAX_PLIES` back to 3,
+which would have been a search-behavior change rather than a fix. Clean
+rebuild, no warnings, batch self-test passes.
+
+Note for anyone reading counters from a binary built BEFORE this fix --
+including the 1710-game match in 10.1 -- tier 0's re-search rate and
+`u64LMPPruned` are corrupted there. The game results are unaffected.
+
+### 10.7e. Re-derive the admission threshold, if 10.7a says depth is worth buying
+
+Nothing currently justifies 500 over any other value (section 10.6). If 10.7a shows
+depth converts, sweep it against a metric that survives scrutiny -- move
+agreement against a deep oracle over a few hundred positions gives far more
+samples per position than a binary solved/unsolved on 90, and measures
+"better decisions per node" directly.
+
+## 10.8. Why magnitude is shallow: fully reconciled, plus a unit landmine fixed
+
+Follow-up to 10.3 ("magnitude is still overwhelmingly base-only"), which
+reported the symptom without explaining it. Driven by a direct question:
+if we have a 4-axis table that judges P(fail-high), why fall back on a
+"more primitive" `log(depth)*log(moves)` formula to size the bet? We are
+at a blackjack table and should bet more when we have an edge.
+
+The answer turns out to be that the magnitude *function* was never the
+binding constraint, so swapping it changed almost nothing -- and the
+reason it isn't binding is a deliberate invariant, not an oversight.
+
+### The table is not signal-starved
+
+Reassembled the `FOURAXISRAW` collection cells from the three 10.2 `sd10`
+runs (87.8M eligible tries, 25,159 distinct raw cells) and looked up every
+cell's shipped prediction. Tries-weighted, over the admitted population:
+
+| predicted FH | share |
+|---|---|
+| < 0.001% | 3.49% |
+| 0.001-0.005% | 18.97% |
+| 0.005-0.072% | 48.07% |
+| 0.072-0.5% | 18.84% |
+| >= 0.5% (admission veto) | 10.36% |
+
+**70.5% of admitted moves read below 0.072%** (the old TIER1 boundary) and
+22.5% read below 0.005%. So "we rarely encounter moves the table thinks are
+safe" is false -- it thinks most of the population is safe.
+
+### The two magnitude sources are a wash
+
+Over LMR's actual population (see the correction below), mean extra plies:
+**4-axis table 0.224, flat log(d)*log(m) 0.244** -- the flat formula is
+marginally MORE generous, and they put the same ~62% of traffic in tier 0.
+Within the depth>=4 slice where extras are legal at all, 0.919 vs 1.002.
+They disagree per-cell on 36% of depth>=4 tries (21% where the table wants
+less, 15% where it wants more) at essentially identical means.
+
+So section 8 (original)'s swap did not meaningfully change how much we
+reduce. It changed WHICH moves get the extras, at the same aggregate. That
+also means section 8 (original)'s headline result -- ringers 23.04M ->
+19.56M nodes, attributed to this swap -- **is not explained by the change
+it was attributed to.** Still unexplained; flagging rather than guessing.
+
+### What actually binds: the one-full-width-ply invariant
+
+`Search()` drops into `QSearch` at `uDepth < THREE_QUARTERS_PLY`
+(search.c:462). A reduced move consumes TWO plies before any extra (the
+normal `- ONE_PLY` decrement plus the base `-ONE_PLY` carried in
+`iExtend`), so `JOINT_EXTRA_MIN_REMAINING_DEPTH = ONE_PLY` encodes exactly
+"a reduced move still gets at least one genuine full-width ply before the
+horizon." `searchsup.c`'s `iMaxExtra` then works out to
+**`remaining_plies - 3`**, and `uDepth > TWO_PLY` (search.c:1254) enforces
+the same invariant for the base ply (at remaining depth 3, base-only leaves
+exactly `ONE_PLY`). One invariant, two gates -- the design is coherent, and
+this floor is load-bearing rather than conservative padding.
+
+Consequence: **no extra ply is permissible below remaining depth-bucket 4,
+for any signal, from any source.** Depth mix of the LMR population:
+
+| depth bucket | share of LMR pop | iMaxExtra |
+|---|---|---|
+| 2 (uDepth 129-191) | 45.7% | 0 |
+| 3 | 30.5% | 0 |
+| 4 | 14.7% | 1 |
+| 5 | 5.8% | 2 |
+| 6 | 2.3% | 3 |
+| 7+ | 1.1% | 4+ |
+
+Only ~23.9% of the population can receive an extra ply at all. This is just
+the shape of an alpha-beta tree: node count grows with ply-from-root, so the
+mass is near the leaves, where there is no depth left to cut.
+
+### Reconciliation of the model against the engine
+
+10.3 left a 2.7x discrepancy between a first (wrong) model and the measured
+mean. Both causes found:
+
+1. **Wrong population floor (my error).** The first pass filtered at depth
+ bucket >= 3, but `uDepth > TWO_PLY` admits bucket 2 (`uDepth` 129-191),
+ where `iMaxExtra` is 0. Bucket 2 is 21.25% of the eligible population and
+ was excluded entirely. Note also that `FOURAXISRAW` collection is gated on
+ `fPruneReduceEligible` ALONE -- it does not apply LMR's own
+ `uDepth > TWO_PLY`, so the collection population is ~4x LMR's and must be
+ filtered before any magnitude projection. Easy trap; it is a different
+ population than the one `u64FHJointHist` counts.
+2. **The dynamic component**, as predicted. Read straight off
+ `u64FHJointHist` with no rebuild: `fDynamicDanger` (own rate > the 0.400%
+ baseline, with any samples at all) zeroes the extras on **27.4%** of LMR
+ decisions (27.0 / 25.1 / 28.6% across the three suites, 46.9M calls).
+ 62.1% are warm-with-rate-exactly-0 and 7.1% pure cold; both pass.
+
+| model | mean extra | x (1 - 0.274) | measured |
+|---|---|---|---|
+| bucket >= 3 (wrong floor) | 0.440 | 0.320 | 0.164 |
+| **bucket >= 2 (correct)** | **0.244** | **0.177** | **0.164** |
+
+Within 8%, and the residual has a known sign: `FOURAXISRAW` increments
+happen AFTER the EFP decision, so EFP-pruned moves get a
+`GetLMRReduction` call (counted in the 46.9M histogram) but no collection
+increment. EFP fires near the horizon, so the proxy is biased away from the
+shallowest depths, which inflates the model exactly as observed.
+
+**So the magnitude story is: the clamp is the dominant factor, the dynamic
+veto is the second, and the choice of magnitude function is third and
+nearly irrelevant.**
+
+### Unit landmine fixed (2026-10-03)
+
+`GetMoveFailHighThousandths` returned PER-MILLE (0-1000) while
+`g_uJointFHThousandths` is in thousandths of a PERCENT -- 100x finer. Every
+caller was individually correct, but the two families were indistinguishable
+by name, and `ShouldLatePrune` held both four lines apart:
+`LMP_DYNAMIC_MAX_THOUSANDTHS (1)` meaning 0.1% beside
+`LMP_STATIC_MAX_THOUSANDTHS (1)` meaning 0.001%. The function's own doc
+comment asserted it shared the static table's unit "so a caller can compare
+this move's own dynamic rate directly against the static table's
+prediction" -- which is how the trap got planted.
+
+Fixed by converting the whole dynamic family to the table's unit (not just
+the two constants originally flagged -- leaving EFP and LMP in per-mille
+would have relocated the trap):
+
+| constant | was | now |
+|---|---|---|
+| `GetMoveFailHighPctThousandths` (renamed) | 0-1000 | 0-100000 |
+| `LMR_FH_BASELINE_PCT_THOUSANDTHS` | 4 | 400 (0.400%) |
+| `LMR_STATIC_ADMIT_PCT_THOUSANDTHS` | 500 | 500 (unchanged) |
+| `EFP_FH_PRUNE_THRESHOLD_PCT_THOUSANDTHS` | 100 | 10000 (10%) |
+| `LMP_DYNAMIC_MAX_PCT_THOUSANDTHS` | 1 | 100 (0.100%) |
+| `LMP_STATIC_MAX_PCT_THOUSANDTHS` | 1 | unchanged |
+
+Documented in-code: the old `> 4` per-mille test could only fire once
+`fh/attempts >= 0.005`, so the SHIPPED gate behaved like 0.5%, not the
+0.400% it documented. 400 is faithful to the intent and a hair stricter than
+what shipped; the affected band needs >= 200 observations of a single move
+identity to express and is essentially unpopulated per `u64FHJointHist`.
+Intended behavior preserved; shipped behavior changed immeasurably.
+
+## 10.9. Future directions from 10.8
+
+Premise, per Scott: when the table says a move is safe we should be more
+willing to (1) reduce beyond the clamp, letting the move drop straight into
+qsearch, or (2) prune it outright. Both are right in kind; they differ by
+more than an order of magnitude in expected payoff, and the measurement
+says which is which.
+
+### 10.9a. Reduce into qsearch when the table certifies safety -- a SAFETY probe, not a depth lever
+
+Make `JOINT_EXTRA_MIN_REMAINING_DEPTH` table-conditional: `ONE_PLY`
+normally, 0 when predicted FH clears a strict bar. This is the FIRST place
+in this project where the table would do something the flat
+`log(d)*log(moves)` formula structurally cannot express -- a conditional
+floor. It is also strictly safer than pruning: the move still receives a
+qsearch score and still trips the existing fail-high re-search
+(search.c:1484), so it stays recoverable in the one direction we can detect.
+
+Measured payoff ceiling (mean extra plies, before/after the 27.4% dynamic
+veto):
+
+| variant | mean extra | after dyn veto |
+|---|---|---|
+| today (`MIN_REMAINING = ONE_PLY`) | 0.224 | 0.162 |
+| unconditional `MIN_REMAINING = 0` | 0.506 | 0.368 |
+| conditional, pred < 0.005% | 0.302 | 0.219 |
+| conditional, pred < 0.001% | 0.227 | 0.165 |
+
+**Then apply section 5 (original)'s own damping measurement** (1.0 extra
+ply/decision -> +0.29 measured root ply): the conditional-at-0.005% variant
+buys ~+0.057 extra ply/decision, i.e. roughly **+0.017 root ply.** The
+unconditional version reaches ~+0.06 root ply. Both are an order of
+magnitude below the +0.65 ply this candidate ALREADY achieves, and well
+below measurement noise.
+
+**So this cannot be justified as a strength experiment -- the arithmetic
+will not support it.** Its real value is as the cheapest available probe of
+whether the table's "safe" verdict survives a much harsher consequence: if
+pred<0.005% cells can lose their last full-width ply with no measurable
+damage, that is direct evidence the table is trustworthy at the extreme,
+which is the precondition 10.9b needs anyway. Run it at the strict
+threshold, judge it on silent-miss rate (reusing `JOINT_EXTRA_VERIFY` plus
+the `NEUTRALIZE_NULL_BAILOUT` fix, per section 8 (original)'s lesson), and
+do NOT expect or claim a depth gain.
+
+Risk class to watch specifically: the refutation that is itself a quiet move
+one ply deep. That is precisely the documented genuine-miss class 2 (the
+`Bb6` deflection, invisible to qsearch by construction), and it is the exact
+thing the discarded full-width ply existed to catch.
+
+### 10.9b. Prune outright -- where the order of magnitude actually is
+
+Reduction saves plies off a subtree; pruning removes the subtree. The
+savings scale completely differently, which is why this dominates 10.9a:
+
+- 1.88% of eligible tries sit at depth >= 4 with predicted FH < 0.005%
+ (0.87% at depth >= 5, 0.38% at depth >= 6).
+- By SPEC.md's own order-of-magnitude check a depth-6 subtree is ~20,000
+ nodes against 30-100 for a near-leaf node.
+- A 1.88% population of 100-1000x larger subtrees dominates the node budget
+ in a way no magnitude change can approach.
+
+The machinery already exists (`ShouldLatePrune`, searchsup.c, disabled at
+search.c:1228) and section 7 (original)'s concrete lead is still unused:
+**43 cells with a genuine 0.0% observed fail-high backed by >= 288,000
+tries, concentrated at hang/dumb bucket 9 essentially regardless of
+depth/gap/late, the largest with 5.1M tries and zero fail-highs.** The
+criterion is structural ("this move obviously hangs material for nothing"),
+not positional, and it replaces the current backwards near-leaf depth
+ceiling -- which section 7 (original) already documents as permitting
+pruning exactly where our own table says risk is HIGHEST.
+
+Prerequisites, unchanged from section 7 (original) and SPEC.md's agreed
+design requirements: ship per-cell sample counts (`bake_joint.py` +
+`data.c`) so a rule-of-three bound can gate on evidence rather than on
+remaining depth, and build the prune-specific shadow-verification harness
+BEFORE pruning for real.
+
+### 10.9c. Coverage into the populations currently excluded outright
+
+Neither 10.9a nor 10.9b touches anything outside the already-admitted
+leftover pool. Still untouched: the 10.4% that admission vetoes, every
+`NODE_PV` node, and every non-leftover (sub-`GOOD_MOVE`-failing) move --
+`fPruneReduceEligible` excludes the latter two structurally. Crafty and
+Ethereal both reduce all quiet moves past a move-count threshold, and
+"coverage beats magnitude" is the most replicated finding in this document
+(section 3 (original), section 6 (original), 10.2). This is the largest
+remaining lever by population size, and 10.8 is the explanation for WHY it
+is larger than magnitude: magnitude is capped by the depth distribution,
+coverage is not.
+
+Also in this family, and nearly free: the three unused inputs from 10.7b
+(`iImprovement`, a declared-but-unread parameter of `GetLMRReduction`;
+`pi->eNodeType`; history as a scale on R rather than a binary veto).
+
+### 10.9d. Standing caveat over all of the above
+
+10.7a (HEAD vs HEAD at 2:1 time odds) still gates every one of these. If a
+ply is worth ~25 Elo in this engine rather than the ~60-70 a healthy engine
+shows, then even 10.9b's node savings convert poorly and the real lever is
+eval/speed, not search. That measurement is cheaper than any direction here
+and decides whether to pursue them at all.
+
+### Two errors of mine recorded, because the reasoning is the useful part
+
+1. **"Relax `JOINT_EXTRA_MIN_REMAINING_DEPTH` to 0 as a cheap way to buy
+ magnitude"** -- withdrawn. It is not a magnitude tweak; it converts
+ reduction into a qsearch-only evaluation, i.e. stealth LMP, and the
+ invariant being relaxed is load-bearing (see 10.8). It survives only in
+ the much narrower, table-gated, safety-probe form of 10.9a.
+2. **Filtering the collection population at depth bucket >= 3** instead of
+ >= 2, which over-predicted mean magnitude by ~1.8x and briefly made the
+ clamp look like it hard-capped 56% of decisions rather than the correct
+ ~76% (buckets 2 and 3 combined). Corrected in 10.8; the lesson is that
+ `FOURAXISRAW` and `u64FHJointHist` count DIFFERENT populations and
+ neither is LMR's without explicit filtering.
+
+## 10.10. The time-odds calibration, and the Crafty gap decomposed (2026-10-03)
+
+Two measurements that between them reframe this whole project. Neither
+depends on ECM solve counts or any proxy.
+
+### 10.10a. A ply is worth ~95 Elo to this engine
+
+HEAD vs HEAD at 2:1 time odds (`--st 2 --odds 2`, same binary both sides,
+no SPRT -- measuring a quantity, not testing a hypothesis). Stopped by hand
+at 170 games once the interval was decisive:
+
+| games | score | Elo | depth (cand vs head) |
+|---|---|---|---|
+| 20 | 0.700 | +147 +/-109 | 11.93 vs 11.08 |
+| 70 | 0.650 | +108 +/- 58 | 12.37 vs 11.33 |
+| 120 | 0.646 | +105 +/- 45 | 12.33 vs 11.47 |
+| 170 | 0.621 | **+86 +/- 37** | 12.37 vs 11.47 |
+
+**Result: a doubling of time buys ~0.90 ply and ~+86 Elo, i.e. ~95 Elo per
+ply** at this time control on this box. The 95% interval [48, 123] excludes
+by a wide margin the "a ply is worth almost nothing here" hypothesis that
+10.7a entertained.
+
+**This is the yardstick the project never had.** Any future depth-buying
+change can be pre-registered against it: predicted ceiling = measured ply
+gain x ~95 Elo. A change that buys +0.3 ply has a ~28 Elo ceiling (so
+`elo0=0 elo1=15` will resolve it in hours); a change that buys +0.05 ply
+cannot clear any bar worth caring about and should not get a match at all.
+That single number retires the entire class of 2,000-game inconclusive runs
+this document is otherwise a history of.
+
+### 10.10b. Reduction-bought plies are worth ~1/11th of time-bought plies
+
+Side by side, same engine, same box:
+
+| source of depth | ply gain | Elo | Elo/ply |
+|---|---|---|---|
+| 2x time | +0.90 | +86 | **~95** |
+| LMR candidate (10.1) | +0.65 | ~+5 | **~8.5** |
+| the section 5 (original) overnight "dodgy thresholds" run | +0.9-1.0 | even-or-better | ~0-10 |
+
+Two independent LMR configurations convert at under 10 Elo/ply while time
+converts at ~95. **The conversion loss is structural to reduction, not a
+function of how hard it is pushed** -- the more aggressive config did not
+convert better. So "squeeze LMR until it reaches 1.0 ply" predicts roughly
++10 Elo, not +95, and at r~2.0 (below) reaching 1.0 ply means HALVING node
+count, against the ~27% savings achieved today.
+
+Leading explanation (Scott's, and it fits): nominal root depth advances
+honestly (PV nodes are never reduced), but the REFUTATION lines -- the late
+quiet moves at ALL nodes whose fail-low proofs justify the move choice --
+are searched shallower than before. Alpha-beta's answer is only as good as
+its cutoff proofs, and reduction weakens exactly those. Time-bought depth
+deepens everything proportionally; reduction-bought depth deepens the PV
+while thinning its own verification.
+
+### 10.10c. `local_ebf` was dead on arrival; the real branching factor is ~2.0, not ~3.9
+
+Three stacked bugs hid this, found while trying to answer "is our branching
+factor really 3.9?" (Scott's suspicion: it is ~2.0, and the harness's number
+is inflated by aspiration/root-PV-change re-search cost rather than real tree
+growth):
+
+1. `match_play.py`'s `ebf` is `nodes ** (1.0/depth)` on the CUMULATIVE node
+ count -- not a per-ply ratio. Its own docstring (2026-09-18) documents
+ this and says `local_ebf` was added to fix it.
+2. `local_ebf` -- the correct estimator (geometric mean of
+ `nodes_now/nodes_prev` across consecutive completed iterations) -- read
+ `0.000` in every match since it was added. `_ITER_RE` was applied only in
+ the trailing stats parser, but the per-iteration PV lines arrive DURING
+ the search, in the move-scan loop, which read and discarded them.
+3. Even after fixing (2), still 0.000: `Engine.__init__`
+ (`tune_eval_dna.py:120`) sends **`nopost`**, and `root.c:248` gates
+ per-iteration PV output on `g_Options.fShouldPost`, so the lines were
+ never emitted at all. Now opt-in via `match_play.py --post`.
+
+Measured once working, `sd 9`, two engines: **local_ebf 2.02 and 2.12**
+against the same runs' cumulative `ebf` of 4.12 and 4.14. Scott's estimate
+was right and the published ebf has been wrong in every match report in this
+project. Independently corroborated by the time-odds depth response: 2x time
+-> +0.90 ply implies a per-ply ratio of `2^(1/0.90)` ~= 2.1.
+
+Also found: **`--st` takes integer seconds** -- `--st 0.5` yields "Soft time
+limit 00:00.000", i.e. an effectively instant search. Any sub-second `--st`
+value in a past experiment produced garbage games.
+
+### 10.10d. The Crafty gap: 4.7 plies of branching + 2.7 plies of speed
+
+Direct measurement, 12 real-game positions from `tests/cross_sectional_1.ep_`,
+both engines at fixed depths 6-12, 64M hash, single-threaded, book off
+(`/tmp/typhoon/cr_measure/`). Node accounting verified comparable
+beforehand by code reading: Crafty increments per `MakeMove`
+(`search.c:452`, `quiesce.c:185/238/352`, `root.c:101` -- qsearch nodes ARE
+counted), typhoon once per `Search`/`QSearch` entry (`searchsup.c:2019`).
+
+Fitting `nodes(d) = C * r^d`, with `r` taken from consecutive-ratio
+geometric means (NOT from the regression -- `C` and `r` trade off badly over
+a 7-point depth range, visible as absurd `C` of 1,362-4,102 on Crafty's
+low-`r` positions):
+
+| | typhoon | Crafty |
+|---|---|---|
+| **r** (median) | **2.17** | **1.74** |
+| r range across 12 positions | 1.75-2.50 | 1.44-1.99 |
+
+Decomposition of the ~7-ply gap Scott observes in live games (Crafty at ~19
+when typhoon is at 12):
+
+| term | contribution |
+|---|---|
+| branching (r 2.17 vs 1.74 -> depth x 1.391) | **+4.7 ply** |
+| speed (1.43M vs 6.6M nps = 4.6x nodes) | **+2.7 ply** |
+| predicted Crafty depth at typhoon's 12 | **19.4** (observed ~19) |
+
+Two independently measured quantities reproduce the observation with no
+residual.
+
+**A hypothesis this KILLS, recorded because it was mine and it was wrong:**
+that the gap lives in `C` (unit-subtree cost) because typhoon's qsearch is
+too wide. typhoon's qsearch is indeed ~84% of its nodes against Crafty's
+~15% (same position, `sd 8`: typhoon 482,156 nodes / 406,205 qnodes; Crafty
+65,688 nodes / ~10K evals) -- but `C` fits LOWER for typhoon (~287) than
+Crafty (~448), and `C` sits inside the logarithm, so even a 20% `C`
+reduction is worth ~0.3 ply. The qsearch width difference is real and
+costs almost nothing in depth. The earlier arithmetic that made `C` look
+decisive assumed equal `r` and failed a plausibility check (it implied
+Crafty's depth-1 search costs 11 nodes, when it must visit ~35 children).
+
+So: **the branching term is the bigger half of the gap and the one that can
+be attacked.** Scott's instinct to work on search efficiency was right.
+
+### 10.10e. Crafty's lazy-eval rates, measured
+
+Settling a disagreement about whether Crafty's eval is cruder. Crafty has
+TWO lazy exits and the printed fields distinguish them (`iterate.c:624-630`,
+both percentages over denominator `evaluations + lazy_evaluations`):
+`lazy=` is the FIRST (material-only, `evaluate.c:60/64`), `full=` went
+through piece scoring, and the residual is the SECOND lazy exit
+(`evaluate.c:243`, skips piece scoring).
+
+12 positions, `sd 10`: material-only **28.8% mean** (median 30, range 0-45);
+second lazy exit **24.9%**; full eval **46.3%**. So **~54% of Crafty's eval
+calls skip the expensive piece scoring**, which supports Scott's 40-50%
+recollection; the material-only tier alone is ~29%, not the 20% claimed
+earlier in this session from a single position at a single depth (the same
+position reads 30% at depth 10 -- that number was a bad sample, and
+depth-dependent).
+
+typhoon has the matching three-way split (`u64SuperLazyEvals`/
+`u64LazyEvals`/`u64FullEvals` plus cycle counts, `root.c:508-542`) but
+behind `#ifdef LAZY_EVAL`, not in the standard profile -- one rebuild gets a
+like-for-like comparison.
+
+**Caveat on what this can support**: lazy exits fire when the score is
+already far outside the window, i.e. in lopsided positions where detail does
+not matter. These counters measure eval COST, not eval QUALITY where it
+counts. Neither side of the eval-quality argument is settled by them.
+
+### 10.10f. Exchange truncation: the mechanism, and three fixes with very different costs
+
+Scott's symptom: losing a pawn or knight "at the end of an exchange" against
+Crafty, prompting an idea to extend all captures fractionally.
+
+NOT the futility margin -- `iFutility = iAlpha -
+(FUTILITY_BASE_MARGIN_BY_SOURCE[pi->uLastPositionalSource] +
+rgiPositional[...]) - iEval` is indexed by eval's own positional source,
+i.e. genuinely eval-calibrated rather than hand-picked.
+
+The mechanism is the SEE-ordered early exit at the top of `QSearch`'s move
+loop: moves are SEE-sorted, and the first `iValue <= 0` does `goto end`,
+abandoning every remaining move at that node. So qsearch only ever explores
+captures SEE scores as non-losing, and **any exchange whose resolution
+requires passing through a SEE-negative intermediate capture is outside the
+horizon entirely.**
+
+That cutoff is sound only to the extent SEE is right, and SPEC.md already
+documents that it is not, specifically: `_WhoAttacksSquareBB` (see.c) has no
+pin or legality awareness -- the finding that made the queen-vs-queen dumb
+bucket read backwards and got that signal deleted. A pinned defender still
+counts as a recapturer, so SEE calls a winning capture losing and qsearch
+never tries it. Same root cause, different symptom.
+
+| fix | touches | cost |
+|---|---|---|
+| make SEE pin-aware (see.c) | `C` | ~0.3 ply for a 20% `C` rise; also improves every other SEE consumer (`fPruneReduceEligible`'s capture rule, EFP, `MVF_SEE_*`) |
+| admit SEE<0 captures at shallow qsearch depth | `C` | bounded, tunable |
+| extend all captures fractionally | **`r`** | exponential; at ~95 Elo/ply, expensive |
+
+Recommendation: fix the cause on the `C` side before spending `r`.
+
+### 10.10g. Next steps implied
+
+1. **Measure Delta-r directly for extension changes, do not infer it from
+ the counters.** `root.c:568-577` prints every extension source count, but
+ those count GRANTS, before `MINU(ONE_PLY, ...)` caps and before
+ `g_uExtensionReduction[]` decays -- so they cannot give the applied ply
+ inflation that actually moves `r`. The 12-position C/r sweep
+ (`/tmp/typhoon/cr_measure/run.sh`) is now the right instrument: build with
+ extensions trimmed toward Crafty's single SEE-gated 1-ply check and
+ re-run it. Crafty has exactly ONE extension (`search.c:476/483`, checks
+ only, `check_depth=1`, SEE-gated, and it disables reductions for that
+ move); typhoon has ten fractional per-move sources plus five node-level
+ ones. Whether typhoon's 3/4-ply-but-decayed check extension "washes out"
+ against Crafty's 1-ply-but-SEE-gated one is exactly the question a Delta-r
+ measurement answers and counter arithmetic does not.
+2. **Route matters more than destination.** Closing `r` from 2.17 to 1.74 is
+ worth 4.7 plies ~= 450 Elo at the 10.10a yardstick. But reduction-bought
+ plies convert at ~8.5 Elo/ply (10.10b), so the question is not "how do we
+ lower r" but "how does Crafty lower r SOUNDLY". Extension trimming is the
+ first candidate identified that is not "reduce harder": it declines to
+ spend effort rather than suppressing information.
+3. **Clamp removal is off the table.** Stockfish floors the reduced depth at
+ 1 ply (`search.cpp:1370`, `std::max(1, ...)`) -- the same invariant
+ typhoon has -- and achieves its leanness through separate explicit
+ pruning (Step 15, `search.cpp:1166`). Crafty has no such floor and does
+ not even verify a move reduced into qsearch (the full-depth re-search at
+ `search.c:957` is inside the `depth > 0` branch only). The strongest
+ engine keeps typhoon's invariant. Combined with 10.9a's arithmetic
+ (+0.017 root ply), two independent lines say drop this idea.
+4. The 10.9b hang-bucket-9 pruning lead is unaffected by any of the above
+ and remains the best-evidenced pruning target (3.3% of eligible tries,
+ observed FH 0.0112%, 44x safer than hang bucket 0).
+
+## 10.11. Where the branching gap actually comes from, and the prioritized backlog (2026-10-03)
+
+### 10.11a. Crafty's r advantage is move-list TRUNCATION, not reduction grading
+
+Settling the "should we grade reduction magnitude harder" question for good.
+Crafty's entire LMR adjustment, `search.c:579-581`:
+
+```c
+reduce = LMR[Min(depth, 31)][Min(order, 63)];
+if (reduce && (pv_node || extend))
+ reduce--;
+```
+
+One decrement. No history, no improving, no node-type, no per-move signal --
+and it reaches r = 1.74 against typhoon's 2.17. **So adjustments cannot be
+where their leanness comes from.** (Stockfish does have a dozen such
+adjustments; Crafty does not, and Crafty is still far leaner than typhoon.
+This retires the "typhoon's candidate is SF's base with no adjustments"
+theory floated earlier in this session, and matches Scott's report of five
+prior LMR attempts with assorted factor-mucking, none of which paid off.)
+
+What sits immediately ABOVE those lines in Crafty's move loop does the work:
+
+```c
+if (depth < FP_depth && !check && MaterialSTM(wtm) + FP_margin[depth] <= alpha && !pv_node) {
+ tree->moves_fpruned++; break; // abandons ALL remaining moves
+}
+if (order > LMP[depth] && depth < LMP_depth && !pv_node && !check && ...) {
+ tree->moves_mpruned++; break; // abandons ALL remaining moves
+}
+```
+
+Measured in one `sd 8` sample: `fp=10.0K mcp=10.3K` against `nodes=65688` --
+**~20,000 move-list truncations in a 66,000-node search.** typhoon has EFP (a
+per-move `continue`) and a disabled LMP, and **no truncation mechanism at
+all**. That is the structural difference behind the 4.7-ply branching term
+in 10.10d.
+
+### 10.11b. "We are holding the 4D table wrong" -- the specific sense
+
+The table is currently consulted for reduction ADMISSION at a threshold that
+vetoes **2.3% of the cell space** (733 of 21,780 cells read >= 500, and 224 of
+those are in the sentinel'd dumb bucket 1). Section 8 (original) removed it
+from magnitude; section 6 (original) demoted the dynamic half to extras-only.
+Step by step a surface with a ~28,000x dynamic range in predicted FH% was
+reduced to making a near-binary decision that fires on ~10% of traffic.
+
+Its dynamic range only earns anything where the decision's stakes are
+correspondingly asymmetric:
+
+- A **reduction** is recoverable and its payoff is a ply or two off one
+ subtree. Fine-grained risk estimates are wasted there -- and 10.8 measured
+ exactly that: table-driven magnitude 0.224 mean extra plies vs the flat
+ `log(d)*log(m)` table's 0.244. A wash.
+- A **prune** is irreversible and its payoff scales with SUBTREE SIZE
+ (~20,000 nodes at depth 6 vs 30-100 near the leaf). That is where
+ "0.0112% observed FH across 2.9M tries, 44x safer than baseline" is worth
+ having, and where Crafty operates blind on depth-and-movecount alone.
+
+**So: Crafty gets r = 1.74 by truncating move lists on a crude rule, and we
+have a measured FH surface they do not. Holding the table right means using
+it to decide WHERE TO TRUNCATE.**
+
+### 10.11c. Scott's dimension argument: a sound derivation of Crafty's `break`
+
+Of the four axes, only **dumbness** is genuinely move-specific:
+
+| axis | scope |
+|---|---|
+| gap to alpha | node-level constant |
+| remaining depth | node-level constant |
+| lateness (move number) | move-specific, but MONOTONE -- every later move is at least as late |
+| dumbness (hang cp) | move-specific, varies arbitrarily |
+
+Therefore: if the table **marginalized over dumbness** reads "never fails
+high" for this (depth, gap, late) cell, that read applies at least as
+strongly to every remaining move at the node, because two axes are fixed and
+the third only worsens. That justifies abandoning the rest of the move list
+-- Crafty's `break`, derived from our own data instead of assumed.
+
+**This does NOT contradict the 2026-09-27 dumb-axis-pooling bug**, and the
+distinction is subtle enough to be worth stating: that bug was
+`ConfirmedZeroFailHigh` using the dumb-SUMMED marginal to vouch for a
+*specific* move sitting in a rare, dangerous dumb bucket -- unsound, because
+the cell's zero was produced by ordinary moves dominating its volume while
+the rare queen-hanging move had never been tested on its own. The argument
+here runs the opposite direction: it uses the pooled read to vouch for the
+**dominant** population. Since dumb=0 is 82.3% of all tries, a pooled rate
+of ~0 mathematically requires the dumb=0 subpopulation to be ~0 too, so
+pooling is CONSERVATIVE for this purpose.
+
+Preferred implementation (Scott's): per-move `continue` rather than one
+`break`, so the move-specific dumbness veto still applies to the move in
+hand. Nearly the same node savings, none of the blanket assumption.
+
+### 10.11d. The near-leaf depth ceiling is backwards and should be deleted
+
+`ShouldLatePrune`'s `LMP_MAX_REMAINING_DEPTH <= 6*ONE_PLY` was inherited
+from Crafty, where a near-leaf restriction IS the whole risk model because
+they have no table. Our own table shows predicted FH% FALLING ~25x from
+depth bucket 0 to bucket 11, so the ceiling permits pruning where risk is
+highest and forbids it where risk is lowest (already documented in
+section 7 (original), never patched).
+
+The cost of a wrong prune does rise with remaining depth -- which is what a
+rule-of-three CONFIDENCE BAR is for, not a depth cutoff. Per Scott: pruning
+a move with depth 11 under it is fine *if* the static read says "no way" and
+the dynamic vetoes are silent; benefit rises exponentially with remaining
+depth while measured risk falls.
+
+### 10.11e. Per-cell sample counts are available NOW -- no re-collection
+
+`joint_table_collection/pickles/raw_2026-10-02.pkl` survives and holds
+per-cell `(fh, tries)` for all 20,075 populated cells; `grand_tries` =
+690,062,068,603 over 56,530 positions. Running SPEC.md's rule-of-three bars
+(`N ~= 3 * applications_per_game / target_misses_per_game`, ~4,800
+applications/game):
+
+| miss budget | required N | zero-FH cells clearing it |
+|---|---|---|
+| 1 game in 20 | 288,000 | **43** |
+| 1 game in 100 | 1,440,000 | **9** |
+| 1 game in 1000 | 14,400,000 | 0 |
+
+This reproduces section 7 (original)'s "43 cells" lead exactly and adds the
+1/100 tier. **The blocker named in section 7 -- that `g_uJointFHThousandths`
+ships only the rate, not the sample count behind it -- is a
+`splice_joint.py` + `data.c` change against an existing pickle, not a new
+collection run.**
+
+### 10.11f. Prioritized backlog
+
+**1. HIGHEST IMPACT -- deep pruning via the table (10.11b/c/d/e).**
+Rationale: Crafty's 4.7-ply branching advantage is demonstrably truncation
+(10.11a); typhoon has no truncation; the payoff is exponential in remaining
+depth; the data exists; 43 cells already clear a 1/20-games bar. Closing r
+from 2.17 toward ~1.9 is worth ~1.6 ply ~= ~150 Elo at the 10.10a yardstick
+-- an order of magnitude more than anything else on this list.
+Steps: (a) ship per-cell tries via `splice_joint.py`/`data.c`; (b) build the
+prune-specific shadow-verification harness FIRST, with the
+`NEUTRALIZE_NULL_BAILOUT` fix applied from day one (SPEC.md's agreed design
+requirement #4); (c) re-point `ShouldLatePrune` at a rule-of-three-cleared
+sample bar + dumb-marginalized monotonicity criterion, deleting the depth
+ceiling; (d) keep the dynamic per-move-identity veto (requirement #5).
+
+**2. CHEAP AND UNATTENDED -- HEAD vs HEAD with LMR disabled.**
+The control condition 25 years of this work has lacked. The 10.10a yardstick
+measures what TIME-bought depth is worth; nothing has ever measured whether
+HEAD's own flat -1 ply LMR converts. If removing it costs 0.5 ply and ~45
+Elo, reduction-bought depth does convert for HEAD and the candidate broke
+something specific. If it costs 0.5 ply and ~5 Elo, reduction-bought depth
+has never been worth anything in this engine and the answer is LESS
+reduction, not better reduction. One build, one match, runs while other work
+proceeds. Use `SPRT_ARGS="--sprt --elo0 0 --elo1 15"`.
+
+**3. DURABLE INFRASTRUCTURE -- the move-agreement test.**
+Replaces curated-suite solve counts (shown to be noise at these sample sizes
+in 10.6) with a metric that has a usable signal-to-noise ratio:
+ - take ~300 real-game positions (`tests/cross_sectional_1.ep_`);
+ - build an ORACLE best move per position -- ideally Crafty at high depth,
+ independently implemented and cheap for them;
+ - run each candidate at a fixed `sn` budget; record % agreement with oracle.
+Every position contributes a signal (not just the ~20% that are tactical
+puzzles with a known answer), there is no dependence on EPD solution
+correctness, and 300 positions gives a binomial SE of ~2.9%.
+Immediate use: at EQUAL node budget, does the LMR candidate pick the oracle's
+move less often than HEAD? Yes => its extra nominal depth costs real decision
+quality. No => the depth was simply never worth anything (a different problem
+with a different fix). Refinement: score both moves with the oracle and only
+count a disagreement when it prefers its own by a real margin.
+
+**4. SMALL BUT CLEAN -- drop the six negligible extension sources.**
+From a real `sd 15` dump: `zug` fired **9** times, `endg` 352, `mult+` 1,410,
+`recap` 10,339, `sing` 10,296, `q+` 35,427 -- against the check extension's
+1,483,277. These cannot affect anything at those volumes and need no gate;
+just confirm grant counts go to zero and node counts do not move. Weighted
+by ply amount the check extension is **~78% of all extension spend**; every
+other source combined is ~0.35 ply ~= a ~33 Elo ceiling, and individually
+3-9 Elo each -- unmeasurable by self-play (~25,000 games to separate 5 Elo
+from 0). So test the mid-tier (threat + bm + 1mv + !kmvs + pawn) as ONE
+bundle, not five matches.
+
+**5. SCREEN-ONLY -- check-extension magnitude variants.**
+Non-negotiable as a mechanism (Scott: prior attempts at SEE-gating caused
+sacrifice-blindness; "king moves out" variants exploded the branching
+factor). But the AMOUNT and gate are open: 3/4 -> 1/2 ply; material gate ->
+Crafty's SEE gate; 3/4 -> 1 full ply (Crafty's amount -- extensions are not
+automatically bad). ~4 minutes each on the C/r sweep
+(`/tmp/typhoon/cr_measure/run.sh`). Note the 3/4-ply SEE>=0 branch is
+700,479 grants = 2.1M of ~2.9M check quarter-plies, i.e. 57% of ALL
+extension spend -- that is the branch worth screening, not the 1/4-ply one.
+
+**6. DEAD -- do not revisit.**
+ - *LMR magnitude grading / per-factor adjustment*: five prior attempts,
+ plus 10.8 (table vs flat is a wash) plus 10.11a (Crafty has one
+ decrement and is leaner anyway).
+ - *Clamp removal / reducing into qsearch*: 10.9a arithmetic (+0.017 root
+ ply) plus Stockfish flooring reduced depth at 1 ply
+ (`search.cpp:1370`) while getting its leanness from separate explicit
+ pruning.
+ - *SEE<0 check extension removal*: measured Delta-r = -0.004 (median over 12
+ positions), a ~3 Elo ceiling. Also independently reported to cause
+ sacrifice-blindness in earlier work. The counter-based estimate that
+ sized this at 21% of extension spend was wrong by ~20x because
+ `uCheckByQuarterPly[1]` pools the SEE<0 and low-enemy-material branches
+ -- **screen with Delta-r, never size an extension from grant counts.**
+ - *Attacking qsearch width to close the Crafty gap*: 10.10d -- `C` fits
+ LOWER for typhoon (~287) than Crafty (~448), and `C` sits inside the
+ logarithm, so a 20% change is worth ~0.3 ply.
+
+## 10.12. The control experiment, the conversion-rate model, and why flat LMR is hard to beat (2026-10-03, later)
+
+### 10.12a. LMR vs no LMR -- the control condition 25 years of this work never had
+
+`DISABLE_LMR` (GNUmakefile) makes `GetLMRReduction` return 0 before consulting
+any signal. Two binaries from the SAME tree, differing only in that flag, run
+head-to-head with NO SPRT (measuring a quantity, not testing a hypothesis --
+same reasoning as the time-odds calibration in 10.10a). `noLMR` is the
+baseline, so a positive score means LMR helps.
+
+| games | score | Elo | depth (withLMR vs noLMR) |
+|---|---|---|---|
+| 50 | 0.550 | +35 +/- 69 | 13.75 vs 12.59 (+1.16) |
+| 90 | 0.550 | +35 +/- 51 | 13.96 vs 12.79 (+1.17) |
+| 110 | 0.559 | +41 +/- 46 | 13.72 vs 12.57 (+1.15) |
+| 150 | 0.553 | +37 +/- 40 | 13.59 vs 12.43 (+1.16) |
+| 180 | 0.575 | +53 +/- 36 | 13.51 vs 12.36 (+1.15) |
+| 210 | 0.569 | **+48 +/- 34** | 13.54 vs 12.35 (+1.19) |
+
+**LMR is worth roughly +40-50 Elo and buys ~1.17 ply.** Remarkably stable
+across six checkpoints. The depth delta is the largest and steadiest of any
+candidate in this document's history.
+
+This settles a question nobody had ever asked: LMR *does* earn its keep here.
+Scott's prior ("LMR has never earned its keep in this engine") was wrong, and
+so was the stronger form of the author's "reduction-bought depth is debased"
+thesis.
+
+### 10.12b. The conversion-rate model: diminishing returns, not a flat rate
+
+Pulling every Elo-per-ply measurement together:
+
+| mechanism | what it buys | measured rate |
+|---|---|---|
+| **LMR vs NO LMR** (the FIRST reduction) | +1.17 ply | **~40 Elo/ply** |
+| graded LMR *on top of* flat LMR | +0.65 ply | ~8 Elo/ply |
+| pruning (`PRUNE_LIVE=10`) *on top of* graded LMR | +0.88 ply | ~8 Elo/ply |
+| 2x time (NEW resource, not a transfer) | +0.90 ply | ~95 Elo/ply |
+
+That is a decreasing-marginal-returns curve, and it is a better model than the
+flat "~8 Elo/ply for all tree-shaping" claim made earlier in this session.
+
+**Why the earlier claim was wrong**: the graded-LMR and prune candidates were
+both measured against `head_reference`, WHICH ALREADY HAS LMR. Their +0.65 and
++0.88 ply were MARGINAL plies stacked on an already-reduced tree. 10.12a
+measures the FIRST ply of reduction, which is where the value is.
+
+**Practical calibration (Scott's framing, and it is the right one)**: there is
+not another 40 Elo available from LMR/LMP/pruning work. There is maybe 40-45
+Elo TOTAL in the whole reduction program, of which the dumb flat -1 ply
+already captures ~85%. Five failed attempts to improve on it do not mean it
+cannot be improved -- they mean expectations should be calibrated to single
+digits, and that single digits cannot be measured in affordable self-play
+(resolving +5 vs 0 needs ~10,000 games).
+
+**ALSO NOTE a yardstick error this corrects.** 10.10a divided the time-odds
+result (+86 Elo for +0.90 ply) and called it "~95 Elo per ply," then used that
+to compute ceilings all session. That division assumes depth is the ONLY
+channel through which extra nodes help. It is not: at fixed nodes, a ply bought
+by shrinking the tree is worth far less. So "95 Elo/ply" is the rate for
+TIME-bought plies specifically and must never be applied to tree-shaping
+changes. Every ceiling computed from it earlier in this document was inflated
+(e.g. `PRUNE_LIVE=10`'s ceiling was quoted as 33 Elo; corrected to ~7, and it
+measured +7 -- it hit its real ceiling exactly rather than underperforming).
+
+### 10.12c. Why a flat -1 ply is so hard to beat -- the actual mechanism
+
+The curve is already present (`_FlatTableExtraPlies` reads
+`g_iLMRQuietReduction`, dynamic.c's `log(depth)*log(moves)` table). The problem
+is WHERE it can act. Evaluating `0.7844 + log(d)*log(m)/2.4696`:
+
+| remaining depth | moves tried | formula | extra plies |
+|---|---|---|---|
+| 2 | 16 | 1.56 | **0** |
+| 3 | 16 | 2.02 | 1 |
+| 4 | 16 | 2.34 | 1 |
+| 10 | 20 | 3.58 | 2 |
+
+**At remaining depth <= 2 the curve prescribes ~1 ply -- which IS flat -1.**
+And 76% of the LMR population sits at depth buckets 2-3 (10.8). So flat is not
+a crude approximation of the curve; at the depths where the mass lives, the two
+AGREE.
+
+Measured consequence: the curve contributes mean **0.164 extra plies** over the
+LMR population, which through section 5 (original)'s damping (~1.0 extra
+ply/decision -> +0.29 root ply) is about **+0.05 root ply**. That is the
+curve's entire current contribution, and it explains the null results directly.
+
+The curve's remaining headroom is the ~11% of moves at remaining depth >= 4,
+further limited by `iMaxExtra = depth - 3` (the one-full-width-ply invariant).
+
+**Open and never cleanly tested**: curve-vs-flat in self-play. The 2212-game
+run (section "Self-play status", 2026-09-29) tested the joint-table TIER
+magnitude under the pre-unification `uLegalMoves > 5` gate, NOT section 8's
+`log(d)*log(m)` version, which was only ever validated on sd10 node counts and
+sn5M suites. An earlier draft of this section wrongly treated that null as
+having answered the question.
+
+### 10.12d. Retraction: eNodeType is too unreliable to modulate reduction
+
+10.7b and 10.9c recommended using `pi->eNodeType` to reduce more at CUT nodes
+than ALL nodes. **Withdrawn.** After fixing the node-type surprise counters
+(10.12f) the full-width rates read: ALL nodes fail HIGH 69-71% of the time,
+CUT nodes fail low 12-14%. An "ALL node" that fails high two-thirds of the time
+is not behaving like an ALL node -- the structural label (assigned purely from
+the parent's type) is close to uninformative here. Modulating reduction off a
+signal that unreliable adds complexity for noise.
+
+### 10.12e. QSearch / QSearchFromCheckNoStandPat review
+
+Reviewed for logic errors in the custom stand-pat and check-generation rules.
+
+**SHIPPED: option B -- waive the opponent-stand-pat requirement at PV nodes.**
+`fIncludeChecks`' first clause suppressed check generation whenever the
+opponent could stand pat at an ancestor. That suppression rests on a sound
+argument *for null-window nodes*: once their stand-pat has raised alpha, no
+line below that score can change what they return, so resolving mate-in-10 vs
+"drops a queen" is wasted work -- and even when their stand-pat did NOT raise
+alpha, the node is heading for a fail-low, which IS the parent's beta cutoff,
+so any score at or below alpha answers the question identically. It does NOT
+hold at a WIDE-WINDOW node, where the exact score propagates to the root and
+those two outcomes are a different PV and a different reported score -- which
+is the forced-mate-from-qsearch case this machinery exists for. Now:
+`(fCouldStandPat[FLIP(uToMove)] == FALSE) || (NODE_PV == pi->eNodeType)`,
+keeping the material requirement (an independent sanity check, not a
+suppression). Reads eNodeType, never writes it. Uses eNodeType rather than
+recomputing `(iBeta != iAlpha+1)` because iAlpha may have just been raised by
+the stand-pat, which would make a recomputed width test read narrow for a node
+that is genuinely PV.
+
+**SHIPPED: `iExtensionAmount` unit fix.** `QSearchFromCheckNoStandPat` wrote
+`uQsearchCheckExtension` (0 or 2, counting qsearch CHECK PLIES) into a field
+that is ONE_PLY-scaled everywhere else (root.c writes HALF_PLY, split.c writes
+iLMR, search.c writes iExtend) -- a latent 32x error. Benign as it stood (both
+readers only test sign) but the field is already a documented stale-state
+hazard. Now scaled; sign and all reader behavior unchanged.
+
+**SHIPPED: `pi->iEval` published** when `QSearchFromCheckNoStandPat`'s lazy
+`Eval()` actually runs. It used to stay purely local, so that ply slot kept a
+stale value. Still stale on the common in-check path where Eval is never called
+at all (no stand-pat is offered, so there is nothing to compute it for) --
+making that safe needs invalidation on entry, a separate change.
+
+**TRIED AND REVERTED THE SAME DAY: `iBestScore = MAX(iAlpha, iEval -
+iInTrouble)`.** The observation behind it was real -- the beta test lives only
+in the else-branch, so a node with `iEval >> iBeta` and hanging material
+returns a fail-low instead of cutting off. The fix was wrong twice over, and
+self-play caught it in 10 games (candidate depth collapsed to +0.05 ply vs
+head, from the +0.65 the same tree showed without it):
+
+ Work the signs. A parent searching this node passed window
+ `(-iBeta_p, -iAlpha_p)`, so `iAlpha` here == `-iBeta_p`. The parent fails high
+ iff `-V >= iBeta_p`, i.e. iff `V <= iAlpha`. So returning exactly `iAlpha` is
+ not a crude placeholder -- **it is precisely the value that hands the parent
+ its cutoff.** Returning MAX(...) instead reports a better-than-alpha value and
+ denies that cutoff (~0.45 ply of lost depth), AND asserts the node is worth at
+ least `iEval - iInTrouble` without searching anything -- a stand-pat credit at
+ a discount, exactly what the branch exists to refuse.
+
+`MIN(iAlpha, iEval - iInTrouble)` was then proposed and also rejected: it is
+depth-neutral (V <= iAlpha always, so the cutoff always fires) but it cannot
+produce the cutoff the original observation was about (MIN never exceeds
+iAlpha), and it asserts an unearned TIGHTER upper bound, which propagates as an
+inflated lower bound into the parent's `StoreLowerBound`. The original
+observation is still unfixed; a sound fix needs a bound the node has EARNED.
+Probably moot: a node with material in trouble almost always has a capture
+available, and captures sort above the `iValue <= 0` early exit, so the move
+loop usually recovers the cutoff by search.
+
+**Confirmed correct** (checked, no change needed): the qsearch-entry reset
+invariant (`search.c:573` and `command.c:739` clear `fCouldStandPat[]`,
+`uQsearchNodes`, `uQsearchDepth` and reset `uQsearchCheckDepth` before every
+drop into qsearch; the only other QSearch call site is
+`CALIBRATE_QSEARCH_FUTILITY`-only and already inside a qsearch state);
+`fCouldStandPat` scoping (each node saves/restores only its own side's flag and
+never writes the opponent's, so `fCouldStandPat[FLIP(uToMove)]` genuinely
+reflects the parent); paired `uQsearchDepth`/`uQsearchCheckDepth` increments
+with no `goto end` between any pair; and the `uLegalMoves > 0` guard on escape
+futility pruning, which correctly prevents a bogus mate score even when leading
+escapes are illegal.
+
+**The exchange-truncation mechanism**, for the record: it is NOT the futility
+margin (`iFutility = iAlpha - (FUTILITY_BASE_MARGIN_BY_SOURCE[
+pi->uLastPositionalSource] + rgiPositional[...]) - iEval` is genuinely
+eval-calibrated). It is the SEE-ordered early exit at the top of QSearch's move
+loop: moves are SEE-sorted and the first `iValue <= 0` does `goto end`,
+abandoning every remaining move. So qsearch only explores captures SEE scores
+as non-losing, and any exchange whose resolution needs a SEE-negative
+intermediate capture is outside the horizon. That cutoff is sound only to the
+extent SEE is right, and SPEC.md already documents `_WhoAttacksSquareBB`
+(see.c) having no pin/legality awareness. Three fixes with very different
+costs: make SEE pin-aware (`C`-side, ~0.3 ply for a 20% C rise, and improves
+every other SEE consumer); admit SEE<0 captures at shallow qsearch depth
+(`C`-side, bounded); extend all captures fractionally (`r`-side, exponential).
+Fix the cause on the C side before spending r.
+
+### 10.12f. Node-type surprise line: four defects
+
+Scott's question ("how can 47% of ALL nodes fail high?") was right, and chasing
+it found four separate defects in one Trace line:
+
+1. **Denominator was total nodes**, not nodes of that type.
+2. **Qsearch was included** (~84% of the tree). A qsearch node whose stand-pat
+ beats beta returns `iBestScore >= iBeta` immediately; typed ALL, that reads
+ as "ALL failed high" while being the most ordinary event in qsearch. Now
+ excluded from numerator (QSearch's copy MOVED to its own slots) and
+ denominator (`u64NodeTypeCount`, incremented in `Search()` only).
+3. **Stale node types.** Every early exit in `CommonSearchInit` returns TRUE
+ *before* `pi->eNodeType` is assigned, and the caller's `goto end` lands on
+ the counters -- which scored the node against whatever type the PREVIOUS
+ node at that ply left behind. That produced an impossible "106.2% of PV
+ nodes". Now guarded by `fNodeTypedThisVisit`.
+4. **The fail-low test used the in-place-raised `iAlpha`.** `search.c:2048`
+ does `iAlpha = iScore`, so a node that SUCCESSFULLY raised alpha ends with
+ `iAlpha == iBestScore` and `(iBestScore <= iAlpha)` reads true -- every
+ success counted as a fail-low. The PV row read 99.1%. Now uses
+ `iInitialAlpha`, which already existed for exactly this distinction
+ (`search.c:2105`).
+
+Also added, per Scott's QNODE suggestion applied to the counters rather than
+the enum: `u64QNodeTypeCount`/`u64QNodeSurprise`, plus a **PVS identity
+cross-check**. The identity: a CUT node failing low returns `score <= alpha_C`,
+so its parent (an ALL node) sees `-score_C >= beta_P` and fails HIGH, and that
+parent cannot have had an earlier child fail low (it would have cut off first)
+-- so #(CUT fails low) and #(ALL fails high) should run ~1:1. Counting qsearch
+separately brought the measured ratio from 3.6 to **1.86**. The residual means
+~46% of ALL-node fail-highs come from nodes resolved with NO child fail-low at
+all: qsearch stand-pat cutoffs, interior-node recognizer/EGTB fail-highs, and
+null-move cutoffs (including the Tord Romstad bailout, which fabricates a score
+and searches nothing). The NODE_TYPE enum and every behavioral consumer (all of
+which only test against NODE_PV) are untouched.
+
+### 10.12g. PERF_COUNTERS cleanup -- the standard profile was carrying investigation debris
+
+An audit for real WORK (not counter increments) inside `#ifdef PERF_COUNTERS`
+found five sites in three categories. All three were fixed.
+
+**Category 1 -- three `rdtsc` calls leaking into the standard build.**
+`eval.c`'s king-safety hash probe bracketed itself with
+`SystemReadTimeStampCounter()` on entry and exit, in one of the hottest paths
+in the engine; `rdtsc` is ~20-30 cycles and partially serializing. An
+`EVAL_TIME=1` flag already exists for exactly this ("make a version that counts
+cycles spent in eval") and already guards the other **32** rdtsc sites in that
+file -- these three were the only leaks. Moved to EVAL_TIME. (The report side
+in root.c was already EVAL_TIME-guarded, so nothing dangles.)
+
+**Category 2 -- two one-off diagnostics, deleted outright.** Both did per-
+decision lookups plus integer division at the hottest decision point in the
+search, and both had comments saying to remove them once their investigations
+concluded:
+ - `u64HeadStyleEligible`/`u64HeadStyleWouldReduce` (called
+ `GetMoveFailHighRawCounts`). Its investigation concluded -- it produced
+ 10.2's 92-94%-vs-70-75% admission finding.
+ - `u64FHJointHist` (called `GetMoveFailHighPctThousandths`). Also concluded --
+ it produced section 4 (original)'s bimodal-dynamic-rate finding.
+Removed the counters, the `g_Options` mirrors, the suite accumulation and BOTH
+report blocks -- a diagnostic that prints zeros forever is worse than none.
+`AttemptsBucket`/`RateBucket` went with them. Binary shrank 7,760 bytes.
+
+**Category 3 -- the 4D collection block, re-gated to FH_TABLE_COLLECTION.**
+`search.c`'s `u64JointTries`/`u64JointFH` collection calls
+`EstimateHangingMaterialCp` (real attack-bitboard work) per eligible move. It
+is COLLECTION instrumentation, and CLAUDE.md already says FH_TABLE_COLLECTION
+is deliberately kept out of the standard profile *because this measurably costs
+nps* -- this block had escaped that gating. Verified by guard inspection:
+`search.c:1637` is now `PERF_COUNTERS > FH_TABLE_COLLECTION`; the only other
+`EstimateHangingMaterialCp` call site is the prune predictor, correctly behind
+`PRUNE_VERIFY`/`PRUNE_LIVE`.
+**Bonus:** it was computing the hang bucket TWICE per move in any build with a
+prune gate (the gate needs the same value), which inflated the `PRUNE_LIVE`
+candidate's nps deficit (-6.0% vs head) and made that match mildly unfair to
+the candidate.
+
+**nps impact: NOT MEASURED.** Two attempts were made and both are worthless
+because the box was running a 32-engine self-play match. The second attempt
+produced `perf_clean` FASTER than `noperf`, which is impossible (the cleaned
+build still has every counter increment `noperf` lacks) -- proof that noise
+exceeded the effect. Also a methodology error worth not repeating: the script
+ran all of arm A, then all of B, then all of C, so a load drift masquerades as
+an effect. **Re-measure on an idle box with ROUND-ROBIN interleaved arms**,
+fixed node budget (`sn 8000000`), same position, 3+ runs each.
+
+### 10.12h. The deep-prune audit and miss verification
+
+`PRUNE_VERIFY` (GNUmakefile) -- audit-only shadow verification for the prune
+gate. It evaluates the table's prediction for every `fPruneReduceEligible` move,
+records it banded by predicted FH%, then **declines to prune** and watches
+whether the move fails high in its own ordinary search. No bespoke shadow
+`Search()` call is needed, which sidesteps the null-move-bailout fabrication
+that made `JOINT_EXTRA_VERIFY`'s first numbers 80-99.8% artifact. Banding by the
+decision variable means one run answers "is `prune if pred <= K` sound" for
+every candidate K at once.
+
+Pooled over the three curated suites (65.6M flagged decisions), measured FH%
+ran **3-19x BELOW predicted** in every prunable band -- the Wilson re-bake
+(10.10/10.11e) doing what it was for. But `cross_sectional` positions (250
+samples, the real-game distribution) came back **5-11x WORSE than the curated
+suites** in the low bands, with `measured > predicted`. The explanation is
+distributional and inverts SPEC.md's stated worry: in a tactical puzzle the
+action is in forcing moves, which gate-1 excludes by construction, so leftover
+quiet moves there really are the dregs; in an ordinary middlegame the best move
+is frequently a quiet one, landing squarely in the leftover pool. **Validating
+a prune gate on puzzles understates its risk for real games.**
+
+Verification of the 71 low-band (`pred <= 2`) cross_sectional misses against
+clean deeper searches: **27 confirmed real, 44 artifacts.** The margin column
+discriminates cleanly -- `margin ~= 0` means artifact (a score that only just
+clears a tight inherited PVS window), huge margin means a genuine forced mate.
+That inverts the author's initial guess, which had it backwards.
+
+One confirmed band-0 miss is worth keeping as a counter-example to the dumb
+axis: `6k1/p4p1p/3p2q1/2p1rP2/2P3p1/P7/1P1Qn1PK/3R1R2 w`, after `Qg6`. The
+queen only LOOKS hung -- White is losing too badly to take it (best line grabs
+the e2 knight; White is -14 at depth 19), so a move the table rates
+"obviously hangs material for nothing, never fails high" is actually winning.
+A structural mispredict that no amount of sampling conservatism catches, which
+is precisely why this harness had to exist before pruning went live.
+
+**Prune threshold measurements** (12-position C/r sweep, paired per-position
+deltas -- note the author initially reported difference-of-medians instead of
+median-of-differences, which flipped the sign on two of three arms):
+
+| threshold | share of eligible | median dr | ply @d12 | verified miss rate |
+|---|---|---|---|---|
+| `pred <= 2` | 4.3% | -0.0040 | +0.03 | ~1 in 481,000 |
+| `pred <= 5` | 16.6% | -0.0281 | +0.21 | ~1 in 360,000 |
+| `pred <= 10` | 30.6% | -0.0465 | +0.35 | ~1 in 121,000 |
+
+Band 6-10 alone carries 79% of the misses for 29% of the flagged volume, so
+`pred <= 5` keeps ~60% of the depth for ~16% of the risk. `PRUNE_LIVE=10`
+self-play: +0.88 ply, +7 Elo at 150 games (llr flat) -- which is exactly its
+corrected ~7 Elo ceiling (10.12b), so it performed as well as it possibly could
+and that is still not interesting.
+
+Also measured and worth recording: **a dynamic veto cannot save these prunes.**
+Cross-tabulating prunable-band candidates by (own-FH attempts x own-FH rate),
+11 of 23 low-band misses sit at **rate = 0** -- the per-move history agrees with
+the static table and both are wrong. A rule catching the 12 high-rate misses
+would refuse ~390,000 sound prunes to do it (~49,000 per miss prevented), the
+same failure mode that retired the `GetMoveFailHighPercentage` veto before.
+
+## 10.13. Open action items as of 2026-10-03 EOD
+
+Ordered by expected value. Items 1-2 are the only ones with a plausible
+double-digit Elo payoff.
+
+### 1. Give EFP the `dumb` axis -- the best remaining idea
+EFP is keyed on gap-to-alpha and blind to the other three. Measured over EFP's
+actual operating envelope (10.2): the **dumb axis swings FH% 15-30x** within an
+otherwise-fixed cell while **gap -- EFP's only lever -- swings ~4x** across its
+entire knight-to-rook range. EFP is reading the weakest signal and ignoring the
+strongest. And unlike every LMR/LMP experiment, this is an **accuracy** play
+rather than a depth purchase: EFP prunes the same volume either way, so it is
+not buying marginal plies at ~8 Elo/ply and nothing measured so far bounds its
+upside. It also finally gives the 4D table a consumer that uses its dynamic
+range. Caveat from section "the tactical-position gap": the first table-driven
+EFP attempt lost ECM.272 and four others and threshold-tightening rescued only
+one -- diagnosed as a training-population problem. The table has been
+re-collected and Wilson-re-baked since, so retry rather than assume.
+
+### 2. The nps gap
+Crafty measured at **5.5M nps** on a real 10-second search (an earlier 7.5M
+figure came from a 0.01s search where the timer is meaningless -- do not quote
+it) against typhoon's ~1.4M. Even discounting for Crafty counting illegal moves
+as nodes (`search.c:452` increments before the legality test), that is roughly
+4x. Time-bought depth converts at ~95 Elo/ply, so the speed term in the Crafty
+gap (~2.7 ply) is worth ~250 Elo -- the only lever on the table with that
+magnitude. Scott's profiling says abandoning the 0x88/bitboard hybrid buys
+maybe 500knps, which does not close it, and the mechanism for their remaining
+advantage is not understood.
+
+### 3. Re-measure the PERF_COUNTERS cleanup on an idle box
+Round-robin interleaved arms, `sn 8000000`, same position, 3+ runs each. Two
+prior attempts are worthless (see 10.12g). Arms are built:
+`/tmp/typhoon/candidates/typhoon_{perf,perf_clean,noperf}`.
+
+### 4. Curve-vs-flat LMR, cleanly
+Never tested in self-play (10.12c). One flag: current tree vs current tree with
+`_FlatTableExtraPlies` forced to 0. Expected ~1-3 Elo from the arithmetic, so
+self-play cannot resolve it -- measure **node count at fixed depth** instead,
+where 0.164 mean extra plies should be visible.
+
+### 5. Decide the fate of the 4D table
+Its only live shipping consumer is `LMR_STATIC_ADMIT_PCT_THOUSANDTHS`, which
+vetoes 509 of 21,780 cells (2.3% of cell space, 7.4% of traffic) -- and that
+threshold was chosen in section 6 (original) on solve-count deltas that 10.6
+showed were noise. **Cheap decisive test**: build with admission effectively
+disabled (`= 100000`) and run against current. If neutral, the table buys
+nothing in a shipping build. NOTE item 1 may make this moot by giving the table
+a real consumer -- do item 1 first.
+
+### 6. Triage six uncatalogued ECM positions into ../tests/ECM.md
+`ECM.348`, `ECM.018`, `ECM.084`, `ECM.516`, `ECM.640`, `ECM.774` all flipped in
+the 10.6 per-position diff and are not in the flaky/legit catalogue.
+
+### 7. Smaller / hygiene
+- **Strip the remaining one-off diagnostics**: `u64LMPPruned` and the
+ `JOINT_EXTRA_VERIFY` family are still wired under PERF_COUNTERS.
+- **Share the hang bucket** between the collection instrumentation and any
+ prune gate instead of computing `EstimateHangingMaterialCp` twice (10.12g).
+- **`pi->iEval` is still stale** on the common in-check qsearch path (Eval is
+ never called there). Only reader is `search.c:760`'s `iImprovement`, which
+ should never see a qsearch ply given the layering -- add an ASSERT recording
+ that invariant.
+- **`JOINT_EXTRA_TIER1/2/3_PCT_THOUSANDTHS`** are vestigial (TIER1 appears only
+ in comments, TIER3 only in the disabled LMP path).
+- **A sound fix for the `iEval >> iBeta` lost cutoff** in QSearch's
+ material-in-trouble branch is still open; both MAX and MIN forms were tried
+ and rejected (10.12e). Needs a bound the node has EARNED. Probably low value
+ -- the move loop usually recovers the cutoff by search.
+- **`ShouldLatePrune`'s depth ceiling is still backwards** (`<= 6*ONE_PLY`
+ permits pruning where the table says risk is highest). Moot while LMP is
+ disabled.
+
+### 8. Shelved deliberately
+- **`PRUNE_LIVE`/LMP**: ~7 Elo, hit its ceiling exactly, needs the table most.
+ Code stays behind its flag; do not ship.
+- **`DISABLE_LMR`**: diagnostic only. 10.12a says removing LMR costs ~40-50
+ Elo, so this is NOT a shipping candidate.
+- **Clamp removal / reducing into qsearch**: twice falsified (10.9a arithmetic;
+ Stockfish floors reduced depth at 1 ply while getting its leanness from
+ separate explicit pruning).
+- **LMR magnitude grading via the joint table**: 10.8 (table vs flat is a wash)
+ plus 10.11a (Crafty has ONE `reduce--` and is leaner anyway).
+- **`eNodeType` as an LMR signal**: 10.12d.
+- **SEE<0 check-extension removal**: measured dr = -0.004, ~3 Elo ceiling, and
+ independently reported to cause sacrifice-blindness.