diff options
| author | Scott Gasch <[email protected]> | 2026-10-04 08:17:52 -0700 |
|---|---|---|
| committer | Scott Gasch <[email protected]> | 2026-10-04 08:17:52 -0700 |
| commit | bbdf1bda8f410df6aa9be0a38e71b7db04982198 (patch) | |
| tree | e8315247f700ee94875cd6d2eb5dd81aef71395d /src/lmr_testing | |
| parent | 9b28080f8b3dca4588cb2aaaf748451f45e3c626 (diff) | |
Dynamic move-ordering/LMR session: honest FH tracking, decoupled admission/magnitude, bugfixes, tooling
Lands the 2026-09-28..10-03 move-ordering investigation (full history in
lmr_testing/*.md). Summary of what ships:
LMR/move-ordering redesign (searchsup.c, dynamic.c, data.c, chess.h):
- GetLMRReduction's admission decision (do we reduce at all) is now
decoupled from magnitude (how much): LMR_STATIC_ADMIT_THOUSANDTHS=500
gates admission via the 4D joint fail-high table (data.c) plus a
countermove-match veto; magnitude is a flat Crafty/Ethereal-shaped
log(depth)*log(moves) table (g_iLMRQuietReduction), which measured as
good or better than the old per-tier joint-table payout while being
structurally immune to the per-bucket miscalibration that sank the
tiered approach. g_FailHighs honesty fix (real 0/0 init, not a fake
prior) underlies the dynamic-danger signal.
- Retired NumLeftoverMovesToSelect's depth-indexed selection budget --
evidence showed the leftover pool has real findable signal a bailout
was discarding.
- ShouldLatePrune (overt late-move pruning) implemented but disabled at
its call site -- measured net-negative at the tested threshold; kept
for reference and as a base for a future hang-bucket-9-targeted retry.
- PRUNE_VERIFY, PRUNE_LIVE, DISABLE_LMR: opt-in diagnostic build flags
(GNUmakefile), off in the standard profile.
Bug fixes found and fixed while exercising this work:
- JOINT_EXTRA_TIER_COUNT 4->6 (chess.h): fixed an out-of-bounds counter
write reachable at remaining depth >= 14 (unreachable at sd10, routine
in real search/self-play).
- Node-type instrumentation (search.c, root.c, searchsup.c): fixed four
compounding defects that produced impossible percentages (e.g. 106.2%
of PV nodes) -- wrong denominator, qsearch misclassified as main
search, untyped early-exit nodes, and fail-low tests using the
in-place-raised alpha instead of the initial one.
- FH-rate unit unification (search.c, searchsup.c, dynamic.c, chess.h):
GetMoveFailHigh* returned per-mille while g_uJointFHThousandths is
thousandths-of-a-percent, two different units sharing a "_THOUSANDTHS"
suffix. Unified to one scale, renamed to "_PCT_THOUSANDTHS".
RateBucket's band edges fixed to match.
- QSearch option B (search.c): waives the opponent-stand-pat requirement
at NODE_PV only (sound there since the exact score propagates; not
sound at null-window nodes). iExtensionAmount unit fix (was 0/2 into a
ONE_PLY-scaled field). pi->iEval now published when Eval() runs.
- eval_tune/match_play.py local_ebf: was reporting 0.000 in every match
since 2026-09-18 (regex only matched the trailing-stats parser, not
the live move-scan loop; engines were also spawned with "nopost" so
root.c never emitted the lines at all). True per-ply EBF is ~2.0, not
the ~3.9 every prior match report implied.
- eval_tune/test_vs_head.sh: positional-arg shift ordering bug caused
every no-extra-args run to silently rebuild ./typhoon with a
non-standard profile and, separately, could feed a flag's value in as
the candidate binary path.
- PERF_COUNTERS cleanup (eval.c, search.c): moved 3 leaked rdtsc sites to
the existing EVAL_TIME guard, deleted two concluded one-off diagnostics
(u64HeadStyle*, u64FHJointHist/RateBucket/AttemptsBucket) and their
report plumbing, and re-gated a per-move hang-material estimate behind
FH_TABLE_COLLECTION instead of running it unconditionally.
Infrastructure:
- match_play.py time-odds support (--odds/--st-head/--st-candidate),
validated against the engine seat rather than color.
- Wilson-upper-bound re-bake of the joint table (data.c): cells store a
confidence-aware bound instead of a raw point estimate, so a consumer
can threshold on the value alone; monotone, so every consumer became
strictly more conservative. Verified behavior-neutral for LMR.
Known-open items (not resolved here, see lmr_testing/GET_LMR_REDUCTION.md
section 10 and 2026-10-03_UNCOMMITTED_TRIAGE.md): the self-play gate on
this candidate is underpowered (+5 Elo, 95% CI roughly [-6, +17]); the
curated-suite solve-count deltas used earlier in the investigation turned
out to be noise-dominated cancellation when diffed per position, not the
clean signal they first appeared to be.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Diffstat (limited to 'src/lmr_testing')
| -rw-r--r-- | src/lmr_testing/2026-09-28_four_axis_joint_table_plan.md | 238 | ||||
| -rw-r--r-- | src/lmr_testing/2026-10-03_LMR_REDESIGN.md | 172 | ||||
| -rw-r--r-- | src/lmr_testing/2026-10-03_UNCOMMITTED_TRIAGE.md | 137 | ||||
| -rw-r--r-- | src/lmr_testing/GET_LMR_REDUCTION.md | 1971 | ||||
| -rw-r--r-- | src/lmr_testing/SPEC.md | 1543 |
5 files changed, 4061 insertions, 0 deletions
diff --git a/src/lmr_testing/2026-09-28_four_axis_joint_table_plan.md b/src/lmr_testing/2026-09-28_four_axis_joint_table_plan.md new file mode 100644 index 0000000..4a67561 --- /dev/null +++ b/src/lmr_testing/2026-09-28_four_axis_joint_table_plan.md @@ -0,0 +1,238 @@ +# Four-axis joint FH% table -- next steps, 2026-09-28 + +Checkpoint: `git stash` (`stash@{0}`, message starting "four-axis (dumb/gap/ +late/depth) marginal calibration instrumentation...") holds the working +instrumentation this plan builds on: `fPruneReduceEligible` (search.c), the +`bbMinorAttacks`/`bbRookAttacks` snapshot fix, and four independently-wired +marginal calibration axes (dumb, gap, late, depth), each confirmed clean at +real volume on `tests/ecm_ringers.ep_`, `sd 14`, single position family. Pop +that stash to resume from exactly this point. + +## Where each axis landed (marginal, single-position-family sample) + +All four measured against the *eligible* leftover baseline (0.31% FH, +matching HEAD's own EFP/LMR eligibility checklist plus losing-capture +admission via cached SEE sign, plus `iAlpha < +NMATE`): + +- **dumb** (material hung, `EstimateHangingMaterialCp`): every bucket below + baseline except 1-100cp (~3x baseline, traced to SEE-scored losing + captures + underpromotions -- a real SEE limitation, not a bug; plan is to + exclude this bucket from pruning/reduction eligibility, not fix SEE). +- **gap** (`iAlpha - iEval`): clean, ~4 orders of magnitude, 3.80% (gap<=0) + down to ~0.00% (900cp+), strictly decreasing. +- **late** (this move's ordinal position, `uLegalMoves`): clean, ~23x decline + move 2 (1.61%) to move 27 (0.07%), small uptick in the deep tail (raw-move- + number-vs-fraction-of-list issue, already known from earlier work). +- **depth** (`uDepth / ONE_PLY` at this node, before this move's own + extension/reduction): clean, ~10-15x decline depth 0 (0.44%) to depth 9-13 + (~0.00-0.06%). Hypothesis: shallow subtrees are noisier -- the opponent + hasn't had enough plies to find its real defense, so a "leftover" move + spuriously crosses beta more often near the horizon than deep, where the + subtree has room to converge to the move's true (usually bad) value. This + is in tension with the traditional "EFP is safest near the horizon" + argument, which is about *cost per mistake* (small subtree), not + *frequency of mistakes* -- our data says frequency is actually *highest* + right where the traditional argument calls it safest. The tradeoff is + real, not resolved by this data alone. + +## Why a joint table now, not a cheap combined estimate + +Discussed and rejected the "naive-Bayes combine the four marginals, skip +the joint collection" shortcut as the first move: cheaper, but doesn't +address the actual open question, which is what to do about the fact that +*every* prior attempt in this project to turn a calibrated signal into a +live pruning/reduction win has landed at "roughly even," never a clear win, +regardless of how correctness-verified the signal is. That pattern won't be +fixed by better precision; if it's real, it's more likely explained by +information the search loses when a move gets skipped (history/TT/killer +contributions later parts of the tree would have used) or by distribution +shift (the table is trained under full search, deployed under a search that +now prunes/reduces differently based on it). Building the joint table is +still worth doing -- Scott wants the real thing, not an approximation, and +a short test run is cheap -- but the *first live test* of it must be via +**reduction, not pruning** (see "Consumption, phase 2" below), specifically +because reduction is self-correcting (existing re-search-on-unexpected- +raise machinery) in a way pruning structurally cannot be. If reduction +*also* lands at "even," that is real evidence the ceiling is structural, +not a signal-quality problem, and no further table refinement will fix it. + +## Table shape + +Same four axes as the marginal work, but joint, cache-shaped, and +coarsened where the marginal curves already went flat: + +| axis | marginal buckets | shipped buckets | boundary notes | +|-------|-----------------:|----------------:|-----------------| +| depth | 25 | **12** | exact 0-9 (all real signal/volume lives here), then 10-13 grouped, 14+ overflow | +| dumb | 11 | **11** | unchanged -- already tight, semantically meaningful (0=not dumb, 1-9=100cp bins, 10=901cp+) | +| gap | 11 | **11** | unchanged -- already tight (0=gap<=0, 1-9=100cp bins, 10=901cp+) | +| late | 33 | **15** | raw move 1 is never produced (`uMoveNum` is 1-based) and raw move 2 is the real floor (the `fPruneReduceEligible`/collection gate requires `uLegalMoves > 1`) -- exact moves 2-13 (biggest swings live here), then 14-21 / 22-31 grouped, 32+ overflow. See 2026-09-30 correction note below: the original bucketing shifted by only 1 (assuming raw 1 was the floor), leaving shipped bucket 0 permanently empty. | + +Total shipped cells: 12 x 11 x 11 x 15 = **21,780**. + +**Axis order in the C array: depth outermost.** Depth is the one axis fixed +for an entire node's move loop; dumb/gap/late all vary move-to-move within +that same loop. Declaring the array as `[SEARCH_DEPTH_BUCKETS][DUMB][GAP] +[LATE]` means every lookup during one node's whole move loop stays inside +one depth-slice: 11 x 11 x 15 = 1,815 cells. At 2 bytes/cell (see below) +that slice is **~3.5KB** -- small enough to stay resident in L1 for the +entire node, not just L2 for the whole table. First touch per node pays one +cache-miss; everything else in that node is hot. The internal order of +dumb/gap/late doesn't matter much for locality (all three vary per-move +similarly) -- pick whatever's most readable; `[depth][dumb][gap][late]` +matches the order we normally discuss them in. + +## Cell encoding: thousandths of a percent, not hundredths + +`USHORT`, storing `FH% * 1000` (so 1 == 0.001%, 1000 == 1.000%). Max +representable ~65.535%, far more than needed (every real cell we've seen is +under ~6%) -- the range headroom isn't the point, the *low-end* precision +is: real cells run from ~0.00% up through maybe 2-4%, and confirmed-zero +tests and dumb/gap tail buckets both live down in the 0.00-0.10% range +where hundredths-of-a-percent (the old table's unit) was already too coarse +to distinguish "genuinely near-zero" from "true zero." Thousandths gives +10x finer resolution exactly where decisions get made. No special encoding +needed for the high end -- Scott's "anything over ~0.39% is useless to us +anyway" observation is true (pruning/reduction decisions never care whether +a bad cell is 2% or 6%), but it doesn't require clamping or a different +scale; a linear thousandths encoding already wastes no precision there, +because we simply never read the top of the range for a decision. Leave it +be. + +Sentinel: `0xFFFF` (`FH_TABLE_SENTINEL`-equivalent) means "below this +run's sample floor, don't trust this cell yet" -- see below. + +## Sample floor, this run only + +This is a 90-minute preliminary collection, not the eventual continuously- +updated table. For *this* pass, bake with a `MIN_SAMPLES` floor (propose +500, tune after seeing real per-cell counts) -- any cell below the floor +gets the sentinel instead of a real percentage. A consuming (reduction) +lookup that hits a sentinel cell should fall back to the coarser gap-only +or dumb-only marginal estimate rather than treat "insufficient data" as +"confirmed safe." Once this table is run for real (a week, or continuously +in the background, matching the original project's stated intent), the +floor can loosen or come off entirely -- Scott's point that "we don't need +to care whether 0.0% came from 10k or 10B tries" is correct *for a table +that keeps accumulating*, not for a single 90-minute snapshot. + +## How to declare and populate it + +1. **New raw collection counters** (chess.h `COUNTERS.tree`, mirrored into + `g_Options` the same way the four marginal axes already are): a single + joint 4D array at *marginal* (uncoarsened) resolution -- + `UINT64 u64JointTries[SEARCH_DEPTH_BUCKETS_RAW(25)][HANG_CP_BUCKETS(11)] + [GAP_CP_BUCKETS(11)][MOVE_NUM_BUCKETS_RAW(33)]` and the matching + `u64JointFH` -- 25*11*11*33 = 99,825 cells * 16 bytes (two UINT64) = + ~1.6MB per counter pair. This is collection-only, lives in + `SEARCHER_THREAD_CONTEXT` (already ~75MB), and is never shipped -- size + doesn't matter here, only in the final baked table. Collecting at full + marginal resolution (not pre-coarsened) preserves the option to try a + different final bucket shape later without re-collecting, matching how + the gap-bucket rework reused the same joint-run pickle by re-bucketing + at bake time instead of re-running collection. + +2. **Wiring in `search.c`**: reuse the exact same + `fPruneReduceEligible`-gated block the four marginal axes already sit + in. Compute all four bucket indices (`_HangCpBucket`, `_GapCpBucket`, + `_MoveNumBucket`, `_DepthBucket` -- already written, unchanged) and + increment the single joint cell `u64JointTries[depthBucket][dumbBucket] + [gapBucket][lateBucket]` once per eligible move, `u64JointFH[...]` once + per eligible fail-high, in addition to (not instead of) the four + existing marginal increments -- keep the marginals live during this run + as a cheap cross-check that the joint reduces to what we already + understand. + +3. **Reassembly format**: a new `FOURAXISRAW depth dumb gap late fh tries` + line, printed the same checkpoint-friendly way as the old + `FHGAPRAW`/`FHJOINTRAW` lines were (full cumulative dump every N + positions via the suite-report path, last-checkpoint-only on + reassembly, never summed across checkpoints). + +4. **Shard/collect**: 16 shards, `--cpus 1` each (matches every prior + collection run in this project -- split.c's parallel helpers don't feed + these counters), `sd 16` or deeper (need real depth to populate the + depth 8-13 buckets at all -- shallower runs structurally can't reach + them), 90 minutes wall-clock per shard via `timeout`, mixed real-game + (`cross_sectional_200k.ep_`) + modest tactical (`ecm.ep_`) position pool, + same ~90/10 blend as prior runs. Output to `/usr/local/tmp/typhoon/` + (persistent, not `/tmp` -- see the disk-full incident earlier this + project). **Do the same smoke-test discipline as every prior run**: 3-5 + minute all-16-shards test first, confirm 0% pruned/no crashes/sane disk + growth, *then* commit to the full 90 minutes. + +5. **Bake**: python script reads each shard's last checkpoint block, + aggregates `FOURAXISRAW` lines into a dict keyed by the *raw* (25 x 11 x + 11 x 33) indices, re-buckets depth and late down to the shipped (12, + 15) shape (dumb and gap pass through unchanged), applies the sample + floor, converts surviving cells to thousandths-of-a-percent, emits a C + initializer for `g_uJointFHThousandths[12][11][11][15]` matching the + depth-outermost declaration above, and splices it into `data.c` the same + brace-depth-tracked way prior bakes did (never a fixed-line-count + assumption -- table bodies are huge and shift size on every rebuild). + +## Consumption, phase 2 (after the table exists -- not this pass) + +Replace the current flat one-ply LMR with a graduated additive term at the +same call site, using the *already-computed* eligibility gate and bucket +indices (no new per-move cost beyond the table lookup itself, since +`fPruneReduceEligible` and all four bucket functions already run on every +candidate move once this lands): + +``` +uPredictedFH = LookupJointTable(depthBucket, dumbBucket, gapBucket, lateBucket); +extra_reduction_plies = f(-ln(uPredictedFH)) // more permissive than pruning + // ever was -- see the doc's + // own "on reduction vs + // pruning" section for why + // reduction can tolerate a + // much less conservative + // threshold (self- + // correcting via re-search, + // unlike a prune) +``` + +Exclude the dumb axis's 1-100cp bucket from ever triggering *extra* +reduction beyond whatever the other three axes independently justify -- +that bucket is understood to be SEE's blind spot (structural undervaluing +of certain sacrifices/underpromotions), not a real "this move is fine" +signal, and treating it as "safe to reduce more" would be trusting exactly +the population we already know fools the estimate. + +First real test: self-play against `head_reference` (current flat one-ply +LMR), same protocol as every prior match in this project (LOWER95 >= 0.5 +gate, not raw score; fixed `sd`/`sn` matched to what's actually being +tested). Watch specifically for whether this breaks the "always even" +pattern -- if it doesn't, that's the signal to stop iterating on this +approach rather than build a bigger/more precise table. + +## Correction, 2026-09-30: late-bucket-0 was permanently dead, now fixed + +Found while reviewing coverage on a fresh 16-shard collection run: +`_ShipLateBucket`/`ship_late_bucket`'s `-1` shift (search.c, dashboard.py) +assumed the real floor for raw `late` was 1 (`uMoveNum` is 1-based). It +isn't -- the collection/consultation gate itself (`fPruneReduceEligible`, +`search.c`, requires `uLegalMoves > 1`) means raw `late` is never less +than 2. So shipped bucket 0 (meant for raw==1) could never receive a +sample: it read `0xFFFF` (sentinel) in every baked table to date, not +because of insufficient collection time but because the bucketing wasted +a slot on an unreachable raw value. This alone accounted for 1,452 of the +21,780 shipped cells (a full `depth x dumb x gap` slice) permanently +reading "no data," inflating the apparent coverage gap by ~6.7 percentage +points in every prior coverage report. + +**Fix**: shift by 2 instead of 1, anchored on the real floor (raw >= 2): +raw 2-13 -> shipped 0-11 (singleton), 14-21 -> 12, 22-31 -> 13, 32+ -> 14. +Same bucket widths as before, just correctly anchored. Applied in lockstep +to `search.c`'s `_ShipLateBucket` (live lookup path), `dashboard.py`'s +`ship_late_bucket` (coverage reporting), and the newly-written +`joint_table_collection/bake_joint.py` (which imports `dashboard.py`'s +version rather than redefining it, so it can't drift out of sync). The +table in `data.c` was rebaked from the 2026-09-30 collection run with the +corrected bucketing; real coverage came out to 18,775/21,780 (86.2%) +populated, vs. 17,444/21,780 (80.1%) that the old (wrong) bucketing would +have reported on the same data. + +No prior collection data needed to be discarded for this -- it's purely a +bucketing/reporting bug, not a population/gate problem (a separate concern +raised and ruled out the same day; see SPEC.md's corresponding note). diff --git a/src/lmr_testing/2026-10-03_LMR_REDESIGN.md b/src/lmr_testing/2026-10-03_LMR_REDESIGN.md new file mode 100644 index 0000000..68ed068 --- /dev/null +++ b/src/lmr_testing/2026-10-03_LMR_REDESIGN.md @@ -0,0 +1,172 @@ +# Re-imagining the baseline LMR, 2026-10-03 + +Question: the dumb flat `-1 ply on every leftover move` is load-bearing to +~40-50 Elo and +1.17 ply (`GET_LMR_REDUCTION.md` 10.12a). Five attempts to +improve on it have all measured as noise. If we rebuilt the baseline from +scratch with what we now know, what would it be? + +Not looking for a miracle. Looking for ~5 Elo and some sound depth. + +--- + +## The reframing that matters + +**Every one of the five attempts pushed in the same direction: trade accuracy +for depth.** Graded magnitude, wider admission, bigger cuts, LMP, deep +pruning -- all of them reduce or remove more search in exchange for a higher +root iteration number. + +We now have that direction's exchange rate: **~8 Elo per marginal ply** +(10.12b). It is a bad rate, and it is bad for a structural reason -- a +reduction is a *transfer*, not new resource. It lowers the depth of the +refutation lines (where the move choice is actually decided) to raise the +depth of the PV. Time-bought plies, which raise the whole distribution, are +worth ~95 Elo/ply. + +**Nobody has ever tried the other direction: trade depth for accuracy.** + +That is the gap. If accuracy converts better than depth -- and there is no +reason to assume it converts at 8 Elo/ply, because it is not buying plies at +all -- then the remaining few Elo are on that side of the ledger, not this one. + +--- + +## What the five attempts actually taught us + +| lesson | evidence | +|---|---| +| **Coverage beats magnitude.** Breadth of admission moves total tree size far more than depth-of-cut per move. | section 3 (HEAD-style reduces 76% of decisions and beat the "honest" 62-66% design on node count); section 6 | +| **Magnitude is capped by the depth DISTRIBUTION, not the formula.** 76% of the LMR population sits at remaining depth buckets 2-3, where `iMaxExtra = depth - 3` permits zero extra plies -- and where the `log(d)*log(m)` curve prescribes ~1 ply anyway, i.e. exactly flat. The curve's whole contribution is **mean 0.164 extra plies ~= +0.05 root ply**. | 10.8, 10.12c | +| **Learned per-situation grading loses to a flat formula.** Table-driven tiers vs flat `log(d)*log(m)`: 0.224 vs 0.244 mean extra plies -- a wash, and the tier version broke with a re-search-rate inversion when widened. | section 8, 10.8 | +| **Per-move dynamic signals cannot identify the dangerous moves.** 11 of 23 verified low-band prune misses sit at own-FH-rate **exactly 0** -- the per-move history agreed with the static table and both were wrong. A veto catching the rest would refuse ~49,000 sound prunes per miss prevented. | 10.12h | +| **`eNodeType` is uninformative here.** Full-width ALL nodes fail HIGH 69-71% of the time. The structural label is close to meaningless. | 10.12f, 10.12d | +| **Move ordering is the real sophistication, and it is already good.** FMBC 90-95%; `r ~= 2.17`. By the time a move is in the leftover pool the generator has already triaged it -- the table is re-triaging an already-triaged population. | 10.11a, 10.12c | +| **Only fail-HIGHS are verified.** The re-search path fires on `iScore >= iBeta`. A reduced search that fails low is accepted silently, forever. | search.c move loop | + +That last row is the one nobody has acted on. + +--- + +## The redesign + +### A. Keep (this is where the 40 Elo lives) +- **Broad admission.** ~93% of eligible decisions get reduced. Five attempts + to be cleverer about *which* moves all failed. +- **One magnitude formula, `log(depth)*log(moves)`.** Already present + (`g_iLMRQuietReduction`, dynamic.c, built by `InitLMRTable`). Keep it as the + single source of magnitude. No table, no tiers. +- **The one-full-width-ply floor.** `JOINT_EXTRA_MIN_REMAINING_DEPTH = ONE_PLY` + guarantees a reduced move still gets a real node before qsearch. Stockfish + keeps the identical invariant (`search.cpp:1370`, `std::max(1, ...)`) and + gets its leanness from separate explicit pruning. Twice falsified as a + target (10.9a). +- **Killer-adjacency vetoes** (this ply's killers, ply-3's). Cheap, exact, + principled -- a specific move at a specific ply, not an aggregate rate. + +### B. ADD: verify fail-lows near the margin <-- the actual proposal +The only unverified error direction, attacked with the search's own evidence +rather than a static prior. + +Today, in `search.c`'s move loop: +```c +iScore = -Search(ctx, -iAlpha - 1, -iAlpha, uNextDepth); /* reduced */ +if ((iExtend < 0) && (iScore >= iBeta)) { ...re-search full depth... } +``` +At a null window `iBeta == iAlpha + 1`, so that branch means "the reduced +search beat alpha". A reduced search that returns `iScore <= iAlpha` is taken +at face value. Proposed addition: + +```c +else if ((iExtend < 0) && (iScore > iAlpha - LMR_NEAR_MISS_MARGIN)) +{ + /* Reduced search failed low, but only just. The reduced search is the + weakest evidence in the tree and this is the one direction the + existing machinery never checks. Re-search at full depth. */ + uNextDepth -= iExtend; + pi->iExtensionAmount = 0; /* defuse the null-move bailout */ + iScore = -Search(ctx, -iAlpha - 1, -iAlpha, uNextDepth); +} +``` + +Why this is different from everything tried: +- **Sound by construction.** It only ever ADDS search. It cannot introduce a + wrong answer; the worst case is wasted nodes. Every previous attempt risked + correctness to buy depth. +- **It trades depth for accuracy** -- the untried direction. It will cost + nominal plies (marginal plies are worth ~8 Elo, so that is a cheap thing to + spend) in exchange for catching refutations the reduction hid. +- **Self-measuring, and falsifiable in one suite run.** Instrument + `near_miss_researches` and `near_miss_flips` (how many came back + `>= iBeta`). If the flip rate is ~0, the idea is dead immediately and + cost nothing. If it is a few percent, those are refutations currently being + missed, and the margin can be swept. +- **It is targeted.** A move returning `alpha - 5` nearly made it; one + returning `alpha - 300` did not. Score proximity is real evidence, unlike + the cell averages the 4D table offers. + +Sweep `LMR_NEAR_MISS_MARGIN` over 0 / 10 / 25 / 50 / 100 cp. Measure flip rate +and node cost per margin before any self-play. + +### C. DELETE (each is unmeasured or measured-as-noise, and each costs real +### work per decision in a SHIPPING build) +- **The 4D table admission gate.** `LMR_STATIC_ADMIT_PCT_THOUSANDTHS` vetoes + 509 of 21,780 cells -- 2.3% of cell space, 7.4% of traffic -- and the + threshold was chosen in section 6 on solve-count deltas 10.6 showed were + noise. Removing it also removes `EstimateHangingMaterialCp` plus four bucket + functions plus a table lookup *per eligible move*, which is the same class of + per-move cost the PERF_COUNTERS cleanup just went after. (Hold if EFP ends up + needing the table -- see 10.13 item 1.) +- **`fDynamicDanger`'s extras gate.** Fires on 27.4% of decisions and only + zeroes the extra plies -- which 85% of reduced moves never get anyway. The + section 1 "honesty fix" that motivated the whole 2026-10-02 session is + therefore nearly inert in shipped behaviour. +- **The bare countermove-match veto.** Unconditional, unmeasured, and + generate.c's own use of a countermove match is evidence-GATED precisely + because a bare match was shown not to carry signal. +- **`JOINT_EXTRA_TIER1/2/3_PCT_THOUSANDTHS`** -- vestigial (TIER1 in comments + only, TIER3 only in the disabled LMP path). + +### D. CONSIDER: widen coverage past the leftover pool +`fPruneReduceEligible` requires `fIsLeftoverMove` (sub-`GOOD_MOVE`). Crafty and +Stockfish reduce **all** quiet moves past a move-count threshold, including ones +the generator scored into a higher tier. Coverage is the single lever with +replicated evidence behind it (section 3, section 6, 10.2), and this is the +largest untapped slice. + +But: it buys marginal plies, so expect ~8 Elo/ply and bound expectations +accordingly. Do it *after* B, because B makes additional coverage safer -- the +verification catches the extra mistakes wider coverage would make. + +### E. DO NOT (all falsified; reasons in 10.13 item 8) +Per-situation magnitude grading; `eNodeType` modulation; clamp removal / +reducing into qsearch; dynamic per-move vetoes; chasing bigger cuts. + +--- + +## What this adds up to + +A baseline LMR that is **simpler** than today's (no 4D table, no dynamic +extras gate, no countermove veto, one magnitude formula) and has **one new +mechanism** that is sound by construction and pushes in the only untried +direction. + +Honest expectation: the deletions are roughly Elo-neutral and buy a little +nps; the near-miss verification is the only part with real upside, and its +upside is unknown rather than bounded at 8 Elo/ply, because it is not buying +plies. If its flip rate comes back near zero it dies in one suite run, which +is the cheapest possible failure mode. + +That is the whole pitch: stop buying depth at 8 Elo/ply, spend some of it back +on the one error the search never checks. + +## Order of work +1. Instrument the near-miss flip rate ONLY (no behaviour change): count how + many reduced fail-lows land within each margin band, and -- behind a + verify flag -- how many of those flip to `>= iBeta` at full depth. One + suite run answers whether B is worth building. +2. If the flip rate is material, implement B and sweep the margin on node + count + `sn` suites, then self-play with `elo0=0 elo1=15`. +3. Do the deletions in C as a separate commit, measured on nps (idle box, + round-robin arms) rather than on Elo -- they are a complexity/speed play, + not a strength play. +4. D last, and only if B worked. diff --git a/src/lmr_testing/2026-10-03_UNCOMMITTED_TRIAGE.md b/src/lmr_testing/2026-10-03_UNCOMMITTED_TRIAGE.md new file mode 100644 index 0000000..f379e96 --- /dev/null +++ b/src/lmr_testing/2026-10-03_UNCOMMITTED_TRIAGE.md @@ -0,0 +1,137 @@ +# Uncommitted-change triage, 2026-10-03 EOD + +`git status` shows 15 modified files, **5,313 insertions / 194 deletions**. +That spans several sessions, not just today (date markers in the added lines: +33 from 2026-10-03, 24 from 10-02, 21 from 09-28, plus 09-29/09-30/10-01). +This file sorts it into KEEP / DECIDE / DISCARD so a commit can be assembled +without re-deriving the reasoning. + +Nothing here is committed. Evidence for every claim is in +`GET_LMR_REDUCTION.md` section 10 and `SPEC.md`. + +--- + +## KEEP -- unambiguous, no open questions + +### Bug fixes (all found by exercising code, all verified) + +| what | file | why it is solid | +|---|---|---| +| `JOINT_EXTRA_TIER_COUNT` 4 -> 6 | chess.h | `JOINT_EXTRA_MAX_PLIES` is 5, so `*puExtraTier` could be 4 or 5 and both consumers indexed 4-element arrays out of bounds -- into `u64JointExtraResearch[0]` and into `u64LMPPruned`. Unreachable at sd10 (needs remaining depth >= 14), live in any real search. Counters-only, but it corrupted the exact calibration diagnostic candidates get judged on. 10.4 | +| Node-type surprise line: 4 defects | search.c, root.c, searchsup.c | (1) denominator was total nodes not per-type; (2) qsearch included, ~84% of the tree, so a stand-pat cutoff read as "ALL failed high"; (3) `CommonSearchInit` early exits reached the counters un-typed and scored against a stale `eNodeType` -- produced an impossible "106.2% of PV nodes"; (4) fail-low test used the IN-PLACE-RAISED `iAlpha`, so every successful alpha-raise counted as a fail-low (PV row read 99.1%). Now uses `iInitialAlpha`. 10.12f | +| FH-rate unit unification | search.c, searchsup.c, dynamic.c, chess.h | `GetMoveFailHigh*` returned per-mille while `g_uJointFHThousandths` is thousandths-of-a-PERCENT -- 100x apart, same `_THOUSANDTHS` suffix. `ShouldLatePrune` held `LMP_DYNAMIC_MAX_THOUSANDTHS (1)` meaning 0.1% four lines from `LMP_STATIC_MAX_THOUSANDTHS (1)` meaning 0.001%. Every caller was individually correct; the naming made that luck. Converted the whole dynamic family to the table's unit, renamed to `..._PCT_THOUSANDTHS`. 10.8 | +| `RateBucket` band edges x100 | searchsup.c | Collateral of the unit change: edges were per-mille, so every rate >= 0.1% collapsed into the top bucket and five middle bands read zero. Caught because the histogram printed only its first and last columns. | +| `local_ebf` fix | eval_tune/match_play.py | Reported 0.000 in **every match since it was added (2026-09-18)**. Two causes: `_ITER_RE` was applied only in the trailing stats parser but the PV lines arrive during the search in the move-scan loop; and `Engine.__init__` sends `nopost`, so `root.c:248` never emitted them at all. Now parsed in both loops plus an opt-in `--post`. This is the metric that proved the published `ebf` (cumulative `nodes**(1/depth)`) has been wrong in every match report -- true per-ply ratio is **~2.0, not ~3.9**. 10.10c | +| `test_vs_head.sh` shift ordering | eval_tune/test_vs_head.sh | Positionals were shifted BEFORE `$2` was read, so (a) every no-extra-args run took the rebuild branch and clobbered `./typhoon` with a non-standard profile (`gmake release MP=1 PERF_COUNTERS=1`, missing GENETIC/SIXTYFOUR), and (b) the first pass-through flag's VALUE became the candidate binary path -- producing a baffling `PermissionError` from Popen trying to execute a directory. Also `${VAR:-default}` -> `${VAR-default}` so an explicitly empty `SPRT_ARGS=""` is honored. | + +### PERF_COUNTERS cleanup (10.12g) +The standard profile was carrying investigation debris that did real WORK, not +counter increments. + +- **3 `rdtsc` sites moved to `EVAL_TIME`** (eval.c). Bracketed every + king-safety hash probe. An `EVAL_TIME` flag already existed for exactly this + and already guarded the other **32** rdtsc sites in the file -- these were + the only leaks. Report side in root.c was already EVAL_TIME-guarded. +- **2 one-off diagnostics deleted outright**: `u64HeadStyle*` (called + `GetMoveFailHighRawCounts`) and `u64FHJointHist` (called + `GetMoveFailHighPctThousandths`). Both had comments saying to remove them + once their investigations concluded; both concluded, and both findings are + recorded (10.2's admission rates; section 4's bimodal dynamic rate). Removed + counters, `g_Options` mirrors, suite accumulation AND both report blocks -- + a diagnostic printing zeros forever is worse than none. Binary -7,760 bytes. +- **4D collection block re-gated to `FH_TABLE_COLLECTION`** (search.c). It + calls `EstimateHangingMaterialCp` per eligible move; CLAUDE.md already keeps + that flag out of the standard profile *because it costs nps*. Verified by + guard inspection: `search.c:1637` is now `PERF_COUNTERS > FH_TABLE_COLLECTION`. + It was also computing the hang bucket TWICE per move in any build with a + prune gate. + +**nps payoff NOT MEASURED** -- two attempts invalid (32-engine match on the +box; one produced `perf_clean` faster than `noperf`, which is impossible). +Re-measure idle, round-robin arms. The changes are justified on inspection +regardless. + +### Infrastructure +- **`match_play.py` time-odds support** (`--odds`, `--st-head`, + `--st-candidate`). Strict validation because a degraded odds match still + scores ~0.500 and looks healthy. Odds follow the ENGINE, not the seat -- + verified live, since the harness swaps colours every other game, so keying on + White would silently balance the match. `TC=` stamped into the result line. + This is what produced the time-odds calibration. +- **`PRUNE_VERIFY`** (GNUmakefile, search.c, chess.h, root.c, script.c). + Audit-only: evaluates the prune gate, records banded by predicted FH%, then + declines to prune and watches the move's own ordinary search. No shadow + `Search()` call, so no null-move-bailout exposure. Banding by the decision + variable answers "is `prune if pred <= K` sound" for every K in one run. This + is the instrument any future pruning work needs. 10.12h +- **`DISABLE_LMR`** (GNUmakefile, searchsup.c). Diagnostic flag that produced + the control experiment. Keep the flag, never ship the configuration. + +### QSearch changes (10.12e) -- smoke-tested with asserts +- **Option B**: `fIncludeChecks` waives the opponent-stand-pat requirement at + `NODE_PV`, keeping the material requirement. The suppression is sound for + null-window nodes (a fail-low there IS the parent's cutoff, so precision buys + nothing) but not for wide-window nodes, where the exact score propagates and + mate-vs-queen-loss is a different PV -- the forced-mate case the machinery + exists for. Reads `eNodeType`, never writes it. +- **`iExtensionAmount` unit fix**: was written as 0 or 2 (qsearch check PLIES) + into a ONE_PLY-scaled field -- a latent 32x error, benign only because both + readers test sign. +- **`pi->iEval` published** when the lazy `Eval()` actually runs. + +### Wilson re-bake (data.c, joint_table_collection/splice_joint.py) -- 10.11e +Cells store the **Wilson upper bound** instead of the point estimate, so a +cell's value encodes its own sample confidence and a consumer can threshold on +the value ALONE. Under point estimates **8,248 of 21,780 cells read exactly 0**, +mostly on little evidence, so any "prune where the table says 0" rule was far +broader than the data supported; under Wilson, 136 do. Monotone (no cell's value +is ever lowered), so every consumer became strictly more conservative. +Verified behaviour-neutral for LMR: admission 92.79% vs 92.88% before, tier +split and ringers solve count unchanged. `LMR_STATIC_ADMIT_PCT_THOUSANDTHS` +stays at 500 -- it was briefly raised to 1000 on a CELL-count argument, which +was wrong because cell counts are not traffic-weighted. + +--- + +## DECIDE -- real, but with an open question + +| what | open question | +|---|---| +| **The 4D table itself** (data.c, 1,777 lines) + `joint_table_collection/` | Its only live shipping consumer is `LMR_STATIC_ADMIT_PCT_THOUSANDTHS`, which vetoes 2.3% of cell space / 7.4% of traffic -- and that threshold was chosen on solve-count deltas 10.6 showed were noise. Decisive cheap test: disable admission (`= 100000`) and run vs current; if neutral the table buys nothing in a shipping build. **But item 1 of 10.13 (give EFP the dumb axis) may give it a real consumer -- do that first.** | +| **Graded LMR** (`LMR_STATIC_ADMIT`, flat-table magnitude, dynamic/countermove vetoes) | Worth ~8 Elo/ply marginal, i.e. a few Elo total, never cleanly self-play tested against flat -1 (10.12c). The `log(d)*log(m)` magnitude contributes mean **0.164 extra plies** because at remaining depth <= 2 the curve prescribes ~1 ply, which IS flat -1. Keep or simplify per the redesign in `2026-10-03_LMR_REDESIGN.md`. | +| **`PRUNE_LIVE`** | Measured +0.88 ply, +7 Elo -- exactly its corrected ceiling. Keep the flag for measurement; do not ship the configuration. | +| **`split.c` zero-snapshot passing** | `HelpSearch` passes all-zero attack snapshots to `GetLMRReduction`, so `EstimateHangingMaterialCp` degrades there. Documented and deliberate (every comparison protocol runs `--cpus 1`), but it means MP search reduces differently from single-threaded. Fine as long as that stays true. | + +--- + +## DISCARD -- scratch that leaked into the tree + +Per CLAUDE.md's scratch policy these belong under `/tmp/typhoon/`, not `src/`: + +- `cdefault_games.pgn`, `chkdefault_games.pgn`, `codds_games.pgn`, + `dflt_games.pgn` -- from this session's `test_vs_head.sh` argument-construction + tests (GAMES=0/2 runs). **Delete.** +- `kiwi_log.txt`, `eval_tune/kiwi_log.txt`, `eval_tune/eval_comp.txt`, + `blunders.pgn`, `players.dat`, `.xboardrc` -- untracked clutter from earlier + sessions; not mine to judge, but none look like deliberate artifacts. + +Deliberately-kept untracked directories (do NOT delete): `head_reference/`, +`joint_table_collection/`, `nullmove_calib/`, `eval_tune/position_cache/`, +`eval_tune/opening_cache/`. + +--- + +## Suggested commit split + +1. **Bug fixes** -- OOB counter, node-type x4, unit unification, RateBucket, + `local_ebf`, `test_vs_head.sh` shift. Independently correct, no behaviour + bets. Run `./precommit_check.sh` first (search.c behaviour did change via + the QSearch work). +2. **PERF_COUNTERS cleanup** -- after an idle-box nps measurement, so the + commit message can state what it bought. +3. **Infrastructure** -- time-odds, `PRUNE_VERIFY`, `DISABLE_LMR`. +4. **QSearch changes** -- option B et al. Smoke-tested; ideally also a short + self-play run, since option B is a real behaviour change never measured in + isolation. +5. **Table + graded LMR + PRUNE_LIVE** -- hold until 10.13 items 1 and 5 are + resolved. diff --git a/src/lmr_testing/GET_LMR_REDUCTION.md b/src/lmr_testing/GET_LMR_REDUCTION.md new file mode 100644 index 0000000..5b40031 --- /dev/null +++ b/src/lmr_testing/GET_LMR_REDUCTION.md @@ -0,0 +1,1971 @@ +# GetLMRReduction overhaul -- session handoff (2026-10-02) + +**Note on filename**: this supersedes `lmr_testing/GET_LMR_REDUCION.md` +(missing a T -- an earlier typo in this same session). That file's +content is reproduced/updated below; don't treat the two as +independent histories, the old one is just a mid-session snapshot. + +This session started by making `GetMoveFailHighPercentage`'s dynamic +fail-high table honest, discovered that honesty alone broke LMR's +node-savings, spent a long stretch diagnosing and redesigning the +gate through several dead ends, and ended with a real, working +redesign: self-play validation is pending as of this handoff (the +user is launching it separately). This file is the full history. + +**Current code state** (verified by grep immediately before writing +this): `searchsup.c`'s `GetLMRReduction` uses + +- `LMR_STATIC_ADMIT_THOUSANDTHS = 500` -- wide-open admission ("do we + reduce at all"). +- `JOINT_EXTRA_TIER1/2/3_THOUSANDTHS = 72/5/1` -- now vestigial for + the live magnitude decision (see section 8); TIER1 only appears in + comments, TIER3 is reused by `LMP_STATIC_MAX_THOUSANDTHS` for the + (currently disabled) LMP experiment. +- `LMR_FH_BASELINE_THOUSANDTHS = 4` -- dynamic-danger baseline, + unchanged all session. +- Magnitude ("how much extra") is now computed by + `_FlatTableExtraPlies(uRemainingDepth, uLegalMoves)`, which reads + `g_iLMRQuietReduction` (dynamic.c) -- a Crafty/Ethereal-shaped flat + `log(depth)*log(moves)` table that existed, fully built, completely + unread, since before this session started. This is the live, + working mechanism as of this handoff -- see section 8. +- `JOINT_EXTRA_MAX_PLIES = 5` (cap on extra plies stacked on the base + `-ONE_PLY`). +- LMP (overt late-move pruning, section 7) is implemented in + `ShouldLatePrune` (searchsup.c) but **disabled** at its call site + (`search.c:1228`, `if (FALSE && ...)`) -- measured net-negative, kept + for reference, not deleted. + +Build is clean (`gmake -j5 GENETIC=1 PERF_COUNTERS=1 MP=1 SIXTYFOUR=1`, +no warnings) at every checkpoint described below. + +## 1. The honesty fix (where this started) + +`g_FailHighs` (dynamic.c) used to seed every move-identity to a fake +`1 FH / 1 attempt` (100%) on `ClearDynamicMoveOrdering`, so +`GetMoveFailHighPercentage` never returned a real 0% -- a move needed +~9-10 *clean* observations before its decaying fake prior dropped +below the old `<= 10%` gate. Fixed: + +- `ClearDynamicMoveOrdering` now zeroes the table for real (0/0). +- Renamed to `GetMoveFailHighThousandths` (dynamic.c) -- returns the + rate in thousandths (0-1000), not whole percent, since the real + population baseline (data.c's comment: **0.400%** tries-weighted, + across the 500B-node joint-table collection run) needs sub-1% + resolution to mean anything. +- Added `GetMoveFailHighRawCounts` (dynamic.c) -- raw FH/attempts, used + by diagnostics below. + +Also while in there: killed the stale doc-comment in `_NewCounterMove` +(claimed depth-gated overwrites; code actually always overwrites on +recency, confirmed by its own inline A/B-test comment -- just the +routine description never got updated). Removed `uCounterDepth` +(write-only, zero readers, the "read-time bonus" it was meant to feed +was never built) and the dead `FIRST_COUNTER_MOVE`/`SECOND_COUNTER_MOVE` +flag constants (chess.h). + +Added a countermove-table veto to `GetLMRReduction` itself: a bare +match against `ctx->sPlyInfo[ctx->uPly-2].mv` (the move played +immediately before `mv` -- **two** plies back from here, since `mv` +has already been made by the time `GetLMRReduction` runs) withholds +reduction, independent of generate.c's evidence-gated tier promotion. + +## 2. Consolidation: one function, not two files + +The joint-table-driven "extra reduction" logic used to live in +search.c as post-processing on top of `GetLMRReduction`'s own baseline +decision -- split across two files/two inconsistent dynamic checks +purely to minimize the original diff against HEAD. Moved the whole +decision into `GetLMRReduction` (searchsup.c): + +- `EstimateHangingMaterialCp`, `HangCpBucket`, `GapCpBucket`, + `MoveNumBucket`, `DepthBucket` are no longer file-static -- they're + declared in chess.h now because search.c's own joint-table + *collection* instrumentation (gated on `fPruneReduceEligible` alone, + independent of the LMR decision) still needs them. +- `GetLMRReduction` gained three params: `bbMinorAttacksSnapshot[2]`, + `bbRookAttacksSnapshot[2]` (for `EstimateHangingMaterialCp`; + split.c's HelpSearch has no such snapshot, passes all-zero arrays, + which degrade safely), and `OUT ULONG *puExtraTier` (optional, feeds + search.c's re-search-rate calibration instrumentation). +- search.c's move loop is now just one call + two counter increments. + +## 3. The surprise: depth went DOWN, not up + +First redesign attempt kept the *old* two-check shape (dynamic rate +must be low AND bucket-specific, both gates, a bare 0.4%-baseline +veto) but made it honest. Measured against `head_reference` (committed +baseline, no joint-table mechanism at all -- its `GetLMRReduction` +only ever applies a flat `-ONE_PLY`, old fake-prior-decay gate): + +- **sd10 node count went UP 2-6%** across all three curated suites -- + the honest gate was *more* conservative than HEAD's old loose one. +- A single position (ECM.233) pushed to sd20 showed candidate using + 3x HEAD's node count at the same depth (18 ply) -- undeniable, not + a suite-aggregate artifact. +- A diagnostic (`u64HeadStyleEligible`/`u64HeadStyleWouldReduce`, + chess.h) replays HEAD's exact old gate formula -- + `(rawFH+1)*100/(rawAttempts+1) <= 10` -- against the SAME live + trajectory as the real decision. Result: **HEAD-style reduces + ~76-77% of eligible decisions, always by exactly 1 ply.** The honest + design (0.4% baseline) only reduced 62-66%, but with some decisions + getting 2-4 plies. Average ply-saved-per-eligible-call was actually + *slightly higher* for the new design (0.80-0.88 vs HEAD's 0.73-0.76) + -- so the problem wasn't magnitude, it was **coverage**: breadth + (how many moves get touched at all) matters more to total tree size + than depth-of-cut-per-touch, and extra-tier magnitude carries a + re-search tax (2-3%/tier) that eats part of its own benefit. This + theme (coverage beats magnitude, magnitude has a tax) recurs + throughout the rest of the session -- see sections 8/9. +- A pure threshold sweep on the EXTRA-tier ceiling alone (holding the + honest 0.4% baseline fixed) never recovered HEAD's node count, not + even at the old deliberate-extreme-experiment setting + (9000/8300/1000 thousandths) -- confirming magnitude-tuning on an + already-small qualified population is the wrong lever. +- A sweep of the 0.4% *baseline* itself told a different story: + loosening it to 100 thousandths (10%, matching HEAD's old literal + ceiling) recovered coverage (75.6%) and **beat HEAD's node count by + 35%**. Disabling the countermove-match veto or the bucket-prediction + veto *individually* moved node count by only 1-10% -- noise, + basically. **The flat 0.4% baseline was almost the entire story**, + not the "smart" vetoes. This is the first sighting of the pattern + that eventually won: a simple, broad admission lever beats precise + per-decision judgment. + +## 4. Why the population shape matters: the FH histogram + +A joint histogram of `(uAttempts bucket) x (uFHThousandths bucket)` at +the live decision point (`u64FHJointHist`, chess.h; +`AttemptsBucket`/`RateBucket`, searchsup.c -- still live +`PERF_COUNTERS`-only diagnostic code, not yet removed) showed, on +confident_quick (18.7M eligible decisions): + +- **~71% sit at exactly 0% observed rate**, including a large, + well-evidenced chunk (3.29M with 100+ attempts, genuinely zero FH + across real volume, not a fluke). +- **A real, separate, well-evidenced population sits durably at + 1-10%** -- most of that mass is in the `attempts=100+` row (713K at + 1-1.9%, 1.1M at 2-4.9%, 657K at 5-9.9%), i.e. **not** small-sample + noise. (A real, narrower small-n artifact *does* exist separately: + at `attempts <= 9`, rates between ~1% and ~10% are structurally + unreachable -- `1/9 ≈ 11.1%` is the smallest nonzero rate at 9 + attempts -- but this is a distinct, smaller phenomenon from the + large well-evidenced 1-10% population above.) +- **~7% are pure-cold** (`uAttempts == 0`) and, under the original + two-population design, got flatly refused any reduction regardless + of what the static joint table said. The static table needs NO game + history (keyed on depth/hang/gap/late, not move identity) -- gating + it behind a dynamic-history requirement threw away its whole point + for this slice. + +So the dynamic rate is **bimodal**, not unimodal around 0.4% -- the +population average is a real summary statistic of a mixture, not +representative of either mode. + +## 5. Static-first and three-population redesigns (SUPERSEDED) + +Two intermediate redesigns, both now superseded by section 8's simpler +mechanism, but worth recording for what they established: + +- **Static-first**: flipped priority so the static bucket prediction + became primary ("should we reduce at all"), dynamic signals became + vetoes on top rather than a precondition. Established the + qualitative lesson (cold moves should defer to the static table, not + be refused outright) that survived into every later design. +- **Three-population**: cold / warm+safe / warm+dangerous, with + warm+dangerous capped to base-only rather than fully vetoed. This + *also* established a real, lasting bug: `JOINT_EXTRA_TIER1_ + THOUSANDTHS` was doing double duty as both the admission threshold + ("reduce at all") and the magnitude threshold ("+1 extra ply") -- + decoupling these (section 6) is what unblocked real progress. + +**CONTAMINATION WARNING, still relevant if this file's old numbers +ever resurface**: partway through this session, +`JOINT_EXTRA_TIER1/2/3_THOUSANDTHS` got silently stuck at +**9000/8300/1000** (an old deliberate-extreme-experiment setting) for +a long stretch, because a `cp searchsup.c searchsup.c.orig_full` +snapshot captured a sweep script's leftover state without checking it +first. Every measurement taken during that window (the HeadStyle +diagnostic, the FH histogram, the first static-first pass) was +measured under that contamination. **Lesson for next time**: after ANY +sed-based sweep script, explicitly grep the tier/baseline constants +before trusting a `cp`'d file as a clean base for anything else. + +Despite the contamination, one result survived it, independently +corroborated later: **depth gain from LMR changes is heavily damped**. +Even at the extreme (accidentally-active) ceiling, sn5M average search +depth only moved 0.1-0.3 ply vs HEAD. This matches a documented +finding already in search.c's history (2026-10-01 comment): the same +9000/8300/1000 setting, in an *overnight self-play run*, predicted +~1.0 extra ply/decision but measured only **+0.29 actual root-depth +ply** -- a ~0.3x damping ratio, attributed to EBF: iterative +deepening's cost scales by the effective branching factor per ply, so +affording one more full iteration needs saving roughly `(EBF-1)/EBF` +(~70-75% for EBF 3.5-4) of the previous iteration's total cost. + +**Crucial later correction to this section's pessimism** (from the +user, mid-session): a *different* overnight self-play run, using the +old p5/p25/p75-derived "dodgy" thresholds, did NOT just match HEAD -- +it ran **0.9-1.0 ply deeper and was even-or-better in actual game +strength**. This is the one piece of ground-truth (real game outcome, +not an ECM proxy) evidence the whole session has, and it directly +contradicts "tune LMR for modest percentage savings is the wrong order +of magnitude lever" -- it says aggressive-even-if-locally-unsound +reduction, tolerated because iterative deepening self-corrects +individual mistakes on the next pass, is exactly the right lever. This +reframing drove everything in sections 6-9. + +## 6. Decoupling admission from magnitude + +Advisor-reviewed fix, directly motivated by the three-population bug +above and the "0.9-1.0 ply overnight" evidence: **introduced a +separate constant, `LMR_STATIC_ADMIT_THOUSANDTHS`, used only for "do +we reduce at all"**, independent of `JOINT_EXTRA_TIER1_THOUSANDTHS` +("does this earn +1 extra ply"). Also fixed the sentinel (no static +data) branch: previously required `uAttempts > 0` even when static had +no opinion either, needlessly refusing cold+sentinel moves; changed to +admit whenever not dynamically dangerous, since absence of data is not +evidence of danger. + +Swept `LMR_STATIC_ADMIT_THOUSANDTHS` on sn5M (all three suites, vs +`head_reference`): + +| setting | ringers depth/solve | confident_quick depth/solve | hard_quick depth/solve | +|---|---|---|---| +| HEAD | 11.6 / 6 | 12.4 / 78 | 11.4 / 19 | +| 500 | 11.7 / 6 | 12.8 / 77 | 11.9 / 19 | +| 1000 (no-op gate) | 11.9 / 4 | 12.8 / 76 | 12.1 / 18 | + +500 was flat-to-noise on solves (0/-1/0) while gaining real depth +(+0.1/+0.4/+0.5 ply). 1000 was deeper still but **lost solves on every +single suite simultaneously** (-2/-2/-1) -- a consistent, same-direction +signal, not noise, meaning the extra depth from over-wide admission +wasn't paying for itself. **500 was adopted as the working admission +value**; it remains the current setting (section "Current code state" +above). + +**Methodology note the user corrected mid-session**: sd10 (fixed +depth) solve-count drops are expected and uninteresting (searching +fewer nodes before the ceiling naturally solves fewer problems) -- +solve counts only matter as a signal at **sn5M (fixed node budget)**, +where a drop means the extra depth isn't translating to correctness. +Don't conflate the two contexts when reading solve deltas. + +## 7. Late move pruning (LMP) -- tried, net-negative, disabled not deleted + +Prompted by comparing against Crafty's actual source +(`crafty-25.2/search.c:542,580`): Crafty's LMP is a hard `break` (skip +ALL remaining moves at a node, not just one) gated purely on +`depth < LMP_depth(16)` and `order > LMP[depth]` (a precomputed +`3 + (i+.5)^1.9` table) -- no static table, no dynamic history, no +per-move judgment at all. Implemented a conservative analog, +`ShouldLatePrune` (searchsup.c): overt skip (not reduction) for the +narrowest, most-certain tail only -- `uRemainingDepth <= +LMP_MAX_REMAINING_DEPTH(6 plies)`, move count past a per-depth floor, +AND both static (`JOINT_EXTRA_TIER3_THOUSANDTHS`, the strictest tier) +and dynamic (well-evidenced, near-zero rate) signals agreeing it's +ultra-safe, with killer/countermove exemptions carried over unchanged. +Wired into search.c ahead of LMR/EFP so a pruned move skips both; +reused `fThisMoveEFPPruned`/`fAnyMoveEFPPruned`'s existing hash- +correctness plumbing rather than duplicating it. + +Measured on top of the admission=500 baseline, sn5M: + +| suite | admit=500 alone | +LMP | +|---|---|---| +| ringers | 11.7 / 6 | 11.7 / 6 (unchanged) | +| confident_quick | 12.8 / 77 | 12.8 / **76** | +| hard_quick | 11.9 / 19 | 11.9 / **18** | + +Depth/EBF barely moved (same as admit=500 alone), but solve count +dropped one further on two of three suites at sn5M (the metric that +matters per section 6's note) -- a real cost with no offsetting +benefit. **Disabled at the call site (`search.c:1228`, `if (FALSE && +...)`), not deleted** -- the mechanism may be worth revisiting with +different thresholds (the tested band was very narrow -- only +~250K-moves-pruned out of ~23M total nodes on sd10 ringers, i.e. this +barely touched anything, so "net-negative" here means "net-negative +at this specific narrow threshold," not "pruning can't work." Crafty's +own version is dramatically more aggressive (breaks on ALL remaining +moves, not a per-move judgment call) -- this was flagged mid-session +as "clearly stronger but way dodgier," and was deliberately not +pursued further given the explicit instruction not to get crazy with +pruning (irreversible, no re-search safety net, unlike LMR). + +**Known, unfixed issue in `ShouldLatePrune`'s depth gate** (found +reviewing the Crafty comparison, documented in-code at +`searchsup.c`'s `LMP_MAX_REMAINING_DEPTH` comment but not patched, +since LMP is disabled anyway): the function only permits pruning near +the leaf (`uRemainingDepth <= 6*ONE_PLY`), mirroring Crafty's +`FP_depth`/`LMP_depth` near-leaf restriction. But our own joint table +shows predicted FH% **declining** as remaining depth grows (checked +directly, dumb=0/gap=0/late=0 slice: depth-bucket 0 → 17119 +thousandths, bucket 5 → 4033, bucket 11 → 617, a ~25x drop) -- so the +ceiling permits pruning exactly where our own data says risk is +*highest*, and forbids it where risk is *lowest*. Crafty isn't +self-contradicting the same way: it has no static table at all, so +the near-leaf restriction *is* its entire risk model (a mistake near +the leaf is cheap to re-discover). We inherited the structural gate +without inheriting its justification, stacked on a mechanism that +already has a better, contradicting signal. + +The user's correct reframing: the *cost* of a wrong prune does +legitimately grow with remaining depth, but that should tighten the +**confidence bar**, not forbid the decision outright past a cutoff. The +principled fix is in `lmr_testing/SPEC.md`'s "how much statistical +confidence is actually needed" section: gate on the **static table's +own per-cell sample count** via a rule-of-three bound (`N ≈ 3 * +applications_per_game / target_misses_per_game`) -- not on remaining +depth, not on the dynamic per-move-identity table. SPEC.md derives +`N ≈ 14.4M` for a 1/1000-games miss target (no deep-zone zero-FH cell +clears this) and, after Scott's pushback that 1/1000 was picked, not +derived, a looser 1/100-games target gives `N ≈ 1.44M` (largest real +cell: 1.34M, close) or 1/20-games gives `N ≈ 288K` (16 deep-zone +zero-FH cells clear this today). **Blocker**: `g_uJointFHThousandths` +(data.c) ships only the rate per cell, not the sample count behind it +(cells need ≥500 tries to ship at all, but a 500-try cell and a +1.34M-try cell are indistinguishable once shipped) -- applying this +bound correctly needs a re-bake (`bake_joint.py` + `data.c`) adding a +companion per-cell count array. Not done this session; a stand-in +(depth-scaled dynamic-sample floor) was drafted and then deliberately +reverted rather than leaving an uncalibrated heuristic in place -- +current code still has the original (acknowledged-wrong) depth +ceiling, now with this critique documented alongside it. + +**Concrete lead for whoever re-bakes this**: querying the raw +collection counts directly (not the shipped table) found **43 cells +with a genuine 0.0% fail-high prediction (0 fail-highs across all +tries) backed by ≥288,000 tries** -- clearing the 1/20-games bound +above. These are heavily concentrated at **hang/dumb bucket 9** +(`HangCpBucket`'s near-top bucket, `EstimateHangingMaterialCp`'s +largest "this move hangs a lot of material for nothing" reading), +essentially regardless of depth/gap/late bucket -- the single largest +qualifying cell has **5.1M tries, zero fail-highs**. This isn't a +sparse-sampling coincidence: it's the "dumb" axis doing exactly the +job it was built for at its most extreme setting -- a move that +obviously just hangs material for nothing is essentially never +actually the best move at a node, so "it raises alpha" can't happen +independent of depth/gap/lateness context. **This is the real +population LMP should target once sample counts are shippable**: not +"near the leaf" (the current, backwards ceiling), but "hang bucket 9, +any depth" -- a structural, not positional, criterion. + +## 8. Magnitude-loosening attempts: the real dead end, and the actual fix + +With admission settled at 500, the next lever tried was MAGNITUDE -- +how many extra plies a reduced move gets. Two attempts: + +**Attempt A -- widen the bucket-tier thresholds.** Changed +`72/5/1/cap-3` to `150/20/5/cap-4`. Rationale: Crafty's own LMR table, +at the remaining-depth range our suites actually exercise (5-10 +plies), applies larger cuts (2-6 plies) than our tiers were allowing. +**Result: sd10 ringers node count went UP, 23.0M -> 27.7M -- WORSE +than even HEAD's 26.9M.** The re-search-rate diagnostic showed why: +tier 1's research rate (4.13%) became *higher* than tier 0's (2.73%) +-- exactly the "distribution-shift failure" the code's own calibration +comment warns about (low-predicted-FH tiers should show a RISING +research rate with tier, never an inversion). Widening tier 1's +*admission* boundary let in a bucket of moves genuinely more dangerous +than baseline, and the resulting re-search tax outweighed the savings. +Reverted immediately. + +**Attempt B -- raise only the tier 2/3 *payout*, leave all admission +boundaries (72/5/1) untouched.** Rationale: tiers 2/3 already showed +LOWER research rates than tier 0 even before any change -- i.e. +headroom, not a crack, unlike tier 1. Bumped tier2 payout 2->3, tier3 +3->5, cap to 5. **Result: sd10 node count barely moved (23.04M -> +22.97M, 0.3%)** -- essentially null. Diagnosed (advisor-assisted) +after the fact: **89% of all reduced moves land in tier 0 (no extra +ply at all)**, 10.3% in tier 1, and only ~1.1% combined in tiers 2/3 -- +so tuning tiers 2/3's payout only ever touches the smallest sliver of +the population; it mathematically cannot move the aggregate much +either way. Separately, the new `cap=5` needed `remaining_plies >= 8` +to even engage (`iMaxExtra` floor logic), which these suites' typical +5-10-ply remaining depth rarely reaches -- a second reason this came +back null. + +**The actual fix -- stop grading magnitude with the joint table at +all.** The diagnosis above reframed the real question: 89% of the +population (tier 0) was getting a flat, ungraded `-ONE_PLY` with no +magnitude mechanism touching it whatsoever -- every experiment so far +had been tuning the payout of an 11% minority. Separately, a grep +turned up `g_iLMRQuietReduction[MAX_PLY_PER_SEARCH+1][LMR_TABLE_MAX_MOVES+1]` +(dynamic.c, built by `InitLMRTable()`, called from `main.c`) -- a +**Crafty/Ethereal-shaped flat `log(depth)*log(moves)` table, fully +built at startup, read by nothing, anywhere in the codebase.** A prior +session had built the simple mechanism and then gone and built the +whole joint-table/dynamic-FH system instead, without ever wiring the +simple one in. Sanity-checked its magnitudes against Crafty's actual +table at the depth range that matters (5-10 plies remaining): d=5,m=8 +gives ~2.1 plies here vs. Crafty's 2.3; d=10,m=8 gives ~2.7 vs. 3.0 -- +same ballpark, same shape, and structurally immune to the "one bucket +silently miscalibrated" failure mode that broke Attempt A, because +it's not a lookup into learned per-situation fail-high predictions at +all -- just depth and move order. + +Wired `_FlatTableExtraPlies(uRemainingDepth, uLegalMoves)` in as the +magnitude source, **replacing** the old `_JointTableExtraPlies` +tier-lookup (which is now dead code, deleted) -- admission logic +(static/dynamic agreement, killer/countermove vetoes, +`LMR_STATIC_ADMIT_THOUSANDTHS=500`) is completely unchanged; only "how +much extra" changed. The table returns the FULL reduction in +ONE_PLY-scaled ticks; subtracting the existing `ONE_PLY` base and +flooring at 0 gives the extra-ply count, clamped by the existing +`JOINT_EXTRA_MIN_REMAINING_DEPTH` floor logic as before. + +**Result -- the best of the whole session, every metric improved or +flat simultaneously:** + +sd10 ringers node count: **19.56M**, vs. HEAD's 26.94M (-27.4%) and +vs. the admit=500-alone baseline's 23.04M (-15.1% further). Solve +count unchanged (3/11). + +sn5M, all three suites, vs. HEAD and vs. the previous best +(admit=500 alone, no LMP, no magnitude change): + +| suite | HEAD | admit=500 alone | **flat-table magnitude** | +|---|---|---|---| +| ringers | 11.6 ply / 6 solved | 11.7 / 6 | **11.8 / 6** | +| confident_quick | 12.4 / 78 | 12.8 / 77 | **12.8 / 77** | +| hard_quick | 11.4 / 19 | 11.9 / 19 | **12.0 / 19** | + +Depth improved on every suite vs. both HEAD and the previous best +(+0.2/+0.4/+0.6 ply vs. HEAD); hard_quick's solve count is perfectly +flat (19/19) despite the largest depth gain of any candidate this +session; no suite lost a solve relative to the admit=500 baseline. +This is the first candidate all session where no metric had to be +traded off against another. + +## 9. The pattern across the whole session + +Every lever that worked was a **simple, broad, structural** one +(admission width, Crafty's flat depth x order table); every lever that +failed or plateaued was a **precise, learned, per-situation** one +(bucket-tier magnitude grading, narrow LMP thresholds). The common +failure mode of the precise levers was a correlated miscalibration in +one slice of the population (tier 1's admission boundary) that a flat +mechanism structurally cannot have, because it has no slices to +miscalibrate -- only a single smooth surface in (depth, move-order). +Crafty's own source confirms this isn't a coincidence: neither its LMP +nor its LMR table consults any static or dynamic per-move signal at +all, and it reaches ply 19 at node counts where this engine was +reaching ply 11-12 before this session. The dynamic/static per-move +tables are still useful -- they're what's deciding *whether* to +reduce at all (admission), where they've demonstrably worked -- the +mistake was asking the same machinery to also grade *how much*. + +## Where we left off / next steps + +1. **Self-play validation is the next step, launched by the user + separately from this handoff.** Every metric used all session (sd10 + node count, sn5M depth/EBF, sn5M solve count) is a proxy; the ECM + suites are small and documented-noisy (`../tests/ECM.md`); the only + ground-truth evidence all session is the overnight self-play run + referenced in section 5. **Use a genuine time-per-move or fixed- + node self-play gate, never `sd` (fixed depth defeats the entire + mechanism being tested -- confirmed in `typhoon_lmr_sd11_sn10m_ + protocol` memory).** Check which existing script + (`eval_tune/match_play.py`, `.dna`-file-oriented, built for eval + tuning -- or `eval_tune/test_vs_head.sh`, currently modified in the + working tree) actually supports two *binaries* head-to-head, since + `match_play.py` as built takes two DNA files against one binary, + not two search-behavior variants. +2. **Cross-check any solve-count deltas against `../tests/ECM.md`** + before trusting them -- gained/lost positions skew heavily + flaky/noise; the project's own doc tracks which ECM numbers are + known-flaky vs. known-legit. +3. The current working config (admission=500, flat-table magnitude, + LMP disabled) has NOT been tried with LMP re-enabled using a + structurally different criterion than the narrow one tested in + section 7. Section 7's concrete lead (43 real cells, hang/dumb + bucket 9, up to 5.1M tries, genuine 0% FH -- a "this move obviously + hangs material" criterion, not a depth-near-leaf one) is the + specific next experiment: ship per-cell sample counts + (`bake_joint.py`/`data.c` re-bake), then gate `ShouldLatePrune` on + hang-bucket-9-or-similar + a rule-of-three-cleared sample count, + dropping the backwards depth ceiling entirely. Not attempted this + session (the re-bake is real work, and the explicit instruction was + to not get crazy with pruning without a principled basis -- this + gives one). +4. Before running any further sed-based sweep, confirm the baseline + file is actually clean (section 5's contamination warning) -- + explicitly grep the tier/baseline/admission constants, don't just + trust a `cp` from a few steps ago. +5. The `u64FHJointHist`/`AttemptsBucket`/`RateBucket`/ + `u64HeadStyleEligible`/`u64HeadStyleWouldReduce`/`u64LMPPruned` + diagnostics (chess.h, searchsup.c, root.c, script.c) are explicitly + one-off and still wired in under `#ifdef PERF_COUNTERS`. Harmless to + leave (no behavior change, diagnostic-only), but strip them out once + this investigation concludes. +6. `JOINT_EXTRA_TIER1_THOUSANDTHS`/`JOINT_EXTRA_TIER2_THOUSANDTHS` are + now unused outside of comments/the disabled LMP path respectively -- + candidates for removal once the flat-table magnitude approach is + confirmed by self-play, along with the now-dead `_JointTableExtraPlies` + function (already deleted) and its tier-sweep history in this doc, + which is kept here only as a record of why the simpler mechanism won. +7. If self-play confirms this is a real strength gain, update + `head_reference/` (binary + logs + README) to make this the new + comparison baseline, per this codebase's own stated protocol + (`CLAUDE.md`'s "head_reference protocol" section). + +--- + +# Update, 2026-10-03: admission-rate measurement, a counter-bounds bug, and +# why this session's solve-count evidence does not support what it was used for + +Written while the self-play validation from "Where we left off" item 1 was +running (`--st 4`, SPRT elo0=0/elo1=5, 12+ workers). At 1710 games: +`-426 =831 +453`, score 0.508, llr +0.41, avg depth candidate 13.55 vs head +12.90, ebf 3.464 vs 3.647, nps essentially tied (1.360M vs 1.365M). + +All measurements below were taken with the live candidate binary +(`./typhoon`, confirmed `-DPERF_COUNTERS` and newer than every source file), +`--cpus 1`, `--hash 256m`, three curated suites run serially. +`eval_tune/match_play.py` confirmed to spawn engines with `--cpus 1`, so +`split.c`'s degraded all-zero-snapshot `GetLMRReduction` path is not involved +in the self-play numbers either. + +## 10.1. The match result is underpowered, not null + +Score 0.5079 over 1710 games with that draw rate gives an SD of the mean of +0.0087, i.e. **+5.5 Elo with a 95% CI of roughly [-6, +17]**. The llr is +drifting toward H1, not away. The honest statement is that this match spans +everything from a small loss to a solid gain and cannot yet distinguish them +-- not "the depth didn't translate," and not "it's fine." + +The extra depth is real rather than nominal: `ln(3.647)/ln(3.464) = 1.0414`, +and `1.0414 x 12.90 = 13.43` against 13.55 measured. The ebf reduction fully +accounts for the depth gained, with no residual. + +**That "no residual" is itself the interesting part.** A well-targeted +reduction should beat the arithmetic, because the saved nodes go +preferentially to lines that matter, so accuracy-at-fixed-nodes improves +beyond what pure depth predicts. Getting exactly the arithmetic and nothing +more is the signature of a near-uniform tax on the leftover pool rather than +a reallocation. See section 10.7b. + +## 10.2. Admission rate: the candidate reduces far MORE than HEAD, not less + +Ran the still-wired `u64HeadStyleEligible`/`u64HeadStyleWouldReduce` +diagnostic (section 3 (original)) at `sd 10`. Last cumulative checkpoint per suite: + +| suite | eligible calls | candidate: SOME reduction | HEAD-style: flat -1 | delta | +|---|---|---|---|---| +| ecm_ringers | 2,897,448 | 92.88% | 71.60% | +21.3 | +| ecm_confident_quick | 15,345,704 | 94.47% | 70.37% | +24.1 | +| ecm_hard_quick | 28,632,443 | 93.26% | 74.99% | +18.3 | + +The section 3 (original) under-coverage problem is comprehensively gone. Coverage is now +~19-24 points WIDER than HEAD-style's. + +**Cause, checked rather than assumed.** The first hypothesis was that the +smoothed re-bake widened admission, since `data.c`'s current header reads +"Every cell is populated -- no 0xffff sentinels remain" with 5,241/21,780 +cells shrunk toward a coarser marginal. That hypothesis is wrong, and +backwards: the sentinel branch in `GetLMRReduction` was the PERMISSIVE one +(`fBaseEligible = (FALSE == fDynamicDanger)`, admit unless dynamically +dangerous) while the value branch is CONDITIONAL +(`uPredictedFH < LMR_STATIC_ADMIT_THOUSANDTHS`). Moving cells from sentinel +to value can only TIGHTEN. + +Parsing `g_uJointFHThousandths` out of data.c directly: only **733 of 21,780 +cells (3.4%) read >= 500**, and 224 of those sit in dumb bucket 1 where +`_JointTablePredictedFHThousandths` returns the sentinel regardless -- so only +~509 cells (2.3% of the space) can actually veto. They concentrate in gap +bucket 0 (480 of 733, i.e. near-alpha) and depth buckets 0-1. So +`LMR_STATIC_ADMIT_THOUSANDTHS = 500` has ALWAYS been close to wide open; +section 6 (original)'s 500-vs-1000 sweep was distinguishing a ~6% slice of calls, not a +broad gate. + +What actually widened coverage from section 3 (original)'s 62-66% to today's 93-94% was +**section 6 (original)'s decoupling**: `fDynamicDanger` was demoted from vetoing the base +reduction to merely forcing `uExtraPlies = 0`. A code change, not a data +change. + +**Corollary: section 1 (original)'s `g_FailHighs` honesty fix is now nearly inert.** +`fDynamicDanger` no longer vetoes the base reduction (except for dumb-bucket-1 +moves, which take the sentinel branch), and it only zeroes the extra plies -- +which ~85% of reduced moves never receive anyway (section 3 (original) below). The lever +that started that whole session has almost no remaining effect on shipped +behavior. Worth knowing before investing further in that table's accuracy. + +## 10.3. Magnitude is still overwhelmingly base-only + +Tier share of all REDUCED moves, across the three suites: tier 0 ~85-86%, +tier 1 ~11-13%, tier 2 ~2.2-2.4%, tier 3 ~0.0-0.01%. Section 8 (original)'s "89% of all +reduced moves land in tier 0" survived the switch from `_JointTableExtraPlies` +to `_FlatTableExtraPlies` essentially unchanged. + +So the +0.65 ply in self-play is bought almost entirely by BREADTH (93-94% +admission at a flat -1 ply), not by depth-of-cut. The flat table replaced the +fragile tier payouts without actually paying out much more than they did. + +## 10.4. BUG: `JOINT_EXTRA_MAX_PLIES` (5) overruns `JOINT_EXTRA_TIER_COUNT` (4) + +``` +chess.h:999 #define JOINT_EXTRA_TIER_COUNT (4) // valid indices 0..3 +chess.h:1143 UINT64 u64JointExtraTries[JOINT_EXTRA_TIER_COUNT]; +chess.h:1144 UINT64 u64JointExtraResearch[JOINT_EXTRA_TIER_COUNT]; +chess.h:1149 UINT64 u64LMPPruned; +searchsup.c:514 #define JOINT_EXTRA_MAX_PLIES (5) +searchsup.c:934-6 if (uExtraPlies > JOINT_EXTRA_MAX_PLIES) uExtraPlies = JOINT_EXTRA_MAX_PLIES; +searchsup.c:942 *puExtraTier = uExtraPlies; // 0..5 escapes to the caller +search.c:1279 INC(ctx->sCounters.tree.u64JointExtraTries[uExtraTier]); +search.c:1489 INC(ctx->sCounters.tree.u64JointExtraResearch[uThisMoveJointExtraTier]); +``` + +With `uExtraTier == 4`: `search.c:1279` writes `u64JointExtraTries[4]`, which +lands on `u64JointExtraResearch[0]`; `search.c:1489` writes +`u64JointExtraResearch[4]`, which is past BOTH arrays and lands on +`u64LMPPruned`. The only other clamp, `iMaxExtra` (searchsup.c:922), bounds +against remaining depth, not against the array -- at remaining depth 14 it +permits 11. + +**Why no suite run ever caught it.** `_FlatTableExtraPlies` returns +`(total_ticks - ONE_PLY)/ONE_PLY` with +`total = 0.7844 + log(d)*log(m)/2.4696`. Tier 4 needs `total >= 5.0`: + +| remaining depth | moves tried | total | tier | +|---|---|---|---| +| 10 | 63 (LMR_TABLE_MAX_MOVES) | 4.65 | 3 | +| 14 | 31 | 4.45 | 3 | +| 14 | 63 | 5.21 | **4** | +| 20 | 63 | 5.81 | **4** | + +At `sd 10` remaining depth never exceeds 10, so tier 3 is the ceiling and +every suite measurement in this document (section 8 (original)'s included) is clean. The +live match averages 13.55 ply, so tier 4 fires there routinely. + +**Blast radius**: counters only. Nothing reads these to make a search +decision, so the running match's RESULT IS VALID, and `root.c:1558-1561`'s +memcpys use `sizeof` the 4-element destinations so it does not propagate into +`g_Options`. What it corrupts is (a) tier 0's apparent re-search rate, i.e. +precisely the calibration diagnostic the next candidate would be judged on, +and (b) `u64LMPPruned`, which can read nonzero while LMP is disabled at +`search.c:1228` -- a thoroughly confusing symptom for whoever hits it next. + +**Fix**: raise `JOINT_EXTRA_TIER_COUNT` to 6 (`script.c`'s report loop is +already bounded by the constant, so reporting stays faithful). Clamping +`JOINT_EXTRA_MAX_PLIES` to 3 instead would change search behavior and is +therefore not a pure bug fix. + +## 10.5. The by-tier re-search-rate diagnostic no longer means what section 8 (original) used it for + +``` + tier0 tier1 tier2 tier3 +ringers 2.63% 3.84% 3.55% 3.85% (77 tries) +confident_quick 2.69% 2.94% 2.87% 3.98% (1,204 tries) +hard_quick 2.60% 3.74% 3.30% 2.88% (1,768 tries) +``` + +Section 8 (original) rejected Attempt A because tier 1's research rate (4.13%) came in +HIGHER than tier 0's (2.73%), calling it the "distribution-shift failure." +That was a valid test WHEN tier was a predicted-FH% band +(`_JointTableExtraPlies`): a rising rate meant the risk ordering had broken. + +Under `_FlatTableExtraPlies`, tier is pure MAGNITUDE. A bigger cut +mechanically needs more re-searches regardless of how well-calibrated +anything is, so the inversion carries no calibration signal at all. Note +tier 2 < tier 1 on all three suites, which no FH-ordering story predicts but +a "bigger cut costs more, noisily" story accommodates fine. + +To be useful again this diagnostic must be re-based on PREDICTED FH% +independent of applied magnitude. As currently reported it should not be used +to accept or reject a candidate. Aggregate re-search tax is ~2.6-2.8% of +reduced moves, which is modest. + +## 10.6. The solve-count evidence in this document does not support what it was used for + +Ran `sn 5000000` on all three suites (the metric section 6 (original) established as the +one that matters) against `head_reference/logs/sn5m_*.out`, then diffed +PER POSITION instead of reading the aggregate: + +| suite | HEAD solved/depth | candidate solved/depth | positions FLIPPED | +|---|---|---|---| +| ecm_ringers | 6/11 @ 11.6 ply | 6/11 @ 11.8 ply | 4 of 11 (36%) | +| ecm_confident_quick | 78/90 @ 12.4 ply | 77/90 @ 12.8 ply | 7 of 90 ( 8%) | +| ecm_hard_quick | 19/90 @ 11.4 ply | 20/90 @ 12.0 ply | 15 of 90 (17%) | + +Depth reproduces section 6 (original)/8's admit=500 row closely (+0.2/+0.4/+0.6 ply) and +is the trustworthy axis. The solve counts are not, and cross-referencing every +flip against `../tests/ECM.md` shows the near-flat aggregates are +CANCELLATION, not stability: + +- **ringers (net 0)**: gains `ECM.213` (catalogued flaky, +0.37 unconfirmed + repetition line) and `ECM.270` (catalogued BAD SOLUTION -- score exactly + 0.00, Crafty's own sd20 prefers Rxc5); losses `ECM.721` and `ECM.746`, both + catalogued CLEAN. **Net 0 is two real losses masked by two junk gains.** +- **confident_quick (net -1)**: real gains `ECM.519` (clean, +7.33) and + `ECM.736` (Crafty confirms the same move across depths 24-28 while HEAD + wobbles); real loss `ECM.348` (NOT CATALOGUED); `ECM.357` gain and + `ECM.715`/`716`/`826` losses all catalogued flaky or explicit horizon + artifacts. **De-noised this suite is roughly +2/-1 -- the OPPOSITE sign + from its aggregate.** +- **hard_quick (net +1)**: real losses `ECM.128` (clean, +2.66, no + oscillation) and `ECM.846` (explicitly RECLASSIFIED NON-flaky on 2026-09-11 + from two-engine deep stable agreement); real gains `ECM.476` + (Crafty-confirmed legit) and `ECM.803` (Crafty finds mate in 10 from the + solution move). `ECM.866` is catalogued verbatim as "a genuinely coin-flip + position." `ECM.018`/`084`/`516`/`640`/`774` NOT CATALOGUED. + +**Uncatalogued positions that flipped and should be triaged into ECM.md**: +`ECM.348`, `ECM.018`, `ECM.084`, `ECM.516`, `ECM.640`, `ECM.774`. + +### Implication: section 6 (original)'s admission decision may rest on noise + +`LMR_STATIC_ADMIT_THOUSANDTHS = 500` was chosen over 1000 because 1000 "lost +solves on every single suite simultaneously (-2/-2/-1) -- a consistent, +same-direction signal, not noise." But a single configuration churns 4/11, +7/90 and 15/90 positions here, with each net produced by catalogued-junk +cancelling catalogued-clean in BOTH directions. A -2/-2/-1 aggregate sits +well inside that churn. + +Direction-consistency across three small aggregates was treated as evidence +against noise. That inference does not hold when each aggregate is itself a +near-cancellation of 8-36% churn. + +The same caveat applies to **section 7 (original)'s LMP rejection** (-1 on two suites) +and **section 8 (original)'s "no suite lost a solve relative to the admit=500 +baseline."** None of those conclusions are refuted -- they are unsupported, +which is different. 500 is not shown to be wrong; it is shown to rest on +nothing. + +**Protocol change this implies**: never report a curated-suite solve delta as +an aggregate again. Diff per position, classify each flip against ECM.md, and +report "N real gains / M real losses / K junk / J uncatalogued." The script +used here is three lines of `grep`/`sed`/`join` over the two `.out` files and +should be checked in rather than re-derived. + +## 10.7. Proposed directions, in priority order + +**Superseded in part by 10.9** -- written before the magnitude question was +reconciled (10.8). 10.7a and 10.7c still stand as written; 10.7b is absorbed +into 10.9c; 10.7d is DONE; 10.7e still stands. Read 10.9 for the current +direction list. + +### 10.7a. HEAD vs HEAD at 2:1 time odds -- the decisive unrun experiment + +Grepped `lmr_testing/*.md`, `../src/CLAUDE.md` and `../tests/ECM.md`: this +has never been run or recorded. It is the highest-value measurement available +and it depends on neither ECM nor any proxy. + +**CORRECTED 2026-10-03** -- an earlier version of this section had the +ebf/time conversion INVERTED (it claimed +0.65 ply was "a 0.19x doubling"). +The correct arithmetic, at ebf 3.6: + +- One ply costs a factor of ebf in nodes, so one DOUBLING of time buys + `ln(2)/ln(3.6)` = **0.54 ply**, not ~1.6. +- Conversely this candidate's +0.65 ply costs `3.6^0.65` = 2.30x the nodes, + i.e. it is worth about **1.20 doublings** of time-equivalent. + +That inversion mattered, because it made one of the two branches below look +plausible when it is not. With the correct conversion, if the measured +Elo-per-doubling is D, then a sound +0.65 ply "should" be worth +`1.20 x D` Elo: + +| measured D (Elo/doubling) | expected Elo for +0.65 ply | vs. the ~+2 to +5 measured | +|---|---|---| +| 60 | ~72 | 15-30x shortfall | +| 40 | ~48 | 10-20x shortfall | +| 20 | ~24 | 5-12x shortfall | +| ~3 | ~4 | matches | + +So for the candidate's measured result to be "exactly what depth is worth +here", a doubling of time would have to be worth only ~2-4 Elo -- i.e. +doubling the thinking time barely helps this engine at all. That is a strong, +falsifiable claim, and the measurement settles it: + +- **D is small (< ~10 Elo/doubling)**: typhoon is severely eval-limited. + Depth is nearly worthless to it, no search-efficiency work can pay, and the + lever is eval/speed. Consistent with the known 1.5M-vs-7.5M nps gap vs + Crafty and with eval being the separately-identified bottleneck. +- **D is normal (~20-70 Elo/doubling)**: then reduction-bought plies are + worth drastically less than time-bought plies of the same nominal size, and + the question becomes WHY. That is a search-QUALITY investigation (what is + being lost in the reduced tree), not a tuning exercise -- and it would + explain the whole project's "always slightly positive, never significant" + pattern in one stroke. + +Note which way the remaining bias runs: Elo-per-doubling generally SHRINKS +with depth (diminishing returns), so a D measured at a short time control is +an upper bound on D at a longer one. Using a short-TC D therefore +over-estimates the expected Elo and over-states the shortfall. Record the +time control the calibration was actually run at. + +### 10.7b. Direct the cuts rather than widening them further + +At 93-94% admission and 85% flat -1 ply, the mechanism is close to "reduce +every late quiet move by one ply" -- a uniform tax. A uniform tax lowers ebf +and buys depth, and it structurally CANNOT protect critical lines, because +nothing in it distinguishes them. Section 1 (original)'s dynamic signal was demoted to +extras-only, section 8 (original) removed the joint table from magnitude, and the +admission gate it still owns vetoes 2.3% of the cell space: step by step, each +individually justified, the mechanism converged on undirected. + +Three inputs for directing it are already in scope and unused: + +1. **`iImprovement`** is a declared parameter of `GetLMRReduction` + (searchsup.c:638) and is never read in the body. Not-improving nodes + conventionally reduce ~1 ply more. +2. **`pi->eNodeType`** is computed for every node and is exact + (`NODE_PV` iff wide window, searchsup.c:2203). CUT nodes tolerate more + reduction than ALL nodes, which must search everything to prove a fail-low. + LMR ignores the distinction. +3. **`g_HistoryCounters`/`g_ContinuationHistory`** feed move ordering but + reach reduction only as a binary FH-rate veto, never as a scale on R. + +Frame these as flat structural +/-1 adjustments on a single smooth surface -- +explicitly NOT the "precise learned per-situation" grading that failed in +section 8 (original). They have no buckets to miscalibrate, which is why section 9 (original)'s own +pattern predicts they behave differently from the tier payouts. + +### 10.7c. Re-examine the original section 9's conclusion + +Section 9 (original) concluded that simple broad structural levers work and precise +learned per-situation levers fail. That is an accurate description of what +moved NODE COUNTS. It may be the wrong inference about Elo: the broad lever is +what produced the no-residual, flat-accuracy depth in section 1 (original), and all three +of section 9 (original)'s supporting measurements (sd10 node count, sn5M solve count, +by-tier research rate) are now either noise-level (section 6 (original)) or no longer +meaningful (section 10.5). Node count is not Elo. This is the inference in this +document I would trust least. + +### 10.7d. Fix the counter bug before the next calibration read -- DONE 2026-10-03 + +Section 10.4. Fixed: `JOINT_EXTRA_TIER_COUNT` 4 -> 6 (chess.h:999), with the +broken lockstep against `JOINT_EXTRA_MAX_PLIES` documented in place. Every +user of the constant was already bounded by it (both counter structs, the +`JOINT_EXTRA_VERIFY` arrays, all six `script.c` loops), so nothing else +needed touching. Raised the count rather than clamping `MAX_PLIES` back to 3, +which would have been a search-behavior change rather than a fix. Clean +rebuild, no warnings, batch self-test passes. + +Note for anyone reading counters from a binary built BEFORE this fix -- +including the 1710-game match in 10.1 -- tier 0's re-search rate and +`u64LMPPruned` are corrupted there. The game results are unaffected. + +### 10.7e. Re-derive the admission threshold, if 10.7a says depth is worth buying + +Nothing currently justifies 500 over any other value (section 10.6). If 10.7a shows +depth converts, sweep it against a metric that survives scrutiny -- move +agreement against a deep oracle over a few hundred positions gives far more +samples per position than a binary solved/unsolved on 90, and measures +"better decisions per node" directly. + +## 10.8. Why magnitude is shallow: fully reconciled, plus a unit landmine fixed + +Follow-up to 10.3 ("magnitude is still overwhelmingly base-only"), which +reported the symptom without explaining it. Driven by a direct question: +if we have a 4-axis table that judges P(fail-high), why fall back on a +"more primitive" `log(depth)*log(moves)` formula to size the bet? We are +at a blackjack table and should bet more when we have an edge. + +The answer turns out to be that the magnitude *function* was never the +binding constraint, so swapping it changed almost nothing -- and the +reason it isn't binding is a deliberate invariant, not an oversight. + +### The table is not signal-starved + +Reassembled the `FOURAXISRAW` collection cells from the three 10.2 `sd10` +runs (87.8M eligible tries, 25,159 distinct raw cells) and looked up every +cell's shipped prediction. Tries-weighted, over the admitted population: + +| predicted FH | share | +|---|---| +| < 0.001% | 3.49% | +| 0.001-0.005% | 18.97% | +| 0.005-0.072% | 48.07% | +| 0.072-0.5% | 18.84% | +| >= 0.5% (admission veto) | 10.36% | + +**70.5% of admitted moves read below 0.072%** (the old TIER1 boundary) and +22.5% read below 0.005%. So "we rarely encounter moves the table thinks are +safe" is false -- it thinks most of the population is safe. + +### The two magnitude sources are a wash + +Over LMR's actual population (see the correction below), mean extra plies: +**4-axis table 0.224, flat log(d)*log(m) 0.244** -- the flat formula is +marginally MORE generous, and they put the same ~62% of traffic in tier 0. +Within the depth>=4 slice where extras are legal at all, 0.919 vs 1.002. +They disagree per-cell on 36% of depth>=4 tries (21% where the table wants +less, 15% where it wants more) at essentially identical means. + +So section 8 (original)'s swap did not meaningfully change how much we +reduce. It changed WHICH moves get the extras, at the same aggregate. That +also means section 8 (original)'s headline result -- ringers 23.04M -> +19.56M nodes, attributed to this swap -- **is not explained by the change +it was attributed to.** Still unexplained; flagging rather than guessing. + +### What actually binds: the one-full-width-ply invariant + +`Search()` drops into `QSearch` at `uDepth < THREE_QUARTERS_PLY` +(search.c:462). A reduced move consumes TWO plies before any extra (the +normal `- ONE_PLY` decrement plus the base `-ONE_PLY` carried in +`iExtend`), so `JOINT_EXTRA_MIN_REMAINING_DEPTH = ONE_PLY` encodes exactly +"a reduced move still gets at least one genuine full-width ply before the +horizon." `searchsup.c`'s `iMaxExtra` then works out to +**`remaining_plies - 3`**, and `uDepth > TWO_PLY` (search.c:1254) enforces +the same invariant for the base ply (at remaining depth 3, base-only leaves +exactly `ONE_PLY`). One invariant, two gates -- the design is coherent, and +this floor is load-bearing rather than conservative padding. + +Consequence: **no extra ply is permissible below remaining depth-bucket 4, +for any signal, from any source.** Depth mix of the LMR population: + +| depth bucket | share of LMR pop | iMaxExtra | +|---|---|---| +| 2 (uDepth 129-191) | 45.7% | 0 | +| 3 | 30.5% | 0 | +| 4 | 14.7% | 1 | +| 5 | 5.8% | 2 | +| 6 | 2.3% | 3 | +| 7+ | 1.1% | 4+ | + +Only ~23.9% of the population can receive an extra ply at all. This is just +the shape of an alpha-beta tree: node count grows with ply-from-root, so the +mass is near the leaves, where there is no depth left to cut. + +### Reconciliation of the model against the engine + +10.3 left a 2.7x discrepancy between a first (wrong) model and the measured +mean. Both causes found: + +1. **Wrong population floor (my error).** The first pass filtered at depth + bucket >= 3, but `uDepth > TWO_PLY` admits bucket 2 (`uDepth` 129-191), + where `iMaxExtra` is 0. Bucket 2 is 21.25% of the eligible population and + was excluded entirely. Note also that `FOURAXISRAW` collection is gated on + `fPruneReduceEligible` ALONE -- it does not apply LMR's own + `uDepth > TWO_PLY`, so the collection population is ~4x LMR's and must be + filtered before any magnitude projection. Easy trap; it is a different + population than the one `u64FHJointHist` counts. +2. **The dynamic component**, as predicted. Read straight off + `u64FHJointHist` with no rebuild: `fDynamicDanger` (own rate > the 0.400% + baseline, with any samples at all) zeroes the extras on **27.4%** of LMR + decisions (27.0 / 25.1 / 28.6% across the three suites, 46.9M calls). + 62.1% are warm-with-rate-exactly-0 and 7.1% pure cold; both pass. + +| model | mean extra | x (1 - 0.274) | measured | +|---|---|---|---| +| bucket >= 3 (wrong floor) | 0.440 | 0.320 | 0.164 | +| **bucket >= 2 (correct)** | **0.244** | **0.177** | **0.164** | + +Within 8%, and the residual has a known sign: `FOURAXISRAW` increments +happen AFTER the EFP decision, so EFP-pruned moves get a +`GetLMRReduction` call (counted in the 46.9M histogram) but no collection +increment. EFP fires near the horizon, so the proxy is biased away from the +shallowest depths, which inflates the model exactly as observed. + +**So the magnitude story is: the clamp is the dominant factor, the dynamic +veto is the second, and the choice of magnitude function is third and +nearly irrelevant.** + +### Unit landmine fixed (2026-10-03) + +`GetMoveFailHighThousandths` returned PER-MILLE (0-1000) while +`g_uJointFHThousandths` is in thousandths of a PERCENT -- 100x finer. Every +caller was individually correct, but the two families were indistinguishable +by name, and `ShouldLatePrune` held both four lines apart: +`LMP_DYNAMIC_MAX_THOUSANDTHS (1)` meaning 0.1% beside +`LMP_STATIC_MAX_THOUSANDTHS (1)` meaning 0.001%. The function's own doc +comment asserted it shared the static table's unit "so a caller can compare +this move's own dynamic rate directly against the static table's +prediction" -- which is how the trap got planted. + +Fixed by converting the whole dynamic family to the table's unit (not just +the two constants originally flagged -- leaving EFP and LMP in per-mille +would have relocated the trap): + +| constant | was | now | +|---|---|---| +| `GetMoveFailHighPctThousandths` (renamed) | 0-1000 | 0-100000 | +| `LMR_FH_BASELINE_PCT_THOUSANDTHS` | 4 | 400 (0.400%) | +| `LMR_STATIC_ADMIT_PCT_THOUSANDTHS` | 500 | 500 (unchanged) | +| `EFP_FH_PRUNE_THRESHOLD_PCT_THOUSANDTHS` | 100 | 10000 (10%) | +| `LMP_DYNAMIC_MAX_PCT_THOUSANDTHS` | 1 | 100 (0.100%) | +| `LMP_STATIC_MAX_PCT_THOUSANDTHS` | 1 | unchanged | + +Documented in-code: the old `> 4` per-mille test could only fire once +`fh/attempts >= 0.005`, so the SHIPPED gate behaved like 0.5%, not the +0.400% it documented. 400 is faithful to the intent and a hair stricter than +what shipped; the affected band needs >= 200 observations of a single move +identity to express and is essentially unpopulated per `u64FHJointHist`. +Intended behavior preserved; shipped behavior changed immeasurably. + +## 10.9. Future directions from 10.8 + +Premise, per Scott: when the table says a move is safe we should be more +willing to (1) reduce beyond the clamp, letting the move drop straight into +qsearch, or (2) prune it outright. Both are right in kind; they differ by +more than an order of magnitude in expected payoff, and the measurement +says which is which. + +### 10.9a. Reduce into qsearch when the table certifies safety -- a SAFETY probe, not a depth lever + +Make `JOINT_EXTRA_MIN_REMAINING_DEPTH` table-conditional: `ONE_PLY` +normally, 0 when predicted FH clears a strict bar. This is the FIRST place +in this project where the table would do something the flat +`log(d)*log(moves)` formula structurally cannot express -- a conditional +floor. It is also strictly safer than pruning: the move still receives a +qsearch score and still trips the existing fail-high re-search +(search.c:1484), so it stays recoverable in the one direction we can detect. + +Measured payoff ceiling (mean extra plies, before/after the 27.4% dynamic +veto): + +| variant | mean extra | after dyn veto | +|---|---|---| +| today (`MIN_REMAINING = ONE_PLY`) | 0.224 | 0.162 | +| unconditional `MIN_REMAINING = 0` | 0.506 | 0.368 | +| conditional, pred < 0.005% | 0.302 | 0.219 | +| conditional, pred < 0.001% | 0.227 | 0.165 | + +**Then apply section 5 (original)'s own damping measurement** (1.0 extra +ply/decision -> +0.29 measured root ply): the conditional-at-0.005% variant +buys ~+0.057 extra ply/decision, i.e. roughly **+0.017 root ply.** The +unconditional version reaches ~+0.06 root ply. Both are an order of +magnitude below the +0.65 ply this candidate ALREADY achieves, and well +below measurement noise. + +**So this cannot be justified as a strength experiment -- the arithmetic +will not support it.** Its real value is as the cheapest available probe of +whether the table's "safe" verdict survives a much harsher consequence: if +pred<0.005% cells can lose their last full-width ply with no measurable +damage, that is direct evidence the table is trustworthy at the extreme, +which is the precondition 10.9b needs anyway. Run it at the strict +threshold, judge it on silent-miss rate (reusing `JOINT_EXTRA_VERIFY` plus +the `NEUTRALIZE_NULL_BAILOUT` fix, per section 8 (original)'s lesson), and +do NOT expect or claim a depth gain. + +Risk class to watch specifically: the refutation that is itself a quiet move +one ply deep. That is precisely the documented genuine-miss class 2 (the +`Bb6` deflection, invisible to qsearch by construction), and it is the exact +thing the discarded full-width ply existed to catch. + +### 10.9b. Prune outright -- where the order of magnitude actually is + +Reduction saves plies off a subtree; pruning removes the subtree. The +savings scale completely differently, which is why this dominates 10.9a: + +- 1.88% of eligible tries sit at depth >= 4 with predicted FH < 0.005% + (0.87% at depth >= 5, 0.38% at depth >= 6). +- By SPEC.md's own order-of-magnitude check a depth-6 subtree is ~20,000 + nodes against 30-100 for a near-leaf node. +- A 1.88% population of 100-1000x larger subtrees dominates the node budget + in a way no magnitude change can approach. + +The machinery already exists (`ShouldLatePrune`, searchsup.c, disabled at +search.c:1228) and section 7 (original)'s concrete lead is still unused: +**43 cells with a genuine 0.0% observed fail-high backed by >= 288,000 +tries, concentrated at hang/dumb bucket 9 essentially regardless of +depth/gap/late, the largest with 5.1M tries and zero fail-highs.** The +criterion is structural ("this move obviously hangs material for nothing"), +not positional, and it replaces the current backwards near-leaf depth +ceiling -- which section 7 (original) already documents as permitting +pruning exactly where our own table says risk is HIGHEST. + +Prerequisites, unchanged from section 7 (original) and SPEC.md's agreed +design requirements: ship per-cell sample counts (`bake_joint.py` + +`data.c`) so a rule-of-three bound can gate on evidence rather than on +remaining depth, and build the prune-specific shadow-verification harness +BEFORE pruning for real. + +### 10.9c. Coverage into the populations currently excluded outright + +Neither 10.9a nor 10.9b touches anything outside the already-admitted +leftover pool. Still untouched: the 10.4% that admission vetoes, every +`NODE_PV` node, and every non-leftover (sub-`GOOD_MOVE`-failing) move -- +`fPruneReduceEligible` excludes the latter two structurally. Crafty and +Ethereal both reduce all quiet moves past a move-count threshold, and +"coverage beats magnitude" is the most replicated finding in this document +(section 3 (original), section 6 (original), 10.2). This is the largest +remaining lever by population size, and 10.8 is the explanation for WHY it +is larger than magnitude: magnitude is capped by the depth distribution, +coverage is not. + +Also in this family, and nearly free: the three unused inputs from 10.7b +(`iImprovement`, a declared-but-unread parameter of `GetLMRReduction`; +`pi->eNodeType`; history as a scale on R rather than a binary veto). + +### 10.9d. Standing caveat over all of the above + +10.7a (HEAD vs HEAD at 2:1 time odds) still gates every one of these. If a +ply is worth ~25 Elo in this engine rather than the ~60-70 a healthy engine +shows, then even 10.9b's node savings convert poorly and the real lever is +eval/speed, not search. That measurement is cheaper than any direction here +and decides whether to pursue them at all. + +### Two errors of mine recorded, because the reasoning is the useful part + +1. **"Relax `JOINT_EXTRA_MIN_REMAINING_DEPTH` to 0 as a cheap way to buy + magnitude"** -- withdrawn. It is not a magnitude tweak; it converts + reduction into a qsearch-only evaluation, i.e. stealth LMP, and the + invariant being relaxed is load-bearing (see 10.8). It survives only in + the much narrower, table-gated, safety-probe form of 10.9a. +2. **Filtering the collection population at depth bucket >= 3** instead of + >= 2, which over-predicted mean magnitude by ~1.8x and briefly made the + clamp look like it hard-capped 56% of decisions rather than the correct + ~76% (buckets 2 and 3 combined). Corrected in 10.8; the lesson is that + `FOURAXISRAW` and `u64FHJointHist` count DIFFERENT populations and + neither is LMR's without explicit filtering. + +## 10.10. The time-odds calibration, and the Crafty gap decomposed (2026-10-03) + +Two measurements that between them reframe this whole project. Neither +depends on ECM solve counts or any proxy. + +### 10.10a. A ply is worth ~95 Elo to this engine + +HEAD vs HEAD at 2:1 time odds (`--st 2 --odds 2`, same binary both sides, +no SPRT -- measuring a quantity, not testing a hypothesis). Stopped by hand +at 170 games once the interval was decisive: + +| games | score | Elo | depth (cand vs head) | +|---|---|---|---| +| 20 | 0.700 | +147 +/-109 | 11.93 vs 11.08 | +| 70 | 0.650 | +108 +/- 58 | 12.37 vs 11.33 | +| 120 | 0.646 | +105 +/- 45 | 12.33 vs 11.47 | +| 170 | 0.621 | **+86 +/- 37** | 12.37 vs 11.47 | + +**Result: a doubling of time buys ~0.90 ply and ~+86 Elo, i.e. ~95 Elo per +ply** at this time control on this box. The 95% interval [48, 123] excludes +by a wide margin the "a ply is worth almost nothing here" hypothesis that +10.7a entertained. + +**This is the yardstick the project never had.** Any future depth-buying +change can be pre-registered against it: predicted ceiling = measured ply +gain x ~95 Elo. A change that buys +0.3 ply has a ~28 Elo ceiling (so +`elo0=0 elo1=15` will resolve it in hours); a change that buys +0.05 ply +cannot clear any bar worth caring about and should not get a match at all. +That single number retires the entire class of 2,000-game inconclusive runs +this document is otherwise a history of. + +### 10.10b. Reduction-bought plies are worth ~1/11th of time-bought plies + +Side by side, same engine, same box: + +| source of depth | ply gain | Elo | Elo/ply | +|---|---|---|---| +| 2x time | +0.90 | +86 | **~95** | +| LMR candidate (10.1) | +0.65 | ~+5 | **~8.5** | +| the section 5 (original) overnight "dodgy thresholds" run | +0.9-1.0 | even-or-better | ~0-10 | + +Two independent LMR configurations convert at under 10 Elo/ply while time +converts at ~95. **The conversion loss is structural to reduction, not a +function of how hard it is pushed** -- the more aggressive config did not +convert better. So "squeeze LMR until it reaches 1.0 ply" predicts roughly ++10 Elo, not +95, and at r~2.0 (below) reaching 1.0 ply means HALVING node +count, against the ~27% savings achieved today. + +Leading explanation (Scott's, and it fits): nominal root depth advances +honestly (PV nodes are never reduced), but the REFUTATION lines -- the late +quiet moves at ALL nodes whose fail-low proofs justify the move choice -- +are searched shallower than before. Alpha-beta's answer is only as good as +its cutoff proofs, and reduction weakens exactly those. Time-bought depth +deepens everything proportionally; reduction-bought depth deepens the PV +while thinning its own verification. + +### 10.10c. `local_ebf` was dead on arrival; the real branching factor is ~2.0, not ~3.9 + +Three stacked bugs hid this, found while trying to answer "is our branching +factor really 3.9?" (Scott's suspicion: it is ~2.0, and the harness's number +is inflated by aspiration/root-PV-change re-search cost rather than real tree +growth): + +1. `match_play.py`'s `ebf` is `nodes ** (1.0/depth)` on the CUMULATIVE node + count -- not a per-ply ratio. Its own docstring (2026-09-18) documents + this and says `local_ebf` was added to fix it. +2. `local_ebf` -- the correct estimator (geometric mean of + `nodes_now/nodes_prev` across consecutive completed iterations) -- read + `0.000` in every match since it was added. `_ITER_RE` was applied only in + the trailing stats parser, but the per-iteration PV lines arrive DURING + the search, in the move-scan loop, which read and discarded them. +3. Even after fixing (2), still 0.000: `Engine.__init__` + (`tune_eval_dna.py:120`) sends **`nopost`**, and `root.c:248` gates + per-iteration PV output on `g_Options.fShouldPost`, so the lines were + never emitted at all. Now opt-in via `match_play.py --post`. + +Measured once working, `sd 9`, two engines: **local_ebf 2.02 and 2.12** +against the same runs' cumulative `ebf` of 4.12 and 4.14. Scott's estimate +was right and the published ebf has been wrong in every match report in this +project. Independently corroborated by the time-odds depth response: 2x time +-> +0.90 ply implies a per-ply ratio of `2^(1/0.90)` ~= 2.1. + +Also found: **`--st` takes integer seconds** -- `--st 0.5` yields "Soft time +limit 00:00.000", i.e. an effectively instant search. Any sub-second `--st` +value in a past experiment produced garbage games. + +### 10.10d. The Crafty gap: 4.7 plies of branching + 2.7 plies of speed + +Direct measurement, 12 real-game positions from `tests/cross_sectional_1.ep_`, +both engines at fixed depths 6-12, 64M hash, single-threaded, book off +(`/tmp/typhoon/cr_measure/`). Node accounting verified comparable +beforehand by code reading: Crafty increments per `MakeMove` +(`search.c:452`, `quiesce.c:185/238/352`, `root.c:101` -- qsearch nodes ARE +counted), typhoon once per `Search`/`QSearch` entry (`searchsup.c:2019`). + +Fitting `nodes(d) = C * r^d`, with `r` taken from consecutive-ratio +geometric means (NOT from the regression -- `C` and `r` trade off badly over +a 7-point depth range, visible as absurd `C` of 1,362-4,102 on Crafty's +low-`r` positions): + +| | typhoon | Crafty | +|---|---|---| +| **r** (median) | **2.17** | **1.74** | +| r range across 12 positions | 1.75-2.50 | 1.44-1.99 | + +Decomposition of the ~7-ply gap Scott observes in live games (Crafty at ~19 +when typhoon is at 12): + +| term | contribution | +|---|---| +| branching (r 2.17 vs 1.74 -> depth x 1.391) | **+4.7 ply** | +| speed (1.43M vs 6.6M nps = 4.6x nodes) | **+2.7 ply** | +| predicted Crafty depth at typhoon's 12 | **19.4** (observed ~19) | + +Two independently measured quantities reproduce the observation with no +residual. + +**A hypothesis this KILLS, recorded because it was mine and it was wrong:** +that the gap lives in `C` (unit-subtree cost) because typhoon's qsearch is +too wide. typhoon's qsearch is indeed ~84% of its nodes against Crafty's +~15% (same position, `sd 8`: typhoon 482,156 nodes / 406,205 qnodes; Crafty +65,688 nodes / ~10K evals) -- but `C` fits LOWER for typhoon (~287) than +Crafty (~448), and `C` sits inside the logarithm, so even a 20% `C` +reduction is worth ~0.3 ply. The qsearch width difference is real and +costs almost nothing in depth. The earlier arithmetic that made `C` look +decisive assumed equal `r` and failed a plausibility check (it implied +Crafty's depth-1 search costs 11 nodes, when it must visit ~35 children). + +So: **the branching term is the bigger half of the gap and the one that can +be attacked.** Scott's instinct to work on search efficiency was right. + +### 10.10e. Crafty's lazy-eval rates, measured + +Settling a disagreement about whether Crafty's eval is cruder. Crafty has +TWO lazy exits and the printed fields distinguish them (`iterate.c:624-630`, +both percentages over denominator `evaluations + lazy_evaluations`): +`lazy=` is the FIRST (material-only, `evaluate.c:60/64`), `full=` went +through piece scoring, and the residual is the SECOND lazy exit +(`evaluate.c:243`, skips piece scoring). + +12 positions, `sd 10`: material-only **28.8% mean** (median 30, range 0-45); +second lazy exit **24.9%**; full eval **46.3%**. So **~54% of Crafty's eval +calls skip the expensive piece scoring**, which supports Scott's 40-50% +recollection; the material-only tier alone is ~29%, not the 20% claimed +earlier in this session from a single position at a single depth (the same +position reads 30% at depth 10 -- that number was a bad sample, and +depth-dependent). + +typhoon has the matching three-way split (`u64SuperLazyEvals`/ +`u64LazyEvals`/`u64FullEvals` plus cycle counts, `root.c:508-542`) but +behind `#ifdef LAZY_EVAL`, not in the standard profile -- one rebuild gets a +like-for-like comparison. + +**Caveat on what this can support**: lazy exits fire when the score is +already far outside the window, i.e. in lopsided positions where detail does +not matter. These counters measure eval COST, not eval QUALITY where it +counts. Neither side of the eval-quality argument is settled by them. + +### 10.10f. Exchange truncation: the mechanism, and three fixes with very different costs + +Scott's symptom: losing a pawn or knight "at the end of an exchange" against +Crafty, prompting an idea to extend all captures fractionally. + +NOT the futility margin -- `iFutility = iAlpha - +(FUTILITY_BASE_MARGIN_BY_SOURCE[pi->uLastPositionalSource] + +rgiPositional[...]) - iEval` is indexed by eval's own positional source, +i.e. genuinely eval-calibrated rather than hand-picked. + +The mechanism is the SEE-ordered early exit at the top of `QSearch`'s move +loop: moves are SEE-sorted, and the first `iValue <= 0` does `goto end`, +abandoning every remaining move at that node. So qsearch only ever explores +captures SEE scores as non-losing, and **any exchange whose resolution +requires passing through a SEE-negative intermediate capture is outside the +horizon entirely.** + +That cutoff is sound only to the extent SEE is right, and SPEC.md already +documents that it is not, specifically: `_WhoAttacksSquareBB` (see.c) has no +pin or legality awareness -- the finding that made the queen-vs-queen dumb +bucket read backwards and got that signal deleted. A pinned defender still +counts as a recapturer, so SEE calls a winning capture losing and qsearch +never tries it. Same root cause, different symptom. + +| fix | touches | cost | +|---|---|---| +| make SEE pin-aware (see.c) | `C` | ~0.3 ply for a 20% `C` rise; also improves every other SEE consumer (`fPruneReduceEligible`'s capture rule, EFP, `MVF_SEE_*`) | +| admit SEE<0 captures at shallow qsearch depth | `C` | bounded, tunable | +| extend all captures fractionally | **`r`** | exponential; at ~95 Elo/ply, expensive | + +Recommendation: fix the cause on the `C` side before spending `r`. + +### 10.10g. Next steps implied + +1. **Measure Delta-r directly for extension changes, do not infer it from + the counters.** `root.c:568-577` prints every extension source count, but + those count GRANTS, before `MINU(ONE_PLY, ...)` caps and before + `g_uExtensionReduction[]` decays -- so they cannot give the applied ply + inflation that actually moves `r`. The 12-position C/r sweep + (`/tmp/typhoon/cr_measure/run.sh`) is now the right instrument: build with + extensions trimmed toward Crafty's single SEE-gated 1-ply check and + re-run it. Crafty has exactly ONE extension (`search.c:476/483`, checks + only, `check_depth=1`, SEE-gated, and it disables reductions for that + move); typhoon has ten fractional per-move sources plus five node-level + ones. Whether typhoon's 3/4-ply-but-decayed check extension "washes out" + against Crafty's 1-ply-but-SEE-gated one is exactly the question a Delta-r + measurement answers and counter arithmetic does not. +2. **Route matters more than destination.** Closing `r` from 2.17 to 1.74 is + worth 4.7 plies ~= 450 Elo at the 10.10a yardstick. But reduction-bought + plies convert at ~8.5 Elo/ply (10.10b), so the question is not "how do we + lower r" but "how does Crafty lower r SOUNDLY". Extension trimming is the + first candidate identified that is not "reduce harder": it declines to + spend effort rather than suppressing information. +3. **Clamp removal is off the table.** Stockfish floors the reduced depth at + 1 ply (`search.cpp:1370`, `std::max(1, ...)`) -- the same invariant + typhoon has -- and achieves its leanness through separate explicit + pruning (Step 15, `search.cpp:1166`). Crafty has no such floor and does + not even verify a move reduced into qsearch (the full-depth re-search at + `search.c:957` is inside the `depth > 0` branch only). The strongest + engine keeps typhoon's invariant. Combined with 10.9a's arithmetic + (+0.017 root ply), two independent lines say drop this idea. +4. The 10.9b hang-bucket-9 pruning lead is unaffected by any of the above + and remains the best-evidenced pruning target (3.3% of eligible tries, + observed FH 0.0112%, 44x safer than hang bucket 0). + +## 10.11. Where the branching gap actually comes from, and the prioritized backlog (2026-10-03) + +### 10.11a. Crafty's r advantage is move-list TRUNCATION, not reduction grading + +Settling the "should we grade reduction magnitude harder" question for good. +Crafty's entire LMR adjustment, `search.c:579-581`: + +```c +reduce = LMR[Min(depth, 31)][Min(order, 63)]; +if (reduce && (pv_node || extend)) + reduce--; +``` + +One decrement. No history, no improving, no node-type, no per-move signal -- +and it reaches r = 1.74 against typhoon's 2.17. **So adjustments cannot be +where their leanness comes from.** (Stockfish does have a dozen such +adjustments; Crafty does not, and Crafty is still far leaner than typhoon. +This retires the "typhoon's candidate is SF's base with no adjustments" +theory floated earlier in this session, and matches Scott's report of five +prior LMR attempts with assorted factor-mucking, none of which paid off.) + +What sits immediately ABOVE those lines in Crafty's move loop does the work: + +```c +if (depth < FP_depth && !check && MaterialSTM(wtm) + FP_margin[depth] <= alpha && !pv_node) { + tree->moves_fpruned++; break; // abandons ALL remaining moves +} +if (order > LMP[depth] && depth < LMP_depth && !pv_node && !check && ...) { + tree->moves_mpruned++; break; // abandons ALL remaining moves +} +``` + +Measured in one `sd 8` sample: `fp=10.0K mcp=10.3K` against `nodes=65688` -- +**~20,000 move-list truncations in a 66,000-node search.** typhoon has EFP (a +per-move `continue`) and a disabled LMP, and **no truncation mechanism at +all**. That is the structural difference behind the 4.7-ply branching term +in 10.10d. + +### 10.11b. "We are holding the 4D table wrong" -- the specific sense + +The table is currently consulted for reduction ADMISSION at a threshold that +vetoes **2.3% of the cell space** (733 of 21,780 cells read >= 500, and 224 of +those are in the sentinel'd dumb bucket 1). Section 8 (original) removed it +from magnitude; section 6 (original) demoted the dynamic half to extras-only. +Step by step a surface with a ~28,000x dynamic range in predicted FH% was +reduced to making a near-binary decision that fires on ~10% of traffic. + +Its dynamic range only earns anything where the decision's stakes are +correspondingly asymmetric: + +- A **reduction** is recoverable and its payoff is a ply or two off one + subtree. Fine-grained risk estimates are wasted there -- and 10.8 measured + exactly that: table-driven magnitude 0.224 mean extra plies vs the flat + `log(d)*log(m)` table's 0.244. A wash. +- A **prune** is irreversible and its payoff scales with SUBTREE SIZE + (~20,000 nodes at depth 6 vs 30-100 near the leaf). That is where + "0.0112% observed FH across 2.9M tries, 44x safer than baseline" is worth + having, and where Crafty operates blind on depth-and-movecount alone. + +**So: Crafty gets r = 1.74 by truncating move lists on a crude rule, and we +have a measured FH surface they do not. Holding the table right means using +it to decide WHERE TO TRUNCATE.** + +### 10.11c. Scott's dimension argument: a sound derivation of Crafty's `break` + +Of the four axes, only **dumbness** is genuinely move-specific: + +| axis | scope | +|---|---| +| gap to alpha | node-level constant | +| remaining depth | node-level constant | +| lateness (move number) | move-specific, but MONOTONE -- every later move is at least as late | +| dumbness (hang cp) | move-specific, varies arbitrarily | + +Therefore: if the table **marginalized over dumbness** reads "never fails +high" for this (depth, gap, late) cell, that read applies at least as +strongly to every remaining move at the node, because two axes are fixed and +the third only worsens. That justifies abandoning the rest of the move list +-- Crafty's `break`, derived from our own data instead of assumed. + +**This does NOT contradict the 2026-09-27 dumb-axis-pooling bug**, and the +distinction is subtle enough to be worth stating: that bug was +`ConfirmedZeroFailHigh` using the dumb-SUMMED marginal to vouch for a +*specific* move sitting in a rare, dangerous dumb bucket -- unsound, because +the cell's zero was produced by ordinary moves dominating its volume while +the rare queen-hanging move had never been tested on its own. The argument +here runs the opposite direction: it uses the pooled read to vouch for the +**dominant** population. Since dumb=0 is 82.3% of all tries, a pooled rate +of ~0 mathematically requires the dumb=0 subpopulation to be ~0 too, so +pooling is CONSERVATIVE for this purpose. + +Preferred implementation (Scott's): per-move `continue` rather than one +`break`, so the move-specific dumbness veto still applies to the move in +hand. Nearly the same node savings, none of the blanket assumption. + +### 10.11d. The near-leaf depth ceiling is backwards and should be deleted + +`ShouldLatePrune`'s `LMP_MAX_REMAINING_DEPTH <= 6*ONE_PLY` was inherited +from Crafty, where a near-leaf restriction IS the whole risk model because +they have no table. Our own table shows predicted FH% FALLING ~25x from +depth bucket 0 to bucket 11, so the ceiling permits pruning where risk is +highest and forbids it where risk is lowest (already documented in +section 7 (original), never patched). + +The cost of a wrong prune does rise with remaining depth -- which is what a +rule-of-three CONFIDENCE BAR is for, not a depth cutoff. Per Scott: pruning +a move with depth 11 under it is fine *if* the static read says "no way" and +the dynamic vetoes are silent; benefit rises exponentially with remaining +depth while measured risk falls. + +### 10.11e. Per-cell sample counts are available NOW -- no re-collection + +`joint_table_collection/pickles/raw_2026-10-02.pkl` survives and holds +per-cell `(fh, tries)` for all 20,075 populated cells; `grand_tries` = +690,062,068,603 over 56,530 positions. Running SPEC.md's rule-of-three bars +(`N ~= 3 * applications_per_game / target_misses_per_game`, ~4,800 +applications/game): + +| miss budget | required N | zero-FH cells clearing it | +|---|---|---| +| 1 game in 20 | 288,000 | **43** | +| 1 game in 100 | 1,440,000 | **9** | +| 1 game in 1000 | 14,400,000 | 0 | + +This reproduces section 7 (original)'s "43 cells" lead exactly and adds the +1/100 tier. **The blocker named in section 7 -- that `g_uJointFHThousandths` +ships only the rate, not the sample count behind it -- is a +`splice_joint.py` + `data.c` change against an existing pickle, not a new +collection run.** + +### 10.11f. Prioritized backlog + +**1. HIGHEST IMPACT -- deep pruning via the table (10.11b/c/d/e).** +Rationale: Crafty's 4.7-ply branching advantage is demonstrably truncation +(10.11a); typhoon has no truncation; the payoff is exponential in remaining +depth; the data exists; 43 cells already clear a 1/20-games bar. Closing r +from 2.17 toward ~1.9 is worth ~1.6 ply ~= ~150 Elo at the 10.10a yardstick +-- an order of magnitude more than anything else on this list. +Steps: (a) ship per-cell tries via `splice_joint.py`/`data.c`; (b) build the +prune-specific shadow-verification harness FIRST, with the +`NEUTRALIZE_NULL_BAILOUT` fix applied from day one (SPEC.md's agreed design +requirement #4); (c) re-point `ShouldLatePrune` at a rule-of-three-cleared +sample bar + dumb-marginalized monotonicity criterion, deleting the depth +ceiling; (d) keep the dynamic per-move-identity veto (requirement #5). + +**2. CHEAP AND UNATTENDED -- HEAD vs HEAD with LMR disabled.** +The control condition 25 years of this work has lacked. The 10.10a yardstick +measures what TIME-bought depth is worth; nothing has ever measured whether +HEAD's own flat -1 ply LMR converts. If removing it costs 0.5 ply and ~45 +Elo, reduction-bought depth does convert for HEAD and the candidate broke +something specific. If it costs 0.5 ply and ~5 Elo, reduction-bought depth +has never been worth anything in this engine and the answer is LESS +reduction, not better reduction. One build, one match, runs while other work +proceeds. Use `SPRT_ARGS="--sprt --elo0 0 --elo1 15"`. + +**3. DURABLE INFRASTRUCTURE -- the move-agreement test.** +Replaces curated-suite solve counts (shown to be noise at these sample sizes +in 10.6) with a metric that has a usable signal-to-noise ratio: + - take ~300 real-game positions (`tests/cross_sectional_1.ep_`); + - build an ORACLE best move per position -- ideally Crafty at high depth, + independently implemented and cheap for them; + - run each candidate at a fixed `sn` budget; record % agreement with oracle. +Every position contributes a signal (not just the ~20% that are tactical +puzzles with a known answer), there is no dependence on EPD solution +correctness, and 300 positions gives a binomial SE of ~2.9%. +Immediate use: at EQUAL node budget, does the LMR candidate pick the oracle's +move less often than HEAD? Yes => its extra nominal depth costs real decision +quality. No => the depth was simply never worth anything (a different problem +with a different fix). Refinement: score both moves with the oracle and only +count a disagreement when it prefers its own by a real margin. + +**4. SMALL BUT CLEAN -- drop the six negligible extension sources.** +From a real `sd 15` dump: `zug` fired **9** times, `endg` 352, `mult+` 1,410, +`recap` 10,339, `sing` 10,296, `q+` 35,427 -- against the check extension's +1,483,277. These cannot affect anything at those volumes and need no gate; +just confirm grant counts go to zero and node counts do not move. Weighted +by ply amount the check extension is **~78% of all extension spend**; every +other source combined is ~0.35 ply ~= a ~33 Elo ceiling, and individually +3-9 Elo each -- unmeasurable by self-play (~25,000 games to separate 5 Elo +from 0). So test the mid-tier (threat + bm + 1mv + !kmvs + pawn) as ONE +bundle, not five matches. + +**5. SCREEN-ONLY -- check-extension magnitude variants.** +Non-negotiable as a mechanism (Scott: prior attempts at SEE-gating caused +sacrifice-blindness; "king moves out" variants exploded the branching +factor). But the AMOUNT and gate are open: 3/4 -> 1/2 ply; material gate -> +Crafty's SEE gate; 3/4 -> 1 full ply (Crafty's amount -- extensions are not +automatically bad). ~4 minutes each on the C/r sweep +(`/tmp/typhoon/cr_measure/run.sh`). Note the 3/4-ply SEE>=0 branch is +700,479 grants = 2.1M of ~2.9M check quarter-plies, i.e. 57% of ALL +extension spend -- that is the branch worth screening, not the 1/4-ply one. + +**6. DEAD -- do not revisit.** + - *LMR magnitude grading / per-factor adjustment*: five prior attempts, + plus 10.8 (table vs flat is a wash) plus 10.11a (Crafty has one + decrement and is leaner anyway). + - *Clamp removal / reducing into qsearch*: 10.9a arithmetic (+0.017 root + ply) plus Stockfish flooring reduced depth at 1 ply + (`search.cpp:1370`) while getting its leanness from separate explicit + pruning. + - *SEE<0 check extension removal*: measured Delta-r = -0.004 (median over 12 + positions), a ~3 Elo ceiling. Also independently reported to cause + sacrifice-blindness in earlier work. The counter-based estimate that + sized this at 21% of extension spend was wrong by ~20x because + `uCheckByQuarterPly[1]` pools the SEE<0 and low-enemy-material branches + -- **screen with Delta-r, never size an extension from grant counts.** + - *Attacking qsearch width to close the Crafty gap*: 10.10d -- `C` fits + LOWER for typhoon (~287) than Crafty (~448), and `C` sits inside the + logarithm, so a 20% change is worth ~0.3 ply. + +## 10.12. The control experiment, the conversion-rate model, and why flat LMR is hard to beat (2026-10-03, later) + +### 10.12a. LMR vs no LMR -- the control condition 25 years of this work never had + +`DISABLE_LMR` (GNUmakefile) makes `GetLMRReduction` return 0 before consulting +any signal. Two binaries from the SAME tree, differing only in that flag, run +head-to-head with NO SPRT (measuring a quantity, not testing a hypothesis -- +same reasoning as the time-odds calibration in 10.10a). `noLMR` is the +baseline, so a positive score means LMR helps. + +| games | score | Elo | depth (withLMR vs noLMR) | +|---|---|---|---| +| 50 | 0.550 | +35 +/- 69 | 13.75 vs 12.59 (+1.16) | +| 90 | 0.550 | +35 +/- 51 | 13.96 vs 12.79 (+1.17) | +| 110 | 0.559 | +41 +/- 46 | 13.72 vs 12.57 (+1.15) | +| 150 | 0.553 | +37 +/- 40 | 13.59 vs 12.43 (+1.16) | +| 180 | 0.575 | +53 +/- 36 | 13.51 vs 12.36 (+1.15) | +| 210 | 0.569 | **+48 +/- 34** | 13.54 vs 12.35 (+1.19) | + +**LMR is worth roughly +40-50 Elo and buys ~1.17 ply.** Remarkably stable +across six checkpoints. The depth delta is the largest and steadiest of any +candidate in this document's history. + +This settles a question nobody had ever asked: LMR *does* earn its keep here. +Scott's prior ("LMR has never earned its keep in this engine") was wrong, and +so was the stronger form of the author's "reduction-bought depth is debased" +thesis. + +### 10.12b. The conversion-rate model: diminishing returns, not a flat rate + +Pulling every Elo-per-ply measurement together: + +| mechanism | what it buys | measured rate | +|---|---|---| +| **LMR vs NO LMR** (the FIRST reduction) | +1.17 ply | **~40 Elo/ply** | +| graded LMR *on top of* flat LMR | +0.65 ply | ~8 Elo/ply | +| pruning (`PRUNE_LIVE=10`) *on top of* graded LMR | +0.88 ply | ~8 Elo/ply | +| 2x time (NEW resource, not a transfer) | +0.90 ply | ~95 Elo/ply | + +That is a decreasing-marginal-returns curve, and it is a better model than the +flat "~8 Elo/ply for all tree-shaping" claim made earlier in this session. + +**Why the earlier claim was wrong**: the graded-LMR and prune candidates were +both measured against `head_reference`, WHICH ALREADY HAS LMR. Their +0.65 and ++0.88 ply were MARGINAL plies stacked on an already-reduced tree. 10.12a +measures the FIRST ply of reduction, which is where the value is. + +**Practical calibration (Scott's framing, and it is the right one)**: there is +not another 40 Elo available from LMR/LMP/pruning work. There is maybe 40-45 +Elo TOTAL in the whole reduction program, of which the dumb flat -1 ply +already captures ~85%. Five failed attempts to improve on it do not mean it +cannot be improved -- they mean expectations should be calibrated to single +digits, and that single digits cannot be measured in affordable self-play +(resolving +5 vs 0 needs ~10,000 games). + +**ALSO NOTE a yardstick error this corrects.** 10.10a divided the time-odds +result (+86 Elo for +0.90 ply) and called it "~95 Elo per ply," then used that +to compute ceilings all session. That division assumes depth is the ONLY +channel through which extra nodes help. It is not: at fixed nodes, a ply bought +by shrinking the tree is worth far less. So "95 Elo/ply" is the rate for +TIME-bought plies specifically and must never be applied to tree-shaping +changes. Every ceiling computed from it earlier in this document was inflated +(e.g. `PRUNE_LIVE=10`'s ceiling was quoted as 33 Elo; corrected to ~7, and it +measured +7 -- it hit its real ceiling exactly rather than underperforming). + +### 10.12c. Why a flat -1 ply is so hard to beat -- the actual mechanism + +The curve is already present (`_FlatTableExtraPlies` reads +`g_iLMRQuietReduction`, dynamic.c's `log(depth)*log(moves)` table). The problem +is WHERE it can act. Evaluating `0.7844 + log(d)*log(m)/2.4696`: + +| remaining depth | moves tried | formula | extra plies | +|---|---|---|---| +| 2 | 16 | 1.56 | **0** | +| 3 | 16 | 2.02 | 1 | +| 4 | 16 | 2.34 | 1 | +| 10 | 20 | 3.58 | 2 | + +**At remaining depth <= 2 the curve prescribes ~1 ply -- which IS flat -1.** +And 76% of the LMR population sits at depth buckets 2-3 (10.8). So flat is not +a crude approximation of the curve; at the depths where the mass lives, the two +AGREE. + +Measured consequence: the curve contributes mean **0.164 extra plies** over the +LMR population, which through section 5 (original)'s damping (~1.0 extra +ply/decision -> +0.29 root ply) is about **+0.05 root ply**. That is the +curve's entire current contribution, and it explains the null results directly. + +The curve's remaining headroom is the ~11% of moves at remaining depth >= 4, +further limited by `iMaxExtra = depth - 3` (the one-full-width-ply invariant). + +**Open and never cleanly tested**: curve-vs-flat in self-play. The 2212-game +run (section "Self-play status", 2026-09-29) tested the joint-table TIER +magnitude under the pre-unification `uLegalMoves > 5` gate, NOT section 8's +`log(d)*log(m)` version, which was only ever validated on sd10 node counts and +sn5M suites. An earlier draft of this section wrongly treated that null as +having answered the question. + +### 10.12d. Retraction: eNodeType is too unreliable to modulate reduction + +10.7b and 10.9c recommended using `pi->eNodeType` to reduce more at CUT nodes +than ALL nodes. **Withdrawn.** After fixing the node-type surprise counters +(10.12f) the full-width rates read: ALL nodes fail HIGH 69-71% of the time, +CUT nodes fail low 12-14%. An "ALL node" that fails high two-thirds of the time +is not behaving like an ALL node -- the structural label (assigned purely from +the parent's type) is close to uninformative here. Modulating reduction off a +signal that unreliable adds complexity for noise. + +### 10.12e. QSearch / QSearchFromCheckNoStandPat review + +Reviewed for logic errors in the custom stand-pat and check-generation rules. + +**SHIPPED: option B -- waive the opponent-stand-pat requirement at PV nodes.** +`fIncludeChecks`' first clause suppressed check generation whenever the +opponent could stand pat at an ancestor. That suppression rests on a sound +argument *for null-window nodes*: once their stand-pat has raised alpha, no +line below that score can change what they return, so resolving mate-in-10 vs +"drops a queen" is wasted work -- and even when their stand-pat did NOT raise +alpha, the node is heading for a fail-low, which IS the parent's beta cutoff, +so any score at or below alpha answers the question identically. It does NOT +hold at a WIDE-WINDOW node, where the exact score propagates to the root and +those two outcomes are a different PV and a different reported score -- which +is the forced-mate-from-qsearch case this machinery exists for. Now: +`(fCouldStandPat[FLIP(uToMove)] == FALSE) || (NODE_PV == pi->eNodeType)`, +keeping the material requirement (an independent sanity check, not a +suppression). Reads eNodeType, never writes it. Uses eNodeType rather than +recomputing `(iBeta != iAlpha+1)` because iAlpha may have just been raised by +the stand-pat, which would make a recomputed width test read narrow for a node +that is genuinely PV. + +**SHIPPED: `iExtensionAmount` unit fix.** `QSearchFromCheckNoStandPat` wrote +`uQsearchCheckExtension` (0 or 2, counting qsearch CHECK PLIES) into a field +that is ONE_PLY-scaled everywhere else (root.c writes HALF_PLY, split.c writes +iLMR, search.c writes iExtend) -- a latent 32x error. Benign as it stood (both +readers only test sign) but the field is already a documented stale-state +hazard. Now scaled; sign and all reader behavior unchanged. + +**SHIPPED: `pi->iEval` published** when `QSearchFromCheckNoStandPat`'s lazy +`Eval()` actually runs. It used to stay purely local, so that ply slot kept a +stale value. Still stale on the common in-check path where Eval is never called +at all (no stand-pat is offered, so there is nothing to compute it for) -- +making that safe needs invalidation on entry, a separate change. + +**TRIED AND REVERTED THE SAME DAY: `iBestScore = MAX(iAlpha, iEval - +iInTrouble)`.** The observation behind it was real -- the beta test lives only +in the else-branch, so a node with `iEval >> iBeta` and hanging material +returns a fail-low instead of cutting off. The fix was wrong twice over, and +self-play caught it in 10 games (candidate depth collapsed to +0.05 ply vs +head, from the +0.65 the same tree showed without it): + + Work the signs. A parent searching this node passed window + `(-iBeta_p, -iAlpha_p)`, so `iAlpha` here == `-iBeta_p`. The parent fails high + iff `-V >= iBeta_p`, i.e. iff `V <= iAlpha`. So returning exactly `iAlpha` is + not a crude placeholder -- **it is precisely the value that hands the parent + its cutoff.** Returning MAX(...) instead reports a better-than-alpha value and + denies that cutoff (~0.45 ply of lost depth), AND asserts the node is worth at + least `iEval - iInTrouble` without searching anything -- a stand-pat credit at + a discount, exactly what the branch exists to refuse. + +`MIN(iAlpha, iEval - iInTrouble)` was then proposed and also rejected: it is +depth-neutral (V <= iAlpha always, so the cutoff always fires) but it cannot +produce the cutoff the original observation was about (MIN never exceeds +iAlpha), and it asserts an unearned TIGHTER upper bound, which propagates as an +inflated lower bound into the parent's `StoreLowerBound`. The original +observation is still unfixed; a sound fix needs a bound the node has EARNED. +Probably moot: a node with material in trouble almost always has a capture +available, and captures sort above the `iValue <= 0` early exit, so the move +loop usually recovers the cutoff by search. + +**Confirmed correct** (checked, no change needed): the qsearch-entry reset +invariant (`search.c:573` and `command.c:739` clear `fCouldStandPat[]`, +`uQsearchNodes`, `uQsearchDepth` and reset `uQsearchCheckDepth` before every +drop into qsearch; the only other QSearch call site is +`CALIBRATE_QSEARCH_FUTILITY`-only and already inside a qsearch state); +`fCouldStandPat` scoping (each node saves/restores only its own side's flag and +never writes the opponent's, so `fCouldStandPat[FLIP(uToMove)]` genuinely +reflects the parent); paired `uQsearchDepth`/`uQsearchCheckDepth` increments +with no `goto end` between any pair; and the `uLegalMoves > 0` guard on escape +futility pruning, which correctly prevents a bogus mate score even when leading +escapes are illegal. + +**The exchange-truncation mechanism**, for the record: it is NOT the futility +margin (`iFutility = iAlpha - (FUTILITY_BASE_MARGIN_BY_SOURCE[ +pi->uLastPositionalSource] + rgiPositional[...]) - iEval` is genuinely +eval-calibrated). It is the SEE-ordered early exit at the top of QSearch's move +loop: moves are SEE-sorted and the first `iValue <= 0` does `goto end`, +abandoning every remaining move. So qsearch only explores captures SEE scores +as non-losing, and any exchange whose resolution needs a SEE-negative +intermediate capture is outside the horizon. That cutoff is sound only to the +extent SEE is right, and SPEC.md already documents `_WhoAttacksSquareBB` +(see.c) having no pin/legality awareness. Three fixes with very different +costs: make SEE pin-aware (`C`-side, ~0.3 ply for a 20% C rise, and improves +every other SEE consumer); admit SEE<0 captures at shallow qsearch depth +(`C`-side, bounded); extend all captures fractionally (`r`-side, exponential). +Fix the cause on the C side before spending r. + +### 10.12f. Node-type surprise line: four defects + +Scott's question ("how can 47% of ALL nodes fail high?") was right, and chasing +it found four separate defects in one Trace line: + +1. **Denominator was total nodes**, not nodes of that type. +2. **Qsearch was included** (~84% of the tree). A qsearch node whose stand-pat + beats beta returns `iBestScore >= iBeta` immediately; typed ALL, that reads + as "ALL failed high" while being the most ordinary event in qsearch. Now + excluded from numerator (QSearch's copy MOVED to its own slots) and + denominator (`u64NodeTypeCount`, incremented in `Search()` only). +3. **Stale node types.** Every early exit in `CommonSearchInit` returns TRUE + *before* `pi->eNodeType` is assigned, and the caller's `goto end` lands on + the counters -- which scored the node against whatever type the PREVIOUS + node at that ply left behind. That produced an impossible "106.2% of PV + nodes". Now guarded by `fNodeTypedThisVisit`. +4. **The fail-low test used the in-place-raised `iAlpha`.** `search.c:2048` + does `iAlpha = iScore`, so a node that SUCCESSFULLY raised alpha ends with + `iAlpha == iBestScore` and `(iBestScore <= iAlpha)` reads true -- every + success counted as a fail-low. The PV row read 99.1%. Now uses + `iInitialAlpha`, which already existed for exactly this distinction + (`search.c:2105`). + +Also added, per Scott's QNODE suggestion applied to the counters rather than +the enum: `u64QNodeTypeCount`/`u64QNodeSurprise`, plus a **PVS identity +cross-check**. The identity: a CUT node failing low returns `score <= alpha_C`, +so its parent (an ALL node) sees `-score_C >= beta_P` and fails HIGH, and that +parent cannot have had an earlier child fail low (it would have cut off first) +-- so #(CUT fails low) and #(ALL fails high) should run ~1:1. Counting qsearch +separately brought the measured ratio from 3.6 to **1.86**. The residual means +~46% of ALL-node fail-highs come from nodes resolved with NO child fail-low at +all: qsearch stand-pat cutoffs, interior-node recognizer/EGTB fail-highs, and +null-move cutoffs (including the Tord Romstad bailout, which fabricates a score +and searches nothing). The NODE_TYPE enum and every behavioral consumer (all of +which only test against NODE_PV) are untouched. + +### 10.12g. PERF_COUNTERS cleanup -- the standard profile was carrying investigation debris + +An audit for real WORK (not counter increments) inside `#ifdef PERF_COUNTERS` +found five sites in three categories. All three were fixed. + +**Category 1 -- three `rdtsc` calls leaking into the standard build.** +`eval.c`'s king-safety hash probe bracketed itself with +`SystemReadTimeStampCounter()` on entry and exit, in one of the hottest paths +in the engine; `rdtsc` is ~20-30 cycles and partially serializing. An +`EVAL_TIME=1` flag already exists for exactly this ("make a version that counts +cycles spent in eval") and already guards the other **32** rdtsc sites in that +file -- these three were the only leaks. Moved to EVAL_TIME. (The report side +in root.c was already EVAL_TIME-guarded, so nothing dangles.) + +**Category 2 -- two one-off diagnostics, deleted outright.** Both did per- +decision lookups plus integer division at the hottest decision point in the +search, and both had comments saying to remove them once their investigations +concluded: + - `u64HeadStyleEligible`/`u64HeadStyleWouldReduce` (called + `GetMoveFailHighRawCounts`). Its investigation concluded -- it produced + 10.2's 92-94%-vs-70-75% admission finding. + - `u64FHJointHist` (called `GetMoveFailHighPctThousandths`). Also concluded -- + it produced section 4 (original)'s bimodal-dynamic-rate finding. +Removed the counters, the `g_Options` mirrors, the suite accumulation and BOTH +report blocks -- a diagnostic that prints zeros forever is worse than none. +`AttemptsBucket`/`RateBucket` went with them. Binary shrank 7,760 bytes. + +**Category 3 -- the 4D collection block, re-gated to FH_TABLE_COLLECTION.** +`search.c`'s `u64JointTries`/`u64JointFH` collection calls +`EstimateHangingMaterialCp` (real attack-bitboard work) per eligible move. It +is COLLECTION instrumentation, and CLAUDE.md already says FH_TABLE_COLLECTION +is deliberately kept out of the standard profile *because this measurably costs +nps* -- this block had escaped that gating. Verified by guard inspection: +`search.c:1637` is now `PERF_COUNTERS > FH_TABLE_COLLECTION`; the only other +`EstimateHangingMaterialCp` call site is the prune predictor, correctly behind +`PRUNE_VERIFY`/`PRUNE_LIVE`. +**Bonus:** it was computing the hang bucket TWICE per move in any build with a +prune gate (the gate needs the same value), which inflated the `PRUNE_LIVE` +candidate's nps deficit (-6.0% vs head) and made that match mildly unfair to +the candidate. + +**nps impact: NOT MEASURED.** Two attempts were made and both are worthless +because the box was running a 32-engine self-play match. The second attempt +produced `perf_clean` FASTER than `noperf`, which is impossible (the cleaned +build still has every counter increment `noperf` lacks) -- proof that noise +exceeded the effect. Also a methodology error worth not repeating: the script +ran all of arm A, then all of B, then all of C, so a load drift masquerades as +an effect. **Re-measure on an idle box with ROUND-ROBIN interleaved arms**, +fixed node budget (`sn 8000000`), same position, 3+ runs each. + +### 10.12h. The deep-prune audit and miss verification + +`PRUNE_VERIFY` (GNUmakefile) -- audit-only shadow verification for the prune +gate. It evaluates the table's prediction for every `fPruneReduceEligible` move, +records it banded by predicted FH%, then **declines to prune** and watches +whether the move fails high in its own ordinary search. No bespoke shadow +`Search()` call is needed, which sidesteps the null-move-bailout fabrication +that made `JOINT_EXTRA_VERIFY`'s first numbers 80-99.8% artifact. Banding by the +decision variable means one run answers "is `prune if pred <= K` sound" for +every candidate K at once. + +Pooled over the three curated suites (65.6M flagged decisions), measured FH% +ran **3-19x BELOW predicted** in every prunable band -- the Wilson re-bake +(10.10/10.11e) doing what it was for. But `cross_sectional` positions (250 +samples, the real-game distribution) came back **5-11x WORSE than the curated +suites** in the low bands, with `measured > predicted`. The explanation is +distributional and inverts SPEC.md's stated worry: in a tactical puzzle the +action is in forcing moves, which gate-1 excludes by construction, so leftover +quiet moves there really are the dregs; in an ordinary middlegame the best move +is frequently a quiet one, landing squarely in the leftover pool. **Validating +a prune gate on puzzles understates its risk for real games.** + +Verification of the 71 low-band (`pred <= 2`) cross_sectional misses against +clean deeper searches: **27 confirmed real, 44 artifacts.** The margin column +discriminates cleanly -- `margin ~= 0` means artifact (a score that only just +clears a tight inherited PVS window), huge margin means a genuine forced mate. +That inverts the author's initial guess, which had it backwards. + +One confirmed band-0 miss is worth keeping as a counter-example to the dumb +axis: `6k1/p4p1p/3p2q1/2p1rP2/2P3p1/P7/1P1Qn1PK/3R1R2 w`, after `Qg6`. The +queen only LOOKS hung -- White is losing too badly to take it (best line grabs +the e2 knight; White is -14 at depth 19), so a move the table rates +"obviously hangs material for nothing, never fails high" is actually winning. +A structural mispredict that no amount of sampling conservatism catches, which +is precisely why this harness had to exist before pruning went live. + +**Prune threshold measurements** (12-position C/r sweep, paired per-position +deltas -- note the author initially reported difference-of-medians instead of +median-of-differences, which flipped the sign on two of three arms): + +| threshold | share of eligible | median dr | ply @d12 | verified miss rate | +|---|---|---|---|---| +| `pred <= 2` | 4.3% | -0.0040 | +0.03 | ~1 in 481,000 | +| `pred <= 5` | 16.6% | -0.0281 | +0.21 | ~1 in 360,000 | +| `pred <= 10` | 30.6% | -0.0465 | +0.35 | ~1 in 121,000 | + +Band 6-10 alone carries 79% of the misses for 29% of the flagged volume, so +`pred <= 5` keeps ~60% of the depth for ~16% of the risk. `PRUNE_LIVE=10` +self-play: +0.88 ply, +7 Elo at 150 games (llr flat) -- which is exactly its +corrected ~7 Elo ceiling (10.12b), so it performed as well as it possibly could +and that is still not interesting. + +Also measured and worth recording: **a dynamic veto cannot save these prunes.** +Cross-tabulating prunable-band candidates by (own-FH attempts x own-FH rate), +11 of 23 low-band misses sit at **rate = 0** -- the per-move history agrees with +the static table and both are wrong. A rule catching the 12 high-rate misses +would refuse ~390,000 sound prunes to do it (~49,000 per miss prevented), the +same failure mode that retired the `GetMoveFailHighPercentage` veto before. + +## 10.13. Open action items as of 2026-10-03 EOD + +Ordered by expected value. Items 1-2 are the only ones with a plausible +double-digit Elo payoff. + +### 1. Give EFP the `dumb` axis -- the best remaining idea +EFP is keyed on gap-to-alpha and blind to the other three. Measured over EFP's +actual operating envelope (10.2): the **dumb axis swings FH% 15-30x** within an +otherwise-fixed cell while **gap -- EFP's only lever -- swings ~4x** across its +entire knight-to-rook range. EFP is reading the weakest signal and ignoring the +strongest. And unlike every LMR/LMP experiment, this is an **accuracy** play +rather than a depth purchase: EFP prunes the same volume either way, so it is +not buying marginal plies at ~8 Elo/ply and nothing measured so far bounds its +upside. It also finally gives the 4D table a consumer that uses its dynamic +range. Caveat from section "the tactical-position gap": the first table-driven +EFP attempt lost ECM.272 and four others and threshold-tightening rescued only +one -- diagnosed as a training-population problem. The table has been +re-collected and Wilson-re-baked since, so retry rather than assume. + +### 2. The nps gap +Crafty measured at **5.5M nps** on a real 10-second search (an earlier 7.5M +figure came from a 0.01s search where the timer is meaningless -- do not quote +it) against typhoon's ~1.4M. Even discounting for Crafty counting illegal moves +as nodes (`search.c:452` increments before the legality test), that is roughly +4x. Time-bought depth converts at ~95 Elo/ply, so the speed term in the Crafty +gap (~2.7 ply) is worth ~250 Elo -- the only lever on the table with that +magnitude. Scott's profiling says abandoning the 0x88/bitboard hybrid buys +maybe 500knps, which does not close it, and the mechanism for their remaining +advantage is not understood. + +### 3. Re-measure the PERF_COUNTERS cleanup on an idle box +Round-robin interleaved arms, `sn 8000000`, same position, 3+ runs each. Two +prior attempts are worthless (see 10.12g). Arms are built: +`/tmp/typhoon/candidates/typhoon_{perf,perf_clean,noperf}`. + +### 4. Curve-vs-flat LMR, cleanly +Never tested in self-play (10.12c). One flag: current tree vs current tree with +`_FlatTableExtraPlies` forced to 0. Expected ~1-3 Elo from the arithmetic, so +self-play cannot resolve it -- measure **node count at fixed depth** instead, +where 0.164 mean extra plies should be visible. + +### 5. Decide the fate of the 4D table +Its only live shipping consumer is `LMR_STATIC_ADMIT_PCT_THOUSANDTHS`, which +vetoes 509 of 21,780 cells (2.3% of cell space, 7.4% of traffic) -- and that +threshold was chosen in section 6 (original) on solve-count deltas that 10.6 +showed were noise. **Cheap decisive test**: build with admission effectively +disabled (`= 100000`) and run against current. If neutral, the table buys +nothing in a shipping build. NOTE item 1 may make this moot by giving the table +a real consumer -- do item 1 first. + +### 6. Triage six uncatalogued ECM positions into ../tests/ECM.md +`ECM.348`, `ECM.018`, `ECM.084`, `ECM.516`, `ECM.640`, `ECM.774` all flipped in +the 10.6 per-position diff and are not in the flaky/legit catalogue. + +### 7. Smaller / hygiene +- **Strip the remaining one-off diagnostics**: `u64LMPPruned` and the + `JOINT_EXTRA_VERIFY` family are still wired under PERF_COUNTERS. +- **Share the hang bucket** between the collection instrumentation and any + prune gate instead of computing `EstimateHangingMaterialCp` twice (10.12g). +- **`pi->iEval` is still stale** on the common in-check qsearch path (Eval is + never called there). Only reader is `search.c:760`'s `iImprovement`, which + should never see a qsearch ply given the layering -- add an ASSERT recording + that invariant. +- **`JOINT_EXTRA_TIER1/2/3_PCT_THOUSANDTHS`** are vestigial (TIER1 appears only + in comments, TIER3 only in the disabled LMP path). +- **A sound fix for the `iEval >> iBeta` lost cutoff** in QSearch's + material-in-trouble branch is still open; both MAX and MIN forms were tried + and rejected (10.12e). Needs a bound the node has EARNED. Probably low value + -- the move loop usually recovers the cutoff by search. +- **`ShouldLatePrune`'s depth ceiling is still backwards** (`<= 6*ONE_PLY` + permits pruning where the table says risk is highest). Moot while LMP is + disabled. + +### 8. Shelved deliberately +- **`PRUNE_LIVE`/LMP**: ~7 Elo, hit its ceiling exactly, needs the table most. + Code stays behind its flag; do not ship. +- **`DISABLE_LMR`**: diagnostic only. 10.12a says removing LMR costs ~40-50 + Elo, so this is NOT a shipping candidate. +- **Clamp removal / reducing into qsearch**: twice falsified (10.9a arithmetic; + Stockfish floors reduced depth at 1 ply while getting its leanness from + separate explicit pruning). +- **LMR magnitude grading via the joint table**: 10.8 (table vs flat is a wash) + plus 10.11a (Crafty has ONE `reduce--` and is leaner anyway). +- **`eNodeType` as an LMR signal**: 10.12d. +- **SEE<0 check-extension removal**: measured dr = -0.004, ~3 Elo ceiling, and + independently reported to cause sacrifice-blindness. diff --git a/src/lmr_testing/SPEC.md b/src/lmr_testing/SPEC.md new file mode 100644 index 0000000..55e6c5c --- /dev/null +++ b/src/lmr_testing/SPEC.md @@ -0,0 +1,1543 @@ +# EXECUTIVE SUMMARY: + +We are implementing late move reduction / late move pruning in a chess +engine based on a "simple idea": measure the propensity of a given +move at a given node in the search tree (search.c/split.c) to fail +high using (at least) four signals: + +- The move's "lateness", i.e. the order it was generated by the move + generator code (generate.c). Because this code generates moves in + order with the best moves first, the only moves left at the end are + either quiet moves or losing captures, usually. Not very + interesting ones, usually. To be extra certain that the late moves + we are looking to reduce/prune are uninteresting, we also prohibit + checking moves, escapes from check, moves on PV nodes, moves with + extensions, killer moves, and, in general, do not prune when any of + several signals indicate that this "uninteresting quiet move" might + be more than meets the eye. +- The move's "dumbness", i.e. does it seem to hang material? A move + of our queen next to an enemy pawn might be a brilliant sacrifice + but most of the time it's a blunder. +- The situation at the node (position) at which the move was + generated, i.e. is the side to move "winning" or are things "going + badly"? Since our search is an alpha..beta search, the gap between + a static evaluation of the current node (iEval) and alpha (iAlpha) + indicates "how far the side to move would have to improve the score + at the current position in order for our move to matter (reach the + PV and/or fail high). The bigger this gap, the less likely a move + is to fail high because to do so requires a stronger move. +- The move's remaining depth. This one is counter-intuitive but if + you do the work to compute the probability of a move to fail high + then you find that the deeper we search the move tree under that + move, the *less* likely the move is to fail high. This is a really + interesting phenomenon -- it indicates that scores (and therefore + PVs) returned by shallow searches are very noisy but once you search + 3-4-5-8-12 ply under the move, the score usually stabilizes and + actually diminishes the deeper you look. + +We call these four signals "axes" and have computed a probability table +where, given a score for each of these signals, you can look up the +computed probability that the move (and node) exhibiting these signals +will fail high. This is the core of our approach. + +# GOAL: + +The goal is to reduce the size of the chess search tree that we search +(and therefore to reduce our effective branching factor) by reducing +the depth we search the subset of moves we believe will not fail high +or outright pruning them (i.e. not searching under them at all). Each +node saved (via reduction or pruning) is re-allocated to cause the +engine to be able to search deeper on (hopefully) more interesting and +meaningful nodes/moves elsewhere in the tree. + +# DANGER: + +When we reduce the search under a move we introduce error in the score +returned. It can bite us in two ways: +- The move can fail high even though we did not expect it to. If a + move fails high with a reduced depth search, search.c re-searches + it at full depth to confirm the fail high so this is not fatal; it + just wastes some time and double searches the tree under the move + in question. +- The move can fail low even though if we would have searched it with + full depth it would have failed high. This is a fatal miss; we have + lost information here. To help defend against this, we have built a + version of the engine that, when it selects a move to reduce, it + then proceeds to run the search at full depth anyway and pays + attention to how many times a reduced move would have ended up + failing high. This is very rare but not impossible. It is, by far + the most dangerous thing about reducing moves. + +All in all, though, reducing the search tree under a weak, +uninteresting move is less dangerous than the alternative: pruning the +move completely. Pruning means never searching under the move at all +-- skipping it completely. Therefore, the threshold to reduce a move +is much less than the threshold needed to prune a move outright. +When we are making irrecoverable decisions, we must be conservative! + +# Four-axis joint FH% table -- next steps, 2026-09-28 + +See `lmr_testing/2026-09-28_four_axis_joint_table_plan.md` for the full +plan (axis definitions, table shape/encoding, collection/bake recipe, +consumption design). This section used to duplicate that entire file +verbatim; removed 2026-09-30 (alongside the late-bucket fix below) so +future edits only need to touch one copy. + +**Correction, 2026-09-30, carried over from the plan doc**: the table's +`late` axis shipped-bucket mapping had an off-by-one. Raw `late` +(`uLegalMoves` at the point it's sampled) can never be less than 2 -- the +collection/consultation gate (`fPruneReduceEligible`, `search.c`) requires +`uLegalMoves > 1`, and `_MoveNumBucket`'s own 1-based-index comment +already ruled out 0. The old bucketing (`raw - 1`, real floor assumed to +be 1) left shipped bucket 0 permanently unreachable -- a full +`depth x dumb x gap` slice (1,452 of 21,780 shipped cells) that read the +`0xFFFF` sentinel forever, not from insufficient collection but because +no sample could ever land there. Fixed by shifting `raw - 2` instead (real +floor is 2): raw 2-13 -> shipped 0-11, 14-21 -> 12, 22-31 -> 13, 32+ -> 14. +Applied in `search.c`'s `_ShipLateBucket`, `dashboard.py`'s +`ship_late_bucket`, and `joint_table_collection/bake_joint.py` (newly +written -- imports `dashboard.py`'s bucket functions rather than +redefining them). `data.c`'s `g_uJointFHThousandths` was rebaked from the +2026-09-30 collection run with the fix in place: real coverage is +18,775/21,780 (86.2%) populated, not the 17,444/21,780 (80.1%) the old +bucketing would have reported on the same data. No data was discarded -- +this was a bucketing/reporting bug, not a population/gate mismatch (a +separate concern raised and ruled out the same day, see below). + +# FH%-by-(depth, move, gap) table, EFP==LMP unification, and LMR as a +# fallback -- brain dump, 2026-09-26 + +This is a different, unrelated project from the `README.md`/ +`searchsup_GetLMRReduction.c`/`generate_counter_move_block.c` files +already in this directory (that's the 2026-08-27 graded-LMR-curve + +counter-move sweep, reverted to a clean baseline and stashed for +possible later reuse). This write-up is about a same-week but +independent effort: building a real, measured `P(fail-high)` surface +and using it to drive EFP, and eventually LMR, instead of hand-picked +constants or a hand-picked reduction curve. + +## The core idea + +EFP is a pruning method currently in use in the engine; we are +considering amending it or replacing it. + +Historically, EFP (extended futility pruning) was Ernst Heinz's +two-tier design: only active within ~2.75 ply of the qsearch horizon, +gated by a hand-picked material margin (`VALUE_KNIGHT` at <=1.75 ply, +`VALUE_ROOK` at <=2.75 ply), with a per-move `ComputeMoveScore(mv) < +uFutilityMargin` test on top. LMP (late move pruning) didn't exist in +this codebase at all; Crafty-style engines that have it usually key it +on move number/depth alone, blind to how far below alpha the position +actually looks. + +The thesis this session worked from: **EFP and LMP are answering the +same underlying question** -- "is it safe to stop searching from here" +-- just entering from different angles (EFP: "this move's static score +already looks bad, gap-driven"; LMP: "we've tried enough moves, +move-count-driven"). Once you have real data on `P(fail-high | depth, +move, gap-to-alpha)`, there's no reason to maintain them as two +separate hand-tuned mechanisms with two separate blind spots. One +table, consulted from either direction, replaces both. See the +conversation this file's SessionId corresponds to for the full +back-and-forth that arrived at this; the short version is EFP was found +to be nearly move-number-blind (a move at position 3 and a move at +position 25 with the *same* gap get treated identically by the old +margin test, despite real data showing a 6-12x difference in FH% between +them at the same depth/gap), and that's what motivated building the +joint table instead of patching EFP's old formula. + +## The data: what it is, exactly + +`f(remaining_depth, move_number, gap_to_alpha) -> P(fail-high)`, +where: + +- **remaining_depth**: `uDepth / ONE_PLY`, whole plies, NOT capped at + the old EFP's 2-3 ply window -- deliberately extended, because the + data shows depth 1-2 is the *riskiest* part of the curve, not the + safest (FH% falls ~5x from depth 0-1 to depth 9-12, confirmed + independently holding move number fixed, so it's a real depth effect + not a move-mix artifact). EFP's old activation window covered exactly + the highest-risk region and left the safer, deeper region entirely + unpruned. +- **move_number**: 1-based ordinal position in the move list. Also + measured as move-*fraction* (fraction of the move list exhausted, + 20 buckets of 5%) as a parallel table, since "move 15 of 18" and + "move 15 of 54" are different signals and raw move count doesn't + capture that. The fraction table exists but the raw-move-number one + is what's actually wired into the live lookup right now. +- **gap_to_alpha**: `iAlpha - iEval`, the NODE's static eval deficit + below alpha, in centipawns, bucketed 50cp wide (`FH_GAP_BUCKET_WIDTH`), + from -200cp up through +1400cp+ (`FH_GAP_BUCKETS = 32`, chess.h). + Deliberately the cheap, already-computed node-level gap (`iEval` is + computed for every node regardless of EFP) rather than a per-move gap + built on `ComputeMoveScore` -- the costlier per-move version was + considered and explicitly deferred, since computing it unconditionally + for every leftover move (to avoid the exact censoring bias described + below) would add real per-move cost everywhere, not just within EFP's + old narrow activation window. + +Of the three, **gap dominates by far** -- a ~28,000x swing in FH% from +the worst to best gap bucket (measured on real data: 28.3% at -200.. +-150cp down to 0.001% at +1300..+1350cp), versus roughly 5-12x from +depth or move number alone. But move number and depth both carry real, +independent signal on top of gap -- confirmed by slicing FH% at a FIXED +gap bucket and watching it still vary 4-12x across move number, and +still vary ~5x across depth. All three matter; none alone is sufficient. + +## The population the data is measured over: "gate-1" + +This is the single most important methodological point, and got fixed +late after an expensive lesson (see "the first real attempt" below). +**The table must be trained on exactly the population that will +actually reach it in real search, not on "all leftover moves."** + +Note: a "leftover move" is not a capture, killer or good countermove. +They are the quiet moves and losing captures. They are the tier of +moves produced by generator.c with a score < GOOD_MOVE. + +If you train an FH-rate table on a broader population that includes +checks, killer-adjacent moves, threat-node members, and normal +captures -- categories that a real EFP/LMR gate would ALREADY exempt +via cheap, separately-validated exemptions -- the resulting table is a +systematic OVERESTIMATE of risk for the population that actually +reaches it, because those excluded categories are (by design, since +they earned their exemption from evidence) higher-FH-rate populations +diluting the measurement. + +The "gate-1" eligibility checklist, applied both when COLLECTING the +data and when CONSULTING the table live, is: + +- `fIsLeftoverMove` (sub-`GOOD_MOVE` -- not a winning/even capture, + killer, killer-mate, or sufficiently-evidenced countermove) +- `!IS_CHECKING_MOVE(mv)` +- `!IS_ESCAPING_CHECK(mv)` +- `iExtend <= 0` (no extension already applied) +- `!fThreat` (this node's own null-move probe didn't raise a threat + signal) +- `(!IS_CAPTURE_OR_PROMOTION(mv)) || (iMoveSee < 0)` -- captures/ + promotions excluded UNLESS losing (SEE < 0), which are admitted. This + was a deliberate widening from the old EFP's blanket capture + exemption: a losing capture is character-wise much closer to a bad + leftover move than a good one, and `iMoveSee` is already computed for + exactly this class of move at shallow depth (`uDepth < THREE_PLY`), + so admitting it costs nothing extra. Caveat: `iMoveSee` defaults to 0 + (not a real SEE value, treated as "non-losing") for captures outside + that depth range where `GetMoveSee` never actually ran -- a + non-issue for the current depth-1-9 beachhead, but worth remembering + if the table's depth range is ever pushed further out without also + extending where SEE gets computed. + +The first data-collection pass (before this fix) trained on the +unrestricted leftover population and showed real cells with FH% up to +20-30% at aggressive-looking gap/move combos -- looked alarming, until +the `fThisMoveEFPCandidate` audit mechanism (below) showed the ACTUAL +population reaching a table lookup in real search had real FH% around +0.1-0.15% at a "5.0% predicted" cutoff, a 30-50x gap entirely explained +by the population mismatch, not a flaw in the gap/depth/move signal +itself. Re-collecting on the gate-1-restricted population removed this +bias; the retrained table's own audit numbers then matched its +predictions almost exactly (within ~1-2%), confirmed by comparing real +outcomes against the volume-weighted average PREDICTED value for +exactly the flagged population -- NOT against the raw threshold used to +flag it, which is a different, much more permissive number and an easy +place to fool yourself (a threshold is a ceiling on what gets flagged, +not the typical value of what actually does). + +## How the data is collected + +Three parallel counter tables live in `chess.h`'s `COUNTERS.tree` +(mirrored into `g_Options` for suite-level reporting), all gated behind +`#ifdef FH_TABLE_COLLECTION` (its own opt-in `gmake` flag, NOT part of +the standard `PERF_COUNTERS` profile -- this instrumentation measurably +costs NPS when active on every gate-1-eligible move, and there's no +reason to pay that cost outside a dedicated collection run): + +- `u64LeftoverTriesByDepthMove` / `u64LeftoverFHByDepthMove` -- raw + move-number x depth, `[65][64]`. +- `u64LeftoverTriesByDepthMoveFrac` / `...FHByDepthMoveFrac` -- move + FRACTION x depth, `[65][20]`. Watch out: `MOVE_COUNT(ctx, ctx->uPly)` + can be 0 when the hash move is tried before this ply's moves are + generated (a distinct search stage) -- skip the frac increment + entirely in that case, don't clamp/floor it. An earlier version used + a `MAXU(1, ...)` floor instead of skipping, which silently dumped + every such move into the "95-100% exhausted" bucket regardless of its + real position, corrupting it with a huge, spuriously-low-FH% + population -- caught by a smoke test where that one bucket had + 20-100x the sample count of its neighbors. **Always smoke-test a new + counter design on a tiny run before trusting a big collection pass.** +- `u64LeftoverTriesByDepthMoveGap` / `...FHByDepthMoveGap` -- the full + 3-way table, `[65][64][32]`. This one's cell count is ~65x the other + two combined, and printing it in full (even just the compact + `FHGAPRAW` reassembly line, not a human-readable duplicate) produces + tens of thousands of lines per checkpoint on a real collection run -- + budget log volume accordingly (this box's `/tmp` is a 30GB tmpfs; + don't let it fill during an unattended run). + +Reassembly across shards: each shard's raw output is greppable for a +`FHRAW`/`FHFRACRAW`/`FHGAPRAW` line format +(`TAG depth move [gap_bucket] fh_count tries`), one line per populated +cell, printed at the end of a `script` run via `_PrintSuiteReport` +(`script.c`). **Periodic checkpointing matters for anything that runs +long enough to risk a timeout**: `ScriptCommand`'s position loop +re-prints the full (cumulative, not delta) report every +`CHECKPOINT_INTERVAL` (currently 5) positions, not just once at the +true end -- found the hard way, the first version of this only printed +once at the end and a shard that missed its `timeout` budget by 0.05 +seconds lost 100% of its data. **When reassembling from checkpointed +output, take the LAST checkpoint block in a shard's log, never sum +every occurrence** -- each checkpoint is a full cumulative snapshot, +not a delta; summing multiple checkpoints double- (or 18x-) counts. + +A second, unrelated gotcha in the checkpoint counter itself: it used to +key off `g_SuiteCounters.uTotal`, which is gated by `WeAreRunningASuite()` +-- true only when the current position has a recorded `solution`/`avoid` +move. Plain `setboard`/`go` position pools (like +`cross_sectional_200k.ep_`, no EPD solution annotations) never +increment `uTotal` at all, so the checkpoint silently never fired on +that file format despite the code looking correct. Fixed with a +separate, unconditionally-incrementing `uPositionsProcessed` counter. +**If a checkpoint/report mechanism silently does nothing on a specific +file format, suspect a hidden dependency on EPD solution metadata.** + +Position source used so far: `tests/cross_sectional_200k.ep_` -- per +`TESTING_PLAN.md`, this pool's actual documented purpose is +`fuzz_test.sh`'s crash-fuzzing corpus, not curated for FH-rate +calibration specifically. No generation script for it exists anywhere +in the repo (checked); its contents look like ordinary, plausible +real-game positions on inspection, but that's "looks fine," not +"verified representative" -- worth keeping in mind, and see "the +tactical-position gap" below for why this specific pool's composition +turned out to matter a lot. + +## The upcoming large sharded run + +Plan: run this collection sharded across many single-threaded +(`--cpus 1` -- `split.c`'s parallel helpers are NOT wired into these +counters, deliberately, so multi-threaded runs would silently under- +count) processes, at real depth (the whole point of collecting deep- +remaining-depth cells is that they're thin unless the root search goes +substantially PAST the depth being measured -- a node with remaining +depth R sits at ply ~(root_depth - R) from the root, and node count +grows with ply-from-root, so `sd 6` can never produce a single +remaining-depth-10 sample; you need `sd 16+` to make that shell +non-trivially populated), for on the order of a week, against a large +position sample. + +**Composition of that sample matters and was the subject of a real, +expensive lesson this session (see "the tactical-position gap" below): +mostly real-game cross-sectional positions, with a MODEST tactical +fraction mixed in -- not a puzzle-heavy pool.** Scott's explicit framing: +"we want an engine that plays chess, not solves tricky puzzles." A +pool dominated by tactical puzzles would teach the table that leftover +moves in general are riskier than they really are (puzzles are +adversarially selected to have a non-obvious best move), producing a +table that under-prunes in the ordinary quiet middlegame positions +that make up most of a real game. Something like a 80/20 or 90/10 +real-game/tactical blend is a reasonable starting point to test, not a +locked-in ratio. + +Practical constraints learned from the smaller collection runs already +done: +- Per-position cost at real depth varies enormously (34s to 300+s + observed at `sd 14-16` on different positions, single-threaded) -- + don't budget wall-clock by assuming uniform per-position cost. + Feeding each shard a generous position POOL (more than it's expected + to finish) and bounding by wall-clock `timeout` rather than position + count means the run naturally uses whatever time is available without + needing to guess position count in advance. +- The `/tmp` tmpfs on this box is shared and has been observed at 99% + full from OTHER processes mid-session (not our own files) -- don't + assume a `du`-based capacity check at the start of a run is still + valid hours later. +- A stray, unrelated process (an interactive `xboard -fcp ./typhoon + -scp crafty` game, apparently left running from earlier work) has + shown up sharing the box's cores during a collection run. Worth a + `ps aux | grep typhoon` sanity check before assuming all available + cores are actually free. + +## What the data is FOR: EFP == LMP, LMR as a lower-risk fallback + +Once gate-1 is passed, the joint table gives a real `P(fail-high)` +estimate. Two consumers, deliberately sequenced -- prune first (since +it was the smaller, more contained change), reduction later: + +**EFP/LMP (unified)**: skip the move outright if predicted FH% is below +some `epsilon_prune`. Irrecoverable, so `epsilon_prune` needs to be very +conservative and, critically, should be picked from a real measured +precision/aggressiveness curve (sweep several candidate thresholds +through the SAME audit mechanism used to validate the table, plot real +FH% and candidate volume against each), not guessed. Runtime-tunable +via `set EFPCandidateThresholdTenths <tenths-of-a-percent>` specifically +so this sweep doesn't require a rebuild per value. + +**LMR (future work, not yet touched this session beyond discussion)**: +the SAME table, but as a reduction-amount driver instead of a binary +skip -- `reduction_plies ~ f(-ln(predicted FH%))`, monotonically more +aggressive the safer the cell looks. The key asymmetry that justifies a +MORE permissive threshold for reduction than for pruning: a reduction +is recoverable in (at least) the direction we can detect -- the +existing "research deeper if history pruning failed" path already +re-verifies a reduced move whose shallow search unexpectedly raises +alpha. What it does NOT catch is the other direction: a reduced search +that fails low, silently accepted, with nobody ever checking whether +the FULL-depth search would have found a raise there instead. Built an +audit for exactly this (`LMR_AUDIT` build flag, its own opt-in profile +since it costs a real extra `Search()` call per audited candidate, not +just a counter increment) -- for a move that got reduced and then +failed low at the reduced depth, re-run the SAME scout window at full +(reduction-undone) depth and check if THAT raises alpha. First real +numbers (small sample): miss rate 4-10% at remaining depth 2, dropping +under 1% at depth 3-4, then creeping back up to 3-7% at depth 5-6 -- a +non-monotonic, "high-low-high" shape that wasn't the expected smooth +decay and hasn't been investigated further yet. + +Both consumers, plus per-move veto signals (below), sit on top of the +same gate-1-gated table lookup -- this is the concrete sense in which +EFP and LMP "are the same thing now." + +## Per-move veto signals: two tried, both hit cliffs, no smooth middle ground + +The idea: once the table flags a move as a pruning candidate, consult a +cheaper-or-not-much-more-expensive per-move signal as a final veto, +since the table only knows the aggregate risk for this (depth, move, +gap) CELL, not anything about this specific move's own tactical merit. +Both signals tried this session are audit-only-first, real-pruning- +second, and both turned out to be effectively binary rather than a +smooth dial: + +1. **`ComputeMoveScore` veto, Heinz-style depth-scaled margin.** + Generalized the old two-tier `VALUE_KNIGHT`(1.75ply)/`VALUE_ROOK` + (2.75ply) activation margin to a linear scale across the whole + depth-1-9 range (`uEFPVetoMarginBase + (depth-1) * + uEFPVetoMarginPerPly`, both runtime-tunable). Swept the margin from + 0 up to 300/200: below ~(200-300, 100-150) nothing gets rescued at + all; at (300, 200) every one of 5 known-loss positions gets rescued, + but node count on `ecm_ringers` comes back to within 0.6% of the + ORIGINAL unpruned baseline -- i.e. that setting is functionally "never + prune," not a real middle ground. **Retired this veto** in favor of + (2) below; not worth resurrecting without a fundamentally different + margin shape (this was linear-in-depth; something with real curvature + might behave differently, untested). + +2. **`GetMoveFailHighPercentage` veto** (per-move-IDENTITY history, + `dynamic.c`'s `g_FailHighs` table, keyed by `mv.uMove & 0xFFFFF` -- + near move-exact, not just piece+destination). Exempt a flagged + candidate from pruning if it has real, evidenced history of failing + high: `(uFHAttempts >= MinSamples) && (uFHPct > Threshold)`. + **Critical, easy-to-miss fact about this table's lifecycle**: every + cell resets to `0x00010001` (1 attempt, 1 fail-high = a 100% FH-rate + PRIOR), not zero, on `ClearDynamicMoveOrdering()` -- which fires on + `setboard`/`new` (`PreGameReset`'s callers), i.e. **once per position + in a suite run, but only once per GAME in real play, never between + moves.** At the most permissive setting tested (`MinSamples=1, + Threshold=0`), the veto protects EVERY move by default until real + accumulated failures (only `_RecordMoveFailure` ever touches these + cells during a normal search; it increments the attempts denominator + without touching the fail-high numerator, diluting the percentage + toward 0 -- reaching an integer-truncated 0% takes on the order of + 100+ observed failures of that exact move-identity within the + current search) prove a specific move safe. This is NOT "smart + triage using real evidence" so much as "protect nearly everything + by default, until proven otherwise" -- confirmed by measuring the + actual prune rate among flagged candidates: **70-80% of everything + the table flags gets vetoed and fully searched anyway**, at every + depth checked. That's a real, quantified NPS cost for zero node + savings on the majority of "candidate" evaluations. + Swept `(MinSamples, Threshold)` away from the extreme (1,0) setting + -- same cliff pattern as (1): every tested combo away from (1,0) + lost 4-5 of 5 known-loss rescues while only partially reducing the + waste rate; none beat "just tighten the table threshold with no veto + at all" (which alone recovers exactly 1 of 5, `ECM.272`, the one + case that was ever a pure threshold-calibration problem rather than + a population/data problem). + +**Open question, not yet resolved**: does this veto's real-game behavior +degrade over the COURSE of a single game, given the table never resets +mid-game? Live self-play match data shows the candidate's average- +search-depth advantage over the unmodified baseline shrinking as the +match's game count grows (within one fixed-configuration match: ++1.09 ply at 30 games, +0.57 at 60, +0.49 at 300) -- consistent with, +though not yet proven to be caused by, staleness accumulating within +individual games as they get longer. Needs a direct test: instrument +depth-gain (or the candidate/actually-pruned ratio) as a function of +MOVE NUMBER WITHIN a game, not just averaged across whole games, using +actual game logs from a running match rather than inferring it from the +aggregate trend. + +## `sd` vs `sn` vs real games -- and getting this backwards cost real time + +**`sd N` (fixed depth) is structurally the WRONG test for anything that +trades tree size for depth.** At equal depth, a leaner-tree change can +only ever cost solves relative to baseline -- it never gets credited +for the depth it's supposed to buy, because the test held depth fixed +instead of letting the efficiency convert into extra plies. This +session ran an expensive round of `sd10` suite comparisons that showed +what looked like a severe regression (14 net lost solutions across two +curated suites) before catching that the test itself was the problem, +not (only) the mechanism: re-run at `sn 5000000` (fixed node budget, +the fair comparison, since it lets the efficiency actually convert into +depth the way it's supposed to), the SAME positions and SAME build +showed net +6 solved across all three curated suites with deeper +average search depth everywhere. **Any test of a pruning/reduction +change needs to specify which of these three it's using and why**: +`sd` for "does this change the tree SHAPE at fixed effort" (the right +question for judging LMR/EFP's aggressiveness in isolation), `sn` for +"does this deliver more accuracy at fixed COST" (the right question +for judging whether a leaner-tree change is worth it), real self-play +games for "does this actually win more, accounting for everything +neither synthetic test can capture" (time-control-driven node budgets +that vary naturally by position, and the compounding effect of +thousands of small decisions across a whole game rather than a handful +of curated positions). + +Related, and worth its own callout: **curated ECM suites have real, +documented noise** (see `tests/ECM.md`'s own "flaky/garbage" catalog -- +oscillating PVs, near-0.00 scores, single-line coin-flips). A ±1-2 +solve swing on a 90-position suite, or any result on the 11-position +`ecm_ringers`, is not strong evidence on its own. Chasing a specific +named position's recovery (the 5-position known-loss set used +throughout this session's veto sweeps) can actively mislead: the +0.5%-threshold configuration that achieved a PERFECT 5/5 recovery on +that named set actually performed WORSE in full-suite aggregate (net ++2) than the 1.0%-threshold configuration it was being compared against +(net +6) -- optimizing against a small, hand-picked set pulled toward a +setting that was worse on the broader, still-imperfect-but-larger +sample. The named positions are useful as a fast smoke test for "did +this obviously break something we already know about," not as an +optimization target. + +## The tactical-position gap: why this matters more than it first looked + +The first live-pruning attempt (table threshold 1.0%, no veto) showed a +real, `ECM.md`-confirmed regression: legitimate tactics lost, including +`ECM.272` -- a position independently, manually re-verified from scratch +this same session (traced the exact sacrifice line, confirmed our own +engine solves it cleanly and stably from depth 7, confirmed by an +independent Crafty search converging to the same move) as a genuine, +non-flaky tactic. Root cause, confirmed by a threshold sweep specific to +the 5 known-loss positions: tightening the table threshold all the way +to 0.1% (10x tighter than the default that failed) rescued exactly 1 of +5 (`ECM.272`); the other 4 stayed wrong regardless. **That's the tell +that it's a population/training-data problem, not a threshold- +calibration problem** -- if tightening 10x doesn't move 4 of 5 cases at +all, the table's own predicted value for whatever cell is involved is +wrong, not just permissively thresholded. `cross_sectional_200k.ep_` +(real games only) apparently underrepresents the "quiet-looking but +critical" move shape tactical positions are specifically curated to +contain, enough that the table confidently mispredicts it. This is the +direct motivation for mixing a modest tactical fraction into the next, +larger collection pass (see above) -- not to make the engine solve +puzzles better, but because the training population needs to actually +contain examples of the shape it's being asked to judge. + +## Status as of this write-up + +- Real pruning is OFF pending resolution of the open questions above + (in-game staleness, and the not-yet-attempted ECM-augmented + retraining). What's live right now (via `EFPCandidateThresholdTenths`/ + `EFPFHVetoMinSamples`/`EFPFHVetoThreshold`, all runtime-settable) is + the most conservative configuration tested (0.1% table threshold, + `(1, 0)` veto), currently running as `candidate` in an ongoing SPRT- + style self-play match against unmodified `head` -- last checkpoint: + 300 games, score 0.473, llr -0.50 (inconclusive, hasn't crossed either + bound; trending toward parity after an unfavorable dip at 60 games + that looks in hindsight like small-sample noise, consistent with the + "ECM suites are noisy, don't over-read small samples" lesson above). +- `FH_TABLE_COLLECTION` build flag exists and works; a real, week-long, + larger, tactically-augmented collection run is the planned next major + step, informed by everything above. +- The `ComputeMoveScore` veto is retired (dead end, see above); the + `GetMoveFailHighPercentage` veto's mid-game staleness question is + open and needs direct instrumentation, not just inference from match + trends. +- LMR (reduction) work has not actually started yet beyond the + `LMR_AUDIT` miss-rate measurement infrastructure and this discussion + of how the risk threshold should differ from pruning's -- next in + line once the pruning side's open questions are resolved. + +--- + +# Update, 2026-09-27/28: table-driven EFP goes fully live, two real +# bugs found and fixed, self-play verdict is "roughly even" + +Continuation of the above, same overall project. This section covers +what changed from "status as of 2026-09-26" (real pruning OFF, +conservative audit-only config) through making it genuinely live, +finding and fixing two real bugs via self-play (not suites), and the +honest current verdict. + +## Old EFP fully replaced, not unioned + +Early in this stretch, discovered the old margin-based EFP block and +the new table-driven block were BOTH live simultaneously (unioned via +a shared `fThisMoveEFPPruned` flag) -- an artifact of iterating on the +veto experiment without reconciling the two blocks. Fixed: old EFP +wrapped in `if (FALSE && ...)`, permanently disabled; the table-driven +mechanism is now tested as a genuine replacement, not an addition. + +Also added, matching old EFP's own behavior that the table-driven +block had been missing: explicit `NODE_PV != pi->eNodeType` exclusion, +and the losing-capture admission switched from a depth-gated local +`iMoveSee` (only computed for `uDepth < THREE_PLY`, silently +defaulting to "non-losing" outside that range) to the cached +`MVF_SEE_KNOWN`/`MVF_SEE_NONNEGATIVE` move-stack flags `generate.c` +already sets unconditionally at generation time for every capture/ +promotion -- correct at every depth, not just near the horizon. + +**Also discovered `ComputeMoveScore` is a no-op for quiet/leftover +moves** under `MOVE_SCORING_NORMAL` (always returns 0) -- meaning old +EFP's real selectivity, historically, came ONLY from its coarse +depth/margin gate plus the killer/FH-history exemptions, never from +any actual per-move tactical read. Worth remembering next time +something is attributed to "EFP's margin test was smart about this +move" -- it wasn't; it was smart about depth alone. + +## Depth-scaled threshold, not a flat one + +A flat FH% threshold applied uniformly across the whole prune-eligible +depth range is structurally wrong: old EFP only ever operated within +~1-2.75 ply of the horizon, where a wrong prune costs little (a small, +bounded subtree). This mechanism's point is to extend pruning deeper, +where a wrong prune is far more expensive (an exponentially larger +subtree). `_EFPThresholdForDepth()` (search.c) interpolates +GEOMETRICALLY (not linearly -- the two endpoints span orders of +magnitude) between `uEFPThresholdShallowHundredths` (depth 1, +permissive -- old EFP's own real gate tolerated FH% up to the tens of +percent at 1-2 ply and still won) and `uEFPThresholdDeepHundredths` +(at `uEFPMaxPruneDepth`, near the table's measurement floor). + +## `MoveLooksDumb`: from boolean, to magnitude bucket, to the joint table + +A new 4th signal, alongside depth/move/gap: does this quiet move +statically look like it hangs material? Went through several rounds: + +1. **First attempt**: three OR'd sub-checks (a "reversal" move, a + pawn-attack, an ignored-hanging/trapped-piece read). Isolation + testing found only the pawn-attack idea behaves as intuition + predicts. The other two were retired for real, understood reasons, + not just "didn't help": the reversal check had a genuine indexing + bug (compared against the WRONG ply's move, `uPly-2` instead of + `uPly-3`) that masked a deeper conceptual problem once fixed -- + reversal moves measure BACKWARDS (higher FH%, not lower), because a + reversal responds to the opponent's own intervening move and is + often a genuine correct reaction, not a wasted tempo. The + hanging/trapped-piece check also measured backwards: `cEnprise`/ + `cTrapped` are written by the same parent `Eval()` call that + produces `iEval` (the gap axis's own input) -- a real confound with + gap, not independent signal. +2. **Second attempt** (kept, still live): "does this move hand + material away for no reason" -- unconditionally for a strictly + cheaper attacker (pawn attacks any non-pawn; minor attacks a rook; + minor or rook attacks a queen), and conditionally for a + queen-vs-queen "undefended" case discussed and then REMOVED below. + Tried and retired: an equal-value-attacker-AND-zero-defenders test + for minors/rooks -- structurally broken, because + `pos->bbMinorAttacks[uMover]`/`bbRookAttacks[uMover]` (computed + PRE-move) already includes the mover's own pre-move reach to its + destination square (that's what makes the path legal), so "zero + defenders" was almost never true with multiple minors/rooks per + side and no cheap way to tell "the piece that just left" from "a + different piece of the same type still defending." The QUEEN case + sidesteps this because there's normally exactly one queen per side + (gated on `FastCountBits(bbPieces[uMover][QUEEN]) == 1`). +3. **Boolean -> magnitude bucket**: "dropping a bishop to a pawn" and + "dropping a queen to a pawn" were both just `TRUE` before, throwing + away real information (a ~28,000x FH% swing across gap buckets + alone says magnitude matters). `MoveLooksDumbLossCp()` now returns + an approximate centipawn loss; `MoveLooksDumbLossBucket()` + coarsens it into `DUMB_LOSS_BUCKETS` (chess.h, =6) piece-shaped + tiers. +4. **A real UnmakeMove-ordering bug**, caught before it shipped: the + FH-side counter increment originally called `MoveLooksDumbLossCp` + a SECOND time, after `UnmakeMove` had already restored the board + and flipped side-to-move -- silently classifying a different + position. Produced an impossible >100%-FH cell in testing. Fixed by + computing the dumb bucket ONCE (before `UnmakeMove`) and reusing + the stored value on both the tries-side and FH-side counters. +5. **Joint (4D) table**: built `(depth, move, gap, dumb)` jointly, not + just as two separate marginal cuts, to test whether "dumb" + compounds with "gap" rather than being redundant. Confirmed real + compounding (e.g. depth 3, gap bucket ~4: dumb=0 -> 0.948% FH, + dumb=4 (post-coarsening numbering) -> 0.008% FH, a ~120x reduction + at 3.2M samples) -- this justified keeping dumb as a real 4th axis + instead of folding it into gap. + +### Queen-vs-queen removed outright (2026-09-27, this session's tail) + +The queen-vs-queen-undefended sub-check (originally bucket 5, ~975cp +loss) was the one case that never behaved like the rest. Direct +analysis of the 45-minute joint-run data: marginal FH% by dumb bucket +was 0.057%/0.072%/0.098%/0.010% for buckets 1-4, but **0.457% for +bucket 5 -- higher than "not dumb" (0.335%) baseline**, and this +persisted (0.0147% vs. 0.0195% baseline, still elevated relative to +bucket 4's 0.0045%) even after controlling for the gap axis. Root +cause suspected but NOT confirmed: `_WhoAttacksSquareBB` (see.c) has +no pin/legality awareness, so a pinned enemy queen still "shows" as +attacking. Rather than ship a signal nobody understands, **the +queen-vs-queen check was deleted from `MoveLooksDumbLossCp` entirely** +(search.c), not just excluded downstream -- "losing a rook to a pawn +is never a good idea" is a claim we can defend; "queen attacked by +queen looks safe to prune" turned out not to be, for reasons still +unexplained. `MoveLooksDumbLossBucket()` still has 5 tiers (0=not +dumb, 1-4 = increasing loss magnitude: ~200/~400-475/~675/~875cp); +bucket 5 (975cp) is simply unreachable now. `DUMB_LOSS_BUCKETS` stays +at 6 in chess.h so the baked tables (sized `[...][6]`) don't need +reshaping -- the top slot is just always zero. + +**A bug caught during this edit, worth remembering**: the first pass +at simplifying `MoveLooksDumbLossBucket()` collapsed the 675cp and +875cp tiers into one bucket, based on an incorrect assumption that +removing the queen-vs-queen check also removed the 875cp value. It +didn't -- 875cp (a queen attacked by a pawn) is still produced by the +generic, unconditional pawn-attack branch at the top of the function, +which was never touched. Caught by rereading the actual reachable +value set before trusting the "simplification." + +## Real pruning: Tier 1 / Tier 2, then unified into one mechanism + +Two designs were tried, in order: + +**Tier 1 + Tier 2 (retired)**: Tier 1 = percentage-curve pruning, +capped at `uEFPMaxPruneDepth`. Tier 2 = `JointMoveNeverFailsHigh()` +(data.c) -- prune unconditionally at ANY remaining depth when the 4D +joint table shows a confirmed zero fail-high over a large-enough +sample (rule-of-three statistical confidence), EXCLUDING dumb buckets +0 (not dumb -- this table says nothing about ordinary moves) and 5 +(queen-vs-queen, even before it was removed from the signal entirely). +Audited before trusting: found real misses at depth 3-4 even at +`MinSamples=10000` -- one exact miss traced to a genuine, deliberate +queen sacrifice (`4rrk1/6p1/p7/1p1Np3/1bq1P3/7Q/PPP5/1K1R2R1 w - - 3 0`, +`Qh6`) near the enemy king, not a bug. Added `Tier2MinDepth=5` after +confirming clean audits at depth >=5 across all three curated suites. + +**Unified (current, live)**: per explicit direction ("I think we +should not have tier1 and tier2 ... prune everywhere again but use a +graduated threshold" -- then, when the deep end of the percentage +curve flatlined at its representable floor, "I don't want 0.01%. I +want 0 FHs ever seen with >5k positions, >10k positions, etc"): + +``` +uCandDepth = uDepth / ONE_PLY +if (uCandDepth < g_Options.uEFPMaxPruneDepth): + prune if PredictedFHPercentHundredths(...) < _EFPThresholdForDepth(uCandDepth) +else: + prune if ConfirmedZeroFailHigh(uCandDepth, uMoveNum, gap, dumbBucket) +``` + +`ConfirmedZeroFailHigh()` (data.c) requires a literal zero observed +fail-high over a sample count that DOUBLES each ply past +`uEFPMaxPruneDepth` (base `g_Options.uZeroConfBaseSamples`, default +5000 at the boundary depth, 10000 the next ply, 20000 the next, ...). +**As of 2026-09-27 this reads the same 4D joint table Tier 2 used** +(`g_uJointTries`/`g_uJointFH`), not a separate gap-only marginal -- +see "the self-play regression and its real cause" below for why that +change was necessary. + +## The self-play regression and its real cause + +First live self-play run of the unified design (30 games): **-8=21+1, +score=0.383, llr=-0.42** trending firmly toward reject, with `ebf` +WORSE for candidate (3.590 vs head 3.522) and candidate searching +SHALLOWER (12.87 vs 12.92 avg depth) at similar nps -- exactly +backwards from what a working pruning mechanism should show. This was +"clearly a bug, not just an aggressive tradeoff" territory, matching +Scott's own read: "I think there's a bug in the pruning impl." + +An advisor consultation (mid-investigation) proposed a structural +theory -- pruning removes move-ordering/TT information that's +load-bearing elsewhere in the tree, and this cost scales with how deep +the prune fires, which old EFP never paid because it stayed within +1-2.75 ply of the horizon. That's a real, general risk (see "on +reduction vs. pruning" below), but a direct code audit found something +more concrete and immediately actionable: **`ConfirmedZeroFailHigh` +was reading `g_uGapRawTries`/`g_uGapRawFH` -- the (depth, move, gap) +marginal SUMMED OVER THE DUMB AXIS.** A cell could read "zero FH in +5000+ tries" purely because ordinary, non-dumb moves dominated that +cell's volume, while a rare queen-hanging move sharing the exact same +(depth, move, gap) cell had never actually been safely tested on its +own. Fixed by switching `ConfirmedZeroFailHigh` onto the 4D joint +table (adding a `uDumbBucket` parameter, same table +`JointMoveNeverFailsHigh` already used), refusing to trust a zero read +for the queen-vs-queen bucket outright (matching Tier 2's own +caution, at the time this bucket still existed). + +**Result, same match restarted after the fix (60 games total, 30 new +under the fixed binary)**: ebf flipped from "candidate worse" to +EXACTLY TIED (3.619 = 3.619), and average depth flipped from +"candidate shallower" to "candidate deeper" (13.05 vs 12.81) -- the +signature of a real fix, not noise. `g_uGapRawTries`/`FH` and the +`GAPRAW_*` macros are now fully dead code, left in place (unused) at +the time of this write-up but candidates for deletion. + +**The `g_uGapRawTries`/`g_uGapRawFH` tables were later deleted +entirely** (2026-09-27, same session) once the gap-bucket rework below +made keeping them around pointless. + +## Quantifying "how much statistical confidence is actually needed" + +A real question raised directly: given a per-cell "confirmed zero" +read only bounds the true rate via rule-of-three (~3/N for N samples), +and this gate fires many times per real game, how much does that +matter? Worked from real counters, not guesses: + +- Real `sd10` counters (post-fix): ~303 actual deep-zone (depth >=6) + prunes per 31.3M nodes searched, i.e. ~1 per 103,000 nodes. +- Scaled to a real blitz budget (1.5M nps x 350s ~= 500M nodes/game): + **~4,800 deep-zone prunes per game.** +- Rule-of-three bound at the current sample bar (5000 at depth 6, + doubling deeper): expected WORST-CASE misses per game ~= 2 (not + thousands -- an earlier back-of-envelope estimate using a wrong, + much larger node-count assumption was corrected in-session). +- To hit a "1 unlucky game in 1000" target at that per-game + application volume: `required_N ~= 3 * applications_per_game / + target_misses_per_game ~= 3 * 4800 / 0.001 ~= 14.4 million` samples + in the specific cell being trusted, not 5,000-40,000. +- **Brute-force collection to reach that is not viable**: the + 45-minute run put ~5,000 samples into a representative deep cell: + getting to 15M is ~3000x more volume, i.e. ~94 days wall-clock even + running all 16 shards this box supports in parallel (already at the + ceiling -- more shards doesn't help past core count). Confirmed and + accepted as a dead end in-session. + +**2026-10-01 correction: the "1 unlucky game in 1000" target itself was +never validated, and the whole framing here was too conservative.** +Scott's direct pushback: nobody decided 1/1000 was the right number -- +it was picked, not derived -- and 1/100 or even 1/20 is reasonable. Two +things compound this: + +1. **Not every prune mistake costs the game.** A silent miss (we prune + a line that would have fail-highed) only matters if that line + actually would have changed the game's outcome -- a strict subset of + "a miss occurred." The rule-of-three math above bounds the rate of + MISSES, not the rate of GAMES ACTUALLY LOST to a miss, so even the + loosened budgets below are still conservative relative to the thing + that actually matters (Elo). +2. **The deeper problem: this section was optimizing the wrong + quantity.** It treats "minimize miss probability until some arbitrary + confidence bar" as the goal in itself. The actual goal is net playing + strength: aggregate nodes saved by pruning/reducing get reinvested as + extra search depth elsewhere, and depth finds tactics -- that's a real + benefit that has to be weighed against the real (but apparently + small, see "the self-play regression" and the LMR self-play match + above) cost of occasional misses. A `g_uJointFHThousandths`-driven + gate exists precisely so this tradeoff can be made *intelligently* + (deeper/leaner where the data says it's safe-ish) rather than via a + single global, hand-picked confidence bar -- but picking that bar by + "make misses astronomically rare" short-circuits the whole exercise + and defaults back to maximal conservatism regardless of what the + table actually knows. The open comparison against Crafty (much + leaner trees, routinely 7-8 plies deeper at equal node budgets) is + the concrete signal that this project has been on the over-cautious + side of that tradeoff. + +Recomputed the same formula at looser budgets, same 4,800 +applications/game assumption: + +``` +1/1000 games -> N ~= 14.4M (0 deep-zone zero-FH cells clear this, as of this run) +1/100 games -> N ~= 1.44M (0 deep-zone zero-FH cells clear this -- largest is 1.34M, close) +1/20 games -> N ~= 288K (16 deep-zone zero-FH cells clear this TODAY, no more collection needed) +``` + +**Decided direction, not yet executed**: stop treating the +rule-of-three sample bar as the primary lever at all. Keep it as a +floor against outright garbage (near-zero-sample cells), but the real +experiment going forward is to try materially more aggressive +settings on both mechanisms -- the 1/20-budget prune bar (288K, usable +immediately) and reduce tiers picked to target real extra depth (aim +for whole plies, not the +0.3 ply the first p75/p25/p5 self-play run +produced) -- and read the result by playing strength and search depth +in self-play, not by how rare a miss is in isolation. The right +long-term framing: **the actual metric to tune against is net Elo +(or a cheap proxy -- depth/nps/ebf movement under a fixed node budget), +with the FH% table used to bias WHERE cuts happen, not to justify an +arbitrarily tiny global miss rate.** +- **What DOES help, cheaply**: coarsening the bucket grid concentrates + the same collected volume into fewer, denser cells -- see next + section. This buys real headroom without any extra collection time. + +## Open gap, 2026-10-01: compounding errors under reinvested depth is unmeasured and unmeasurable by anything built so far + +Every statistic gathered for this project so far -- raw FH%, the +`JOINT_EXTRA_VERIFY` silent-miss rate, the rule-of-three sample bound +above -- is a SINGLE-DECISION statistic: it characterizes the chance +that one gate, at one node, misjudges. None of it says anything about +what happens once that misjudgment is embedded in a continuation that +is then searched DEEPER, specifically because the nodes saved by +pruning/reducing elsewhere get reinvested as exactly that extra depth. + +Scott's framing: we have no information about what happens if we prune +wrong ~5x/game and reduce wrong such that a reduced-depth search +fails low where a full-depth search would have failed high, AND in +doing so we bought ourselves ~2 extra plies per move on average -- +because searching those 2 extra plies means the SAME gate fires again, +2 more times, deeper under the same kind of position, with more real +tree beneath each repetition. + +**Why this could be qualitatively worse than the per-decision rates +suggest, not just "the same risk, more often":** if the 4-axis model's +misjudgments were i.i.d. noise, compounding wouldn't be very dangerous +-- a short chain of independently low-probability errors is still +low-probability, same as any noise process. But this project has +already found the model has CORRELATED, structured blind spots, not +just noise: the queen-vs-queen dumb-bucket (removed outright, see +above -- "safe" reads that weren't, for reasons still not fully +understood) and the null-move-bailout signature bug (silent-miss +investigation, see above) were both systematic, not random. Reinvesting +saved nodes as depth specifically UNDERNEATH the branches just judged +"safe" means that if a blind spot like those exists, extra depth +doesn't dilute it with more search -- it drills further into exactly +the part of the tree most likely to share it. + +**This cannot be measured by instrumentation built so far, even in +principle.** `JOINT_EXTRA_VERIFY`'s silent-miss rate and the rule-of- +three bound both describe P(one gate misfires once); there is no way +to decompose an aggregate self-play score back into "cost from +compounding errors under reinvested depth" vs. "benefit from genuinely +sound extra depth" -- they are inseparably confounded in that single +number by construction. Per-node instrumentation was always going to +be structurally blind to this, no matter how much more data is +collected at the per-cell level. + +**Practical implication for the next (more aggressive) self-play run**: +don't just read the final SPRT score. Pull the actual lost/drawn games +under the aggressive settings and check whether losses cluster around +a recognizable, repeated position type (a live blind spot, same shape +as queen-vs-queen) versus being spread uniformly (ordinary noise at a +higher-variance operating point). `SILENT_MISS_TRACE`'s existing +FEN/move/score logging is most of the needed infrastructure already -- +it would need to stay attached through to which game a flagged miss +occurred in (not just logged as an isolated per-node event) to make +this check possible. Not yet built; flagging as the concrete next +instrumentation gap before trusting a bolder self-play result as +"just noise" either way. + +## Gap-bucket redesign: 32 uniform bins -> 6 piece-value-aligned bins + +Per direct instruction ("Why do we have 32 gap buckets?! I think they +should be like: alpha-100, alpha-300, alpha-500, alpha-900, +alpha-1500, worse than that"), `FH_GAP_BUCKETS` (chess.h) went from +32 uniform 50cp-wide bins (`FH_GAP_BUCKET_MIN=-200`, +`FH_GAP_BUCKET_WIDTH=50`) down to 6: + +``` +bucket 0 : gap < 100 +bucket 1 : 100 <= gap < 300 +bucket 2 : 300 <= gap < 500 +bucket 3 : 500 <= gap < 900 +bucket 4 : 900 <= gap < 1500 +bucket 5 : gap >= 1500 +``` + +Rationale: the fine 50cp resolution was mostly wasted precision in the +near-alpha region (highest FH%, 9%+ -- never going to pass any sane +pruning threshold regardless of exactly which of ~6 fine bins it lands +in) and starved precision in the tail, which is exactly where a +pruning DECISION actually lives. A 32->6 collapse is a ~5.3x reduction +in total cell count, i.e. a ~5.3x increase in effective samples per +cell for the SAME collection run, for free. + +**`GapCpToBucket(iGapCp)` (data.c) is now the single source of truth** +for this mapping -- previously the -200/50 arithmetic was duplicated +inline at SIX separate call sites (two lookup functions, one +collection increment... actually three lookup functions and three +collection sites) with no shared helper, exactly the kind of +duplication that drifts silently. Every lookup +(`PredictedFHPercentHundredths`, `JointMoveNeverFailsHigh`, +`ConfirmedZeroFailHigh`) and every collection increment (search.c, two +sites) now calls this one function. + +**Both baked tables were rebuilt from scratch** using this new +bucketing, sourced from the SAME 45-minute joint-run pickle +(`reassembled_joint.pkl`'s `gap` marginal for the percentage table, +`joint` dict for the 4D table) -- this also fixes a real, previously +flagged inconsistency: the percentage table used to be baked from an +EARLIER, smaller, differently-composed collection run +(`reassembled_gate1.pkl`) than the joint table, meaning the two halves +of the "one seamless mechanism" disagreed about what population they +described. Now both come from one dataset. + +**Verified after rebake** (`sd10`, all three curated suites): solve +counts essentially unchanged (one single-position flip on +`ecm_confident_quick`, within normal suite noise), and candidate +volume at depth 6-9 on that suite jumped from ~132/85/50/1 tries +(before coarsening) to **4,177/4,603/3,150/170** (after) -- a ~30-60x +increase in samples per surviving cell, exactly the intended effect, +with real observed FH still 0.000% at every one of those depths. + +## Self-play verdict as of this write-up (2026-09-28) + +A fresh match (post queen-vs-queen removal + gap-bucket rebake) +against `head_reference`, tracked over time: + +| games | score | llr | ebf (cand vs head) | avg depth (cand vs head) | +|---|---|---|---|---| +| 100 | 0.520 | +0.10 | 3.641 < 3.654 (better) | 13.21 > 12.93 (better) | +| 170 | 0.521 | +0.16 | 3.679 < 3.685 (better) | 12.94 > 12.69 (better) | +| 1000 | **0.479** | **-1.34** | 3.617 > 3.600 (worse) | 13.19 > 13.05 (still better) | + +**Honest read: the early (100-170 game) positive trend did not hold +at 1000 games.** llr swung from +0.16 to -1.34 -- more than a third of +the way to the H0=-2.94 reject boundary. Average search depth still +favors candidate at every checkpoint (the mechanism genuinely is +reaching deeper), but that extra depth is not converting into a score +edge, and ebf has flipped to favor head at the larger sample. This is +roughly consistent with nearly every measurement across this whole +project once the concrete bugs (EFP/table union, dumb-axis pooling, +queen-vs-queen) were fixed: **"even," not "clearly ahead."** + +This matches the advisor's structural argument from the mid-session +consultation: pruning at depth ranges beyond old EFP's narrow 1-2.75 +ply removes move-ordering/TT information that's load-bearing +elsewhere in the tree, and that cost doesn't fully disappear just +because the individual prune decisions are well-calibrated. Fixing +concrete bugs recovers lost ground (30-game catastrophe -> 1000-game +parity) but does not, on its own, produce a clear win. + +**Decision point reached, not yet resolved**: either (a) let the +current match run to a real statistical conclusion (llr crossing +either bound) before drawing further conclusions from noisy +checkpoints, or (b) accept "even" as pruning's ceiling at this depth +range and pivot to reduction (see next section) instead of continuing +to chase a clear win from harder pruning alone. + +## On reduction vs. pruning, as the likely next step + +Not yet implemented. The core argument for why reduction should not +need anywhere near the ~15M-samples-per-cell bar pruning would need at +a permissive threshold: **a reduction is self-correcting when wrong.** +The standard PVS/LMR re-search machinery already re-verifies a reduced +move whose shallow search unexpectedly raises alpha or fails high. A +wrong prune is permanently lost; a wrong reduction costs one extra +re-search. That changes the risk calculus completely and was always +the second half of the original plan ("beat EFP at its own game and +then use this table to layer on reduction with a less conservative +threshold"). + +Sketch discussed, not yet built: stack an additive `extra_reduction_ +plies` term onto whatever `GetLMRReduction()` already computes for a +gate-1-eligible move, driven by `PredictedFHPercentHundredths`: + +``` +FH% >= 5% : +0 (no extra reduction) +1% <= FH% < 5% : +1 ply +0.1% <= FH% < 1% : +2 plies +0.01% <= FH% < 0.1% : +3 plies +confirmed zero, or + FH% < 0.01% : +4 plies +``` + +Clamped so total depth never goes below some floor, same clamping +discipline `ComputeMoveExtension`/`GetLMRReduction` already use. +Reuse the SAME gate-1-eligible population and the SAME table this +whole document is about -- no new data collection needed to start +this, only new search.c wiring at the LMR call site. + +## How to regenerate the tables (reproducibility notes) + +Both baked tables (`g_uFHPercentHundredths` and `g_uJointTries`/ +`g_uJointFH`, all in data.c) are derived from `.pkl` files produced by +reassembling a sharded collection run's raw `FHGAPRAW`/`FHJOINTRAW` +output lines (see "how the data is collected" above for the format). +**These `.pkl` files and the reassembly/codegen scripts live under +`/tmp/typhoon/` (scratch, per this repo's `CLAUDE.md` policy) and are +NOT checked into git** -- if they're gone, the tables can only be +regenerated by re-running collection from scratch. As of this +write-up they still exist at `/tmp/typhoon/fh_joint_run/ +reassembled_joint.pkl` (the current, live source for BOTH tables). + +To regenerate from an existing reassembled pickle after a bucket/table +shape change (what was actually done for the 2026-09-27 gap-bucket +rework): + +1. Load the pickle (`pickle.load`) -- top-level dict with `joint` + (key: `(depth, move_idx[0..15], old_gap[0..31], dumb[0..5])` -> + `[fh_count, tries]`, note **move_idx here is already 0-based, + pre-capped at collection time** -- do NOT subtract 1 again) and + `gap` (key: `(depth, move_raw[1..63], old_gap[0..31])` -> + `[fh_count, tries]`, note **this one IS 1-based raw move number**, + a genuinely different convention between the two dicts that cost a + real debugging cycle to catch -- verify which convention a given + dict uses before writing a regeneration script, don't assume they + match). +2. Re-bucket the gap axis using whatever the CURRENT `GapCpToBucket` + boundaries are (old bucket `g`'s underlying range is + `[-200+50g, -200+50(g+1))`; map each old bucket to whichever new + bucket contains that range -- boundaries were deliberately chosen + as multiples of the old 50cp bin width so this mapping is exact + except for the old table's final overflow bin, which has no + defined upper edge and needs a documented judgment call). +3. Sum `[fh, tries]` for every old cell that maps to the same new + `(depth, move, new_gap, dumb)` key. +4. Emit C initializer text matching the existing array declarations + exactly (dimension order, `// move N` / `// depth N` comments for + readability), then splice it into data.c replacing the old array + body between its declaration line and matching closing `};` (brace- + depth-tracked, not a fixed line-count assumption -- table bodies + are tens of thousands of lines and shift size on every rebuild). +5. Rebuild (`gmake clean && gmake -j5 GENETIC=1 PERF_COUNTERS=1 MP=1 + SIXTYFOUR=1`) and re-run the `sd10` three-suite sanity check before + trusting the result -- solve counts should be stable (+/-1 on + `ecm_confident_quick` is normal suite noise) and the EFP candidate + audit block's real-FH% should still read ~0.000% at depth >= + `uEFPMaxPruneDepth`. + +To regenerate from a FRESH collection run instead of an existing +pickle: re-run the same sharded, `--cpus 1`, `FH_TABLE_COLLECTION`- +gated collection process described earlier in this document (16 +shards, wall-clock-bounded via `timeout`, mixed real-game/tactical +position pool), reassemble each shard's LAST checkpoint block's +`FHGAPRAW`/`FHJOINTRAW` lines into the same pickle shape, then follow +steps 2-5 above. + +--- + +# Update, 2026-09-28/29: phase 2 (graduated LMR), a real measurement +# bug found and fixed, EFP's per-move check confirmed dead, and the +# "deep prune" idea + +Picks up where the previous section left off (reduction as the +likely next step after pruning's self-play verdict landed at "even"). +This is a companion, largely independent table build -- a fresh 4-axis +`(depth, dumb, gap, late)` joint table baked from a new 60-90 minute +16-shard `sd 16` collection run (see +`lmr_testing/2026-09-28_four_axis_joint_table_plan.md` for the full +collection/bake recipe), `g_uJointFHThousandths[12][11][11][15]` in +`data.c`, thousandths-of-a-percent resolution, `0xFFFF` sentinel below +a 500-tries-per-cell floor. Same underlying idea as the EFP work +above, applied to reduction instead of pruning, and building its own +independent measurement infrastructure along the way. + +## The mechanism: strictly additive over `GetLMRReduction` + +Deliberately the smallest first experiment: HEAD's existing flat +`-ONE_PLY` reduction (`GetLMRReduction`, searchsup.c) is left +completely unchanged and still decides whether a move gets reduced at +all. The table is only ever consulted to add MORE plies on top of a +move already reduced -- never less, never on a move `GetLMRReduction` +wouldn't have touched anyway. `EFP is left untouched too (including +its known bug, see below) -- this round is measuring the LMR +augmentation in isolation, on purpose. + +Tier thresholds (tries-weighted percentiles of the table's predicted- +FH% distribution, computed against the ACTUAL LMR-eligible population +-- `uLegalMoves > 5`, real `uDepth > TWO_PLY`, dumb bucket 1 excluded +as ~97% SEE-scored losing captures/underpromotions misfiled onto the +quiet-move axis, same lesson as the EFP work's dumb-bucket-1 exclusion +above): + +``` +FH% >= p75 (0.072%) : +0 (no extra reduction, baseline -1 ply only) +FH% < p75 : +1 ply +FH% < p25 (0.005%) : +2 plies +FH% < p5 (0.001%) : +3 plies (max) +``` + +`JOINT_EXTRA_MIN_REMAINING_DEPTH` floor: currently `ONE_PLY` (the +original, most aggressive setting) after a round-trip described below +-- briefly raised to `FOUR_PLY` out of caution, then reverted once the +apparent danger was shown to be a measurement artifact, not a real +safety problem. + +Main move loop (`search.c`) only, same as the EFP work -- not wired +into `split.c`'s `HelpSearch`, for the same reason (per-node snapshot +arrays with no path into a helper thread's own context; every +comparison protocol in this project already runs `--cpus 1`, so this +costs nothing in what can currently be validated). + +## The null-move-bailout measurement bug (the big one this round) + +Built `JOINT_EXTRA_VERIFY` (shadow-search verification: after a real +reduced search fails to raise beta, re-run at +`uNextDepth + tier*ONE_PLY` and check if THAT would have failed high -- +same idea as the EFP-side "audit before trusting" discipline +throughout this whole project) and `SILENT_MISS_TRACE` (log FEN/ +scores/moves on every detected miss). First numbers were alarming: up +to 1-11% silent-miss rates across various configurations -- a +"reduced search fails low, hiding a fail-high the real search would +have found" rate that would have meant this mechanism was unsafe. + +**Root cause, found via live gdb stepping through an actual flagged +case, not guessed**: `TryNullmovePruning`'s (searchsup.c) intentional +Tord Romstad idea -- + +```c +// This is an idea from Tord Romstad on ccc: if we are below a +// history reduced node and we try a nullmove and the nullmove +// fails low and the move that refuted the nullmove involves +// the same piece moved in the reduced move at ply-1 then bail +// out of the reduced depth search now and research at full +// depth. ... +if ((mv.cFrom == (pi-1)->mv.cTo) && ((pi-1)->iExtensionAmount < 0)) +{ + ... + *piNullScore = iAlpha - 1; + return TRUE; +} +``` + +deliberately FABRICATES a score of `iAlpha - 1` and returns `TRUE` +instead of doing a real re-search, specifically because that +fabricated value negates up to exactly `iBeta+1` one level up -- +correctly triggering the REAL search's own existing +re-search-on-fail-high machinery. This is correct, intentional, and +safe in the real search path. It is NOT safe inside our bespoke, +one-off shadow-verification `Search()` call, which has no surrounding +wrapper to catch that "please re-search me" signal -- our own +verification code misread the fabricated fail-high as a genuine one. + +**Fix**: `NEUTRALIZE_NULL_BAILOUT` (GNUmakefile flag) -- temporarily +zero `pi->iExtensionAmount` (save/restore) around just our own shadow +`Search()` call, disabling only this one precondition for our own +verification, leaving every other occurrence of the real mechanism +(anywhere else in the tree, on any other move) completely unaffected. + +**Result**: true silent-miss rate, measured correctly, is ~0.00056% +(~1 in 178,000) even at the most aggressive `ONE_PLY` floor -- the +80-99.8% of "misses" the earlier numbers showed were this measurement +artifact, not real danger. This is why the floor was reverted from +`FOUR_PLY` back to `ONE_PLY`: the conservative retreat had been a +reaction to contaminated data. + +**Lesson for next time a shadow/audit `Search()` call is built +anywhere in this codebase**: any node-local mechanism that +communicates results by fabricating a boundary score and relying on a +CALLER a level up to interpret it correctly (this null-move bailout, +and structurally anything similar) is a landmine for exactly this +kind of one-off internal verification call. Check for this class of +mechanism specifically before trusting a shadow-search-based audit's +raw miss rate. + +## Genuine (post-fix) misses: two distinct classes, neither cheaply catchable + +Investigated the ~handful of genuine misses surviving the fix by hand, +one at a time, using a fresh, independent `sd1` vs `sd8`-through-`sd12` +comparison per position (NOT the `SILENT_MISS_TRACE` hash-chain +`shadowRefutation` guess, which was separately confirmed unreliable -- +`GetPonderMove`'s hash-chain walk can read stale/unrelated entries +since the hash table is global, not scoped to the specific +verification call; one hand-traced "refutation" turned out to be a +completely different line than what an independent `sd12` search +actually found). Two real, distinct patterns found: + +1. **Narrow-window boundary noise, not a real miss.** One traced + example showed `realScore`/`shadowScore` swinging wildly + (`-500` to `+144`) at the trace's logged `alpha=143 beta=144` -- + but independently re-searching the SAME position freely (`sd1` + through `sd12`, no artificial window) showed a stable, consistent + assessment the whole way (-3.5 to -3.75), no swing at all. The + trace's narrow inherited PVS window means `shadowScore >= beta` + only proves "clears this specific tight ancestor-relative + threshold," not "the position is actually good" -- a real but + mundane form of alpha-beta boundary noise, not a blind spot a + heuristic could detect or avoid. +2. **Genuine horizon-effect quiet tactics.** A second traced example + (`3r1rk1/1pq1bppp/p7/n2R4/P1N3Q1/1P6/1B3PPP/R5K1 b - - 3 0`, after + White's `Qd4`) was real: `sd1` likes `Qd4` (+2.06), but `sd8` + through `sd12` all agree it loses to `Bb6` (a quiet, non-capturing + deflection attacking the queen -- `Qxb6 Qxd7` nets Black the + exchange). `Bb6` isn't a capture or check, so it's entirely outside + `sd1`'s horizon; this is inherent to any bounded qsearch + (captures/checks only), not a specific bug. + +**Tried and abandoned, per direct experimentation with +`QSearchFromCheckNoStandPat`'s (search.c) check-extension gate**: +loosened the `+2 uQsearchCheckExtension` conditions three times in a +row (narrowed the inner OR-conditions, then removed the +`CountKingSafetyDefects(pos, ...) > 4` outer gate entirely), rebuilding +and re-running the SAME automated `sd1`/`sd8` batch comparison +(224 unique genuine-miss positions) after each change. **All three +variants showed no measurable difference from baseline** (median +swing 1.35, mean ~2.71-2.73, same top-10 largest-swing list/PVs, ~20.5% +sd1==sd8 move-agreement rate, every time) -- this specific lever +doesn't touch the class of miss the genuine examples actually show +(the `Bb6` example isn't a check, so a check-generation-budget change +was never going to catch it). Reverted to the original condition set. +Conclusion, by mutual agreement: this is a quirk inherent to low-depth +search, not a systematic, cheaply-fixable blind spot -- punt on +chasing it further. + +## EFP's `ComputeMoveScore` per-move check confirmed dead, quantitatively + +Separately (but directly relevant to "should the table eventually +replace EFP"), traced `ComputeMoveScore`/`DecodeMoveValueEstimate` +(searchsup.c) end to end and confirmed by code reading (not just +inference) that EFP's per-move sanity check -- + +```c +(ComputeMoveScore(ctx, mv, (x - 1)) < uFutilityMargin) +``` + +-- is a tautology given the gates already passed at that point in the +`if`: every quiet (non-capture/promotion) move decodes to exactly `0` +under `MOVE_SCORING_NORMAL`/`MOVE_SCORING_QSEARCH_*`, and +`IS_CAPTURE_OR_PROMOTION(mv)` is already excluded earlier in the same +condition -- so this check can never evaluate to anything but `0 < +uFutilityMargin`, which is already guaranteed true by the +`(uFutilityMargin)` gate a few lines up. It contributes nothing; EFP's +real (and only) per-move selectivity has always come from the +killer-adjacency and `GetMoveFailHighPercentage` history exemptions +below it, confirming and extending the earlier (2026-09-27) finding in +this same doc that "EFP's margin test was never smart about the +specific move." + +## Quantifying what EFP is blind to, using the new table + +Queried `g_uJointFHThousandths` directly over EFP's actual typical +operating envelope (1-2 plies remaining, gap 300-500cp below alpha -- +the `VALUE_KNIGHT`-to-`VALUE_ROOK` margin -- leftover moves, +move-number bucket ~6-30+): + +- Overall range across that envelope: ~0.000% to ~0.37%, median + ~0.009% (p10-p90: 0.001%-0.046%). +- **The `dumb` axis -- EFP is completely blind to this -- swings FH% + 15-30x within an otherwise-fixed cell** (e.g. depth=1, gap=4, + late=8: `dumb=0` -> 0.032%, `dumb=5` -> 0.002%, `dumb=9` -> 0.001%). +- **The `gap` axis -- EFP's only real lever -- only swings FH% ~4x** + across its ENTIRE knight-to-rook operating range (gap=3 -> 0.068%, + gap=5 -> 0.017%). +- The `late` axis (partially, structurally captured by EFP's + `uLegalMoves > 1` gate, but not gradedly) also swings ~4-5x + (late=5 -> 0.041%, late=14 -> 0.009%). + +**The axis EFP cannot see (dumb) has a bigger effect than the one axis +it does look at (gap).** Direct, quantified evidence that the table +carries real information EFP structurally cannot use, independent of +the `ComputeMoveScore` bug above. + +## The "deep prune" idea: cut hopeless-looking branches much higher in the tree + +Scott's proposal, motivated by the above: rather than (or in addition +to) EFP's high-volume, low-value pruning right at the horizon (1-2 +plies remaining, ~30-100 node subtrees, mostly qsearch), use the same +table to prune a MUCH smaller number of branches much higher up (6-8 +plies remaining) where a real hopeless/late/dumb-material cell reads a +genuinely evidence-backed (500+ tries per the shipped floor) exact +`0.0000%` observed FH rate. + +**Order-of-magnitude check, from real measured numbers (self-play +`ebf`~3.6, and an observed ~84% qsearch-node share in a real `sd8` +trace)**: full-width-nodes-times-qsearch-multiplier estimate puts a +1-2-ply subtree at ~30-100 nodes (matches EFP's actual footprint) vs. +~20,000 (depth 6) to ~250,000 (depth 8) nodes for a deep subtree -- +several orders of magnitude more payoff per prune decision. + +**Confirmed the table actually has real, evidence-backed near-zero +cells there**: at depth=6-8, gap=10 (>=901cp behind -- roughly a queen +or more), late=14 (very late/leftover), dumb=6-9 (materially hanging), +observed FH% reads exactly `0.0000%`, and because every non-sentinel +cell already cleared the 500-tries collection floor, this is a real +"watched 500+ times, never once failed high" statement, not a guess. +The most extreme combination (`dumb=10` AND `gap=10` together) doesn't +even have data yet -- too rare to have cleared the floor in the +current (60-90 minute) collection run, which is itself part of the +argument for a bigger rebuild (see next section). + +**This is explicitly the highest-conviction next direction discussed +this session** -- higher potential payoff than either the LMR +augmentation above or an EFP patch, but also acknowledged as the +riskiest (an irreversible prune, firing deeper in the tree where a +wrong call costs an exponentially bigger subtree than either EFP or +this round's LMR work ever risked). + +## Agreed design requirements for the deep-prune mechanism, before it's built for real + +1. **One unified eligibility gate for ALL pruning/reduction + mechanisms** (EFP, this round's LMR augmentation, any future LMP, + and the deep prune) -- extend the existing `fPruneReduceEligible` + checklist (already shared infrastructure from the earlier EFP work) + to explicitly cover: no checks, no escaping check, leftover moves + only, not a killer (previous ply or this ply), no reason to extend + this node, no reason the LAST move by this side was extended + (EFP's existing gate has this piece already), `!fThreat`, not a + capture unless losing, `iAlpha < +NMATE`. Rationale, per Scott + directly: these preconditions define "unremarkable move," which is + also exactly the population the table itself was trained against -- + using a different gate at consult time than at collection time is + the same population-mismatch mistake the EFP work already paid for + once (see "the population the data is measured over" above). + +2. **A rebuilt, larger, deeper table before trusting "confirmed + zero" for anything irreversible.** Current table: 60-90 minutes, + 16 shards, `sd 16`. Planned rebuild: overnight or longer (8-12+ + hours/shard), deeper (`sd 18-20`+, to populate depth 10-13 with + real signal instead of the coarsened overflow bucket), bigger/more + varied position pool specifically to get real sample volume into + the deep+hopeless+late cells that are currently unpopulated (e.g. + `dumb=10`+`gap=10` combined). Blocked for now: this box doesn't + have spare cores to run a 16-shard `--cpus 1` collection run + alongside the ongoing 12-worker self-play match (each `match_play.py` + worker spawns 2 single-threaded engine processes, so 12 workers is + already ~24 concurrent processes) -- waiting for the self-play LLR + to resolve (cross either SPRT bound) before starting this. + +3. **"Certain" means literally `0.0%` observed, at real sample + volume** -- not a low-but-nonzero tier, per Scott's explicit + framing. A follow-on, not-yet-designed refinement: how much sample + volume is actually enough given how many times per game this gate + would fire (see the earlier EFP section's "how much statistical + confidence is actually needed" analysis -- same rule-of-three + reasoning applies here and needs re-deriving for whatever + application rate the deep-prune gate turns out to have). + +4. **A prune-specific shadow-verification harness, built BEFORE any + real pruning, same discipline as `JOINT_EXTRA_VERIFY` above but + answering a different question**: not "would a deeper search of + this reduced move have failed high" but "would searching this move + to real depth at all have failed high, given we were about to skip + it entirely." Must apply the (now understood) `NEUTRALIZE_NULL_ + BAILOUT` fix from day one, and must dump FEN + full details to a + log on every actual miss found, for direct manual investigation + (matching how the two genuine-miss classes above were understood) -- + agreed explicitly: any wrong guess gets looked at and reasoned + about, not just counted. + +5. **A dynamic, per-move-identity component is needed and is + currently MISSING from this round's design** -- `GetLMRReduction`'s + own existing pattern (require the specific move's own + `GetMoveFailHighPercentage` history to independently agree, not + just the static table cell) was flagged mid-session as something + the deep-prune gate must also have: require both the static table + cell to read genuine evidence-backed zero AND this exact move's own + accumulated in-search history to show zero fail-highs over some + minimum tries, before ever pruning for real. Countermove-conditioned + dynamic veto was raised as a further, explicitly UNTESTED idea for + later, not part of the first attempt. + +## Self-play status as of this write-up (2026-09-29, ongoing) + +First live test of the LMR-only augmentation (EFP left untouched, +including its known `ComputeMoveScore` bug) against `head_reference`, +`match_play.py`, `--st 4`, 12 workers, SPRT (`elo0=0, elo1=5`). + +**Pruning/reduction heuristic settings live for this entire match** +(`search.c` constants, unchanged for the whole run -- see "the mechanism: +strictly additive over `GetLMRReduction`" and "tier thresholds" sections +above for how these were originally derived): + +``` +JOINT_EXTRA_TIER1_THOUSANDTHS = 72 // ~p75 of LMR-slice FH%: below this, +1 ply +JOINT_EXTRA_TIER2_THOUSANDTHS = 5 // ~p25: below this, +2 plies +JOINT_EXTRA_TIER3_THOUSANDTHS = 1 // ~p5: below this, +3 plies +JOINT_EXTRA_MAX_PLIES = 3 +JOINT_EXTRA_MIN_REMAINING_DEPTH = ONE_PLY // most aggressive floor, post round-trip (see above) +``` + +i.e.: HEAD's own baseline `GetLMRReduction` (`-ONE_PLY`) is left untouched; +on top of it, a gate-1-eligible move whose 4D joint-table predicted FH% +falls under 0.072% gets +1 extra ply, under 0.005% gets +2, under 0.001% +gets +3 (capped by `JOINT_EXTRA_MAX_PLIES` and by the +`JOINT_EXTRA_MIN_REMAINING_DEPTH` floor so combined reduction never drops +remaining depth below one ply). EFP was deliberately left untouched for +this round (see below). + +| games | score | llr | avg depth (cand vs head) | ebf (cand vs head) | nps (cand vs head) | +|---|---|---|---|---|---| +| 110 | 0.532 | +0.16 | 12.90 > 12.77 | 3.561 < 3.664 | 1,625,612 > 1,570,311 | +| 290 | 0.533 | +0.45 | 13.21 > 12.80 | 3.545 < 3.651 | 1,618,609 > 1,576,194 | +| 730 | 0.510 | +0.28 | 13.30 > 12.99 | 3.546 < 3.622 | 1,621,970 > 1,589,133 | +| 2212 (final, killed by hand) | 0.502 | -0.19 | 13.28 > 12.99 | 3.543 < 3.615 | 1,604,237 > 1,577,331 | + +Trending positive early (110-730 games) but the match was killed by hand +at 2212 games once it settled back to dead-even: `SPRT=INCONCLUSIVE +LLR=-0.189 BOUNDS=[-2.944,2.944] ELO=+1.4+/-14.5 LOWER95=0.4812`. At this +sample size SPRT hadn't crossed only because `elo1=5` sits inside the ~12 +Elo-wide 95% confidence interval the match had already converged to -- +another 75 hours (the printed ETA) would have bought a label, not new +information, so continuing to let it run was judged not worth it. + +**Caveat added 2026-10-01**: this entire match ran under the +pre-unification gate (`uLegalMoves > 5`, the population the p75/p25/p5 +tier thresholds above were themselves derived against). EFP/LMR were +unified onto `fPruneReduceEligible` (`uLegalMoves > 1`) afterward, which +widens the live population these thresholds now classify without +re-deriving or re-testing them. So this "even, no strength gain" verdict +is evidence about the *original* p75/p25/p5 thresholds against the +*narrower* pre-unification population -- it says nothing yet about +their behavior (or the current hand-adjusted 150/5/1 thresholds, see +`search.c`'s `JOINT_EXTRA_TIER*_THOUSANDTHS`) against the current, +wider `uLegalMoves > 1` population. Needs its own self-play run before +treating this verdict as settled for the current code. + +**Verdict, Scott's read, 2026-09-30**: the mechanism *works* in the narrow +sense that it delivers real, consistent extra depth/speed (+0.3 ply, +~+1.7% nps, leaner `ebf`) for no loss in measured playing strength -- but +that 0.3 ply isn't enough on its own to show up as a score edge; the +extra depth has to be large enough to let the engine see things HEAD +doesn't. Two candidate next steps, not yet decided between: +1. **Find more *sound* pruning/reduction opportunities** via the planned + larger/deeper table rebuild (below) -- current table has real + low-traffic cells (sparse or unpopulated deep+hopeless+late + combinations) that a bigger collection run should fill in, unlocking + additional reduction without cost. +2. **Tune the existing curve to be a bit more aggressive as a starting + point.** Scott's specific framing: the current tier thresholds + (`p75`/`p25`/`p5` -> `+0`/`+1`/`+2`/`+3` plies, see above) aren't + hurting playing strength but also aren't buying enough depth/speed to + realize a gain -- worth pushing slightly harder on the SAME curve + shape before waiting on a full table rebuild. +Leaning toward trying (2) as a cheap, fast experiment first (doesn't need +the rebuild or its core budget) while (1)'s rebuild runs in parallel once +the match's ~24 `match_play.py` engine processes are confirmed dead. + +**The ~3-4% nps and ~0.4-ply depth advantage are almost certainly the +SAME underlying effect measured two ways, not two independent +confirmations.** This is a fixed-wall-clock-per-move match (`--st 4`), +so depth reached is a direct function of average cost per node. If +extra reduction shifts the tree's node mix toward proportionally more +(cheap) qsearch nodes and fewer (expensive) full-width nodes -- which +the lower `ebf` for candidate at both checkpoints is directly +consistent with -- that simultaneously raises the blended nodes/second +figure AND lets iterative deepening reach one more ply in the same +time budget. Confirmed as a real, structural effect (not process +scheduling noise): stable over an hour-plus of real gameplay across +~24 concurrent single-threaded engine processes on a shared, +contended box, not a short or single-sample reading. + +## Build hygiene note (2026-09-29) + +Before this self-play run started, did a full audit + clean rebuild to +confirm the shipped binary carries none of the investigation-only +instrumentation: `JOINT_EXTRA_VERIFY`, `SILENT_MISS_TRACE`, +`NEUTRALIZE_NULL_BAILOUT`, `DISABLE_EFP`, `DEPTH_ONLY_EXTRA_REDUCTION` +are all `ifdef`-gated behind their own opt-in `GNUmakefile` flags, none +part of the standard `GENETIC=1 PERF_COUNTERS=1 MP=1 SIXTYFOUR=1` +profile; confirmed absent from a clean build's own +`--command ""`-printed `Make profile used:` line. `QSearchFromCheckNo +StandPat`'s three rounds of live manual experimentation (see above) +were confirmed reverted to their original condition set via `git diff`. +No stray debugger hooks (`BREAKPOINT`) remained in `search.c`. Modified +files vs. HEAD for this whole effort: `GNUmakefile`, `chess.h`, +`data.c`, `root.c`, `script.c`, `search.c`. + +## Status / open questions as of this write-up + +- LMR augmentation: live, in self-play, inconclusive but trending + positive (llr +0.45 at 290 games). No further code changes planned + until this resolves one way or the other -- can't run the planned + table rebuild concurrently with self-play on this box's core count. +- EFP: left deliberately untouched (including the confirmed-dead + `ComputeMoveScore` check) for this round, on purpose, to keep the + LMR-only test interpretable. Two candidate next steps discussed and + NOT yet decided between: (a) a conservative "veto EFP's prune when + the table thinks this move is surprisingly dangerous" patch (small, + reversible, directly fixes the confirmed dead-check bug), or (b) + skip straight to the deep-prune mechanism and let IT eventually + subsume/retire EFP rather than patching EFP incrementally. Leaning + toward (b) as the real long-term direction, but agreed the + incremental-experiment discipline that's served this whole project + well should still apply -- not committing to skipping (a) outright. +- Deep-prune mechanism: designed in discussion (this section), not yet + implemented. Blocked on: (1) the overnight/longer table rebuild + (itself blocked on self-play finishing, core-count constrained), (2) + building the prune-specific shadow-verification harness, (3) the + unified-gate refactor, (4) deciding on the dynamic per-move-identity + component's exact shape. +- Open, explicitly undecided strategic question: does the table + eventually fully replace EFP (one mechanism, deep+shallow both + handled by table-driven decisions), or do EFP (shallow, high-volume, + patched) and a new deep-prune mechanism (deep, low-volume, + table-driven) coexist as two mechanisms answering the same + underlying question from different depths? Deferred until real + numbers exist from both the current self-play match and a future + deep-prune experiment. |
