1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
|
# Retiring EFP in favor of LMP
Status: **plan only, not started.** Supersedes item 4 ("EFP-vs-LMP
substitution experiment") in `EXTENDED_PRUNING_DEPTH.md`, which flagged this
as the next step but didn't have the concrete numbers below. Do not start
until the current uncommitted working-tree changes (presently being
evaluated against HEAD via `match_play.py`, score trending positive) are
either committed or reverted -- this work should start from a clean,
committed baseline, not stack on top of an in-flight, unresolved candidate.
## Motivation
EFP (`search.c`'s `fTryEFPHere`/`fThisMoveEFPPruned`, ~line 1440-1522) and
LMP (`ShouldLatePrune`, `searchsup.c`) are two separate forward-pruning
mechanisms that fire on heavily overlapping populations at remaining depth
0-2 ply, maintained independently, with no shared tuning discipline between
them. EFP uses a hand-picked margin (Heinz's paper: `VALUE_KNIGHT`/
`VALUE_ROOK` by 2-tier depth) plus two fail-high-history vetoes (static
table, dynamic per-move). LMP uses the same underlying 5D joint fail-high
table (`g_uJointFHThousandths`, `data.c`) directly, with no margin test and
its own, differently-calibrated pair of vetoes. Goal: retire EFP entirely,
let LMP's table-driven decision govern that depth range on its own, without
losing EFP's pruning volume (tree size must not grow) and without adopting
Crafty's `break`-the-whole-node semantics (every mechanism here stays
per-move, `continue`-based -- rejected explicitly, see "What we are
deliberately not doing" below).
## What's already been verified (no code changes made yet)
1. **The joint table has no sentinels.** `g_uJointFHThousandths` (`data.c`)
is fully backoff-smoothed (Wilson-upper-bound, shrinkage to coarser
marginals) as of the mid-run splice documented in
`../dual_table_run/README.md` -- "No 0xffff sentinels remain." Both
`ShouldLatePrune`'s and EFP's sentinel checks are currently dead code,
not a live concern.
2. **The table's collection was censored by EFP/LMP themselves, and that's
expected, not a flaw.** `search.c`'s `FH_TABLE_COLLECTION` instrumentation
sits textually *after* the `if (fThisMoveEFPPruned || fThisMoveLMPPruned)
{ ...; continue; }` block (~line 1550-1613) during the collection run
that built this table. A move either mechanism decided to prune never
reached the collection counters. This means the table is accurate for
the population it actually saw (eligible moves that survived both
mechanisms) but has **zero data** on the population EFP has been
discarding -- not biased data, no data. There is no way to query "what
was the real FH% of EFP's historically-pruned moves" from existing logs;
it was never measured. (Corrects an earlier, imprecise claim in this
investigation that bucket[0]/[1] were "thin" there -- they aren't thin,
they're simply about a different population.)
3. **EFP's own margin, read through our own table, is already far more
conservative than its nominal static-veto bar.** Looking up EFP's tier-1
margin (`VALUE_KNIGHT`≈325cp, depth bucket 1, gap bucket 4) in the live
table gives a measured FH% of 0.01-0.05% across lateness buckets --
roughly 10x *under* EFP's own `uPredictedFH < 300` (0.3%) static veto.
That veto has essentially never been the operative gate; the margin test
alone already does the real filtering, more conservatively than its own
stated threshold.
4. **LMP's existing static ceiling already has slack to cover EFP's old
population, unmodified.** Comparing EFP's old population's real
table-measured FH% against LMP's current `uLMPStaticMaxPctThousandths[]`
ceiling by depth bucket:
| depth bucket | LMP's current ceiling | EFP's-old-population measured FH% |
|---|---|---|
| 0 | 0.200% | 0.010% |
| 1 | 0.200% | 0.020% |
| 2 | 0.150% | 0.036% |
| 3 | 0.075% | 0.063% |
| 4 | 0.050% | 0.071% (now exceeds ceiling -- correct, by design) |
At depth buckets 0-2 (EFP's actual operating range), LMP's own ceiling
already permits 5-20x more risk than EFP's population actually carries.
The static side of LMP should, on its own, already prune *at least* as
much as EFP did there. By depth bucket 3+, LMP's own schedule correctly
starts refusing -- that tightening-with-depth behavior is intentional,
not a gap to fix.
5. **The real asymmetry is the DYNAMIC veto, not the static one, and this
is the mechanism most likely to cause a tree-size regression on naive
retirement.** Both EFP and LMP gate on real, per-move-identity observed
fail-high history in addition to the static table, but at wildly
different strictness:
| | min samples | max tolerated dynamic FH% |
|---|---|---|
| EFP (`search.c`) | 5 | **10%** (`EFP_FH_PRUNE_THRESHOLD_PCT_THOUSANDTHS=10000`) |
| LMP (`searchsup.c`) | 1 | **0.001%** (`LMP_DYNAMIC_MAX_PCT_THOUSANDTHS=1`) |
A 10,000x gap. `LMP_DYNAMIC_MAX_PCT_THOUSANDTHS=1` is, in practice, a
"this move identity has never once failed high, for any nonzero sample
count" gate -- not a calibrated percentage. Any move identity with real,
nonzero fail-high history anywhere in the tree (common for moves tried
more than a handful of times) gets exempted from LMP pruning even when
the static table is fully confident. EFP rarely hit this problem because
its own dynamic bar (10%) almost never tripped. **Predicted outcome if
EFP is retired without also touching this constant: LMP's static side
alone is not sufficient to match EFP's old pruning volume, because this
gate silently vetoes a meaningful fraction of what the static table
would otherwise allow.** This is the paper's central, falsifiable
prediction -- see Plan step 2.
6. **The apparent "Crafty's ply-2 margin predicts a LOWER fail-high rate
than ply-3, despite a smaller margin" oddity is a bucket-resolution
artifact, not a real inversion.** Crafty's 150cp (remaining ply 2) and
200cp (remaining ply 3) both land in our 100cp-wide gap bucket 2.
Holding gap bucket fixed and varying depth shows the expected
monotonic-with-depth increase (0.039% -> 0.053% -> 0.089% -> 0.176% ...
at gap bucket 2, late bucket 4, dumb=0, improve=neutral) -- "more depth
remaining = more chance to recover" holds throughout; our 100cp gap
bucket is just too coarse to resolve Crafty's 50cp margin steps at the
shallow end. Not something this plan needs to fix to proceed, but worth
remembering if a future finer-grained gap axis is ever considered.
7. **Reference points from Crafty/Stockfish, translated into our table
(dumb=0, improve=neutral, several lateness buckets), for calibration
context only -- not adopted as targets:**
- Crafty's `FP_margin[]` at remaining ply 1 (100cp) implies **0.2-3.2%**
FH% in our table depending on lateness -- the single largest
"implies more room exists" data point of this whole investigation,
concentrated at the shallowest depth and earliest move numbers.
Crafty's deeper tiers (150-400cp, ply 2-6) land close to our own
existing 0.05-0.3% range -- roughly agreeing with us already.
- Stockfish's quiet-move futility margin translates to **0.01-0.13%**
throughout -- more conservative than Crafty in our terms, not less;
consistent with SF leaning on other layered mechanisms (move-count
quiet-move skip, continuation-history pruning, SEE-based pruning) to
do work Crafty's single margin does alone. SF's own margin test is
therefore not a fair apples-to-apples number by itself.
- Neither engine's number is being proposed as a direct target --
they're priors for where real engines have found a workable
risk/reward point, used to sanity-check, not dictate, what we test.
## What we are deliberately not doing
**No `break`.** Crafty `break`s its entire move loop on either its
futility or LMP check -- abandons every remaining move at that node, not
just the one under consideration. Stockfish's futility tests are `continue`
(per-move, like ours); its one break-like mechanism (`skip_quiet_moves()`)
is narrower still -- it only stops trying further *quiet* moves, captures
and checks still get considered afterward. Both EFP and LMP already use
`continue`, evaluating every move's prune/no-prune decision independently,
and that is staying exactly as-is. Nothing in this plan should introduce a
node-wide bailout of any kind.
## Plan
1. **Baseline.** Confirm `head_reference/` is current at a clean, committed
HEAD (rebuild if the in-flight uncommitted changes mentioned above get
committed first). Capture `sd 10` and `sn 5000000` logs for all three
curated suites (`ecm_ringers`, `ecm_confident_quick`, `ecm_hard_quick`)
if not already fresh.
2. **Two-variant node-count experiment, before touching `match_play.py` at
all.** Build and compare two variants against the baseline, fixed `sd`,
all three curated suites, node count + first-move-cutoff-rate as the
signal (per CLAUDE.md's existing move-ordering-experiment methodology --
measure the pick, not the game):
- **Variant A**: delete EFP (`fTryEFPHere`, `fThisMoveEFPPruned`, and its
contribution to the final prune-or-continue check) entirely.
`LMP_DYNAMIC_MAX_PCT_THOUSANDTHS` left untouched at `1`.
**Predicted result**: tree size grows relative to baseline, confirming
finding 5 above -- the dynamic veto asymmetry is real and costs
pruning volume LMP's static side alone doesn't recover.
- **Variant B**: same EFP deletion, plus raise
`LMP_DYNAMIC_MAX_PCT_THOUSANDTHS` toward EFP's old bar (start at
`10000` to match EFP exactly, as the cleanest single-variable swap;
narrower values can be swept later if this overshoots).
**Predicted result**: tree size lands close to or tighter than
baseline, since finding 4 already showed the static side has headroom
once the dynamic gate stops over-vetoing.
- If both predictions hold, that's strong, cheap (no `match_play.py`
games needed yet) confirmation of the mechanism before spending any
self-play budget. If either prediction is wrong, stop and re-diagnose
before proceeding -- a wrong prediction here means the mental model of
which gate is load-bearing is incomplete, and tuning further on top of
a wrong model wastes the expensive match-play stage.
3. **`match_play.py` gate on Variant B only** (not A -- A is a diagnostic,
not a candidate; it's not expected to be an improvement on its own).
No cheap/fast elimination stage exists in this project's tooling --
there's no shortcut cheaper than a full run, so don't plan around one.
Use step 2's node-count + fail-high-rate-by-bucket diagnostic as the
only pre-filter (lean on it harder than a minimal single-value check --
compare across all three suites and several buckets, not just
aggregate node count, since it's the last free signal before paying
for self-play). Then commit to **one full-length, long-run
`match_play.py` game count** (matching the run already used for the
current unrelated candidate) directly on Variant B at
`LMP_DYNAMIC_MAX_PCT_THOUSANDTHS=10000` -- don't pre-plan a sweep.
Gate on `LOWER95 >= 0.5`, not raw score, per existing project
convention. Only spend a second full run sweeping a narrower
`LMP_DYNAMIC_MAX_PCT_THOUSANDTHS` value if this first run comes back
close-but-not-passing; if it misses badly, that's a signal the
dynamic-veto model itself is off (go to step 5), not that a nearby
value would have worked.
4. **If Variant B wins**: this becomes the EFP-retirement commit. Remove
the now-dead EFP code paths, `EFP_FH_MIN_SAMPLES`/
`EFP_FH_PRUNE_THRESHOLD_PCT_THOUSANDTHS` macros, and update
`ShouldLatePrune`'s docstring again to drop the "second gate, applied
after EFP" framing (it's no longer applied after anything -- it's the
only gate). Rebuild `head_reference/` as the new baseline.
5. **If Variant B does not win** (tree size recovers per the prediction but
playing strength doesn't hold, or the dynamic-veto fix overshoots into
real tactical misses): fall back to sweeping
`LMP_DYNAMIC_MAX_PCT_THOUSANDTHS` between `1` and `10000` rather than
either extreme, same one-variable-at-a-time discipline, before
concluding EFP's margin test was catching something the table's axes
genuinely can't see (in which case EFP's own tiers should be recalibrated
against measured surprise rate instead of retired, per
`EXTENDED_PRUNING_DEPTH.md`'s original fallback plan).
## Open items, not part of this plan's critical path
- **Horizon-tier loosening beyond matching EFP** (bucket 0/1's static
ceiling raised past its current 200/1000, toward Crafty's own
~1-3%-at-early-lateness risk tolerance) is a separate, later experiment --
only worth running once EFP retirement itself is confirmed at parity.
Don't conflate "retire EFP without regressing" with "also get more
aggressive than EFP ever was" in the same test; they have different,
separable risk profiles and should be gated independently.
- **`split.c` has neither EFP nor LMP today** (confirmed: no
`fTryEFPHere`/`fThisMoveEFPPruned`/`ShouldLatePrune` references in
`split.c`). This plan doesn't change that; `UNIFY_SEARCH.md` already
tracks bringing LMP (and, after this plan, LMP-without-EFP) to the split
path as its own separate work.
- **Gap-bucket resolution** (currently 100cp-wide, per finding 6) is coarser
than Crafty's own margin schedule moves in. Not a blocker for this plan,
but flagged for whoever eventually tackles the horizon-tier-loosening
follow-on above, since a coarse bucket there would blur exactly the
region that follow-on cares about most.
|
