summaryrefslogtreecommitdiff
path: root/tests
diff options
context:
space:
mode:
authorScott Gasch <[email protected]>2026-10-04 08:17:52 -0700
committerScott Gasch <[email protected]>2026-10-04 08:17:52 -0700
commitbbdf1bda8f410df6aa9be0a38e71b7db04982198 (patch)
treee8315247f700ee94875cd6d2eb5dd81aef71395d /tests
parent9b28080f8b3dca4588cb2aaaf748451f45e3c626 (diff)
Dynamic move-ordering/LMR session: honest FH tracking, decoupled admission/magnitude, bugfixes, tooling
Lands the 2026-09-28..10-03 move-ordering investigation (full history in lmr_testing/*.md). Summary of what ships: LMR/move-ordering redesign (searchsup.c, dynamic.c, data.c, chess.h): - GetLMRReduction's admission decision (do we reduce at all) is now decoupled from magnitude (how much): LMR_STATIC_ADMIT_THOUSANDTHS=500 gates admission via the 4D joint fail-high table (data.c) plus a countermove-match veto; magnitude is a flat Crafty/Ethereal-shaped log(depth)*log(moves) table (g_iLMRQuietReduction), which measured as good or better than the old per-tier joint-table payout while being structurally immune to the per-bucket miscalibration that sank the tiered approach. g_FailHighs honesty fix (real 0/0 init, not a fake prior) underlies the dynamic-danger signal. - Retired NumLeftoverMovesToSelect's depth-indexed selection budget -- evidence showed the leftover pool has real findable signal a bailout was discarding. - ShouldLatePrune (overt late-move pruning) implemented but disabled at its call site -- measured net-negative at the tested threshold; kept for reference and as a base for a future hang-bucket-9-targeted retry. - PRUNE_VERIFY, PRUNE_LIVE, DISABLE_LMR: opt-in diagnostic build flags (GNUmakefile), off in the standard profile. Bug fixes found and fixed while exercising this work: - JOINT_EXTRA_TIER_COUNT 4->6 (chess.h): fixed an out-of-bounds counter write reachable at remaining depth >= 14 (unreachable at sd10, routine in real search/self-play). - Node-type instrumentation (search.c, root.c, searchsup.c): fixed four compounding defects that produced impossible percentages (e.g. 106.2% of PV nodes) -- wrong denominator, qsearch misclassified as main search, untyped early-exit nodes, and fail-low tests using the in-place-raised alpha instead of the initial one. - FH-rate unit unification (search.c, searchsup.c, dynamic.c, chess.h): GetMoveFailHigh* returned per-mille while g_uJointFHThousandths is thousandths-of-a-percent, two different units sharing a "_THOUSANDTHS" suffix. Unified to one scale, renamed to "_PCT_THOUSANDTHS". RateBucket's band edges fixed to match. - QSearch option B (search.c): waives the opponent-stand-pat requirement at NODE_PV only (sound there since the exact score propagates; not sound at null-window nodes). iExtensionAmount unit fix (was 0/2 into a ONE_PLY-scaled field). pi->iEval now published when Eval() runs. - eval_tune/match_play.py local_ebf: was reporting 0.000 in every match since 2026-09-18 (regex only matched the trailing-stats parser, not the live move-scan loop; engines were also spawned with "nopost" so root.c never emitted the lines at all). True per-ply EBF is ~2.0, not the ~3.9 every prior match report implied. - eval_tune/test_vs_head.sh: positional-arg shift ordering bug caused every no-extra-args run to silently rebuild ./typhoon with a non-standard profile and, separately, could feed a flag's value in as the candidate binary path. - PERF_COUNTERS cleanup (eval.c, search.c): moved 3 leaked rdtsc sites to the existing EVAL_TIME guard, deleted two concluded one-off diagnostics (u64HeadStyle*, u64FHJointHist/RateBucket/AttemptsBucket) and their report plumbing, and re-gated a per-move hang-material estimate behind FH_TABLE_COLLECTION instead of running it unconditionally. Infrastructure: - match_play.py time-odds support (--odds/--st-head/--st-candidate), validated against the engine seat rather than color. - Wilson-upper-bound re-bake of the joint table (data.c): cells store a confidence-aware bound instead of a raw point estimate, so a consumer can threshold on the value alone; monotone, so every consumer became strictly more conservative. Verified behavior-neutral for LMR. Known-open items (not resolved here, see lmr_testing/GET_LMR_REDUCTION.md section 10 and 2026-10-03_UNCOMMITTED_TRIAGE.md): the self-play gate on this candidate is underpowered (+5 Elo, 95% CI roughly [-6, +17]); the curated-suite solve-count deltas used earlier in the investigation turned out to be noise-dominated cancellation when diffed per position, not the clean signal they first appeared to be. Co-Authored-By: Claude Sonnet 5 <[email protected]>
Diffstat (limited to 'tests')
-rw-r--r--tests/ECM.md3
1 files changed, 3 insertions, 0 deletions
diff --git a/tests/ECM.md b/tests/ECM.md
index 62c5bd4..d3077a3 100644
--- a/tests/ECM.md
+++ b/tests/ECM.md
@@ -88,6 +88,9 @@ session).
| ECM.717 | ringers | `r1qb1r1k/2p3pp/p1n1bp2/1p1Np2Q/P3P3/1BP3R1/1P3PPP/R1B3K1 w - - 0 0` | axb5 | Abandons a `++`-confirmed axb5 (depth 10, +3.16) for an unconfirmed Rxg7 (+3.17) at the very last iteration -- same shape as ECM.357. Lost by the f698c02 eval-margin-redesign build (2026-09-18) |
| ECM.601 | hard_quick | `8/p5k1/2p2p2/4p1p1/1p2P3/1P4PP/1PP5/5K2 b - - 0 0` | g4 | No iteration lines printed at all before the depth limit; final score a dead-equal +0.07 -- no confirmation of any kind. Lost by the f698c02 eval-margin-redesign build (2026-09-18) |
| ECM.193 | hard_quick | `r2r1bk1/3qp2p/3pp1p1/p2n2N1/2N3Q1/BP4P1/P4PP1/2R3K1 w - - 0 0` | Nb6 | Wavers between Qh3/Qf3 through most iterations, lands on Nb6 only at the sn5M node-budget cutoff, unconfirmed. Lost by the f698c02 eval-margin-redesign build (2026-09-18) at sn5M |
+| ECM.770 | hard_quick | `rnb2rk1/pp3pbp/2p3pB/2q5/4P3/1BN5/PPP3PP/R2Q1R1K w - -` | Rxf7 | HEAD locks onto (an unconfirmed but stable) `Rxf7` through depth 12, then abandons it for `Bf4` from depth 12 (second entry) through depth 16+ (+2.60 -> +2.98 -> +2.57). **Crafty independently confirms `Bf4`, not `Rxf7`**, stably from depth 22 through 28 (+1.78 -> +2.61, six-plus stable iterations) -- the labeled "solution" appears to be wrong, not a search miss. Same wobble reproduced identically under both `JOINT_EXTRA_TIER1_THOUSANDTHS=72` and `=150` (2026-09-30 threshold-sweep investigation), confirming it predates and is unrelated to that change |
+| ECM.826 | confident_quick | `1rbq2k1/b1p3pp/p1Pp4/3Ppr2/8/5N2/PPQ2PPP/R1B2RK1 b - -` | Rxf3 | Sits on `Bd4` (-0.73) through depth 10, only switches to (and then stably confirms) `Rxf3` at depth 11-14. Identical behavior, same scores, same switch point under both `JOINT_EXTRA_TIER1_THOUSANDTHS=72` and `=150` (2026-09-30) -- a depth-budget/horizon effect on both thresholds, not a regression |
+| ECM.491 | hard_quick | `1r2r1k1/p4ppb/q1pbnn1p/4p3/N1P1P3/pPB2NP1/2Q1RPBP/R5K1 b - -` | Nxe4 | Wobbles `c5` (depths 9-11) -> `Bxe4` (depth 11-13) -> `Nxe4` (matching "solution", depth 13-14), never settling before the sd10/sn5M test budget. Identical wobble, same depths, same scores under both `JOINT_EXTRA_TIER1_THOUSANDTHS=72` and `=150` (2026-09-30) -- predates the threshold change |
## Gains seen (candidate solves, HEAD doesn't) -- not deeply vetted