summaryrefslogtreecommitdiff
path: root/tests/ECM.md
diff options
context:
space:
mode:
authorScott Gasch <[email protected]>2026-10-10 10:29:48 -0700
committerScott Gasch <[email protected]>2026-10-10 10:29:48 -0700
commit03545c4600df81b1a47dbc63fc606c065e857d8b (patch)
treea433fdce69fd27359679ad91e0ebedd52e2506cd /tests/ECM.md
parentfbdcb9d76dd2444d8bc631331eaeb9435cc247f8 (diff)
Eval mobility/pseudo-mobility tweaks; drop unproductive move-ordering experiments.
eval.c: add bishop/rook x-ray pseudo-mobility terms (squares seen past a directly-attacked higher-value piece), trim pawn-centrality edge penalty and bishop-pair bonus. chess.h/dynamic.c/generate.c/search.c/searchsup.c: revert three move-ordering experiments (recapture bonus, flee-to-safety GOOD_MOVE promotion, ply-2 move-pair bonus) that calibrated with real fail-high signal but never converted into a measurable node-count or solve-rate win on the curated suites -- see MOVE_ORDERING_IDEAS.md for the evidence and reasoning behind each drop. Add MOVE_ORDERING_EVIDENCE_COLLECTION as its own GNUmakefile flag, independent of PERF_COUNTERS, for any future such harness. command.c: remove the KING_DEFECT_DUMP-gated evalkingdump command, dead since the king-safety remine work moved to eval_tune/king_safety_data/. Co-Authored-By: Claude Sonnet 5 <[email protected]>
Diffstat (limited to 'tests/ECM.md')
-rw-r--r--tests/ECM.md6
1 files changed, 6 insertions, 0 deletions
diff --git a/tests/ECM.md b/tests/ECM.md
index d3077a3..33bc905 100644
--- a/tests/ECM.md
+++ b/tests/ECM.md
@@ -60,6 +60,10 @@ session).
| ECM.746 | ringers | `r3k2r/p1qb1p2/1p2p2p/3pPpN1/P1nP3Q/8/2P2PPP/R1B1R1K1 w kq - 0 0` | Nxf7 | `++`-confirmed at depth 10 (+2.27), same move held through the final depth-10 re-search (+2.62) -- no wavering. Lost by the f698c02 eval-margin-redesign build (2026-09-18) |
| ECM.280 | confident_quick | `r1b2rk1/1pqp1ppp/p1n1pn2/5N2/8/2PBB3/P1P2PPP/R2QR1K1 w - - 0 0` | Nxg7 | Same first move (`Nxg7`) held stable across all 4 depth-8/9/10 iterations including a `++` confirm at depth 9, despite the score itself swinging (-0.29 to +0.93 to +0.89) -- move choice never wavered. Lost by the f698c02 eval-margin-redesign build (2026-09-18) |
| ECM.597 | hard_quick | `rnb2rk1/pp4pp/2pb1n2/3N1p1q/2P5/3N2P1/P1Q1PPBP/R1B2RK1 w - - 0 0` | Bf4 | Same move (`Bf4`) held stable across all 3 iterations (depths 8-10), no wavering, though never gets an explicit `++` mark. Lost by the f698c02 eval-margin-redesign build (2026-09-18) |
+| ECM.339 | confident_quick | `r1b1r1k1/pp3p1p/1q2p1pQ/2b1P1B1/8/P2B3P/1P3PP1/2R1R1K1 w - -` | Rxc5 | Same shape as ECM.442/672: HEAD solves at depth 10 (310k nodes); the bishop-mobility-rewrite build (2026-10-10) still finds it, just one ply later (depth 11, 1.3M nodes) -- a depth-budget delay, not a blind spot |
+| ECM.450 | confident_quick | `6r1/6rk/3p3p/p2Rp3/R1PqPp1b/2N2Q2/6PP/7K b - -` | Rxg2 | Same depth-budget-delay shape: HEAD solves before depth 10 even prints (<278k nodes); the bishop-mobility-rewrite build (2026-10-10) solves at depth 11 (454k nodes) -- later, not missed |
+| ECM.496 | confident_quick | `r2q1rk1/pp1n2pp/2pb4/1P1p3n/P1PPpP2/4P2b/1BNNBP1P/R2QR1K1 b - -` | Nxf4 | Same depth-budget-delay shape: HEAD solves at depth 10 (2.0M nodes); the bishop-mobility-rewrite build (2026-10-10) solves at depth 11 (2.43M nodes). Neither build actually understands the tactic until depth 14-15 (score jumps 1.4 -> 3.1 -> 3.8) -- the depth-10 "solve" on either side is itself a shallow, barely-there read |
+| ECM.672 | confident_quick | `5rk1/pbrn1ppp/1ppN1q2/2P1p3/3P4/1PRB4/P3QPPP/5RK1 w - - 0 0` | Bxh7+ | Same shape again, pushed back further this time: the rook-mobility-rewrite build (2026-10-10) never touches `Bxh7+` within the sd10 window at all (stays on a quiet `dxe5` line throughout, +1.05), unlike the bishop-only intermediate build which still solved it. Re-run to `sd 16` on the same position: `Bxh7+` appears and gets `++`-confirmed twice at depth 11 (+1.80, +3.30), climbing stably through depth 16 (+6.96) -- a 3-ply depth-budget delay, not a blind spot. Consistent with this position's existing track record (already shown depth-delay-prone under the unrelated 2026-09-11 LMR graded-curve change too, per that session's note above) |
## Flaky / garbage (suite noise -- don't read a loss here as a real regression)
@@ -91,6 +95,8 @@ session).
| ECM.770 | hard_quick | `rnb2rk1/pp3pbp/2p3pB/2q5/4P3/1BN5/PPP3PP/R2Q1R1K w - -` | Rxf7 | HEAD locks onto (an unconfirmed but stable) `Rxf7` through depth 12, then abandons it for `Bf4` from depth 12 (second entry) through depth 16+ (+2.60 -> +2.98 -> +2.57). **Crafty independently confirms `Bf4`, not `Rxf7`**, stably from depth 22 through 28 (+1.78 -> +2.61, six-plus stable iterations) -- the labeled "solution" appears to be wrong, not a search miss. Same wobble reproduced identically under both `JOINT_EXTRA_TIER1_THOUSANDTHS=72` and `=150` (2026-09-30 threshold-sweep investigation), confirming it predates and is unrelated to that change |
| ECM.826 | confident_quick | `1rbq2k1/b1p3pp/p1Pp4/3Ppr2/8/5N2/PPQ2PPP/R1B2RK1 b - -` | Rxf3 | Sits on `Bd4` (-0.73) through depth 10, only switches to (and then stably confirms) `Rxf3` at depth 11-14. Identical behavior, same scores, same switch point under both `JOINT_EXTRA_TIER1_THOUSANDTHS=72` and `=150` (2026-09-30) -- a depth-budget/horizon effect on both thresholds, not a regression |
| ECM.491 | hard_quick | `1r2r1k1/p4ppb/q1pbnn1p/4p3/N1P1P3/pPB2NP1/2Q1RPBP/R5K1 b - -` | Nxe4 | Wobbles `c5` (depths 9-11) -> `Bxe4` (depth 11-13) -> `Nxe4` (matching "solution", depth 13-14), never settling before the sd10/sn5M test budget. Identical wobble, same depths, same scores under both `JOINT_EXTRA_TIER1_THOUSANDTHS=72` and `=150` (2026-09-30) -- predates the threshold change |
+| ECM.298 | hard_quick | `2b1qr1k/7p/p1pR2p1/1pP1p1Q1/4P3/1B6/PP4PP/6K1 w - -` | Re6 | Re-checked under the rook-mobility-rewrite build (2026-10-10): both HEAD (`Re6`, matching the suite label) and the new build (`Rf6`) print *zero* per-iteration lines before hitting the depth limit -- the exact "no iteration lines printed at all, no confirmation of any kind" signature this doc already calls flaky elsewhere (see ECM.601). HEAD's own "solve" here is just as unconfirmed as the new build's "miss" -- not a real loss. (Separately, an unrelated 2026-09-14 session found a *different* candidate playing `h4` here, independently confirmed stronger than the suite's own labeled `Re6` by a deep Crafty check -- see that session's note below. The suite's expected answer for this position may simply be wrong.) |
+| ECM.186 | hard_quick | `1R3b2/3r3k/2p1bp1p/r1q1pNpQ/2PpP2P/6B1/P4PP1/1R4K1 w - -` | Nxh6 | Same depth-budget-delay shape as 339/450/496/672: HEAD solves at depth 11 (+5.75), climbing cleanly through depth 13 (`++` confirmed, +8.13). The queen-mobility-rewrite build (2026-10-10, bishop+rook+queen combined) still prefers `Qe8` at depth 11, but switches to and solves with `Nxh6` at depth 12 (+7.26) -- pushed out one ply, not missed |
## Gains seen (candidate solves, HEAD doesn't) -- not deeply vetted