diff options
| author | Scott Gasch <[email protected]> | 2026-09-18 20:11:14 -0700 |
|---|---|---|
| committer | Scott Gasch <[email protected]> | 2026-09-18 20:11:14 -0700 |
| commit | 73b82a01c6f1b15489608c7e2cf757180746f7b8 (patch) | |
| tree | 587de6f6d2825f67063692878c93f8377e62c046 /tests/ECM.md | |
| parent | 187e321c450740dc0fba2c1d29917851d45a822f (diff) | |
ECM.md: classify f698c02's new ECM solve losses (5 legit, 4 flaky)
Investigated per the established HEAD-confidence methodology after the
f698c02 checkpoint's aggregate sd10/sn5M drop (98/191->94/191,
99/191->98/191) looked concerning despite winning self-play against old
HEAD, typhoon_conservative, and typhoon_liberal. Reran archived
head_reference/archive/typhoon-4a7ec9a against all three curated suites
to get real before/after logs (the in-place head_reference/logs/ had
already been overwritten by update_head_reference.sh).
5 of the new losses are real (ECM.746/280/597, plus already-known
736/251); 4 are suite noise (ECM.357/717 abandon a confirmed move for an
unconfirmed one at the last iteration, ECM.601 has no confirmation
signature at all, ECM.193 wavers to the node-budget cutoff) -- roughly
flat net real signal, not the large regression the raw aggregate
suggested.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Diffstat (limited to 'tests/ECM.md')
| -rw-r--r-- | tests/ECM.md | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/tests/ECM.md b/tests/ECM.md index 5ff1ed0..64b4fa5 100644 --- a/tests/ECM.md +++ b/tests/ECM.md @@ -57,6 +57,9 @@ session). | ECM.735 | hard_quick | `r3r1k1/pp3ppp/8/1Nbp3q/3NnBn1/2P5/PPQ2PPP/3R1RK1 b - -` | Ngxf2 | HEAD's own line is clean and solid; Crafty's defense uses the *other* knight (Nexf2) but that's a "which piece" quibble, not a refutation of the idea | | ECM.803 | hard_quick | `2b2r1k/2p3pp/2Nb4/pP5q/2PP4/P4pP1/5P1P/R1BQ2K1 b - -` | Bxg3 | HEAD's line is clean; Crafty independently finds **mate in 10** starting with Bxg3 -- about as solid as confirmation gets | | ECM.442 | hard_quick | `4qn1k/1b3rpp/pb2pB2/1pp1P2P/3p1NQ1/P2P2PB/1PP4K/5R2 w - -` | Ng6+ | Not actually a miss -- the king-file-storm-defect candidate (2026-09-14) settles on `Bh4` at `sd10`, but its own deeper iterations (depth 14-15) find and `++`-confirm `Ng6+` (+1.94, climbing to +2.47 by depth 16). A depth-budget delay, not a blind spot; see the 2026-09-14 session note below | +| ECM.746 | ringers | `r3k2r/p1qb1p2/1p2p2p/3pPpN1/P1nP3Q/8/2P2PPP/R1B1R1K1 w kq - 0 0` | Nxf7 | `++`-confirmed at depth 10 (+2.27), same move held through the final depth-10 re-search (+2.62) -- no wavering. Lost by the f698c02 eval-margin-redesign build (2026-09-18) | +| ECM.280 | confident_quick | `r1b2rk1/1pqp1ppp/p1n1pn2/5N2/8/2PBB3/P1P2PPP/R2QR1K1 w - - 0 0` | Nxg7 | Same first move (`Nxg7`) held stable across all 4 depth-8/9/10 iterations including a `++` confirm at depth 9, despite the score itself swinging (-0.29 to +0.93 to +0.89) -- move choice never wavered. Lost by the f698c02 eval-margin-redesign build (2026-09-18) | +| ECM.597 | hard_quick | `rnb2rk1/pp4pp/2pb1n2/3N1p1q/2P5/3N2P1/P1Q1PPBP/R1B2RK1 w - - 0 0` | Bf4 | Same move (`Bf4`) held stable across all 3 iterations (depths 8-10), no wavering, though never gets an explicit `++` mark. Lost by the f698c02 eval-margin-redesign build (2026-09-18) | ## Flaky / garbage (suite noise -- don't read a loss here as a real regression) @@ -79,6 +82,10 @@ session). | ECM.875 | hard_quick | `2kr1br1/ppp2p1p/2n1p1q1/2Nn4/3P4/1Q2PB2/PP1B1P1P/2R1K2R b K - 0 0` | Qg1+ | Score exactly `0.00` on a repetition line (`<=>`) across 6 straight iterations (depths 8-13) -- textbook flaky signature. Contradicts the "Gains seen" table's loose note calling this Crafty-confirmed legit in an earlier review; that note appears to have been wrong, or about a different search reaching a different (non-drawing) conclusion. Gained by the candidate LMR-graded-curve build (2026-09-11) | | ECM.879 | hard_quick | `r2r2k1/pp3ppp/2q5/2p5/2P1P1n1/P4P1b/1BQ1BP1P/R3R2K b - - 0 0` | Rd2 | Score exactly `0.00` on a repetition line (`<=>`) -- same signature as ECM.875. Gained by the candidate LMR-graded-curve build (2026-09-11) | | ECM.866 | hard_quick | `7R/1p1r4/2b2p2/2Pp1qk1/3P1bp1/3N4/2Q2PB1/6K1 w - -` | Rg8+ | HEAD's own score is negative (-0.12 to -0.15); Crafty's independent top choice (Qd1) is also near-zero. **Note**: has shown up as a *gain* (candidate solves it, HEAD doesn't) in at least one later run too -- consistent with a genuinely coin-flip position, not a stable read either direction | +| ECM.357 | confident_quick | `r3r1k1/4pp1p/b5pB/q1pP4/3b4/2N4P/PP4P1/R2RQ2K w - - 0 0` | Be3 | Abandons a `++`-confirmed Be3 (depth 10, +0.70) for an unconfirmed Rxd4 (+0.80) at the very last iteration -- textbook flaky signature. Lost by the f698c02 eval-margin-redesign build (2026-09-18) | +| ECM.717 | ringers | `r1qb1r1k/2p3pp/p1n1bp2/1p1Np2Q/P3P3/1BP3R1/1P3PPP/R1B3K1 w - - 0 0` | axb5 | Abandons a `++`-confirmed axb5 (depth 10, +3.16) for an unconfirmed Rxg7 (+3.17) at the very last iteration -- same shape as ECM.357. Lost by the f698c02 eval-margin-redesign build (2026-09-18) | +| ECM.601 | hard_quick | `8/p5k1/2p2p2/4p1p1/1p2P3/1P4PP/1PP5/5K2 b - - 0 0` | g4 | No iteration lines printed at all before the depth limit; final score a dead-equal +0.07 -- no confirmation of any kind. Lost by the f698c02 eval-margin-redesign build (2026-09-18) | +| ECM.193 | hard_quick | `r2r1bk1/3qp2p/3pp1p1/p2n2N1/2N3Q1/BP4P1/P4PP1/2R3K1 w - - 0 0` | Nb6 | Wavers between Qh3/Qf3 through most iterations, lands on Nb6 only at the sn5M node-budget cutoff, unconfirmed. Lost by the f698c02 eval-margin-redesign build (2026-09-18) at sn5M | ## Gains seen (candidate solves, HEAD doesn't) -- not deeply vetted |
