path: root/tests/ECM.md

# ECM position notes

Per-position metadata accumulated while investigating solve-rate losses on
the curated suites (`ecm_ringers.ep_`, `ecm_confident_quick.ep_`,
`ecm_hard_quick.ep_`) during the 2026-09-10/11 extension-taper/qsearch-prune
session. IDs match the `id "ECM.###"` labels `script`/`suite_diff.pl` print
(note: due to how the `.ep_` files pipeline `id`/`go`/`setboard`/`solution`
-- each `id` line is followed by the *previous* block's `setboard`/
`solution`, not the one textually after it -- so don't assume `grep`ing the
raw `.ep_` file for an id gives you the matching FEN without checking this).

## Methodology

A position's HEAD-vs-candidate solve flip was investigated by:

1. Reading HEAD's own `sd10` log for that position: does the PV settle on
   one move with a stable or improving score across iterations, ideally
   with a `++`/`+` confirmed-fail-high mark? Or does it flip between
   different candidate moves, land on an unconfirmed score at the very
   last iteration (especially if it abandons an already-*confirmed*
   alternative), or return a score suspiciously close to `0.00`/negative
   for the side that's supposedly winning?
2. Where available, an independent second opinion: Crafty's own search
   (`sd 15`-`sd 20`) on the same position, and/or typhoon-candidate's own
   result at a deeper fixed depth (`sd 15`) than the suite's `sd 10`.
3. A position only counts as **legit** if HEAD's own read was solid *and*
   nothing independent contradicts it. A single wobble or disagreement is
   enough to call it **flaky**.

This is a real, if informal, signal -- not a substitute for actually
re-running the suites when a change lands, just a way to avoid re-deriving
"is this specific solve-count delta noise" from scratch every time a
similar-shaped loss reappears (which it does -- most of the positions below
have shown up as a flip across *multiple*, otherwise-unrelated changes this
session).

## Legit (real tactic, HEAD understood it solidly -- a genuine loss if lost)

| ID | Suite | FEN | Solution | Why legit |
|---|---|---|---|---|
| ECM.035 | confident_quick | `2r2bk1/4qp2/3n2p1/2R1p1Np/2p1N3/r6P/1Q3PP1/3R2K1 w - -` | Rxc8 | HEAD confirms (`++`, +2.31) before finalizing at +2.79; Crafty's independent deep search converges to ~+4.6; candidate's own `sd 15` also confirms |
| ECM.080 | confident_quick | `5r1k/pb2r1bp/1p2B1p1/n7/4qNQ1/4B3/P4PPP/2RR2K1 w - -` | Rd4 | Single clean line, +3.26, fast solve, no oscillation |
| ECM.128 | hard_quick | `rn2qbr1/2p4k/p2p1nb1/1p1Pp2p/2P4P/2NBBPN1/PP1Q4/2KR2R1 w - -` | Nxh5 | Single clean line, +2.66, no oscillation |
| ECM.071 | hard_quick | `b1n1r1k1/2q2p1p/p4Pp1/1pBP4/4B1P1/2P1Q3/P4K1P/4R3 w - - 0 0` | Qh6 | Confirmed (`++`, +3.34) at depth 9, climbs monotonically to +4.74 by depth 12, same move throughout. Crafty independently finds the position winning even bigger (+7-9) but via a different move (`Kg2`, a quiet consolidation) -- doesn't contradict `Qh6`'s own soundness, just suggests it isn't the single objectively-best try. Gained by the candidate LMR-graded-curve build (2026-09-11) |
| ECM.250 | confident_quick | `2r1rbk1/5pp1/bq5p/1pnBPN2/6Q1/N7/5PPP/R2R2K1 w - -` | Bxf7+ | Single clean line, +3.31, fast solve |
| ECM.251 | hard_quick | `1k1r1r2/2b2R2/Bppp2p1/2q4p/Q2pP3/P2P4/2P3PP/1R4K1 w - -` | Bb7 | Instant solve; Crafty's own deep search independently plays Bb7 too (+1.87) |
| ECM.445 | confident_quick | `7r/2p1q1k1/1p3r1p/p1pPpPp1/2P1P1P1/PP6/3Q2KR/7R w - -` | Qxg5+ | HEAD never solves this at `sd10` at all (only flips under `sn5m`); Crafty confirms the move is objectively good and stable (+2.31 to +2.35 across depths 18-20) |
| ECM.519 | confident_quick | `1k1r2r1/p1p3q1/2p2p2/3pn3/1P1b2bB/1QN5/P4PPP/1R2RBK1 b - -` | Nf3+ | Huge, unambiguous score (+7.33), no wavering at all; candidate's own deeper search confirms and strengthens it |
| ECM.147 | confident_quick | `r2b1rk1/5qp1/2p2P2/2p3Pp/p6P/2Q5/PPP5/2KR2R1 w - - 0 0` | g6 | HEAD's own line switches ideas mid-search (Qxc5 -> g6) but the new idea gets its own `++` confirmation (depth 10, +0.75) before the final unconfirmed bump to +0.89; Crafty independently plays the same `g6` and stably converges to +0.49-0.66 across depths 19-22. Thin margin, but move-choice agreement from an independent engine is the deciding evidence -- lost by the candidate LMR-graded-curve build (2026-09-11) |
| ECM.672 | confident_quick | `5rk1/pbrn1ppp/1ppN1q2/2P1p3/3P4/1PRB4/P3QPPP/5RK1 w - - 0 0` | Bxh7+ | HEAD confirms (`++` twice, +4.09/+5.59) with the same first move stable from depth 7 on, score climbing monotonically to +6.22 at depth 10; Crafty independently plays the same move, reaching +9.5 by depth 27. Lost by the candidate LMR-graded-curve build (2026-09-11) -- notably, the *same candidate binary* run to deeper iterative depths (not suite-capped `sd10`) does eventually find `Bxh7+` too, jumping from +2.46 (depth 12, still on a `cxb6`-based line) to +7.59/+7.90 at depths 15-16. Not a permanent blind spot: the graded curve over-reduces this specific sacrifice line early enough to delay discovery by ~7-8 plies past what `sd10` allows, not miss it outright |
| ECM.721 | ringers | `4rrk1/2pn2pb/p1p1qp2/1pb1pN2/P3P1PN/1P1P4/1BP2PK1/R2Q3R w - -` | Nxg7 | Clean, unwavering, +3.19; candidate confirms/strengthens at deeper depth (+4.29) |
| ECM.733 | confident_quick | `2r3k1/pbq1np1p/1p1b1Bp1/8/6QP/2P2N2/B4PP1/4R1K1 w - -` | Bxf7+ | HEAD confirms once (`++`, +1.52); same first move stable across all entries, only the deeper continuation changes -- stronger signal than a plain unconfirmed line. Reconfirmed by Crafty (stable `Bxf7+`, +4.2-4.4, depths 18-22) during the 2026-09-11 LMR graded-curve investigation, which also lost this position |
| ECM.736 | confident_quick | `r2r2k1/1b2qpp1/1p2n3/p1b1p2p/4P1n1/PP1B2PP/1B1NQP2/R3NRK1 b - - 0 0` | Nxf2 | HEAD abandons a confirmed `Nd4` (`++`, +1.51) for `Nxf2`, unconfirmed and briefly dipping (+2.05 -> +1.93) -- looked flaky from HEAD's log alone. Crafty independently and stably plays the same `Nxf2` across 5 straight iterations (depths 24-28, -1.43 to -1.90 from Black's perspective), confirming the move choice despite HEAD's own wobble. Lost by the candidate LMR-graded-curve build (2026-09-11) |
| ECM.734 | hard_quick | `r3r1k1/1bqn1ppp/1pp2p2/8/3P4/1B4N1/PP3PPP/R2QR1K1 w - -` | Bxf7+ | Same shape as ECM.733: confirmed once (`++`, +1.74), same top move stable throughout even as the deeper line's score settles lower |
| ECM.789 | hard_quick | `2r1r1k1/pb1n1pp1/1p1qpn1p/4N1B1/2PP4/3B4/P2Q1PPP/3RR1K1 w - - 0 0` | Bxh6 | Never gets an explicit `++` mark but the move is stable across the last 3 iterations with the score climbing (1.06 -> 1.27 -> 1.69). Crafty independently confirms the same move, also growing (1.89 -> 2.06 -> 2.13). Gained by the candidate LMR-graded-curve build (2026-09-11) |
| ECM.846 | hard_quick | `3qr1k1/rp3pbp/2p3p1/3p1b2/N1nP4/P2NP3/1PQ2PPP/1BRR2K1 b - -` | Rxe3 | **Reclassified from Flaky (2026-09-11)**: this entry previously read "two unconfirmed lines, both under +0.5" from an earlier build. The candidate LMR-graded-curve build, run to real (non-node-capped) depth, stabilizes firmly on `Rxe3` from depth 11 through 15 (+0.31 -> +0.67, no further wobble). Crafty independently and stably confirms the same move across depths 25-30 (-0.82 to -1.12, its own sign convention). Two-engine, deep, stable agreement overturns the old flaky call -- the earlier read was likely just a shallower/different build catching this position mid-wobble before it settles |
| ECM.735 | hard_quick | `r3r1k1/pp3ppp/8/1Nbp3q/3NnBn1/2P5/PPQ2PPP/3R1RK1 b - -` | Ngxf2 | HEAD's own line is clean and solid; Crafty's defense uses the *other* knight (Nexf2) but that's a "which piece" quibble, not a refutation of the idea |
| ECM.803 | hard_quick | `2b2r1k/2p3pp/2Nb4/pP5q/2PP4/P4pP1/5P1P/R1BQ2K1 b - -` | Bxg3 | HEAD's line is clean; Crafty independently finds **mate in 10** starting with Bxg3 -- about as solid as confirmation gets |
| ECM.442 | hard_quick | `4qn1k/1b3rpp/pb2pB2/1pp1P2P/3p1NQ1/P2P2PB/1PP4K/5R2 w - -` | Ng6+ | Not actually a miss -- the king-file-storm-defect candidate (2026-09-14) settles on `Bh4` at `sd10`, but its own deeper iterations (depth 14-15) find and `++`-confirm `Ng6+` (+1.94, climbing to +2.47 by depth 16). A depth-budget delay, not a blind spot; see the 2026-09-14 session note below |
| ECM.746 | ringers | `r3k2r/p1qb1p2/1p2p2p/3pPpN1/P1nP3Q/8/2P2PPP/R1B1R1K1 w kq - 0 0` | Nxf7 | `++`-confirmed at depth 10 (+2.27), same move held through the final depth-10 re-search (+2.62) -- no wavering. Lost by the f698c02 eval-margin-redesign build (2026-09-18) |
| ECM.280 | confident_quick | `r1b2rk1/1pqp1ppp/p1n1pn2/5N2/8/2PBB3/P1P2PPP/R2QR1K1 w - - 0 0` | Nxg7 | Same first move (`Nxg7`) held stable across all 4 depth-8/9/10 iterations including a `++` confirm at depth 9, despite the score itself swinging (-0.29 to +0.93 to +0.89) -- move choice never wavered. Lost by the f698c02 eval-margin-redesign build (2026-09-18) |
| ECM.597 | hard_quick | `rnb2rk1/pp4pp/2pb1n2/3N1p1q/2P5/3N2P1/P1Q1PPBP/R1B2RK1 w - - 0 0` | Bf4 | Same move (`Bf4`) held stable across all 3 iterations (depths 8-10), no wavering, though never gets an explicit `++` mark. Lost by the f698c02 eval-margin-redesign build (2026-09-18) |
| ECM.339 | confident_quick | `r1b1r1k1/pp3p1p/1q2p1pQ/2b1P1B1/8/P2B3P/1P3PP1/2R1R1K1 w - -` | Rxc5 | Same shape as ECM.442/672: HEAD solves at depth 10 (310k nodes); the bishop-mobility-rewrite build (2026-10-10) still finds it, just one ply later (depth 11, 1.3M nodes) -- a depth-budget delay, not a blind spot |
| ECM.450 | confident_quick | `6r1/6rk/3p3p/p2Rp3/R1PqPp1b/2N2Q2/6PP/7K b - -` | Rxg2 | Same depth-budget-delay shape: HEAD solves before depth 10 even prints (<278k nodes); the bishop-mobility-rewrite build (2026-10-10) solves at depth 11 (454k nodes) -- later, not missed |
| ECM.496 | confident_quick | `r2q1rk1/pp1n2pp/2pb4/1P1p3n/P1PPpP2/4P2b/1BNNBP1P/R2QR1K1 b - -` | Nxf4 | Same depth-budget-delay shape: HEAD solves at depth 10 (2.0M nodes); the bishop-mobility-rewrite build (2026-10-10) solves at depth 11 (2.43M nodes). Neither build actually understands the tactic until depth 14-15 (score jumps 1.4 -> 3.1 -> 3.8) -- the depth-10 "solve" on either side is itself a shallow, barely-there read |
| ECM.672 | confident_quick | `5rk1/pbrn1ppp/1ppN1q2/2P1p3/3P4/1PRB4/P3QPPP/5RK1 w - - 0 0` | Bxh7+ | Same shape again, pushed back further this time: the rook-mobility-rewrite build (2026-10-10) never touches `Bxh7+` within the sd10 window at all (stays on a quiet `dxe5` line throughout, +1.05), unlike the bishop-only intermediate build which still solved it. Re-run to `sd 16` on the same position: `Bxh7+` appears and gets `++`-confirmed twice at depth 11 (+1.80, +3.30), climbing stably through depth 16 (+6.96) -- a 3-ply depth-budget delay, not a blind spot. Consistent with this position's existing track record (already shown depth-delay-prone under the unrelated 2026-09-11 LMR graded-curve change too, per that session's note above) |

## Flaky / garbage (suite noise -- don't read a loss here as a real regression)

| ID | Suite | FEN | "Solution" | Why flaky |
|---|---|---|---|---|
| ECM.213 | ringers | `k3r3/pR5p/PppR1p1p/4nP2/1PP5/8/4B1rP/2K5 w - -` | Re7 | Came off a draw-by-repetition line (`Kd1 ... <=>`) at only +0.37, unconfirmed |
| ECM.270 | ringers | `6k1/1p3p2/4r1p1/1PPrbq1p/2QN4/P2R1P1P/R4KP1/8 b - -` | Bg3+ | Score exactly `0.00`, single unconfirmed line; Crafty's own `sd20` search prefers Rxc5 instead |
| ECM.336 | hard_quick | `r1bqr1k1/pp3pp1/2p4p/3PN1n1/3P1b2/2NB4/PPQ2PPP/3R1RK1 b - -` | Rxe5 | Abandons a *twice*-confirmed cxd5 (swung -0.81 -> -0.16 -> +1.34) for an unconfirmed Rxe5 at the very end |
| ECM.337 | confident_quick | `r3r1k1/pbqn1ppp/1p3n2/1Pb5/2p5/5N1P/PBQ1BPP1/1N1R1RK1 b - -` | Rxe2 | Cycles through 3 different first moves within depth 10, lands unconfirmed |
| ECM.340 | confident_quick | `1k2r2r/1bpQN2p/1p6/P3p2q/4np2/2P2N2/2P2PPP/R3R1K1 b - -` | Rxe7 | Abandons a *thrice*-confirmed Qh6 (+2.49) for a near-identical unconfirmed Rxe7 (+2.76) |
| ECM.370 | hard_quick | `1r2r1k1/1pqb2pp/p2bp1n1/2p3BQ/3pN3/1P1P2P1/P1P3BP/4RRK1 w - -` | Bd8 | Score exactly `0.00` on a repetition line; Crafty prefers a different move at only +0.15 |
| ECM.443 | hard_quick | `2b5/2qrrpk1/5Rp1/2p4p/1pB1PR1P/1P1P2P1/5Q1K/8 w - -` | Qb2 | Two unconfirmed lines, thin margins (+0.92, +1.10) |
| ECM.637 | hard_quick | `r3qr1k/1b4pp/p2RQ3/Bp2p3/4p3/6P1/PPP2P1P/3R2K1 b - -` | Qh5 | Swapped a confirmed e3 (+2.18) for an unconfirmed Qh5 (+2.19) -- a coin-flip; Crafty's own eval also wobbled between -2.33 and -0.80 across nearby depths |
| ECM.715 | confident_quick | `3r1rk1/5pp1/pq1n1n1p/2pPR3/2B2P2/1PBQ2RP/P5P1/6K1 w - -` | Rxg7+ | Abandons a *twice*-confirmed Rge3 (+4.41) for a lower-scoring, unconfirmed Rxg7+ (+3.05) |
| ECM.771 | hard_quick | `3qr1k1/1br1bp1p/p3p1pB/1p1nN3/3PB3/7Q/PP3PPP/3RR1K1 w - -` | Nxf7 | HEAD never breaks +0.56, unconfirmed; Crafty's independent top choice is a different move near dead equal |
| ECM.813 | confident_quick | `r2q1rk1/pp2ppb1/3pn1pQ/3R4/2P3B1/4BR2/PP4PP/6K1 w - -` | Qxg6 | Both of HEAD's candidate lines score *negative* (-0.74, -0.69) -- HEAD's own eval doesn't think its own "solution" is good for the side supposedly winning |
| ECM.422 | hard_quick | `6k1/rnqb3p/5ppQ/2pPp3/p1N1P3/2PB3P/5PP1/1R4K1 w - - 0 0` | f4 | Same move (`f4`) held across 5 iterations but never gets a `++` confirm, thin margin (+0.70 to +0.95). Crafty independently prefers a different move (`Qh4`) at a similarly thin margin (+0.68 to +1.06) -- a genuine disagreement on a razor-thin position, not corroboration. Gained by the candidate LMR-graded-curve build (2026-09-11) but not a confident real solve |
| ECM.830 | confident_quick | `1n2r3/p1pq1kp1/1b1pNpp1/3P4/5RP1/3Q3P/1B3P2/6K1 w - -` | Bxf6 | Oscillates Nxg7/Bxf6 the whole way, no confirmation either side |
| ECM.842 | confident_quick | `r1b1k1nr/3n1p1p/1qpBp1p1/pp2b3/4PN2/PBN2Q2/1PP2PPP/2KR3R w kq - 0 0` | Nxe6 | HEAD abandons a confirmed `Bxe6` (`++`, +1.14) for an unconfirmed `Nxe6`, never reconfirmed. Crafty's own search at `sd 22` doesn't corroborate either -- wobbles between `Qg3` and `Bxe6` across depths 20-22, never playing `Nxe6`. Independent check does not support HEAD's answer, unlike ECM.147/672/736 below -- lost by the candidate LMR-graded-curve build (2026-09-11) but stays classified flaky |
| ECM.875 | hard_quick | `2kr1br1/ppp2p1p/2n1p1q1/2Nn4/3P4/1Q2PB2/PP1B1P1P/2R1K2R b K - 0 0` | Qg1+ | Score exactly `0.00` on a repetition line (`<=>`) across 6 straight iterations (depths 8-13) -- textbook flaky signature. Contradicts the "Gains seen" table's loose note calling this Crafty-confirmed legit in an earlier review; that note appears to have been wrong, or about a different search reaching a different (non-drawing) conclusion. Gained by the candidate LMR-graded-curve build (2026-09-11) |
| ECM.879 | hard_quick | `r2r2k1/pp3ppp/2q5/2p5/2P1P1n1/P4P1b/1BQ1BP1P/R3R2K b - - 0 0` | Rd2 | Score exactly `0.00` on a repetition line (`<=>`) -- same signature as ECM.875. Gained by the candidate LMR-graded-curve build (2026-09-11) |
| ECM.866 | hard_quick | `7R/1p1r4/2b2p2/2Pp1qk1/3P1bp1/3N4/2Q2PB1/6K1 w - -` | Rg8+ | HEAD's own score is negative (-0.12 to -0.15); Crafty's independent top choice (Qd1) is also near-zero. **Note**: has shown up as a *gain* (candidate solves it, HEAD doesn't) in at least one later run too -- consistent with a genuinely coin-flip position, not a stable read either direction |
| ECM.360 | hard_quick | `2r3k1/p1r1ppb1/2bp1np1/q5N1/1p1BP2Q/3P4/PP2N1PP/2R2R1K w - -` | Bxf6 | Wobbles Ra1 -> Bxf6 (depth 10, matching "solution") -> Ra1 (depth 11) -> Ra1 `++`-confirmed (depth 12) -> Nf4 (depth 12, second entry) -> Nf4 stable through depth 15. A separate `go` run at a 30s time control (depths 17-22) settles firmly and stably on Nf4, never touching Bxf6 -- the sd10 gate just sampled the one brief window sitting on the "solution" move mid-wobble. Gained by the pawn-eval isolation/backward-pawn session (2026-09-19) but not a real solve |
| ECM.420 | hard_quick | `3rr1k1/1p1qb2p/pP4p1/2p2p2/4RQ2/3P4/1PPB2PP/4R1K1 w - -` | Bc3 | Wobbles Re6 (depths 8-10 first entry) -> Bc3 (depth 10 second entry, matching "solution", through depth 12 first entry) -> Re6 (depth 12 second entry) -> Re6 stable through depth 15. A deeper run (depths 16-18) confirms Re6 firmly, with `Bc3+` only ever appearing a few plies into the Re6 line (as a follow-up check: `...Rxe8+ Qxe8 Qxc5 Bxc3...`), not as the intended root move. Gained by the pawn-eval isolation/backward-pawn session (2026-09-19) but not a real solve |
| ECM.357 | confident_quick | `r3r1k1/4pp1p/b5pB/q1pP4/3b4/2N4P/PP4P1/R2RQ2K w - - 0 0` | Be3 | Abandons a `++`-confirmed Be3 (depth 10, +0.70) for an unconfirmed Rxd4 (+0.80) at the very last iteration -- textbook flaky signature. Lost by the f698c02 eval-margin-redesign build (2026-09-18) |
| ECM.717 | ringers | `r1qb1r1k/2p3pp/p1n1bp2/1p1Np2Q/P3P3/1BP3R1/1P3PPP/R1B3K1 w - - 0 0` | axb5 | Abandons a `++`-confirmed axb5 (depth 10, +3.16) for an unconfirmed Rxg7 (+3.17) at the very last iteration -- same shape as ECM.357. Lost by the f698c02 eval-margin-redesign build (2026-09-18) |
| ECM.601 | hard_quick | `8/p5k1/2p2p2/4p1p1/1p2P3/1P4PP/1PP5/5K2 b - - 0 0` | g4 | No iteration lines printed at all before the depth limit; final score a dead-equal +0.07 -- no confirmation of any kind. Lost by the f698c02 eval-margin-redesign build (2026-09-18) |
| ECM.193 | hard_quick | `r2r1bk1/3qp2p/3pp1p1/p2n2N1/2N3Q1/BP4P1/P4PP1/2R3K1 w - - 0 0` | Nb6 | Wavers between Qh3/Qf3 through most iterations, lands on Nb6 only at the sn5M node-budget cutoff, unconfirmed. Lost by the f698c02 eval-margin-redesign build (2026-09-18) at sn5M |
| ECM.770 | hard_quick | `rnb2rk1/pp3pbp/2p3pB/2q5/4P3/1BN5/PPP3PP/R2Q1R1K w - -` | Rxf7 | HEAD locks onto (an unconfirmed but stable) `Rxf7` through depth 12, then abandons it for `Bf4` from depth 12 (second entry) through depth 16+ (+2.60 -> +2.98 -> +2.57). **Crafty independently confirms `Bf4`, not `Rxf7`**, stably from depth 22 through 28 (+1.78 -> +2.61, six-plus stable iterations) -- the labeled "solution" appears to be wrong, not a search miss. Same wobble reproduced identically under both `JOINT_EXTRA_TIER1_THOUSANDTHS=72` and `=150` (2026-09-30 threshold-sweep investigation), confirming it predates and is unrelated to that change |
| ECM.826 | confident_quick | `1rbq2k1/b1p3pp/p1Pp4/3Ppr2/8/5N2/PPQ2PPP/R1B2RK1 b - -` | Rxf3 | Sits on `Bd4` (-0.73) through depth 10, only switches to (and then stably confirms) `Rxf3` at depth 11-14. Identical behavior, same scores, same switch point under both `JOINT_EXTRA_TIER1_THOUSANDTHS=72` and `=150` (2026-09-30) -- a depth-budget/horizon effect on both thresholds, not a regression |
| ECM.491 | hard_quick | `1r2r1k1/p4ppb/q1pbnn1p/4p3/N1P1P3/pPB2NP1/2Q1RPBP/R5K1 b - -` | Nxe4 | Wobbles `c5` (depths 9-11) -> `Bxe4` (depth 11-13) -> `Nxe4` (matching "solution", depth 13-14), never settling before the sd10/sn5M test budget. Identical wobble, same depths, same scores under both `JOINT_EXTRA_TIER1_THOUSANDTHS=72` and `=150` (2026-09-30) -- predates the threshold change |
| ECM.298 | hard_quick | `2b1qr1k/7p/p1pR2p1/1pP1p1Q1/4P3/1B6/PP4PP/6K1 w - -` | Re6 | Re-checked under the rook-mobility-rewrite build (2026-10-10): both HEAD (`Re6`, matching the suite label) and the new build (`Rf6`) print *zero* per-iteration lines before hitting the depth limit -- the exact "no iteration lines printed at all, no confirmation of any kind" signature this doc already calls flaky elsewhere (see ECM.601). HEAD's own "solve" here is just as unconfirmed as the new build's "miss" -- not a real loss. (Separately, an unrelated 2026-09-14 session found a *different* candidate playing `h4` here, independently confirmed stronger than the suite's own labeled `Re6` by a deep Crafty check -- see that session's note below. The suite's expected answer for this position may simply be wrong.) |
| ECM.186 | hard_quick | `1R3b2/3r3k/2p1bp1p/r1q1pNpQ/2PpP2P/6B1/P4PP1/1R4K1 w - -` | Nxh6 | Same depth-budget-delay shape as 339/450/496/672: HEAD solves at depth 11 (+5.75), climbing cleanly through depth 13 (`++` confirmed, +8.13). The queen-mobility-rewrite build (2026-10-10, bishop+rook+queen combined) still prefers `Qe8` at depth 11, but switches to and solves with `Nxh6` at depth 12 (+7.26) -- pushed out one ply, not missed |

## Gains seen (candidate solves, HEAD doesn't) -- not deeply vetted

These showed up as improvements across various runs. Only spot-checked, not
put through the full methodology above -- listed for reference, not as a
confirmed-legit set the way the "Legit" table above is.

| ID | Suite | Notes |
|---|---|---|
| ECM.193 | hard_quick | Crafty-confirmed legit in earlier review |
| ECM.476 | hard_quick | Crafty-confirmed legit in earlier review |
| ECM.544 | hard_quick | Crafty-confirmed legit in earlier review |
| ECM.613 | hard_quick | Seen as a gain across multiple runs |
| ECM.716 | confident_quick | Seen as a gain across multiple runs |
| ECM.779 | hard_quick | Seen as a gain across multiple runs |
| ECM.822 | hard_quick | Seen as a gain across multiple runs |
| ECM.875 | hard_quick | Seen as a gain across multiple runs |
| ECM.489 | hard_quick | Seen once |

## 2026-09-11 LMR graded-curve session

A separate investigation replaced typhoon's flat `-ONE_PLY` LMR reduction
with a graded depth/move-number curve (fit to this engine's own measured
tail-fail-high-rate data, not a borrowed formula -- see `dynamic.c`'s
`InitLMRTable`), plus a QSearch-drop guard in `GetLMRReduction` (never
reduce far enough to land below `THREE_QUARTERS_PLY` remaining depth).
Measured against `head_reference` (`ddf932b`) on `confident_quick`:
`sd10` net -4/+2 (net -3 across all three suites combined), `sn5000000`
net -3/+3 depth-adjusted (net +4 combined, `hard_quick` alone +7 with
+0.5 extra ply -- a real depth-payoff signal).

Re-investigated all 6 `confident_quick` losses (identical set at both
`sd10` and `sn5m` -- a reproducible effect, not scattered noise) and both
gains against `head_reference`'s own log plus a fresh independent Crafty
check (`sd 22`, not run through the full methodology before this
session). Net result: **4 of 6 losses are legit** (733, 672, 147, 736 --
147 and 736 were previously unclassified and looked flaky from HEAD's
log alone, but Crafty's independent move-choice agreement overturned
that read), only 2 are flaky (715, 842). Both gains (519, 035) are
already-established legit. That's a real, non-noise net-negative
tactical cost on `confident_quick` specifically -- the aggregate
suite-level numbers understated this at first glance (`sd10` -4 raw
looked mostly explainable as noise until the per-position Crafty check
was actually run).

ECM.672 is worth remembering for future LMR work specifically: the same
candidate binary, given enough iterative depth (not suite-capped
`sd10`), *does* eventually find the correct `Bxh7+` sacrifice (depth
15-16, jumping from +2.46 to +7.59/+7.90) after several iterations
spent on an unrelated `cxb6`-based line. The graded curve isn't blind to
this tactic, it's delaying discovery by roughly 7-8 plies past what a
depth-10 budget allows -- a genuine cost at fixed/shallow search depths,
but not the kind of permanent blind spot a bad exemption rule would
produce. Worth checking whether this delay is concentrated in
sacrifice/attacking lines specifically (where the "safe to reduce"
heuristics this curve is built from may be least trustworthy) before
concluding the curve needs a different shape rather than just deeper
search to pay off.

**`hard_quick`'s `sn5000000` gains investigated too** (11 gains, 4 losses
-- 193, 476, 779, 866, not re-investigated here since they're losses on
the suite that was already net-positive). Of the 11 gains: **7 are legit**
(071, 251, 734, 735, 789, 803, 846 -- 846 is a reclassification, see its
updated Legit-table entry above: an earlier build's shallower read
tagged it flaky, but this candidate's deeper search plus Crafty both
stably converge on the same move), **3 are flaky** (422 -- genuine
engine-vs-engine disagreement on a thin margin, not corroboration; 875,
879 -- both exact `0.00` on a repetition line), and **1 is genuinely
unresolved** (640): the candidate's own extended search twice-confirms
`Rxb2` at +3.66 (the `sn5m`-capped run's PV/played-move mismatch was
just the node budget cutting off *before* this switch, not a bug), but
Crafty independently searches to depth 30 and never once prefers
`Rxb2`, staying on a different move (`Bd8`) near dead equal. Two
real-depth searches disagreeing after real depth is a different, more
interesting kind of uncertainty than the usual flaky signature (score
exactly 0, unconfirmed thin margins) -- worth a dedicated line-by-line
look at the `Rxb2` continuation before counting it either way.

**Combined verdict across both suites, real (non-flaky) positions only**:
9 legit gains (035, 519, 071, 251, 734, 735, 789, 803, 846) vs. 4 legit
losses (733, 672, 147, 736), with ECM.640 unresolved. A materially more
favorable picture than the raw suite aggregates suggested on their own
-- this is why the per-position Crafty-corroboration check matters more
than trusting `sd10`/`sn5000000` solve counts at face value, especially
at this suite size.

## 2026-09-14 king-safety file-storm-defect session

Retuned `eval.c`'s `KingFileDefects`/`KingStormingPawnDefects` tables
(inside `_ComputeFileStormDefects`) per `eval_tune/king_safety_data/HANDOFF.md`'s
tier-1 finding: `{+2,+1,+0}` -> `{+4,+2,+0}` and
`{+0,+4,+3,+1,+0,+0,+0,+0}` -> `{+0,+7,+5,+2,+0,+0,+0,+0}`, a real-game
mate-mining logistic regression's two most dominant, most stable
predictors, previously weighted the same as every other hand-picked
king-safety defect. Measured against `head_reference` (`ddf932b`) at
`sd10`: `ecm_ringers` -2, `ecm_confident_quick` -1, `ecm_hard_quick` -1.

Per-position investigation (same methodology as the LMR session above)
found almost none of this is real:
- `ECM.715`, `ECM.866` (losses) and `ECM.813`, `ECM.443`, `ECM.771`
  (gains) were already tagged flaky above.
- `ECM.716`, `ECM.193`, `ECM.476` (losses) were already listed in the
  "Gains seen" table from other sessions -- a position that has already
  flip-flopped direction before isn't new evidence of a regression.
- `ECM.501`, `ECM.801` (ringers losses, previously unclassified): both
  show HEAD *itself* wobbling between candidate moves and landing on its
  final answer only at the last iteration with no `++` confirmation
  (`ECM.501`: Rf7 -> Rc8 -> Rf7 -> Rb8 -> Nxf3; `ECM.801`: Nd4 held
  stably for 3 iterations then an unconfirmed jump to Nh6+ only at
  depth 10) -- textbook flaky signature, not a real loss either
  direction.
- `ECM.442` (hard_quick loss): not a miss at all -- see its updated
  entry in the Legit table above. The candidate's own deeper iterations
  (depth 14-15) find and `++`-confirm the same `Ng6+` HEAD plays at
  `sd10`; the `sd10` "loss" is purely a depth-budget delay (climbing
  from +0.91 at depth 10 to a confirmed +1.94 by depth 14), the same
  shape as `ECM.672`'s entry above.
- `ECM.298` (hard_quick loss, previously unclassified): HEAD converges
  cleanly on `Re6`/`Rf6` (+2.7 to +2.9) and the suite's labeled solution
  is `Re6`, but the candidate instead finds and `++`-confirms `h4`
  (+5.49 by depth 11, climbing to +8.87 by depth 17) -- and Crafty's own
  independent deep search (`sd 27`, ~9.3s to switch onto it after
  spending depths 20-23 stuck on `Rf6` at only +2.9-3.2) agrees `h4` is
  objectively much stronger (+10.17) than the suite's labeled answer.
  Notably the candidate found this line in 2.5s vs. Crafty's 9.3s.
  Not a regression -- if anything, evidence the candidate is finding a
  better resource than the suite's own labeled solution, faster than an
  independent reference engine. Worth relabeling this suite's expected
  answer if it's ever revisited, rather than trusting `Re6` as ground
  truth.

**Net read**: of the 4 nominal net solve losses (-2/-1/-1 across the
three suites), 7 of 9 individual position losses were already-known or
freshly-confirmed noise, one (`ECM.442`) is a depth-budget artifact, and
the last (`ECM.298`) is arguably a candidate improvement mislabeled as a
loss by the suite's fixed expected answer. No surviving evidence of a
real tactical regression from this change at the position level, despite
the suite-aggregate solve counts looking mildly negative -- another data
point for not trusting raw `sd10` solve deltas on suites this size
without the per-position check.

Self-play (`match_play.py`, ongoing) at 110 games: -29 =47 +34
(score 0.523, llr=+0.11, still far from either SPRT bound) -- too early
to call, but notably `avg depth: candidate=9.96 head=10.25` (candidate
searching ~0.3 ply shallower at the same `st` budget) with
`avg ebf: candidate=4.442 head=4.396` (branching factor slightly
*higher*, i.e. the tree got slightly fatter, not narrower) -- a
non-trivial shape change in the search tree from what was purely an
eval-weight retune around king safety, presumably via move-ordering/
pruning decisions that read king danger differently now (e.g. anything
gated on `KING_SAFETY_BY_COUNTER`'s bucket or a null-move/LMR
verification threshold sensitive to eval score swings). Worth keeping
in mind for future king-safety retunes: even a "just the numbers"
change to an eval sub-term can visibly shift average search depth, not
just leaf-node scores.

## Caveat

This list is a snapshot from one session's worth of investigation, not a
permanent classification -- a position flagged "flaky" here reflects HEAD's
behavior *at the commit this session started from* (`9d1e32d`) plus
whatever candidate builds were tested through it. If the eval or search
changes significantly later, re-derive rather than trust this table blindly
-- especially the "Gains" table, which was never run through the full
methodology in the first place.