summaryrefslogtreecommitdiff
path: root/tests/ECM.md
diff options
context:
space:
mode:
authorScott Gasch <[email protected]>2026-09-13 23:18:07 -0700
committerScott Gasch <[email protected]>2026-09-13 23:18:07 -0700
commit4ff9170fd96e3da049c747bf5215e908a94ff145 (patch)
tree6c8e6f846b42518d84357bb3b83014feddc83202 /tests/ECM.md
parent78d1889fe38db1fd45b58509c8b3bd535ce691b7 (diff)
Add king-safety mate-mining regression pipeline; retune file-storm/
queen-proximity king-safety weights and add a back-flight bonus from it. Mines real human-game checkmates and matched sound-draw controls (eval_tune/mine_king_mates.py, shard_mine.py, extract_king_features.py, match_controls.py, train_king_safety_model.py), then fits a logistic regression against _EvalKing's raw per-defect breakdown (engine-side support: chess.h's KING_DEFECT_DUMP_INFO, eval.c's KING_DEFECT_DUMP instrumentation, command.c's evalkingdump command, all gated behind the KING_DEFECT_DUMP=1 build flag) to see which defects actually predict getting mated. See eval_tune/king_safety_data/HANDOFF.md for the full methodology and findings. Two rounds of retuning landed from this data so far, both validated via sd10 curated-suite comparison plus per-position Crafty corroboration (tests/ECM.md) and gated self-play: - KingFileDefects/KingStormingPawnDefects (_ComputeFileStormDefects) bumped ~1.75-2x relative to the rest of the ring-scan defects -- the regression's most dominant, most stable tier-1 predictors, previously weighted the same as every other hand-picked defect. - KING_QUEEN_PROXIMITY_DANGER rescaled from -8/count to -32/count, and a new KING_BACK_FLIGHT_BONUS (+1/square) term added for empty, unattacked ring squares on the side of the king away from the attacker -- a real, non-collinear protective signal the current EmptyUnattackedSquare table didn't credit at all. Both calibrated by unstandardizing the regression's coefficients and anchoring against KING_MISSING_ONE_CASTLE_OPTION's already-live -23 value (same direct-EVAL_TERM unit family). Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01P9EDZVcqu6E2bNRm4zDrTf
Diffstat (limited to 'tests/ECM.md')
-rw-r--r--tests/ECM.md246
1 files changed, 246 insertions, 0 deletions
diff --git a/tests/ECM.md b/tests/ECM.md
new file mode 100644
index 0000000..5ff1ed0
--- /dev/null
+++ b/tests/ECM.md
@@ -0,0 +1,246 @@
+# ECM position notes
+
+Per-position metadata accumulated while investigating solve-rate losses on
+the curated suites (`ecm_ringers.ep_`, `ecm_confident_quick.ep_`,
+`ecm_hard_quick.ep_`) during the 2026-09-10/11 extension-taper/qsearch-prune
+session. IDs match the `id "ECM.###"` labels `script`/`suite_diff.pl` print
+(note: due to how the `.ep_` files pipeline `id`/`go`/`setboard`/`solution`
+-- each `id` line is followed by the *previous* block's `setboard`/
+`solution`, not the one textually after it -- so don't assume `grep`ing the
+raw `.ep_` file for an id gives you the matching FEN without checking this).
+
+## Methodology
+
+A position's HEAD-vs-candidate solve flip was investigated by:
+
+1. Reading HEAD's own `sd10` log for that position: does the PV settle on
+ one move with a stable or improving score across iterations, ideally
+ with a `++`/`+` confirmed-fail-high mark? Or does it flip between
+ different candidate moves, land on an unconfirmed score at the very
+ last iteration (especially if it abandons an already-*confirmed*
+ alternative), or return a score suspiciously close to `0.00`/negative
+ for the side that's supposedly winning?
+2. Where available, an independent second opinion: Crafty's own search
+ (`sd 15`-`sd 20`) on the same position, and/or typhoon-candidate's own
+ result at a deeper fixed depth (`sd 15`) than the suite's `sd 10`.
+3. A position only counts as **legit** if HEAD's own read was solid *and*
+ nothing independent contradicts it. A single wobble or disagreement is
+ enough to call it **flaky**.
+
+This is a real, if informal, signal -- not a substitute for actually
+re-running the suites when a change lands, just a way to avoid re-deriving
+"is this specific solve-count delta noise" from scratch every time a
+similar-shaped loss reappears (which it does -- most of the positions below
+have shown up as a flip across *multiple*, otherwise-unrelated changes this
+session).
+
+## Legit (real tactic, HEAD understood it solidly -- a genuine loss if lost)
+
+| ID | Suite | FEN | Solution | Why legit |
+|---|---|---|---|---|
+| ECM.035 | confident_quick | `2r2bk1/4qp2/3n2p1/2R1p1Np/2p1N3/r6P/1Q3PP1/3R2K1 w - -` | Rxc8 | HEAD confirms (`++`, +2.31) before finalizing at +2.79; Crafty's independent deep search converges to ~+4.6; candidate's own `sd 15` also confirms |
+| ECM.080 | confident_quick | `5r1k/pb2r1bp/1p2B1p1/n7/4qNQ1/4B3/P4PPP/2RR2K1 w - -` | Rd4 | Single clean line, +3.26, fast solve, no oscillation |
+| ECM.128 | hard_quick | `rn2qbr1/2p4k/p2p1nb1/1p1Pp2p/2P4P/2NBBPN1/PP1Q4/2KR2R1 w - -` | Nxh5 | Single clean line, +2.66, no oscillation |
+| ECM.071 | hard_quick | `b1n1r1k1/2q2p1p/p4Pp1/1pBP4/4B1P1/2P1Q3/P4K1P/4R3 w - - 0 0` | Qh6 | Confirmed (`++`, +3.34) at depth 9, climbs monotonically to +4.74 by depth 12, same move throughout. Crafty independently finds the position winning even bigger (+7-9) but via a different move (`Kg2`, a quiet consolidation) -- doesn't contradict `Qh6`'s own soundness, just suggests it isn't the single objectively-best try. Gained by the candidate LMR-graded-curve build (2026-09-11) |
+| ECM.250 | confident_quick | `2r1rbk1/5pp1/bq5p/1pnBPN2/6Q1/N7/5PPP/R2R2K1 w - -` | Bxf7+ | Single clean line, +3.31, fast solve |
+| ECM.251 | hard_quick | `1k1r1r2/2b2R2/Bppp2p1/2q4p/Q2pP3/P2P4/2P3PP/1R4K1 w - -` | Bb7 | Instant solve; Crafty's own deep search independently plays Bb7 too (+1.87) |
+| ECM.445 | confident_quick | `7r/2p1q1k1/1p3r1p/p1pPpPp1/2P1P1P1/PP6/3Q2KR/7R w - -` | Qxg5+ | HEAD never solves this at `sd10` at all (only flips under `sn5m`); Crafty confirms the move is objectively good and stable (+2.31 to +2.35 across depths 18-20) |
+| ECM.519 | confident_quick | `1k1r2r1/p1p3q1/2p2p2/3pn3/1P1b2bB/1QN5/P4PPP/1R2RBK1 b - -` | Nf3+ | Huge, unambiguous score (+7.33), no wavering at all; candidate's own deeper search confirms and strengthens it |
+| ECM.147 | confident_quick | `r2b1rk1/5qp1/2p2P2/2p3Pp/p6P/2Q5/PPP5/2KR2R1 w - - 0 0` | g6 | HEAD's own line switches ideas mid-search (Qxc5 -> g6) but the new idea gets its own `++` confirmation (depth 10, +0.75) before the final unconfirmed bump to +0.89; Crafty independently plays the same `g6` and stably converges to +0.49-0.66 across depths 19-22. Thin margin, but move-choice agreement from an independent engine is the deciding evidence -- lost by the candidate LMR-graded-curve build (2026-09-11) |
+| ECM.672 | confident_quick | `5rk1/pbrn1ppp/1ppN1q2/2P1p3/3P4/1PRB4/P3QPPP/5RK1 w - - 0 0` | Bxh7+ | HEAD confirms (`++` twice, +4.09/+5.59) with the same first move stable from depth 7 on, score climbing monotonically to +6.22 at depth 10; Crafty independently plays the same move, reaching +9.5 by depth 27. Lost by the candidate LMR-graded-curve build (2026-09-11) -- notably, the *same candidate binary* run to deeper iterative depths (not suite-capped `sd10`) does eventually find `Bxh7+` too, jumping from +2.46 (depth 12, still on a `cxb6`-based line) to +7.59/+7.90 at depths 15-16. Not a permanent blind spot: the graded curve over-reduces this specific sacrifice line early enough to delay discovery by ~7-8 plies past what `sd10` allows, not miss it outright |
+| ECM.721 | ringers | `4rrk1/2pn2pb/p1p1qp2/1pb1pN2/P3P1PN/1P1P4/1BP2PK1/R2Q3R w - -` | Nxg7 | Clean, unwavering, +3.19; candidate confirms/strengthens at deeper depth (+4.29) |
+| ECM.733 | confident_quick | `2r3k1/pbq1np1p/1p1b1Bp1/8/6QP/2P2N2/B4PP1/4R1K1 w - -` | Bxf7+ | HEAD confirms once (`++`, +1.52); same first move stable across all entries, only the deeper continuation changes -- stronger signal than a plain unconfirmed line. Reconfirmed by Crafty (stable `Bxf7+`, +4.2-4.4, depths 18-22) during the 2026-09-11 LMR graded-curve investigation, which also lost this position |
+| ECM.736 | confident_quick | `r2r2k1/1b2qpp1/1p2n3/p1b1p2p/4P1n1/PP1B2PP/1B1NQP2/R3NRK1 b - - 0 0` | Nxf2 | HEAD abandons a confirmed `Nd4` (`++`, +1.51) for `Nxf2`, unconfirmed and briefly dipping (+2.05 -> +1.93) -- looked flaky from HEAD's log alone. Crafty independently and stably plays the same `Nxf2` across 5 straight iterations (depths 24-28, -1.43 to -1.90 from Black's perspective), confirming the move choice despite HEAD's own wobble. Lost by the candidate LMR-graded-curve build (2026-09-11) |
+| ECM.734 | hard_quick | `r3r1k1/1bqn1ppp/1pp2p2/8/3P4/1B4N1/PP3PPP/R2QR1K1 w - -` | Bxf7+ | Same shape as ECM.733: confirmed once (`++`, +1.74), same top move stable throughout even as the deeper line's score settles lower |
+| ECM.789 | hard_quick | `2r1r1k1/pb1n1pp1/1p1qpn1p/4N1B1/2PP4/3B4/P2Q1PPP/3RR1K1 w - - 0 0` | Bxh6 | Never gets an explicit `++` mark but the move is stable across the last 3 iterations with the score climbing (1.06 -> 1.27 -> 1.69). Crafty independently confirms the same move, also growing (1.89 -> 2.06 -> 2.13). Gained by the candidate LMR-graded-curve build (2026-09-11) |
+| ECM.846 | hard_quick | `3qr1k1/rp3pbp/2p3p1/3p1b2/N1nP4/P2NP3/1PQ2PPP/1BRR2K1 b - -` | Rxe3 | **Reclassified from Flaky (2026-09-11)**: this entry previously read "two unconfirmed lines, both under +0.5" from an earlier build. The candidate LMR-graded-curve build, run to real (non-node-capped) depth, stabilizes firmly on `Rxe3` from depth 11 through 15 (+0.31 -> +0.67, no further wobble). Crafty independently and stably confirms the same move across depths 25-30 (-0.82 to -1.12, its own sign convention). Two-engine, deep, stable agreement overturns the old flaky call -- the earlier read was likely just a shallower/different build catching this position mid-wobble before it settles |
+| ECM.735 | hard_quick | `r3r1k1/pp3ppp/8/1Nbp3q/3NnBn1/2P5/PPQ2PPP/3R1RK1 b - -` | Ngxf2 | HEAD's own line is clean and solid; Crafty's defense uses the *other* knight (Nexf2) but that's a "which piece" quibble, not a refutation of the idea |
+| ECM.803 | hard_quick | `2b2r1k/2p3pp/2Nb4/pP5q/2PP4/P4pP1/5P1P/R1BQ2K1 b - -` | Bxg3 | HEAD's line is clean; Crafty independently finds **mate in 10** starting with Bxg3 -- about as solid as confirmation gets |
+| ECM.442 | hard_quick | `4qn1k/1b3rpp/pb2pB2/1pp1P2P/3p1NQ1/P2P2PB/1PP4K/5R2 w - -` | Ng6+ | Not actually a miss -- the king-file-storm-defect candidate (2026-09-14) settles on `Bh4` at `sd10`, but its own deeper iterations (depth 14-15) find and `++`-confirm `Ng6+` (+1.94, climbing to +2.47 by depth 16). A depth-budget delay, not a blind spot; see the 2026-09-14 session note below |
+
+## Flaky / garbage (suite noise -- don't read a loss here as a real regression)
+
+| ID | Suite | FEN | "Solution" | Why flaky |
+|---|---|---|---|---|
+| ECM.213 | ringers | `k3r3/pR5p/PppR1p1p/4nP2/1PP5/8/4B1rP/2K5 w - -` | Re7 | Came off a draw-by-repetition line (`Kd1 ... <=>`) at only +0.37, unconfirmed |
+| ECM.270 | ringers | `6k1/1p3p2/4r1p1/1PPrbq1p/2QN4/P2R1P1P/R4KP1/8 b - -` | Bg3+ | Score exactly `0.00`, single unconfirmed line; Crafty's own `sd20` search prefers Rxc5 instead |
+| ECM.336 | hard_quick | `r1bqr1k1/pp3pp1/2p4p/3PN1n1/3P1b2/2NB4/PPQ2PPP/3R1RK1 b - -` | Rxe5 | Abandons a *twice*-confirmed cxd5 (swung -0.81 -> -0.16 -> +1.34) for an unconfirmed Rxe5 at the very end |
+| ECM.337 | confident_quick | `r3r1k1/pbqn1ppp/1p3n2/1Pb5/2p5/5N1P/PBQ1BPP1/1N1R1RK1 b - -` | Rxe2 | Cycles through 3 different first moves within depth 10, lands unconfirmed |
+| ECM.340 | confident_quick | `1k2r2r/1bpQN2p/1p6/P3p2q/4np2/2P2N2/2P2PPP/R3R1K1 b - -` | Rxe7 | Abandons a *thrice*-confirmed Qh6 (+2.49) for a near-identical unconfirmed Rxe7 (+2.76) |
+| ECM.370 | hard_quick | `1r2r1k1/1pqb2pp/p2bp1n1/2p3BQ/3pN3/1P1P2P1/P1P3BP/4RRK1 w - -` | Bd8 | Score exactly `0.00` on a repetition line; Crafty prefers a different move at only +0.15 |
+| ECM.443 | hard_quick | `2b5/2qrrpk1/5Rp1/2p4p/1pB1PR1P/1P1P2P1/5Q1K/8 w - -` | Qb2 | Two unconfirmed lines, thin margins (+0.92, +1.10) |
+| ECM.637 | hard_quick | `r3qr1k/1b4pp/p2RQ3/Bp2p3/4p3/6P1/PPP2P1P/3R2K1 b - -` | Qh5 | Swapped a confirmed e3 (+2.18) for an unconfirmed Qh5 (+2.19) -- a coin-flip; Crafty's own eval also wobbled between -2.33 and -0.80 across nearby depths |
+| ECM.715 | confident_quick | `3r1rk1/5pp1/pq1n1n1p/2pPR3/2B2P2/1PBQ2RP/P5P1/6K1 w - -` | Rxg7+ | Abandons a *twice*-confirmed Rge3 (+4.41) for a lower-scoring, unconfirmed Rxg7+ (+3.05) |
+| ECM.771 | hard_quick | `3qr1k1/1br1bp1p/p3p1pB/1p1nN3/3PB3/7Q/PP3PPP/3RR1K1 w - -` | Nxf7 | HEAD never breaks +0.56, unconfirmed; Crafty's independent top choice is a different move near dead equal |
+| ECM.813 | confident_quick | `r2q1rk1/pp2ppb1/3pn1pQ/3R4/2P3B1/4BR2/PP4PP/6K1 w - -` | Qxg6 | Both of HEAD's candidate lines score *negative* (-0.74, -0.69) -- HEAD's own eval doesn't think its own "solution" is good for the side supposedly winning |
+| ECM.422 | hard_quick | `6k1/rnqb3p/5ppQ/2pPp3/p1N1P3/2PB3P/5PP1/1R4K1 w - - 0 0` | f4 | Same move (`f4`) held across 5 iterations but never gets a `++` confirm, thin margin (+0.70 to +0.95). Crafty independently prefers a different move (`Qh4`) at a similarly thin margin (+0.68 to +1.06) -- a genuine disagreement on a razor-thin position, not corroboration. Gained by the candidate LMR-graded-curve build (2026-09-11) but not a confident real solve |
+| ECM.830 | confident_quick | `1n2r3/p1pq1kp1/1b1pNpp1/3P4/5RP1/3Q3P/1B3P2/6K1 w - -` | Bxf6 | Oscillates Nxg7/Bxf6 the whole way, no confirmation either side |
+| ECM.842 | confident_quick | `r1b1k1nr/3n1p1p/1qpBp1p1/pp2b3/4PN2/PBN2Q2/1PP2PPP/2KR3R w kq - 0 0` | Nxe6 | HEAD abandons a confirmed `Bxe6` (`++`, +1.14) for an unconfirmed `Nxe6`, never reconfirmed. Crafty's own search at `sd 22` doesn't corroborate either -- wobbles between `Qg3` and `Bxe6` across depths 20-22, never playing `Nxe6`. Independent check does not support HEAD's answer, unlike ECM.147/672/736 below -- lost by the candidate LMR-graded-curve build (2026-09-11) but stays classified flaky |
+| ECM.875 | hard_quick | `2kr1br1/ppp2p1p/2n1p1q1/2Nn4/3P4/1Q2PB2/PP1B1P1P/2R1K2R b K - 0 0` | Qg1+ | Score exactly `0.00` on a repetition line (`<=>`) across 6 straight iterations (depths 8-13) -- textbook flaky signature. Contradicts the "Gains seen" table's loose note calling this Crafty-confirmed legit in an earlier review; that note appears to have been wrong, or about a different search reaching a different (non-drawing) conclusion. Gained by the candidate LMR-graded-curve build (2026-09-11) |
+| ECM.879 | hard_quick | `r2r2k1/pp3ppp/2q5/2p5/2P1P1n1/P4P1b/1BQ1BP1P/R3R2K b - - 0 0` | Rd2 | Score exactly `0.00` on a repetition line (`<=>`) -- same signature as ECM.875. Gained by the candidate LMR-graded-curve build (2026-09-11) |
+| ECM.866 | hard_quick | `7R/1p1r4/2b2p2/2Pp1qk1/3P1bp1/3N4/2Q2PB1/6K1 w - -` | Rg8+ | HEAD's own score is negative (-0.12 to -0.15); Crafty's independent top choice (Qd1) is also near-zero. **Note**: has shown up as a *gain* (candidate solves it, HEAD doesn't) in at least one later run too -- consistent with a genuinely coin-flip position, not a stable read either direction |
+
+## Gains seen (candidate solves, HEAD doesn't) -- not deeply vetted
+
+These showed up as improvements across various runs. Only spot-checked, not
+put through the full methodology above -- listed for reference, not as a
+confirmed-legit set the way the "Legit" table above is.
+
+| ID | Suite | Notes |
+|---|---|---|
+| ECM.193 | hard_quick | Crafty-confirmed legit in earlier review |
+| ECM.476 | hard_quick | Crafty-confirmed legit in earlier review |
+| ECM.544 | hard_quick | Crafty-confirmed legit in earlier review |
+| ECM.613 | hard_quick | Seen as a gain across multiple runs |
+| ECM.716 | confident_quick | Seen as a gain across multiple runs |
+| ECM.779 | hard_quick | Seen as a gain across multiple runs |
+| ECM.822 | hard_quick | Seen as a gain across multiple runs |
+| ECM.875 | hard_quick | Seen as a gain across multiple runs |
+| ECM.489 | hard_quick | Seen once |
+
+## 2026-09-11 LMR graded-curve session
+
+A separate investigation replaced typhoon's flat `-ONE_PLY` LMR reduction
+with a graded depth/move-number curve (fit to this engine's own measured
+tail-fail-high-rate data, not a borrowed formula -- see `dynamic.c`'s
+`InitLMRTable`), plus a QSearch-drop guard in `GetLMRReduction` (never
+reduce far enough to land below `THREE_QUARTERS_PLY` remaining depth).
+Measured against `head_reference` (`ddf932b`) on `confident_quick`:
+`sd10` net -4/+2 (net -3 across all three suites combined), `sn5000000`
+net -3/+3 depth-adjusted (net +4 combined, `hard_quick` alone +7 with
++0.5 extra ply -- a real depth-payoff signal).
+
+Re-investigated all 6 `confident_quick` losses (identical set at both
+`sd10` and `sn5m` -- a reproducible effect, not scattered noise) and both
+gains against `head_reference`'s own log plus a fresh independent Crafty
+check (`sd 22`, not run through the full methodology before this
+session). Net result: **4 of 6 losses are legit** (733, 672, 147, 736 --
+147 and 736 were previously unclassified and looked flaky from HEAD's
+log alone, but Crafty's independent move-choice agreement overturned
+that read), only 2 are flaky (715, 842). Both gains (519, 035) are
+already-established legit. That's a real, non-noise net-negative
+tactical cost on `confident_quick` specifically -- the aggregate
+suite-level numbers understated this at first glance (`sd10` -4 raw
+looked mostly explainable as noise until the per-position Crafty check
+was actually run).
+
+ECM.672 is worth remembering for future LMR work specifically: the same
+candidate binary, given enough iterative depth (not suite-capped
+`sd10`), *does* eventually find the correct `Bxh7+` sacrifice (depth
+15-16, jumping from +2.46 to +7.59/+7.90) after several iterations
+spent on an unrelated `cxb6`-based line. The graded curve isn't blind to
+this tactic, it's delaying discovery by roughly 7-8 plies past what a
+depth-10 budget allows -- a genuine cost at fixed/shallow search depths,
+but not the kind of permanent blind spot a bad exemption rule would
+produce. Worth checking whether this delay is concentrated in
+sacrifice/attacking lines specifically (where the "safe to reduce"
+heuristics this curve is built from may be least trustworthy) before
+concluding the curve needs a different shape rather than just deeper
+search to pay off.
+
+**`hard_quick`'s `sn5000000` gains investigated too** (11 gains, 4 losses
+-- 193, 476, 779, 866, not re-investigated here since they're losses on
+the suite that was already net-positive). Of the 11 gains: **7 are legit**
+(071, 251, 734, 735, 789, 803, 846 -- 846 is a reclassification, see its
+updated Legit-table entry above: an earlier build's shallower read
+tagged it flaky, but this candidate's deeper search plus Crafty both
+stably converge on the same move), **3 are flaky** (422 -- genuine
+engine-vs-engine disagreement on a thin margin, not corroboration; 875,
+879 -- both exact `0.00` on a repetition line), and **1 is genuinely
+unresolved** (640): the candidate's own extended search twice-confirms
+`Rxb2` at +3.66 (the `sn5m`-capped run's PV/played-move mismatch was
+just the node budget cutting off *before* this switch, not a bug), but
+Crafty independently searches to depth 30 and never once prefers
+`Rxb2`, staying on a different move (`Bd8`) near dead equal. Two
+real-depth searches disagreeing after real depth is a different, more
+interesting kind of uncertainty than the usual flaky signature (score
+exactly 0, unconfirmed thin margins) -- worth a dedicated line-by-line
+look at the `Rxb2` continuation before counting it either way.
+
+**Combined verdict across both suites, real (non-flaky) positions only**:
+9 legit gains (035, 519, 071, 251, 734, 735, 789, 803, 846) vs. 4 legit
+losses (733, 672, 147, 736), with ECM.640 unresolved. A materially more
+favorable picture than the raw suite aggregates suggested on their own
+-- this is why the per-position Crafty-corroboration check matters more
+than trusting `sd10`/`sn5000000` solve counts at face value, especially
+at this suite size.
+
+## 2026-09-14 king-safety file-storm-defect session
+
+Retuned `eval.c`'s `KingFileDefects`/`KingStormingPawnDefects` tables
+(inside `_ComputeFileStormDefects`) per `eval_tune/king_safety_data/HANDOFF.md`'s
+tier-1 finding: `{+2,+1,+0}` -> `{+4,+2,+0}` and
+`{+0,+4,+3,+1,+0,+0,+0,+0}` -> `{+0,+7,+5,+2,+0,+0,+0,+0}`, a real-game
+mate-mining logistic regression's two most dominant, most stable
+predictors, previously weighted the same as every other hand-picked
+king-safety defect. Measured against `head_reference` (`ddf932b`) at
+`sd10`: `ecm_ringers` -2, `ecm_confident_quick` -1, `ecm_hard_quick` -1.
+
+Per-position investigation (same methodology as the LMR session above)
+found almost none of this is real:
+- `ECM.715`, `ECM.866` (losses) and `ECM.813`, `ECM.443`, `ECM.771`
+ (gains) were already tagged flaky above.
+- `ECM.716`, `ECM.193`, `ECM.476` (losses) were already listed in the
+ "Gains seen" table from other sessions -- a position that has already
+ flip-flopped direction before isn't new evidence of a regression.
+- `ECM.501`, `ECM.801` (ringers losses, previously unclassified): both
+ show HEAD *itself* wobbling between candidate moves and landing on its
+ final answer only at the last iteration with no `++` confirmation
+ (`ECM.501`: Rf7 -> Rc8 -> Rf7 -> Rb8 -> Nxf3; `ECM.801`: Nd4 held
+ stably for 3 iterations then an unconfirmed jump to Nh6+ only at
+ depth 10) -- textbook flaky signature, not a real loss either
+ direction.
+- `ECM.442` (hard_quick loss): not a miss at all -- see its updated
+ entry in the Legit table above. The candidate's own deeper iterations
+ (depth 14-15) find and `++`-confirm the same `Ng6+` HEAD plays at
+ `sd10`; the `sd10` "loss" is purely a depth-budget delay (climbing
+ from +0.91 at depth 10 to a confirmed +1.94 by depth 14), the same
+ shape as `ECM.672`'s entry above.
+- `ECM.298` (hard_quick loss, previously unclassified): HEAD converges
+ cleanly on `Re6`/`Rf6` (+2.7 to +2.9) and the suite's labeled solution
+ is `Re6`, but the candidate instead finds and `++`-confirms `h4`
+ (+5.49 by depth 11, climbing to +8.87 by depth 17) -- and Crafty's own
+ independent deep search (`sd 27`, ~9.3s to switch onto it after
+ spending depths 20-23 stuck on `Rf6` at only +2.9-3.2) agrees `h4` is
+ objectively much stronger (+10.17) than the suite's labeled answer.
+ Notably the candidate found this line in 2.5s vs. Crafty's 9.3s.
+ Not a regression -- if anything, evidence the candidate is finding a
+ better resource than the suite's own labeled solution, faster than an
+ independent reference engine. Worth relabeling this suite's expected
+ answer if it's ever revisited, rather than trusting `Re6` as ground
+ truth.
+
+**Net read**: of the 4 nominal net solve losses (-2/-1/-1 across the
+three suites), 7 of 9 individual position losses were already-known or
+freshly-confirmed noise, one (`ECM.442`) is a depth-budget artifact, and
+the last (`ECM.298`) is arguably a candidate improvement mislabeled as a
+loss by the suite's fixed expected answer. No surviving evidence of a
+real tactical regression from this change at the position level, despite
+the suite-aggregate solve counts looking mildly negative -- another data
+point for not trusting raw `sd10` solve deltas on suites this size
+without the per-position check.
+
+Self-play (`match_play.py`, ongoing) at 110 games: -29 =47 +34
+(score 0.523, llr=+0.11, still far from either SPRT bound) -- too early
+to call, but notably `avg depth: candidate=9.96 head=10.25` (candidate
+searching ~0.3 ply shallower at the same `st` budget) with
+`avg ebf: candidate=4.442 head=4.396` (branching factor slightly
+*higher*, i.e. the tree got slightly fatter, not narrower) -- a
+non-trivial shape change in the search tree from what was purely an
+eval-weight retune around king safety, presumably via move-ordering/
+pruning decisions that read king danger differently now (e.g. anything
+gated on `KING_SAFETY_BY_COUNTER`'s bucket or a null-move/LMR
+verification threshold sensitive to eval score swings). Worth keeping
+in mind for future king-safety retunes: even a "just the numbers"
+change to an eval sub-term can visibly shift average search depth, not
+just leaf-node scores.
+
+## Caveat
+
+This list is a snapshot from one session's worth of investigation, not a
+permanent classification -- a position flagged "flaky" here reflects HEAD's
+behavior *at the commit this session started from* (`9d1e32d`) plus
+whatever candidate builds were tested through it. If the eval or search
+changes significantly later, re-derive rather than trust this table blindly
+-- especially the "Gains" table, which was never run through the full
+methodology in the first place.