# King-safety defect regression: handoff ## Goal `_EvalKing` (eval.c) sums a pile of hand-picked "defect" constants into one scalar (`uCounter`), then runs that through a single lookup table (`KING_SAFETY_BY_COUNTER`). The hand-picked weights assume every defect counts equally, which is suspect -- e.g. a queen touching an undefended ring square and a knight touching one add the exact same `+1`. This project mines real games to see which defects actually predict getting mated, so `eval.c`'s weights can be informed by data instead of guesswork. **Method**: find real middlegame checkmates in human games, look at the losing king's position N plies earlier, and compare its raw defect pattern against a matched "sound" control position (drawn games) with the same attacker material and development level. Fit a logistic regression; features with a real, stable coefficient are candidates for reweighting in `eval.c`. See `CLAUDE.md`'s "King-safety eval improvement project" section (search for it) for the full narrative this file summarizes. ## Where everything is - **Final training data**: `remine_v4/king_features_matched_v4.csv` -- 86,362 rows (43,181 matched positive/control pairs). **Use this one.** - `remine_v4/merged.csv` -- raw mined FENs before feature extraction (172,735 rows: 43,181 positives + 129,554 control candidates). - `remine_v4/king_features_v4.csv` -- all 172,735 rows with full `_EvalKing` defect breakdown, before matching trims it down to the 86,362 confound-controlled rows. - Pipeline scripts live in `eval_tune/` (sibling of this dir), not here: `mine_king_mates.py`, `shard_mine.py`, `extract_king_features.py`, `match_controls.py`, `train_king_safety_model.py`. - Engine side: `chess.h`'s `KING_DEFECT_DUMP_INFO` struct, `_EvalKing`'s `#ifdef KING_DEFECT_DUMP` instrumentation in eval.c, and command.c's `evalkingdump` command -- all gated behind the `KING_DEFECT_DUMP=1` gmake build flag (NOT part of a normal build). `../typhoon_kingdump` is a saved binary built with that flag; rebuild it (`gmake clean` first -- flag changes don't trigger recompilation on their own, see "Gotchas" below) if `eval.c`/`chess.h`/`command.c` change. ## Final result (from `king_features_matched_v4.csv`) Logistic regression, 80/20 train/test split, standardized coefficients. Test AUC 0.91 (plausible for "engineered danger features vs. proven-sound draws", not suspiciously perfect). Ranked by |coefficient|, most-to-least trustworthy tier first (stability = full-population vs. top-tercile-army- strength fits agreeing): **Tier 1 -- dominant, very stable**: `filestormopenfile` (+0.57 to +0.80), `filestormpawn` (+0.51 to +0.54). These are eval.c's `KingFileDefects[3]` and `KingStormingPawnDefects[8]` tables (open/half-open file near the king; how far an enemy pawn has advanced up that file), already independent tables -- lowest-risk place to start a real retune. **Tier 2 -- strong, stable**: `front_offboard`, `front_def`, `sq4_offboard`, `front_undef`, `front_atk_minor`, `front_atk_rook`, `misscastleshort`, `front_flight` (+0.33 to +0.45). "front" = the diagonal/vertical ring squares one rank away, in the direction each side attacks toward (see "Feature schema" below for the exact slot mapping -- do NOT assume this means "in front of the king" the way a human would first guess; see the correction note there). `sq3`/`sq4` are the two same-rank side squares, kept individually resolved. **Tier 3 -- moderate, less stable**: `back_offboard` (drifts the most of any surviving feature, treat with suspicion), `sq3_undef`, `attackerarmyscaler`, `queenproximity`, `back_atk_minor`, `front_atk_queen`. **Confirmed noise -- do not retune based on this data**: `minorsathome` (coefficient ~0.001, was previously the #1 "signal" before two sampling bugs got fixed -- see "Bugs found" below), `initloc`, `castled` (exactly 0.0000, fully collinear with `misscastleshort`/`misscastlelong`). ## Translating this into eval.c -- confidence and plan This is NOT a "throw away eval.c and use the model" situation. The plan is targeted retuning of existing constants/tables, informed by relative coefficient *ratios*, not literal log-odds-to-centipawn conversion: 1. **`KingFileDefects`/`KingStormingPawnDefects` deserve a real weight increase relative to everything else** -- they're the strongest signal by a clear margin and already live in their own small, isolated tables (`_ComputeFileStormDefects` in eval.c). Lowest risk, highest expected payoff, start here. 2. **`EmptyAttackedSquare`/`EmptyUnattackedSquare`'s front/back asymmetry is directionally right but probably too weak a spread** -- front- bucket features consistently outrank back-bucket ones here by a good margin. 3. **`misscastleshort`'s constant likely deserves a bump.** 4. **Don't touch `KING_INITIAL_COUNTER_BY_LOCATION` or the undeveloped- minors term** based on this data -- both came back as noise. 5. Neither `KingFileDefects`/`KingStormingPawnDefects` nor the ring-scan tables are currently `DNA_ARRAY`-registered (checked: only `KING_SAFETY_BY_COUNTER` and `KING_QUEEN_PROXIMITY_DANGER` are under king-safety). Retuning them means a manual edit (optionally promoting them to file scope + registering as DNA arrays for future tuning), not "rerun the tuner." 6. **Absolute magnitude has to be calibrated the normal way** -- run any change through `sd10` ECM suite comparison against `head_reference/` and `match_play.py`'s gated head-to-head match, same as any other hand-tuned eval constant. The regression tells you *shape* (which defects matter more relative to each other), not the right centipawn magnitude for a chess engine's scoring scale. 7. `CountKingSafetyDefects` (the cheap lazy-eval proxy) stays untouched until a real change lands and is validated -- agreed early in this project, still the right call. ## Feature schema (read this before extending the pipeline) `KingSafetyDeltas` in eval.c (`-17,-16,-15,-1,+1,+15,+16,+17`) defines 8 ring slots, indices 0-7 in that order. **Correction made mid-project**: slots 3 and 4 (deltas -1/+1) are the two SAME-RANK side squares, not "front of the king" -- an earlier description in conversation got this wrong. The diagonal/vertical slots (0,1,2 and 5,6,7) are the ones with a real front/back asymmetry, matching `EmptyAttackedSquare`'s own existing color-dependent weighting (WHITE's front is slots 0/1/2, BLACK's is 5/6/7 -- see `_EvalKing`'s comments and `KING_RING_FRONT`-style logic). `extract_king_features.py` dumps full per-slot resolution (all 8 slots x 4 piece types x several state flags) since the engine-side cost of doing so is free once you're calling `evalkingdump` anyway. `train_king_safety_model.py` is where slots 0/1/2/5/6/7 get bucketed into "front"/"back" sums (color-dependent per row's `mated_color_or_sampled_color`), while slots 3/4 stay individually resolved -- this bucketing was added in this session after the diagonal per-slot coefficients turned out to be the noisiest, most sign-flipping entries in every full-vs-subset drift comparison. `filestorm` is deliberately NOT used as a training feature -- its two independent sub-components (`filestormopenfile`, `filestormpawn`) are used instead, since they map onto two already-separate eval.c tables and there's no reason to make the regression re-derive their relative importance from one pre-summed number. ## Bugs found and fixed this session (read before trusting old data) If you find any `.csv` file in this project OTHER than `remine_v4/*_v4.csv` and its merged.csv, IT PREDATES ONE OR MORE OF THESE FIXES AND SHOULD NOT BE USED. (Older files were deleted during cleanup, but if this pipeline gets rerun and produces intermediate files again, this is the checklist for whether they're trustworthy.) 1. **Bare-king endgame mates leaking into "middlegame" positives.** Only checking the attacker's material (`>= 800cp`, roughly rook+bishop) let trivial K+Q-vs-lone-K technique mates through. Fixed by also requiring `total_non_pawn_piece_count(board) >= 8` (mirrors `_EvalKing`'s own `pos->uNonPawnCount[WHITE][0] + pos->uNonPawnCount[BLACK][0] < 8` endgame boundary exactly) -- see `is_middlegame()` in `mine_king_mates.py`. 2. **`min_ply` too low (10, i.e. move 5) for control sampling.** A large fraction of the pool's draws are short draws-by-agreement close to `filter_pgn.py`'s own 40-ply floor. Combined with a large end-guard (staying clear of the game's end), the only valid sampling window on a short draw was the early opening -- where minors are legitimately still undeveloped for completely ordinary reasons. This alone caused `minorsathome` to look like a massive, spurious "signal" (controls showed `minorsathome>=2` in ~85% of samples). Verified the engine's own computation was correct on a real developed-middlegame FEN -- 100% a sampling-window bug, not an eval bug. Fixed: `min_ply` raised to 20 (move 10), and a game too short to satisfy both `min_ply` and the end-guard is now skipped entirely rather than force-sampled early. 3. **Control-sampling ply selection secretly always picked the earliest plies in a game's valid window.** `rng.shuffle(candidates)` followed by `for idx in sorted(candidates)` completely undid the shuffle. Fixed in two steps (the first fix wasn't sufficient on its own -- see the code comment in `mine_controls()` for why): randomly subsample which plies to try, replay ascending through all of them ONCE (required -- boards can only be walked forward), collect every result that passes filters into a list, THEN randomly choose which results to actually keep via `rng.sample` on that list. Verified: sampled-ply-as-fraction-of-game-length now has a median around 0.46 with real tail into 60%+, instead of clustering at 20-30%. 4. **Matching-radius bug treated two very differently-scaled confound columns with one shared distance metric.** `attackerarmyscaler` spans 0-31 (`ASSERT`-confirmed hard clamp ceiling), `minorsathome` spans 0-4. A single Chebyshev radius meant a "tight" radius-2 match still let `minorsathome` drift by up to 2 -- literally half its entire range. This is why `minorsathome` was STILL the #1 "signal" even after an initial matching attempt. Fixed: `match_controls.py` now takes `--exact-cols` (must match exactly, no radius search -- `minorsathome` defaults here) and `--radius-cols` (allowed to expand outward, `attackerarmyscaler` defaults here) as separate concepts. 5. **KDTree-based nearest-neighbor matching matched almost nothing** (308 of 43,181 positives) because the match columns are low-cardinality integers -- huge exact-value clusters mean a KDTree's "k nearest" are massively tied, so the first few positives processed in a cluster exhaust the shared candidate set and everyone else in that cluster fails to match. Replaced with exact-bucket + expanding-integer-radius matching (see `match_controls.py`), which has no such collision problem. 6. **`_EvalKing`'s per-square ring scan implicitly assumed all 8 slots were geometrically equivalent for feature purposes**; slots 3/4 (side squares) vs. the diagonal slots turned out to behave differently enough that per-slot resolution on the diagonals was pure noise at the sample sizes available -- see "Feature schema" above. 7. **Board-diagram spam.** `setboard` prints a full ASCII board dump on every call by default (`g_Options.fShouldPost`, see `DumpPosition` in gamelist.c), which dominated per-row I/O cost during feature extraction. Fixed by prepending `nopost` to every extraction script. 8. **Two separate concurrent-process races during interactive mining**, both self-inflicted mid-session: (a) `$!` after a `cd ... && nohup ...` pipeline captured the wrong PID, leaving a stale duplicate process racing on the same output file as a supposedly-fresh rerun; (b) killing that process didn't fully stop it before a new one started. No lasting damage (caught via `ps aux` before trusting any output), but: **always verify a background job is actually singular via `ps aux | grep