diff options
| author | Scott Gasch <[email protected]> | 2026-09-15 13:31:36 -0700 |
|---|---|---|
| committer | Scott Gasch <[email protected]> | 2026-09-15 13:31:36 -0700 |
| commit | 390e675849ff42c8b7e1806f75552de988741fff (patch) | |
| tree | 17b8792aeaf74373bbaef3bac6b71bb5c0d74c69 /src/eval_tune/test_vs_head.sh | |
| parent | ca73728f7dbf01c4b39c7496729f81b1fbffa00b (diff) | |
Rewrite _EvalKing as a linear ring-scan and promote its terms to DNA-tunable constants; retune via mate-mining regression, ElasticNet/Ridge, and logistic-vs-material refits.
Splits _EvalKing's own king-location table from CountKingSafetyDefects' search-gate copy so future eval retunes can't silently perturb extension/reduction gate frequency. Bumps KING_BACK_FLIGHT_BONUS (+1 -> +3) and reorders/nudges the ring-scan sub-terms (minor/pawn/xray/undefended) to match relative rankings that replicated across three independent calibration methods, while leaving KING_QUEEN_PROXIMITY_DANGER and KING_MISSING_ONE_CASTLE_OPTION untouched pending further discussion.
Self-play (640 games) is roughly even (score 0.509, llr=+0.17, far from either SPRT bound) but consistently faster (nps candidate=1362140 vs head=1333682) at equal average depth -- landing as a wash/speed-neutral checkpoint, not a confirmed tactical improvement, for further tuning to build on.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Diffstat (limited to 'src/eval_tune/test_vs_head.sh')
| -rwxr-xr-x | src/eval_tune/test_vs_head.sh | 46 |
1 files changed, 40 insertions, 6 deletions
diff --git a/src/eval_tune/test_vs_head.sh b/src/eval_tune/test_vs_head.sh index 871b801..29c050f 100755 --- a/src/eval_tune/test_vs_head.sh +++ b/src/eval_tune/test_vs_head.sh @@ -1,11 +1,45 @@ #!/usr/bin/env bash +# +# Gated self-play match against a baseline binary. Historically this +# always compared the live working-tree build against head_reference/ +# -- generalized (2026-09-14) to take both binary paths as args so two +# arbitrary builds (e.g. a file-storm-only worktree build vs. the +# current working tree, to isolate one change at a time when several +# have landed together -- see king_safety_data/HANDOFF.md's session +# notes on not measuring two eval changes in one match) can be compared +# without hand-rolling a match_play.py invocation each time. +# +# Usage: test_vs_head.sh [head_binary] [candidate_binary] [label] +# +# head_binary Baseline engine. Defaults to head_reference/typhoon +# (the usual "is this better than last checkpoint" +# question). +# candidate_binary Candidate engine. Defaults to a fresh copy of the +# live working-tree build (../typhoon) into +# /tmp/typhoon/typhoon_match_play_candidate -- pass +# an explicit path (already built) to skip that +# copy/freeze step entirely, e.g. when comparing +# two prebuilt worktree binaries. +# label Distinguishes scratch dir/log/pgn-out so multiple +# matches can run concurrently without colliding. +# Defaults to "match". +# +# Examples: +# test_vs_head.sh # old default behavior +# test_vs_head.sh /tmp/typhoon/filestorm_only /tmp/typhoon/candidate queenprox_backflight set -e -CANDIDATE=/tmp/typhoon/typhoon_match_play_candidate -HEAD=../../head_reference/typhoon +HEAD="${1:-../../head_reference/typhoon}" +LABEL="${3:-match}" + +if [ -n "$2" ]; then + CANDIDATE="$2" +else + CANDIDATE=/tmp/typhoon/typhoon_match_play_candidate + cp -f ../typhoon ${CANDIDATE} +fi -cp -f ../typhoon ${CANDIDATE} python3 ./match_play.py \ ${HEAD} \ ${CANDIDATE} \ @@ -14,6 +48,6 @@ python3 ./match_play.py \ --workers 12 \ --st 1 \ --sprt --elo0 0 --elo1 5 \ - --scratch /usr/local/tmp/typhoon_match_overnight \ - --log overnight_match.log \ - --pgn-out ../self_play_games.pgn + --scratch /usr/local/tmp/typhoon_match_${LABEL} \ + --log ${LABEL}.log \ + --pgn-out ../${LABEL}_games.pgn |
