summaryrefslogtreecommitdiff
path: root/src/eval_tune/test_vs_head.sh
diff options
context:
space:
mode:
authorScott Gasch <[email protected]>2026-09-15 13:31:36 -0700
committerScott Gasch <[email protected]>2026-09-15 13:31:36 -0700
commit390e675849ff42c8b7e1806f75552de988741fff (patch)
tree17b8792aeaf74373bbaef3bac6b71bb5c0d74c69 /src/eval_tune/test_vs_head.sh
parentca73728f7dbf01c4b39c7496729f81b1fbffa00b (diff)
Rewrite _EvalKing as a linear ring-scan and promote its terms to DNA-tunable constants; retune via mate-mining regression, ElasticNet/Ridge, and logistic-vs-material refits.
Splits _EvalKing's own king-location table from CountKingSafetyDefects' search-gate copy so future eval retunes can't silently perturb extension/reduction gate frequency. Bumps KING_BACK_FLIGHT_BONUS (+1 -> +3) and reorders/nudges the ring-scan sub-terms (minor/pawn/xray/undefended) to match relative rankings that replicated across three independent calibration methods, while leaving KING_QUEEN_PROXIMITY_DANGER and KING_MISSING_ONE_CASTLE_OPTION untouched pending further discussion. Self-play (640 games) is roughly even (score 0.509, llr=+0.17, far from either SPRT bound) but consistently faster (nps candidate=1362140 vs head=1333682) at equal average depth -- landing as a wash/speed-neutral checkpoint, not a confirmed tactical improvement, for further tuning to build on. Co-Authored-By: Claude Sonnet 5 <[email protected]>
Diffstat (limited to 'src/eval_tune/test_vs_head.sh')
-rwxr-xr-xsrc/eval_tune/test_vs_head.sh46
1 files changed, 40 insertions, 6 deletions
diff --git a/src/eval_tune/test_vs_head.sh b/src/eval_tune/test_vs_head.sh
index 871b801..29c050f 100755
--- a/src/eval_tune/test_vs_head.sh
+++ b/src/eval_tune/test_vs_head.sh
@@ -1,11 +1,45 @@
#!/usr/bin/env bash
+#
+# Gated self-play match against a baseline binary. Historically this
+# always compared the live working-tree build against head_reference/
+# -- generalized (2026-09-14) to take both binary paths as args so two
+# arbitrary builds (e.g. a file-storm-only worktree build vs. the
+# current working tree, to isolate one change at a time when several
+# have landed together -- see king_safety_data/HANDOFF.md's session
+# notes on not measuring two eval changes in one match) can be compared
+# without hand-rolling a match_play.py invocation each time.
+#
+# Usage: test_vs_head.sh [head_binary] [candidate_binary] [label]
+#
+# head_binary Baseline engine. Defaults to head_reference/typhoon
+# (the usual "is this better than last checkpoint"
+# question).
+# candidate_binary Candidate engine. Defaults to a fresh copy of the
+# live working-tree build (../typhoon) into
+# /tmp/typhoon/typhoon_match_play_candidate -- pass
+# an explicit path (already built) to skip that
+# copy/freeze step entirely, e.g. when comparing
+# two prebuilt worktree binaries.
+# label Distinguishes scratch dir/log/pgn-out so multiple
+# matches can run concurrently without colliding.
+# Defaults to "match".
+#
+# Examples:
+# test_vs_head.sh # old default behavior
+# test_vs_head.sh /tmp/typhoon/filestorm_only /tmp/typhoon/candidate queenprox_backflight
set -e
-CANDIDATE=/tmp/typhoon/typhoon_match_play_candidate
-HEAD=../../head_reference/typhoon
+HEAD="${1:-../../head_reference/typhoon}"
+LABEL="${3:-match}"
+
+if [ -n "$2" ]; then
+ CANDIDATE="$2"
+else
+ CANDIDATE=/tmp/typhoon/typhoon_match_play_candidate
+ cp -f ../typhoon ${CANDIDATE}
+fi
-cp -f ../typhoon ${CANDIDATE}
python3 ./match_play.py \
${HEAD} \
${CANDIDATE} \
@@ -14,6 +48,6 @@ python3 ./match_play.py \
--workers 12 \
--st 1 \
--sprt --elo0 0 --elo1 5 \
- --scratch /usr/local/tmp/typhoon_match_overnight \
- --log overnight_match.log \
- --pgn-out ../self_play_games.pgn
+ --scratch /usr/local/tmp/typhoon_match_${LABEL} \
+ --log ${LABEL}.log \
+ --pgn-out ../${LABEL}_games.pgn