typhoon.git/src/eval_tune/match_play.py, branch master

Remove dead autoplay/autoplayer trees, land GNUmakefile/eval_tune companions

2026-09-09T03:25:26Z

- Remove autoplay/ and autoplayer/ entirely: old opponent-automation scrapers/harnesses (child_process.cc test scaffolding, compiled a.out binaries and a .dSYM bundle, saved book/position files, macOS ._-prefixed resource-fork cruft) that predate this repo's current tooling and were never referenced by anything still in use. - GNUmakefile: add DIAG_NO_QSEARCH_FUTILITY/CALIBRATE_QSEARCH_FUTILITY profile flags and testrecogn.o to the TEST=1 object list. Both are companions to already-committed work that never got their own build support committed: search.c's qsearch-futility calibration harness needs the two profile flags to be buildable at all, and testrecogn.c (added alongside the recogn.c bugfix, now committed here too) needs to be in TEST=1's OBJS to actually compile/link. - eval_tune/match_play.py, eval_tune/test_vs_head.sh: real fixes found and applied earlier this session -- match_play.py's opening-book leak (games weren't actually book-free), missing --hash/--cpus (games ran on the 64k-entry/single-cpu memset-zero defaults instead of this project's normal 256m/1cpu), a shared-logfile race across concurrent match workers, and a report-parsing deadlock on engine resignation. test_vs_head.sh reverted to comparing against head_reference/typhoon + the live working-tree binary -- it had been pointed at a since-deleted, long-stale one-off comparison binary (typhoon_allbitboards) since 92fc412, silently invalidating every "vs head" self-play check run through it since Sep 4. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01Ka9o3S2eKqh4jNfmxVZ6fH

Cherry-pick non-LMR fixes and tooling from the "LMR" stash

2026-09-04T00:40:17Z

Pulled the parts of the stashed LMR work that are genuinely independent of the reduction logic itself, leaving the actual LMR redesign for separate review: - Fix extension-taper table overflow: remove the flat MAX_EXTEND_PER_LINE cap and instead clamp the depth used to build g_uExtensionReduction[] so a deep `sd` request can't leave the whole taper table stuck at "0 penalty" (every index unreachable). - Remove a spuriously-firing ASSERT(fMovesRescoredByIID) in Search(): RescoreMovesViaSearch's own fail-high branch deliberately leaves that flag FALSE by contract, so the assert could fire on any DEBUG build given an unlucky rescore, making the DEBUG/TEST harness unreliable. - Misc correctness/portability fixes: unix.c pointer-truncation casts, chess.h's CONTAINING_STRUCT/IS_ENPASSANT/ABS_DIFF macro hardening (plus gating the branchless bit-tricks on _X64_ too, not just _X86_), removal of dead Slide*WithoutSigs prototypes, main.c's hash default bumped to 256m and its CPP self-test's arch gate widened to _X64_. - eval_tune/match_play.py: cosmetic SPRT progress-bar/output rework. - Delete eval_tune/run_ecm.sh (superseded, unreferenced elsewhere). - run_tests.sh: parameterize suites/SD/SN via args/env vars instead of hardcoding the three curated suites and sd10/sn5M (defaults kept pointing at the existing curated suites, since the stash's own lmr_sensitive_30/lmr_control_30 default suites aren't present in the repo). Deliberately left out of this commit: the stash's actual LMR reduction logic, the M-SIGNAL-SHADOW diagnostic subsystem, the large PERF_COUNTERS instrumentation buildout, the history-table gravity rework, and the FindEnprisePiece pre-move staleness fix (skipped per request pending a decision on whether to also change EFP's pruning behavior). Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01MjdDfHry3i2jfJzyDXaG8A

Rewrite match_play.py: binary-vs-binary comparison, real SPRT, EBF/NPS/first-move-beta stats, fix a completion-order bias in the live progress readout.

2026-08-30T05:51:37Z

match_play.py previously compared one fixed binary with two loaded evaldna files -- built for the Texel auto-tuning pipeline this project has since moved away from in favor of hand-tuned constants baked directly into eval.c. Converted to compare two separate compiled binaries instead (head_engine/candidate_engine positional args, no DNA loading at all) -- a real build is now required per side, but that's the right model: what's being compared is two source trees, not two parameter files loaded into an otherwise-identical process. Found and deleted eval_tune/binary_match_play.py, a pre-existing (never committed) sibling that already did binary-vs-binary comparison but predates and is now strictly superseded by this rewrite (no SPRT, no stats capture, and the same completion-order bias fixed below) -- keeping both would have left two overlapping tools to drift out of sync, the same duplication problem this session spent all night removing from eval.c itself. Added a real SPRT (Sequential Probability Ratio Test), the same formulation fishtest/cutechess-cli use: two Elo hypotheses (H0/H1) tested via the log-likelihood ratio of a normal approximation to the per-game trinomial (W/D/L) score, variance re-estimated from the running W/D/L mix as games accumulate. Verified against a Monte Carlo simulation before landing: correctly resolves H0 at true_elo=0 (~6-19k games) and H1 at game counts matching the theoretical fixed-N table almost exactly (30 Elo: ~1.4-2k, 20 Elo: ~2.3-4.5k, 10 Elo: ~5-9k, 5 Elo: ~10-25k). Required restructuring the game scheduler from "submit everything upfront, as_completed" to a bounded rolling window (at most --workers games in flight) so it can actually stop early once SPRT concludes instead of having thousands of already-launched futures it can't usefully cancel. Fixed a real, previously-unnoticed bug that explains a specific observed symptom (candidate consistently scoring high for the first ~500 games of a 1000-game run, then eroding toward 0.5 -- every run, same direction, which is what tipped this off as systematic rather than noise): the live "score so far" readout processed games in *completion* order (as_completed), not submission order, and decisive games plausibly finish faster than grindy draws/losses (a winning side wraps up before --max-plies; a losing/drawing side often runs long). That means candidate wins arrive disproportionately early and the live average was a biased mid-run estimator -- high at first, eroding as slower non-win games trickle in. The *final* score was never actually wrong (order-independent, sums the same regardless of arrival order), just the progress narrative watched live. Fixed by buffering out-of-order completions and only advancing the printed running score through games in their original submission order. Also fixed --sd defaulting to 8 even when --st was passed -- the argparse mutually-exclusive group only stops both flags being given together, it does nothing about one flag's default silently applying when only the other was specified. --st alone was being silently ignored in favor of sd=8 the whole time. Now --sd defaults to None and only falls back to 8 when neither --sd nor --st is given. Added EngineStats: per-engine (not per-color, since candidate/baseline swap sides every other game) speed and tree-shape capture -- NPS, EBF (nodes**(1/depth) per move, same formula script.c's suite runs use), and first-move-beta-cutoff-rate, all pulled from the same PostMoveSearchReport block every move already prints (confirmed this prints after EVERY move, not just script-run suite summaries, and confirmed the "move" line prints BEFORE the stats block against root.c, not after -- the read loop needed restructuring to keep reading past the move line rather than stopping on it). No separate benchmark pass needed; these come from the same games already being played for the strength comparison. Deliberately not built tonight, flagged as a possible follow-up: an opening-pool independence concern (sample_openings picks random byte offsets into the TWIC pool, which could pull duplicate/correlated lines if TWIC has many games following the same trendy opening in a season -- would make the standard-error math slightly overconfident, not wrong in direction). Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01M9ZDiJhiUajUxh95mTXCFJ