# typhoon chess engine A chess engine by Scott Gasch (scott.gasch@gmail.com), originally started 2004. Previously named "monsoon" -- any old references to that name mean this same codebase. Speaks the xboard/WinBoard protocol. FreeBSD host, 64-bit. ## Scratch files and logs **Put ad-hoc test/debug output under `/tmp/typhoon/`, not loose in `src/` or loose in `/tmp` itself.** Past sessions repeatedly dropped one-off ECM run logs, build logs, bisect binaries, and gprof output directly into `src/` (`ecm_v17_sn4m.stdout`, `lmr_A.log`, stray `typhoon_*` binaries, etc.) and directly into `/tmp` (`build6.log`, `nn_ringers.log`, `typhoon_pool_test3/`, multi-hundred-MB `typhoon_gprof.txt`, etc.), both of which build up uncommitted clutter that's easy to mistake for something meaningful and tedious to clean up later. Anything that isn't a deliberately-kept comparison baseline (`head_reference/`, the curated suites in `tests/`, `eval_tune/`'s pipeline scripts and `.dna` outputs) is scratch and belongs under `/tmp/typhoon/`. ## Building `build.sh`'s OS detection (`expr $OSTYPE = "darwin"`) is broken in this shell environment and falls through to a 32-bit `gcc`-based profile that doesn't exist here (no `gcc`, and the resulting `.o` files are architecture-mismatched against everything else). **Don't use build.sh as-is.** Build directly with: ```sh gmake -j5 GENETIC=1 PERF_COUNTERS=1 MP=1 SIXTYFOUR=1 ``` This uses `clang`/`clang++` (already the GNUmakefile default) and produces a 64-bit binary matching the historically-built one. After changing any header touched by many translation units, `gmake clean` first to avoid 32/64-bit object mismatches at link time. ## Running / xboard protocol basics No command-line flags are required for interactive use; the binary drops into an xboard-protocol REPL on stdin/stdout. Key commands used during development: - `force` -- stop the engine from auto-playing; you control both sides' moves. - `setboard ` -- set the position. - `eval` -- print `Static eval: ` (calls `Eval()` directly, no search). - `evaldna dump` / `evaldna read ` / `evaldna write ` -- the eval-constant import/export system (`g_EvalDNA` in eval.c). `evaldna` alone (no subcommand) just prints usage -- you need `evaldna dump` explicitly. - `sd ` / `st ` -- fixed depth vs. fixed time search limits. - `go` -- search and play one move for whichever side is on move, then stop (when paired with `force` after). Repeated `go`/`force` cycles correctly drive a full self-play game -- the engine tracks its own position across calls, no need to feed the opponent's move back manually. - `script ` -- run a list of commands from a file, used for test suites (see ECM below). Prints solved/unsolved stats at the end. - `--dnafile ` (startup flag) or `evaldna read ` (runtime) load a DNA file produced by tuning; `--egtbpath ` or `set EGTBPath ` configure Syzygy tablebase lookup. **Default EGTB path is hardcoded** in `main.c:297` to `/zscratch/egtb`, used whenever `--egtbpath` isn't passed. That directory currently has full 5-man WDL+DTZ Syzygy coverage but is **missing the WDL half of its 6-man set** (only `.rtbz`/DTZ files exist for 6-man combos) -- `init_tb()` in `fathom/tbprobe.c:752` requires the WDL file to exist before registering a combination at all, so every 6-man entry is silently skipped and the engine reports "up to 5 men" even though 6-man `.rtbz` files are sitting right there. Not a config issue; the fix is downloading the missing 6-man `.rtbw` files. ## Testing: the ECM tactical suite `tests/ecm.ep_` is a 3517-line / 881-problem EPD tactical test suite. `test.sh ` runs it and diffs against `lastrun.log` via `suite_diff.pl`. Two things to know: 1. **`suite_diff.pl`'s shebang (`#!/usr/bin/perl`) is wrong on this box** -- perl lives at `/usr/local/bin/perl`. Invoke it as `perl ./suite_diff.pl ` rather than `./suite_diff.pl`. 2. **Prefer `sd ` over `st ` for before/after eval comparisons.** A time-based limit lets identical positions reach different search depths across runs purely from incidental machine load (this session saw large timing swings from unrelated processes/scrubs) -- `sd 10` (or similar) holds search effort constant so a solve-rate difference is actually attributable to the eval change being tested. **Invocation**, matching what `test.sh` and `head_reference/logs/` use -- piping commands over stdin does not reliably run a `script` command to completion; use `--batch --command` instead, with the opening book pointed at a nonexistent path so early moves aren't book lookups (see the `match_play.py` bugs below for why that matters) and explicit `--hash`/ `--cpus` rather than the tiny memset-zero defaults: ```sh ./typhoon --cpus 1 --hash 256m --logfile /tmp/typhoon//sd10_.log \ --batch --command "force; book name /nonexistent.book.bin; sd 10; script ../tests/.ep_" ``` **Run these in the background and use Monitor to wait, don't block the foreground on them.** Each curated suite takes on the order of minutes at `sd 10`; a `wait` in a synchronous Bash call routinely exceeds a 2-minute tool timeout even though the run itself is fine. Launch with `run_in_background: true` (or equivalent), then use Monitor (or just continue the conversation -- a background command notifies on completion) rather than polling/sleeping in a loop. Current baseline (`lastrun.log`, `st 1`, unmodified eval constants): **606/879 solved.** A first full DNA-tuning pass against 600k TWIC positions raised this to **622/879** (see `eval_tune/`). `Trace()` (used for almost all engine output, including ECM's per-problem results) only `fflush()`s stdout, **not** the logfile -- `Log()`/`Bug()` do flush the logfile every call, but `Trace()` doesn't. A `--logfile` file can lag well behind actual progress while a run is in flight; it catches up fully once the process exits. Don't trust a logfile's line count as a live progress indicator for a running batch job. ### The three curated suites and the `head_reference` protocol Day-to-day search-change evaluation uses three small, curated EPD files under `tests/` rather than the full 881-problem `ecm.ep_` (too slow to iterate on) or a single random sample (too noisy -- a handful of flips either way swamps the signal at that size): - **`tests/ecm_ringers.ep_`** (11 positions) -- clean discriminators hand-picked because a baseline engine solves them reliably and a regressed one reliably doesn't (or vice versa). Fastest signal, but small enough that +/-1 solve is not necessarily real. - **`tests/ecm_confident_quick.ep_`** (90 positions) -- a larger, still-fast sample the engine is expected to do well on. - **`tests/ecm_hard_quick.ep_`** (90 positions) -- a harder sample, more sensitive to search-quality changes (pruning/reduction/ordering) since these positions need real depth/precision to crack. Run all three together, not just one -- a change can look great on one and cost solves on another (see `lmr_testing/RESULTS.md` for several examples this cut both ways). **Always compare against a fixed, known-good reference binary+logs, one variable at a time -- never against "the working tree as it happened to be a few hours ago."** `head_reference/` (sibling of `src/`, not `tests/`) holds exactly this: a binary built from a specific commit, plus full logs for all three suites at both `sd 10` (fixed depth) and `sn 5000000` (fixed node budget per position) in `head_reference/logs/`, and a `README.md` recording the commit hash and confirmed live/dead state of every search mechanism (LMR, EFP, etc.) at that point -- confirmed by reading the code, not assumed from memory of what a diagnostic build's `if (FALSE)` happened to say. Rebuild this directory (and update its README) whenever committing a change that becomes the new comparison baseline, so the next investigation always has something to check itself against before drawing conclusions -- this exact gap (assuming a disabled-in-the-working-tree mechanism was also disabled at HEAD, when it wasn't) cost a full afternoon re-deriving EFP's actual committed behavior mid-investigation. Pick `sd` vs `sn` based on what the change is expected to affect: `sd` (fixed depth) is the right choice when judging whether a pruning/ ordering change makes the tree smaller or larger for the same search effort; `sn` (fixed node budget) is the right choice when judging how many positions solve within a fixed cost. Don't run both for every single candidate change -- pick whichever matches what's actually being tested (the reference logs in `head_reference/` keep both, precisely so a later investigation can choose either without re-running HEAD). ## Eval tuning pipeline (`eval_tune/`) `tune_eval_dna.py` is a Texel-style tuner: drives `typhoon` as an xboard-protocol subprocess, fits `g_EvalDNA`'s ~1870 raw constants (reduced to ~468 after collapsing cells that share a starting value -- see caveat below) against TWIC game outcomes via sigmoid-squared-error + coordinate descent. `dna_diff.py` renders a human-readable, per-array diff between two `.dna` files (with 8x8 grids for the board-shaped location tables) so you can actually see what a tuning run changed. Position pools are cached under `position_cache/` keyed on source-file mtime+size+filter args, since a 500k-600k position PGN scan takes ~20 minutes. **Two real bugs were found and fixed this session, both in `command.c`, not in the Python tooling:** 1. **`EvalCommand` used to malloc + fully zero (`InitializeSearcherContext`, which memsets the whole ~75MB `SEARCHER_THREAD_CONTEXT` -- `rgPawnHash` and `rgEvalHash` are embedded arrays inside that struct, not separately-managed heap buffers) on every single `eval` call, then free it.** Fixed to keep one persistent context and use the cheap `ReInitializeSearcherContext` (just repositions + zeros counters) on subsequent calls -- ~1000x throughput improvement for scripted eval scoring. 2. **That persistent context's embedded eval hash was never invalidated when new DNA was loaded**, so after the fix above, a position scored once under any DNA would return that *same cached score forever*, silently ignoring every later `evaldna read`. Fixed via a global `g_uDnaGeneration` counter (bumped in `EvalDnaCommand` on successful load, checked in `EvalCommand`) that forces exactly one full re-init per DNA change, not per position. **If this regresses, tuning results will look like it "converged" almost immediately with implausibly little movement -- that's the symptom, not genuine convergence.** Verify by perturbing a few known-live parameters and confirming the reported error actually changes. **Known limitation, not yet fixed:** the distinct-value grouping used to shrink 1870 raw cells to ~468 tunable parameters ties together *any* cells that start out numerically equal, for whatever reason -- true mirror symmetry, an intentional-but-asymmetric design choice (e.g. `KING_INITIAL_COUNTER_BY_LOCATION`'s rank-8 row `1,0,0,0,1,0,0,1` flags a1/e1/h1 specifically, not a geometric mirror), or pure incidental coincidence (e.g. `PAWN_CENTRALITY_BONUS`'s legitimate "no bonus" squares happening to share the value `0` with unrelated off-board padding cells). The padding case is harmless (those indices are never read during real eval) but the coincidental-equality case is a real, unaddressed source of noise in the parameter space, most visible on small/noisy training runs (sentinel `-1` placeholder cells in `TRADE_PIECES` drifted to an arbitrary shared value on a 25k-position smoke test; they stayed put on the full 600k run). **`Engine`/`EnginePool` in `tune_eval_dna.py` must be used as a context manager (`with Engine(...) as e:`) or have `.quit()` called explicitly.** Each spawned engine holds a kernel SysV semaphore (`semget(IPC_PRIVATE, ...)` in `unix.c`) for its input-dispatch loop, which is **only released by a graceful `quit`, not by the OS when the process is killed**. `kern.ipc.semmni` (max semaphore sets system-wide) is only 50 on this box; a session's worth of force-killed test engines can leak past that limit, after which *every new engine* silently falls back to a 100ms-per- command polling path (`_WaitUntilTheresInputToRead` in `input.c:63`) instead of the fast semaphore wait -- a ~1000x slowdown that looks exactly like "the machine is under heavy load" and cost significant debugging time before the real cause (`ipcs -s`, then `ipcrm -s ` for each leaked one) was found. ## Automated eval-tuning loop (`eval_tune/cycle.sh`) A later session built a full wash/rinse/repeat pipeline on top of the manual tuning tools above, so a cycle goes baseline -> tune -> head-to-head gate -> bake-in-if-better, repeatable indefinitely against a growing game pool. - **`filter_pgn.py --min-elo 2400 --min-ply 40`** -- fast text-only pre-filter (no python-chess board replay, so it runs in ~100s over a multi-GB pool) that drops short/low-rated games before they ever reach `tune_eval_dna.py`'s slower, board-replay-based filtering. Confirmed most of a raw TWIC pool's mass is short draws/low-rated games: filtering the full merged `twic.pgn` (4,099,094 games) down to >=2400 Elo / >40 ply kept 642,268 games (556MB) -- that's `twic_filtered.pgn`, the pool actually used for tuning and match play now. - **`match_play.py --pgn --games N --sd D`** -- head-to-head gate match, N games (games/2 unique openings, each played twice with colors swapped, so opening luck cancels), fixed search depth (not clock time, same sd-over-st reasoning as ECM below). Prints `CANDIDATE_SCORE=... LOWER95=...` -- **gate on `LOWER95 >= 0.5`, not the raw score.** Search is non-deterministic (MP=1 multithreaded), confirmed live: two engines loaded with *identical* DNA still split games unevenly at small N, so a bare score >=0.5 is not evidence of a real improvement, only a 95%-confidence lower bound that still clears break-even is. Three real bugs were found and fixed getting this to give sane numbers: 1. **The opening book was still live**, so early moves in every game were book lookups that never touched eval at all -- defeats the purpose of a match meant to compare eval quality. Fixed by pointing `book name` at a nonexistent file each game; `root.c`'s `g_uBookProbeFailures >= 3` gate then disables book probing for the rest of that game after 3 cheap failed opens. 2. **Engines were spawned with no `--hash`/`--cpus` at all**, so every game ran on `main.c`'s memset-zero default (a 65536-entry hash table) instead of the 256m used everywhere else in this project's own testing -- worse move ordering, much more re-search in complex positions. Fixed by passing `--hash 256m --cpus 1` explicitly (added an `extra_args` param to `tune_eval_dna.py`'s `Engine` class for this). 3. **Every engine process also defaulted to a *shared* `typhoon.log`** in cwd (same memset-zero default). With `--workers N` running `2*N` concurrent engines, they were all racing on `BackupFile()`-then-`fopen(wb+)` at startup and `fflush()`ing that one file on every `Log()`/`Bug()` call -- real lock contention, observed live as unexplained match slowdown. Fixed with `--logfile -`, which `main.c` special-cases to skip file logging entirely (no rename race, no fopen, no fflush). 4. Separately (not a match_play.py bug but cost real wall-clock time chasing it as one): `sample_openings()`'s original implementation did a sequential `chess.pgn.read_game()` scan from byte 0 of the pool file to pick even a couple of random openings -- minutes of wasted I/O/parsing on a 556MB pool. Rewritten to seek to random byte offsets and parse just the one game found there, cached the same way as `tune_eval_dna.py`'s `position_cache/` (keyed on pool identity + params) under `opening_cache/`. **If a match run seems to hang or take far longer than a lone `sd N` search should:** check for leftover engine processes from a previous killed/interrupted run first (`ps aux | grep typhoon`) before assuming a new bug -- a stray process from an earlier debug session silently eating a full CPU core was mistaken for a pipeline bug for a while before being found and killed. - **`bake_dna.py --eval-c eval.c --out eval.c`** -- writes a winning candidate's values into `eval.c`'s array initializers *in place*, preserving all formatting/comments/8x8 grid layout (regex-replaces only the numeric literals found outside comments, in `g_EvalDNA[]` order, so a resulting `git diff` shows only the numbers that actually changed). Round-trip verified: dumping the currently-compiled-in DNA and baking it right back produces a byte-identical `eval.c`. **Positional, not name-matched** -- a `.dna` file only works against the exact `eval.c` revision it was tuned against; a line-count mismatch against the current `g_EvalDNA[]` aborts loudly rather than silently misassigning values. - **`dna_trend.py cycle1.dna cycle2.dna ... [--cycles-dir eval_tune/cycles]`** -- across a chronological sequence of *kept* candidates, flags whether each raw DNA cell is trending (every step moves it the same direction -- real signal) or flip-flopping (moves a lot, nets out near zero -- chasing sampling noise in that cycle's position batch). Worth running every few kept cycles; a consistently flip-flopping cell is a candidate to exclude from future tuning passes rather than let it keep eating budget on noise. - **`cycle.sh [pgn] [n_games] [workers] [max_positions] [max_passes] [sd_depth]`** -- wires all of the above into one cycle: dump baseline -> tune -> gate match -> bake-in-and-rebuild if `LOWER95 >= 0.5`, else discard and leave `eval.c` untouched. **Never runs `git commit`** -- a win leaves `eval.c` modified-but-uncommitted for manual review, by design. `sd_depth` defaults to 10; a dry run at that depth was observed still running after 15+ minutes on one game before the bugs above were found -- use a shallow depth (4-6) for pipeline-wiring smoke tests, not 10. - **`evaldna write ` refuses to overwrite an existing file** (silent `Error writing dna file.` with no explanation of why) -- `WriteEvalDNA` in eval.c does `if (SystemDoesFileExist(...)) goto end;` rather than clobbering. Delete the target first when re-dumping a baseline. - **Watch for apples-to-oranges ECM comparisons across depths/time controls.** A baseline solved-count recorded at `st 1` is not a fair comparison point against a candidate scored at `sd 20` -- rescoring the baseline at the *same* fixed depth as the candidate revealed a real gap (711 vs 707 at `sd 20`, both far below the apparent 606-vs-711 jump you'd get comparing against the old `st 1` number) that was noise-level, not the large improvement it first looked like. Always diff ECM numbers taken at the same `sd` depth. ## Dynamic move ordering experiments (2026-08-29) A single long session that redesigned how quiet ("leftover", i.e. sub-`GOOD_MOVE`) moves get ordered, replacing several hand-picked-and-never- revisited heuristics with evidence-driven ones. Landed changes: retired `NumLeftoverMovesToSelect` (see below), added a continuation-history table, retired hung-piece-escape's unconditional tier promotion in favor of a same-tier `FLEE_BONUS` nudge, and made the countermove table's tier promotion evidence-gated instead of automatic. See `generate.c`'s `_ScoreAllMoves` routine-description comment for the resulting move-ordering hierarchy. The methodology below is the more durable takeaway -- reusable for any future move-ordering question, not just the ones it already answered. **Core technique: measure the *pick*, not the *game*.** Aggregate solve counts on a 90-position suite are too noisy to tune a single move-ordering knob against (a handful of positions flipping either way swamps the signal) -- confirmed independently twice now, once by a prior session (see `lmr_testing/RESULTS.md`'s "given-up leftover"/"would-prune surprise rate" methodology) and once by this one. Instead, instrument the *local, per-move outcome* (did trying this move raise alpha or fail high?) at a much larger sample size, and read percentages off that instead of solve/pass counts. - **The "select nothing" / zero-selection-budget trick**: temporarily force the leftover-selection budget to 0 (a compile-time constant swap + rebuild, not a runtime flag -- keep this diagnostic-only, never commit it) so that only the single "discovered we're in leftover territory" transition move gets a full `SelectBestWithHistory` scan per node. That scan still runs over the *entire* remaining pool regardless of budget, so its result is genuinely "the best-of-remaining leftover, if we could only afford to pick one" -- a clean, isolated read on whether the move-ordering heuristic itself is any good, uncontaminated by how many picks the budget allows. This is what first revealed that the countermove table's exact- match signal was real (a forced-dominant version of it lifted the best-leftover fail-high rate by about +1 percentage point, consistently, across three curated suites) after a *scaled* continuation-history version of the same signal showed a flat, uninformative response curve across a 256x range -- the two experiments together showed scale wasn't the missing variable, key resolution was (exact-move match vs. a coarser [piece][to]-keyed proxy). - **The contested-node A/B harness**, for "should class A rank above or below class B" questions (e.g. countermove match vs. ply-2 killer): aggregate per-class fail-high rates are confounded by *censoring* -- whichever class is ranked higher gets tried first, so a lower-ranked class's measured rate only ever reflects the subset of nodes where nothing higher-ranked already resolved the position. Fix: at each node, cheaply detect (piggybacked on the *first* `SelectBestWithHistory` call's already-full-pool scan, no separate pass needed) whether *both* classes' candidates are present as genuinely different moves; only log an outcome on nodes where that's true, and only for the first candidate tried that belongs to either class. **Caveat discovered by running this twice** (once for countermove-vs-killer, once for hung-escape-vs-killer): whichever class is *structurally disadvantaged* in a given test setting only wins its rare contests via unusually strong self-reinforcing history/ continuation evidence -- a self-selection effect that inflates the disadvantaged class's apparent quality and deflates the favored class's, independent of which is actually the better signal. Don't read a single A/B setting's numbers at face value; compare each class's *least-filtered* sample (the setting where it's favored) against the other's, not the "loser" numbers from either individual run. - **Evidence calibration**: once a structural pattern-match (countermove table hit, en-prise escape) is identified, bucket every occurrence by its own accumulated `g_HistoryCounters`+`g_ContinuationHistory` evidence (log-ish bands: 0, 1-99, 100-999, ...) and plot fail-high rate per bucket. This is the test for "is this pattern-match alone trustworthy, or does it need a track record?" -- countermove matches with zero evidence scored statistically identically to an ordinary unprivileged leftover (~0.6- 0.85% FH) while evidence >=10,000 cleared 53-64%; hung-piece-escape's zero-evidence population was 250-1000x larger than countermove's *and* scored at the plain-leftover baseline too, revealing that the then- shipping unconditional promotion was handing free tier-escape treatment to a huge population that had done nothing to earn it. **Load-bearing gotcha, easy to reintroduce by accident**: a bonus added inside `SelectBestWithHistory` (a selection-time-only nudge to the local comparison value) never touches a move's persisted `iValue`, so it cannot change `fIsLeftoverMove`/EFP eligibility no matter how large it is -- useful for a same-tier nudge like `FLEE_BONUS`, useless if the goal is an actual tier promotion. A real promotion (like the evidence-gated countermove bonus) has to be written into `iValue` at *generation* time (`generate.c`), not selection time. Confirmed by testing: an earlier "dominant countermove" diagnostic added its huge bonus at selection time and, despite clearly winning every internal comparison, never once actually exempted a move from EFP or leftover classification -- the data it produced was still valid (still genuinely measuring leftover-pool behavior) but the mechanism didn't do what it looked like it should. **`NumLeftoverMovesToSelect` retirement**: the depth-indexed budget that decided how many leftover moves got a full `SelectBestWithHistory` scan before falling back to unsorted order was removed entirely once the evidence above showed the leftover pool has real, findable signal a bailout was discarding -- `search.c`'s main move loop now always fully selects. Aggregate node-count deltas from this and related changes bounced around by double-digit percentages on the smallest curated suite (`ecm_ringers`, 11 positions) with no corresponding solve change -- **on a suite this small, a..b search's chaotic sensitivity to move order (a few more depth-N-to-N+1 re-searches, a root fail-high or two) can move node counts a lot for reasons unrelated to the change being tested. Fail-high percentages and solve counts are the signal; raw node counts on small suites mostly aren't.** ## Environment notes - Shared, multi-user FreeBSD box with genuinely variable load (other Claude Code sessions, a `bhyve` VM, periodic `zpool scrub` on both `zroot` and `zscratch`). Don't trust a single throughput measurement without a back-to-back comparison under the same conditions; this session saw the same benchmark vary by 10x+ run to run for reasons that turned out to be the semaphore leak above, not genuine machine contention. - `/zscratch/egtb` and `/usr/home/scott/typhoon/pgn/twic.pgn` are the two large external resources this codebase's tooling depends on. `twic.pgn` was originally 554,135 games (up through TWIC issue 608, fetched by the long-dead `pgn/twic.pl`); a later session fetched all subsequent issues (920-1658) directly from theweekinchess.com (its WAF blocks bare `wget`, needs a real browser `User-Agent` + `Accept` header via `curl`) and merged them in, growing it to 4,099,094 games (3.5GB). Note issues 609-919 are still a gap (fetched range starts at 920) if completeness ever matters. `pgn/twic_filtered.pgn` (642,268 games, 556MB, >=2400 Elo / >40 ply, via `filter_pgn.py`) is the pool actually used for tuning/match-play now, not the raw file.