# typhoon chess engine A chess engine by Scott Gasch (scott.gasch@gmail.com), originally started 2004. Previously named "monsoon" -- any old references to that name mean this same codebase. Speaks the xboard/WinBoard protocol. FreeBSD host, 64-bit. ## Scratch files and logs **Put ad-hoc test/debug output under `/tmp/typhoon/`, not loose in `src/` or loose in `/tmp` itself.** Past sessions repeatedly dropped one-off ECM run logs, build logs, bisect binaries, and gprof output directly into `src/` (`ecm_v17_sn4m.stdout`, `lmr_A.log`, stray `typhoon_*` binaries, etc.) and directly into `/tmp` (`build6.log`, `nn_ringers.log`, `typhoon_pool_test3/`, multi-hundred-MB `typhoon_gprof.txt`, etc.), both of which build up uncommitted clutter that's easy to mistake for something meaningful and tedious to clean up later. Anything that isn't a deliberately-kept comparison baseline (`head_reference/`, the curated suites in `tests/`, `eval_tune/`'s pipeline scripts and `.dna` outputs) is scratch and belongs under `/tmp/typhoon/`. ## Building `build.sh`'s OS detection (`expr $OSTYPE = "darwin"`) is broken in this shell environment and falls through to a 32-bit `gcc`-based profile that doesn't exist here (no `gcc`, and the resulting `.o` files are architecture-mismatched against everything else). **Don't use build.sh as-is.** Build directly with: ```sh gmake -j5 GENETIC=1 PERF_COUNTERS=1 MP=1 SIXTYFOUR=1 ``` This uses `clang`/`clang++` (already the GNUmakefile default) and produces a 64-bit binary matching the historically-built one. After changing any header touched by many translation units, `gmake clean` first to avoid 32/64-bit object mismatches at link time. ## Running / xboard protocol basics No command-line flags are required for interactive use; the binary drops into an xboard-protocol REPL on stdin/stdout. Key commands used during development: - `force` -- stop the engine from auto-playing; you control both sides' moves. - `setboard ` -- set the position. - `eval` -- print `Static eval: ` (calls `Eval()` directly, no search). - `evaldna dump` / `evaldna read ` / `evaldna write ` -- the eval-constant import/export system (`g_EvalDNA` in eval.c). `evaldna` alone (no subcommand) just prints usage -- you need `evaldna dump` explicitly. - `sd ` / `st ` -- fixed depth vs. fixed time search limits. - `go` -- search and play one move for whichever side is on move, then stop (when paired with `force` after). Repeated `go`/`force` cycles correctly drive a full self-play game -- the engine tracks its own position across calls, no need to feed the opponent's move back manually. - `script ` -- run a list of commands from a file, used for test suites (see ECM below). Prints solved/unsolved stats at the end. - `--dnafile ` (startup flag) or `evaldna read ` (runtime) load a DNA file produced by tuning; `--egtbpath ` or `set EGTBPath ` configure Syzygy tablebase lookup. **Default EGTB path is hardcoded** in `main.c:297` to `/zscratch/egtb`, used whenever `--egtbpath` isn't passed. That directory currently has full 5-man WDL+DTZ Syzygy coverage but is **missing the WDL half of its 6-man set** (only `.rtbz`/DTZ files exist for 6-man combos) -- `init_tb()` in `fathom/tbprobe.c:752` requires the WDL file to exist before registering a combination at all, so every 6-man entry is silently skipped and the engine reports "up to 5 men" even though 6-man `.rtbz` files are sitting right there. Not a config issue; the fix is downloading the missing 6-man `.rtbw` files. ## Testing: the ECM tactical suite `tests/ecm.ep_` is a 3517-line / 881-problem EPD tactical test suite. `test.sh ` runs it and diffs against `lastrun.log` via `suite_diff.pl`. Two things to know: 1. **`suite_diff.pl`'s shebang (`#!/usr/bin/perl`) is wrong on this box** -- perl lives at `/usr/local/bin/perl`. Invoke it as `perl ./suite_diff.pl ` rather than `./suite_diff.pl`. 2. **Prefer `sd ` over `st ` for before/after eval comparisons.** A time-based limit lets identical positions reach different search depths across runs purely from incidental machine load (this session saw large timing swings from unrelated processes/scrubs) -- `sd 10` (or similar) holds search effort constant so a solve-rate difference is actually attributable to the eval change being tested. **Invocation**, matching what `test.sh` and `head_reference/logs/` use -- piping commands over stdin does not reliably run a `script` command to completion; use `--batch --command` instead, with the opening book pointed at a nonexistent path so early moves aren't book lookups (see the `match_play.py` bugs below for why that matters) and explicit `--hash`/ `--cpus` rather than the tiny memset-zero defaults: ```sh ./typhoon --cpus 1 --hash 256m --logfile /tmp/typhoon//sd10_.log \ --batch --command "force; book name /nonexistent.book.bin; sd 10; script ../tests/.ep_" ``` **Run these in the background and use Monitor to wait, don't block the foreground on them.** Each curated suite takes on the order of minutes at `sd 10`; a `wait` in a synchronous Bash call routinely exceeds a 2-minute tool timeout even though the run itself is fine. Launch with `run_in_background: true` (or equivalent), then use Monitor (or just continue the conversation -- a background command notifies on completion) rather than polling/sleeping in a loop. Current baseline (`lastrun.log`, `st 1`, unmodified eval constants): **606/879 solved.** A first full DNA-tuning pass against 600k TWIC positions raised this to **622/879** (see `eval_tune/`). `Trace()` (used for almost all engine output, including ECM's per-problem results) only `fflush()`s stdout, **not** the logfile -- `Log()`/`Bug()` do flush the logfile every call, but `Trace()` doesn't. A `--logfile` file can lag well behind actual progress while a run is in flight; it catches up fully once the process exits. Don't trust a logfile's line count as a live progress indicator for a running batch job. ### The three curated suites and the `head_reference` protocol Day-to-day search-change evaluation uses three small, curated EPD files under `tests/` rather than the full 881-problem `ecm.ep_` (too slow to iterate on) or a single random sample (too noisy -- a handful of flips either way swamps the signal at that size): - **`tests/ecm_ringers.ep_`** (11 positions) -- clean discriminators hand-picked because a baseline engine solves them reliably and a regressed one reliably doesn't (or vice versa). Fastest signal, but small enough that +/-1 solve is not necessarily real. - **`tests/ecm_confident_quick.ep_`** (90 positions) -- a larger, still-fast sample the engine is expected to do well on. - **`tests/ecm_hard_quick.ep_`** (90 positions) -- a harder sample, more sensitive to search-quality changes (pruning/reduction/ordering) since these positions need real depth/precision to crack. Run all three together, not just one -- a change can look great on one and cost solves on another (see `lmr_testing/RESULTS.md` for several examples this cut both ways). **Always compare against a fixed, known-good reference binary+logs, one variable at a time -- never against "the working tree as it happened to be a few hours ago."** `head_reference/` (sibling of `src/`, not `tests/`) holds exactly this: a binary built from a specific commit, plus full logs for all three suites at both `sd 10` (fixed depth) and `sn 5000000` (fixed node budget per position) in `head_reference/logs/`, and a `README.md` recording the commit hash and confirmed live/dead state of every search mechanism (LMR, EFP, etc.) at that point -- confirmed by reading the code, not assumed from memory of what a diagnostic build's `if (FALSE)` happened to say. Rebuild this directory (and update its README) whenever committing a change that becomes the new comparison baseline, so the next investigation always has something to check itself against before drawing conclusions -- this exact gap (assuming a disabled-in-the-working-tree mechanism was also disabled at HEAD, when it wasn't) cost a full afternoon re-deriving EFP's actual committed behavior mid-investigation. Pick `sd` vs `sn` based on what the change is expected to affect: `sd` (fixed depth) is the right choice when judging whether a pruning/ ordering change makes the tree smaller or larger for the same search effort; `sn` (fixed node budget) is the right choice when judging how many positions solve within a fixed cost. Don't run both for every single candidate change -- pick whichever matches what's actually being tested (the reference logs in `head_reference/` keep both, precisely so a later investigation can choose either without re-running HEAD). ## Eval tuning pipeline (`eval_tune/`) `tune_eval_dna.py` is a Texel-style tuner: drives `typhoon` as an xboard-protocol subprocess, fits `g_EvalDNA`'s ~1870 raw constants (reduced to ~468 after collapsing cells that share a starting value -- see caveat below) against TWIC game outcomes via sigmoid-squared-error + coordinate descent. `dna_diff.py` renders a human-readable, per-array diff between two `.dna` files (with 8x8 grids for the board-shaped location tables) so you can actually see what a tuning run changed. Position pools are cached under `position_cache/` keyed on source-file mtime+size+filter args, since a 500k-600k position PGN scan takes ~20 minutes. **Two real bugs were found and fixed this session, both in `command.c`, not in the Python tooling:** 1. **`EvalCommand` used to malloc + fully zero (`InitializeSearcherContext`, which memsets the whole ~75MB `SEARCHER_THREAD_CONTEXT` -- `rgPawnHash` and `rgEvalHash` are embedded arrays inside that struct, not separately-managed heap buffers) on every single `eval` call, then free it.** Fixed to keep one persistent context and use the cheap `ReInitializeSearcherContext` (just repositions + zeros counters) on subsequent calls -- ~1000x throughput improvement for scripted eval scoring. 2. **That persistent context's embedded eval hash was never invalidated when new DNA was loaded**, so after the fix above, a position scored once under any DNA would return that *same cached score forever*, silently ignoring every later `evaldna read`. Fixed via a global `g_uDnaGeneration` counter (bumped in `EvalDnaCommand` on successful load, checked in `EvalCommand`) that forces exactly one full re-init per DNA change, not per position. **If this regresses, tuning results will look like it "converged" almost immediately with implausibly little movement -- that's the symptom, not genuine convergence.** Verify by perturbing a few known-live parameters and confirming the reported error actually changes. **Known limitation, not yet fixed:** the distinct-value grouping used to shrink 1870 raw cells to ~468 tunable parameters ties together *any* cells that start out numerically equal, for whatever reason -- true mirror symmetry, an intentional-but-asymmetric design choice (e.g. `KING_INITIAL_COUNTER_BY_LOCATION`'s rank-8 row `1,0,0,0,1,0,0,1` flags a1/e1/h1 specifically, not a geometric mirror), or pure incidental coincidence (e.g. `PAWN_CENTRALITY_BONUS`'s legitimate "no bonus" squares happening to share the value `0` with unrelated off-board padding cells). The padding case is harmless (those indices are never read during real eval) but the coincidental-equality case is a real, unaddressed source of noise in the parameter space, most visible on small/noisy training runs (sentinel `-1` placeholder cells in `TRADE_PIECES` drifted to an arbitrary shared value on a 25k-position smoke test; they stayed put on the full 600k run). **`Engine`/`EnginePool` in `tune_eval_dna.py` must be used as a context manager (`with Engine(...) as e:`) or have `.quit()` called explicitly.** Each spawned engine holds a kernel SysV semaphore (`semget(IPC_PRIVATE, ...)` in `unix.c`) for its input-dispatch loop, which is **only released by a graceful `quit`, not by the OS when the process is killed**. `kern.ipc.semmni` (max semaphore sets system-wide) is only 50 on this box; a session's worth of force-killed test engines can leak past that limit, after which *every new engine* silently falls back to a 100ms-per- command polling path (`_WaitUntilTheresInputToRead` in `input.c:63`) instead of the fast semaphore wait -- a ~1000x slowdown that looks exactly like "the machine is under heavy load" and cost significant debugging time before the real cause (`ipcs -s`, then `ipcrm -s ` for each leaked one) was found. ## Automated eval-tuning loop (`eval_tune/cycle.sh`) A later session built a full wash/rinse/repeat pipeline on top of the manual tuning tools above, so a cycle goes baseline -> tune -> head-to-head gate -> bake-in-if-better, repeatable indefinitely against a growing game pool. - **`filter_pgn.py --min-elo 2400 --min-ply 40`** -- fast text-only pre-filter (no python-chess board replay, so it runs in ~100s over a multi-GB pool) that drops short/low-rated games before they ever reach `tune_eval_dna.py`'s slower, board-replay-based filtering. Confirmed most of a raw TWIC pool's mass is short draws/low-rated games: filtering the full merged `twic.pgn` (4,099,094 games) down to >=2400 Elo / >40 ply kept 642,268 games (556MB) -- that's `twic_filtered.pgn`, the pool actually used for tuning and match play now. - **`match_play.py --pgn --games N --sd D`** -- head-to-head gate match, N games (games/2 unique openings, each played twice with colors swapped, so opening luck cancels), fixed search depth (not clock time, same sd-over-st reasoning as ECM below). Prints `CANDIDATE_SCORE=... LOWER95=...` -- **gate on `LOWER95 >= 0.5`, not the raw score.** Search is non-deterministic (MP=1 multithreaded), confirmed live: two engines loaded with *identical* DNA still split games unevenly at small N, so a bare score >=0.5 is not evidence of a real improvement, only a 95%-confidence lower bound that still clears break-even is. Three real bugs were found and fixed getting this to give sane numbers: 1. **The opening book was still live**, so early moves in every game were book lookups that never touched eval at all -- defeats the purpose of a match meant to compare eval quality. Fixed by pointing `book name` at a nonexistent file each game; `root.c`'s `g_uBookProbeFailures >= 3` gate then disables book probing for the rest of that game after 3 cheap failed opens. 2. **Engines were spawned with no `--hash`/`--cpus` at all**, so every game ran on `main.c`'s memset-zero default (a 65536-entry hash table) instead of the 256m used everywhere else in this project's own testing -- worse move ordering, much more re-search in complex positions. Fixed by passing `--hash 256m --cpus 1` explicitly (added an `extra_args` param to `tune_eval_dna.py`'s `Engine` class for this). 3. **Every engine process also defaulted to a *shared* `typhoon.log`** in cwd (same memset-zero default). With `--workers N` running `2*N` concurrent engines, they were all racing on `BackupFile()`-then-`fopen(wb+)` at startup and `fflush()`ing that one file on every `Log()`/`Bug()` call -- real lock contention, observed live as unexplained match slowdown. Fixed with `--logfile -`, which `main.c` special-cases to skip file logging entirely (no rename race, no fopen, no fflush). 4. Separately (not a match_play.py bug but cost real wall-clock time chasing it as one): `sample_openings()`'s original implementation did a sequential `chess.pgn.read_game()` scan from byte 0 of the pool file to pick even a couple of random openings -- minutes of wasted I/O/parsing on a 556MB pool. Rewritten to seek to random byte offsets and parse just the one game found there, cached the same way as `tune_eval_dna.py`'s `position_cache/` (keyed on pool identity + params) under `opening_cache/`. **If a match run seems to hang or take far longer than a lone `sd N` search should:** check for leftover engine processes from a previous killed/interrupted run first (`ps aux | grep typhoon`) before assuming a new bug -- a stray process from an earlier debug session silently eating a full CPU core was mistaken for a pipeline bug for a while before being found and killed. - **`bake_dna.py --eval-c eval.c --out eval.c`** -- writes a winning candidate's values into `eval.c`'s array initializers *in place*, preserving all formatting/comments/8x8 grid layout (regex-replaces only the numeric literals found outside comments, in `g_EvalDNA[]` order, so a resulting `git diff` shows only the numbers that actually changed). Round-trip verified: dumping the currently-compiled-in DNA and baking it right back produces a byte-identical `eval.c`. **Positional, not name-matched** -- a `.dna` file only works against the exact `eval.c` revision it was tuned against; a line-count mismatch against the current `g_EvalDNA[]` aborts loudly rather than silently misassigning values. - **`dna_trend.py cycle1.dna cycle2.dna ... [--cycles-dir eval_tune/cycles]`** -- across a chronological sequence of *kept* candidates, flags whether each raw DNA cell is trending (every step moves it the same direction -- real signal) or flip-flopping (moves a lot, nets out near zero -- chasing sampling noise in that cycle's position batch). Worth running every few kept cycles; a consistently flip-flopping cell is a candidate to exclude from future tuning passes rather than let it keep eating budget on noise. - **`cycle.sh [pgn] [n_games] [workers] [max_positions] [max_passes] [sd_depth]`** -- wires all of the above into one cycle: dump baseline -> tune -> gate match -> bake-in-and-rebuild if `LOWER95 >= 0.5`, else discard and leave `eval.c` untouched. **Never runs `git commit`** -- a win leaves `eval.c` modified-but-uncommitted for manual review, by design. `sd_depth` defaults to 10; a dry run at that depth was observed still running after 15+ minutes on one game before the bugs above were found -- use a shallow depth (4-6) for pipeline-wiring smoke tests, not 10. - **`evaldna write ` refuses to overwrite an existing file** (silent `Error writing dna file.` with no explanation of why) -- `WriteEvalDNA` in eval.c does `if (SystemDoesFileExist(...)) goto end;` rather than clobbering. Delete the target first when re-dumping a baseline. - **Watch for apples-to-oranges ECM comparisons across depths/time controls.** A baseline solved-count recorded at `st 1` is not a fair comparison point against a candidate scored at `sd 20` -- rescoring the baseline at the *same* fixed depth as the candidate revealed a real gap (711 vs 707 at `sd 20`, both far below the apparent 606-vs-711 jump you'd get comparing against the old `st 1` number) that was noise-level, not the large improvement it first looked like. Always diff ECM numbers taken at the same `sd` depth. ## Environment notes - Shared, multi-user FreeBSD box with genuinely variable load (other Claude Code sessions, a `bhyve` VM, periodic `zpool scrub` on both `zroot` and `zscratch`). Don't trust a single throughput measurement without a back-to-back comparison under the same conditions; this session saw the same benchmark vary by 10x+ run to run for reasons that turned out to be the semaphore leak above, not genuine machine contention. - `/zscratch/egtb` and `/usr/home/scott/typhoon/pgn/twic.pgn` are the two large external resources this codebase's tooling depends on. `twic.pgn` was originally 554,135 games (up through TWIC issue 608, fetched by the long-dead `pgn/twic.pl`); a later session fetched all subsequent issues (920-1658) directly from theweekinchess.com (its WAF blocks bare `wget`, needs a real browser `User-Agent` + `Accept` header via `curl`) and merged them in, growing it to 4,099,094 games (3.5GB). Note issues 609-919 are still a gap (fetched range starts at 920) if completeness ever matters. `pgn/twic_filtered.pgn` (642,268 games, 556MB, >=2400 Elo / >40 ply, via `filter_pgn.py`) is the pool actually used for tuning/match-play now, not the raw file.