# TREEDUMP: a shared search-tree trace format for typhoon vs. Stockfish Both engines, when built with a `TREEDUMP` compile-time flag, write a flat, line-oriented, always-on trace of search decisions to a file named `treedump.log` in the current directory. There is no runtime toggle or filtering -- keep a dump small by using a shallow `sd N` (typhoon) / `go depth N` (Stockfish), not by restricting what gets written. The point is to run both engines on the *same FEN* at the *same shallow depth*, then diff/meld the two `treedump.log` files to see exactly where their trees diverge -- which move each engine reduces, prunes, or searches in full, and by how much. ## Building - typhoon: `gmake release PERF_COUNTERS=1 TREEDUMP=1` (or `debug`). Writes `treedump.log` in the working directory from process start to exit (`TDOpen`/`TDClose` in `main.c`). Single-threaded runs only (`--cpus 1`) -- the file write isn't locked against concurrent threads, matching this project's own `--cpus 1` convention for diagnostic/comparison runs. - Stockfish: `gmake build ARCH=x86-64-avx2 COMP=clang EXTRACXXFLAGS=-DTREEDUMP` (FreeBSD: use `gmake`, not `make` -- the stock `make` chokes on this Makefile's GNU-only syntax; `COMP=clang` because there's no `gcc` on this box; `ARCH=x86-64-avx2`, not bare `x86-64` or `ARCH=native` -- both of the latter either fail FreeBSD's arch-detection outright or silently build a popcnt-less binary that reproducibly returns a dead/empty search, 0 nodes, on every `go`, with no error -- see the gotcha below, this is a different failure mode from it and easy to conflate with it). Also writes `treedump.log` in the working directory, for the life of the process. Single-threaded only (`Threads` UCI option defaults to 1; don't raise it for a comparison run). **`EXTRACXXFLAGS` isn't tracked as a `make` dependency** -- if you rebuild the same ARCH with/without it back to back, `search.cpp` (the only file TREEDUMP touches) won't be recompiled and you'll silently get the previous binary; `touch search.cpp` first, or `gmake clean` first, whenever toggling the flag. ### A Stockfish-specific gotcha: piping input races the search thread If you drive Stockfish non-interactively by piping commands in and letting the pipe close right after (`printf '...\ngo depth N\n' | ./stockfish`), the engine's input-reading thread can see EOF and begin shutting the engine down *before* the search thread has actually started producing nodes -- the symptom is `info depth 1 ... nodes 0 ... pv ` (empty) immediately followed by a `bestmove`, for every UCI limit type (`depth`, `nodes`, `movetime` all equally ignored), with exit code 0 (not a crash). This reproduced on this box on a **clean, unmodified** Stockfish build too, so it's not a TREEDUMP artifact -- it's inherent to how this fork's shared-memory NNUE bootstrap (`shm.h`/`shm_unix.h`, "Network replica N: Shared memory" in the startup banner) races against stdin closing, and it was intermittent enough (roughly 1 good run in ~15-18 attempts, observed empirically) that blind retrying is not a reliable workaround. **Fix: hold stdin open past the point the search should finish**, e.g.: ```sh (printf 'position fen \ngo depth N\n'; sleep 2) | ./stockfish ``` An interactive terminal session (typing the commands directly, or redirecting from a real tty) never hits this, which is why it's easy to not reproduce it while debugging interactively and then hit it again the next time a script pipes input in. ## Why path-keyed, not node-numbered Internal node counters are meaningless across two different engines -- their recursion order, node numbering, and even which positions get revisited (transposition table hits) differ completely. The only thing that identifies "the same position" in both engines is the sequence of moves played from the root, since both start from the same FEN. Every line is therefore keyed by `path=""` -- sort or `grep` on that field to line up corresponding decisions from the two logs regardless of visit order. A null move is spelled `0000` in a path (both engines already use this UCI convention for `Move::null()`/`NULLMOVE` internally -- it's not an invented token). ## Record types Two kinds of line, both single-line, both flushed immediately (so a killed/crashed process still leaves a readable partial log): ``` MOVE path="" depth= alpha= beta= kind= act= r= ext= score= RESULT path="" depth= alpha= beta= eval= bound= score= ``` - `depth`: remaining search depth in whole plies (float, since typhoon's internal depth is fixed-point in `ONE_PLY`-sized ticks and extensions/reductions can be fractional ply amounts; Stockfish's is already integer plies and just prints with no fractional part). - `alpha`/`beta`: the window at the node the move/result belongs to. For a `RESULT` line, this is the node's *original* (pre-aspiration- widening) window, so the `bound` field's classification is consistent. - `kind` (MOVE only): rough move classification -- `FIRST`, `HASH`, `CAP`, `QUIET`, `NULL`, `PROBCUT`, freeform otherwise. Not identical granularity between the two engines (see Known Gaps below) -- treat it as a hint, not a guaranteed-matching taxonomy. - `act` (MOVE only): what happened to this move -- `FULL` (searched at full/nominal depth), `LMR` (late-move-reduced), `LMR-RESEARCH` (Stockfish only: a reduced search failed high and got a full-depth re-search), `EXT` (extended), `LMP` (pruned without being searched, "late move pruning"), `FP-CAP`/`SEE-CAP`/ `HIST-PRUNE`/`FP-QUIET`/`SEE-QUIET` (Stockfish's shallow-depth move pruning variants -- typhoon currently has no direct equivalent to several of these, which is itself one of the interesting things to look for in a diff), `NMP` (null-move probe), `PROBCUT` (Stockfish only -- typhoon has no ProbCut at all). - `r`/`ext` (MOVE only): magnitude, in whole plies, of any reduction (`r`) or extension (`ext`) applied. Exactly one is normally nonzero. - `score`: the score this move/node returned, or the literal `SKIPPED` on a `MOVE` line for a move that was pruned without ever being searched. - `eval` (RESULT only): the node's static eval. May be stale/0 on a typhoon node that returned before `Eval()` ran (e.g. an immediate hash/recognizer cutoff) -- diagnostic only, don't trust it blindly. ## Known gaps / asymmetries (read before drawing conclusions from a diff) - **Neither engine traces into qsearch.** Qsearch nodes vastly outnumber full-width nodes even at `sd 4`/`go depth 4` and would swamp a diff of the full-width move-ordering/pruning decisions that are actually of interest. This means a TREEDUMP diff only shows the full-width tree shape, not what happens once captures start resolving near the horizon -- keep comparisons shallow enough that qsearch isn't where most of the interesting divergence lives, or treat this as future work if it turns out qsearch matters too. - **`kind` is not a matched taxonomy.** typhoon's `TDMove` call sites classify moves coarsely (`FIRST`/`CAP`/`QUIET`/`NULL`); Stockfish's do the same but the two engines' internal move-ordering concepts (killers, countermoves, TT move, etc.) don't line up 1:1. Don't diff on `kind` expecting an exact match -- diff on `act`, `r`/`ext`, and `score` for a given `path`, and use `kind` only as a sanity check. - **typhoon has no ProbCut.** Any `act=PROBCUT` line will only ever appear in Stockfish's log. This is itself a structural difference worth noting, not a bug in the instrumentation. - **Stockfish's null-move `depth` field is the full pre-reduction parent depth minus R; typhoon's null-move `MOVE` line's `depth` field is the already-reduced probe depth** (`TryNullmovePruning` never receives the pre-reduction parent depth) -- compare the *absolute* depth value in context, don't assume the two `r` fields for null moves are computed the same way (typhoon's is left `0.00` for null moves rather than faked). - **typhoon's `RESULT.eval` can be stale (0 or a leftover value)** on any node that returned before `Eval()` ran. Stockfish's `eval` field comes from `ss->staticEval`, which has its own, different set of early-return paths that skip a real eval. Don't diff raw eval numbers across engines expecting them to be comparable at all (they use different evaluation functions) -- only use `eval` to sanity- check a single engine's own `RESULT` line against its own earlier `MOVE` lines. ## Suggested workflow ```sh # typhoon side cd /usr/home/scott/typhoon/src gmake release PERF_COUNTERS=1 TREEDUMP=1 rm -f treedump.log ./typhoon --cpus 1 --hash 8m --logfile - --batch --command \ "force; book name /nonexistent.book.bin; setboard ; sd 4; go" cp treedump.log /tmp/typhoon/td_typhoon.log # stockfish side cd ~/programming/chess/stockfish/src touch search.cpp # EXTRACXXFLAGS isn't a tracked dependency -- force the recompile gmake build ARCH=x86-64-avx2 COMP=clang EXTRACXXFLAGS=-DTREEDUMP rm -f treedump.log (printf 'position fen \ngo depth 4\n'; sleep 2) | ./stockfish cp treedump.log /tmp/typhoon/td_sf.log # compare (sort by path first so corresponding positions line up # regardless of each engine's own visit order) sort -t'"' -k2 /tmp/typhoon/td_typhoon.log > /tmp/typhoon/td_typhoon.sorted sort -t'"' -k2 /tmp/typhoon/td_sf.log > /tmp/typhoon/td_sf.sorted meld /tmp/typhoon/td_typhoon.sorted /tmp/typhoon/td_sf.sorted ``` Rebuild each engine back to its normal release profile afterward (`TREEDUMP` should never be the profile left in place for a real run -- it's diagnostic-only, same spirit as this project's other collection- build flags).