path: root/src/docs/ANALYZE_GAME.md

# Analyzing a game from blunders.pgn

A working playbook for pulling a specific game out of `blunders.pgn` and
figuring out *where the eval/search actually went wrong*, as opposed to
just noticing that the final result was bad. Written after a session spent
doing exactly this on the 2026-10-06 `typhoon vs Crafty-25.2` loss (a
KRP-vs-KQ endgame that looked like it had a scoring bug but turned out to
be legitimately lost, plus two real engine bugs found along the way).

## 0. What's actually in blunders.pgn

Each game's move comments are **only** `{score/depth time}`, e.g.
`{-5.09/34 6}` -- score in pawns from White's perspective (not side-to-move;
confirmed by the sign trending consistently negative as White loses, across
both White's and Black's own move comments), search depth reached, and
wall-clock seconds spent on that move. **There is no PV text and no engine
tag in the PGN at all** -- not `<REC>`, not `<EGTB>`, not `<#>`, nothing.
Whatever wrapper plays these self-play/vs-Crafty games strips the PV down to
just the eval number before writing the comment. If you want to see *why*
a move was chosen -- the actual PV, or whether a recognizer/tablebase hit
produced the score -- you have to replay the position through a live engine
yourself; the PGN alone can't tell you.

Don't go looking for `<EGTB>`/`<#>` tags in old PGNs for another reason too:
those sentinels (`EGTBMOVE` plus the `RECOGN_EGTB` wiring) are new, added in
the 2026-10-06 session that produced this document. Any game recorded before
that commit landed couldn't have produced them even if the PGN format did
carry PV text.

## 1. Pulling the game and exact FENs with python-chess

`python3 -c "import chess"` works in this environment (python-chess is
installed) -- use it rather than hand-parsing SAN, especially for games deep
enough that move numbers and ply counts diverge from intuition.

```python
import chess.pgn
f = open('blunders.pgn')
games = []
g = chess.pgn.read_game(f)
while g:
    games.append(g)
    g = chess.pgn.read_game(f)
last = games[-1]            # or games[N] for a specific game
board = last.board()
moves = list(last.mainline_moves())
```

**Getting a FEN at a specific PGN move number**: ply count and PGN move
number are NOT the same thing and it's easy to get this wrong (did exactly
this once already this session -- cost a wasted round-trip). The mapping:

```python
def ply(movenum, color):           # color: 'w' or 'b'
    return 2*(movenum-1) + (1 if color == 'w' else 2)
```

`moves[ply(69, 'w') - 1]` is White's 69th move (0-indexed list, 1-indexed
ply). Push moves up through the target ply and read `board.fen()` --
`board.halfmove_clock` is directly available too and matters a lot (see
"fifty-move-rule proximity" below).

To scan a whole stretch and watch halfmove_clock / material change move by
move (useful for finding exactly where a pawn/capture resets the fifty-move
counter, or where material transitions from one tablebase signature to
another):

```python
for i, m in enumerate(moves, start=1):
    before = board.copy()
    board.push(m)
    if lo <= i <= hi:
        print(i, before.san(m), 'halfmove_clock=', board.halfmove_clock,
              'fen=', board.fen())
```

## 2. Replaying a position through typhoon

Use `--batch --command` with `setboard`, not interactive stdin piping (per
CLAUDE.md, piping commands over stdin to a live REPL doesn't reliably
complete a multi-step script). Always pass `--egtbpath` explicitly if you
want to be sure which EGTB directory is in play (the hardcoded default is
`/zscratch/egtb`, which currently has full WDL+DTZ coverage through 6-man --
see CLAUDE.md -- but don't assume that silently when writing up findings;
state it).

```sh
./typhoon --cpus 1 --hash 256m --egtbpath /zscratch/egtb \
  --logfile /tmp/typhoon/<name>.log \
  --batch --command "force; setboard <fen>; eval; sd <depth>; go"
```

- `eval` alone (no search) gives the raw static eval -- useful as a sanity
  baseline before trusting a search score.
- `sd <depth>` fixes search depth; prefer this over `st <seconds>` for
  reproducibility (machine load varies a lot on this shared box -- see
  CLAUDE.md's ECM section).
- Watch stdout for these diagnostic lines, which tell you *how* the move was
  chosen, not just what it was:
  - `ONLY MOVE --> stop searching now` -- forced, no real search happened.
  - `TABLEBASE MOVE --> stop searching now` -- root EGTB (DTZ) root-move
    short-circuit; score will show as `PV=<EGTB>` and a flat `±NMATE`-ish
    "mate" display that is **not** a real mate distance (see section 4).
  - `FORCED MATE --> stop searching now` -- the iterative loop detected a
    mate score at the current depth and is stopping; the printed PV comes
    from `szLastPV`, which on a **root fail-high** is just the bare move
    (`MoveToSan`), not a real PV walk -- see section 5, don't expect `<#>`
    here.
  - `DEPTH LIMIT --> stop searching now` -- ordinary completion, nothing
    special.
- Grep the output/log for `tellothers` to get the final
  `score, time, nps, PV=...` summary line in one place.

## 3. Cross-checking with Crafty

`/usr/local/bin/crafty` is available and gives a genuinely independent
second opinion -- different engine, different (or absent) EGTB usage,
different search. Use it exactly the way you'd use typhoon:

```sh
echo "setboard <fen>
sd <depth>
go
quit" | timeout 20 /usr/local/bin/crafty 2>&1 | tail -30
```

Crafty's output is far chattier (full eval component breakdown, PV with
move count, nodes/nps) -- look for the `depth ... score ... variation`
block and the `note: scores are for the white side` line (confirms sign
convention, which is directly comparable to typhoon's White-POV scores).

**Why this matters**: if typhoon reports something wild (a huge mate-ish
score with a PV that goes nowhere, or a score frozen for 20+ plies in a row)
and Crafty reports an ordinary, non-mate score with a PV that's actually
converging, that's your signal the *typhoon* number is suspect, not that the
position itself is unusual. This exact comparison is what caught the
`ProbeEGTBRoot` DTZ-as-mate-distance bug (section 4) -- typhoon said
`-MATE4`, Crafty's independent 16-ply search said `-7.38` with no mate
anywhere in a 13-move PV.

## 4. Things that look like bugs but are fifty-move-rule artifacts (and one that was a real bug)

**Watch `halfmove_clock` whenever a long endgame grind produces a
suspicious score.** Syzygy tables report **WDL** (win/draw/loss) and
**DTZ** (distance to the next fifty-move-rule-*zeroing* move -- a capture or
pawn push), not DTM (distance to mate). This project's EGTB directory has
**zero `.rtbm` files** -- DTM tables were never downloaded, and Fathom's DTM
code (`probe_dtm_table`, `TB_probe_dtm`, etc.) is present in the source but
dead weight without data behind it. So:

- A genuinely won endgame can sit at a **small DTZ** while still being many
  plies from actual mate, especially once `halfmove_clock` is climbing
  toward 100 (a reset becomes forced soon regardless of how far mate
  actually is).
- A score that stays flat for many consecutive moves, or a score that
  jumps to a huge "mate-ish" magnitude with a PV that doesn't resolve
  anywhere close to that many plies later, is the signature of this --
  **not** evidence of missing/corrupt tablebase files by itself. Check
  `ls /zscratch/egtb | grep <material-signature>` to rule out a genuine
  coverage gap first, but if the files are present, suspect the
  DTZ-vs-mate-distance conflation next.
- This was a **real, fixed bug** as of 2026-10-06: `probe.c`'s
  `ProbeEGTBRoot` used to compute `*piScore = INFINITY_SCORE ∓ uDtz` and
  report it through the mate-score display path -- i.e. it fabricated a
  "mate in N" number out of a quantity that was never a mate distance. Fixed
  to report a flat `+NMATE`/`-NMATE` (same convention `ProbeEGTB`/
  `RecognLookup` already use at interior nodes) instead. If you're
  diagnosing a *pre-fix* game (anything before this fix landed -- check
  `git log probe.c`), expect to see exactly this artifact and don't chase
  it as a new finding.
- The **move** Fathom picks at the root is unaffected by any of this --
  `tb_probe_root()` does its own internal DTZ-based ranking before
  returning, so the chosen move is already DTZ-optimal regardless of
  whether the *score* we derived from it was sane. Don't second-guess the
  move choice because the displayed score looked wrong; check the score
  logic and the move logic as separate questions.

## 5. Reading PV sentinels (and when you won't see them)

`WalkPV` (`util.c`) terminates a printed PV early and appends a tag when it
hits one of five sentinel "moves" (`data.c`): `HASHMOVE` → `<TT>`,
`RECOGNMOVE` → `<REC>`, `DRAWMOVE` → `<=>`, `MATEMOVE` → `<#>`, `EGTBMOVE` →
`<EGTB>`. These mark *why the PV stopped*, not what the move was:

- `<REC>` / `<EGTB>`: an interior-node `RecognLookup` hit resolved the
  score outright. `RECOGN_EGTB` (→ `<EGTB>`) means an actual on-disk Syzygy
  WDL probe succeeded; `RECOGN_EXACT` (→ `<REC>`) means one of `recogn.c`'s
  hand-written classic recognizers (KPK, KBKP, etc.) fired instead. Before
  2026-10-06 these were indistinguishable in the PV (both reported
  `RECOGNMOVE`) -- `RecognLookup` had an `RECOGN_EGTB` enum value sitting
  unused the whole time, always returning `RECOGN_EXACT` even on a genuine
  Fathom hit. If you're looking at an old log/build, `<REC>` could mean
  either; `<EGTB>` specifically didn't exist yet.
- `<#>`: search found an actual forced checkmate (zero legal moves + in
  check) at a PV leaf.
- `<=>`: a drawn leaf (zero legal moves, not in check == stalemate; this
  sentinel is also used for recognized/forced draws elsewhere).
- **Root tablebase moves get `<EGTB>` in `szLastPV` directly** (as a literal
  string, not via `WalkPV` -- `ProbeEGTBRoot`'s success path short-circuits
  the whole iterative loop and never calls `WalkPV` at all for that move).

**The big gotcha**: `PrintPV` (`util.c`) only calls the real `WalkPV` walk
-- the one that can surface any of these tags -- when the reported score is
a genuine PV node, `iAlpha < iScore < iBeta`. On a **root fail-high**
(`iScore > iBeta`, i.e. the aspiration window is still widening upward),
it falls back to printing just the bare best move via `MoveToSan`, with no
PV walk and therefore no sentinel, ever -- even if the position is a
dead-obvious mate-in-1 whose leaf genuinely wrote `MATEMOVE` two plies down.
So: **don't read "no `<#>` in the output" as "no mate was found"** -- check
whether the line says `FORCED MATE` (a fail-high-driven stop) vs. a normal
depth-complete report before concluding anything about whether the engine
actually resolved the mating line as an exact PV.

## 6. The timer-elapsed-time gotcha

If you see `Searched for -<huge number> seconds` in output, that's not a
real timing anomaly to chase -- or rather, it *was* a real bug (fixed
2026-10-06): `Iterate()` (`root.c`) used to stamp
`g_MoveTimer.dEndTime = SystemTimeStamp()` partway through the function,
*before* the shared `end:` label that every early-exit path (`ONLY MOVE`,
`TABLEBASE MOVE`, pre-move terminal states) jumps to via `goto end`. Those
paths skipped the stamp entirely, leaving `dEndTime` stale (0, or whatever
the previous unrelated search last set it to), which produces exactly this
kind of nonsense negative "elapsed time" when the end-of-search report
computes `dEndTime - dStartTime`. Fixed by moving the stamp to right after
the `end:` label itself. If you're diagnosing a pre-fix build and see this,
it's cosmetic (doesn't affect move choice or score), but it's also a tell
that you're looking at a root short-circuit path (tablebase or only-move),
which is useful context for *why* no real iterative search happened at that
node.

## 7. General workflow summary

1. Identify the game (usually "last game in the file") and get the exact
   FEN at the move(s) you care about via python-chess -- don't eyeball SAN
   move numbers against ply counts by hand.
2. Check `board.halfmove_clock` at that FEN before anything else if it's an
   endgame -- rules out/in the fifty-move-rule class of artifact immediately.
3. Replay the FEN through `./typhoon --batch --command "force; setboard
   <fen>; sd <depth>; go"`, read the diagnostic `--> stop searching now`
   line to understand *which* code path produced the move, and check
   `tellothers`'s `PV=` field.
4. Get a second opinion from `/usr/local/bin/crafty` on the same FEN at
   comparable depth. Large disagreement (especially "my engine claims near-
   mate, Crafty's PV goes nowhere near that far") is the signal to dig into
   the score-computation code, not the position.
5. If something still looks wrong, read the actual source for the code path
   the diagnostic line pointed at (`root.c`'s `Iterate`, `probe.c`'s
   `ProbeEGTB`/`ProbeEGTBRoot`, `recogn.c`'s `RecognLookup`) rather than
   guessing from behavior alone -- every bug found this way so far
   (`ProbeEGTBRoot`'s DTZ-as-mate-score, the `dEndTime` placement, the
   dead `RECOGN_EGTB` branch) was a straightforward, small, findable-in-
   minutes fix once the actual code was in front of us; none of them
   required deep speculation.