NNUE search integration: FH% tables, NODE_TYPE refinement, TREEDUMP rewrite

- data.c/chess.h: widen the joint FH% table's late axis 16->17, dedicating
  the new bucket 0 to a whole-node ("will something here fail high at
  all") sample rather than sharing it with the first-leftover-move curve.
  Add an NNUE-eval counterpart table (g_uJointFHThousandths_NNUE) and an
  NNUE counterpart of the beta-side table, selected at runtime based on
  whether NNUE eval is active.
- search.c/searchsup.c: add _RefineNodeTypeGuess, which uses the FH%
  tables (via g_NodeTypeGuessTable) to refine a node's NODE_TYPE guess
  instead of the old heuristic; thread uParentMoveFHEstimate through
  PLY_INFO so a node can see its parent's prediction for the move that
  led to it.
- generate.c: initialize sKillers[2]/[3].uBonus unconditionally (an
  uninitialized-stack read caught by valgrind under LTO; harmless today
  but a real bug).
- dumptree.c/chess.h/GNUmakefile: replace the long-dead DUMP_TREE XML
  tree dump (written for a GUI this project no longer has) with TREEDUMP,
  a flat, path-keyed move/result trace meant to diff against an
  identically-instrumented Stockfish build -- see docs/TREEDUMP_FORMAT.md.
- docs/DEEPER_PRUNING.md: notes on closing the search-efficiency gap vs
  Stockfish.
- crafty_egtb/stockfish_xboard/typhoon_nnue: wrapper scripts so
  match_play.py's Engine class can drive Crafty, Stockfish (via
  PolyGlot), and an NNUE-enabled typhoon the same way it drives HCE
  typhoon.
- tests/lop: rename lop.txt -> lop.ep_; update compare_lop.cpp.
- annotate_pgns: add aggregate_marginsafety.py, dedup_pgn.py,
  fit_lazy_probe.py; update annotate.py/run_nnue_pipeline.py for the NNUE
  training pipeline.
- .gitignore: ignore compile_commands.json (clangd build DB) and
  eval.nnue (a symlink into the untracked head_reference/ comparison
  baseline), neither of which belongs in the repo.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

1 files changed, 52 insertions, 0 deletions

diff --git a/src/crafty_egtb b/src/crafty_egtb
new file mode 100755
index 0000000..89b0f54
--- /dev/null
+++ b/src/crafty_egtb
@@ -0,0 +1,52 @@
+#!/usr/bin/env bash
+#
+# crafty_egtb -- thin wrapper so match_play.py's Engine class (which always
+# spawns its path with hardcoded TYPHOON flags, "--hash 256m --cpus 1
+# --logfile -" -- see match_play.py's engine_args) can drive Crafty instead.
+# Crafty speaks xboard protocol natively (Engine sends "xboard" at startup,
+# per match_play.py's own comments), so unlike stockfish_xboard this needs
+# no PolyGlot adapter -- but it also can't take typhoon's command-line
+# flags, so (same pattern as stockfish_xboard) every argument this script
+# is called with is deliberately DISCARDED.
+#
+# Crafty's own Syzygy/Fathom-based EGTB support reads the exact same
+# .rtbw/.rtbz files typhoon and stockfish_xboard point at (confirmed
+# 2026-10-18 via `strings crafty`: message strings "SYZYGY EGTB access
+# enabled, %d piece TBs found" / "Could not find %s.rtbw"). There is no
+# command-line flag or environment variable for this, despite a
+# `CRAFTY_TB_PATH` string existing in the binary (tried, confirmed it does
+# nothing) -- the ONLY way is two separate interactive commands:
+#   tbpath <dir>   -- just records the directory, no confirmation output
+#   egtb           -- actually TOGGLES probing on; THIS is what prints
+#                      "SYZYGY EGTB access enabled, N piece TBs found" and
+#                      is required in addition to tbpath, not instead of it.
+# Verified against a KBNK (bishop+knight mate) position: `go` without these
+# two commands searches ~877K/656K nodes per move; with them, depth 0
+# instantly returns "Won ... <EGTB>" in the PV.
+#
+# Since these are interactive commands (not args or env), they're injected
+# by piping them in ahead of whatever xboard/match_play.py writes to stdin
+# next -- `cat` with two input sources (this script's own heredoc, then
+# stdin) concatenates them for Crafty to read as one continuous stream, the
+# same way a human would type tbpath/egtb first and then start playing.
+#
+# Override via env vars if needed:
+#   CRAFTY_BIN   defaults to "crafty" (resolved via PATH, i.e.
+#                /usr/local/bin/crafty on this box -- NOT any older
+#                ~/programming/chess/crafty-25.2/crafty build, see
+#                eval_tune/compare_vs_crafty.py's comments on why that
+#                distinction matters)
+#   SYZYGY_PATH  defaults to /zscratch/egtb (same default typhoon's
+#                --egtbpath and stockfish_xboard's SYZYGY_PATH use). Set to
+#                "" to disable TB probing entirely (skips tbpath/egtb).
+
+set -e
+
+CRAFTY_BIN="${CRAFTY_BIN:-crafty}"
+SYZYGY_PATH="${SYZYGY_PATH:-/zscratch/egtb}"
+
+if [ -n "$SYZYGY_PATH" ]; then
+    { printf 'tbpath %s\negtb\n' "$SYZYGY_PATH"; cat; } | exec stdbuf -oL "$CRAFTY_BIN"
+else
+    exec stdbuf -oL "$CRAFTY_BIN" < /dev/stdin
+fi