typhoon.git/annotate_pgns/run_nnue_pipeline.py, branch master

NNUE search integration: FH% tables, NODE_TYPE refinement, TREEDUMP rewrite

2026-10-23T20:19:17Z

- data.c/chess.h: widen the joint FH% table's late axis 16->17, dedicating the new bucket 0 to a whole-node ("will something here fail high at all") sample rather than sharing it with the first-leftover-move curve. Add an NNUE-eval counterpart table (g_uJointFHThousandths_NNUE) and an NNUE counterpart of the beta-side table, selected at runtime based on whether NNUE eval is active. - search.c/searchsup.c: add _RefineNodeTypeGuess, which uses the FH% tables (via g_NodeTypeGuessTable) to refine a node's NODE_TYPE guess instead of the old heuristic; thread uParentMoveFHEstimate through PLY_INFO so a node can see its parent's prediction for the move that led to it. - generate.c: initialize sKillers[2]/[3].uBonus unconditionally (an uninitialized-stack read caught by valgrind under LTO; harmless today but a real bug). - dumptree.c/chess.h/GNUmakefile: replace the long-dead DUMP_TREE XML tree dump (written for a GUI this project no longer has) with TREEDUMP, a flat, path-keyed move/result trace meant to diff against an identically-instrumented Stockfish build -- see docs/TREEDUMP_FORMAT.md. - docs/DEEPER_PRUNING.md: notes on closing the search-efficiency gap vs Stockfish. - crafty_egtb/stockfish_xboard/typhoon_nnue: wrapper scripts so match_play.py's Engine class can drive Crafty, Stockfish (via PolyGlot), and an NNUE-enabled typhoon the same way it drives HCE typhoon. - tests/lop: rename lop.txt -> lop.ep_; update compare_lop.cpp. - annotate_pgns: add aggregate_marginsafety.py, dedup_pgn.py, fit_lazy_probe.py; update annotate.py/run_nnue_pipeline.py for the NNUE training pipeline. - .gitignore: ignore compile_commands.json (clangd build DB) and eval.nnue (a symlink into the untracked head_reference/ comparison baseline), neither of which belongs in the repo. Co-Authored-By: Claude Sonnet 5

NNUE: re-fit raw-to-score calibration, NNU4 weight format, super-lazy default-on, lazy probe measured net-negative

2026-10-21T01:37:20Z

Fits an affine (a + b*raw) correction into every exported .nnue file (export_nnue_weights.py's CalibrateScoreMapping) so the engine's consumed centipawn output tracks Stockfish's real score scale rather than the net's nominal raw WDL-sigmoid output -- training variance alone shifts this mapping across retrains even with an identical recipe, so it's re-fit and baked in per export rather than assumed constant. Weight format bumped to "NNU4" to carry the two new floats; older NNU1/2/3 files are no longer readable. Super-lazy (material-only) pre-filter promoted to default-on (g_fUseNnueSuperLazy = TRUE) with a recalibrated per-material-bucket margin table (measured against the recalibrated net, 16-shard cross_sectional_200k.ep_ sample) -- real, measured nps win across all three curated suites. The separate lazy-probe gate (accumulator-based early exit) is measured net-negative on real search traffic at this net's probe-vs-forward latency ratio and stays off by default (g_fUseNnueLazyProbe = FALSE); code kept live since margin tuning could still flip this, per docs/NNUE.md. Also: direct per-call latency instrumentation for NnueProbeScore/ NnueForward (root.c, EVAL_TIME build) to make the lazy-probe breakeven math measurable directly instead of inferred; QSearch futility margins given real-data-derived NNUE_FULL/NNUE_LAZY starting points. Co-Authored-By: Claude Sonnet 5

Wire NNUE eval into search, build the training pipeline, and add a fast

2026-10-20T16:26:44Z

move/eval-agreement benchmark against Stockfish (tests/lop) Implements the architecture decisions in src/docs/NNUE.md: HalfKAv2_hm feature set, int16/int8 quantization, per-ply accumulator backup for king moves, and a lazy-eval-ready integration into search.c/eval.c. Adds the full annotate_pgns/ training pipeline (extraction, K-fit/ labeling, quiet filtering, tensor building, training, export) plus an end-to-end orchestrator (run_nnue_pipeline.py) and validation tooling (sanity_check_net.py, compare_vs_stockfish_net.py). Adds tests/lop: a 10k-position list-of-positions benchmark comparing engine eval rank and move choice against pre-computed Stockfish/lc0 references, much cheaper than a full self-play match for day-to-day progress checks. First sd10-matched self-play result against classic HCE: LLR +2.28 (H1=+2.94), -72=119+169 over 360 games, confirming the net is "smarter per node" at matched depth. A real-time (--st 4) match is currently weaker (candidate loses ~3 plies of depth to a ~3.3-3.7x nps gap against an unrecalibrated search), consistent with NNUE.md's adoption-criterion section -- pruning/reduction constants still need retuning for the new eval's score scale before a fair real-time comparison is possible. Co-Authored-By: Claude Sonnet 5