Wire NNUE eval into search, build the training pipeline, and add a fast

move/eval-agreement benchmark against Stockfish (tests/lop)

Implements the architecture decisions in src/docs/NNUE.md: HalfKAv2_hm
feature set, int16/int8 quantization, per-ply accumulator backup for
king moves, and a lazy-eval-ready integration into search.c/eval.c.
Adds the full annotate_pgns/ training pipeline (extraction, K-fit/
labeling, quiet filtering, tensor building, training, export) plus an
end-to-end orchestrator (run_nnue_pipeline.py) and validation tooling
(sanity_check_net.py, compare_vs_stockfish_net.py). Adds tests/lop:
a 10k-position list-of-positions benchmark comparing engine eval rank
and move choice against pre-computed Stockfish/lc0 references, much
cheaper than a full self-play match for day-to-day progress checks.

First sd10-matched self-play result against classic HCE: LLR +2.28
(H1=+2.94), -72=119+169 over 360 games, confirming the net is
"smarter per node" at matched depth. A real-time (--st 4) match is
currently weaker (candidate loses ~3 plies of depth to a ~3.3-3.7x
nps gap against an unrecalibrated search), consistent with
NNUE.md's adoption-criterion section -- pruning/reduction constants
still need retuning for the new eval's score scale before a fair
real-time comparison is possible.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

1 files changed, 135 insertions, 0 deletions

diff --git a/tests/lop/lop_out_to_triplets.py b/tests/lop/lop_out_to_triplets.py
new file mode 100755
index 0000000..627e4f5
--- /dev/null
+++ b/tests/lop/lop_out_to_triplets.py
@@ -0,0 +1,135 @@
+#!/usr/bin/env python3
+"""Convert a typhoon --logfile transcript of a lop.txt run (plain
+`setboard <fen>` / `go` script, xboard protocol) into the triplet
+format tests/lop/compare_lop expects: <fen>\\n<score>\\n<move>\\n
+repeated, one group per position.
+
+- <move> must be UCI coordinate notation (e.g. "d2c2"), matching the
+  convention already used in tests/lop/lop_10k_sf.txt and
+  lop_10k_lc0.txt -- NOT the SAN the engine prints ("move Kg5" in the
+  --logfile transcript). SAN is resolved to UCI via python-chess,
+  using the FEN that was actually sent for that position (the
+  "SCRIPT> <fen>" echo line), not the engine's internal re-print
+  ("FEN: ...", which zeroes the move-count fields and is not what was
+  asked).
+- <score> is an integer centipawn-ish value, side-to-move-relative,
+  from the "tellothers d<depth>, <score>, ..." summary line. A
+  "+MATE<N>"/"-MATE<N>" score is mapped to +-(100000 - N), matching
+  the existing reference files' observed range (e.g. mate-in-1 ->
+  99999) -- there is no mate notation in those files to confirm this
+  exact formula against, so this is a best-effort match to their
+  magnitude/sign convention, not a verified-identical one. A bare
+  "+MATE"/"-MATE" (no digits -- util.c's FormatScore prints this when
+  the reported move delivers mate itself) is treated as N=1.
+
+- A position where the engine never prints a "move"/"tellothers" line
+  before the next FEN arrives (observed for K-vs-K insufficient-
+  material draws, where the engine short-circuits without searching)
+  is written as score=0 with a blank move field, rather than silently
+  dropped. Detected generically, by absence (no move captured by the
+  time the next FEN shows up) -- not by matching the draw-announcement
+  text itself ("tellics draw" is written by main.c's Trace() call but
+  does not actually appear in --logfile output in --batch mode, so
+  matching for it directly does not work). A blank move field never
+  strcmp-matches a real reference move, which is correct: there is
+  nothing to agree or disagree with.
+"""
+import argparse
+import re
+import sys
+
+import chess
+
+FEN_RE = re.compile(r"^SCRIPT> (.+)$")
+MOVE_RE = re.compile(r"^move (\S+)$")
+SCORE_RE = re.compile(r"^tellothers d\d+, ([+-][\d.]+|[+-]MATE\d*),")
+
+
+def convert_score(score_str):
+    if "MATE" in score_str:
+        sign = 1 if score_str[0] == "+" else -1
+        digits = score_str.split("MATE")[1]
+        # util.c's FormatScore prints bare "+MATE"/"-MATE" (no digits) when
+        # the move being reported delivers mate itself (distance-to-mate 0
+        # plies past this move) -- treat that as mate-in-1 to match the
+        # existing reference files' observed mate-in-1 value (99999).
+        n = int(digits) if digits else 1
+        return sign * (100000 - n)
+    return int(round(float(score_str) * 100))
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument("logfile", help="engine --logfile transcript, e.g. src/lop.out")
+    ap.add_argument("out", help="output triplet file for compare_lop")
+    args = ap.parse_args()
+
+    cur_fen = None
+    cur_move_san = None
+    n_written = 0
+    n_san_failed = 0
+    n_bare_draws = 0
+
+    with open(args.logfile) as f, open(args.out, "w") as out:
+        for line in f:
+            line = line.rstrip("\n")
+
+            m = FEN_RE.match(line)
+            if m:
+                fen = m.group(1).strip()
+                if fen != "go":
+                    # A prior FEN that never got a move+score before this
+                    # next one arrived means the engine short-circuited
+                    # without searching (observed for K-vs-K insufficient-
+                    # material draws -- "tellics draw" isn't actually
+                    # written to --logfile output in --batch mode, so
+                    # detecting this generically, by absence, is more
+                    # robust than matching that text). Flush it as a
+                    # score=0/blank-move row rather than silently
+                    # dropping the position.
+                    if cur_fen is not None and cur_move_san is None:
+                        out.write(f"{cur_fen}\n0\n\n")
+                        n_written += 1
+                        n_bare_draws += 1
+                    cur_fen = fen
+                    cur_move_san = None
+                continue
+
+            mm = MOVE_RE.match(line)
+            if mm and cur_fen is not None:
+                cur_move_san = mm.group(1)
+                continue
+
+            ms = SCORE_RE.match(line)
+            if ms and cur_fen is not None and cur_move_san is not None:
+                score = convert_score(ms.group(1))
+                try:
+                    board = chess.Board(cur_fen)
+                    uci = board.parse_san(cur_move_san).uci()
+                except ValueError:
+                    n_san_failed += 1
+                    uci = cur_move_san  # fallback, will mismatch on strcmp
+                out.write(f"{cur_fen}\n{score}\n{uci}\n")
+                n_written += 1
+                cur_fen = None
+                cur_move_san = None
+
+        # Same short-circuit flush as above, for the last position in the
+        # file (no following SCRIPT> line to trigger it in the loop).
+        if cur_fen is not None and cur_move_san is None:
+            out.write(f"{cur_fen}\n0\n\n")
+            n_written += 1
+            n_bare_draws += 1
+
+    print(f"{n_written} positions written to {args.out}", file=sys.stderr)
+    if n_bare_draws:
+        print(f"  ({n_bare_draws} of those are bare-draw short-circuits "
+              f"with score=0 and a blank move field)", file=sys.stderr)
+    if n_san_failed:
+        print(f"WARNING: {n_san_failed} SAN moves failed to parse against "
+              f"their FEN -- written verbatim (will not UCI-match the "
+              f"reference file)", file=sys.stderr)
+
+
+if __name__ == "__main__":
+    main()