Wire NNUE eval into search, build the training pipeline, and add a fast

move/eval-agreement benchmark against Stockfish (tests/lop)

Implements the architecture decisions in src/docs/NNUE.md: HalfKAv2_hm
feature set, int16/int8 quantization, per-ply accumulator backup for
king moves, and a lazy-eval-ready integration into search.c/eval.c.
Adds the full annotate_pgns/ training pipeline (extraction, K-fit/
labeling, quiet filtering, tensor building, training, export) plus an
end-to-end orchestrator (run_nnue_pipeline.py) and validation tooling
(sanity_check_net.py, compare_vs_stockfish_net.py). Adds tests/lop:
a 10k-position list-of-positions benchmark comparing engine eval rank
and move choice against pre-computed Stockfish/lc0 references, much
cheaper than a full self-play match for day-to-day progress checks.

First sd10-matched self-play result against classic HCE: LLR +2.28
(H1=+2.94), -72=119+169 over 360 games, confirming the net is
"smarter per node" at matched depth. A real-time (--st 4) match is
currently weaker (candidate loses ~3 plies of depth to a ~3.3-3.7x
nps gap against an unrecalibrated search), consistent with
NNUE.md's adoption-criterion section -- pruning/reduction constants
still need retuning for the new eval's score scale before a fair
real-time comparison is possible.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

1 files changed, 46 insertions, 6 deletions

diff --git a/annotate_pgns/annotate.py b/annotate_pgns/annotate.py
index 240b55f..1f8d02d 100644
--- a/annotate_pgns/annotate.py
+++ b/annotate_pgns/annotate.py
@@ -29,8 +29,13 @@ import chess.pgn
 from quiet_common import position_is_quiet
 
 TELLOTHERS_RE = re.compile(
-    r"^tellothers d(\d+), ([+-]?[\d.]+), [\d.]+s,\s*\d+knps, PV=(.*)$"
+    r"^tellothers d(\d+), ([+-]?[\w.]+), [\d.]+s,\s*\d+knps, PV=(.*)$"
 )
+FLOAT_RE = re.compile(r"^[+-]?\d+\.\d+$")
+MATE_RE = re.compile(r"^([+-])MATE(\d*)$")  # digit(s) optional -- the
+# "FORCED MATE --> stop searching now" immediate-short-circuit path
+# (search.c) prints a bare "+MATE"/"-MATE" with no distance number at
+# all, not just the ordinary "+MATE<N>" case.
 
 
 def load_games(input_paths, max_games=None):
@@ -122,17 +127,52 @@ def run_shard(engine_path, script_path, out_path, hash_size, cpus):
 
 
 def parse_shard_output(out_path):
-    """Return a list of (depth, score, pv_string) in script order."""
+    """Return a list of (depth, score_token, pv_string) in script
+    order. score_token is kept as the engine's raw tellothers token
+    (an ordinary "+1.23"-style float string, or "+MATE<N>"/"-MATE<N>")
+    rather than eagerly parsed to float here -- a MATE token has no
+    pawns magnitude to parse in the first place, and silently dropping
+    any line that doesn't match a plain float is exactly the bug that
+    used to desync this function's output from `index` (every position
+    after a dropped mate/EGTB line in a shard got the wrong score) --
+    see docs/NNUE.md's "Bug found and fixed in annotate.py's output
+    parser" status entry, previously patched only in the separate
+    extract_positions.py reader, not here."""
     results = []
     with open(out_path, encoding="utf-8", errors="replace") as f:
         for line in f:
             m = TELLOTHERS_RE.match(line.rstrip("\n"))
             if m:
-                depth, score, pv = m.groups()
-                results.append((int(depth), float(score), pv.strip()))
+                depth, score_tok, pv = m.groups()
+                results.append((int(depth), score_tok, pv.strip()))
     return results
 
 
+def _format_comment(depth, score_tok, pv):
+    """Render one (depth, score_token, pv) result as a PGN comment in
+    the same "{+-}SCORE/DEPTH PV" convention match_play.py's
+    _format_eval_comment uses for self-play games (and
+    extract_selfplay_positions.py's COMMENT_RE expects on read) --
+    translates the engine's own raw "+MATE<N>"/"-MATE<N>" tellothers
+    token into that convention's "+M"/"-M" (no distance digit, matching
+    match_play.py exactly -- mate distance isn't independently verified
+    here either). The PV's own sentinel token (<EGTB>/<REC>/<=>/<#>),
+    when present, is passed through verbatim since that is exactly what
+    both extract_positions.py and extract_selfplay_positions.py key
+    their exactness detection off of."""
+    m = MATE_RE.match(score_tok)
+    if FLOAT_RE.match(score_tok):
+        score_str = score_tok
+    elif m:
+        score_str = "+M" if m.group(1) == "+" else "-M"
+    else:
+        score_str = score_tok  # unrecognized token: pass through as-is
+    comment = f"{score_str}/{depth}"
+    if pv:
+        comment += f" {pv}"
+    return comment
+
+
 def annotate_shard(shard_games_list, index, results):
     """Attach comments to each game's mainline nodes in place. index and
     results are both in script order and the same length; index maps
@@ -148,8 +188,8 @@ def annotate_shard(shard_games_list, index, results):
         while node.variations:
             node = node.variations[0]
             if pi in scores:
-                depth, score, pv = scores[pi]
-                node.comment = f"{score:+.2f}/{depth} {pv}"
+                depth, score_tok, pv = scores[pi]
+                node.comment = _format_comment(depth, score_tok, pv)
             pi += 1