Wire NNUE eval into search, build the training pipeline, and add a fast

move/eval-agreement benchmark against Stockfish (tests/lop)

Implements the architecture decisions in src/docs/NNUE.md: HalfKAv2_hm
feature set, int16/int8 quantization, per-ply accumulator backup for
king moves, and a lazy-eval-ready integration into search.c/eval.c.
Adds the full annotate_pgns/ training pipeline (extraction, K-fit/
labeling, quiet filtering, tensor building, training, export) plus an
end-to-end orchestrator (run_nnue_pipeline.py) and validation tooling
(sanity_check_net.py, compare_vs_stockfish_net.py). Adds tests/lop:
a 10k-position list-of-positions benchmark comparing engine eval rank
and move choice against pre-computed Stockfish/lc0 references, much
cheaper than a full self-play match for day-to-day progress checks.

First sd10-matched self-play result against classic HCE: LLR +2.28
(H1=+2.94), -72=119+169 over 360 games, confirming the net is
"smarter per node" at matched depth. A real-time (--st 4) match is
currently weaker (candidate loses ~3 plies of depth to a ~3.3-3.7x
nps gap against an unrecalibrated search), consistent with
NNUE.md's adoption-criterion section -- pruning/reduction constants
still need retuning for the new eval's score scale before a fair
real-time comparison is possible.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

1 files changed, 78 insertions, 16 deletions

diff --git a/src/eval_tune/test_vs_head.sh b/src/eval_tune/test_vs_head.sh
index 71b2282..fa44d44 100755
--- a/src/eval_tune/test_vs_head.sh
+++ b/src/eval_tune/test_vs_head.sh
@@ -75,12 +75,41 @@
 # engine_args to both sides):
 #
 #   NNUE_WEIGHTS=/path/to/net.nnue test_vs_head.sh
+#
+# NNUE_PROBE_WEIGHTS: path to a probe weights file (as produced by
+# annotate_pgns's probe-fitting script, "NPRB" format -- see docs/
+# NNUE.md's "Lazy-evaluation tiering plan"). Requires NNUE_WEIGHTS to
+# also be set (the probe gate only does anything once NNUE eval itself
+# is enabled) -- adds "--nnueprobeweights $NNUE_PROBE_WEIGHTS" to the
+# same wrapper, enabling the lazy-eval accumulator probe gate at
+# startup (main.c's --nnueprobeweights sets g_fUseNnueLazyProbe=TRUE
+# automatically, same convention as --nnueweights):
+#
+#   NNUE_WEIGHTS=/path/to/net.nnue \
+#   NNUE_PROBE_WEIGHTS=/path/to/probe.bin \
+#       test_vs_head.sh
 
 set -e
 
-HEAD="${1:-../../head_reference/typhoon}"
-CANDIDATE_ARG="${2:-}"
-LABEL="${3:-match}"
+# Only consume LEADING args that don't look like a flag as the three
+# positionals (head/candidate/label) -- 2026-10-19 fix: a bare
+# ./test_vs_head.sh --sd 10 used to land "--sd" in HEAD and "10" in
+# CANDIDATE_ARG (since the old code unconditionally took "$1"/"$2"/"$3"
+# regardless of content), the exact same footgun class the comment
+# below already describes once biting the --shift-then-test-"$2"
+# version of this script. A flag-looking first arg now means "no
+# positionals given at all, everything is pass-through" instead.
+POSITIONALS=()
+while [ $# -gt 0 ] && [ "${#POSITIONALS[@]}" -lt 3 ]; do
+    case "$1" in
+        -*) break ;;
+        *) POSITIONALS+=("$1"); shift ;;
+    esac
+done
+
+HEAD="${POSITIONALS[0]:-../../head_reference/typhoon}"
+CANDIDATE_ARG="${POSITIONALS[1]:-}"
+LABEL="${POSITIONALS[2]:-match}"
 # Note the `-` (not `:-`) for the three env overrides below: an
 # explicitly EMPTY value must be honored, not replaced by the default.
 # SPRT_ARGS="" is the documented way to run without a hypothesis test
@@ -90,18 +119,17 @@ TC_ARGS="${TC_ARGS---st 4}"
 SPRT_ARGS="${SPRT_ARGS---sprt --elo0 0 --elo1 5}"
 GAMES="${GAMES-10000}"
 
-# Drop the (up to) three positional args so "$@" holds only extra
-# pass-through flags for match_play.py. This MUST come after the three
-# positionals have been captured into HEAD/CANDIDATE_ARG/LABEL above:
-# an earlier version of this shifted first and then tested "$2" to
-# decide whether to rebuild, which (a) made every no-extra-args run
-# take the rebuild branch and silently clobber ../typhoon with a
-# different make profile, and (b) made the first pass-through FLAG VALUE
-# land in "$2" and get used as the candidate BINARY PATH -- producing a
-# baffling "PermissionError: ... Permission denied: <scratch dir>" from
-# Popen trying to execute a directory.
-for _ in 1 2 3; do
-    if [ $# -gt 0 ]; then shift; fi
+# If the caller already passed --sd or --st through as an extra
+# argument (e.g. ./test_vs_head.sh --sd 10), don't ALSO inject the
+# baked-in TC_ARGS ("--st 4" by default) -- match_play.py's --sd/--st
+# are mutually exclusive, so the two would collide
+# ("error: argument --st: not allowed with argument --sd") even though
+# the caller only asked for one of them. Pass-through wins; TC_ARGS is
+# only the fallback when the caller didn't specify either.
+for arg in "$@"; do
+    case "$arg" in
+        --sd|--st) TC_ARGS="" ;;
+    esac
 done
 
 if [ -n "$CANDIDATE_ARG" ]; then
@@ -117,14 +145,47 @@ fi
 
 if [ -n "${NNUE_WEIGHTS:-}" ]; then
     NNUE_WRAPPER=/tmp/typhoon/typhoon_match_play_candidate_nnue
+    NNUE_FLAGS="--nnueweights \"$NNUE_WEIGHTS\""
+    if [ -n "${NNUE_PROBE_WEIGHTS:-}" ]; then
+        NNUE_FLAGS="$NNUE_FLAGS --nnueprobeweights \"$NNUE_PROBE_WEIGHTS\""
+    fi
+    # NNUE_LAZY_MARGIN / NNUE_SUPERLAZY_MARGIN: 2026-10-16 looser-
+    # margin experiment (see docs/NNUE.md and command.c's "nnue
+    # lazymargin"/"nnue superlazyon"). NNUE_LAZY_MARGIN overrides every
+    # material bucket's lazy-probe margin with one flat cp value
+    # instead of the calibrated p98 table; NNUE_SUPERLAZY_MARGIN turns
+    # on the even-cheaper material-only pre-filter (off by default)
+    # with the given flat cp margin. Both require NNUE_WEIGHTS; the
+    # lazy one further requires NNUE_PROBE_WEIGHTS to do anything.
+    if [ -n "${NNUE_LAZY_MARGIN:-}" ]; then
+        NNUE_FLAGS="$NNUE_FLAGS --nnuelazymargin $NNUE_LAZY_MARGIN"
+    fi
+    if [ -n "${NNUE_SUPERLAZY_MARGIN:-}" ]; then
+        NNUE_FLAGS="$NNUE_FLAGS --nnuesuperlazymargin $NNUE_SUPERLAZY_MARGIN"
+    fi
     cat > "$NNUE_WRAPPER" <<WRAPPER
 #!/usr/bin/env bash
-exec "$CANDIDATE" --nnueweights "$NNUE_WEIGHTS" "\$@"
+exec "$CANDIDATE" $NNUE_FLAGS "\$@"
 WRAPPER
     chmod +x "$NNUE_WRAPPER"
     CANDIDATE="$NNUE_WRAPPER"
 fi
 
+# NNUE_CANDIDATE_NAME: PGN White/Black identity recorded for the
+# candidate engine, forwarded to match_play.py's --candidate-name
+# (2026-10-16, see that flag's own comment). Defaults to a name that
+# flags "don't trust this as training data" whenever NNUE is in play,
+# since self_play_games.pgn is reused later to train the NNUE net
+# itself and a differently-calibrated evaluator's scores shouldn't be
+# mixed in as labels without this being mechanically filterable.
+# Override explicitly if a given NNUE config's scores are actually
+# meant to be trusted.
+NAME_ARGS=""
+if [ -n "${NNUE_WEIGHTS:-}" ]; then
+    NNUE_CANDIDATE_NAME="${NNUE_CANDIDATE_NAME:-candidate_nnue}"
+    NAME_ARGS="--candidate-name $NNUE_CANDIDATE_NAME"
+fi
+
 python3 ./match_play.py \
         ${HEAD} \
         ${CANDIDATE} \
@@ -134,6 +195,7 @@ python3 ./match_play.py \
         --post \
         ${TC_ARGS} \
         ${SPRT_ARGS} \
+        ${NAME_ARGS} \
         --scratch /usr/local/tmp/typhoon_match_${LABEL} \
         --log ${LABEL}.log \
         "$@"