NNUE: re-fit raw-to-score calibration, NNU4 weight format, super-lazy default-on, lazy probe measured net-negative

Fits an affine (a + b*raw) correction into every exported .nnue file
(export_nnue_weights.py's CalibrateScoreMapping) so the engine's
consumed centipawn output tracks Stockfish's real score scale rather
than the net's nominal raw WDL-sigmoid output -- training variance
alone shifts this mapping across retrains even with an identical
recipe, so it's re-fit and baked in per export rather than assumed
constant. Weight format bumped to "NNU4" to carry the two new floats;
older NNU1/2/3 files are no longer readable.

Super-lazy (material-only) pre-filter promoted to default-on
(g_fUseNnueSuperLazy = TRUE) with a recalibrated per-material-bucket
margin table (measured against the recalibrated net, 16-shard
cross_sectional_200k.ep_ sample) -- real, measured nps win across all
three curated suites. The separate lazy-probe gate (accumulator-based
early exit) is measured net-negative on real search traffic at this
net's probe-vs-forward latency ratio and stays off by default
(g_fUseNnueLazyProbe = FALSE); code kept live since margin tuning
could still flip this, per docs/NNUE.md.

Also: direct per-call latency instrumentation for NnueProbeScore/
NnueForward (root.c, EVAL_TIME build) to make the lazy-probe
breakeven math measurable directly instead of inferred; QSearch
futility margins given real-data-derived NNUE_FULL/NNUE_LAZY
starting points.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

1 files changed, 55 insertions, 13 deletions

diff --git a/src/nnueeval.c b/src/nnueeval.c
index 7caadef..583c2db 100644
--- a/src/nnueeval.c
+++ b/src/nnueeval.c
@@ -45,6 +45,24 @@ CHAR g_rgfNnueFc3Weight[NNUE_L3];
 float g_rgfNnueFc3Bias;
 float g_rgfNnueActScale[3] = { 1.0f, 1.0f, 1.0f };
 float g_rgfNnueWeightScale[3] = { 1.0f, 1.0f, 1.0f };
+// Affine correction so the engine's consumed centipawn output tracks
+// Stockfish's own real score, not the net's nominal raw WDL-sigmoid
+// scale -- see export_nnue_weights.py's CalibrateScoreMapping and the
+// 2026-10-20 "keeping this stable across retrains" conversation. A
+// fixed K/lambda/loss-exponent training recipe is NOT enough on its
+// own to guarantee the search's consumed cp scale stays constant
+// across future retrains -- training variance (data, convergence)
+// still shifts the empirical raw-to-score mapping even with an
+// identical recipe. Re-fitting and re-baking this correction into
+// every export, rather than trusting a fixed multiplier, is what
+// actually decouples engine-side search tuning from needing the raw
+// net's internal scale to stay perfectly constant forever. Defaults
+// (0,1) are the identity transform -- the pre-2026-10-20 behavior --
+// so an old build loading a newer file (or vice versa, before the
+// dimension check below even runs) degrades to the old bare-*100
+// convention rather than silently miscalibrating.
+float g_fNnueCalibA = 0.0f;
+float g_fNnueCalibB = 1.0f;
 FLAG g_fNnueWeightsLoaded = FALSE;
 FLAG g_fUseNnueEval = FALSE;
 
@@ -86,12 +104,13 @@ Return value:
     }
 
     if ((1 != fread(szMagic, 4, 1, fp)) ||
-        (0 != memcmp(szMagic, "NNU3", 4)))
+        (0 != memcmp(szMagic, "NNU4", 4)))
     {
         Trace("NnueLoadWeights: %s is not a valid NNUE weights file "
-              "(bad magic -- expected \"NNU3\"; an older \"NNU1\"/"
-              "\"NNU2\" file predates the FC head's int8 quantization "
-              "and is no longer readable)\n", szFilename);
+              "(bad magic -- expected \"NNU4\"; an older \"NNU1\"/"
+              "\"NNU2\"/\"NNU3\" file predates the raw-to-score "
+              "calibration fields and is no longer readable)\n",
+              szFilename);
         fclose(fp);
         return FALSE;
     }
@@ -130,6 +149,14 @@ Return value:
         fclose(fp);
         return FALSE;
     }
+    if ((1 != fread(&g_fNnueCalibA, sizeof(g_fNnueCalibA), 1, fp)) ||
+        (1 != fread(&g_fNnueCalibB, sizeof(g_fNnueCalibB), 1, fp)))
+    {
+        Trace("NnueLoadWeights: %s truncated (score calibration)\n",
+              szFilename);
+        fclose(fp);
+        return FALSE;
+    }
 
 #define READ_OR_FAIL(ptr, elsize, count)                               \
     do {                                                                \
@@ -265,7 +292,7 @@ Return value:
                g_rgfNnueFc3Bias;
     }
 
-    return (SCORE)(fOut * 100.0f);
+    return (SCORE)((g_fNnueCalibA + g_fNnueCalibB * fOut) * 100.0f);
 }
 
 
@@ -420,14 +447,23 @@ FLAG g_fUseNnueLazyProbe = FALSE;
 // docs/NNUE.md "paradox: safety doesn't win games" discussion.
 // g_iNnueLazyMarginOverride < 0 means "use eval.c's calibrated
 // per-material-bucket table as before"; >= 0 means "use this flat cp
-// value for every bucket instead". g_fUseNnueSuperLazy/
-// g_iNnueSuperLazyMargin gate a brand new, even cheaper pre-filter
-// (pure material balance, no net at all) ahead of the probe -- this
-// was previously measurement-only (see eval.c's CALIBRATE_MARGIN_SAFETY
-// block) and is now wired into the real Eval() path for the first time.
+// value for every bucket instead". g_fUseNnueSuperLazy gates an even
+// cheaper pre-filter (pure material balance, no net at all) ahead of
+// the probe. g_iNnueSuperLazyMargin follows the identical convention
+// as of 2026-10-20 (was a flat-only override before that date, with no
+// per-bucket fit at all): < 0 means "use eval.c's calibrated
+// iNnueSuperLazyMarginByMatBucket table", >= 0 means "use this flat cp
+// value for every bucket instead".
 INT g_iNnueLazyMarginOverride = -1;
-FLAG g_fUseNnueSuperLazy = FALSE;
-SCORE g_iNnueSuperLazyMargin = 0;
+// 2026-10-20: super-lazy defaults ON -- measured a clean win (nps up,
+// no consistent solve-rate cost) across all three curated ECM suites
+// at both sd 10 and sn 5000000, using the calibrated per-bucket
+// iNnueSuperLazyMarginByMatBucket table (eval.c), unlike the lazy
+// probe gate just above, which stays off by default (measured net
+// negative -- see its own table's comment in eval.c for the full
+// story). Still only takes effect when g_fUseNnueEval is also on.
+FLAG g_fUseNnueSuperLazy = TRUE;
+SCORE g_iNnueSuperLazyMargin = -1;
 
 
 FLAG
@@ -554,5 +590,11 @@ Return value:
         fSum += v * g_rgfNnueProbeWeight[NNUE_ACCUMULATOR_SIZE + i];
     }
 
-    return (SCORE)(fSum * 100.0f);
+    // fSum approximates NnueForward's pre-calibration raw fOut (it was
+    // fit by ridge regression against exactly that target -- see
+    // docs/NNUE.md's "Lazy-evaluation tiering plan") -- apply the same
+    // (g_fNnueCalibA, g_fNnueCalibB) correction so this stays directly
+    // comparable to NnueForward's actual output, per this function's
+    // own doc comment above.
+    return (SCORE)((g_fNnueCalibA + g_fNnueCalibB * fSum) * 100.0f);
 }