Phase A NNUE: feature-indexing spec, annotation/training pipeline, dump nnuefeatures

Builds out the NNUE Phase A prototype per docs/NNUE.md:

- src/nnuefeatures.c: standalone HalfKAv2_hm feature-index reference
  implementation (new `dump nnuefeatures` command), sourced from
  Stockfish's half_ka_v2_hm.{h,cpp} (sf_16 tag), including the 0x88->64
  square conversion typhoon's own board representation needs.
- annotate_pgns/: sharded TWIC/self-play annotation harness
  (annotate.py), position extraction, quiet-position filtering (flat
  and game-shaped, plus an upstream --prefilter-quiet option),
  train/holdout game-level stratified split, WDL-sigmoid K fitting,
  the Python HalfKAv2_hm reference encoder (nnue_feature_spec.py) and
  its bit-for-bit cross-checker against the C side
  (cross_check_features.py), and a first PyTorch training pipeline
  (nnue_model.py, nnue_train.py, build_training_tensors.py) -- a
  256x2-32-32-1-ish net trained locally on CPU against ~1.85M quiet
  TWIC positions as a pipeline/framework validation pass.
- Finish an in-progress InitMagic/InitLMRTable/InitEval ->
  InitializeBitboardMagic/InitializeLMRTable/InitializeEval rename:
  chess.h's prototypes hadn't been updated to match the already-renamed
  definitions/call sites, leaving 4 implicit-function-declaration
  warnings.
- tests/arasan.epd: new curated EPD suite.
- docs/: add the project's design docs (previously untracked).

Co-Authored-By: Claude Sonnet 5 <[email protected]>

1 files changed, 213 insertions, 0 deletions

diff --git a/src/docs/NULLMOVE_SKIP_CALIBRATION.md b/src/docs/NULLMOVE_SKIP_CALIBRATION.md
new file mode 100644
index 0000000..50a050e
--- /dev/null
+++ b/src/docs/NULLMOVE_SKIP_CALIBRATION.md
@@ -0,0 +1,213 @@
+# Null-move-skip calibration plan (2026-10-12)
+
+## Motivation
+
+Two FH%-prediction tables now exist in `data.c`:
+
+- `g_uJointFHThousandths` ("alpha table") -- predicts a move's FH% given
+  [depth][hang][gap-to-alpha][late][improve]. Used today for LMP/EFP/LMR
+  decisions on individual leftover moves.
+- `g_uBetaFHThousandths` ("beta table") -- predicts a node's FH% given
+  [depth][gap-to-beta][ownImprove][enemyImprove], for nodes where
+  `iEval > iBeta`. Used today for the static-NM shortcut
+  (`search.c`'s `fTryBetaPruningHere` block, >95% -> return `iBeta`
+  without searching).
+
+Question: is there a *low*-confidence corner in either table reliable
+enough to justify **skipping the real null-move search attempt**
+entirely at a node (not pruning the node itself -- just not trying the
+null probe, falling through to the normal move loop: hash move,
+killers, etc.)? Unlike the static-NM shortcut, this is a pure
+efficiency play with no score-injection risk: if we're wrong, the real
+move loop still runs and finds the correct score. We just forgo the
+cheap shortcut on the rare occasion it would have worked.
+
+## Why the alpha table, not the beta table, for this
+
+At non-PV nodes, `iAlpha == iBeta - 1`, so `iEval <= iAlpha` and
+`iEval >= iBeta` together cover the entire range -- no gap. Checked
+both tables' actual baked data (`data.c`) for a usable low-FH corner:
+
+- **Alpha table**, sliced at hang=0 ("nothing hung"), late=0 ("even the
+  single best-looking move"): deep remaining depth (shipped buckets
+  8-10) x large gap-to-alpha (buckets 8-10) x improve bucket 0 (own
+  eval declining) gives real, non-fallback predicted FH% as low as
+  0.001-0.1% -- two-plus orders of magnitude below the ~0.4-0.9%
+  baseline leftover-move rate. A strong, well-observed signal.
+- **Beta table**: its 2026-10-11 inversion (storing gap-to-100% instead
+  of FH% directly, to keep resolution near 100% for the static-NM
+  shortcut) saturates every cell predicting below 34.466% FH to the
+  same stored value. No resolution exists below that floor in the
+  current bake -- there is no usable low corner to find here. Would
+  need a dedicated unsaturated rebake to pursue; out of scope for now.
+
+So: implement the skip using the alpha table only, for the
+`iEval <= iAlpha` side.
+
+## The EV algebra
+
+Trying null-move at a node has expected value (in nodes saved):
+
+```
+EV(try) = p * realLoopCost - nullProbeCost
+```
+
+where `p` = probability null-move fails high, `nullProbeCost` = the
+reduced-depth probe's node cost, `realLoopCost` = what the real move
+loop costs to reach its own cutoff (the cost skipped entirely when the
+probe succeeds). Skip trying null-move iff
+`p < nullProbeCost / realLoopCost`.
+
+- `nullProbeCost` is trivial to measure: a node-counter delta around
+  the probe's recursive `Search()` call (same as `TEST_NULL`'s existing
+  `uNullCalibProbeNodeCost`).
+- `realLoopCost` is conceptually simple (it's the cost the node always
+  pays via the real move loop when null isn't tried) but the
+  *counterfactual* isolation -- measuring it specifically for the
+  "null was never attempted" branch, without contamination from
+  production's own null-move logic firing afterward -- needs a
+  deliberate harness. Not yet built; see "Future work" below.
+
+### R-factor / branching-factor estimate
+
+Using `p* ~= b^(-R)` with `b = 2.5` (assumed average branching) and
+`R = 3 + floor(depth/6)` (actual `SelectNullmoveRFactor` formula):
+
+| remaining depth (plies) | R | p* = 2.5^-R |
+|---|---|---|
+| 0-5   | 3 | 6.4%  |
+| 6-11  | 4 | 2.6%  |
+| 12-17 | 5 | 1.0%  |
+| 18-23 | 6 | 0.41% |
+| 24-29 | 7 | 0.16% |
+
+The threshold **tightens** (gets stricter) with depth, not looser --
+`realLoopCost` grows with full remaining depth while `nullProbeCost`
+only shrinks mildly via R's slow growth. The alpha-table corner found
+above (shipped depth 8-10, raw ~8-13 plies, landing in the R=4-5 rows,
+`p* ~= 1-2.6%`) showed predicted FH% of 0.001-0.15% there -- clearing
+the math-derived bar by one to two orders of magnitude. Good sign the
+corner is real, not just noise, even under a crude model.
+
+## Nested-null-move handling
+
+Raised explicitly: does a null-move search recursing into *another*
+null-move search deeper in its own subtree break any of this?
+
+- **Inside the probe's own subtree**: not a problem. `fAvoidNullmove`
+  is a single shared flag, not stacked per-ply -- it only blocks two
+  *consecutive* nulls, toggled tightly around the probe call and reset
+  immediately after. Nested null-moves several plies down inside the
+  probe are real and representative, exactly as they'd behave in
+  production. This does mean `nullProbeCost` is a high-variance
+  per-sample quantity (one lucky nested cutoff can collapse most of a
+  probe's subtree) -- argues for a large sample floor per bucket
+  (reuse the ~500-try convention `dual_table_run` already uses)
+  before trusting any bucket average, not a correctness fix.
+- **Inside the "skip branch" realLoopCost measurement**: this is where
+  it matters. If/when a skip-rule is compiled into `Search()`, nested
+  nodes inside that measured real-loop subtree will also apply the
+  same rule recursively -- calibration data collected this way depends
+  on whatever rule is *already* active at calibration time. Circular.
+  Resolution (same pattern already used for every other baked table
+  here): **Pass 1** calibrates with no skip-rule compiled in anywhere
+  (matches `TEST_NULL`'s existing "structural eligibility only, don't
+  consult the thing being evaluated" design). Ship from that. **Pass
+  2**: re-measure/re-splice once live, since nested applications then
+  change descendant costs too (likely compounding favorably, since a
+  collapsing-eval signal is autocorrelated down a few plies -- a
+  hypothesis, not yet verified).
+
+## What's implemented now (2026-10-12)
+
+A single flat, compile-time sweep knob -- not yet the full
+depth-bucketed table:
+
+- `search.c:93`: `#define NULLMOVE_SKIP_FH_CEILING_THOUSANDTHS (0)`
+  -- `0` disables (default/shipped state).
+- `search.c`, inside `fTryBetaPruningHere`, opposite-direction check
+  from the existing static-NM block: when `iEval <= iAlpha`, look up
+  `_JointTablePredictedFHThousandths(uRawDepthB, 0, GapCpBucket(iAlpha
+  - iEval), 2, uRawImproveB)` (hang=0, late=2 -- the table's own domain
+  floor, since collection required `uLegalMoves > 1`, so there's no
+  literal "move 1" bucket; 2 is the nearest available proxy for "even
+  the single best-looking move"). `uRawDepthB`/`uRawImproveB` are
+  reused from the already-computed per-node locals (lines ~551, ~726),
+  not recomputed. If predicted FH% is below the ceiling, skip calling
+  `WeShouldTryNullmovePruning` for this node entirely, falling through
+  to the real move loop.
+- `chess.h`: new counter `u64NullMoveSkippedLowFH` (tree counters
+  struct only, matching `u64QuickNullMoveSuccess`'s precedent -- not
+  MP-merged, diagnostic-only).
+
+## Empirical sweep results
+
+### bench (6 fixed positions, fast but small)
+
+Baseline (ceiling=0): 2,714,949 nodes. Shallow dip at ceiling 10-30
+(~0.4-0.5% fewer nodes), rising above 100, blowing up by 3000-10000.
+
+### ecm_ringers (11 positions, `sd 10`) -- too small, non-monotonic
+
+| ceiling | total nodes | solved/11 |
+|---|---|---|
+| 0    | 4,293,355 | 3 |
+| 10   | 4,435,817 (+3.3%) | 3 |
+| 30   | 4,314,830 (+0.5%) | 3 |
+| 100  | 4,250,553 (-1.0%) | 3 |
+| 300  | 4,089,390 (-4.7%) | 3 |
+| 1000 | 4,700,197 (+9.5%) | **2** (lost ECM.783) |
+| 3000 | 3,904,085 (-9.1%) | 3 |
+
+Matches this project's own documented caveat: ringers is too small and
+chaotically order-sensitive to trust a single run's shape.
+
+### ecm_confident_quick + ecm_hard_quick (90+90 positions, `sd 10`)
+
+Ran as 8 parallel background processes (4 ceilings x 2 suites, 24-core
+box, ~idle at the time) using 4 separately-built binaries (ceiling is
+a compile-time `#define`, can't be swept at runtime without rebuilding).
+
+| ceiling | combined nodes (180 pos) | delta | combined solved (baseline 79) |
+|---|---|---|---|
+| 0    | 48,588,231 | --     | 79 |
+| 300  | 47,468,723 | -2.3%  | 76 (-3) |
+| 1000 | 47,783,919 | -1.7%  | 77 (-2) |
+| 3000 | 45,907,126 | **-5.5%** | **79 (net 0)** |
+
+Per-suite: `ecm_confident_quick` solved count degrades monotonically
+(62/60/61/59) as ceiling rises; `ecm_hard_quick` is flat/down through
+300-1000 (17/16/16) then jumps to **20** at 3000. The jump is what
+makes 300/1000 look worse than their neighbors on both sides --
+non-monotonic, not a stable curve.
+
+**Conclusion so far**: no clean single minimum for a flat ceiling --
+consistent with (not just predicted by) the depth-dependence argument
+above. Ceiling=3000 is the one genuinely encouraging point (real node
+savings, net-zero solve cost across 180 positions) but a flat knob
+can't properly track a threshold that should move with depth; finding
+a better flat number isn't the right next move.
+
+## Future work
+
+1. **Don't keep sweeping the flat knob.** Build the depth-bucketed
+   version instead: extend `TEST_NULL` to measure `nullProbeCost` and
+   `realLoopCost` per (depth, gap, improve) bucket (Pass 1, no live
+   skip-rule anywhere, per the nested-null discussion above), bake a
+   per-bucket `p*` the same way `dual_table_run`/`splice_dual.py`
+   already bakes the other tables, re-measure/re-splice once shipped
+   (Pass 2).
+2. If that proves too complex or doesn't pan out, fall back to a
+   coarser depth-only piecewise ceiling (a handful of depth bands,
+   each with its own flat threshold derived from this sweep's
+   per-depth behavior) rather than one global number.
+3. Before committing anything: full `ecm.ep_` run at the chosen
+   setting(s), `precommit_check.sh`, and the usual `head_reference`
+   comparison protocol.
+4. The beta-side (`iEval > iBeta`) mirror of this idea is still
+   blocked on a resolution-preserving rebake of `g_uBetaFHThousandths`
+   below its current 34.466% saturation floor -- not worth pursuing
+   until the alpha-side version proves out.
+
+Sweep artifacts (binaries, logs, stdout) are under
+`/tmp/typhoon/nullskip_sweep/`, scratch -- not kept.