path: root/src/docs/REGRESSION.md

# Learned FH% regression: design and collection plan

Context: `g_uJointFHThousandths`/`g_uBetaFHThousandths` (`data.c`) are 5D/4D
histogram tables predicting "will this move/node fail high," used to gate
LMR/LMP admission (alpha side) and static null-move cutoff + node-type
refinement (beta side). This doc designs their replacement: two small
logistic regressions, one per side of the window, fit offline and baked
into the engine as quantized integer weight vectors.

## Why move off the table

1. **Dimension growth is combinatorial, sample growth isn't.** The shipped
   alpha table is already `12x11x11x17x11` ~= 268K cells; the NNUE
   collection run (`../dual_table_run_nnue/README.md`) found 121,192 of
   271,524 cells still below the 500-sample floor even after a multi-day,
   16-shard run. Adding even one more binary axis roughly doubles the cell
   count and thins every existing cell further. A regression's cost per
   new feature is one more weight, not a multiplicative storage/sample
   penalty.
2. **New discriminating signals don't fit cleanly as new axes.** The
   `docs/DEEPER_PRUNING.md` HASHCAP_COLLECTION side-study found two real
   effects -- node-type (CUT vs ALL/PV) carries a measured ~9x effect on
   leftover-move FH%, and whether the hash move was a capture carries a
   separate ~2x effect, and the two *interact* (a CUT-node-survived-this-far
   signal is itself evidence the CUT guess was wrong). Wiring either into
   the existing table means a full axis-dimension expansion; a regression
   just adds features and an interaction term.
3. **Measured discrimination is real and worth capturing properly.**
   Pulled directly from `../dual_table_run/pickles/baked_now.pkl`
   (`compare_improve_axis.py` plus one-off marginal scripts written this
   session):

   Alpha side (`iEval <= iAlpha`), global marginal rate ratio per axis:

   | axis | ratio (max/min FH%) |
   |---|---|
   | gap | 1647x |
   | improve | 128x |
   | dumb (hang estimate) | 96x |
   | late (move lateness) | 40x |
   | depth | 24x |

   Beta side (`iEval >= iBeta`), using **miss-rate** (`100% - FH%`) ratio,
   since raw FH% sits near a 95-99% ceiling where a percentage-ratio
   understates real discrimination:

   | axis | ratio (miss-rate max/min) |
   |---|---|
   | depth | 155x |
   | gap | 91x |
   | improve | 12x |
   | own_improve | 4.8x |

   Note depth is the *strongest* axis on the beta side and a middling one
   on the alpha side -- a real, measured asymmetry, not an assumption. This
   is the direct evidence behind building **two independent regressions**
   rather than one shared model: even a feature both sides have in common
   carries a different weight in each regime.

## Model family: logistic regression, not trees or a net

Considered and rejected for the inner loop:

- **Small gradient-boosted tree ensemble** (e.g. 50 trees, depth 4): finds
  interactions automatically, but each split is a data-dependent branch the
  predictor can't learn a pattern for -- ~15-20 cycles mispredict penalty
  *per split*, 50 trees x 4 levels is in the ~1us range. Disqualifying at
  the throughput this needs to run at (every candidate move, every node).
- **Small NNUE-style net**: fast once SIMD'd, but heavier infra for no
  clear win over logistic regression at this feature count, and NNUE eval
  itself is already the dominant per-node cost in this engine (~400-500K
  nps vs HCE's ~1.3M+) -- not the place to add more float-heavy inference.
- **Logistic regression**: one dot product (~16-20 terms) + one sigmoid.
  Fully branch-free, pipelines well, same cost class as the existing
  Wilson-bound table lookup + `MIN`. This is the one we're building.

**Quantization**: fit in float offline (scikit-learn `LogisticRegression`,
seconds on commodity CPU for millions of rows) exactly like
`tune_eval_dna.py`'s DNA constants are tuned in float and only baked to
engine-native representation afterward. Quantize the learned float weights
to fixed-point integers (choose the scale *after* seeing real weight
magnitudes -- don't squeeze to int16 preemptively, there's no storage
pressure here forcing it the way there was with table axes). Inference:
`score = (sum(w_i * f_i)) >> shift`, int64 accumulator, then one lookup
into a small precomputed sigmoid table keyed on a coarse bucket of `score`
-- no floating point anywhere in the hot path, no runtime `exp()`/`log()`.

## Nonlinear raw quantities need an explicit feature transform

Logistic regression is linear in log-odds *of the features you feed it*,
not of whatever raw quantity you happened to measure. If the true
relationship between depth and log-odds(FH) is logarithmic (which is why
Stockfish's own reduction formula is `log(depth)*log(moveIndex)` to begin
with), feeding raw depth in forces the model to approximate a curve with a
straight line -- transform first (`log(depth)`, `log(moveIndex)`), don't
rely on the model family to discover the right shape.

Checked directly against the alpha table's real data (`baked_now.pkl`,
marginalized per axis, logit = `log(p/(1-p))`):

- **gap**: logit deltas per bucket are `-1.88, -2.30, -0.81, -0.48, -0.50,
  -0.04, -0.07, -0.23, -0.28, -0.85` -- large early drops that flatten
  hard, classic compressive-transform shape. `log(gap)` (or
  `log(gap + const)`) is the right starting point for a continuous
  feature; note the *existing* axis's bucket boundaries already encode
  some of this nonlinearity implicitly via non-uniform bucket widths, so
  this isn't a new discovery so much as confirmation to carry forward
  explicitly.
- **depth**: trends down through bucket ~12 but not cleanly log-shaped
  (deltas bounce: `-0.12, -0.27, -0.43, -0.12, -0.06, -0.04, -0.31, ...`),
  then goes noisy/non-monotonic past depth 13 (`+0.79`, `-0.90` swings) --
  consistent with the NNUE README's own finding that per-cell sample
  counts collapse toward zero past depth 13. Use `log(depth)` as the
  default transform (matches every other engine's own formula and the
  early-bucket trend direction), but don't trust the depth>13 region's
  shape from this data -- validate against the regression's own residuals
  once real feature-level (not axis-marginal) data exists, and treat
  depth>13 behavior as an extrapolation, same caution the existing table
  already applies there.
- **move ordinal**: the existing `late` axis (alpha side) is the closest
  historical proxy and gives a textbook-clean compressive shape --
  logit deltas shrink smoothly from `-0.465` down to near-zero by bucket
  ~27 (`-0.013`), then drift slightly positive in the extreme tail
  (bucket 32, a wide catch-all bucket, likely a different population
  mix rather than a reversal of the real trend). This is about as strong
  a confirmation of `log(ordinal)` as old data can give. Combined with
  the fact that Crafty and Stockfish both use a log-of-move-index term
  regardless of which side of the window they're on, `log(ordinal)` is
  adopted for **both** the alpha and beta side's move-ordinal feature,
  not left as a side-specific guess.
- **improve** (alpha side, axis 4): trends up with bucket index but
  without gap's clean compressive shape -- deltas are noisy
  (`+0.85, +0.10, +0.46, +0.63, +1.51, +0.74, -0.05, +0.14, +0.51,
  -0.20`) though the overall direction holds across 11 buckets backed by
  billions of tries each. Likely doesn't need an *additional* transform
  beyond whatever the existing bucket boundaries already encode -- try
  the raw underlying continuous quantity untransformed first, confirm
  against the smoke test's own fresh data.
- **improve / own_improve (beta side, axes 2-3)**: noisier still.
  `own_improve` in particular shows a sharp level-shift -- bucket 0 sits
  at 98.48%, every other bucket (1-10) sits in a tighter 92-96% band with
  alternating-sign deltas -- which smells more like "own_improve == 0"
  mattering as a distinct state than like a smooth magnitude effect.
  Worth testing an explicit `(own_improve == 0)` indicator plus a
  separate continuous magnitude feature against a single continuous
  encoding once real data exists, rather than assuming either shape now.
- **dumb (hang estimate, alpha side, axis 1)**: non-monotonic, no smooth
  shape at all -- a huge spike at bucket 10 riding on a tiny 8.8M-try
  sample (vs. neighboring buckets' billions), bucket 1 also spiking well
  above its neighbors. Reads as a genuinely **categorical** signal
  (discrete hang-severity tiers), not a continuous quantity -- which
  confirms the decision already made above (the new, redesigned "move
  value" feature is a continuous signed SEE/hang-estimate, not a
  transform of this old discrete axis) rather than suggesting a log
  transform was missing here.

General rule for this project: any raw feature suspected non-linear in
log-odds gets checked against real axis-marginal data (same style as
above) before locking its transform, not assumed from other engines'
formulas alone. Where no historical axis exists to check against (every
brand-new feature: `is_whole_node`, the running counter, FH%-as-feature,
both cross terms, and the *new* continuous move-value definition), the
smoke test's own output is the first real opportunity to check -- see
"Validating against the existing table" below for how that interacts
with trusting the smoke test's small sample size.

## One model, not two: NNUE-only collection (decided 2026-10-24)

Originally planned as two independent collection runs (HCE and NNUE),
matching `dual_table_run`/`dual_table_run_nnue`'s own split. Revised:
**collect once, against NNUE only, and let HCE use the same model.**
Rationale: NNUE is the strategic direction going forward, and running
two full multi-day collections for a model that's meant to retire
anyway isn't worth the cost. This is a reasonable call but carries a
named, specific risk worth remembering rather than assuming away: the
earlier HCE-vs-NNUE table comparison (see `dual_table_run_nnue/README.md`)
found the **alpha** table's predictions tracked closely between the two
evals (-0.11pp mean delta, flat across depth) -- clean support for
reuse there -- but the **beta** table showed a real, systematic
divergence (NNUE -0.6pp overall, growing to -1.1 to -1.9pp from depth
bucket 2 on), plus a separate, never-investigated 3-9pp gap in one
narrow alpha-side slice (depth=1, dumb=0, gap=0, early leftover moves).
Before trusting one NNUE-trained model under an HCE build, the
"rigorous spot-check against high-confidence table cells" step below
should be run against **both** the HCE and NNUE tables' cells, not
just whichever eval the collection happened to run under -- if the
beta-side predictions land close to HCE's own historical cells too,
that confirms the simplification; if not, that's exactly the thing
worth knowing before shipping one model for both evals.

Practically this requires no change to the collection design below --
`FH_COLLECT_RECORD`/the hook API never reference `g_fUseNnueEval` at
all, so the only real implications are: the collection run itself uses
`typhoon_nnue` exclusively, and the eventual runtime dispatch in
`searchsup.c` collapses to "always use the one model," no
`g_fUseNnueEval` branch needed.

## Scope: MWS (null-window) nodes only, for now

Both regressions are trained and used only where `iAlpha + 1 == iBeta`
(a null-window/MWS child). At a true PV node (`alpha < eval < beta` is a
real, non-degenerate state), "fail high" isn't a well-defined single
question the way it is here -- collecting and modeling PV nodes is
explicitly **deferred as future work**, not attempted in this pass. The
alpha/beta branch below is exhaustive specifically because MWS makes
`iEval` always either `<= iAlpha` or `>= iBeta`, never strictly between:

```c
if (iEval <= iAlpha) {
    uPredictedFHThousandths = RegressionA(...);
} else if (iEval >= iBeta) {
    uPredictedFHThousandths = RegressionB(...);
}
```

## Shared design rules (both regressions)

1. **`is_whole_node` sentinel bit, plus a dedicated `no-move` tier
   category** -- not a raw-zero convention. The NNUE collection run's own
   bucket-0/1/2 collision bug (raw `late` values 0, 1, and 2 all landing in
   shipped bucket 0 because the placeholder and a real low-ordinal
   population's natural value both mapped to zero) is exactly the failure
   mode an implicit zero-as-"absent" convention invites. An explicit
   indicator bit plus a genuinely separate categorical value avoids
   repeating it.
2. **`moves_known` bit + mean-imputation for total legal move count.** In
   production, moves aren't generated until after the null-move and hash-
   move tries, so the real count is genuinely unknown for the whole-node,
   null-move, and hash-move predictions -- not zero, unknown. Pair the
   feature with a binary indicator and impute the dataset mean when
   unknown; the collection *build* can eagerly generate moves for every
   logged event regardless of production's lazy order (collection already
   pays extra cost for ground-truth correctness -- see LMP-disabling
   precedent below), so training data always has the real count even where
   production inference won't.
3. **Node-type guess fed in must be the raw, parent-derived guess only** --
   never the *refined* guess this same regression produces. Feeding a
   model's own (even indirectly derived) output back in as an input is
   leakage: it'll look great in training and be meaningless at inference
   time, where the refined guess doesn't exist yet.
4. **`in_check` is dropped entirely.** It's a hard upstream gate -- no
   pruning, reduction, or null-move is ever attempted while in check -- so
   the training population never contains an `in_check=true` row by
   construction. A feature with zero variance in its own training
   population carries zero information; it's a precondition for the model
   running at all, not something for the model to weigh.
5. **Evidence is the move's own calibrated FH% (Wilson-bound estimate),
   not a raw history/continuation counter.** A raw counter has no natural
   zero point or scale comparable to a percentage; the move's own FH%
   estimate is already computed, already calibrated, and lives on the same
   0-100 scale as every other probability-flavored feature.
6. **Generalized "good moves tried without a cutoff" running counter.**
   Replaces the narrower "was the hash move a capture" binary flag with a
   loop-state feature: the count (and fraction of moves-so-far) of
   winning/even-capture moves already tried at this node that failed to
   produce the relevant outcome. Mutates as the move loop progresses (zero
   before any move, incrementing on each surviving good move) -- directly
   operationalizes the "a CUT guess that survives this long is probably
   wrong" finding rather than encoding one static fact about the hash move
   alone.
7. **Explicit hand-built cross terms for measured interactions.** Plain
   logistic regression won't discover an interaction on its own -- only a
   tree ensemble does that automatically, and we ruled trees out on speed.
   Where there's real measured evidence of an interaction (node-type x
   running-counter, per the ~9x CUT effect), add the product as its own
   feature explicitly.
8. **Nonlinear raw quantities get transformed before entering the model**
   -- see above.

## Feature list -- alpha side (`iEval <= iAlpha`)

Predicts: did this move's (or the whole node's eventual best) resulting
score end up `>= iBeta`.

**Node-level** (computed once per node; shared by the whole-node row and
every move-level row at this node):

| # | feature | type / transform |
|---|---|---|
| 1 | `gap` = iAlpha - iEval | continuous, `log(gap)` |
| 2 | remaining depth | continuous, `log(depth)` |
| 3 | node-type guess (CUT/ALL/PV), **parent-derived only** | categorical |
| 4 | improving | binary/continuous + trend (1st derivative) |
| 5 | enemy improving | binary/continuous + trend |
| 6 | hang/dumb estimate (position-wide) | continuous |
| 7 | total legal move count | continuous, paired with `moves_known` bit + mean-imputation |
| 8 | phase/material bucket | categorical, coarse |
| 9 | `is_whole_node` | binary sentinel |

**Move-level / loop-state** (one row per move actually tried: `null`,
hash, good-capture, killer, countermove-match, leftover-quiet,
losing-capture -- plus the `no-move` sentinel row paired with
`is_whole_node=1`):

| # | feature | type / transform |
|---|---|---|
| 10 | move tier tag (incl. `null`, `no-move`) | categorical |
| 11 | move value, signed (SEE/IID/hang-estimate; +300/0/-900 style) | continuous |
| 12 | move ordinal, raw | continuous, `log(ordinal)` |
| 13 | move ordinal, normalized (ordinal / total legal moves) | continuous |
| 14 | move's own FH% (Wilson-bound estimate) | continuous, 0-100 scale |
| 15 | running counter: good/even captures tried without a cutoff, raw + fraction | continuous |
| 16 | node-type-CUT x running-counter (cross term) | continuous |

## Feature list -- beta side (`iEval >= iBeta`)

Predicts: did this move's (or the null-move probe's, or the whole node's
eventual best) resulting score end up `>= iBeta`. Same label definition as
alpha side.

**Node-level:**

| # | feature | type / transform |
|---|---|---|
| 1 | `gap` = iEval - iBeta | continuous, transform TBD -- check logit-linearity against this side's own data before assuming `log(gap)` carries over unchanged from the alpha side |
| 2 | remaining depth | continuous, `log(depth)` -- strongest single axis measured on this side (155x miss-rate ratio) |
| 3 | node-type guess, **parent-derived only** | categorical |
| 4 | improving / own-improving | binary/continuous + trend, both already real axes in the shipped table |
| 5 | total legal move count | continuous, `moves_known` bit + mean-imputation |
| 6 | phase/material bucket | categorical |
| 7 | `is_whole_node` | binary sentinel |

**Move-level / loop-state** (`null`, hash, good-capture, killer,
countermove-match, leftover-quiet, losing-capture, `no-move`):

| # | feature | type / transform |
|---|---|---|
| 8 | move tier tag | categorical |
| 9 | move value, signed | continuous -- **flagged below, not yet trusted** |
| 10 | move ordinal, raw + normalized | continuous, `log(ordinal)` for raw |
| 11 | move's own FH% (Wilson-bound estimate) | continuous, 0-100 |
| 12 | running counter: good/even captures tried without preserving the cutoff, raw + fraction | continuous |
| 13 | node-type-CUT x running-counter | continuous |

## Open items -- measure before trusting, don't assume symmetry with alpha

- **Move value / SEE on the beta side is reported to have measured weak in
  an earlier (undocumented) attempt, and the reason isn't understood.**
  Candidate explanations, none verified: (a) a ceiling effect -- beta-side
  outcomes already sit at 92-99%, leaving little residual probability for
  any feature to explain; (b) collinearity with `gap` -- the node's static
  eval may already reflect whatever material swing produced the
  advantage, making a candidate move's own SEE partly redundant once gap
  is in the model; (c) the original measurement itself may have been
  flawed (bad bucketing, a confound) rather than the signal being
  genuinely weak. Needs its own small, targeted diagnostic (same shape as
  `HASHCAP_COLLECTION`) logging move value alongside outcome on this
  population specifically, before deciding to keep, drop, or redefine it.
- **The alpha-side `dumb`/hang-estimate axis measured strong (96x); its
  beta-side analog is not assumed to be the same feature.** "Is material
  propping up this favorable eval actually under threat" is a different
  question than "is material hanging in general" -- needs its own
  definition and its own measurement, not a copy-paste from the alpha
  side.
- **Speculative features to log now (cheap to add to the collection build)
  and decide on later via measurement, not logged with any strong prior
  they'll survive:**
  - static eval magnitude itself, not just `gap` (a `+900`-eval node and a
    `+50`-eval node with the same gap may not behave identically)
  - ply-from-root, distinct from remaining depth (horizon/history-evidence
    maturity effects near the root)
  - repetition/50-move-adjacency (probably weak, cheap to check and drop)

## Collection

**Format: dense binary records, not text logs.** The existing
`FH_TABLE_COLLECTION` instrumentation writes full per-node trace text via
`Trace()`, which produced 13GB across 16 shards for the NNUE run and, per
observation on this session's `sd 12` shallow-tier shard, grows especially
fast on the shallow/high-diversity tier (`shard_15.log.bz2`, 5.3GB
*compressed*). Fix this structurally rather than stripping after the fact:
have the collection build emit one small fixed-width struct per logged
event (whole-node row, or one row per move/null-move tried) directly to a
per-shard binary file -- feature values and the outcome label, no text
formatting, no field names repeated per line. This is both far smaller on
disk and faster to load for training (straight into numpy via a dtype,
no text parsing).

**Keep the deep/shallow/medium shard tiering** (same idea as the existing
run: one shard `sd 20`, one `sd 12`, the rest `sd 16`) -- the shallow
tier's high position-diversity is valuable on its own terms and isn't the
thing causing the disk problem; the disk problem was the text-log format,
which the binary-record change solves independently of tiering.

**Collection-build changes needed:**

1. Disable LMP during alpha-side collection (existing `FH_TABLE_COLLECTION`
   precedent -- still needed, same reasoning).
2. Disable the static reverse-futility shortcut during beta-side
   collection (new requirement, same reasoning as #1: the shortcut would
   otherwise short-circuit the whole node before any real moves -- or even
   the whole-node row itself -- get a chance to be logged).
3. Eagerly generate moves for every eligible node regardless of whether
   null-move/hash-move would have short-circuited it in production, so
   `total legal move count` is real (not imputed) in every training row.
4. Log one `is_whole_node` row per eligible node before the move loop
   starts, then one row per move tried thereafter (`null` first where
   applicable, then hash, then the rest in generation order) -- all gated
   on the same `iEval <= iAlpha` / `iEval >= iBeta`, MWS-only eligibility
   check used today.

## Validating against the existing table: sanity check vs. rigorous gate

The existing table's best-populated cells carry enormous sample sizes --
the alpha table alone has individual `(depth,dumb,gap,late)` cells with
over 2 billion tries (see the `improve`-axis within-cell check earlier in
this doc's history). That's real, cheap ground truth, and it should be
used twice, at two different points, for two different purposes --
conflating them risks either dismissing a good model too early or
trusting a bad one too late.

1. **Cheap sanity check, against the smoke-test model.** Pick 3-4 of the
   table's highest-tries cells, build the matching feature vector, and
   confirm the smoke model's predicted FH% is at least in the right
   ballpark (right order of magnitude, right direction) -- not a
   rigorous statistical comparison, just a bug catcher. The smoke test's
   training sample is deliberately small, so a real mismatch here is
   more likely to mean a sign error, a scaling mistake, or a transform
   bug than a genuine quality problem -- treat a failure here as "go
   find the bug," not "the approach doesn't work." Run this right after
   the smoke test's model is fit, before investing in the full run.
2. **Rigorous pre-integration gate, against the full-run model.** After
   the real multi-day collection and fit, compare the final model's
   predictions against the same high-tries cells as the primary
   calibration check before any production code changes -- this is the
   one that actually answers "is this model trustworthy," since both
   sides of the comparison now have enough data to be statistically
   meaningful. Report predicted-vs-actual across a spread of cells (not
   just the handful used for the quick sanity pass), the same
   binned-calibration discipline the table itself would need if its own
   boundaries were ever re-derived.

## Plan

1. ~~Finalize this doc.~~
2. ~~Instrument the collection build.~~ Landed 2026-10-24:
   `FH_REGRESSION_COLLECTION` (replaces `FH_TABLE_COLLECTION` outright --
   see `GNUmakefile`) in `chess.h`/`search.c`/`searchsup.c`/`fhcollect.c`
   (new file)/`main.c`/`root.c`. Call-site design matches the "function
   calls, not big `#ifdef`s in the middle of `Search()`" guidance: three
   always-callable hooks (`FhCollectNodeEntry`/`FhCollectMoveTried`/
   `FhCollectNodeExit`, no-op inline stubs in a non-collection build) plus
   `FhClassifyMoveTier` (decodes `generate.c`'s packed `_ScoreAllMoves`
   sort key -- hash/good-capture/killer/countermove/leftover-quiet/
   losing-capture -- back into a tier, checked in the same priority order
   that scoring function documents, since `COUNTERMOVE_TIER_BONUS` shares
   a bit with `GOOD_MOVE` and needs an exact match, not a bare AND). A
   `FH_COLLECT_PENDING` struct, declared as an ordinary local in
   `Search()` (deliberately not stored in `ctx`/`PLY_INFO` -- recursion
   would clobber a shared copy before the parent node's own `NodeExit`
   ran), carries state from node entry through each move tried to the
   final whole-node commit. Per-thread binary files (`fhcollect_threadNN.bin`,
   one `FH_COLLECT_RECORD` per logged event, buffered in
   `FH_COLLECT_BUFFER_RECORDS`-sized batches) replace the old histogram
   entirely -- no accumulation/merge code needed anywhere, so the
   corresponding `root.c`/`script.c` copy-out-and-dump blocks were deleted
   outright, not just renamed. Verified end-to-end on a smoke run (see
   below) before moving on.
3. **Smoke test before the real run**: ran a single-shard, `sd 6`,
   one-position smoke test (`--cpus 1`, LMP confirmed disabled via "Overt
   move skips: LMP=0 moves" in the engine's own output). Both build
   profiles (`FH_REGRESSION_COLLECTION=1 PERF_COUNTERS=1 MP=1` and the
   normal `GENETIC=1 PERF_COUNTERS=1 MP=1 SIXTYFOUR=1` production
   profile) compile clean with no new warnings. Parsed the resulting
   16,384-record binary file in Python: zero suspicious rows, sane
   table(alpha/beta)/tier/label distributions (dominated by
   `LEFTOVER_QUIET` as expected with LMP off, a plausible handful of
   `NULL`/`HASH`/`KILLER`/rare `COUNTERMOVE` rows), and matching
   `NULL`/`NO_MOVE` row pairs sharing identical position signature/gap/
   depth at the same node, confirming the stash-then-commit two-phase
   design works correctly across the node's lifetime.

   **Full soup-to-nuts pass completed 2026-10-24**, including a real
   `sklearn` fit and a spot-check against the existing historical table
   -- see "Running, training, and validating" below for the commands
   and `regression_run/` for the scripts. Four real bugs were found and
   fixed along the way, all caught by this smoke-testing, none by code
   review alone:
   1. `FhClassifyMoveTier`'s capture check used `TRUE ==
      IS_CAPTURE_OR_PROMOTION(mv)` -- that macro is a bitmask test
      (`mv.uMove & 0x0FF00000`), not a 0/1 boolean, so it was almost
      always false for real captures. Every capture silently
      misclassified as a quiet move (mostly `LEFTOVER_QUIET`, some
      `KILLER` -- a losing capture's negative `iValue` sign-extends
      with its high bits all set, coincidentally matching the killer-
      bit test). Fixed to a plain nonzero check, matching every other
      real call site in the codebase.
   2. The per-move `move_value` feature for quiet moves was reading
      `g_iPSQT[mv.pMoved][mv.cTo]` (a positional square-preference
      number) via the packed sort key, not a material-safety estimate
      -- a different signal than the one actually designed and
      validated (the old table's `dumb` axis, 96x discrimination on
      the alpha side). Fixed to call `EstimateHangingMaterialCp`
      explicitly for quiet moves, matching `searchsup.c`'s own
      existing call to it.
   3. `GetMoveFailHighWilsonUpperBoundThousandths` returns a
      confidently-low value for a move identity with zero evidence
      (the table's own `n=1,k=0` cold-start convention), not a neutral
      "unknown" -- indistinguishable from a genuinely well-evidenced
      safe move without more context. Added `uMoveFhAttempts`
      (`GetMoveFailHighAttempts`, new function) as a companion raw
      evidence-count field.
   4. `total_non_pawn_material` (the phase/material feature, itself
      missing from the first implementation -- see the feature audit
      below) initially summed `pos->uNonPawnMaterial[color]` directly,
      which includes each side's own king at `VALUE_KING` ==
      `INFINITY_SCORE` (32767) -- a ~65534 constant swamping the real
      ~0-6725 of actual piece material riding on top of it. Fixed by
      subtracting `VALUE_KING` per side, the same convention `eval.c`'s
      own `uArmyScaler` already uses.

   **Feature audit against this doc's own tables** (done after the
   above fixes, before trusting anything further): every listed
   feature is present and correctly wired except alpha-side node-level
   feature #6, `hang/dumb estimate (position-wide)` -- never
   implemented at all. Decision: leave it as a documented placeholder
   (0) rather than build new code for it now -- the old table *also*
   pinned this exact whole-node slot to a hardcoded 0 (there's no
   natural move-independent analog of `EstimateHangingMaterialCp`
   without a more expensive full-board threat scan), so this isn't a
   regression, just an unfilled slot with real historical precedent.
   Revisit only if the trained model's calibration specifically
   implicates missing whole-node material-safety signal.

   The MP/`StartParallelSearch` split path remains deliberately
   uninstrumented (collection runs at `--cpus 1`, so this path is never
   exercised in practice).
4. Full multi-shard, multi-day collection run (binary format, same
   deep/shallow/medium tiering), against `typhoon_nnue` only (see "One
   model, not two" above).
5. Fit both regressions offline, quantize, validate calibration
   (predicted vs. actual FH%, binned, same discipline as validating the
   existing table) before touching production code.
6. Integrate into `search.c`/`searchsup.c`: replace the table-lookup call
   sites with the quantized dot-product + sigmoid-lookup inference,
   gated the same way the existing HCE/NNUE table pair is selected today
   (`g_fUseNnueEval && g_fNnueWeightsLoaded`-style runtime switch -- TBD
   whether this model needs a similar HCE/NNUE split or can be eval-
   agnostic; check empirically rather than assume).
7. Full validation ladder per `docs/DEEPER_PRUNING.md`'s existing
   discipline: wiring sanity (`DEBUG=1 TEST=1` + `debug_smoke_test.sh`),
   per-move FH-rate instrumentation, `sd 10` and `sn 5000000` suite diffs
   on the three curated suites, `match_play.py` gated self-play
   (`LOWER95 >= 0.5`) before this ever ships as the default.

## Running, training, and validating

Everything below lives in `regression_run/` (sibling of `src/`, same
convention as `dual_table_run/`), and all of it already works end to
end against real smoke-test data -- this is the actual, exercised
procedure, not a forward-looking plan.

### 1. Build the collection binary

```sh
cd src
gmake clean   # after any chess.h change touching many TUs
gmake -j5 PERF_COUNTERS=1 MP=1 FH_REGRESSION_COLLECTION=1
```

Produces `src/typhoon` with the collection hooks live. **Never ship this
binary for anything other than a collection run** -- LMP and the beta-
side static-null-move shortcut are both disabled under this flag (see
"Collection-build changes needed" above), a real strength regression in
normal play.

### 2. Launch a collection run

Same invocation convention as the rest of this project's ECM testing
(CLAUDE.md's "Testing: the ECM tactical suite" section) -- `--cpus 1`,
book disabled, `easy` to stop background pondering from contaminating
the logs, `sd <depth>` not `st <seconds>` for a reproducible, load-
independent run:

```sh
mkdir -p /tmp/typhoon/<run_name> && cd /tmp/typhoon/<run_name>
/path/to/typhoon --cpus 1 --hash 256m \
    --logfile /tmp/typhoon/<run_name>/run.log \
    --batch --command "force; book name /nonexistent.book.bin; easy; sd 10; \
    script /usr/home/scott/typhoon/tests/<suite>.ep_"
```

Produces `fhcollect_thread00.bin` (and `fhcollect_threadNN.bin` per
helper thread, if `MP=1` and `--cpus > 1`) in the current directory --
one `FH_COLLECT_RECORD` per logged event, 64 bytes each as of this
writing (confirmed via `offsetof`/`sizeof` against the real compiled
struct, not assumed -- see `regression_run/load_records.py`'s own
docstring for why that matters). A 90-position suite at `sd 10` runs in
well under a minute and produces on the order of tens of millions of
records to low hundreds of GB for the real multi-shard run -- launch
in the background (`nohup ... &` or `run_in_background`), don't block
the foreground on it.

### 2b. Or launch a real multi-shard run (16 shards, like dual_table_run*)

`regression_run/build_shards.py` is `dual_table_run_nnue/build_shards.py`
copied verbatim -- it's pure EPD-pool slicing, no awareness of which
table format reads the shard files it builds, so it needed no changes:

```sh
cd regression_run
python3 build_shards.py --repo-root .. --num-shards 16 \
    --medium-positions-per-shard <N> --out-dir shards
```

Then `launch_shards.sh <run_name>` -- same deep/shallow/medium depth
tiering convention as the dual-table runs (1 `sd20`, 1 `sd12`, 14
`sd16`, overridable via env vars, see the script's own header comment
for the all-`sd10` smoke-test invocation), using `../src/typhoon_nnue`
(NNUE-only collection, per the decision above):

```sh
./launch_shards.sh my_run_name
```

**One real difference from the old `FH_TABLE_COLLECTION` shard
scripts, worth understanding before relying on this**: there, each
shard's *data* lived in its own `--logfile`, so all 16 could safely
share one working directory. Here, the data is
`fhcollect_thread00.bin` -- named only by **thread number within a
process** (`chess.h`'s `FH_COLLECT_WRITER` comment), since the file-
naming scheme was designed around multiple *threads* in one engine
(`MP=1`), not multiple independent shard *processes*. 16 shards
sharing a directory would all collide on the same filename. Fixed with
no C changes: `launch_shards.sh` gives each shard its own working
directory (`runs/<run_name>/shard_NN/`) before `exec`ing the engine, so
each one's `fhcollect_thread00.bin` is a distinct file on disk.

**Recombining shards is simpler than the old design, not harder**:
`load_records.py`'s `glob`-and-concatenate already treats "every
matching file under a directory" as one dataset -- pointing it at
`runs/<run_name>/shard_*/fhcollect_thread00.bin` (via the `pattern`
argument) *is* the recombination step. There's no `aggregate.py`/
`bake.py`-style merge needed, because there's no in-process histogram
to sum across shards in the first place -- concatenating raw rows is
the whole operation.

**Checking progress and coverage while shards are still running**:

```sh
./dashboard.py my_run_name
```

Prints each shard's record count/file size and whether its process is
still alive, then re-derives the old table's discrete-cell coverage
view from the raw rows on the fly (via `old_table_buckets.py`'s exact
port of `searchsup.c`'s bucket functions) -- populated-cell count,
count below the same 500-sample floor the original table's own bake
used, and the sparsest populated cells (the "dusty corners" view).
Unlike the old `dashboard.py`, there's no accumulated histogram to
read -- this one loads the shard files directly (with
`snapshot=True`: a trailing partial record from a write caught mid-
flight, or the writer's own `FH_COLLECT_BUFFER_RECORDS`-sized buffered-
but-not-yet-flushed tail, is expected and silently dropped, not an
error -- see `load_records.py`'s own docstring) each time it's run, so
it's always reading real on-disk state, just never perfectly up to the
instant.

**Taking a mid-run snapshot to bake a model without stopping the
shards** (the other thing the old workflow supported): works the same
way -- pass `--snapshot` to either `explore_and_fit.py` or
`spotcheck_vs_table.py` (e.g. `python3 explore_and_fit.py
runs/my_run_name --snapshot` -- note these two scripts take the actual
directory, not the bare run name `dashboard.py` takes) to read
`<dir>/shard_*/fhcollect_thread00.bin` with `snapshot=True`, exactly
like `dashboard.py` already does. The
shards keep running throughout, same as the old workflow's "last
checkpoint is authoritative" model -- nothing here needs them paused.

**Stopping**: `./stop_shards.sh my_run_name` -- safe anytime, kills
each shard by its recorded pid; the only loss on a hard kill (vs. a
clean exit) is each shard's own still-buffered (not yet flushed)
tail, same `FH_COLLECT_BUFFER_RECORDS - 1`-record caveat as any other
abrupt stop.

**Validated mechanically** (2026-10-24): a 2-shard, 10-position,
`sd 8` pipeline smoke test exercised build → launch → dashboard
(mid-run-shaped coverage output, including the sparsest-cells view) →
stop end to end with no issues.

### 3. Load and sanity-check

```sh
cd regression_run
python3 load_records.py /tmp/typhoon/<run_name>
```

Prints row counts by table/tier and a label rate, and fails loudly
(nonzero exit, listed problems) on any structural issue -- out-of-range
table/tier/label/moves_known values, negative gap, implausible depth.
Run this on every new collection directory before trusting it further.

### 4. Explore and fit

```sh
python3 explore_and_fit.py /tmp/typhoon/<run_name>
```

Engineers features (log transforms, one-hot tier/node-type, normalized
ordinal, the running-counter fraction, the node-type x running-counter
cross term -- see `engineer_features()`), prints a per-feature
discrimination report (correlation with label, bottom/top-quartile
rate), fits a standardized logistic regression per table, and prints
log-loss/AUC plus a calibration table (predicted-probability bucket vs.
actual rate). This is the fast, routine command to re-run after any
new collection pass.

### 5. Spot-check against the existing historical table

```sh
python3 spotcheck_vs_table.py /tmp/typhoon/<run_name> \
    ../dual_table_run/pickles/baked_now.pkl
```

Trains both models (same as step 4), then -- using Python
re-implementations of `searchsup.c`'s exact bucket functions
(`old_table_buckets.py`, constants pulled directly from `chess.h`, not
guessed) -- finds real rows in the fresh collection that land in the
*same discrete cell* as the old table's own highest-tries cells
(`tier == LEFTOVER_QUIET` for the alpha side, matching the old table's
real per-move population; `tier == NO_MOVE` for the beta side, matching
its whole-node-only population), and prints old-table-rate vs.
new-model-mean-prediction side by side. This is the "sanity version" of
the two-tier validation plan described earlier in this doc -- cheap,
fast, and meant to catch a sign/scaling bug, not to be the final
calibration gate (that's step 5 of the numbered Plan above, against
the real multi-day run, with the **rigorous** version of this same
comparison).

**What a good result looks like**, from the actual 11-position
`ecm_ringers.ep_` smoke run: the large majority of matched cells (30/30
found on both sides against the old table's top-30-by-tries cells)
track within 1-3 percentage points of the historical rate, including
several beta-side cells with billions of real historical tries. Isolated
larger deviations are expected and not automatically alarming -- e.g.
this run's one real outlier landed on the `late` axis's known-unusual
saturated tail bucket (see "Why past LMR experiments failed"-adjacent
discussion of bucket 32 earlier in this doc); a *systematic* drift
across many cells in the same direction, rather than one isolated
bucket, is the thing that would actually warrant stopping and
investigating before trusting a model trained this way.

### 6. Quantize and bake (not yet automated -- next piece of work)

Not built yet. When this is wired up, the shape is: take the fitted
`sklearn` `LogisticRegression` + `StandardScaler` (as returned by
`explore_and_fit.fit_and_report`), fold the scaler's mean/scale
affinely into the raw-scale weights and intercept (`coef /
scaler.scale_`, with the intercept adjusted by `-sum(coef * mean /
scale)` -- exact algebra, no approximation, since standardization is
just an affine transform), quantize to fixed-point integers at a scale
chosen *after* seeing the real weight magnitudes (don't pre-commit to
e.g. int16 the way the old table's axes had to -- there's no storage
pressure forcing a tight scale here), and emit a small C array (weights
+ bias + a sigmoid lookup table) analogous to how `bake_dna.py` splices
tuned constants into `eval.c` today.

### 7. Wire into the engine (not yet done -- after step 6 and the full validation ladder)

Replace the `_JointTablePredictedFHThousandths`/
`_BetaTablePredictedFHThousandths` call sites in `searchsup.c` with the
quantized dot-product + sigmoid-lookup inference. Concretely:

- The feature vector at each call site needs to be assembled from
  values already computed there today (`iEval`/`iAlpha`/`iBeta` for
  gap, `uDepth` for depth, the existing `uRawImproveB`-style locals,
  the move's own tier/ordinal/`EstimateHangingMaterialCp`/SEE value,
  `GetMoveFailHighWilsonUpperBoundThousandths`/`GetMoveFailHighAttempts`
  for the evidence features) -- nothing new to compute, just reshaped
  into the model's input order.
- `_RefineNodeTypeGuess` and the null-move-skip site both currently
  reuse one table lookup (`uNodeLocalFH`) across multiple call sites
  per node specifically to avoid a duplicate table consult -- preserve
  that same reuse discipline with the new inference call, which is not
  free (a ~20-feature dot product), unlike a bare array index.
- Per the "One model, not two" decision above, there is no
  `g_fUseNnueEval` branch needed at the call site -- always use the one
  NNUE-trained model, under both HCE and NNUE builds.
- Full validation ladder before this ever ships as the default: `gmake
  DEBUG=1 TEST=1` + `debug_smoke_test.sh`, per-move FH-rate
  instrumentation, `sd 10`/`sn 5000000` suite diffs on the three
  curated suites, `match_play.py` gated self-play (`LOWER95 >= 0.5`).
  See `docs/DEEPER_PRUNING.md`'s existing validation-ladder discipline
  -- this is the same ladder, not a new one.

## Session note: depth-dependent sampling bias (2026-10-24 follow-up)

After hours of `nnue_run1` collection with tier-dependent sampling live,
Scott noticed the beta-side rigorous spot-check's depth-blowup
(`spotcheck_vs_table.py`, depth-bucketed error climbing from ~2pp at
shallow remaining depth to 20-37pp at the deepest buckets) wasn't
shrinking despite more hours of collection -- surprising, since more
data usually does narrow this kind of gap.

Root cause: `fhcollect.c`'s tier-dependent subsampling classified
beta-side `NO_MOVE` (the whole-node population, exactly what the
spot-check compares against the old table) as "common," so it got hit
by the aggressive `FH_COLLECT_SAMPLE_DENOM` (e.g. 1-in-250) uniformly
across ALL remaining depths. But a large REMAINING depth means the
node sits close to the root, where the branching factor makes nodes
structurally scarce compared to the shallow-remaining-depth population
near the leaves -- the deep tail's raw volume was always small
relative to the shallow glut, for tree-shape reasons having nothing to
do with how long collection runs. Subsampling both populations at the
same flat rate preserves that same scarcity ratio forever: running
longer scales the deep tail and the shallow glut up together, so the
deep tail's SHARE of the data (what the spot-check's matched-cell `n`
counts depend on) never improves. This is a sampling-design bug, not
evidence the model itself can't fit deep cells -- confirmed by the
matched-cell `n_new` counts themselves staying in the single/low double
digits at depth 5+ even after a full extra collection pass.

Fix: `FhCollectEmit`'s common/rare classification is now ALSO
depth-aware, not just tier-aware -- once `uRemainingDepthTicks / ONE_PLY
>= 3`, a record falls back to the gentle/rare denominator regardless of
tier, on both tables. Cheap to do (the deep population is naturally
small on its own, so relaxing its sampling doesn't meaningfully grow
disk usage) and directly targets exactly the gap the spot-check kept
surfacing.

**Required a full shard restart** (the denominators, and now this
depth check, are read/applied once per process -- a running shard
keeps using the old binary's logic until restarted), same as the
original tier-dependent-sampling fix. Restarting costs nothing but
some redundant re-processing of each shard's already-seen opening
positions (search is deterministic, so no bias is introduced -- see
that section's discussion); existing collected data is kept (append-
mode writer, never truncated). Re-run `spotcheck_vs_table.py` again
after this fix has had a few more hours to accumulate deep-bucket
coverage before concluding whether it actually closed the gap.

## Beta-side spot-check disagreement, investigation, and resolution (2026-10-24)

The rigorous spot-check (`spotcheck_vs_table.py`) consistently showed a
real, unexplained disagreement between the fresh beta model and the
old table, concentrated in two distinct ways at different points in
the investigation -- both eventually resolved (or, for the second,
found NOT to be a model problem), via direct measurement rather than
guessing:

**1. Mate-score window contamination (resolved).** The old table's
`gapbeta_b=20` overflow bucket (gap >= 2000cp) showed the fresh
model badly OVER-predicting (97-98%) against the old table's 25-58%.
Checked directly: that bucket is 99.4% concentrated at `gap` in
[31700, 32767] -- essentially `INFINITY_SCORE` -- with a complete gap
in the data between 3,000 and 20,000, meaning it's not a continuum of
"big material edge getting bigger" but two distinct populations: an
ordinary cp-scale edge, and a detected-mate static eval compared
against an ordinary beta window. The fresh collection's own recorded
label rate for the mate-score population was high (90-99% at
shallow/medium depth) -- the "disagreement" was the FRESH data being
right and the OLD TABLE being stale for this one narrow, structurally
different region (very likely a holdover from before the
NODE_TYPE-refinement rewrite, commit `69525eb`, landed literally
hours before the old table's own bake). Fixed by excluding `gap >
MATE_GAP_THRESHOLD` (5000, chosen in the empirical no-data gap between
the two populations) from BOTH training (`load_records.exclude_mate_window`)
and the old-table comparison population (the old table's own
overflow-bucket cells, since its ground truth there is the
stale/mismatched side) -- see `old_table_buckets.py`'s
`exclude_overflow` parameter to `spotcheck_vs_table.run_comparison`.
This also directly validated Scott's independent hypothesis about
`iBeta > +NMATE` windows being a structurally different regime not
worth training on.

**2. A second, smaller depth/improve-combo disagreement (ruled out as
a model defect -- NOT the same issue).** After the mate-window fix,
3-4pp average / up to 34pp disagreement remained at moderate depth,
ordinary gap, specific own/enemy-improve combos. Two hypotheses were
tested and ruled out: (a) more collection volume -- tested directly by
resampling 5x bigger (15M vs. 3M rows, same already-collected data,
no more time running shards) and the error got slightly WORSE, not
better, which is the signature of a real bias, not sampling noise; (b)
a missing `improve x enemy_improve` interaction term -- added and
refit, no measurable effect. The test that actually settled it (Scott's
own proposed validation criterion -- "run a zillion positions and see
if it approximates the predicted rate"): a targeted calibration check
on EXACTLY the disputed region, held out from training, with real
observed outcomes. Over 258,349 held-out rows in that region, predicted
probability matched realized frequency within 0.04-3.5pp at every
decile, aggregate 46.10% predicted vs. 46.85% actual. The model's
predictions ARE calibrated against real outcomes in the disputed
region -- the old-table disagreement there is most likely explained by
the same staleness mechanism as (1) (old table built before the
NODE_TYPE-refinement rewrite), not a defect in the fresh model.

**Validation methodology this settled on** (the actual answer to "how
do we know this model is trustworthy," distinct from "does it agree
with the old table," which can be unfair when the old table itself is
stale): bucket held-out rows (never used in training) by their own
predicted-probability decile, and check whether the REALIZED label
rate in each decile matches the mean predicted probability for that
decile. This is a direct test against ground truth (what actually
happened), not against a second model (the old table) that may itself
be wrong. Agreement with the old table is useful corroboration when it
holds, but calibration against real outcomes is the primary trust
criterion, and the one that should be re-run after any future
search/eval change (the same risk that made the old table stale
applies to this model too, just on a longer horizon).

## Production implementation (`fhmodel.c`) and a serious open problem found by actually benchmarking it (2026-10-24)

Implemented as fully quantized fixed-point integer arithmetic (Q16 for
table/final-sum arithmetic, Q30 for the handful of raw-feature
weights) -- explicitly NOT floats, per the original design premise
("the whole premise of this regression model is that it would be
fast, have few multiplies and adds, and could be quantized" -- not
optional, corrected after an initial draft wrongly used floats by
analogy to NNUE's L2/L3 tail, which Scott correctly pushed back on).
See `fhmodel.c`/`chess.h` (declarations folded into chess.h, matching
fhcollect.c's own no-separate-header convention) and
`regression_run/export_model.py` (generates `fhmodel_weights.h`).

**Hoisted node-level/move-level split** (Scott: "make sure to hoist as
much of the inference out of the move loop as you can... gap to alpha
is one of the terms that can be hoisted"): `gap`, `depth`, `improve`,
`enemy_improve`, `material`, and `node_type` are all fixed for a
node's entire move loop (none can change based on which move is being
tried), so `FhModelAlphaNodeBaseQ16` computes them exactly ONCE per
node (search.c, same point `FhCollectNodeEntry` already is), and
`SearchOneMove` only adds the genuinely move-varying terms (move
value, ordinal, tier, this move's own historical FH% evidence) per
move.

**Production feature-set scoping**: `good_captures_tried_no_cutoff`
and its cross term were dropped from the production model (though kept
for data collection/analysis) -- using them live would require adding
new always-on state to `search.c`'s move loop (currently
collection-only), and measured cost of dropping them was negligible
(AUC -0.0004/-0.0002 on alpha/beta). `FhClassifyMoveTier` was moved
from `fhcollect.c` (collection-only) to `searchsup.c` (always
compiled) since production inference needs real tier classification
unconditionally, not the collection-only build's no-op stub.

**Real bugs found and fixed via actually building and running this**,
not by inspection:
- The sigmoid-output lookup table overflowed `INT` (32-bit): an
  earlier version Q16-scaled `probability*100000` (max ~100000) on
  top of an already-complete integer unit, producing values up to
  ~6.55 billion, ~3x `INT32_MAX` -- caught via a real
  `-Wconstant-conversion` warning on the actual generated header, not
  by inspection. Fixed by not double-scaling: the table already
  outputs the final integer unit directly.
- `_RefineNodeTypeGuess`'s alpha whole-node branch initially called a
  function name left over from an earlier (pre-quantization) draft
  that no longer existed after the rewrite -- caught by the compiler
  (`undeclared function`), not by review.

**A serious, SERIOUS, still-UNRESOLVED problem found by running
`bench`** (Scott's explicit ask: compare `bench` speed against HEAD
with `git stash`): total nodes searched at the SAME fixed depths
(`bench.c`'s own per-position depth, unaffected by this change)
dropped from 1,678,274 (HEAD, table-based) to 302,251 (regression
model) -- a 5.5x reduction, not a modest speed difference. Traced via
temporary diagnostic instrumentation (added, used, and removed in this
same session -- not left in the tree) directly logging
`FhModelBetaPredictedFHThousandths`'s output: it returns 94.5-99.97%
for EVERY sampled call, regardless of gap (9-1299), depth (1-7
plies), or improve/enemy values varying widely. This saturates
`_RefineNodeTypeGuess`'s band classification into `NODE_CUT` almost
universally, which cascades into far more aggressive pruning
everywhere else in the engine that reads `pi->eNodeType` -- almost
certainly the real cause of the 5.5x node-count collapse, not a
reduced-dynamic-range-but-still-reasonable-magnitude issue.

**Root cause**: `FHM_BETA_INTERCEPT` is `+5.55` in real log-odds units
(`sigmoid(5.55)` ~= 99.6%) -- confirmed present even in the
PRE-quantization float fit, not introduced by the integer rewrite.
`export_model.py`'s `fit_final()` trains on the FULL mixed-tier beta
population (every per-move tier plus `NO_MOVE`) with just an additive
per-tier offset, but the only thing production ever calls
(`_RefineNodeTypeGuess`) is the `NO_MOVE`-specific whole-node case.
`NO_MOVE` rows are inherently high-base-rate (the node's static eval
already exceeds beta by construction of eligibility), and the shared
node-level slopes -- fit across ALL tiers combined, dominated by the
much larger per-move-tier population's own data -- are too flat
relative to that large additive offset to meaningfully move `NO_MOVE`
predictions away from it. The AUC=0.9234 reported during fitting is
real but measures discrimination ACROSS tiers (tier identity alone
separates classes well), not WITHIN the one tier (`NO_MOVE`) production
actually consumes -- exactly the kind of gap the holdout-calibration
methodology above would have caught if it had been re-run against the
FINAL production (trimmed-feature, mixed-population-fit) model
specifically restricted to `NO_MOVE` rows, rather than the earlier,
differently-fit model that calibration check used.

**Not yet fixed. Recommended next step**: refit the beta production
model on the `NO_MOVE`-only population specifically (not the mixed
population used for data collection/the general calibration work),
so the intercept and node-level slopes reflect `NO_MOVE`'s own actual
relationship to gap/depth/improve/material, not an average diluted by
tiers production never queries this function for. Re-run the holdout
calibration check (restricted to `NO_MOVE` rows specifically) and the
`bench` node-count comparison against HEAD again before trusting this
number -- do NOT re-attempt a speed comparison until this is fixed,
since the current 5.5x node-count gap makes any NPS number measured
against it meaningless (comparing search quality/aggressiveness, not
per-node computational cost).

**Build profiles verified clean** (all three): default (table-based,
unchanged), `FH_REGRESSION_COLLECTION=1` (collection), and the new
`FH_USE_REGRESSION_MODEL=1` (this implementation) all compile with
`gmake clean && gmake release PERF_COUNTERS=1 MP=1 [flag]=1` with zero
errors. `gmake DEBUG=1 TEST=1 FH_USE_REGRESSION_MODEL=1`'s self-test
suite ran to completion with no assertion failures or crashes
(`REAL_EXIT=0`) -- the new code path is memory/assert-safe even though
its PRUNING BEHAVIOR is currently wrong per the above. Do not treat
"doesn't crash" as "is correct" -- they were checked separately here on
purpose, and only the first currently passes.

## Trust audit (2026-10-24, same day): a close read before committing anything

Scott's framing going in: the candidate won a 400-game match (54.4% was
the actual score, see below -- "65%" was a misremembering, not what the
log shows) and bench looked ~fixed, but the whole process "felt
haphazard, seat-of-the-pants, rushed, and random," and he specifically
distrusted (a) the beta-side saturation fix, (b) a verbal "the table was
too conservative because of smoothing" explanation for a threshold
change, and (c) a "this feature isn't a strong signal" dismissal of the
good-captures-counter generalization. All three turned out to be worth
distrusting, to different degrees -- see below. This section is the
full audit trail; treat it as the actual state of the beta-side work,
superseding the "Not yet fixed" framing a few sections up wherever they
conflict.

### A real, unrelated bug found while trying to build anything

The working tree's `search.c` didn't compile at all, with no FH flags
present -- a prior edit had collapsed `SCORE iBestScore;` (declared near
the top of `Search()`) into `SCORE iBestScore = -INFINITY_SCORE;` and
left the combined declaration *below* the early-QSearch-jump that
already uses `iBestScore` at line ~402. Fixed by restoring the plain
early declaration and leaving the assignment where it was. Unrelated to
any FH work; just a reminder that "does it build" is step zero, not an
assumption.

### Match-play log: 54.4%, not 65%, and it fails the project's own gate

`eval_tune/regmodel_vs_head_nnue.log` (400 games, built from binaries
postdating the beta NO_MOVE-only refit) is 142W/151D/107L =
**score 0.5438**. Recomputing with `match_play.py`'s own formula (plain
normal-approximation CI, not Wilson): **LOWER95 ≈ 0.495** -- below the
documented `>= 0.5` gate. So, as of that run, this candidate had not
actually cleared the bar it needs to clear; "it won" was not an accurate
read of the saved log.

### Bench: NPS is the wrong metric; total nodes told a different story

`bench.c` prints `"Searched N nodes in T sec"` *before* the NPS line.
Comparing NPS across builds (as a first pass did) can look flat even
when the tree collapsed, because fewer nodes in proportionally less
time preserves throughput. Using the actual node-count line, same fixed
depths both builds:

| build | nodes | ratio vs HEAD |
|---|---|---|
| HEAD | 1,678,274 | 1.00x |
| regression (threshold=96000, the shipped default at the time) | 450,852 | **3.72x collapse** |
| regression (threshold=99000) | 1,489,894 | 1.13x (close) |

The original bug report's 5.5x collapse (pre NO_MOVE-only refit) had
shrunk to 3.72x post-refit -- a real improvement, but "the bench issue
is ~fixed" (said based on NPS alone) was not correct; the collapse was
still present, just smaller, until the threshold was also addressed
(see below).

### ECM suites confirm it, at real cost to tactical strength

`sd 10`, three curated suites, HEAD vs regression at the then-default
threshold (96000):

| suite | HEAD solved | regression solved (96000) | regression solved (99000) |
|---|---|---|---|
| ecm_ringers (11) | 2 | 3 (noise-level, n too small) | -- |
| ecm_confident_quick (90) | 56 | **41** (-15) | **56** (exact match) |
| ecm_hard_quick (90) | 13 | **7** (-6) | **12** (within noise) |

### `good_captures_tried_no_cutoff`: the signal is real, not broken by the generalization

Checked both the doc's claimed ablation (-0.0004/-0.0002 AUC on
alpha/beta for dropping it from the production feature set -- no saved
artifact existed anywhere for this number, but re-measuring it
independently, same methodology, confirms it: +0.0007/+0.0001 on
4M rows, negligible either way) and the raw univariate signal:

```
good_captures_tried_no_cutoff=0   FH_rate=20.42%
                             =1   FH_rate= 3.10%
                             =2   FH_rate= 1.69%
                             =3+  FH_rate~ 1.0-1.8%
```

~15-20x effect on its own, univariate AUC=0.68. It wasn't broken by
Scott's generalization from "hash move is a capture" -- it correlates
+0.41 with `log_ordinal`, a feature that didn't exist in the old
table's axis set and does exist in the new model, so the generalized
counter's real content is now redundant with (absorbed by) `log_ordinal`
in the full model. The marginal-AUC number is a true statement about
*redundancy given the richer feature set*, not a true statement about
*whether the phenomenon is real* -- conflating the two, especially
right next to a mention of not wanting new move-loop state, is exactly
what produced the dismissive-sounding "meh, not a strong signal anyway"
framing. The underlying finding stands; it just doesn't need its own
dedicated feature once `log_ordinal` is present.

### Calibration: the beta model is honest, both in aggregate and sliced

Held-out decile calibration, NO_MOVE population, post-refit model:

```
pred [0.950,0.960)  n=4,050   mean_pred=0.9549  realized=0.9533
pred [0.960,0.970)  n=3,966   mean_pred=0.9649  realized=0.9652
pred [0.970,0.980)  n=3,994   mean_pred=0.9751  realized=0.9735
pred [0.980,0.985)  n=1,979   mean_pred=0.9825  realized=0.9843
```

Sliced further by depth and material within the "would-skip" (pred>=0.96)
population specifically (to hunt for a bad subpopulation before touching
the global threshold): wrong-rate stays flat and low (0.6%-3.6%)
everywhere checked, depth 1-9 plies and material 0-7000cp alike. No
identifiable bad corner. **The "table was too conservative because of
smoothing, the regression model is more accurate" explanation for
needing `FH_BETA_SKIP_THRESHOLD=99000` is not supported by this data** --
there's no smoothing gap to correct; the model's 96% output is already
honest, on average and in every slice checked.

### The real explanation: firing FREQUENCY, not per-decision accuracy

Comparing the table's own real historical wrong-rate at its 96000
threshold (`dual_table_run_nnue/pickles/baked_now.pkl`'s raw NO_MOVE
cells, >=50,000 tries each, mate-window excluded) against the regression
model's held-out wrong-rate at the same nominal threshold:

| | wrong rate @ 96% |
|---|---|
| old table (historical, 66B real tries) | 1.77% |
| regression model (held-out) | 2.02% |

Close -- the two models roughly agree on what "96%" means in real risk,
aggregate. So the ECM/bench collapse is not explained by the regression
model being generically overconfident at 96% relative to the table.
Added a small diagnostic (`u64QuickNullMoves`/`u64QuickNullMoveSuccess`
tree counters, already existed; wired through to `g_Options` in
`root.c`, accumulated across a suite in `script.c`'s
`g_SuiteCounters.u64TotalQuickNullMove{s,Success}`, printed as a new
"quick nullmove fire" line in `_PrintSuiteReport` -- kept, not a
remove-after-session diagnostic, since it's cheap and generally useful)
and re-ran the suites:

| suite | HEAD fire rate | regression fire rate (96000) | regression fire rate (99000) |
|---|---|---|---|
| ecm_confident_quick | 47.09% (714K/1.52M) | 91.61% (745K/813K) | 71.05% (1.31M/1.84M) |
| ecm_hard_quick | 40.83% (833K/2.04M) | 87.49% (800K/914K) | 64.74% (1.45M/2.23M) |

The regression model fires the shortcut on roughly **twice** as many
eligible nodes as the table did, at a similar per-decision risk level.
Since each firing truncates a subtree, a ~2x per-node firing-rate
increase compounding across depth plausibly explains the 3-4x total
node collapse and the resulting solve-rate cost, without requiring
either model to be locally miscalibrated anywhere checked.

**Working hypothesis, not yet verified**: real search trees have
correlated neighboring nodes (siblings/descendants down one line often
share whatever feature combination drove a high prediction -- a recent
capture's eval margin, say -- for several consecutive plies). The
regression model's richer feature set can track that persistent
condition precisely and fire at every one of those correlated nodes;
the table's coarser, smoothed buckets more often scatter similar-but-
not-identical real positions across different cells, diluting the same
real condition below its own 96% cutoff more often. Same honesty per
isolated decision, different firing frequency once correlated decisions
down a real line are strung together. Worth checking directly (e.g.
autocorrelation of the shortcut firing along a single PV) before
leaning on this further.

### Category A vs B: ECM was the wrong instrument to pick the threshold with

`docs/DEEPER_PRUNING.md`'s framework: Category A (tighten safe-by-
construction machinery, validate with FH-rate/suite-diff) vs Category B
(deliberately accept a nonzero error rate for aggregate Elo, validate
*only* with self-play -- "the FH-rate harness tells you where the errors
are, but only self-play can tell you whether shipping them is a net
win"). Tuning `FH_BETA_SKIP_THRESHOLD` to recover ECM solve parity (as
this session initially did) applies the Category A instrument to what is
actually a Category B decision -- ECM puzzles are curated worst-case
tactical positions, exactly the population a deliberately more
aggressive pruner is expected to cost solves on even when it's a real
net win in ordinary games (cf. Crafty's own `search.c` comment on its
forward pruning: "this is classic forward-pruning and can certainly
introduce errors into the search. However, cluster testing has shown
that this improves play in real games"). Given typhoon searches ~800x
more nodes than Stockfish at comparable depth (`DEEPER_PRUNING.md`), the
regression model's higher firing rate is not obviously a regression to
correct back toward the table's rate -- it might be exactly the
direction needed, and ECM solve-count alone cannot distinguish those two
possibilities.

The one artifact that bore on this before the sweep below was the
match-play log quoted earlier, at the *aggressive* default (96000, 2x
table firing rate) -- inconclusive on its own (LOWER95≈0.495).

**Sweep run, 2026-10-24 (same day): 96000 vs 99000, head-to-head.**
200 games, `sd 8`, `twic_filtered.pgn` openings, both sides the
regression build differing only in `FH_BETA_SKIP_THRESHOLD` (two tiny
shell wrappers setting the env var, passed to `match_play.py` as the
two engine paths -- no code changes needed to run this). Result, 99000
as candidate against 96000 as head:

```
CANDIDATE_SCORE=0.8425  GAMES=200  WINS=141  DRAWS=55  LOSSES=4
ELO=+291.3+/-66.1  LOWER95=0.7920  TC=sd8
nodes: candidate(99000)=1,239,521,567   head(96000)=468,297,174
```

Not a marginal result -- `LOWER95=0.79` clears the `>=0.5` gate by a
wide margin, and 96000 lost decisively despite searching *fewer* total
nodes and running at higher aggregate NPS (i.e. it wasn't losing because
it did less work overall in wall-clock terms; it was losing because the
extra static-shortcut firings were cutting real signal, not dead
weight). This settles the Category A/B ambiguity raised above: for this
particular knob, the self-play result and the ECM-based intuition agree
once the right instrument is actually used -- 96000's ~2x-table firing
rate is a real strength cost, not a Crafty-style acceptable trade-off.
**`FH_BETA_SKIP_THRESHOLD`'s default (99000, `search.c`) is now
confirmed by self-play, not just a conservative stand-in.**

**Follow-up sweep, same day: 99500 vs 99000.** Same setup (200 games,
`sd 8`, `twic_filtered.pgn`), 99500 as candidate against 99000 as head:

```
CANDIDATE_SCORE=0.5650  GAMES=200  WINS=64  DRAWS=98  LOSSES=38
ELO=+45.4+/-48.6  LOWER95=0.4963  TC=sd8
nodes: candidate(99500)=976,833,926   head(99000)=683,750,951
```

Inconclusive -- `LOWER95` sits just below the `>=0.5` gate, and the
error bar (+/-48.6 Elo) is bigger than the point estimate. Slight nominal
edge toward 99500, not a confirmed win. Combined picture across all
three thresholds tested: a real cliff between 96000 and 99000 (+291 Elo,
decisive), then a flattening -- consistent with a model that's
genuinely miscalibrated-in-firing-frequency somewhere below ~98-99% and
roughly fine above that. **No case for moving off 99000 based on this
data**; pushing further (99500+) is unproven, not disproven -- would
need a much bigger or SPRT-gated sample to resolve a ~45-Elo effect with
this much game-to-game variance, and isn't worth prioritizing over
finishing the validation ladder.

### Queued: regression-model vs HEAD, both under NNUE eval (overnight, external)

Every match-play comparison so far was threshold-vs-threshold *within*
the regression build -- useful for tuning the knob, but it never
actually compared the regression model against the thing it's meant to
replace (the `data.c` tables) under the real eval the collection was
done against. Queued to run overnight, outside this session:

```sh
cd /usr/home/scott/typhoon/src/eval_tune

python3 match_play.py \
    /tmp/typhoon/threshold_sweep/typhoon_head_nnue.sh \
    /tmp/typhoon/threshold_sweep/typhoon_regmodel_nnue.sh \
    --pgn ../../pgn/twic_filtered.pgn \
    --games 20000 \
    --workers 18 \
    --sd 8 \
    --sprt --elo0 0 --elo1 5 \
    --scratch /usr/local/tmp/typhoon_match_regmodel_vs_head_nnue_overnight \
    --log /tmp/typhoon/threshold_sweep/regmodel_vs_head_nnue_overnight.log
```

Both sides run under NNUE eval (thin wrappers at
`/tmp/typhoon/threshold_sweep/typhoon_{head,regmodel}_nnue.sh`, each
setting `TYPHOON_BIN` and exec'ing `src/typhoon_nnue`'s existing
`--nnueweights`-injecting shim against `src/eval.nnue`, the same
promoted-current net both ways) -- isolates the FH-model question
(table vs. regression) from the eval question, which is the actual
thing this match needs to answer. `typhoon_regmodel` carries the
confirmed 99000 default. `--sprt` so an unattended overnight run stops
on its own rather than always paying for a fixed game count regardless
of effect size. Result pending; fold into this doc once it lands.

### Scope note: the per-move beta prediction was lost, not just weakened

Checked directly: `FhModelBetaPredictedFHThousandths` has exactly one
call site in production (`searchsup.c`'s `_RefineNodeTypeGuess`,
whole-node branch). There is no per-move beta call anywhere in
`search.c`'s move loop. The original motivation for this whole model --
getting an FH% estimate on individual candidate moves when
`eval >= beta` and null-move didn't resolve things, to drive pruning on
specific moves beyond the null-move probe -- isn't implemented at all.
The NO_MOVE-only refit fixed the saturation bug by narrowing scope down
to exactly the one thing still wired in, not by fixing the underlying
shared-population-fit problem in a way that would let a per-move beta
model come back.

**Reframing, 2026-10-24 (Scott + this session)**: given the Category A/B
discussion above, this may not be worth re-pursuing as originally
scoped. The whole-node-only model that actually got built is already a
deliberately-unsound, Crafty/Stockfish-style forward-pruning layer in
spirit (predict the node's own FH% once, before the move loop, and skip
it outright on a high-confidence prediction) -- rather than treating the
missing per-move/post-null-move case as a gap to backfill, it may be
more productive to lean into the whole-node framing that already exists,
validate its firing-rate/threshold choice properly via self-play (above),
and revisit the harder per-move case only if that doesn't pan out.

### Alpha-side trust, quantitatively confirmed

Re-ran `spotcheck_vs_table.py` against the real NNUE table
(`dual_table_run_nnue/pickles/baked_now.pkl`), full rigor (old_tries
>= 50,000 per cell, not the earlier small smoke sample):

- **Alpha**: weighted mean |old_rate - new_rate| = **0.119pp**, median
  0.037pp, across 10,684 matched cells. Worst disagreements (up to
  7.6pp) are all in single-digit-`n_new` deep-depth cells -- sampling
  noise, not a trend. This confirms, with real numbers, Scott's
  standing trust in the alpha-side model: it tracks the old table
  closely almost everywhere there's real data on both sides.
- **Beta** (this same run, using `explore_and_fit`'s general mixed-tier
  fit, i.e. the *pre-refit* shape): 3.528pp weighted mean, growing to
  20-30pp at depth -- reproduces the already-diagnosed saturation
  problem through a script path (`spotcheck_vs_table.py`) that was never
  updated to use the NO_MOVE-only refit. Not new information, but a
  loose end: that script should be updated to match `export_model.py`'s
  actual production fit before it's used as a beta-side trust check
  again.

## Explicitly out of scope for this pass

- PV-node collection/modeling (real `alpha < eval < beta` states) --
  future work, not attempted here.
- Any model family beyond logistic regression for the production path --
  trees and nets were considered and rejected on inference-speed grounds
  for this specific inner-loop use; revisit only if logistic regression's
  calibration proves insufficient after real collection.