Dynamic move ordering overhaul: continuation-history, evidence-gated

countermove promotion, retired hung-piece-escape and NumLeftoverMovesToSelect.

Full session was built on a "measure the pick, not the game" methodology:
aggregate solve counts on curated suites are too noisy to tune move-ordering
knobs against, so most decisions here came from per-move fail-high/alpha-raise
rates at much larger sample sizes (leftover FH% instrumentation, a zero-
selection-budget diagnostic that isolates a single best-of-remaining pick,
and evidence-bucket calibration), not solve-count deltas alone. See
CLAUDE.md's "Dynamic move ordering experiments" section for the reusable
methodology and generate.c's _ScoreAllMoves comment for the resulting
ordering hierarchy.

Changes:
- Added g_ContinuationHistory: same growth/decay math as the existing
  g_HistoryCounters butterfly table, additionally keyed by the previous
  move, so its magnitude is self-calibrated rather than a hand-picked
  constant. Flat, sufficient response across a 256x scale sweep.
- Countermove-table matches now get a real GOOD_MOVE-tier promotion
  (previously the table was write-only, tracked for stats but never read
  for ordering), but only when the match's own accumulated
  history+continuation evidence clears COUNTERMOVE_EVIDENCE_THRESHOLD
  (10,000) -- a raw match with no track record was shown to perform
  identically to an ordinary leftover (~0.6-0.85% FH), so promoting on
  match alone would have repeated hung-piece-escape's mistake below.
- Retired hung-piece-escape's unconditional GOOD_MOVE-tier promotion.
  Evidence-calibration showed the overwhelming majority of triggers (a
  zero-evidence population 250-1000x larger than countermove's) performed
  at the plain-leftover baseline -- the promotion was mostly free tier-
  escape treatment for moves that hadn't earned it. Replaced with
  FLEE_BONUS, a flat same-tier nudge inside SelectBestWithHistory (never
  escapes GOOD_MOVE/leftover classification, unlike a generation-time
  promotion) at the magnitude found to plateau a same-tier-nudge sweep.
- Retired NumLeftoverMovesToSelect (the depth-indexed budget on how many
  leftover moves got a full selection scan before falling back to
  unsorted order). search.c's main move loop now always fully selects --
  the leftover pool was shown to contain real, findable signal a bailout
  budget was discarding for a node-count savings that didn't hold up net-
  net once measured by solve counts and fail-high rates rather than raw
  node counts (noisy on small suites independent of this change).
- Collapsed leftover-move instrumentation from sorted/raw pairs down to a
  single set now that "raw" (unsorted fallback) is structurally
  impossible; kept the countermove evidence-bucket calibration counters
  (ongoing check that COUNTERMOVE_EVIDENCE_THRESHOLD stays well-
  calibrated); removed the contested-node A/B harness and hung-piece
  evidence calibration now that the decisions they were built to inform
  are made.

Net effect on the three curated suites (sd 10): solve counts wash (tied,
+1, -1 across ringers/confident/hard), leftover fail-high rate improved
consistently on all three (the intended, directly-measured target of this
work). Not yet validated beyond sd 10 -- an sn-based run or
eval_tune/match_play.py head-to-head gate is the natural next check before
leaning on this as a proven strength gain rather than a directionally-
sound, sd-10-clean change.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_014XePz6Sk4qQsTaP2jVJWJu

1 files changed, 101 insertions, 0 deletions

diff --git a/src/CLAUDE.md b/src/CLAUDE.md
index c7c7e4a..52a6877 100644
--- a/src/CLAUDE.md
+++ b/src/CLAUDE.md
@@ -317,6 +317,107 @@ bake-in-if-better, repeatable indefinitely against a growing game pool.
   large improvement it first looked like. Always diff ECM numbers taken at
   the same `sd` depth.
 
+## Dynamic move ordering experiments (2026-08-29)
+
+A single long session that redesigned how quiet ("leftover", i.e.
+sub-`GOOD_MOVE`) moves get ordered, replacing several hand-picked-and-never-
+revisited heuristics with evidence-driven ones. Landed changes: retired
+`NumLeftoverMovesToSelect` (see below), added a continuation-history table,
+retired hung-piece-escape's unconditional tier promotion in favor of a
+same-tier `FLEE_BONUS` nudge, and made the countermove table's tier
+promotion evidence-gated instead of automatic. See `generate.c`'s
+`_ScoreAllMoves` routine-description comment for the resulting move-ordering
+hierarchy. The methodology below is the more durable takeaway -- reusable
+for any future move-ordering question, not just the ones it already
+answered.
+
+**Core technique: measure the *pick*, not the *game*.** Aggregate solve
+counts on a 90-position suite are too noisy to tune a single move-ordering
+knob against (a handful of positions flipping either way swamps the
+signal) -- confirmed independently twice now, once by a prior session (see
+`lmr_testing/RESULTS.md`'s "given-up leftover"/"would-prune surprise rate"
+methodology) and once by this one. Instead, instrument the *local, per-move
+outcome* (did trying this move raise alpha or fail high?) at a much larger
+sample size, and read percentages off that instead of solve/pass counts.
+
+- **The "select nothing" / zero-selection-budget trick**: temporarily force
+  the leftover-selection budget to 0 (a compile-time constant swap +
+  rebuild, not a runtime flag -- keep this diagnostic-only, never commit
+  it) so that only the single "discovered we're in leftover territory"
+  transition move gets a full `SelectBestWithHistory` scan per node. That
+  scan still runs over the *entire* remaining pool regardless of budget, so
+  its result is genuinely "the best-of-remaining leftover, if we could only
+  afford to pick one" -- a clean, isolated read on whether the move-ordering
+  heuristic itself is any good, uncontaminated by how many picks the budget
+  allows. This is what first revealed that the countermove table's exact-
+  match signal was real (a forced-dominant version of it lifted the
+  best-leftover fail-high rate by about +1 percentage point, consistently,
+  across three curated suites) after a *scaled* continuation-history
+  version of the same signal showed a flat, uninformative response curve
+  across a 256x range -- the two experiments together showed scale wasn't
+  the missing variable, key resolution was (exact-move match vs. a coarser
+  [piece][to]-keyed proxy).
+- **The contested-node A/B harness**, for "should class A rank above or
+  below class B" questions (e.g. countermove match vs. ply-2 killer):
+  aggregate per-class fail-high rates are confounded by *censoring* --
+  whichever class is ranked higher gets tried first, so a lower-ranked
+  class's measured rate only ever reflects the subset of nodes where
+  nothing higher-ranked already resolved the position. Fix: at each node,
+  cheaply detect (piggybacked on the *first* `SelectBestWithHistory` call's
+  already-full-pool scan, no separate pass needed) whether *both* classes'
+  candidates are present as genuinely different moves; only log an outcome
+  on nodes where that's true, and only for the first candidate tried that
+  belongs to either class. **Caveat discovered by running this twice** (once
+  for countermove-vs-killer, once for hung-escape-vs-killer): whichever
+  class is *structurally disadvantaged* in a given test setting only wins
+  its rare contests via unusually strong self-reinforcing history/
+  continuation evidence -- a self-selection effect that inflates the
+  disadvantaged class's apparent quality and deflates the favored class's,
+  independent of which is actually the better signal. Don't read a single
+  A/B setting's numbers at face value; compare each class's *least-filtered*
+  sample (the setting where it's favored) against the other's, not the
+  "loser" numbers from either individual run.
+- **Evidence calibration**: once a structural pattern-match (countermove
+  table hit, en-prise escape) is identified, bucket every occurrence by its
+  own accumulated `g_HistoryCounters`+`g_ContinuationHistory` evidence
+  (log-ish bands: 0, 1-99, 100-999, ...) and plot fail-high rate per bucket.
+  This is the test for "is this pattern-match alone trustworthy, or does it
+  need a track record?" -- countermove matches with zero evidence scored
+  statistically identically to an ordinary unprivileged leftover (~0.6-
+  0.85% FH) while evidence >=10,000 cleared 53-64%; hung-piece-escape's
+  zero-evidence population was 250-1000x larger than countermove's *and*
+  scored at the plain-leftover baseline too, revealing that the then-
+  shipping unconditional promotion was handing free tier-escape treatment
+  to a huge population that had done nothing to earn it.
+
+**Load-bearing gotcha, easy to reintroduce by accident**: a bonus added
+inside `SelectBestWithHistory` (a selection-time-only nudge to the local
+comparison value) never touches a move's persisted `iValue`, so it cannot
+change `fIsLeftoverMove`/EFP eligibility no matter how large it is --
+useful for a same-tier nudge like `FLEE_BONUS`, useless if the goal is an
+actual tier promotion. A real promotion (like the evidence-gated
+countermove bonus) has to be written into `iValue` at *generation* time
+(`generate.c`), not selection time. Confirmed by testing: an earlier
+"dominant countermove" diagnostic added its huge bonus at selection time
+and, despite clearly winning every internal comparison, never once actually
+exempted a move from EFP or leftover classification -- the data it produced
+was still valid (still genuinely measuring leftover-pool behavior) but the
+mechanism didn't do what it looked like it should.
+
+**`NumLeftoverMovesToSelect` retirement**: the depth-indexed budget that
+decided how many leftover moves got a full `SelectBestWithHistory` scan
+before falling back to unsorted order was removed entirely once the
+evidence above showed the leftover pool has real, findable signal a bailout
+was discarding -- `search.c`'s main move loop now always fully selects.
+Aggregate node-count deltas from this and related changes bounced around
+by double-digit percentages on the smallest curated suite (`ecm_ringers`,
+11 positions) with no corresponding solve change -- **on a suite this
+small, a..b search's chaotic sensitivity to move order (a few more
+depth-N-to-N+1 re-searches, a root fail-high or two) can move node counts
+a lot for reasons unrelated to the change being tested. Fail-high
+percentages and solve counts are the signal; raw node counts on small
+suites mostly aren't.**
+
 ## Environment notes
 
 - Shared, multi-user FreeBSD box with genuinely variable load (other Claude