path: root/src/TODO

Better understanding of stuff like KRKRB == drawish, Q's come off = !drawish
Pawnhash size tunable from cmdline again
Think about when to avoid nullmove pruning based on checks in the line
Think about futility pruning when a >= +NMATE
Stick eval / king score in ply info and use it to trigger extensions

r2r4/p4pk1/2p3p1/1p1nPR2/5p1Q/2N5/PPPq4/1K4R1 w - - 0 0 -- check extension crazy

2026-09-17: search.c's extended futility pruning (VALUE_KNIGHT/VALUE_ROOK
gate, PREPARE_TO_TRY_MOVES) now folds in iImprovement + iPositional[side]
(the same per-tier p50 typical-value estimate QSearch's own futility gate
uses), but still uses one flat margin regardless of which Eval() exit tier
(PLY_INFO.uLastPositionalSource) produced iEval/iPositional at that node.
iPositional corrects for a tier's mean bias, not its spread -- a coarse
(super-lazy) read could carry more variance/tail risk than a fine one even
after mean-centering. Rather than a categorical "never prune from
super-lazy" rule (rejected as too blunt/magic-number-shaped -- iPositional
already replaces trust-by-category with trust-by-measured-bias), the
principled fix is a real EFP_BASE_MARGIN_BY_SOURCE[tier], calibrated the
same way as QSearch's FUTILITY_BASE_MARGIN_BY_SOURCE (CALIBRATE_QSEARCH_
FUTILITY-style harness: search the would-be-pruned move anyway, bucket
surprise rate by distance-short-of-bar AND uLastPositionalSource, check
whether the tiers actually need different margins here too). Not yet
built -- deferred behind self-play validation of today's iPositional/
futility-margin changes landing first.

2026-09-17: self-calibrating lazy/super-lazy margins -- instead of (or on
top of) the static per-bucket percentile tables, maintain a small mutable
per-(tier, material-or-army-bucket) delta, adjusted at runtime: at a low
sample rate (~1%), let a node that would have taken the cheap exit run to
full eval anyway, compare, and nudge the delta -- up fast on a real miss
(swing exceeded the margin), down slowly otherwise (fast-up/slow-down,
not symmetric AIMD). Motivated by this session's CALIBRATE_MARGIN_SAFETY
digging: exceeded-margin nodes aren't spread uniformly across the tree,
they cluster heavily inside a handful of "spicy" root games' subtrees
(see the recurring N1Q5/N1B5/N1R5/... and 5rk1/1b2Q2p/... FEN families
that kept reappearing across every guess this session, all clearly the
same search tree a few plies apart). A single fixed percentile can't
respond to "we're currently deep in a dangerous subtree right now"; a
fast-reacting runtime term could -- widen in response to a real miss
right when the search is actually inside the risky region, then decay
back out once it's past. Two real costs before building this: (1) it
makes eval score depend on search visitation order/history, adding to
the non-determinism MP already introduces and complicating the kind of
fixed-margin A/B comparison work this session did throughout; (2) it's
its own new tuning problem (step-up size, decay rate) layered on top of
the static-table one -- don't start building this until the question of
whether the current 30k-position calibration pool is even large/diverse
enough (see cross_sectional_1/2.ep_ variance check, tests/) is settled,
so it's clear which change is causing which effect. Explicitly rejected
alternative: hand-carving structural exceptions for specific recurring
motifs (e.g. "queen has infiltrated near the enemy king") -- doesn't
generalize, and is exactly the kind of special-casing that rots.

2026-09-18: STILL AN OPEN QUESTION, not decided either way. Scott's
worry re-examining this: reaction speed. At a ~1-in-10k sample rate, an
adaptive term probably can't react fast enough inside a spicy subtree to
matter -- by the time enough samples land there to trip the widen, the
search has likely already made every unsound cutoff in that region
using the too-narrow margin, so the wrong bound is already baked into
that iteration's result before the term ever moves. Raising the sample
rate to compensate directly fights the entire reason the lazy tiers
exist (paying full-eval cost specifically to make the cheap tiers
marginally safer). Separately, the complexity cost compounds with
something already proven expensive this session: guess #1-11 (eval.c)
showed changing one bucket's *static* margin reshapes the whole search
tree in ways that confound attribution (see those comments' "cross-
bucket confound" notes) -- a runtime-adaptive term makes that confound
permanent and load-bearing every game, rather than a one-time tuning
headache, and two engines with identical DNA could diverge on eval
*decisions* (not just timing) run to run. Current lean: shelve it, since
the mechanism can only close the barn door after the horse for the
specific node that triggered it, against a permanent tax on
determinism/reasoning-about-Eval(), for a gain the static percentile
tables already get within ~1% of via the boring approach. Not acted on
-- left open in case reaction-speed concerns turn out to be wrong in
practice (e.g. if "spicy subtree" actually means tens of thousands of
nodes wide, not a handful, the sample-rate math changes).

2026-09-18: build two eval.c lazy-margin variants and compare all three
(current/baseline, "messy but fast", "super conservative") head to head.
"Messy but fast": tune iResidualP2/98ByOwnArmy + iSuperLazyResidualP2/98-
ByOwnArmy to accept ~5% exceeded rate against tests/cross_sectional_200k.ep_
(search under each root position, not just eval in isolation -- want
real search-tree exceeded rate, not a static-position read). "Super
conservative": tune the same tables so the 200k set hardly ever misses.
Then: (1) self-play match_play.py between baseline and each variant, and
between the two variants directly; (2) full tests/ecm.ep_ run (not a
curated subset) under each of the three binaries, same sd, compare
solve counts. Interesting because it's a direct speed-vs-soundness
tradeoff measurement on the exact margin mechanism this session spent a
day mistakenly blaming for the 2026-09-17 regression (see recent git
log/session notes) -- the real bug was searchsup.c's CommonSearchInit
PV staleness, not the margins, so this is now safe to actually explore
properly with a real experiment instead of blind percentile-guessing.
  Cheaper derivation than 11 rounds of guess-rebuild-rerun (see eval.c's
  guess #1-11 history comments): PersistMarginSafetyHistogramsToFile
  (calibrate marginsafety, CALIBRATE_MARGIN_SAFETY build) persists the
  raw per-bucket SWING distribution (real full-eval score minus the
  lazy/super-lazy estimate), not just a boolean exceeded count -- so
  one calibration run against the 200k pool lets a script sweep any
  candidate margin M against the persisted histogram and read "fraction
  of swings > M" directly, giving the M for ~5% exceeded and the M for
  ~0% exceeded from the SAME run. eval_tune/lazy_eval_margins/
  aggregate_marginsafety_shards.py already sums bins across shards;
  needs a small "exceeded rate at threshold M" query added alongside
  hist_percentile.py's percentile query. Caveat (documented repeatedly
  in eval.c's guess history): the recorded swing distribution is itself
  shaped by whatever margin was actually compiled in during that
  calibration run -- a different final M reshapes the search tree,
  which changes which nodes even reach the swing-recording code -- so
  this gives a very good first estimate for both variants, not a
  substitute for the real match_play/ecm validation once each variant
  is actually built.

2026-09-18: sharded ASSERT-fuzzing harness -- the technique that found
the CommonSearchInit PV-staleness bug (DEBUG binary + gdb, batch mode,
`script <shard>.ep_` at a shallow sd, let it run until either an
ASSERT/SIGSEGV or clean exit) generalizes well and should be run
routinely, not just once ad hoc. Shard tests/cross_sectional_200k.ep_
(or any large stratified pool) into N pieces and run N concurrent
DEBUG-build _typhoon processes under gdb, one per shard, each logging
to its own file; investigate any hit (assert message + gdb backtrace)
the same way this session did. Needs a full core budget (N cores) so
it collides with self-play match runs -- schedule it for whenever the
box isn't already saturated by match_play.py workers, not concurrently
with them.

2026-09-18: We should experiment with more aggressive R factors for
nullmove at deep search depth remaining levels.

2026-09-18: blunders.pgn review (typhoon vs Crafty losses). Two distinct
loss shapes, different fixes:
  (1) "Slow grind" -- the majority of losses. Score sits near equal for
  many moves then drifts 0.1-0.3 pawns/move in Crafty's favor with no
  single bad move visible, compounding to -3/-4 over 20-40 moves.
  Annotated depths run ~10-17 ply for us vs ~18-30 for Crafty the whole
  game, consistently -- this is the NPS/depth gap compounding over time,
  not an eval or tactics bug. Only real fix is search efficiency (nodes
  needed per ply) or raw nps, not eval richness -- eval doesn't look wrong
  here, it's just arriving late.
  (2) "Cliff" losses -- a handful of moves per game where the score drops
  3-4 pawns in one move at a suspiciously shallow depth vs the moves
  around it (e.g. game 3, move 20: -0.5 for several moves then
  `20. Nc4 {-4.26/12}`, depth 12 vs neighboring 20-ply reads), mostly in
  40/60 (fast) time controls. Looks like a time-management gap, not
  search quality: candidate fix is EBF-based "don't start ply N+1 if
  there's no time to finish it" (cheap, Scott: "easy") PLUS a
  root-instability-triggered "buy more time" signal for volatile
  positions (PV changing across iterations, last iteration failed
  high/low, score still swinging over the last few completed depths, or
  a narrow best-vs-second-best gap) -- recognizing "this position is
  tactically critical" from scratch is hard/rejected, but search-
  instability is a cheap proxy already sitting in the ID loop. Not built.

2026-09-18: profile a super-lazy-always (material-only Eval() exit,
forced unconditionally, throwaway diagnostic build a la the zero-
selection-budget move-ordering trick -- never commit it) binary against
Crafty at matched depth/positions to find out where our nps actually
goes once eval is ~free. Scott's educated guess is SEE and the
is-this-move-a-check / legal-reply-to-check generators dominate once eval
is out of the way, not eval itself -- also suspect given the earlier
`<=`-to-`<` SEE-skip experiment on line ~4178 (skip SEE on same-piece-
for-same-piece captures) barely changed speed but made trees fatter
(more nodes needed), i.e. that SEE call was doing real pruning work, not
just costing time. Before assuming check-detection costs are irreducible,
confirm "is this move a check" is answered via O(1) bitboard AND against
the enemy king square from the post-move attack bitboard, not via a
generate-and-filter pass -- Scott believes he already does this "cheaply"
but isn't sure it's the fastest available method. Single-legal-reply-to-
check detection is likely the genuinely unavoidable cost of the three
(needs a real legal-move count in the child position) -- worth checking
whether it early-exits as soon as a 2nd legal move is found rather than
generating the full list.

2026-09-18: consider adding cLikelyUnstoppablePasser to the POSITION
structure and using it to extend searches.  Just like we already do
with cTrapped and cEnprise -- signals from eval.c to affect search.

2026-10-01: It seems like a Q vs RR in a pawnful endgame... Q dominates.
In general, we should work on encouraging the engine to maintain a good
pawn structure -- read more of that pawn power in chess book.  Work on
the pawn eval specifically.  Maybe do eval learning on only the pawn
hashed eval terms.