summaryrefslogtreecommitdiff
path: root/src/TODO
diff options
context:
space:
mode:
authorScott Gasch <[email protected]>2026-10-09 14:18:52 -0700
committerScott Gasch <[email protected]>2026-10-09 14:18:52 -0700
commitfbdcb9d76dd2444d8bc631331eaeb9435cc247f8 (patch)
tree602ab360462d6f23188ba51f962c4abb2dc6c7e5 /src/TODO
parent73b4afb12abf7cc442831dea42b41b63a2ce07be (diff)
* Grab bag of eval.c changes.
* Don't bother not hashing when moves are pruned. * Prune moves move librerally with LMP. Inexplicably, this is slightly shallower than HEAD (despite pruning MORE). It may be because it is slightly slower. It consistently beats HEAD in match_play.py though (3x now).
Diffstat (limited to 'src/TODO')
-rw-r--r--src/TODO44
1 files changed, 3 insertions, 41 deletions
diff --git a/src/TODO b/src/TODO
index 48b42cf..0dbdb8d 100644
--- a/src/TODO
+++ b/src/TODO
@@ -4,8 +4,6 @@ Pawnhash size tunable from cmdline again
Think about when to avoid nullmove pruning based on checks in the line
-Think about futility pruning when a >= +NMATE
-
r2r4/p4pk1/2p3p1/1p1nPR2/5p1Q/2N5/PPPq4/1K4R1 w - - 0 0 -- check extension crazy
2026-09-17: search.c's extended futility pruning (VALUE_KNIGHT/VALUE_ROOK
@@ -26,42 +24,6 @@ whether the tiers actually need different margins here too). Not yet
built -- deferred behind self-play validation of today's iPositional/
futility-margin changes landing first.
-2026-09-18: build two eval.c lazy-margin variants and compare all three
-(current/baseline, "messy but fast", "super conservative") head to head.
-"Messy but fast": tune iResidualP2/98ByOwnArmy + iSuperLazyResidualP2/98-
-ByOwnArmy to accept ~5% exceeded rate against tests/cross_sectional_200k.ep_
-(search under each root position, not just eval in isolation -- want
-real search-tree exceeded rate, not a static-position read). "Super
-conservative": tune the same tables so the 200k set hardly ever misses.
-Then: (1) self-play match_play.py between baseline and each variant, and
-between the two variants directly; (2) full tests/ecm.ep_ run (not a
-curated subset) under each of the three binaries, same sd, compare
-solve counts. Interesting because it's a direct speed-vs-soundness
-tradeoff measurement on the exact margin mechanism this session spent a
-day mistakenly blaming for the 2026-09-17 regression (see recent git
-log/session notes) -- the real bug was searchsup.c's CommonSearchInit
-PV staleness, not the margins, so this is now safe to actually explore
-properly with a real experiment instead of blind percentile-guessing.
- Cheaper derivation than 11 rounds of guess-rebuild-rerun (see eval.c's
- guess #1-11 history comments): PersistMarginSafetyHistogramsToFile
- (calibrate marginsafety, CALIBRATE_MARGIN_SAFETY build) persists the
- raw per-bucket SWING distribution (real full-eval score minus the
- lazy/super-lazy estimate), not just a boolean exceeded count -- so
- one calibration run against the 200k pool lets a script sweep any
- candidate margin M against the persisted histogram and read "fraction
- of swings > M" directly, giving the M for ~5% exceeded and the M for
- ~0% exceeded from the SAME run. eval_tune/lazy_eval_margins/
- aggregate_marginsafety_shards.py already sums bins across shards;
- needs a small "exceeded rate at threshold M" query added alongside
- hist_percentile.py's percentile query. Caveat (documented repeatedly
- in eval.c's guess history): the recorded swing distribution is itself
- shaped by whatever margin was actually compiled in during that
- calibration run -- a different final M reshapes the search tree,
- which changes which nodes even reach the swing-recording code -- so
- this gives a very good first estimate for both variants, not a
- substitute for the real match_play/ecm validation once each variant
- is actually built.
-
2026-09-18: sharded ASSERT-fuzzing harness -- the technique that found
the CommonSearchInit PV-staleness bug (DEBUG binary + gdb, batch mode,
`script <shard>.ep_` at a shallow sd, let it run until either an
@@ -88,8 +50,7 @@ the pawn eval specifically. Maybe do eval learning on only the pawn
hashed eval terms.
2026-10-04: We need to go faster.
- - Can we get rid of rgSquares[] completely and use bitboards everywher?
- - Profile and speed hunt again?
+ - Can we get rid of rgSquares[] completely and use bitboards everywhere?
- Eliminate SEE calls?
- profile a super-lazy-always (material-only Eval() exit, forced
unconditionally, throwaway diagnostic build a la the zero-
@@ -116,4 +77,5 @@ hashed eval terms.
2026-10-04: Reign in extensions the make the tree smaller.
2026-10-04: Modify self-play harness to record eval score / PVs per
-move in the PGN output.
+move in the PGN output. Make self-play harness able to play crafty
+and then test the HEAD/candidate against crafty.