| Age | Commit message (Collapse) | Author |
|---|
|
- Remove autoplay/ and autoplayer/ entirely: old opponent-automation
scrapers/harnesses (child_process.cc test scaffolding, compiled a.out
binaries and a .dSYM bundle, saved book/position files, macOS
._-prefixed resource-fork cruft) that predate this repo's current
tooling and were never referenced by anything still in use.
- GNUmakefile: add DIAG_NO_QSEARCH_FUTILITY/CALIBRATE_QSEARCH_FUTILITY
profile flags and testrecogn.o to the TEST=1 object list. Both are
companions to already-committed work that never got their own build
support committed: search.c's qsearch-futility calibration harness
needs the two profile flags to be buildable at all, and testrecogn.c
(added alongside the recogn.c bugfix, now committed here too) needs
to be in TEST=1's OBJS to actually compile/link.
- eval_tune/match_play.py, eval_tune/test_vs_head.sh: real fixes found
and applied earlier this session -- match_play.py's opening-book
leak (games weren't actually book-free), missing --hash/--cpus
(games ran on the 64k-entry/single-cpu memset-zero defaults instead
of this project's normal 256m/1cpu), a shared-logfile race across
concurrent match workers, and a report-parsing deadlock on engine
resignation. test_vs_head.sh reverted to comparing against
head_reference/typhoon + the live working-tree binary -- it had been
pointed at a since-deleted, long-stale one-off comparison binary
(typhoon_allbitboards) since 92fc412, silently invalidating every
"vs head" self-play check run through it since Sep 4.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Ka9o3S2eKqh4jNfmxVZ6fH
|
|
Three changes landed together, verified via the usual pipeline (release
+ DEBUG build, DEBUG smoke test, 40-game st1 match vs clean 434fa04,
score 0.600, llr +0.24):
1. New BOOC (bishops of opposite color) endgame drawish-scaling term
had a real bug in its opposite-color check:
(pos->uWhiteSqBishopCount[WHITE] && !pos->uWhiteSqBishopCount[BLACK])
only detects one of the two possible opposite-color configurations
and silently misses the mirror case (White dark-squared / Black
light-squared). Since each side's bishop-square-color flag is 0 or 1
whenever uNonPawnCount[side][BISHOP] == 1 (already checked above),
"opposite colors" is exactly the XOR of the two flags:
(pos->uWhiteSqBishopCount[WHITE] != pos->uWhiteSqBishopCount[BLACK])
2. Extracted the winning-chances/BOOC/fifty-move drawish scaling out of
Eval() into its own EvalLookForDrawishSituations(pos,
&iScoreForSideToMove) helper -- same semantics, cleaner separation.
Fixed two small issues in the extraction: missing `static` (every
other file-local eval.c helper is static; this had accidental
external linkage with no prototype anywhere) and a typo in a new
EVAL_DUMP trace string ("At of all pieces" -> "After all pieces").
Also reordered Eval()'s two king evaluations to go side-to-move
first / enemy second (via the already-cached uColor/xColor) instead
of always BLACK-then-WHITE -- confirmed safe, no dependency between
the two _EvalKing calls (each only reads attack-bitboard data
already populated by earlier phases).
3. Re-calibrated and trimmed SUPER_LAZY_MARGIN_BY_ARMY and
iSwingFloorByArmy. Both were originally derived by measuring
symmetric |real - lazy| swing, which conflates a swing *toward* the
alpha/beta boundary (the only direction that can make an exit
unsound) with a swing *away* from it (harmless). Re-ran
CALIBRATE_MARGIN_SAFETY against the same 1500-position
tests/twic_sample.ep_ (sd 8) with the harness fixed to measure only
the dangerous-direction swing: true max ran 15-50% below the old
symmetric measurement in most material buckets, several averaged in
the single digits, and every bucket showed exceeded=0 even before
adding any headroom back.
iSwingFloorByArmy: {1069,1069,1069,974,821,876,796,754}
-> {297,297,297,286,461,453,582,600}
SUPER_LAZY_MARGIN_BY_ARMY: {2000,1800,1800,1750,1000,850,850,850}
-> {300,1635,1800,1070,946,850,734,698}
(buckets 2 and 5 unchanged -- already tighter than a fresh 15%
headroom over the new directional max would give)
Motivation: the board-representation-migration branch exists to
close a measured 5x nps gap vs Crafty on the same CPU (profiled:
typhoon spends more of its search time in Eval() than Crafty does
in evaluate()); every lazy/super-lazy exit that fires is Eval()'s
fast path, so trimming unnecessary margin headroom directly
increases how often the cheap path is taken instead of a full
evaluation.
Not yet done: splitting alpha-margin and beta-margin into independent
per-bucket values (currently symmetric per bucket, no principled reason
they need to be) -- would need another calibration pass tracking the
two separately.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Ka9o3S2eKqh4jNfmxVZ6fH
|
|
Brings in the last remaining piece from stash@{0}: the super-lazy exit
point (material-only pre-check before the regular lazy gate), a
material-bucket floor under the regular lazy exit's margin
(iSwingFloorByArmy), and search.c's qsearch futility rework
(FUTILITY_BASE_MARGIN_BY_SOURCE, indexed by which Eval() exit tier
produced the score). Required Eval()'s signature change from a single
SCORE* to SCORE(*)[2] (positional estimate per side instead of one
munged magnitude) -- search.c's futility margin folds in
rgiPositional[pos->uToMove], which the earlier bad-trades investigation
found to be a meaningfully predictive signal.
search.c and chess.h brought in wholesale from the stash (both were
either completely untouched by prior commits or contained no
divergence worth preserving). eval.c required hand-merging on top of
this session's already-applied bad-trades fix, B-over-N removal, and
xColor/reorder cleanups -- ported the super-lazy exit block, the
regular-lazy material floor, the per-color piPositional writes (all
three exit sites: super-lazy, regular-lazy x2, full-eval), the
super-lazy calibration harness (RecordSuperLazyMarginSafetySwing,
dual-regime DumpMarginSafetyCalibration), and moved uArmyScaler/
uNumTrapped initialization to match the new ordering the super-lazy
exit depends on.
Verified: clean release + DEBUG build (only the previously-flagged
_EvalTrappedPieces warning), DEBUG smoke test pass, and a 40-game
st1 match against clean 434fa04 (score 0.487, llr -0.04) -- landing
this margin machinery as-is from the stash, before any retuning, does
not regress strength on its own. This confirms the original
regression (0.15-0.225 score seen early in this investigation) was
fully explained by the bad-trades unsigned-underflow bug, not by these
margins being unsound.
Values are the original, as-derived-from-calibration ones (see
inline comments: SUPER_LAZY_MARGIN_BY_ARMY from 100 positions/sd8
with ~15-20% headroom, iSwingFloorByArmy from 1500 positions/sd8
with ~25% headroom, FUTILITY_BASE_MARGIN_BY_SOURCE from a separate
1500-position/sd6 surprise-rate calibration). Not yet retuned --
suspected to carry more headroom than necessary, which costs real
search speed (speed=depth). Next step: re-run the margin-safety
calibration fresh against the current build and trim the super-lazy
and regular-lazy floor headroom down from the built-in database,
leaving the qsearch futility margins alone (different calibration
method, already risk-tolerant by construction).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Ka9o3S2eKqh4jNfmxVZ6fH
|
|
reorder
Continues re-applying the eval.c overhaul from stash@{0} (see fbb138c),
each piece verified individually against clean 434fa04 (fast st1 matches,
~30-40 games each) before landing:
- _EvalBadTrades: replaced the TRADE_PIECES/DONT_TRADE_PAWNS lookup
tables with direct arithmetic (part of a broader "eval.c is too slow,
cut lookup tables where possible" effort). The stash's first attempt
at this had a real bug: uInverseBehindPieceCount was computed by
subtracting a raw material sum from 9 in unsigned arithmetic, which
underflows to ~4 billion in any non-bare-endgame position with
unequal material -- confirmed via match_play (score 0.15-0.225 over
20 games vs clean 434fa04, a near-total wipeout). Fixed to use the
existing uNonPawnCount piece-count field directly instead of
reinventing it via material math, and restored a cheap zero-pawn
guard (old DONT_TRADE_PAWNS punished "material lead with zero pawns
left" specifically -- the classic KNB-vs-KRP false positive -- which
the arithmetic replacement had dropped entirely). Verified back at
parity (0.475/40 games) after the fix.
- _EvalRook: ROOK_FULL_HALF_OPEN_BONUS is now a startup-computed cache
(InitEval(), refreshed on every DNA reload) instead of a per-call
local array rebuild.
- xColor = FLIP(uColor) cached once instead of recomputed inline:
_EvalBishop, _EvalKnight, CountKingSafetyDefects (also renamed its
uSide/xSide params to uColor/xColor to match), and Eval()'s own
per-piece-type loop.
- _EvalBishopPairs collapsed to one FOREACH_COLOR loop instead of
duplicated per-side code.
- eval.c "diet" cleanup: removed pos->iTempScore (a scalar handoff
between _EvalKing and Eval()'s per-color copy, now redundant --
_EvalKing writes directly into ctx->sPlyInfo[ply].iKingScore[uColor]);
removed several stale comments documenting already-historical
removals; _EvalPassers renamed to _ReEvalPassers; re-enabled a
previously-disabled ASSERT in _EvaluateCandidatePasser (confirmed
live via DEBUG smoke test, does not fire under current DNA).
- Added _SideHasWinningChances (Crafty-style material-only "can this
side force a win at all" classifier) plus the fifty-move-rule
dampening in Eval() -- both scale the score toward g_iDrawScore when
material alone rules out real winning chances, without touching
Eval()'s signature (kept the existing scalar piPositional convention
rather than pulling in the super-lazy exit's per-color rework).
- Removed the "B over N in the endgame with 2 pawn wings" term
entirely, by direct instruction -- BISHOP_OVER_KNIGHT_IN_ENDGAME is
now an orphaned DNA constant (matches the stash's own choice, which
left it similarly unused).
- Eval()'s main per-piece-type loop reordered from "all of our minors,
then all of the enemy's minors, then rooks both colors, then queens
both colors" to "ours then enemy's, interleaved per type" (knights,
then bishops, then rooks, then queens). Verified the real dependency
this ordering has to preserve -- _EvalRook/_EvalQueen read the
enemy's bbMinorAttacks, _EvalKing reads both colors' bbMinorAttacks/
bbQueenAttacks, _ReEvalPassers needs the king scores -- and confirmed
the new interleaving still satisfies it (all minors both colors done
before any rook/queen; both colors done before king).
Verification methodology throughout: gmake clean release build, DEBUG
smoke test (10 random ECM positions at sd 5, asserts compiled in), then
a fast (st 1, 40-game) match_play.py run against a clean 434fa04
reference binary before landing each piece. A same-binary-vs-itself
control match (score 0.500/20 games) confirms the harness itself has no
structural bias, so per-checkpoint scores in the 0.44-0.58 band across
this session reflect real (if noisy) parity, not measurement artifacts.
Still not re-applied (remains in stash@{0}): the super-lazy exit, the
material-based normal-lazy floor, and search.c's qsearch futility
rework -- these three are one coupled unit (the futility margins index
by which Eval() exit tier fired) and go in as the next, more carefully
scrutinized step, since the original regression that started this
whole investigation traced to exactly this area.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Ka9o3S2eKqh4jNfmxVZ6fH
|
|
Confirmed self-play regression traced to a stale test_vs_head.sh reference
binary (typhoon_allbitboards/550ea81, deleted): every "vs head" comparison
since 92fc412 (Sep 4) was checking new work against that fixed Sep-4
snapshot, never against real HEAD or the working tree. Rebuilt clean
reference binaries directly from git and re-verified everything from
scratch.
This commit lands only the pieces confirmed safe against clean 434fa04
(fast st1 match, ~30-40 games, score ~0.44-0.55, consistent with parity;
plus a DEBUG-build smoke test pass):
- recogn.c, fen.c: real bugfixes
- data.c, draw.c, ics.c: whitespace only
- x64.asm: retires the asm GetAttacks implementation now that chess.h's
GetAttacks macro unconditionally selects the already-verified-faster
_GetAttacksBB bitboard version instead of a three-way build-flag
toggle (GETATTACKS_BITBOARD/CROUTINES/asm default)
- see.c, testsee.c: SEE/test-harness updates supporting that default
- root.c: per-tier eval-exit reporting (super-lazy counters currently
always read 0 -- accurate, since no super-lazy exit exists yet)
- main.c: startup banner update, InitEval() call, TestRecogn() added to
the #ifdef TEST self-test sequence
- command.c: InitEval() DNA-reload hook, new qsearchfutility diagnostic
- dynamic.c: minor changes
- chess.h: the GetAttacks default change above, three
FUTILITY_BASE_MARGIN_* compatibility aliases (all still equal to the
original flat FUTILITY_BASE_MARGIN -- search.c has not been split into
per-tier margins here), placeholder super-lazy counters, and an
EvalPasserRaces -> _EvalPasserRacesAgainstLoneKings rename (confirmed
byte-identical body) to match recogn.c's call site
- eval.c: the same rename, plus a no-op InitEval() stub (nothing to
initialize until the ROOK_FULL_HALF_OPEN_BONUS cache below exists)
Deliberately NOT included: the full eval.c overhaul (~1770 lines) and
search.c's qsearch-futility rework (~650 lines), including yesterday's
loosened SUPER_LAZY_MARGIN_BY_ARMY/FUTILITY_BASE_MARGIN_BY_SOURCE tables.
Reverting just those two tables while keeping the rest of the eval.c
overhaul still lost badly to 434fa04 (0.20 over 10 games), so the
regression isn't fully explained by the margins alone -- the eval.c
overhaul needs careful, incremental re-verification against this commit
as the new baseline, not a bulk re-apply. Full original work preserved in
git stash (stash@{0} as of this commit) for that follow-up.
Note: two pre-existing, position/state-dependent assertion crashes were
found during this verification (util.c:1093 WalkPV, recogn.c:1359
_SanityCheckRecognizers), both reproducing on unmodified 434fa04 -- not
introduced by anything here, not yet root-caused.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Ka9o3S2eKqh4jNfmxVZ6fH
|
|
Add ASSERT(uWhite < 32)/ASSERT(uBlack < 32), the bound that actually
matters for g_SwapTable[14][32][32] -- the existing (& 0xFFFFFF00)
checks only caught garbage above bit 7 and would have passed silently
through the exact out-of-bounds indexing fixed in the previous commit,
had a similar bit-layout mistake been made again in the future.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01XxmVi2sTMwpPp4i6WYFjan
|
|
Recalibrate iKingSwingP90 against the bitboard-rewritten
CountKingSafetyDefects (~1.28B samples via new CALIBRATE_POSITIONAL/
CALIBRATE_BASE_MARGIN/CALIBRATE_MARGIN_SAFETY diagnostic build flags,
board_representation/EVAL.md section 9). Add LAZY_EVAL_MIN_MATERIAL:
measured the regular lazy exit's real swing exceeding its own assumed
margin 20.6% of the time in near-bare-king endgames (vs <=0.36%
elsewhere) -- skip lazy eval entirely below that material floor.
Double the stale search.c/searchsup.c CountKingSafetyDefects extension
thresholds as a stopgap pending their own recalibration.
Eval hot-path trimming (measured via EVAL_TIME, ~1759 -> ~1386 avg
cycles/eval on a representative middlegame position):
- Pull _GetFileStormDefects out of EstimatePositionalScore's hot path
(cost more than the "cheap cached lookup" it was assumed to be,
running on ~90% of all Eval() calls).
- Add pos->bbOccupiedSide[2], incrementally maintained alongside
bbOccupied, so _BuildFriendlySideBB is a field read instead of a
6-term OR.
- Switch CoorFromBitBoardRank8ToRank1/Rank1ToRank8 to the existing
static-inline FastFirstBit/FastLastBit (same bsf/bsr instruction,
no call/ret overhead).
- Defer EvalPasserRaces' uRacerDist/fDontCountMeOut past its
no-passer early return.
- Remove the mailbox-era "max mobility in a row" term from
_EvalBishop/_EvalRook (no bitboard-mobility equivalent need for it).
- Simplify _EvalBishopPairs and rook file-openness/passer bonuses to
flat DNA-tunable constants instead of distance/pawn-count-scaled
tables, rook file-openness now a branchless bitboard-indexed lookup.
- Remove pos->cPiece (write-only, no reader anywhere).
- Collapse WHITE/BLACK mirror-branches (castle-rights block,
rook-trapped-in-corner) to color-indexed constants.
- Close the PAWN_BIT..KING_BIT gap (bits 7-3 -> bits 4-0), removing
the bvPattern >>= 3 before its KING_COUNTER_BY_ATTACK_PATTERN
lookup. This also fixes a real bug introduced earlier this session
when _WhoControlsSquareFast was converted to read these constants
directly: g_SwapTable is only [32][32], but the old bit values
(up to 0xF8) indexed far out of bounds on any attacked square --
data.c's InitializeSwapTable was always built assuming the bits
0-4 range this change now actually produces.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01XxmVi2sTMwpPp4i6WYFjan
|
|
57502d6)
Flagged during the ATTACK_BITV/bvAttacks cleanup (5405191) but left
unfixed there deliberately, per direct instruction, since a real fix
changes eval scoring and deserves its own before/after check rather
than being buried in a mechanical rename commit.
The gate ("is this square safe to advance a helper pawn into, i.e.
not enemy-attacked or already friend-defended") used to read the old
combined bvAttacks word for both colors at the target square. This
function runs from _EvalPawns, the first piece type Eval() evaluates
each call -- no non-pawn piece (and, since commit 57502d6, not even
pawns themselves) has written any attack data yet at this point. That
word has therefore been unconditionally zero, and the gate
unconditionally true (silently disabled), since 57502d6 landed.
Fixed using pos->bbPawnAttacks -- the one piece-type bitboard that
actually is valid this early in Eval()'s sequence (populated at the
top of _EvalPawns, before this function runs). Narrower than whatever
the original gate covered (pawns only, not every piece type), but a
real, correct check instead of a fake one, and pawns are the dominant
real-world case for contesting a helper-pawn's advance square anyway.
Applied to both the right- and left-side helper searches (the initial
target-square check and the backward walk-and-search loop each need
their own copy, since the loop's own square changes every iteration).
Verified via precommit_check.sh, then all three curated suites at
sd10 (not just ringers, since this is a real eval-scoring change, not
a mechanical one): net +2/191 (113->115), concentrated in
ecm_hard_quick (18->22) with a small ecm_confident_quick give-back
(85->83) and ringers unchanged (10/11) -- the same "gains lopsided
toward hard_quick" shape this session's other real eval changes have
shown, consistent with a genuine (if modest) positional improvement
rather than suite-specific noise.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01XxmVi2sTMwpPp4i6WYFjan
|
|
Now that king (the last piece writing it) has converted to a bitboard
accumulator, nothing writes rgSquare[c|8].bvAttacks any more -- deletes
the whole mechanism rather than leaving a known-dead struct around:
- chess.h: ATTACK_BITV union gone. SQUARE collapses from a
union-with-ATTACK_BITV to a plain {pPiece, uIndex} struct (the #pragma
pack(1) that only existed for ATTACK_BITV's bitfield layout goes too).
UNSAFE_FOR_ROOK/UNSAFE_FOR_QUEEN and the whole-word MINOR_XRAY_BIT/
ROOK_XRAY_BIT/QUEEN_XRAY_BIT constants deleted -- confirmed unused
(only ever referenced in stale comments, not code) now that every
consumer reads bbXAttacks bitboards directly. PAWN_BIT/MINOR_BIT/
ROOK_BIT/QUEEN_BIT/KING_BIT stay: they're a separate, still-live
local bit-packing scheme _EvalKing/_WhoControlsSquareFast use to
build a per-square attack-pattern index into KING_COUNTER_BY_
ATTACK_PATTERN/g_SwapTable, unrelated to the retired storage struct.
- eval.c: _ClearAttackTables drops its entire macro-unrolled,
128-square clearing loop (CLEAR_A_SQ/CLEAR_A_RANK/CLEAR_SHORT_RANK,
all deleted) -- it only ever existed to zero the old per-square
struct; clearing the 7 bbXAttacks accumulators is the whole function
now. The transitional _IsSquareAttackedByX/_IsSquareXrayedByX helpers
(minor/rook/queen/king, 8 functions total) are deleted outright, not
just simplified -- their only remaining purpose was bridging to the
now-gone struct, and their DEBUG cross-checks were explicitly
migration-only scaffolding, not a permanent invariant. Call sites
(_WhoControlsSquareFast, _EvalKing's bvAttack/bvXray/bvDefend) read
the bbXAttacks bitboards directly instead. _WhoControlsSquareFast
simplifies to a flat OR of 8 bitboard membership tests per color,
down from raw struct reads plus 7 helper calls each.
Found and fixed one real, pre-existing bug while doing this (flagged
and confirmed with the user before touching it, kept as its own
documented change rather than silently folded into the mechanical
rename): _EvaluateCandidatePasser's helper-pawn-safety gate read
rgSquare[c1+8].bvAttacks[...].uWholeThing, but this function runs from
_EvalPawns -- the first piece type Eval() evaluates each call, before
any non-pawn piece (or, since commit 57502d6 retired pawns' own
bvAttacks write, even pawns) has written anything there. That word has
therefore been unconditionally zero, and the gate unconditionally true
(a silent no-op), since 57502d6 landed -- not something today's cleanup
introduced. Left exactly as dead/unconditional (deleted the
now-meaningless condition, kept the body it always ran anyway) rather
than fixed, since a real fix changes eval scoring and deserves its own
before/after check, documented inline for a future session.
Verification: precommit_check.sh (self-test + DEBUG smoke test) passes.
tests/ecm_ringers.ep_ at sd10 vs. the immediately preceding commit:
solve parity holds exactly (10/11 both), and final (depth-10) node
counts are byte-identical for all 11 positions -- the bar for a change
meant to be purely mechanical, unlike king's own conversion. One
harmless artifact noted: ECM.750 has a different depth-6 *intermediate*
best move (a shallow tie-break flip) that already resolves to the
identical PV and node count by depth 7 and holds through depth 10 --
not chased further since the actual (depth-10) result matches exactly
and search is deterministic at --cpus 1.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01XxmVi2sTMwpPp4i6WYFjan
|
|
asymmetry
_EvalKing was the last piece type contributing to the old bvAttacks/
ATTACK_BITV mechanism -- both write sites (the low-enemy-material
early-exit path and the main king-safety loop) replaced with a single
pos->bbKingAttacks[uColor] |= g_KingAttacksBB[c], plus a new
_IsSquareAttackedByKing transitional helper (DEBUG-cross-checked, no
x-ray companion needed since a king can't move through a blocker) for
_WhoControlsSquareFast and _EvalKing's own bvDefend to read.
Two real, intentional behavior changes land with this conversion,
both discussed and confirmed before coding rather than assumed:
1. The main king-safety loop's old per-square write used
KingSafetyDeltas (11 entries: the 8 real king-move squares plus -2/
+2, two squares away on the same rank, present only for that loop's
own file-distance bookkeeping) instead of the real 8-square king
move pattern -- a bug, confirmed by direct instruction, not a
design choice worth preserving. Fixed by writing the real
g_KingAttacksBB[c] pattern once, before the loop, instead of
whatever KingSafetyDeltas happened to visit per-square.
2. _EvalKing's own bvAttack computation (does the *enemy* king attack
a square near this king?) is now symmetric where it used to be an
accidental artifact of evaluation order: kings are evaluated
black-then-white, so the old mailbox code let white's computation
see black's already-written king bit while black's could never see
white's (white hadn't run yet). Rather than preserve or upgrade
that asymmetry now that both colors go through an explicit helper
either way, neither side sees the enemy king as a threat here,
matching the side that already couldn't -- by direct instruction.
This is narrow in practice (two kings can never be legally
adjacent, so it only ever fires at king-vs-king distance 2) but
real.
_WhoControlsSquareFast's own king contribution is unaffected by either
change and stays fully symmetric: it's only ever called after *both*
kings finish evaluating (the passed-pawn re-check and trapped-piece/
danger passes all run after Eval()'s king-eval block), so
pos->bbKingAttacks is always fully populated for both colors by the
time it runs -- confirmed by checking call-site ordering directly, not
assumed from bvAttack's own (different) situation.
CountKingSafetyDefects is untouched by this change (confirmed by
reading it again): it's pure CHECK_VECTOR geometry over piece
locations, computed before any bbXAttacks accumulator exists this
Eval() call, and reads none of them. Its correlation with _EvalKing's
score is therefore preserved by construction, not something requiring
separate re-tuning.
Verification: since this bundles three attributable behavior changes
(the bitboard rewrite itself, the KingSafetyDeltas bug fix, and the
dropped bvAttack asymmetry), used the ringers-suite-plus-bounded-delta
bar from bishop's conversion rather than expecting exact node-count
equality: precommit_check.sh (self-test + DEBUG smoke test, plus
several hand-built king-adjacency/king-proximity positions run
directly against a DEBUG binary to exercise the new cross-check) all
pass; whole-engine tests/ecm_ringers.ep_ at sd10 vs. the immediately
preceding commit holds exact solve parity (10/11 both), with
per-position node-count deltas (-50% to +76%) all attributable to the
three documented changes above, no unexplained outliers.
ATTACK_BITV/bvAttacks itself is intentionally left in place -- nothing
writes it any more (king was the last writer), but the struct deletion
and the resulting simplification of the transitional
_IsSquareAttackedByX/_IsSquareXrayedByX helpers (which collapse to
plain bitboard reads once nothing can ever populate the old structure)
is staged as a deliberate follow-up commit, not bundled here.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01XxmVi2sTMwpPp4i6WYFjan
|
|
rook/bishop mobility to credit squares beyond a battery partner
_EvalQueen's mobility ray-walk and QMobCaseTable switch replaced with
two passes of the same unified chain-walk technique rook/bishop use --
_RookAttacksBB for the orthogonal-direction pass, _BishopAttacksBB for
the diagonal-direction pass (MOVEGEN_MIGRATION.md/EVAL.md already
established a combined 8-ray table measures slower than reusing the
rook/bishop tables separately, per the stashed first bitboard-eval
attempt's _EvalQueenOccupancyBB PoC -- reused that structure instead of
rediscovering the regression). Continue-set per pass: friendly queen in
both, friendly rook only in the rook-direction pass, friendly bishop
only in the bishop-direction pass -- QMOB_FRIEND_ROOK/_BISHOP's old
fOrthogonalRay flag is entirely subsumed by which lookup a blocker
shows up in. Queen never x-rays through any enemy piece (unlike
rook/bishop), so each pass's continue-set has no enemy side. Adds
pos->bbQueenAttacks[2]/bbQueenXrayAttacks[2] and the corresponding
_IsSquareAttackedByQueen/_IsSquareXrayedByQueen transitional helpers
(DEBUG-cross-checked against an independent mailbox walk, same pattern
as the rook/minor helpers), with _WhoControlsSquareFast and _EvalKing's
bvAttack/bvXray/bvDefend/uQueenNearKing computations updated to read
them instead of the old per-square bvAttacks bits queen no longer
writes.
Also fixes a real, previously-unnoticed fidelity gap in the
already-committed rook and bishop conversions: the old mailbox walk
doesn't just x-ray past a friendly battery partner (or, for rook/
bishop specifically, an x-rayable enemy queen/king) for attack-bit
purposes -- it keeps walking and keeps crediting *mobility* for
whatever safe/empty squares and further captures lie beyond, however
many such blockers are stacked on one ray. The originally-committed
bitboard versions only ever computed mobility from the near side (up
to the first blocker via the magic lookup) and treated the chain
purely as an attack-bit population exercise. Caught by direct
comparison against the pre-conversion mailbox binary on battery
positions (rook: two same-color rooks on an open file, old code
credited 13 mobility to one rook via squares beyond its companion vs.
9 for the near-side-only bitboard version; bishop: two same-color
bishops on one diagonal, 12/8 old vs. 11/5 near-side-only). Fixed by
merging each piece's mobility and x-ray-population computation into a
single chain-walk that credits mobility at every layer, not just the
first -- queen's own conversion was written with this fix in from the
start and verified against the same kind of battery position (two
queens on a file, queen+rook, queen+bishop) before landing.
Verified via precommit_check.sh (self-test + DEBUG smoke test, several
hand-built multi-piece-battery positions run directly against a DEBUG
binary to exercise the new cross-checks) after each step.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01XxmVi2sTMwpPp4i6WYFjan
|
|
bishop
_EvalRook's mobility ray-walk and RMobCaseTable switch replaced with
_RookAttacksBB(c, pos->bbOccupied) plus bitboard masks, same technique
as knight/bishop. Adds pos->bbRookAttacks[2]/bbRookXrayAttacks[2],
first contributors alongside bbMinorAttacks/bbMinorXrayAttacks.
Connected-rook bonus and x-ray population derived from the attack
bitboard instead of a per-square dispatch.
Unlike bishop's single-hop x-ray simplification, rook's x-ray
population uses a bounded chain-following loop (recompute with the
newly-found blocker excluded, repeat until no new x-ray-worthy
terminal appears): checked frequency first (board_representation/
EVAL.md), and 14% of the curated-suite positions have a genuine
2+-deep rook/queen battery on some ray, far more common than bishop's
~1% -- a one-hop approximation here would be a real fidelity loss, not
a negligible one. The stashed first bitboard-eval attempt (git
stash@{1}) had already solved this correctly by walking blocker-to-
blocker via bit-scan; this reproduces the same unbounded behavior via
repeated magic-lookup recomputation, cheap because the loop only
iterates again when an actual chained battery exists.
Backported the same bounded-chain fix to bishop's x-ray population
(previously single-hop only) for consistency, now that it's known
cheap and mechanically identical -- bishop's own battery rate is much
lower (~1%) so this mostly just removes an intentional divergence
rather than fixing an active problem.
Also fixes two real correctness gaps this conversion would otherwise
have introduced silently (same failure mode as pawn's earlier
conversion, EVAL.md's progress log): rook no longer writes ROOK_BIT/
ROOK_XRAY_BIT into the old per-square bvAttacks structure, but
_WhoControlsSquareFast and queen's mobility unsafe-check both still
read those bits directly. Added _IsSquareAttackedByRook/
_IsSquareXrayedByRook (same transitional-helper, DEBUG-cross-checked
pattern as the minor-piece helpers) and updated both call sites.
Verified via precommit_check.sh (self-test + DEBUG smoke test) after
each step.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01XxmVi2sTMwpPp4i6WYFjan
|
|
Records what's actually landed (pawns, bbOccupied, dispatch loop,
knight, bishop -- commits 57502d6/2ce3570/6e86450/7e3e6b6/de3f366),
corrects section 4's per-piece #define toggle strategy (not what was
actually used -- direct incremental rewrites with transitional
helpers instead), and documents methodology worth repeating:
- One piece type at a time, each landed and verified before the next
starts, learned the hard way after a combined knight+bishop+rook+
queen+king attempt produced a real bug that was hard to isolate
with five things changed at once and had to be reverted.
- Keep writing DEBUG asserts against the still-live rgSquare mailbox
representation as independent ground truth for as long as it
exists -- this is specifically what caught bishop's x-ray-chain
behavior gap during routine smoke testing.
- Verification bar changes once a step ships a deliberate behavior
change (bishop's x-ray simplification): full ecm_ringers solve
parity + bounded node-count deltas, not byte-identical counts.
- This effort errs on the side of speed over exact fidelity,
explicitly -- a real, recorded change from the original plan's
"must be byte-identical" bar.
- What's next: rook, then queen, then _EvalKing/_WhoControlsSquareFast,
culminating in deleting bvAttacks/ATTACK_BITV/the c|8 mechanism
entirely once nothing writes it anymore. Transitional helpers get
rewritten incrementally as each piece converts, not left to
accumulate special cases.
- Pawn hash hit/miss cost measurements (90 vs 1741 cycles) and the
EVAL_TIME per-term breakdown methodology, both worth reusing on
rook/queen once they land.
- Noted intent to revisit the non-mobility scoring terms in
_EvalKnight/_EvalBishop/_EvalRook/_EvalQueen once the attack-bits
work is done -- separate, later pass, not scoped further yet.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01P6g6iF6mD1Hau6nCZCzwYj
|
|
_EvalBishop's mobility ray-walk (per-square g_iBDeltas delta-walk +
BMobCaseTable switch dispatch) replaced with _BishopAttacksBB(c,
pos->bbOccupied) -- generate.c's magic-bitboard slider lookup,
already used by move generation -- plus bitboard masks for the
mobility count, matching knight's reduction pattern:
enemy-non-pawn-terminal counts unconditionally, empty-or-enemy-pawn-
terminal counts unless pawn-unsafe (via bbPawnAttacks), and the one
case that isn't a pure occupancy mask -- a terminal friendly,
non-stationary pawn on this bishop's own color complex -- keeps its
credit via bbPc (already computed for the existing good/bad
transient-pawn scoring, untouched this change).
Adds POSITION::bbMinorAttacks contributions from bishop (direct
attack bits) and POSITION::bbMinorXrayAttacks (new field) for squares
seen through a friendly bishop/queen battery partner or an
x-rayable enemy rook/queen/king.
Deliberate behavior change from the old ray-walk, made for speed per
direct instruction: the old walk's fStop=FALSE for the x-ray cases
meant it kept going -- and kept counting mobility -- through however
many x-ray-worthy blockers were stacked consecutively on one ray
(e.g. x-raying an enemy rook, then continuing to x-ray *through* an
enemy king sitting right behind it too). The bitboard version only
extends one hop past the first x-ray-worthy blocker; it does not
re-check whether the newly-revealed terminal square is itself
x-ray-worthy and extend again. This was found and deliberately kept
(not fixed) after a DEBUG assert caught the exact case on
8/1R1B4/2B1r3/5k2/2P2P2/1p6/1Kb5/7n w - - during precommit's random-
sample smoke test -- judged an acceptable trade given how rare a ray
with >=2 consecutive x-ray-worthy pieces is.
Two new transitional helpers (_IsSquareAttackedByMinor's bishop
contribution, and new _IsSquareXrayedByMinor) carry bishop's combined
state to every remaining consumer (rook/queen mobility-safety checks,
king's danger computation, _WhoControlsSquareFast) -- both DEBUG-
asserted against an independent from-scratch mailbox ray-walk (using
the still-live rgSquare representation, not any shared code with the
production bitboard technique) implementing this same single-hop
rule, so a real regression fails loudly rather than drifting into a
wrong score.
Verified via precommit_check.sh and the full tests/ecm_ringers.ep_
suite at sd10 against the prior commit (7e3e6b6): same 10/11 solved
(same single miss, ECM.335, in both), node counts now legitimately
differ per position (expected given the semantic change) but stay in
a bounded, reasonable range (-17% to +20%), nothing resembling the
60%+ blowups a real bug produced earlier in this same session before
being caught, isolated, and traced to this exact x-ray-chain gap.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01P6g6iF6mD1Hau6nCZCzwYj
|
|
_EvalKnight's mobility loop (per-square g_iNDeltas delta-walk +
NMobCaseTable switch dispatch) replaced entirely with
g_KnightAttacksBB[c] (generate.c's precomputed table, already used by
move generation) plus two bitboard masks -- knights never x-ray or
have a battery partner, so the old four-case table collapses to
"enemy non-pawn: count unconditionally" and "empty-or-enemy-pawn:
count unless pawn-unsafe."
Adds POSITION::bbMinorAttacks[2] (chess.h), the first of the
bbMinorAttacks/bbRookAttacks/bbQueenAttacks accumulators from
board_representation/EVAL.md section 2 -- knight ORs its full attack
set in directly, no per-square bit-scan needed. Bishop is not
converted yet and still writes its own minor-bit contribution into
the old per-square rgSquare[c|8].bvAttacks structure.
Since knight stopped writing that old structure, every consumer that
needs "does any minor attack this square" (rook/queen's mobility-
safety check, king's danger computation, _WhoControlsSquareFast) now
goes through a new transitional helper, _IsSquareAttackedByMinor,
which ORs the new bitboard (knight) with the old bvAttacks bit
(bishop) in exactly one place rather than each call site hand-rolling
its own combination -- this collapses to a plain bbMinorAttacks read
once bishop converts too, and the helper goes away entirely.
_IsSquareAttackedByMinor is DEBUG-asserted against an independent
recomputation (g_KnightAttacksBB / _BishopAttacksBB, both already-
trusted primitives from move generation, unrelated to either the old
ray-walk's bit bookkeeping or the new accumulator) so a bug in either
mechanism fails loudly in any DEBUG build/smoke-test run rather than
silently drifting into a wrong score deep in search.
Verified via precommit_check.sh and a direct before/after node-count
comparison (sd10, r1bq1rk1/pp2bppp/2n1pn2/2pp4/3P4/2NBPN2/PP3PPP/
R1BQ1RK1 w - - 0 1): byte-identical, 3125335 nodes both builds.
This landed as a deliberately small, single-piece-type step after an
earlier attempt to convert knight+bishop+rook+queen+king in one
combined change produced a real bug (a byte-scale mismatch in xray
bit reconstruction) that was hard to isolate with five things changed
at once, and was reverted back to 6e86450 rather than debugged
further. Bishop, rook, queen, and _EvalKing's own conversion are
follow-up steps, each to be landed and verified the same way, one at
a time.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01P6g6iF6mD1Hau6nCZCzwYj
|
|
The old loop walked cNonPawns[color][1..N] (a flat, arbitrarily-
ordered list mixing knight/bishop/rook/queen), doing a mailbox lookup
plus a p&0x4/IS_KNIGHT branch per piece to decide which _Eval*
function to call, then stashing rooks/queens into cDefer/uDefer
arrays to evaluate in a later pass -- a hard-to-predict branch per
piece on top of a mailbox read the callee already redoes for its own
ASSERT.
pos->bbPieces[color][PIECE_TYPE] already exists (incrementally
maintained by move.c) and was simply unused by Eval() until now.
Replaced the whole dispatch with a direct per-type bitboard walk:
knights, then bishops, for side-to-move, then the same for the other
side, then rooks both colors, then queens both colors -- same phase
order as before (load-bearing for bvAttacks accumulation), just
sourced from a bitboard instead of a mixed list + runtime type
dispatch. Retires the cDefer/uDefer bookkeeping entirely -- with each
type its own direct walk, "rooks after minors" is just "do that walk
after this one," nothing to defer.
Verified genuinely behavior-neutral, not just crash-free: built the
pre-change and post-change binaries side by side and diffed
--batch --command output against tests/ecm_ringers.ep_ at sd10. Node
counts, scores, and PVs are byte-for-byte identical across all 11
positions; only wall-clock time and NPS differ, consistently in the
new version's favor (~3-5% faster on this suite, e.g. 1088239 nps ->
1143141 nps, 1560092 -> 1616844). Also verified via the normal
precommit_check.sh gate.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01P6g6iF6mD1Hau6nCZCzwYj
|
|
First piece type converted per board_representation/EVAL.md section 2:
pawns no longer write into rgSquare[c|8].bvAttacks at all. Every
consumer of "does a pawn attack this square" now reads
pos->bbPawnAttacks[2] directly -- a plain bitboard, computed fresh
each Eval() call from pos->bbPawns[] via the same shift-and-mask
technique generate.c's _GenerateAllPawnMovesBB already uses (zero
per-pawn mailbox iteration, vs. up to 16 delta+IS_ON_BOARD checks
before). Knight/bishop/rook/queen/king are unchanged -- still
populate/read their own bvAttacks bits (uMinor/uRook/uQueen/uKing)
the old way until their own conversions land.
Consumer changes, all in eval.c:
- UNSAFE_FOR_MINOR retired as a macro (it only ever tested the pawn
bit) -- its 3 call sites (knight, bishop x2) now test
bbPawnAttacks directly.
- UNSAFE_FOR_ROOK/_QUEEN masks narrowed to drop the now-dead pawn
bit; call sites OR in an explicit bbPawnAttacks test alongside the
narrowed bvAttacks read.
- Two direct .small.uPawn reads (bishop's transient-pawn mobility
credit, bishop's defended-pawn bonus) switched to bbPawnAttacks
tests.
- _EvalKing's bvAttack/bvDefend (the real king-danger computation)
OR the bitboard bit back in at both read points, careful to
preserve the original ordering where bvDefend must reflect the
king's own just-set defend bit.
- _WhoControlsSquareFast (used by passer-race/trapped-piece/danger
code) ORs the bitboard bit back into its g_SwapTable index at the
same bit position PAWN_BIT always occupied.
- Two bugs caught by manually auditing every remaining |8 site after
the fact (not by any test failing): _EvalPawns' own pawn-duo and
backward-pawn detection read bvAttacks.uWholeThing at a point in
Eval()'s sequence where only pawns could have written it -- once
pawns stopped writing there, both checks went permanently dead
silently. Fixed to read bbPawnAttacks directly. No self-test caught
this; it's exactly the gap EVAL.md section 5's planned exact-score
harness is meant to close.
EVAL_TIME instrumentation: split _EvalPawns' cycle counter into
pawn-hash hit/miss buckets (chess.h/eval.c/root.c), answering whether
the hash is still worth it now that attack-bit population is nearly
free. Measured on one sd12 benchmark position: hits average 90.4
cycles, misses average 1741.3 cycles (~19x), 97.15% hit rate -- the
hash stays a clear win; the miss cost was never mostly attack-bit
population (that's a separate ~1% bucket now, down from ~3.6%), it's
the isolated/doubled/duo/backward-pawn scoring loops, which still do
real per-pawn work on a miss.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01P6g6iF6mD1Hau6nCZCzwYj
|
|
chess.h: add POSITION::bbOccupied (full-board occupancy, both colors,
every piece including kings), maintained incrementally alongside
bbPieces/bbPawns rather than rebuilt on demand -- resolves the open
question in EVAL.md section 0 about whether this is worth doing given
both generate.c and the planned eval.c mobility rewrite need it.
move.c: SlidePiece/SlidePawn/LiftPiece/PlacePiece and their
WithoutSigs variants now maintain bbOccupied at the same choke points
that already maintain bbPieces/bbPawns -- unconditionally, since
occupancy doesn't care about piece type or color. Kings only ever
move through SlidePiece/SlidePieceWithoutSigs (never Lift/Place), so
no separate king-specific update site was needed.
fen.c: populate bbOccupied when parsing a FEN.
board.c: VerifyPositionConsistency cross-checks pos->bbOccupied
against a from-scratch rebuild, same pattern already used for
bbPieces/bbPawns.
generate.c/movesup.c/see.c: replace call sites that rebuilt full
occupancy via _BuildFullOccupiedBB/_BuildOccupiedBB with direct reads
of pos->bbOccupied; delete see.c's _BuildOccupiedBB, which was a
byte-for-byte duplicate of generate.c's _BuildFullOccupiedBB (kept
only as the from-scratch ground truth for the new consistency check
and testgenerate.c's benchmark harness).
testsup.c: GenerateRandomLegalPosition builds POSITIONs by poking
rgSquare/bbPieces/bbPawns directly, bypassing both move.c and fen.c --
a third construction path the above missed. It never set bbOccupied,
so the new VerifyPositionConsistency check failed on every generated
position, and since generation retries until a position verifies, the
self-test suite spun forever (100% CPU, no progress) instead of
crashing outright. Fixed by setting bbOccupied at all four placement
sites (both kings, pawn, non-pawn piece). Full self-test suite and
precommit_check.sh verified clean afterward.
board_representation/EVAL.md: record the bbOccupied decision and
rationale, resolving section 0's open question.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01P6g6iF6mD1Hau6nCZCzwYj
|
|
redundant pawn-location bitboard
board_representation/EVAL.md: rewritten migration plan for a
bitboard-backed Eval() (mobility ray-walks + bvAttacks replacement),
plus a performance-philosophy section recording the profiling-first,
cut-aggressively-except-mobility/safety-awareness approach agreed on
this session, and findings on CountKingSafetyDefects' structural
inability to share bvAttacks-derived state with _EvalKing.
EVAL_TIME per-term instrumentation (chess.h/eval.c/root.c): breaks
the existing whole-Eval() cycle counter down by pawns/knight/bishop/
rook/queen/king, the always-paid pre-lazy-exit segment, and the
full-eval-only post-lazy segment, printed alongside the existing
"Avg. cpu cycles in eval" line. Diagnostic only (EVAL_TIME-gated),
no effect on the normal release profile.
Drop PAWN_HASH_ENTRY's bbPawnLocations[2]: it duplicated
POSITION's own incrementally-maintained bbPawns[2], rebuilt bit-by-bit
on every pawn-hash miss for no reason. eval.c now reads pos->bbPawns[]
directly; removed a stale per-iteration invariant assert in _EvalPawns
that only made sense when the bitboard was being built bit-by-bit in
that same loop.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01P6g6iF6mD1Hau6nCZCzwYj
|
|
default on
Implements the full board_representation/MOVEGEN_MIGRATION.md scope:
bitboard-backed generators for all six not-in-check piece types plus
the JumpTable-avoiding whole-node dispatch fork (_GenerateAllMovesBB),
the in-check escape path (king flight + block/capture), and
movesup.c's ExposesCheck/FasterExposesCheck/ExposesCheckEp/IsAttacked/
InCheck bitboard equivalents. Nine toggles total
(GENERATE_{KNIGHT,KING,ROOK,BISHOP,QUEEN,PAWN}_BITBOARD,
GENERATE_ESCAPES_{KING,BLOCK}_BITBOARD, EXPOSESCHECK_BITBOARD,
ISATTACKED_BITBOARD), all now on by default in GNUmakefile --
DISABLE_BITBOARD_MOVEGEN=1 opts back into the mailbox path, which
remains fully present and compiled either way.
Correctness verified via perft (Kiwipete, Position 4), the move-set
comparison harness across 20,000 random positions, all nine toggles
combined cleanly (15/15 runs, after fixing a GenerateRandomLegalPosition
en-passant-sentinel bug in the test harness), and sd10 on all three
curated suites showing zero solve-count regression vs head_reference
(the ecm_hard_quick delta traced to unrelated intervening commits).
Speed: most individual generators land near parity by design (mailbox's
per-square walk was already close to O(destination count)); the real,
consistent wins are the dispatch-layer fork (up to 23% in dense
positions) and IsAttackedBB (0.73x-0.93x of mailbox).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01AbHkVrm5KUyzLwWd3GHmo6
|
|
Board-representation migration section 4 item 4: seescores <filename>
(command.c, registered alongside script/sd) reads setboard lines from
an EPD file (same convention tests/ecm*.ep_ already use), generates
legal moves per position, and prints (FEN, SAN move, SEE value) for
every capture -- meant to be run once per GetAttacksBB toggle state
and diffed, closing a gap TestGetAttacks's attacker-list comparison
can't: whether _GetAttacksBB's attacker lists, though set-identical to
asm GetAttacks, still produce identical SEE() output once fed through
_MinLegalPiece's exchange simulation.
Run against all three curated suites (747 captures total: 51 + 351 +
345), same-commit asm-vs-GETATTACKS_BITBOARD=1 A/B build (same pair
used for the earlier sd10 comparison) -- output byte-identical, zero
diff, on all three. Confirms _GetAttacksBB is correct all the way
through to the final SEE() score every move-ordering decision actually
uses, not just at the raw attacker-list level.
precommit_check.sh clean. Default (no flag) release binary restored
after testing.
Only match_play.py remains unmet before full retirement of the old
mailbox GetAttacks implementation (section 7) -- deliberately deferred,
not run this pass.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Jntky4yGUTyQVaGCXms4F2
|
|
Board-representation migration section 4 item 5 / section 5 item 2 /
section 7: ran all three curated suites (ecm_ringers,
ecm_confident_quick, ecm_hard_quick) at sd10, same commit built twice
(default asm GetAttacks vs. GETATTACKS_BITBOARD=1), rather than against
the checked-in head_reference/ binary -- that binary predates this
branch's sections 1-6 by a dozen-plus unrelated commits, so diffing
against it would have conflated this change with everything else on
the branch. Isolating the single variable (same commit, one flag
flipped) is the correct comparison here.
Results: solve counts bit-identical on all three suites (10/11, 83/90,
25/90) -- zero regression. Node counts up slightly (+0.36% to +1.24%),
the expected "same attacker set, not necessarily same order" effect on
move-ordering tie-breaking already flagged in this document; didn't
cost a single solve. Aggregate (total nodes / total script time across
all three suites): +8.38% wall-clock NPS (1,164,842 -> 1,262,490),
comfortably absorbing the extra nodes -- confirms section 3's isolated
cycles/call benchmark reflects a real end-to-end win, not an artifact
of the isolated harness.
match_play.py (the remaining section 7 criterion) deliberately
deferred, not run this pass.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Jntky4yGUTyQVaGCXms4F2
|
|
Board-representation migration section 6: a GNUmakefile build flag
(-DGETATTACKS_BITBOARD) makes chess.h's GetAttacks macro resolve to
_GetAttacksBB instead of the real asm implementation (or SlowGetAttacks
under CROUTINES) -- a three-way choice at the same spot the existing
CROUTINES switch already lived. _GetAttacksBB is now reachable from
every real call site (generate.c's check-detection call, see.c's
SEE(), searchsup.c), not just the test/bench harness.
Found and fixed while verifying this: testsee.c's TestGetAttacks and
its benchmark call the identifier GetAttacks meaning "the real
asm/CROUTINES baseline" -- once the macro could resolve to
_GetAttacksBB, those calls would silently compare the new
implementation against itself, turning both the correctness sweep and
the benchmark into false-positive no-ops. Fixed with a local #undef
GetAttacks right after #include "chess.h" in testsee.c, so the harness
always validates against the true baseline regardless of which
implementation is live in production.
Verified: gmake TEST=1 GETATTACKS_BITBOARD=1 passes (self-test suite,
corrected benchmark still reporting real asm vs. _GetAttacksBB
correctly, and a real Search() call exercising _GetAttacksBB live).
precommit_check.sh GETATTACKS_BITBOARD=1 clean for both the TEST=1
self-test and DEBUG=1 smoke test. Default (no flag) build confirmed
unaffected -- GetAttacks still resolves to the real asm function.
See board_representation/MIGRATION.md section 6 for the full writeup.
Sections 4/5/7's remaining items (curated-suite sd10 comparison,
match_play.py gate) are now unblocked but not yet run.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Jntky4yGUTyQVaGCXms4F2
|
|
Drafted after GetAttacks's migration landed, to evaluate extending the
same bbPieces/bbPawns/ray-table substrate to generate.c's seven
piece-type move generators. Kept as a separate document from
MIGRATION.md rather than a new section there, same reasoning as
dropping CountKingSafetyDefects from that plan: this is a
substantially bigger, higher-risk surface (7 functions, ~3400 lines,
no existing reference implementation to diff against, and the
pseudo-legal over-generation contract is load-bearing -- a bitboard
rewrite that accidentally becomes more legal-aware is a silent
behavior change, not a free improvement).
Covers: per-function rollout plan (knight/king first as
lowest-risk/best-precedented, rook/bishop as the real segment-marking
design work, queen mechanical once those land, pawns last and
possibly not worth it), a stronger correctness gate than GetAttacks
had (perft node-count matching against externally-known-correct
numbers, not just internal self-consistency), and a confirmed (not
just flagged) scope gap: _GenerateEscapes, the in-check move
generation path, has its own independent mailbox implementation and
is not covered by the seven piece-type functions this plan targets.
No code changes -- planning only.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Jntky4yGUTyQVaGCXms4F2
|
|
Descope CountKingSafetyDefects entirely, per discussion after landing
GetAttacks's half of section 3: the two functions no longer share
enough to justify one plan. CountKingSafetyDefects (eval.c:2325) turns
out to do no ray-walk/blocker check at all -- it's an unblocked
CHECK_VECTOR proximity heuristic, not a true attack query -- so
_WhoAttacksSquareBB's blocker-aware result isn't a value-identical
drop-in for it; making it bitboard-backed would be a real behavior
change (needing eval re-tuning/re-gating), not a reimplementation, and
a materially different, riskier project than this one.
eval.c is untouched. If CountKingSafetyDefects work happens later, it
should be a new, separate migration document starting from its actual
(unblocked heuristic) behavior, not a resumption of this one.
Also updates section 4-7 status notes to reflect what's actually done
vs. still blocked on section 6's toggle (GetAttacks alone, no longer
entangled with a king-safety timeline).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Jntky4yGUTyQVaGCXms4F2
|
|
Board-representation migration, sections 2-3 (GetAttacks half):
- board.c: VerifyPositionConsistency's bbPieces consistency check
(migration section 2), verified clean via gmake TEST=1 with the
assert live.
- POSITION.bbPawns[2]: new incrementally-maintained per-color pawn
location bitboard (chess.h), maintained at the same 6 move.c sites
as bbPieces, populated from scratch in fen.c. Distinct from the
pawn-hash-keyed bbPawnLocations; this one needs no
SEARCHER_THREAD_CONTEXT, so it's reachable from GetAttacks's actual
call sites (which only ever have a POSITION*).
- data.c/chess.h/main.c: g_RookRayAll/g_BishopRayAll (all 4 per-square
ray directions pre-ORed) and g_PawnAttackOriginBB[2][128] startup
tables, plus FastFirstBit/FastLastBit (static inline bsf/bsr
wrappers, chess.h) -- supporting tables/helpers for the primitive
below.
- see.c: _WhoAttacksSquareBB (bitboard "who attacks square X" query)
and _GetAttacksBB (SEE_LIST-populating PoC wrapping it), side by
side with the existing SlowGetAttacks/asm GetAttacks -- not wired
into the GetAttacks macro yet (section 6), pure addition.
- testsee.c: SeeListsAreEqual made order-independent (SEE() sorts the
list right after GetAttacks returns, so order was never semantically
significant); TestGetAttacks extended to run _GetAttacksBB as a
third comparison across the existing 20,000-random-position sweep;
added an interleaved asm/Slow/BB cycles-per-call benchmark across
opening/middlegame/endgame positions.
- testsup.c: fixed GenerateRandomLegalPosition (used by the sweep
above) to maintain bbPieces/bbPawns at its two hand-placement sites
-- a latent gap since section 1 that made its own
VerifyPositionConsistency legality gate almost always reject
generated positions, causing large, variable retry-loop slowdowns.
Verified: 20,000-position x every-square x both-colors correctness
sweep passes (gmake TEST=1), precommit_check.sh clean (self-test +
DEBUG smoke test). Benchmark: _GetAttacksBB is ~0.53-0.55x asm
GetAttacks's cycles/call (opening/middlegame) and ~0.89x (endgame) --
faster, not just equivalent, primarily from replacing bbOccupied's
up-to-16-iteration pawn loop with two bbPawns ORs, plus a
g_PawnAttackOriginBB table lookup replacing per-call pawn-delta
arithmetic and per-direction/per-side-group early-outs in the slider
walk. See board_representation/MIGRATION.md section 3 for the full
writeup, including a reverted approach that measured slower and why,
and the CountKingSafetyDefects half's re-scoped (not yet implemented)
design.
Also confirmed (not caused by this work, not fixed here): a
pre-existing non-deterministic MP-race assertion in util.c:1093's PV
printing, reproduced independently on a clean HEAD checkout.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Jntky4yGUTyQVaGCXms4F2
|
|
Board-representation migration, section 1: add POSITION.bbPieces[2][8]
(per-color, per-piece-type location bitboards, indexed like the
existing uNonPawnCount) as incrementally-maintained state, not a
per-Eval()-call rebuild -- the structural fix for why the earlier
attack-presence-bitboard work measured slower, not faster.
- chess.h: bbPieces[2][8] field; extern decls for data.c's
g_RookRayToEdge/g_BishopRayToEdge/g_KnightAttacksBB ray tables and
their Initialize* functions (needed by the planned bitboard-backed
GetAttacks/CountKingSafetyDefects primitive, section 3).
- fen.c: populate bbPieces during piece placement; zeroing is free via
the existing memset(p, 0, sizeof(POSITION)).
- move.c: maintain bbPieces at all 6 non-pawn piece-movement functions
(SlidePiece/LiftPiece/PlacePiece and their WithoutSigs siblings used
by UnmakeMove) -- covers every move type: normal moves, captures,
both-side castling, promotion with/without capture, en passant, and
every undo.
- board.c: extend VerifyPositionConsistency's existing non-pawn
piece-list walk with a parallel bbPieces reconstruction-and-compare,
rather than a separate bespoke check.
- data.c/main.c: pulled ray-to-edge/knight-attack tables from stash
(needed by section 3, not section 1 itself, but zero-risk to land
now).
Also, while verifying: COOR_TO_BB was a table lookup (BBSQUARE[idx])
measured ~5-7% slower than the pure-ALU shift already sitting unused in
SLOWCOOR_TO_BB (whose "SLOW" name reflects a stale assumption about
variable shifts never actually tested on this hardware). Switched
COOR_TO_BB to the shift; fixed testbitboard.c's existing but broken
(dead-code-eliminated, silently reporting "0 cycles/op") comparison
benchmark for both while at it.
Verified via gmake TEST=1 (including TestMakeUnmakeMove's explicit
en-passant/promotion-with-capture/both-castling coverage) and
debug_smoke_test.sh, both clean; release build clean and runs normally.
Nothing reads bbPieces yet -- pure addition, zero behavioral risk.
See board_representation/MIGRATION.md for the full plan.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Jntky4yGUTyQVaGCXms4F2
|
|
bitboard helpers
- bitboard.c: CoorFromBitBoardRank1ToRank8 cleared the lowest set bit
unconditionally instead of the reported (highest) one, silently
mis-walking doubled-pawn files in eval.c's passed-pawn detection.
- search.c/searchsup.c: move GetLMRReduction's precondition checks from
inside the function to the caller in search.c (pre-existing work),
finishing the split with a matching gate in split.c's HelpSearch --
the parallel-search call site had no gate at all, letting it call
GetLMRReduction unconditionally (including for checking moves),
reachable only under real multithreading (--cpus > 1) and the
intermittent root cause of assertion crashes seen under --cpus 4.
- eval.c: redirect CountBits/CoorFromBitBoardRank8ToRank1/
CoorFromBitBoardRank1ToRank8 to inline compiler-builtin versions
(gated !CROUTINES) instead of the real out-of-line asm calls, on
eval.c's ~20 existing production call sites. CountBits' asm body
isn't O(1) popcnt, it's a Kernighan bit-clearing loop plus call
overhead, paid on every Eval() call.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Jntky4yGUTyQVaGCXms4F2
|
|
Pulled the parts of the stashed LMR work that are genuinely independent
of the reduction logic itself, leaving the actual LMR redesign for
separate review:
- Fix extension-taper table overflow: remove the flat MAX_EXTEND_PER_LINE
cap and instead clamp the depth used to build g_uExtensionReduction[]
so a deep `sd` request can't leave the whole taper table stuck at "0
penalty" (every index unreachable).
- Remove a spuriously-firing ASSERT(fMovesRescoredByIID) in Search():
RescoreMovesViaSearch's own fail-high branch deliberately leaves that
flag FALSE by contract, so the assert could fire on any DEBUG build
given an unlucky rescore, making the DEBUG/TEST harness unreliable.
- Misc correctness/portability fixes: unix.c pointer-truncation casts,
chess.h's CONTAINING_STRUCT/IS_ENPASSANT/ABS_DIFF macro hardening
(plus gating the branchless bit-tricks on _X64_ too, not just _X86_),
removal of dead Slide*WithoutSigs prototypes, main.c's hash default
bumped to 256m and its CPP self-test's arch gate widened to _X64_.
- eval_tune/match_play.py: cosmetic SPRT progress-bar/output rework.
- Delete eval_tune/run_ecm.sh (superseded, unreferenced elsewhere).
- run_tests.sh: parameterize suites/SD/SN via args/env vars instead of
hardcoding the three curated suites and sd10/sn5M (defaults kept
pointing at the existing curated suites, since the stash's own
lmr_sensitive_30/lmr_control_30 default suites aren't present in the
repo).
Deliberately left out of this commit: the stash's actual LMR reduction
logic, the M-SIGNAL-SHADOW diagnostic subsystem, the large PERF_COUNTERS
instrumentation buildout, the history-table gravity rework, and the
FindEnprisePiece pre-move staleness fix (skipped per request pending a
decision on whether to also change EFP's pruning behavior).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01MjdDfHry3i2jfJzyDXaG8A
|
|
Bundle the common build variable combos (MP+GENETIC, DEBUG+PERF_COUNTERS+GENETIC) with -j5 into single targets.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01XaWq66W3foZ2zpRmMerWW8
|
|
Search()'s Dieter-Brusser hash-hit-leads-to-draw check only verified
that a score of 0 would clear the same alpha/beta bound as the stored
iScore -- it didn't establish that iScore itself was accurate. Since
playing the hash move actually produces a draw, propagate the draw
score upward instead of the stale score computed along a different,
non-repeating path.
While fixing this, centralized every other place that returned a
literal 0 for a draw (search.c's stalemate leaf, searchsup.c's
QSearch draw leaf, probe.c's EGTB draw case, which had a dead
`// g_iDrawValue[...]` comment suggesting this was intended all
along) into a single g_iDrawScore[2] global in draw.c, declared in
chess.h. It's indexed by side to move rather than a scalar so a
future contempt-factor tweak can bias the draw score per color
without touching every call site again; both entries are currently 0,
so behavior is unchanged except for the hash-hit bugfix above.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012e2SaEaQ27JJq1D3wCqryr
|
|
Clean gmake GENETIC=1 PERF_COUNTERS=1 MP=1 SIXTYFOUR=1 build had 100
warnings; DEBUG=1 and TEST=1 builds had more once actually exercised.
- OFFSET_OF/CONTAINING_STRUCT (chess.h) and PTR_TO_ALLOC_HASH (unix.c)
truncated pointers through 32-bit ULONG before use in offset/hash
arithmetic on this 64-bit build -- routed through size_t instead.
- Diagnostic int<->void* round-trips (command.c, root.c, split.c,
sig.c, data.c, unix.c, util.c) widened/narrowed via size_t to avoid
implicit truncation.
- ABS_DIFF on unsigned COOR now casts to int before abs().
- Dropped -fexpensive-optimizations (GCC-only, clang silently ignores
it) from GNUmakefile.
- Removed genuinely dead variables (book.c, gamelist.c, split.c,
testgenerate.c, testhash.c).
- Guarded DEBUG/PERF_COUNTERS/_X86_-only variables and the
_CMEvidenceBucket helper under the #ifdef that actually reads them,
since ASSERT/EVAL_TERM/KEEP_TRACK_OF_FIRST_MOVE_FHs compile away
outside those builds.
- Added missing prototypes for SlidePawn, SlidePawnWithoutSigs,
SlidePieceWithoutSigs (move.c), previously undeclared in chess.h.
- Removed dead _SystemIsRoot (unix.c).
Verified via precommit_check.sh: TEST=1 self-test suite passes,
DEBUG=1 smoke test (10 random ECM positions, sd 4) passes with no
crashes/assertions, release build restored -- all three profiles now
build with zero warnings.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_014Cmv11sJZqVfanrPh6UnWE
|
|
|
|
|
|
Fix a bug (iEval) in HelpSearch
This commit changes the heuristics for: nullmove pruning, LMR, and EFP slightly.
|
|
|
|
dispatch.
The per-square mobility ray-walk called through a function pointer for
every square visited (piece type varies square to square, so the CPU's
indirect-branch predictor couldn't learn it), and nearly every handler
reduced to a couple of constant increments and a stop/continue flag.
Replaced each PMOBILITY_HELPER table with a small case-tag table and an
inlined switch, dropping all now-dead handler functions and the
PMOBILITY_HELPER typedef. Also hoisted the ray-invariant rank/file
direction test (rook's "connected" bonus, queen's bishop/rook xray
cases) out of the per-square switch to compute once per ray instead.
Verified semantics-preserving: node counts, solve/fail sets, and PVs
are byte-for-byte identical to head_reference across all three curated
ECM suites (ringers/confident_quick/hard_quick) at sd 10. bench shows a
consistent ~6-7% NPS improvement over two back-to-back runs.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012Q8wbZABsyG7PZQUd9WBAw
|
|
|
|
Eval() already detects trapped/attacked pieces (for search hints via
RecordEnprisePieceAtPly/RecordTrappedPiece) but never penalized them in
the static score. Add two named, DNA-visible constants: a larger flag
for the opponent-to-move/imminently-capturable case, a smaller one for
the own-move/still-might-escape case -- flat "this is bad" nudges, not
an attempt to price the material outcome, which search still owns.
Verified flat on ecm_ringers/ecm_confident_quick vs head_reference at
sd 10; ecm_hard_quick's lone flip (ECM.370) is a search-instability
artifact of that specific position (its true evaluation was still
moving through depth 14 in independent runs), not a real regression.
|
|
|
|
and Scott's overnight-SPRT shortcut (test_vs_head.sh); gitignore generated caches/PGN output.
tune_eval_dna.py isn't Texel-tuning-specific tooling anymore -- it's a
load-bearing dependency (match_play.py does `from tune_eval_dna import
Engine`), so it needs to be tracked for match_play.py to run at all on
a fresh checkout, independent of whatever happens to the rest of the
auto-tuning pipeline. filter_pgn.py (builds twic_filtered.pgn, the
pool match_play.py's --pgn points at) is similarly not tuning-specific.
test_vs_head.sh is Scott's shortcut for the overnight SPRT run
discussed this session (head_reference/typhoon vs. current build,
--games 20000 --workers 20 --st 1 --sprt --elo0 0 --elo1 5).
Left untracked, Texel-pipeline-specific and matching this session's
move away from auto-tuning: bake_dna.py, dna_diff.py, dna_trend.py,
cycle.sh, run_ecm.sh, compare_ecm_*.py, tuned.dna.
.gitignore: eval_tune/__pycache__/ and eval_tune/opening_cache/ (pure
regeneratable caches) and src/{match_games,self_play_games}.pgn
(match_play.py's --pgn-out game logs, generated output not source) --
the dna/first.dna / dna/original_baseline.dna accidental-commit from
earlier tonight was exactly this class of mistake, catching the
obvious repeat cases now.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01M9ZDiJhiUajUxh95mTXCFJ
|
|
the previous commit.
These were already staged in the index (not by anything in this
commit's own git add) when the previous commit ran, and git commit
without a pathspec commits the whole index, not just what was
explicitly added that call -- should have checked git status right
before committing. Old Texel-era baseline dumps (2026-08-24), unrelated
to tonight's work and contrary to this session's decision to ditch
Texel-based tuning for the hand-tuned baseline.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01M9ZDiJhiUajUxh95mTXCFJ
|
|
so it actually happens instead of being forgotten.
Refreshing head_reference/ (rebuild release binary, sd10+sn5m sweep
across all three curated suites, copy logs, tag the commit, archive
the binary) was, until now, a several-minute manual dance repeated by
hand every time a commit became the new comparison baseline -- exactly
the kind of multi-step ritual an AI assistant with no persistent
memory across sessions will reliably forget to fully repeat. Scripted
the mechanical parts: build, 6-way sweep, log copy, git tag (head-
reference-<date>, the durable/versioned source of truth for "which
commit was checkpoint N" -- see head_reference/README.md's new
"Checkpoint history convention" section), and binary archive (a
rebuild-avoidance cache alongside the tag, not a replacement for it).
Does NOT write the README's prose sections (what changed, why, how to
read the net score) -- that still needs a human/Claude actually
looking at the diff and the numbers this script prints at the end,
not a template trying to guess at them.
Checks for a dirty working tree and warns rather than silently tagging
something that doesn't correspond to a real commit -- the intended
sequence is still git commit first, then this script, same as
precommit_check.sh's gate runs before a commit rather than replacing
it.
precommit_check.sh: two lines printing the actual next-step commands
("git commit", "./update_head_reference.sh") after a passing check, so
the two scripts' relationship is visible right where the first one
succeeds.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01M9ZDiJhiUajUxh95mTXCFJ
|
|
EBF/NPS/first-move-beta stats, fix a completion-order bias in the live progress readout.
match_play.py previously compared one fixed binary with two loaded
evaldna files -- built for the Texel auto-tuning pipeline this project
has since moved away from in favor of hand-tuned constants baked
directly into eval.c. Converted to compare two separate compiled
binaries instead (head_engine/candidate_engine positional args, no
DNA loading at all) -- a real build is now required per side, but
that's the right model: what's being compared is two source trees,
not two parameter files loaded into an otherwise-identical process.
Found and deleted eval_tune/binary_match_play.py, a pre-existing (never
committed) sibling that already did binary-vs-binary comparison but
predates and is now strictly superseded by this rewrite (no SPRT, no
stats capture, and the same completion-order bias fixed below) --
keeping both would have left two overlapping tools to drift out of
sync, the same duplication problem this session spent all night
removing from eval.c itself.
Added a real SPRT (Sequential Probability Ratio Test), the same
formulation fishtest/cutechess-cli use: two Elo hypotheses (H0/H1)
tested via the log-likelihood ratio of a normal approximation to the
per-game trinomial (W/D/L) score, variance re-estimated from the
running W/D/L mix as games accumulate. Verified against a Monte Carlo
simulation before landing: correctly resolves H0 at true_elo=0 (~6-19k
games) and H1 at game counts matching the theoretical fixed-N table
almost exactly (30 Elo: ~1.4-2k, 20 Elo: ~2.3-4.5k, 10 Elo: ~5-9k, 5
Elo: ~10-25k). Required restructuring the game scheduler from
"submit everything upfront, as_completed" to a bounded rolling window
(at most --workers games in flight) so it can actually stop early
once SPRT concludes instead of having thousands of already-launched
futures it can't usefully cancel.
Fixed a real, previously-unnoticed bug that explains a specific
observed symptom (candidate consistently scoring high for the first
~500 games of a 1000-game run, then eroding toward 0.5 -- every run,
same direction, which is what tipped this off as systematic rather
than noise): the live "score so far" readout processed games in
*completion* order (as_completed), not submission order, and decisive
games plausibly finish faster than grindy draws/losses (a winning side
wraps up before --max-plies; a losing/drawing side often runs long).
That means candidate wins arrive disproportionately early and the live
average was a biased mid-run estimator -- high at first, eroding as
slower non-win games trickle in. The *final* score was never actually
wrong (order-independent, sums the same regardless of arrival order),
just the progress narrative watched live. Fixed by buffering
out-of-order completions and only advancing the printed running score
through games in their original submission order.
Also fixed --sd defaulting to 8 even when --st was passed -- the
argparse mutually-exclusive group only stops both flags being given
together, it does nothing about one flag's default silently applying
when only the other was specified. --st alone was being silently
ignored in favor of sd=8 the whole time. Now --sd defaults to None and
only falls back to 8 when neither --sd nor --st is given.
Added EngineStats: per-engine (not per-color, since candidate/baseline
swap sides every other game) speed and tree-shape capture -- NPS, EBF
(nodes**(1/depth) per move, same formula script.c's suite runs use),
and first-move-beta-cutoff-rate, all pulled from the same
PostMoveSearchReport block every move already prints (confirmed this
prints after EVERY move, not just script-run suite summaries, and
confirmed the "move" line prints BEFORE the stats block against
root.c, not after -- the read loop needed restructuring to keep
reading past the move line rather than stopping on it). No separate
benchmark pass needed; these come from the same games already being
played for the strength comparison.
Deliberately not built tonight, flagged as a possible follow-up: an
opening-pool independence concern (sample_openings picks random byte
offsets into the TWIC pool, which could pull duplicate/correlated
lines if TWIC has many games following the same trendy opening in a
season -- would make the standard-error math slightly overconfident,
not wrong in direction).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01M9ZDiJhiUajUxh95mTXCFJ
|
|
real king-safety bug found along the way.
A full-file pass over eval.c hunting for the "positional terms too
loud" feedback from real chess programmers, following the concrete
finding that Crafty prices most structural themes through one term
where this codebase spread the same theme across several (passed
pawns alone via 5-6 separate additive terms that can all fire for one
pawn). Two categories of fix, applied per the rule "if it's counting
the same thing twice, kill it; if it's a genuinely different angle on
the same theme, turn it down rather than remove it":
Pawns (pawn-hash cached, so free regardless of term count -- these are
data/magnitude fixes, not perf fixes):
- Removed ISOLATED_PAWN_PENALTY_BY_COUNT, a whole-position aggregate
that re-priced the same uIsolated[] count already reflected by
summing the per-pawn isolated term once per isolated pawn -- an
exact duplicate, not a different angle.
- CANDIDATE_PASSER_BY_RANK's "in endgame" bonus used to add the
exact same value a second time (a literal clone of the term just
added above it); now a /2 fractional modifier.
- CONNECTED_PASSERS_BY_RANK / SUPPORTED_PASSER_BY_RANK /
OUTSIDE_PASSER_BY_DISTANCE scaled to ~1/3 magnitude: each prices a
genuinely distinct angle on "how good is this passer" (connected
to a partner, pawn-defended, outside the opposing majority) and
can stack for the same pawn, so turned down rather than removed.
- ISOLATED_DOUBLED_PAWN turned down (-11 -> -5): a per-pawn kicker
that stacks with the whole-position DOUBLED_PAWN_PENALTY_BY_COUNT
aggregate for the isolated+doubled subset -- different angle
(single-worst-case flag vs. whole-position severity), not a
duplicate, but a real overlap worth trimming.
Pieces (non-cached, real per-node cost, so these are also legibility/
perf fixes, not just magnitude):
- Bishop: cut BISHOP_IN_CLOSED_POSITION outright -- it duplicated
bishop mobility rather than adding a distinct angle (mobility
already measures per-bishop diagonal blockage directly and more
precisely than a coarse whole-board proxy).
- Knight: killed a stale "don't block unmoved E2/D2 pawns" TODO
(opening-book territory, not eval's job) and "a knight with an
open file behind it is good" (dubious chess reasoning reusing an
unrelated table -- the same lookup as the backward-pawn-blockade
bonus, for a completely different concept).
- Rook: killed ROOK_TRAPPING_EKING (a rook on the 7th/8th aligned
with the enemy king is exactly the geometric pattern
CountKingSafetyDefects' CHECK_VECTOR scan already folds into
uPiecesPointingAtKing -- belongs in king safety, not a rook-
specific bolt-on). Also removed pFriendRook, dead in the same
block.
- Queen: killed "pointing near enemy K" (QUEEN_ATTACKS_SQ_NEXT_TO_
KING) -- computed from the queen's own mobility ray-cast, direct-
attacks only, duplicating what _EvalKing's real (non-lazy-estimate)
danger computation already reads from the identical attack-table
bits a few lines away.
- cTrapped fixed from a single COOR per color to a small [2][4] list
(_RecordTrappedCandidate): the old single-slot design let a later
piece's zero-mobility candidacy silently overwrite an earlier
one's on the same side, discarding a genuinely trapped-and-
attacked piece. This fed into search too (RecordTrappedPiece's
move-ordering hint), not just eval scoring. RecordTrappedPiece's
own per-ply single slot is left alone per design (would double the
cost on the branch that already computes it, this is the
innermost eval loop) -- now reports the MOST VALUABLE of the
candidates found, not just whichever was found last.
King (the actual regression-and-recovery of this session):
- Cutting the queen's "pointing near enemy K" term initially cost
real solves (117->110 on the sd10 curated suites) despite being a
correct duplication kill -- the general king-safety loop's
per-square attacker accounting was piece-type-blind (a queen
attacking a square near the king counted the same as a knight
doing the same geometric thing), so removing the one place that
priced queen-specific severity lost real fidelity, not just a
duplicate. Fixed properly: added KING_QUEEN_PROXIMITY_DANGER,
computed from bvAttacks[...].small.uQueen / .xray.uQueen bits the
king-safety loop already reads for every one of its 11 squares --
free (no new attack-table work) and more accurate than the killed
term (catches x-ray/latent queen threats it never did). Calibrated
against the killed term's own empirical magnitude (uNearKing * 8,
capped at 6) rather than guessed. Recovered to 118/191 (a new
session-best), now with the fidelity gap actually closed instead
of just removed.
- KING_SUPPORTING_OWN_PASSER_BY_RANK split out from
SUPPORTED_PASSER_BY_RANK, which _EvalKing's "kings in front of
passers" endgame bonus was silently reusing -- pawn-support and
king-escort are different concepts (fires when the KING stands
next to its own passer, not when a pawn does); scaling the shared
table down for its real purpose was silently also scaling the
unrelated king-escort bonus. Seeded with the table's original
(pre-scaling) hand-tuned magnitude.
- Collapsed three copy-pasted file-scan blocks (c-1/c/c+1, identical
logic repeated three times) into one loop -- confirmed
behaviorally neutral by isolated sd10 suite testing before
landing alongside the king-safety content changes.
Net result across the three curated suites (sd10, vs. the hand-tuned+
bugfix baseline this built on): 117 -> 118, a new session best, with
every intermediate checkpoint tested via EVAL_DUMP verification +
precommit_check.sh + sd10 sweep before moving to the next change.
Deliberately deferred, written down for a future session rather than
attempted here: a holistic king-safety overhaul (the piece-type-
tropism inconsistency across knight/bishop/queen/the general
CountKingSafetyDefects scan goes deeper than tonight's scoped fixes),
recalibrating EstimatePositionalScore's iKingSwingP90 lazy-eval margin
table (the instrumentation that built it no longer exists in this
tree, and today's changes have already shifted the true swing
distribution it was calibrated against), and training a small king-
danger classifier from TWIC checkmate games (snapshot king safety
features at -10/-15 moves from real checkmates, not resignations) to
calibrate whichever of the above happens first.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01M9ZDiJhiUajUxh95mTXCFJ
|
|
|
|
QSearch mate-magnitude asserts, MATEMOVE PV display, --command truncation, and test.sh's stale egtbpath.
Several small, independent correctness fixes bundled together since
they were all exercised together through today's precommit_check.sh
and curated-suite runs:
- command.c: batch-mode's "Exhausted input" exit was exit(-1), which
truncates to 255 (an 8-bit status) and is indistinguishable from a
real crash's nonzero exit. Changed to exit(0) so debug_smoke_test.sh
can reliably tell a clean batch run apart from a crash by exit status
alone.
- dynamic.c: _NewKillerMove's slot[1] backfill from
mvNullmoveQuietRefutations[uPly] had no check that the backfilled
move differed from the move just placed in slot[0]. When they
coincided, both slots held the identical move, silently wasting a
killer slot in release builds (ASSERT is a no-op there) and tripping
_NewKillerMove's own IS_SAME_MOVE invariant in DEBUG builds. Fixed by
skipping the backfill on collision.
- search.c: removed two ASSERT(iBestScore > -NMATE) calls in QSearch
that encoded an invariant that isn't actually guaranteed -- at an
early full-width root iteration, or after aspiration-window widening
following repeated fail-highs, an ancestor frame's iAlpha/iBeta can
itself already be more extreme than -NMATE with no mate anywhere in
the line, so a legitimate fail-low placeholder or fail-high score can
land in mate-magnitude territory purely as a window artifact.
hash.c's storage path already treats any value <= -NMATE as a sound
upper bound regardless of origin, so this was a false invariant, not
a caught bug. Also: minor whitespace cleanup, an added ASSERT
documenting the futility-margin depth precondition it replaced a
redundant runtime check for, and PV/leaf-count bookkeeping on the
mate/draw-at-root leaf paths that was previously skipped.
- util.c: MATEMOVE sentinel moves weren't handled in PV-to-string
conversion, so a PV ending in a detected mate would either display
garbage or hit the same-move assert. Added an explicit "<#>" marker.
- test.sh: --egtbpath pointed at a nonexistent /egtb/three;/egtb/four;
/egtb/five; corrected to /zscratch/egtb, this box's actual EGTB
location.
- main.c/input.c: --command's initial-command buffer (g_szInitialCommand)
was a fixed 256-byte array; strncpy(..., SMALL_STRING_LEN_CHAR - 2)
silently truncated any longer --command string, and -- worse -- when
the source was long enough not to fit, strncpy doesn't null-terminate
the destination, so the immediately-following strcat(..., "\r\n") could
read/write past the buffer. Long move-replay command strings used
during this session's debugging hit the truncation directly (a ~600
char move list silently cut off mid-token, desyncing the input queue).
Changed g_szInitialCommand to a heap allocation sized to the actual
input length instead of a fixed cap.
- CLAUDE.md: documents the above (this file's own diff is prior
session's writeup of these same fixes, committed now alongside the
code).
All exercised together via precommit_check.sh (self-test suite + DEBUG
smoke test against random ecm.ep_ samples) and the sd10/sn5m curated
suite sweep run for the eval.c hand-tuning commit just before this one.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01M9ZDiJhiUajUxh95mTXCFJ
|
|
a stale king-safety data bug found along the way.
The Texel/coordinate-descent auto-tuning pass (started at 29d73f4) left
several eval terms with non-monotonic or outright sign-flipped values
that several ASSERTs had to be silently commented out to tolerate
(e.g. BACKWARD_SHIELDED_BY_LOCATION scoring a structural pawn defect as
a +12..+17 bonus on most squares, PASSER_BONUS_AS_MATERIAL_COMES_OFF
staying flat until the defending side was down to almost nothing).
Restored all 54 differing constant tables to their last hand-tuned
values (commit df8facc, pre-dating 29d73f4) mechanically -- table
names/shapes are identical between the two commits, only values
differ, so this is a pure data restore with none of the surrounding
code-structure changes since df8facc reverted.
Also fixes a real bug found while investigating: pos->uPiecesPointingAtKing[]
was only refreshed inside EstimatePositionalScore's lazy-eval-margin
path (eval.c ~5648), but _EvalKing reads it unconditionally on every
full eval. Whenever a node's cheap material+pawn score wasn't close
enough to the alpha/beta window to trigger that lazy-margin branch, the
full eval proceeded straight to _EvalKing using a stale
uPiecesPointingAtKing value left over from a prior, unrelated node --
silent, intermittent noise in king-safety scoring on an unpredictable
subset of evaluations. Introduced 2026-08-24/26 (29d73f4, 7857096), so
it predates and was baked into the Texel tuning pass being reverted
here. Fixed by computing it once in an else branch when the lazy-margin
path isn't taken, so it's refreshed exactly once per full eval either
way (this is the innermost eval loop, so avoided doubling the cost on
the branch that already computes it).
sd10/sn5m results across the three curated suites (vs. head_reference,
the prior Texel-tuned HEAD):
sd10: ringers 9/11 (was 10), confident 85/90 (was 88), hard 23/90 (was 17) -- total 117 vs 115
sn5m: ringers 11/11 (was 10), confident 87/90 (was 89), hard 14/90 (was 13) -- total 112 vs 112
Net win at sd10, wash at sn5m, in both cases with a large swing toward
ecm_hard_quick -- consistent with hand-tuned values being more
internally coherent (monotonic curves, no sign flips, no double-counted
whole-position aggregates layered on top of already-summed per-item
terms) even though they were never retuned against this specific suite
or these specific opponents.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01M9ZDiJhiUajUxh95mTXCFJ
|
|
countermove promotion, retired hung-piece-escape and NumLeftoverMovesToSelect.
Full session was built on a "measure the pick, not the game" methodology:
aggregate solve counts on curated suites are too noisy to tune move-ordering
knobs against, so most decisions here came from per-move fail-high/alpha-raise
rates at much larger sample sizes (leftover FH% instrumentation, a zero-
selection-budget diagnostic that isolates a single best-of-remaining pick,
and evidence-bucket calibration), not solve-count deltas alone. See
CLAUDE.md's "Dynamic move ordering experiments" section for the reusable
methodology and generate.c's _ScoreAllMoves comment for the resulting
ordering hierarchy.
Changes:
- Added g_ContinuationHistory: same growth/decay math as the existing
g_HistoryCounters butterfly table, additionally keyed by the previous
move, so its magnitude is self-calibrated rather than a hand-picked
constant. Flat, sufficient response across a 256x scale sweep.
- Countermove-table matches now get a real GOOD_MOVE-tier promotion
(previously the table was write-only, tracked for stats but never read
for ordering), but only when the match's own accumulated
history+continuation evidence clears COUNTERMOVE_EVIDENCE_THRESHOLD
(10,000) -- a raw match with no track record was shown to perform
identically to an ordinary leftover (~0.6-0.85% FH), so promoting on
match alone would have repeated hung-piece-escape's mistake below.
- Retired hung-piece-escape's unconditional GOOD_MOVE-tier promotion.
Evidence-calibration showed the overwhelming majority of triggers (a
zero-evidence population 250-1000x larger than countermove's) performed
at the plain-leftover baseline -- the promotion was mostly free tier-
escape treatment for moves that hadn't earned it. Replaced with
FLEE_BONUS, a flat same-tier nudge inside SelectBestWithHistory (never
escapes GOOD_MOVE/leftover classification, unlike a generation-time
promotion) at the magnitude found to plateau a same-tier-nudge sweep.
- Retired NumLeftoverMovesToSelect (the depth-indexed budget on how many
leftover moves got a full selection scan before falling back to
unsorted order). search.c's main move loop now always fully selects --
the leftover pool was shown to contain real, findable signal a bailout
budget was discarding for a node-count savings that didn't hold up net-
net once measured by solve counts and fail-high rates rather than raw
node counts (noisy on small suites independent of this change).
- Collapsed leftover-move instrumentation from sorted/raw pairs down to a
single set now that "raw" (unsorted fallback) is structurally
impossible; kept the countermove evidence-bucket calibration counters
(ongoing check that COUNTERMOVE_EVIDENCE_THRESHOLD stays well-
calibrated); removed the contested-node A/B harness and hung-piece
evidence calibration now that the decisions they were built to inform
are made.
Net effect on the three curated suites (sd 10): solve counts wash (tied,
+1, -1 across ringers/confident/hard), leftover fail-high rate improved
consistently on all three (the intended, directly-measured target of this
work). Not yet validated beyond sd 10 -- an sn-based run or
eval_tune/match_play.py head-to-head gate is the natural next check before
leaning on this as a proven strength gain rather than a directionally-
sound, sd-10-clean change.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_014XePz6Sk4qQsTaP2jVJWJu
|
|
trace it at startup alongside the build timestamp.
A --logfile trace could previously only be tied back to a build
timestamp, not the exact source state -- distinguishing same-day
rebuilds during A/B testing required diffing binaries. GIT_COMMIT is
injected via GNUmakefile (git rev-parse --short HEAD, kept out of the
PROFILE variable itself since PROFILE gets separately stringified
whole for the "Make profile used" trace line, and this value's
embedded quotes broke that outer string literal when first tried
folded in there).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EortUUkDVpsfrbqshBJYJg
|
