typhoon.git/src/board_representation, branch master

Update EVAL.md: progress log, lessons learned, corrected next steps

2026-09-05T09:22:58Z

Records what's actually landed (pawns, bbOccupied, dispatch loop, knight, bishop -- commits 57502d6/2ce3570/6e86450/7e3e6b6/de3f366), corrects section 4's per-piece #define toggle strategy (not what was actually used -- direct incremental rewrites with transitional helpers instead), and documents methodology worth repeating: - One piece type at a time, each landed and verified before the next starts, learned the hard way after a combined knight+bishop+rook+ queen+king attempt produced a real bug that was hard to isolate with five things changed at once and had to be reverted. - Keep writing DEBUG asserts against the still-live rgSquare mailbox representation as independent ground truth for as long as it exists -- this is specifically what caught bishop's x-ray-chain behavior gap during routine smoke testing. - Verification bar changes once a step ships a deliberate behavior change (bishop's x-ray simplification): full ecm_ringers solve parity + bounded node-count deltas, not byte-identical counts. - This effort errs on the side of speed over exact fidelity, explicitly -- a real, recorded change from the original plan's "must be byte-identical" bar. - What's next: rook, then queen, then _EvalKing/_WhoControlsSquareFast, culminating in deleting bvAttacks/ATTACK_BITV/the c|8 mechanism entirely once nothing writes it anymore. Transitional helpers get rewritten incrementally as each piece converts, not left to accumulate special cases. - Pawn hash hit/miss cost measurements (90 vs 1741 cycles) and the EVAL_TIME per-term breakdown methodology, both worth reusing on rook/queen once they land. - Noted intent to revisit the non-mobility scoring terms in _EvalKnight/_EvalBishop/_EvalRook/_EvalQueen once the attack-bits work is done -- separate, later pass, not scoped further yet. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01P6g6iF6mD1Hau6nCZCzwYj

Maintain pos->bbOccupied incrementally, dedupe its two from-scratch builders

2026-09-05T07:19:59Z

chess.h: add POSITION::bbOccupied (full-board occupancy, both colors, every piece including kings), maintained incrementally alongside bbPieces/bbPawns rather than rebuilt on demand -- resolves the open question in EVAL.md section 0 about whether this is worth doing given both generate.c and the planned eval.c mobility rewrite need it. move.c: SlidePiece/SlidePawn/LiftPiece/PlacePiece and their WithoutSigs variants now maintain bbOccupied at the same choke points that already maintain bbPieces/bbPawns -- unconditionally, since occupancy doesn't care about piece type or color. Kings only ever move through SlidePiece/SlidePieceWithoutSigs (never Lift/Place), so no separate king-specific update site was needed. fen.c: populate bbOccupied when parsing a FEN. board.c: VerifyPositionConsistency cross-checks pos->bbOccupied against a from-scratch rebuild, same pattern already used for bbPieces/bbPawns. generate.c/movesup.c/see.c: replace call sites that rebuilt full occupancy via _BuildFullOccupiedBB/_BuildOccupiedBB with direct reads of pos->bbOccupied; delete see.c's _BuildOccupiedBB, which was a byte-for-byte duplicate of generate.c's _BuildFullOccupiedBB (kept only as the from-scratch ground truth for the new consistency check and testgenerate.c's benchmark harness). testsup.c: GenerateRandomLegalPosition builds POSITIONs by poking rgSquare/bbPieces/bbPawns directly, bypassing both move.c and fen.c -- a third construction path the above missed. It never set bbOccupied, so the new VerifyPositionConsistency check failed on every generated position, and since generation retries until a position verifies, the self-test suite spun forever (100% CPU, no progress) instead of crashing outright. Fixed by setting bbOccupied at all four placement sites (both kings, pawn, non-pawn piece). Full self-test suite and precommit_check.sh verified clean afterward. board_representation/EVAL.md: record the bbOccupied decision and rationale, resolving section 0's open question. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01P6g6iF6mD1Hau6nCZCzwYj

Add Eval() bitboard-migration plan, per-term EVAL_TIME cycle breakdown, drop redundant pawn-location bitboard

2026-09-05T06:54:51Z

board_representation/EVAL.md: rewritten migration plan for a bitboard-backed Eval() (mobility ray-walks + bvAttacks replacement), plus a performance-philosophy section recording the profiling-first, cut-aggressively-except-mobility/safety-awareness approach agreed on this session, and findings on CountKingSafetyDefects' structural inability to share bvAttacks-derived state with _EvalKing. EVAL_TIME per-term instrumentation (chess.h/eval.c/root.c): breaks the existing whole-Eval() cycle counter down by pawns/knight/bishop/ rook/queen/king, the always-paid pre-lazy-exit segment, and the full-eval-only post-lazy segment, printed alongside the existing "Avg. cpu cycles in eval" line. Diagnostic only (EVAL_TIME-gated), no effect on the normal release profile. Drop PAWN_HASH_ENTRY's bbPawnLocations[2]: it duplicated POSITION's own incrementally-maintained bbPawns[2], rebuilt bit-by-bit on every pawn-hash miss for no reason. eval.c now reads pos->bbPawns[] directly; removed a stale per-iteration invariant assert in _EvalPawns that only made sense when the bitboard was being built bit-by-bit in that same loop. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01P6g6iF6mD1Hau6nCZCzwYj

Land bitboard move generation (Part A+B) and movesup.c bitboard queries; default on

2026-09-04T23:46:27Z

Implements the full board_representation/MOVEGEN_MIGRATION.md scope: bitboard-backed generators for all six not-in-check piece types plus the JumpTable-avoiding whole-node dispatch fork (_GenerateAllMovesBB), the in-check escape path (king flight + block/capture), and movesup.c's ExposesCheck/FasterExposesCheck/ExposesCheckEp/IsAttacked/ InCheck bitboard equivalents. Nine toggles total (GENERATE_{KNIGHT,KING,ROOK,BISHOP,QUEEN,PAWN}_BITBOARD, GENERATE_ESCAPES_{KING,BLOCK}_BITBOARD, EXPOSESCHECK_BITBOARD, ISATTACKED_BITBOARD), all now on by default in GNUmakefile -- DISABLE_BITBOARD_MOVEGEN=1 opts back into the mailbox path, which remains fully present and compiled either way. Correctness verified via perft (Kiwipete, Position 4), the move-set comparison harness across 20,000 random positions, all nine toggles combined cleanly (15/15 runs, after fixing a GenerateRandomLegalPosition en-passant-sentinel bug in the test harness), and sd10 on all three curated suites showing zero solve-count regression vs head_reference (the ecm_hard_quick delta traced to unrelated intervening commits). Speed: most individual generators land near parity by design (mailbox's per-square walk was already close to O(destination count)); the real, consistent wins are the dispatch-layer fork (up to 23% in dense positions) and IsAttackedBB (0.73x-0.93x of mailbox). Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01AbHkVrm5KUyzLwWd3GHmo6

Add seescores diagnostic command, verify zero SEE-value drift

2026-09-04T16:55:15Z

Board-representation migration section 4 item 4: seescores (command.c, registered alongside script/sd) reads setboard lines from an EPD file (same convention tests/ecm*.ep_ already use), generates legal moves per position, and prints (FEN, SAN move, SEE value) for every capture -- meant to be run once per GetAttacksBB toggle state and diffed, closing a gap TestGetAttacks's attacker-list comparison can't: whether _GetAttacksBB's attacker lists, though set-identical to asm GetAttacks, still produce identical SEE() output once fed through _MinLegalPiece's exchange simulation. Run against all three curated suites (747 captures total: 51 + 351 + 345), same-commit asm-vs-GETATTACKS_BITBOARD=1 A/B build (same pair used for the earlier sd10 comparison) -- output byte-identical, zero diff, on all three. Confirms _GetAttacksBB is correct all the way through to the final SEE() score every move-ordering decision actually uses, not just at the raw attacker-list level. precommit_check.sh clean. Default (no flag) release binary restored after testing. Only match_play.py remains unmet before full retirement of the old mailbox GetAttacks implementation (section 7) -- deliberately deferred, not run this pass. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01Jntky4yGUTyQVaGCXms4F2

Record whole-engine sd10 results: zero solve regression, +8.38% NPS

2026-09-04T16:50:57Z

Board-representation migration section 4 item 5 / section 5 item 2 / section 7: ran all three curated suites (ecm_ringers, ecm_confident_quick, ecm_hard_quick) at sd10, same commit built twice (default asm GetAttacks vs. GETATTACKS_BITBOARD=1), rather than against the checked-in head_reference/ binary -- that binary predates this branch's sections 1-6 by a dozen-plus unrelated commits, so diffing against it would have conflated this change with everything else on the branch. Isolating the single variable (same commit, one flag flipped) is the correct comparison here. Results: solve counts bit-identical on all three suites (10/11, 83/90, 25/90) -- zero regression. Node counts up slightly (+0.36% to +1.24%), the expected "same attacker set, not necessarily same order" effect on move-ordering tie-breaking already flagged in this document; didn't cost a single solve. Aggregate (total nodes / total script time across all three suites): +8.38% wall-clock NPS (1,164,842 -> 1,262,490), comfortably absorbing the extra nodes -- confirms section 3's isolated cycles/call benchmark reflects a real end-to-end win, not an artifact of the isolated harness. match_play.py (the remaining section 7 criterion) deliberately deferred, not run this pass. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01Jntky4yGUTyQVaGCXms4F2

Add GETATTACKS_BITBOARD toggle, wiring _GetAttacksBB into real search

2026-09-04T16:37:45Z

Board-representation migration section 6: a GNUmakefile build flag (-DGETATTACKS_BITBOARD) makes chess.h's GetAttacks macro resolve to _GetAttacksBB instead of the real asm implementation (or SlowGetAttacks under CROUTINES) -- a three-way choice at the same spot the existing CROUTINES switch already lived. _GetAttacksBB is now reachable from every real call site (generate.c's check-detection call, see.c's SEE(), searchsup.c), not just the test/bench harness. Found and fixed while verifying this: testsee.c's TestGetAttacks and its benchmark call the identifier GetAttacks meaning "the real asm/CROUTINES baseline" -- once the macro could resolve to _GetAttacksBB, those calls would silently compare the new implementation against itself, turning both the correctness sweep and the benchmark into false-positive no-ops. Fixed with a local #undef GetAttacks right after #include "chess.h" in testsee.c, so the harness always validates against the true baseline regardless of which implementation is live in production. Verified: gmake TEST=1 GETATTACKS_BITBOARD=1 passes (self-test suite, corrected benchmark still reporting real asm vs. _GetAttacksBB correctly, and a real Search() call exercising _GetAttacksBB live). precommit_check.sh GETATTACKS_BITBOARD=1 clean for both the TEST=1 self-test and DEBUG=1 smoke test. Default (no flag) build confirmed unaffected -- GetAttacks still resolves to the real asm function. See board_representation/MIGRATION.md section 6 for the full writeup. Sections 4/5/7's remaining items (curated-suite sd10 comparison, match_play.py gate) are now unblocked but not yet run. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01Jntky4yGUTyQVaGCXms4F2

Add move-generation bitboard migration scoping doc (planning only)

2026-09-04T16:33:50Z

Drafted after GetAttacks's migration landed, to evaluate extending the same bbPieces/bbPawns/ray-table substrate to generate.c's seven piece-type move generators. Kept as a separate document from MIGRATION.md rather than a new section there, same reasoning as dropping CountKingSafetyDefects from that plan: this is a substantially bigger, higher-risk surface (7 functions, ~3400 lines, no existing reference implementation to diff against, and the pseudo-legal over-generation contract is load-bearing -- a bitboard rewrite that accidentally becomes more legal-aware is a silent behavior change, not a free improvement). Covers: per-function rollout plan (knight/king first as lowest-risk/best-precedented, rook/bishop as the real segment-marking design work, queen mechanical once those land, pawns last and possibly not worth it), a stronger correctness gate than GetAttacks had (perft node-count matching against externally-known-correct numbers, not just internal self-consistency), and a confirmed (not just flagged) scope gap: _GenerateEscapes, the in-check move generation path, has its own independent mailbox implementation and is not covered by the seven piece-type functions this plan targets. No code changes -- planning only. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01Jntky4yGUTyQVaGCXms4F2

Drop CountKingSafetyDefects from board-representation migration plan

2026-09-04T16:27:35Z

Descope CountKingSafetyDefects entirely, per discussion after landing GetAttacks's half of section 3: the two functions no longer share enough to justify one plan. CountKingSafetyDefects (eval.c:2325) turns out to do no ray-walk/blocker check at all -- it's an unblocked CHECK_VECTOR proximity heuristic, not a true attack query -- so _WhoAttacksSquareBB's blocker-aware result isn't a value-identical drop-in for it; making it bitboard-backed would be a real behavior change (needing eval re-tuning/re-gating), not a reimplementation, and a materially different, riskier project than this one. eval.c is untouched. If CountKingSafetyDefects work happens later, it should be a new, separate migration document starting from its actual (unblocked heuristic) behavior, not a resumption of this one. Also updates section 4-7 status notes to reflect what's actually done vs. still blocked on section 6's toggle (GetAttacks alone, no longer entangled with a king-safety timeline). Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01Jntky4yGUTyQVaGCXms4F2

Add bitboard-backed GetAttacks (section 2/3), verified faster than asm

2026-09-04T16:22:46Z

Board-representation migration, sections 2-3 (GetAttacks half): - board.c: VerifyPositionConsistency's bbPieces consistency check (migration section 2), verified clean via gmake TEST=1 with the assert live. - POSITION.bbPawns[2]: new incrementally-maintained per-color pawn location bitboard (chess.h), maintained at the same 6 move.c sites as bbPieces, populated from scratch in fen.c. Distinct from the pawn-hash-keyed bbPawnLocations; this one needs no SEARCHER_THREAD_CONTEXT, so it's reachable from GetAttacks's actual call sites (which only ever have a POSITION*). - data.c/chess.h/main.c: g_RookRayAll/g_BishopRayAll (all 4 per-square ray directions pre-ORed) and g_PawnAttackOriginBB[2][128] startup tables, plus FastFirstBit/FastLastBit (static inline bsf/bsr wrappers, chess.h) -- supporting tables/helpers for the primitive below. - see.c: _WhoAttacksSquareBB (bitboard "who attacks square X" query) and _GetAttacksBB (SEE_LIST-populating PoC wrapping it), side by side with the existing SlowGetAttacks/asm GetAttacks -- not wired into the GetAttacks macro yet (section 6), pure addition. - testsee.c: SeeListsAreEqual made order-independent (SEE() sorts the list right after GetAttacks returns, so order was never semantically significant); TestGetAttacks extended to run _GetAttacksBB as a third comparison across the existing 20,000-random-position sweep; added an interleaved asm/Slow/BB cycles-per-call benchmark across opening/middlegame/endgame positions. - testsup.c: fixed GenerateRandomLegalPosition (used by the sweep above) to maintain bbPieces/bbPawns at its two hand-placement sites -- a latent gap since section 1 that made its own VerifyPositionConsistency legality gate almost always reject generated positions, causing large, variable retry-loop slowdowns. Verified: 20,000-position x every-square x both-colors correctness sweep passes (gmake TEST=1), precommit_check.sh clean (self-test + DEBUG smoke test). Benchmark: _GetAttacksBB is ~0.53-0.55x asm GetAttacks's cycles/call (opening/middlegame) and ~0.89x (endgame) -- faster, not just equivalent, primarily from replacing bbOccupied's up-to-16-iteration pawn loop with two bbPawns ORs, plus a g_PawnAttackOriginBB table lookup replacing per-call pawn-delta arithmetic and per-direction/per-side-group early-outs in the slider walk. See board_representation/MIGRATION.md section 3 for the full writeup, including a reverted approach that measured slower and why, and the CountKingSafetyDefects half's re-scoped (not yet implemented) design. Also confirmed (not caused by this work, not fixed here): a pre-existing non-deterministic MP-race assertion in util.c:1093's PV printing, reproduced independently on a clean HEAD checkout. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01Jntky4yGUTyQVaGCXms4F2