diff options
| author | Scott Gasch <[email protected]> | 2026-10-20 19:46:51 -0700 |
|---|---|---|
| committer | Scott Gasch <[email protected]> | 2026-10-20 19:46:51 -0700 |
| commit | 6b883e83935f5e8110788565cfcb77cd6fbf8950 (patch) | |
| tree | 2ed8ca325c45edcf65bac76a20ba47e8fe8a348e /src | |
| parent | 4e6e78fc767985b9a06ce0fc5dda392027bcc25a (diff) | |
Restore 5D-alpha + 4D-beta FH% collection (improve axis + whole-node bucket), fix two bugs found via validation
Restores the full dual-table FH% collection instrumentation under
FH_TABLE_COLLECTION, which had regressed to a 4D-only alpha joint with
no beta side at all (see the prior commit's note): adds back the
`improve` axis to the alpha joint and a new 4D beta joint (depth,
gap-to-beta, own-improve, enemy-improve), both with real INC() call
sites and ALPHARAW/BETARAW/ESTRAW dump lines matching dual_table_run/'s
existing (unmodified) aggregate.py/dashboard.py/marginal_crosscheck.py
parsers exactly.
Also adds a genuine NODE-level sample into the alpha table's previously
unused late==0 bucket: "will something at this node fail high" when
iEval <= iAlpha, ground truth = the real move loop's outcome. This is
the alpha-side analog of the beta table (which only covers iEval >
iBeta) -- together they cover every null-window child's self-estimate
with no gap, motivating a follow-up (not part of this change) on
combining a parent's per-move FH% prediction with a child's own
node-level self-estimate for a sharper NODE_CUT/NODE_ALL signal than
CommonSearchInit's current purely-structural PVS alternation.
Collecting real ground truth for both new node-level samples requires
not taking the beta-side static-null-move shortcut's early `goto end`
during collection builds, same reasoning as the existing LMP disable.
Two bugs found and fixed via validation before trusting this for a
multi-day run:
1. goto-skips-initializer: Search()'s early QSearch-depth `goto end`
jumped over the new collection variables' declarations, leaving
them uninitialized (indeterminate stack garbage) on that path. Only
manifested in the release (-O3) build -- a DEBUG (-O0) build worked
by accident (the skipped stack slots happened to read as zero).
Caught via lldb on a real segfault: garbage "eligible" flags feeding
wildly out-of-range array indices. Fixed by moving the declarations
before the early goto.
2. MAX_SPLITS amplification: the new arrays were nested inside
COUNTERS.tree, which is embedded both in SEARCHER_THREAD_CONTEXT and
in split.c's g_SplitInfo[MAX_SPLITS] array -- so the ~19MB growth
got replicated 64x, inflating the binary's BSS from ~220MB to
~1.47GB and breaking valgrind's fixed-address mmap outright (harmless
at runtime for a --cpus 1 collection shard, since g_SplitInfo's
copies are never touched, but it also meant any real MP split would
have silently lost a helper thread's collected data, since no
merge-back code exists for these fields). Fixed by moving them into
a new standalone FH_COLLECTION_COUNTERS struct referenced only from
SEARCHER_THREAD_CONTEXT, not from SPLIT_INFO. BSS dropped to ~281MB;
valgrind (--track-origins=yes) now runs clean: 0 errors, 0 bytes
definitely/indirectly/possibly lost.
Validated end-to-end against real NNUE weights (--nnueweights
eval.nnue), release build, full tests/ecm_ringers.ep_ suite at sd 10:
ALPHARAW/BETARAW/ESTRAW all populated (including real late==0 "whole
node" rows), dual_table_run/'s existing aggregate.py and
marginal_crosscheck.py parse it unmodified -- depth/gap/improve axes
match exactly; the only two "mismatches" (dumb and late bucket 0) are
exactly and fully explained by the new whole-node samples, which
deliberately don't feed those two per-move marginals (hang=0/late=0
are placeholders there, not real measurements) -- not a real bug, and
reproduced byte-for-byte identical after the struct relocation above.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Diffstat (limited to 'src')
| -rwxr-xr-x | src/chess.h | 95 | ||||
| -rw-r--r-- | src/root.c | 30 | ||||
| -rw-r--r-- | src/script.c | 241 | ||||
| -rw-r--r-- | src/search.c | 112 | ||||
| -rw-r--r-- | src/searchsup.c | 13 |
5 files changed, 489 insertions, 2 deletions
diff --git a/src/chess.h b/src/chess.h index 34099b2..8c5d86c 100755 --- a/src/chess.h +++ b/src/chess.h @@ -1942,6 +1942,76 @@ KING_DEFECT_DUMP_INFO; // #define COUNTER_MOVE_TABLE_SIZE (0x20000) +#ifdef FH_TABLE_COLLECTION +// 2026-10-20: full dual-table (5D alpha + 4D beta) collection, +// restored for the NNUE-eval re-collection run -- see docs/NNUE.md. +// Deliberately NOT a member of COUNTERS (unlike every other +// PERF_COUNTERS field) -- COUNTERS is embedded both in +// SEARCHER_THREAD_CONTEXT *and* in SPLIT_INFO's g_SplitInfo[MAX_SPLITS] +// array, so anything added there gets silently multiplied by +// MAX_SPLITS. This struct alone is already ~19MB (the 5D alpha array +// dominates); embedded in COUNTERS it inflated the binary's BSS to +// ~1.47GB (confirmed via `size typhoon` + `nm --size-sort`) purely +// from g_SplitInfo's replication -- harmless at runtime since a +// --cpus 1 collection shard never touches g_SplitInfo's copies (pure +// never-faulted zero pages), but it broke valgrind's fixed-address +// mmap of the BSS segment outright, and would have silently dropped +// any data a helper thread collected during a real MP split (no +// merge-back code exists for these fields, unlike COUNTERS' other +// per-split fields) if collection were ever run with --cpus > 1. +// Giving it its own struct, referenced only from +// SEARCHER_THREAD_CONTEXT below, fixes both: SPLIT_INFO never carries +// a copy at all now. +typedef struct _FH_COLLECTION_COUNTERS +{ + // Alpha axes: [depth][dumb][gap][late][improve], same + // fPruneReduceEligible population as u64JointTries/FH (COUNTERS.tree) + // for late >= 2 (real per-move samples). late == 1 is the existing + // "early move" extrapolation proxy used live at the null-move-skip + // site (search.c) -- NOT collected here, it's a lookup-time + // fiction, never a real sample. late == 0 is reserved and, before + // this change, permanently unproduced (uMoveNum is 1-based and the + // per-move eligibility gate requires uLegalMoves > 1) -- repurposed + // here for a genuine NODE-level sample (hang pinned to 0, no + // specific move) answering "will SOMETHING at this node fail + // high", taken once per eligible node when iEval <= iAlpha, ground + // truth = the real move loop's iBestScore >= iBeta at Search()'s + // end: label. This is the alpha-side analog of the beta table's + // own whole-node sample (which only covers iEval > iBeta) -- + // together the two cover every null-window child's self-estimate + // with no gap, since alpha_child/beta_child are adjacent integers + // for a null window. See docs/NNUE.md's node-type discussion. + UINT64 u64Alpha5Tries[SEARCH_DEPTH_BUCKETS][HANG_CP_BUCKETS][GAP_CP_BUCKETS][MOVE_NUM_BUCKETS][IMPROVE_BUCKETS]; + UINT64 u64Alpha5FH[SEARCH_DEPTH_BUCKETS][HANG_CP_BUCKETS][GAP_CP_BUCKETS][MOVE_NUM_BUCKETS][IMPROVE_BUCKETS]; + // Beta axes: [depth][gapBeta][ownImprove][enemyImprove], one + // sample per eligible node (iEval > iBeta), same real-move-loop + // ground truth. Collecting this requires NOT taking the + // static-null-move shortcut's early `goto end` during collection + // builds (same reasoning as the LMP disable in searchsup.c) -- + // otherwise the current, soon-to-be-replaced data.c table would + // censor exactly the population this collection is trying to + // measure. + UINT64 u64BetaTries[SEARCH_DEPTH_BUCKETS][GAP_BETA_BUCKETS][IMPROVE_BUCKETS][IMPROVE_BUCKETS]; + UINT64 u64BetaFH[SEARCH_DEPTH_BUCKETS][GAP_BETA_BUCKETS][IMPROVE_BUCKETS][IMPROVE_BUCKETS]; + // 1D marginals, independent INC() call sites (not derived from the + // joints above) -- see marginal_crosscheck.py. The alpha table's + // depth/dumb/gap/late marginals already exist in COUNTERS.tree + // (u64DepthEstimateTries etc.) and are reused as-is for the 5D + // cross-check; only the improve marginal is new, here. + UINT64 u64ImproveEstimateTries[IMPROVE_BUCKETS]; + UINT64 u64ImproveEstimateFH[IMPROVE_BUCKETS]; + UINT64 u64BetaDepthEstimateTries[SEARCH_DEPTH_BUCKETS]; + UINT64 u64BetaDepthEstimateFH[SEARCH_DEPTH_BUCKETS]; + UINT64 u64BetaGapEstimateTries[GAP_BETA_BUCKETS]; + UINT64 u64BetaGapEstimateFH[GAP_BETA_BUCKETS]; + UINT64 u64BetaOwnImpEstimateTries[IMPROVE_BUCKETS]; + UINT64 u64BetaOwnImpEstimateFH[IMPROVE_BUCKETS]; + UINT64 u64BetaEnemyImpEstimateTries[IMPROVE_BUCKETS]; + UINT64 u64BetaEnemyImpEstimateFH[IMPROVE_BUCKETS]; +} +FH_COLLECTION_COUNTERS; +#endif // FH_TABLE_COLLECTION + typedef struct _SEARCHER_THREAD_CONTEXT { ULONG uPly; // its distance from root @@ -1985,6 +2055,13 @@ typedef struct _SEARCHER_THREAD_CONTEXT COOR cTrapped[MAX_PLY_PER_SEARCH]; // trapped piece hint PIECE pTrapped[MAX_PLY_PER_SEARCH]; COUNTERS sCounters; +#ifdef FH_TABLE_COLLECTION + // Deliberately NOT inside sCounters -- see FH_COLLECTION_COUNTERS' + // own comment (just above this struct) for why: sCounters is also + // embedded in SPLIT_INFO's g_SplitInfo[MAX_SPLITS] array, and + // anything added there gets silently multiplied by MAX_SPLITS. + FH_COLLECTION_COUNTERS sFhCollection; +#endif ULONG uThreadNumber; SPLIT_INFO *pSplitInfo[NUM_SPLIT_PTRS_IN_CONTEXT]; MOVE mvRootMove; @@ -2124,6 +2201,24 @@ typedef struct _GAME_OPTIONS UINT64 u64LMPPruned; UINT64 u64NullMoveTriesByR[NULL_R_BUCKETS]; UINT64 u64NullMoveSuccessByR[NULL_R_BUCKETS]; +#ifdef FH_TABLE_COLLECTION + // Mirrors SEARCHER_THREAD_CONTEXT's sCounters.tree fields of the + // same name -- see that struct's comment for the full design. + UINT64 u64Alpha5Tries[SEARCH_DEPTH_BUCKETS][HANG_CP_BUCKETS][GAP_CP_BUCKETS][MOVE_NUM_BUCKETS][IMPROVE_BUCKETS]; + UINT64 u64Alpha5FH[SEARCH_DEPTH_BUCKETS][HANG_CP_BUCKETS][GAP_CP_BUCKETS][MOVE_NUM_BUCKETS][IMPROVE_BUCKETS]; + UINT64 u64BetaTries[SEARCH_DEPTH_BUCKETS][GAP_BETA_BUCKETS][IMPROVE_BUCKETS][IMPROVE_BUCKETS]; + UINT64 u64BetaFH[SEARCH_DEPTH_BUCKETS][GAP_BETA_BUCKETS][IMPROVE_BUCKETS][IMPROVE_BUCKETS]; + UINT64 u64ImproveEstimateTries[IMPROVE_BUCKETS]; + UINT64 u64ImproveEstimateFH[IMPROVE_BUCKETS]; + UINT64 u64BetaDepthEstimateTries[SEARCH_DEPTH_BUCKETS]; + UINT64 u64BetaDepthEstimateFH[SEARCH_DEPTH_BUCKETS]; + UINT64 u64BetaGapEstimateTries[GAP_BETA_BUCKETS]; + UINT64 u64BetaGapEstimateFH[GAP_BETA_BUCKETS]; + UINT64 u64BetaOwnImpEstimateTries[IMPROVE_BUCKETS]; + UINT64 u64BetaOwnImpEstimateFH[IMPROVE_BUCKETS]; + UINT64 u64BetaEnemyImpEstimateTries[IMPROVE_BUCKETS]; + UINT64 u64BetaEnemyImpEstimateFH[IMPROVE_BUCKETS]; +#endif // FH_TABLE_COLLECTION #ifdef JOINT_EXTRA_VERIFY UINT64 u64JointExtraVerifyTries[JOINT_EXTRA_TIER_COUNT][POST_REDUCE_DEPTH_BUCKETS]; UINT64 u64JointExtraVerifySilentMiss[JOINT_EXTRA_TIER_COUNT][POST_REDUCE_DEPTH_BUCKETS]; @@ -1689,6 +1689,36 @@ Return value: sizeof(g_Options.u64NullMoveTriesByR)); memcpy(g_Options.u64NullMoveSuccessByR, ctx->sCounters.tree.u64NullMoveSuccessByR, sizeof(g_Options.u64NullMoveSuccessByR)); +#ifdef FH_TABLE_COLLECTION + memcpy(g_Options.u64Alpha5Tries, ctx->sFhCollection.u64Alpha5Tries, + sizeof(g_Options.u64Alpha5Tries)); + memcpy(g_Options.u64Alpha5FH, ctx->sFhCollection.u64Alpha5FH, + sizeof(g_Options.u64Alpha5FH)); + memcpy(g_Options.u64BetaTries, ctx->sFhCollection.u64BetaTries, + sizeof(g_Options.u64BetaTries)); + memcpy(g_Options.u64BetaFH, ctx->sFhCollection.u64BetaFH, + sizeof(g_Options.u64BetaFH)); + memcpy(g_Options.u64ImproveEstimateTries, ctx->sFhCollection.u64ImproveEstimateTries, + sizeof(g_Options.u64ImproveEstimateTries)); + memcpy(g_Options.u64ImproveEstimateFH, ctx->sFhCollection.u64ImproveEstimateFH, + sizeof(g_Options.u64ImproveEstimateFH)); + memcpy(g_Options.u64BetaDepthEstimateTries, ctx->sFhCollection.u64BetaDepthEstimateTries, + sizeof(g_Options.u64BetaDepthEstimateTries)); + memcpy(g_Options.u64BetaDepthEstimateFH, ctx->sFhCollection.u64BetaDepthEstimateFH, + sizeof(g_Options.u64BetaDepthEstimateFH)); + memcpy(g_Options.u64BetaGapEstimateTries, ctx->sFhCollection.u64BetaGapEstimateTries, + sizeof(g_Options.u64BetaGapEstimateTries)); + memcpy(g_Options.u64BetaGapEstimateFH, ctx->sFhCollection.u64BetaGapEstimateFH, + sizeof(g_Options.u64BetaGapEstimateFH)); + memcpy(g_Options.u64BetaOwnImpEstimateTries, ctx->sFhCollection.u64BetaOwnImpEstimateTries, + sizeof(g_Options.u64BetaOwnImpEstimateTries)); + memcpy(g_Options.u64BetaOwnImpEstimateFH, ctx->sFhCollection.u64BetaOwnImpEstimateFH, + sizeof(g_Options.u64BetaOwnImpEstimateFH)); + memcpy(g_Options.u64BetaEnemyImpEstimateTries, ctx->sFhCollection.u64BetaEnemyImpEstimateTries, + sizeof(g_Options.u64BetaEnemyImpEstimateTries)); + memcpy(g_Options.u64BetaEnemyImpEstimateFH, ctx->sFhCollection.u64BetaEnemyImpEstimateFH, + sizeof(g_Options.u64BetaEnemyImpEstimateFH)); +#endif // FH_TABLE_COLLECTION #ifdef JOINT_EXTRA_VERIFY memcpy(g_Options.u64JointExtraVerifyTries, ctx->sCounters.tree.u64JointExtraVerifyTries, sizeof(g_Options.u64JointExtraVerifyTries)); diff --git a/src/script.c b/src/script.c index 7286d17..7de170c 100644 --- a/src/script.c +++ b/src/script.c @@ -97,6 +97,29 @@ typedef struct _SUITE_COUNTERS UINT64 u64TotalJointExtraTries[JOINT_EXTRA_TIER_COUNT]; UINT64 u64TotalJointExtraResearch[JOINT_EXTRA_TIER_COUNT]; UINT64 u64TotalLMPPruned; +#ifdef FH_TABLE_COLLECTION + // Full dual-table (5D alpha + 4D beta) collection -- see chess.h's + // SEARCHER_THREAD_CONTEXT.sCounters.tree comment for the full + // design. Dumped as ALPHARAW/BETARAW (5/4-axis raw cells) and + // ESTRAW (1D marginals) below, never printed as a human-readable + // table here -- same reasoning as u64TotalJointTries/FH above, + // just a much bigger shape (the 5D alpha array alone is + // 25*11*11*33*11 = 1,208,625 raw cells). + UINT64 u64TotalAlpha5Tries[SEARCH_DEPTH_BUCKETS][HANG_CP_BUCKETS][GAP_CP_BUCKETS][MOVE_NUM_BUCKETS][IMPROVE_BUCKETS]; + UINT64 u64TotalAlpha5FH[SEARCH_DEPTH_BUCKETS][HANG_CP_BUCKETS][GAP_CP_BUCKETS][MOVE_NUM_BUCKETS][IMPROVE_BUCKETS]; + UINT64 u64TotalBetaTries[SEARCH_DEPTH_BUCKETS][GAP_BETA_BUCKETS][IMPROVE_BUCKETS][IMPROVE_BUCKETS]; + UINT64 u64TotalBetaFH[SEARCH_DEPTH_BUCKETS][GAP_BETA_BUCKETS][IMPROVE_BUCKETS][IMPROVE_BUCKETS]; + UINT64 u64TotalImproveEstimateTries[IMPROVE_BUCKETS]; + UINT64 u64TotalImproveEstimateFH[IMPROVE_BUCKETS]; + UINT64 u64TotalBetaDepthEstimateTries[SEARCH_DEPTH_BUCKETS]; + UINT64 u64TotalBetaDepthEstimateFH[SEARCH_DEPTH_BUCKETS]; + UINT64 u64TotalBetaGapEstimateTries[GAP_BETA_BUCKETS]; + UINT64 u64TotalBetaGapEstimateFH[GAP_BETA_BUCKETS]; + UINT64 u64TotalBetaOwnImpEstimateTries[IMPROVE_BUCKETS]; + UINT64 u64TotalBetaOwnImpEstimateFH[IMPROVE_BUCKETS]; + UINT64 u64TotalBetaEnemyImpEstimateTries[IMPROVE_BUCKETS]; + UINT64 u64TotalBetaEnemyImpEstimateFH[IMPROVE_BUCKETS]; +#endif // FH_TABLE_COLLECTION #ifdef JOINT_EXTRA_VERIFY UINT64 u64TotalJointExtraVerifyTries[JOINT_EXTRA_TIER_COUNT][POST_REDUCE_DEPTH_BUCKETS]; UINT64 u64TotalJointExtraVerifySilentMiss[JOINT_EXTRA_TIER_COUNT][POST_REDUCE_DEPTH_BUCKETS]; @@ -506,6 +529,75 @@ _ProcessSuiteLine(char *szLine, POSITION *pos) } g_SuiteCounters.u64TotalLMPPruned += g_Options.u64LMPPruned; +#ifdef FH_TABLE_COLLECTION + { + ULONG a, b, c, d; + for (a = 0; a < SEARCH_DEPTH_BUCKETS; a++) + { + for (b = 0; b < HANG_CP_BUCKETS; b++) + { + for (c = 0; c < GAP_CP_BUCKETS; c++) + { + for (d = 0; d < MOVE_NUM_BUCKETS; d++) + { + for (v = 0; v < IMPROVE_BUCKETS; v++) + { + g_SuiteCounters.u64TotalAlpha5Tries[a][b][c][d][v] += + g_Options.u64Alpha5Tries[a][b][c][d][v]; + g_SuiteCounters.u64TotalAlpha5FH[a][b][c][d][v] += + g_Options.u64Alpha5FH[a][b][c][d][v]; + } + } + } + } + } + for (a = 0; a < SEARCH_DEPTH_BUCKETS; a++) + { + for (b = 0; b < GAP_BETA_BUCKETS; b++) + { + for (c = 0; c < IMPROVE_BUCKETS; c++) + { + for (d = 0; d < IMPROVE_BUCKETS; d++) + { + g_SuiteCounters.u64TotalBetaTries[a][b][c][d] += + g_Options.u64BetaTries[a][b][c][d]; + g_SuiteCounters.u64TotalBetaFH[a][b][c][d] += + g_Options.u64BetaFH[a][b][c][d]; + } + } + } + } + for (v = 0; v < IMPROVE_BUCKETS; v++) + { + g_SuiteCounters.u64TotalImproveEstimateTries[v] += + g_Options.u64ImproveEstimateTries[v]; + g_SuiteCounters.u64TotalImproveEstimateFH[v] += + g_Options.u64ImproveEstimateFH[v]; + g_SuiteCounters.u64TotalBetaOwnImpEstimateTries[v] += + g_Options.u64BetaOwnImpEstimateTries[v]; + g_SuiteCounters.u64TotalBetaOwnImpEstimateFH[v] += + g_Options.u64BetaOwnImpEstimateFH[v]; + g_SuiteCounters.u64TotalBetaEnemyImpEstimateTries[v] += + g_Options.u64BetaEnemyImpEstimateTries[v]; + g_SuiteCounters.u64TotalBetaEnemyImpEstimateFH[v] += + g_Options.u64BetaEnemyImpEstimateFH[v]; + } + for (v = 0; v < SEARCH_DEPTH_BUCKETS; v++) + { + g_SuiteCounters.u64TotalBetaDepthEstimateTries[v] += + g_Options.u64BetaDepthEstimateTries[v]; + g_SuiteCounters.u64TotalBetaDepthEstimateFH[v] += + g_Options.u64BetaDepthEstimateFH[v]; + } + for (v = 0; v < GAP_BETA_BUCKETS; v++) + { + g_SuiteCounters.u64TotalBetaGapEstimateTries[v] += + g_Options.u64BetaGapEstimateTries[v]; + g_SuiteCounters.u64TotalBetaGapEstimateFH[v] += + g_Options.u64BetaGapEstimateFH[v]; + } + } +#endif // FH_TABLE_COLLECTION #ifdef JOINT_EXTRA_VERIFY { ULONG w; @@ -897,6 +989,155 @@ _PrintSuiteReport(double dSuiteStart) } } +#ifdef FH_TABLE_COLLECTION + // Full dual-table raw dump for offline reassembly/baking (dual_table_run*/ + // aggregate.py et al): ALPHARAW is the 5D alpha joint (depth, dumb, + // gap, late, improve -- late==0 is the "whole node" sample, see + // chess.h), BETARAW is the 4D beta joint (depth, gap, own-improve, + // enemy-improve), ESTRAW is every independent 1D marginal (used + // only by marginal_crosscheck.py, never for baking itself). Same + // "last checkpoint per shard is authoritative" convention as the + // FOURAXISRAW dump above. + { + ULONG a, b, c, d; + for (a = 0; a < SEARCH_DEPTH_BUCKETS; a++) + { + for (b = 0; b < HANG_CP_BUCKETS; b++) + { + for (c = 0; c < GAP_CP_BUCKETS; c++) + { + for (d = 0; d < MOVE_NUM_BUCKETS; d++) + { + for (v = 0; v < IMPROVE_BUCKETS; v++) + { + if (g_SuiteCounters.u64TotalAlpha5Tries[a][b][c][d][v] > 0) + { + Trace("ALPHARAW %lu %lu %lu %lu %lu %" + COMPILER_LONGLONG_UNSIGNED_FORMAT " %" + COMPILER_LONGLONG_UNSIGNED_FORMAT "\n", + a, b, c, d, v, + g_SuiteCounters.u64TotalAlpha5FH[a][b][c][d][v], + g_SuiteCounters.u64TotalAlpha5Tries[a][b][c][d][v]); + } + } + } + } + } + } + for (a = 0; a < SEARCH_DEPTH_BUCKETS; a++) + { + for (b = 0; b < GAP_BETA_BUCKETS; b++) + { + for (c = 0; c < IMPROVE_BUCKETS; c++) + { + for (d = 0; d < IMPROVE_BUCKETS; d++) + { + if (g_SuiteCounters.u64TotalBetaTries[a][b][c][d] > 0) + { + Trace("BETARAW %lu %lu %lu %lu %" + COMPILER_LONGLONG_UNSIGNED_FORMAT " %" + COMPILER_LONGLONG_UNSIGNED_FORMAT "\n", + a, b, c, d, + g_SuiteCounters.u64TotalBetaFH[a][b][c][d], + g_SuiteCounters.u64TotalBetaTries[a][b][c][d]); + } + } + } + } + } + for (v = 0; v < SEARCH_DEPTH_BUCKETS; v++) + { + if (g_SuiteCounters.u64TotalDepthEstimateTries[v] > 0) + { + Trace("ESTRAW depth %lu %" COMPILER_LONGLONG_UNSIGNED_FORMAT + " %" COMPILER_LONGLONG_UNSIGNED_FORMAT "\n", v, + g_SuiteCounters.u64TotalDepthEstimateFH[v], + g_SuiteCounters.u64TotalDepthEstimateTries[v]); + } + } + for (v = 0; v < HANG_CP_BUCKETS; v++) + { + if (g_SuiteCounters.u64TotalHangEstimateTries[v] > 0) + { + Trace("ESTRAW dumb %lu %" COMPILER_LONGLONG_UNSIGNED_FORMAT + " %" COMPILER_LONGLONG_UNSIGNED_FORMAT "\n", v, + g_SuiteCounters.u64TotalHangEstimateFH[v], + g_SuiteCounters.u64TotalHangEstimateTries[v]); + } + } + for (v = 0; v < GAP_CP_BUCKETS; v++) + { + if (g_SuiteCounters.u64TotalGapEstimateTries[v] > 0) + { + Trace("ESTRAW gap %lu %" COMPILER_LONGLONG_UNSIGNED_FORMAT + " %" COMPILER_LONGLONG_UNSIGNED_FORMAT "\n", v, + g_SuiteCounters.u64TotalGapEstimateFH[v], + g_SuiteCounters.u64TotalGapEstimateTries[v]); + } + } + for (v = 0; v < MOVE_NUM_BUCKETS; v++) + { + if (g_SuiteCounters.u64TotalMoveNumEstimateTries[v] > 0) + { + Trace("ESTRAW late %lu %" COMPILER_LONGLONG_UNSIGNED_FORMAT + " %" COMPILER_LONGLONG_UNSIGNED_FORMAT "\n", v, + g_SuiteCounters.u64TotalMoveNumEstimateFH[v], + g_SuiteCounters.u64TotalMoveNumEstimateTries[v]); + } + } + for (v = 0; v < IMPROVE_BUCKETS; v++) + { + if (g_SuiteCounters.u64TotalImproveEstimateTries[v] > 0) + { + Trace("ESTRAW improve %lu %" COMPILER_LONGLONG_UNSIGNED_FORMAT + " %" COMPILER_LONGLONG_UNSIGNED_FORMAT "\n", v, + g_SuiteCounters.u64TotalImproveEstimateFH[v], + g_SuiteCounters.u64TotalImproveEstimateTries[v]); + } + } + for (v = 0; v < SEARCH_DEPTH_BUCKETS; v++) + { + if (g_SuiteCounters.u64TotalBetaDepthEstimateTries[v] > 0) + { + Trace("ESTRAW betadepth %lu %" COMPILER_LONGLONG_UNSIGNED_FORMAT + " %" COMPILER_LONGLONG_UNSIGNED_FORMAT "\n", v, + g_SuiteCounters.u64TotalBetaDepthEstimateFH[v], + g_SuiteCounters.u64TotalBetaDepthEstimateTries[v]); + } + } + for (v = 0; v < GAP_BETA_BUCKETS; v++) + { + if (g_SuiteCounters.u64TotalBetaGapEstimateTries[v] > 0) + { + Trace("ESTRAW betagap %lu %" COMPILER_LONGLONG_UNSIGNED_FORMAT + " %" COMPILER_LONGLONG_UNSIGNED_FORMAT "\n", v, + g_SuiteCounters.u64TotalBetaGapEstimateFH[v], + g_SuiteCounters.u64TotalBetaGapEstimateTries[v]); + } + } + for (v = 0; v < IMPROVE_BUCKETS; v++) + { + if (g_SuiteCounters.u64TotalBetaOwnImpEstimateTries[v] > 0) + { + Trace("ESTRAW betaown %lu %" COMPILER_LONGLONG_UNSIGNED_FORMAT + " %" COMPILER_LONGLONG_UNSIGNED_FORMAT "\n", v, + g_SuiteCounters.u64TotalBetaOwnImpEstimateFH[v], + g_SuiteCounters.u64TotalBetaOwnImpEstimateTries[v]); + } + } + for (v = 0; v < IMPROVE_BUCKETS; v++) + { + if (g_SuiteCounters.u64TotalBetaEnemyImpEstimateTries[v] > 0) + { + Trace("ESTRAW betaopp %lu %" COMPILER_LONGLONG_UNSIGNED_FORMAT + " %" COMPILER_LONGLONG_UNSIGNED_FORMAT "\n", v, + g_SuiteCounters.u64TotalBetaEnemyImpEstimateFH[v], + g_SuiteCounters.u64TotalBetaEnemyImpEstimateTries[v]); + } + } + } +#endif // FH_TABLE_COLLECTION + #ifdef TEST_NULL // Null-move precision/recall calibration, summed across the whole // suite (see _NullCalibBucketIndex's comment in search.c for the diff --git a/src/search.c b/src/search.c index 7d8d686..6f5e90c 100644 --- a/src/search.c +++ b/src/search.c @@ -374,6 +374,24 @@ Search(IN SEARCHER_THREAD_CONTEXT *ctx, // same QUARTER_PLY off the per-iteration depth budget so this doesn't // just add a blanket 1/4 ply to every search. SCORE iBestScore; +#ifdef FH_TABLE_COLLECTION + // Declared here, BEFORE the QSearch-jump `goto end;` just below -- + // these must be initialized on every path reaching end:, including + // the early QSearch exit, or the commit code there reads + // indeterminate (garbage) values. Declaring them further down + // (alongside the other per-node locals) put their initializers + // after this goto, which skips them entirely on the QSearch-exit + // path -- caught live via a release-build (-O3) segfault that + // didn't reproduce in DEBUG (-O0, where the skipped stack slots + // happened to already read as zero): fBetaCollectEligible/ + // fAlpha0CollectEligible read back as nonzero garbage, feeding + // equally-garbage bucket indices into the array indexing at end:. + FLAG fBetaCollectEligible = FALSE; + ULONG uBetaCollectDepthB = 0, uBetaCollectGapB = 0, + uBetaCollectOwnImpB = 0, uBetaCollectEnemyImpB = 0; + FLAG fAlpha0CollectEligible = FALSE; + ULONG uAlpha0CollectDepthB = 0, uAlpha0CollectGapB = 0; +#endif if (uDepth < THREE_QUARTERS_PLY) { pf->fCouldStandPat[BLACK] = pf->fCouldStandPat[WHITE] = FALSE; @@ -708,6 +726,27 @@ Search(IN SEARCHER_THREAD_CONTEXT *ctx, ULONG uRawEnemyImproveB = ImproveBucket(iEnemyImprovement); INC(ctx->sCounters.tree.u64QuickNullMoves); +#ifdef FH_TABLE_COLLECTION + // Collection builds record a genuine whole-node try here + // (ground truth committed at end:) and must NOT take the + // shortcut below -- it would skip the real move loop + // based on the CURRENTLY LOADED (soon-to-be-replaced) + // data.c table, censoring exactly the population this + // collection is trying to measure. Same reasoning as the + // LMP disable in searchsup.c. + fBetaCollectEligible = TRUE; + uBetaCollectDepthB = uStaticNMDepthB; + uBetaCollectGapB = GapBetaBucket((SCORE)(iEval - iBeta)); + uBetaCollectOwnImpB = uRawImproveB; + uBetaCollectEnemyImpB = uRawEnemyImproveB; + INC(ctx->sFhCollection.u64BetaDepthEstimateTries[uBetaCollectDepthB]); + INC(ctx->sFhCollection.u64BetaGapEstimateTries[uBetaCollectGapB]); + INC(ctx->sFhCollection.u64BetaOwnImpEstimateTries[uBetaCollectOwnImpB]); + INC(ctx->sFhCollection.u64BetaEnemyImpEstimateTries[uBetaCollectEnemyImpB]); + INC(ctx->sFhCollection.u64BetaTries + [uBetaCollectDepthB][uBetaCollectGapB] + [uBetaCollectOwnImpB][uBetaCollectEnemyImpB]); +#else ULONG uBetaPredictedFHThousandths = _BetaTablePredictedFHThousandths( uStaticNMDepthB, GapBetaBucket((SCORE)(iEval - iBeta)), @@ -723,12 +762,37 @@ Search(IN SEARCHER_THREAD_CONTEXT *ctx, iBestScore = iBeta; goto end; } +#endif } -#if (NULLMOVE_SKIP_FH_CEILING_THOUSANDTHS > 0) else if (iEval <= iAlpha) { ULONG uNullSkipDepthB = DepthBucket(uDepth / ONE_PLY); ULONG uRawGapAlphaB = GapCpBucket((SCORE)(iAlpha - iEval)); +#ifdef FH_TABLE_COLLECTION + // Whole-node sample for the alpha table's reserved + // late==0 bucket (see chess.h) -- the alpha-side analog + // of the beta-side whole-node try just above, covering + // the iEval <= iAlpha regime the beta table doesn't. + // hang is pinned to 0 (no specific move; "nothing hung" + // is the only sensible placeholder) and late is pinned to + // 0 (the reserved "whole node" sentinel, never produced + // by a real per-move sample) -- deliberately NOT fed into + // the shared per-move hang/late marginals below (those + // values are placeholders here, not real measurements), + // but depth/gap/improve ARE real, equally-meaningful + // quantities for a whole-node sample, so they DO feed the + // shared marginals. + fAlpha0CollectEligible = TRUE; + uAlpha0CollectDepthB = uNullSkipDepthB; + uAlpha0CollectGapB = uRawGapAlphaB; + INC(ctx->sCounters.tree.u64DepthEstimateTries[uAlpha0CollectDepthB]); + INC(ctx->sCounters.tree.u64GapEstimateTries[uAlpha0CollectGapB]); + INC(ctx->sFhCollection.u64ImproveEstimateTries[uRawImproveB]); + INC(ctx->sFhCollection.u64Alpha5Tries + [uAlpha0CollectDepthB][0][uAlpha0CollectGapB][0][uRawImproveB]); +#endif +#if (NULLMOVE_SKIP_FH_CEILING_THOUSANDTHS > 0) + { USHORT uAlphaPredictedFH = _JointTablePredictedFHThousandths( uNullSkipDepthB, 0, // hang bucket: assume nothing hung @@ -741,8 +805,9 @@ Search(IN SEARCHER_THREAD_CONTEXT *ctx, fSkipNullForLowFH = TRUE; INC(ctx->sCounters.tree.u64NullMoveSkippedLowFH); } - } + } #endif + } if ((FALSE == fSkipNullForLowFH) && WeShouldTryNullmovePruning(ctx, iAlpha, @@ -1138,6 +1203,12 @@ Search(IN SEARCHER_THREAD_CONTEXT *ctx, INC(ctx->sCounters.tree.u64JointFH [uThisMoveDepthBucket][uThisMoveHangBucket] [uThisMoveGapBucket][uThisMoveNumBucket]); +#ifdef FH_TABLE_COLLECTION + INC(ctx->sFhCollection.u64ImproveEstimateFH[uRawImproveB]); + INC(ctx->sFhCollection.u64Alpha5FH + [uThisMoveDepthBucket][uThisMoveHangBucket] + [uThisMoveGapBucket][uThisMoveNumBucket][uRawImproveB]); +#endif } } #endif // PERF_COUNTERS @@ -1271,6 +1342,43 @@ Search(IN SEARCHER_THREAD_CONTEXT *ctx, ASSERT(IS_VALID_SCORE(iAlpha)); ASSERT(IS_VALID_SCORE(iBestScore) || WE_SHOULD_STOP_SEARCHING); ASSERT(PositionsAreEquivalent(pos, &pi->sPosition)); +#ifdef FH_TABLE_COLLECTION + // Commit the whole-node ground truth stashed above, now that + // iBestScore's real fate against iBeta is known -- same two-phase + // pattern as TEST_NULL's commit just below. Skipped on an + // aborted/incomplete search (WE_SHOULD_STOP_SEARCHING): iBestScore + // may not reflect a fully-searched node in that case, and counting + // it as ground truth would bias the table toward whatever got + // searched before the clock/node-limit fired. + if ((!WE_SHOULD_STOP_SEARCHING) && IS_VALID_SCORE(iBestScore)) + { + FLAG fRealFH = (iBestScore >= iBeta); + if (fBetaCollectEligible) + { + if (fRealFH) + { + INC(ctx->sFhCollection.u64BetaDepthEstimateFH[uBetaCollectDepthB]); + INC(ctx->sFhCollection.u64BetaGapEstimateFH[uBetaCollectGapB]); + INC(ctx->sFhCollection.u64BetaOwnImpEstimateFH[uBetaCollectOwnImpB]); + INC(ctx->sFhCollection.u64BetaEnemyImpEstimateFH[uBetaCollectEnemyImpB]); + INC(ctx->sFhCollection.u64BetaFH + [uBetaCollectDepthB][uBetaCollectGapB] + [uBetaCollectOwnImpB][uBetaCollectEnemyImpB]); + } + } + if (fAlpha0CollectEligible) + { + if (fRealFH) + { + INC(ctx->sCounters.tree.u64DepthEstimateFH[uAlpha0CollectDepthB]); + INC(ctx->sCounters.tree.u64GapEstimateFH[uAlpha0CollectGapB]); + INC(ctx->sFhCollection.u64ImproveEstimateFH[uRawImproveB]); + INC(ctx->sFhCollection.u64Alpha5FH + [uAlpha0CollectDepthB][0][uAlpha0CollectGapB][0][uRawImproveB]); + } + } + } +#endif // FH_TABLE_COLLECTION #ifdef TEST_NULL // Commit the calibration record stashed at the diagnostic probe // site above, now that iBestScore's fate against iBeta (this diff --git a/src/searchsup.c b/src/searchsup.c index f3d1370..eed9846 100644 --- a/src/searchsup.c +++ b/src/searchsup.c @@ -1951,6 +1951,19 @@ Return value: INC(ctx->sCounters.tree.u64JointTries [uThisMoveDepthBucket][uThisMoveHangBucket] [uThisMoveGapBucket][uThisMoveNumBucket]); + +#ifdef FH_TABLE_COLLECTION + // Full 5D alpha joint (restored for the NNUE-eval + // re-collection run, see chess.h) -- same four bucket + // indices just computed, plus the caller-supplied + // improve bucket (loop-invariant per node, computed once + // by Search() before the move loop). Independent + // "improve" marginal too, matching the convention above. + INC(ctx->sFhCollection.u64ImproveEstimateTries[uRawImproveB]); + INC(ctx->sFhCollection.u64Alpha5Tries + [uThisMoveDepthBucket][uThisMoveHangBucket] + [uThisMoveGapBucket][uThisMoveNumBucket][uRawImproveB]); +#endif } #endif } |
