diff options
| author | Scott Gasch <[email protected]> | 2026-10-04 08:17:52 -0700 |
|---|---|---|
| committer | Scott Gasch <[email protected]> | 2026-10-04 08:17:52 -0700 |
| commit | bbdf1bda8f410df6aa9be0a38e71b7db04982198 (patch) | |
| tree | e8315247f700ee94875cd6d2eb5dd81aef71395d /src/script.c | |
| parent | 9b28080f8b3dca4588cb2aaaf748451f45e3c626 (diff) | |
Dynamic move-ordering/LMR session: honest FH tracking, decoupled admission/magnitude, bugfixes, tooling
Lands the 2026-09-28..10-03 move-ordering investigation (full history in
lmr_testing/*.md). Summary of what ships:
LMR/move-ordering redesign (searchsup.c, dynamic.c, data.c, chess.h):
- GetLMRReduction's admission decision (do we reduce at all) is now
decoupled from magnitude (how much): LMR_STATIC_ADMIT_THOUSANDTHS=500
gates admission via the 4D joint fail-high table (data.c) plus a
countermove-match veto; magnitude is a flat Crafty/Ethereal-shaped
log(depth)*log(moves) table (g_iLMRQuietReduction), which measured as
good or better than the old per-tier joint-table payout while being
structurally immune to the per-bucket miscalibration that sank the
tiered approach. g_FailHighs honesty fix (real 0/0 init, not a fake
prior) underlies the dynamic-danger signal.
- Retired NumLeftoverMovesToSelect's depth-indexed selection budget --
evidence showed the leftover pool has real findable signal a bailout
was discarding.
- ShouldLatePrune (overt late-move pruning) implemented but disabled at
its call site -- measured net-negative at the tested threshold; kept
for reference and as a base for a future hang-bucket-9-targeted retry.
- PRUNE_VERIFY, PRUNE_LIVE, DISABLE_LMR: opt-in diagnostic build flags
(GNUmakefile), off in the standard profile.
Bug fixes found and fixed while exercising this work:
- JOINT_EXTRA_TIER_COUNT 4->6 (chess.h): fixed an out-of-bounds counter
write reachable at remaining depth >= 14 (unreachable at sd10, routine
in real search/self-play).
- Node-type instrumentation (search.c, root.c, searchsup.c): fixed four
compounding defects that produced impossible percentages (e.g. 106.2%
of PV nodes) -- wrong denominator, qsearch misclassified as main
search, untyped early-exit nodes, and fail-low tests using the
in-place-raised alpha instead of the initial one.
- FH-rate unit unification (search.c, searchsup.c, dynamic.c, chess.h):
GetMoveFailHigh* returned per-mille while g_uJointFHThousandths is
thousandths-of-a-percent, two different units sharing a "_THOUSANDTHS"
suffix. Unified to one scale, renamed to "_PCT_THOUSANDTHS".
RateBucket's band edges fixed to match.
- QSearch option B (search.c): waives the opponent-stand-pat requirement
at NODE_PV only (sound there since the exact score propagates; not
sound at null-window nodes). iExtensionAmount unit fix (was 0/2 into a
ONE_PLY-scaled field). pi->iEval now published when Eval() runs.
- eval_tune/match_play.py local_ebf: was reporting 0.000 in every match
since 2026-09-18 (regex only matched the trailing-stats parser, not
the live move-scan loop; engines were also spawned with "nopost" so
root.c never emitted the lines at all). True per-ply EBF is ~2.0, not
the ~3.9 every prior match report implied.
- eval_tune/test_vs_head.sh: positional-arg shift ordering bug caused
every no-extra-args run to silently rebuild ./typhoon with a
non-standard profile and, separately, could feed a flag's value in as
the candidate binary path.
- PERF_COUNTERS cleanup (eval.c, search.c): moved 3 leaked rdtsc sites to
the existing EVAL_TIME guard, deleted two concluded one-off diagnostics
(u64HeadStyle*, u64FHJointHist/RateBucket/AttemptsBucket) and their
report plumbing, and re-gated a per-move hang-material estimate behind
FH_TABLE_COLLECTION instead of running it unconditionally.
Infrastructure:
- match_play.py time-odds support (--odds/--st-head/--st-candidate),
validated against the engine seat rather than color.
- Wilson-upper-bound re-bake of the joint table (data.c): cells store a
confidence-aware bound instead of a raw point estimate, so a consumer
can threshold on the value alone; monotone, so every consumer became
strictly more conservative. Verified behavior-neutral for LMR.
Known-open items (not resolved here, see lmr_testing/GET_LMR_REDUCTION.md
section 10 and 2026-10-03_UNCOMMITTED_TRIAGE.md): the self-play gate on
this candidate is underpowered (+5 Elo, 95% CI roughly [-6, +17]); the
curated-suite solve-count deltas used earlier in the investigation turned
out to be noise-dominated cancellation when diffed per position, not the
clean signal they first appeared to be.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Diffstat (limited to 'src/script.c')
| -rw-r--r-- | src/script.c | 528 |
1 files changed, 528 insertions, 0 deletions
diff --git a/src/script.c b/src/script.c index f18ab7b..219825c 100644 --- a/src/script.c +++ b/src/script.c @@ -64,6 +64,52 @@ typedef struct _SUITE_COUNTERS UINT64 u64TotalLeftoverFH; UINT64 u64TotalCMEvidenceTries[CM_EVIDENCE_BUCKETS]; UINT64 u64TotalCMEvidenceFH[CM_EVIDENCE_BUCKETS]; + // See chess.h's HANG_CP_BUCKETS comment. Same accumulation/print + // pattern as u64TotalCMEvidenceTries above -- only meaningful + // gathered single-threaded (--cpus 1), same as that one (not + // wired into split.c's MP merge). + UINT64 u64TotalHangEstimateTries[HANG_CP_BUCKETS]; + UINT64 u64TotalHangEstimateFH[HANG_CP_BUCKETS]; + // The real comparison baseline for the buckets above -- same + // fPruneReduceEligible population, not just fIsLeftoverMove. + UINT64 u64TotalEligibleLeftoverTries; + UINT64 u64TotalEligibleLeftoverFH; + // The "down axis" -- (iAlpha - iEval) calibration. Same population + // and comparison baseline as the hang-estimate buckets above. + UINT64 u64TotalGapEstimateTries[GAP_CP_BUCKETS]; + UINT64 u64TotalGapEstimateFH[GAP_CP_BUCKETS]; + // The "late axis" -- this move's own ordinal position calibration. + // Same population and comparison baseline as the other two axes. + UINT64 u64TotalMoveNumEstimateTries[MOVE_NUM_BUCKETS]; + UINT64 u64TotalMoveNumEstimateFH[MOVE_NUM_BUCKETS]; + // The "depth axis" -- remaining search depth calibration. Same + // population and comparison baseline as the other three axes. + UINT64 u64TotalDepthEstimateTries[SEARCH_DEPTH_BUCKETS]; + UINT64 u64TotalDepthEstimateFH[SEARCH_DEPTH_BUCKETS]; + // Joint 4D table, marginal (uncoarsened) resolution -- reassembled + // offline by the bake script via the FOURAXISRAW dump below, never + // printed as a human-readable table here (21,780+ shipped cells, + // 99,825 at this raw resolution -- far too many lines). + UINT64 u64TotalJointTries[SEARCH_DEPTH_BUCKETS][HANG_CP_BUCKETS][GAP_CP_BUCKETS][MOVE_NUM_BUCKETS]; + UINT64 u64TotalJointFH[SEARCH_DEPTH_BUCKETS][HANG_CP_BUCKETS][GAP_CP_BUCKETS][MOVE_NUM_BUCKETS]; + // Phase 2 -- joint-table-driven graduated LMR calibration (see + // search.c's JOINT_EXTRA_TIER*_THOUSANDTHS comment). + UINT64 u64TotalJointExtraTries[JOINT_EXTRA_TIER_COUNT]; + UINT64 u64TotalJointExtraResearch[JOINT_EXTRA_TIER_COUNT]; + UINT64 u64TotalLMPPruned; +#ifdef PRUNE_VERIFY + UINT64 u64TotalPruneVerifyTries[PRUNE_VERIFY_BANDS][SHIP_DEPTH_BUCKETS]; + UINT64 u64TotalPruneVerifyFailHigh[PRUNE_VERIFY_BANDS][SHIP_DEPTH_BUCKETS]; + UINT64 u64TotalPruneVerifyMarginSum[PRUNE_VERIFY_BANDS][SHIP_DEPTH_BUCKETS]; + UINT64 u64TotalPruneVerifyPredSum[PRUNE_VERIFY_BANDS][SHIP_DEPTH_BUCKETS]; + UINT64 u64TotalPruneVerifyDyn[8][8]; + UINT64 u64TotalPruneVerifyDynMiss[8][8]; +#endif +#ifdef JOINT_EXTRA_VERIFY + UINT64 u64TotalJointExtraVerifyTries[JOINT_EXTRA_TIER_COUNT][POST_REDUCE_DEPTH_BUCKETS]; + UINT64 u64TotalJointExtraVerifySilentMiss[JOINT_EXTRA_TIER_COUNT][POST_REDUCE_DEPTH_BUCKETS]; + UINT64 u64TotalJointExtraVerifyNullBailoutSignature[JOINT_EXTRA_TIER_COUNT]; +#endif #ifdef TEST_NULL // Null-move precision/recall calibration, summed across every // position in the suite -- see _NullCalibBucketIndex's comment in @@ -408,6 +454,112 @@ _ProcessSuiteLine(char *szLine, POSITION *pos) g_SuiteCounters.u64TotalCMEvidenceFH[v] += g_Options.u64CMEvidenceFH[v]; } + for (v = 0; v < HANG_CP_BUCKETS; v++) + { + g_SuiteCounters.u64TotalHangEstimateTries[v] += + g_Options.u64HangEstimateTries[v]; + g_SuiteCounters.u64TotalHangEstimateFH[v] += + g_Options.u64HangEstimateFH[v]; + } + g_SuiteCounters.u64TotalEligibleLeftoverTries += + g_Options.u64EligibleLeftoverTries; + g_SuiteCounters.u64TotalEligibleLeftoverFH += + g_Options.u64EligibleLeftoverFH; + for (v = 0; v < GAP_CP_BUCKETS; v++) + { + g_SuiteCounters.u64TotalGapEstimateTries[v] += + g_Options.u64GapEstimateTries[v]; + g_SuiteCounters.u64TotalGapEstimateFH[v] += + g_Options.u64GapEstimateFH[v]; + } + for (v = 0; v < MOVE_NUM_BUCKETS; v++) + { + g_SuiteCounters.u64TotalMoveNumEstimateTries[v] += + g_Options.u64MoveNumEstimateTries[v]; + g_SuiteCounters.u64TotalMoveNumEstimateFH[v] += + g_Options.u64MoveNumEstimateFH[v]; + } + for (v = 0; v < SEARCH_DEPTH_BUCKETS; v++) + { + g_SuiteCounters.u64TotalDepthEstimateTries[v] += + g_Options.u64DepthEstimateTries[v]; + g_SuiteCounters.u64TotalDepthEstimateFH[v] += + g_Options.u64DepthEstimateFH[v]; + } + { + ULONG a, b, c; + for (a = 0; a < SEARCH_DEPTH_BUCKETS; a++) + { + for (b = 0; b < HANG_CP_BUCKETS; b++) + { + for (c = 0; c < GAP_CP_BUCKETS; c++) + { + for (v = 0; v < MOVE_NUM_BUCKETS; v++) + { + g_SuiteCounters.u64TotalJointTries[a][b][c][v] += + g_Options.u64JointTries[a][b][c][v]; + g_SuiteCounters.u64TotalJointFH[a][b][c][v] += + g_Options.u64JointFH[a][b][c][v]; + } + } + } + } + } + for (v = 0; v < JOINT_EXTRA_TIER_COUNT; v++) + { + g_SuiteCounters.u64TotalJointExtraTries[v] += + g_Options.u64JointExtraTries[v]; + g_SuiteCounters.u64TotalJointExtraResearch[v] += + g_Options.u64JointExtraResearch[v]; + } + g_SuiteCounters.u64TotalLMPPruned += + g_Options.u64LMPPruned; + #ifdef PRUNE_VERIFY + { + ULONG w; + for (v = 0; v < PRUNE_VERIFY_BANDS; v++) + { + for (w = 0; w < SHIP_DEPTH_BUCKETS; w++) + { + g_SuiteCounters.u64TotalPruneVerifyTries[v][w] += + g_Options.u64PruneVerifyTries[v][w]; + g_SuiteCounters.u64TotalPruneVerifyFailHigh[v][w] += + g_Options.u64PruneVerifyFailHigh[v][w]; + g_SuiteCounters.u64TotalPruneVerifyMarginSum[v][w] += + g_Options.u64PruneVerifyMarginSum[v][w]; + g_SuiteCounters.u64TotalPruneVerifyPredSum[v][w] += + g_Options.u64PruneVerifyPredSum[v][w]; + } + } + for (v = 0; v < 8; v++) + { + for (w = 0; w < 8; w++) + { + g_SuiteCounters.u64TotalPruneVerifyDyn[v][w] += + g_Options.u64PruneVerifyDyn[v][w]; + g_SuiteCounters.u64TotalPruneVerifyDynMiss[v][w] += + g_Options.u64PruneVerifyDynMiss[v][w]; + } + } + } +#endif +#ifdef JOINT_EXTRA_VERIFY + { + ULONG w; + for (v = 0; v < JOINT_EXTRA_TIER_COUNT; v++) + { + for (w = 0; w < POST_REDUCE_DEPTH_BUCKETS; w++) + { + g_SuiteCounters.u64TotalJointExtraVerifyTries[v][w] += + g_Options.u64JointExtraVerifyTries[v][w]; + g_SuiteCounters.u64TotalJointExtraVerifySilentMiss[v][w] += + g_Options.u64JointExtraVerifySilentMiss[v][w]; + } + g_SuiteCounters.u64TotalJointExtraVerifyNullBailoutSignature[v] += + g_Options.u64JointExtraVerifyNullBailoutSignature[v]; + } + } +#endif } #ifdef TEST_NULL { @@ -513,6 +665,360 @@ _PrintSuiteReport(double dSuiteStart) } } + // EstimateHangingMaterialCp calibration -- does "how much material + // does this leftover move look like it hangs" actually predict + // FH%? See chess.h's HANG_CP_BUCKETS comment for the bucket shape + // (bucket 0 == not dumb; 1..9 are 100cp-wide bins 1-900cp; 10 is + // the 901cp+ overflow). + { + Trace("\nhanging-material estimate calibration (captures + quiet " + "moves, both gated on fPruneReduceEligible):\n"); + Trace(" [baseline] eligible : %5.2f%% FH (%" + COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n", + (100.0 * (double)g_SuiteCounters.u64TotalEligibleLeftoverFH / + ((double)g_SuiteCounters.u64TotalEligibleLeftoverTries + 1.0)), + g_SuiteCounters.u64TotalEligibleLeftoverTries); + Trace(" not dumb (0cp) : %5.2f%% FH (%" + COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n", + (100.0 * (double)g_SuiteCounters.u64TotalHangEstimateFH[0] / + ((double)g_SuiteCounters.u64TotalHangEstimateTries[0] + 1.0)), + g_SuiteCounters.u64TotalHangEstimateTries[0]); + for (v = 1; v < HANG_CP_BUCKETS; v++) + { + if (v < HANG_CP_BUCKETS - 1) + { + Trace(" %4u - %4u cp lost : %5.2f%% FH (%" + COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n", + 1 + (v - 1) * HANG_CP_BUCKET_WIDTH, + v * HANG_CP_BUCKET_WIDTH, + (100.0 * (double)g_SuiteCounters.u64TotalHangEstimateFH[v] / + ((double)g_SuiteCounters.u64TotalHangEstimateTries[v] + 1.0)), + g_SuiteCounters.u64TotalHangEstimateTries[v]); + } + else + { + Trace(" %4u+ cp lost : %5.2f%% FH (%" + COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n", + 1 + (v - 1) * HANG_CP_BUCKET_WIDTH, + (100.0 * (double)g_SuiteCounters.u64TotalHangEstimateFH[v] / + ((double)g_SuiteCounters.u64TotalHangEstimateTries[v] + 1.0)), + g_SuiteCounters.u64TotalHangEstimateTries[v]); + } + } + } + + // The "down axis" -- (iAlpha - iEval) calibration. See chess.h's + // GAP_CP_BUCKETS comment for the bucket shape (bucket 0 == gap + // <= 0, our own static eval already meets/beats alpha; 1..9 are + // 100cp-wide bins for a positive gap of 1-900cp; 10 is the + // 901cp+ overflow). Same [baseline] eligible line above applies + // here too -- same population, same fPruneReduceEligible gate. + { + Trace("\ngap-to-alpha estimate calibration (captures + quiet " + "moves, both gated on fPruneReduceEligible):\n"); + Trace(" gap <= 0 (eval>=alpha): %5.2f%% FH (%" + COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n", + (100.0 * (double)g_SuiteCounters.u64TotalGapEstimateFH[0] / + ((double)g_SuiteCounters.u64TotalGapEstimateTries[0] + 1.0)), + g_SuiteCounters.u64TotalGapEstimateTries[0]); + for (v = 1; v < GAP_CP_BUCKETS; v++) + { + if (v < GAP_CP_BUCKETS - 1) + { + Trace(" %4u - %4u cp below alpha : %5.2f%% FH (%" + COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n", + 1 + (v - 1) * GAP_CP_BUCKET_WIDTH, + v * GAP_CP_BUCKET_WIDTH, + (100.0 * (double)g_SuiteCounters.u64TotalGapEstimateFH[v] / + ((double)g_SuiteCounters.u64TotalGapEstimateTries[v] + 1.0)), + g_SuiteCounters.u64TotalGapEstimateTries[v]); + } + else + { + Trace(" %4u+ cp below alpha : %5.2f%% FH (%" + COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n", + 1 + (v - 1) * GAP_CP_BUCKET_WIDTH, + (100.0 * (double)g_SuiteCounters.u64TotalGapEstimateFH[v] / + ((double)g_SuiteCounters.u64TotalGapEstimateTries[v] + 1.0)), + g_SuiteCounters.u64TotalGapEstimateTries[v]); + } + } + } + + // The "late axis" -- this move's own 1-based ordinal position + // among moves actually tried at this node. See chess.h's + // MOVE_NUM_BUCKETS comment -- direct, unpaired buckets, move + // number (MOVE_NUM_BUCKETS-2)+ collapses into one overflow + // bucket. Same [baseline] eligible line as the other two axes + // applies here too. + { + Trace("\nmove-number estimate calibration (captures + quiet " + "moves, both gated on fPruneReduceEligible):\n"); + for (v = 1; v < MOVE_NUM_BUCKETS; v++) + { + if (v < MOVE_NUM_BUCKETS - 1) + { + Trace(" move %4u : %5.2f%% FH (%" + COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n", + v, + (100.0 * (double)g_SuiteCounters.u64TotalMoveNumEstimateFH[v] / + ((double)g_SuiteCounters.u64TotalMoveNumEstimateTries[v] + 1.0)), + g_SuiteCounters.u64TotalMoveNumEstimateTries[v]); + } + else + { + Trace(" move %4u+ : %5.2f%% FH (%" + COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n", + v, + (100.0 * (double)g_SuiteCounters.u64TotalMoveNumEstimateFH[v] / + ((double)g_SuiteCounters.u64TotalMoveNumEstimateTries[v] + 1.0)), + g_SuiteCounters.u64TotalMoveNumEstimateTries[v]); + } + } + } + + // The "depth axis" -- remaining search depth at this node (uDepth + // / ONE_PLY, whole plies, before this move's own extension/ + // reduction). Direct, unpaired buckets, index 0 legitimately used + // (a heavily-reduced move can arrive at remaining depth 0). Same + // [baseline] eligible line as the other axes applies here too. + { + Trace("\ndepth estimate calibration (captures + quiet moves, " + "both gated on fPruneReduceEligible):\n"); + for (v = 0; v < SEARCH_DEPTH_BUCKETS; v++) + { + if (v < SEARCH_DEPTH_BUCKETS - 1) + { + Trace(" depth %4u : %5.2f%% FH (%" + COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n", + v, + (100.0 * (double)g_SuiteCounters.u64TotalDepthEstimateFH[v] / + ((double)g_SuiteCounters.u64TotalDepthEstimateTries[v] + 1.0)), + g_SuiteCounters.u64TotalDepthEstimateTries[v]); + } + else + { + Trace(" depth %4u+ : %5.2f%% FH (%" + COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n", + v, + (100.0 * (double)g_SuiteCounters.u64TotalDepthEstimateFH[v] / + ((double)g_SuiteCounters.u64TotalDepthEstimateTries[v] + 1.0)), + g_SuiteCounters.u64TotalDepthEstimateTries[v]); + } + } + } + + // Phase 2 -- joint-table-driven graduated LMR calibration: for each + // tier (0 == GetLMRReduction's baseline only, 1..3 == that many + // extra plies the joint table added), what fraction of moves + // assigned that tier triggered the existing re-search-on-fail-high + // path? This is the actual scale-calibration mechanism (not the + // offline collection data) -- if low-predicted-FH tiers (2, 3) show + // an EQUAL OR HIGHER research rate than tier 0/1, that's the + // distribution-shift failure this project's history keeps + // producing, and the tier thresholds (or the whole approach) need + // rethinking before any self-play run. A rising research rate with + // tier is the expected, healthy shape. + { + Trace("\njoint-table extra-reduction tier calibration " + "(re-search rate per tier):\n"); + for (v = 0; v < JOINT_EXTRA_TIER_COUNT; v++) + { + Trace(" tier %u (+%u extra ply%s) : %5.2f%% research-rate (%" + COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n", + v, v, (1 == v) ? "" : "s", + (100.0 * (double)g_SuiteCounters.u64TotalJointExtraResearch[v] / + ((double)g_SuiteCounters.u64TotalJointExtraTries[v] + 1.0)), + g_SuiteCounters.u64TotalJointExtraTries[v]); + } +#ifdef PRUNE_VERIFY + // See chess.h's PRUNE_VERIFY comment. The calibration test is + // MEASURED vs PREDICTED on exactly the flagged population -- + // never measured-vs-threshold, which is a ceiling on what gets + // flagged rather than the typical value of what does (SPEC.md + // records getting that wrong once). A band whose measured rate + // tracks its predicted rate is a band the gate can be trusted + // in; one where measured >> predicted is model error localized + // there, which no amount of Wilson/sampling conservatism fixes. + { + static const char *szBand[PRUNE_VERIFY_BANDS] = + { "==0", "1", "2", "3-5", "6-10", "11-50", "51-500", ">500" }; + ULONG w; + Trace("\ndeep-prune shadow verification (AUDIT ONLY -- nothing " + "was actually pruned):\n"); + Trace(" predFH flagged actual-FH measured%% " + "predicted%% mean-margin\n"); + for (v = 0; v < PRUNE_VERIFY_BANDS; v++) + { + UINT64 u64T = 0, u64F = 0, u64M = 0, u64P = 0; + for (w = 0; w < SHIP_DEPTH_BUCKETS; w++) + { + u64T += g_SuiteCounters.u64TotalPruneVerifyTries[v][w]; + u64F += g_SuiteCounters.u64TotalPruneVerifyFailHigh[v][w]; + u64M += g_SuiteCounters.u64TotalPruneVerifyMarginSum[v][w]; + u64P += g_SuiteCounters.u64TotalPruneVerifyPredSum[v][w]; + } + if (0 == u64T) continue; + Trace(" %-8s %10" COMPILER_LONGLONG_UNSIGNED_FORMAT + " %10" COMPILER_LONGLONG_UNSIGNED_FORMAT + " %8.5f%% %8.5f%% %10.1f\n", + szBand[v], u64T, u64F, + 100.0 * (double)u64F / (double)u64T, + ((double)u64P / (double)u64T) / 1000.0, + u64F ? (double)u64M / (double)u64F : 0.0); + } + Trace(" (by remaining-depth bucket, flagged/actual-FH -- the COST " + "of a wrong\n prune scales with depth even where the RATE " + "does not)\n"); + for (w = 0; w < SHIP_DEPTH_BUCKETS; w++) + { + UINT64 u64T = 0, u64F = 0; + for (v = 0; v < PRUNE_VERIFY_BANDS; v++) + { + u64T += g_SuiteCounters.u64TotalPruneVerifyTries[v][w]; + u64F += g_SuiteCounters.u64TotalPruneVerifyFailHigh[v][w]; + } + if (0 == u64T) continue; + Trace(" depth bucket %2u: %10" COMPILER_LONGLONG_UNSIGNED_FORMAT + " / %" COMPILER_LONGLONG_UNSIGNED_FORMAT + " (%7.5f%%)\n", w, u64T, u64F, + 100.0 * (double)u64F / (double)u64T); + } + // Dynamic-veto cost/benefit, prunable bands only. For any + // rule "refuse when attempts >= A and rate > B", sum the + // Dyn cells it covers (sound prunes lost) against the + // DynMiss cells (real misses caught). + { + static const char *szA[8] = + { "0", "1", "2", "3-4", "5-9", "10-24", "25-99", "100+" }; + ULONG u; + Trace("\n dynamic signals over PRUNABLE-band candidates " + "(flagged / of-those-missed),\n rows = own-FH attempts, " + "cols = own-FH rate in thousandths of a percent:\n"); + Trace(" attempts 0 1-100 101-400 401-900 " + "901-1900 1901-4900 4901-9900 9901+\n"); + for (u = 0; u < 8; u++) + { + UINT64 u64RowT = 0; + for (v = 0; v < 8; v++) + u64RowT += g_SuiteCounters.u64TotalPruneVerifyDyn[u][v]; + if (0 == u64RowT) continue; + Trace(" %-8s", szA[u]); + for (v = 0; v < 8; v++) + { + Trace(" %7" COMPILER_LONGLONG_UNSIGNED_FORMAT + "/%-2" COMPILER_LONGLONG_UNSIGNED_FORMAT, + g_SuiteCounters.u64TotalPruneVerifyDyn[u][v], + g_SuiteCounters.u64TotalPruneVerifyDynMiss[u][v]); + } + Trace("\n"); + } + } + } +#endif + Trace("\nlate-move pruning (overt skip, not reduction): %" + COMPILER_LONGLONG_UNSIGNED_FORMAT " moves pruned\n", + g_SuiteCounters.u64TotalLMPPruned); + + } + +#ifdef JOINT_EXTRA_VERIFY + // "Silent miss" shadow-verification calibration: of the extra- + // reduced moves whose real score never triggered a re-search (the + // population the report block above CAN'T see into), how many + // would have failed high anyway if just the extra plies were + // removed? This is the direct answer to "how much unverified + // playing-strength loss is this mechanism causing" -- a real- + // search-cost-doubling measurement, JOINT_EXTRA_VERIFY builds only. + { + UINT64 u64TierTries, u64TierMiss; + ULONG w; + + Trace("\njoint-table extra-reduction SILENT MISS calibration " + "(shadow re-search, extra-plies-removed):\n"); + for (v = 1; v < JOINT_EXTRA_TIER_COUNT; v++) + { + u64TierTries = 0; + u64TierMiss = 0; + for (w = 0; w < POST_REDUCE_DEPTH_BUCKETS; w++) + { + u64TierTries += g_SuiteCounters.u64TotalJointExtraVerifyTries[v][w]; + u64TierMiss += g_SuiteCounters.u64TotalJointExtraVerifySilentMiss[v][w]; + } + Trace(" tier %u (+%u extra ply%s) : %5.2f%% silent-miss (%" + COMPILER_LONGLONG_UNSIGNED_FORMAT " tries), of which %5.2f%% " + "(%" COMPILER_LONGLONG_UNSIGNED_FORMAT ") match the null-move " + "bailout signature (iShadowScore == beta+1 exactly)\n", + v, v, (1 == v) ? "" : "s", + (100.0 * (double)u64TierMiss / ((double)u64TierTries + 1.0)), + u64TierTries, + (100.0 * (double)g_SuiteCounters.u64TotalJointExtraVerifyNullBailoutSignature[v] / + ((double)u64TierMiss + 1.0)), + g_SuiteCounters.u64TotalJointExtraVerifyNullBailoutSignature[v]); + } + + // Breakdown by EFP's own two-tier margin-schedule boundaries + // (search.c's _PostReduceDepthBucketEfpAligned) rather than + // plain whole-ply buckets -- tests whether silent misses + // concentrate specifically inside EFP's tier windows (a + // different pruning mechanism behaving differently there) or + // are spread uniformly by remaining depth alone. + { + static const char *rgszBucketLabel[POST_REDUCE_DEPTH_BUCKETS] = { + "< 1 ply (should not happen)", + "EFP tier1 (selective futility)", + "EFP tier2 (extended futility)", + "no EFP, nearby", + "no EFP, comfortably deep" + }; + Trace("\n ...broken down by EFP-tier-aligned remaining " + "depth window (is this an EFP-tier-window effect?):\n"); + for (v = 1; v < JOINT_EXTRA_TIER_COUNT; v++) + { + for (w = 0; w < POST_REDUCE_DEPTH_BUCKETS; w++) + { + Trace(" tier %u, %-32s : %5.2f%% silent-miss (%" + COMPILER_LONGLONG_UNSIGNED_FORMAT " tries)\n", + v, rgszBucketLabel[w], + (100.0 * (double)g_SuiteCounters.u64TotalJointExtraVerifySilentMiss[v][w] / + ((double)g_SuiteCounters.u64TotalJointExtraVerifyTries[v][w] + 1.0)), + g_SuiteCounters.u64TotalJointExtraVerifyTries[v][w]); + } + } + } + } +#endif + + // Joint 4D raw dump for offline reassembly/baking -- one line per + // populated cell, marginal (uncoarsened) indices, this checkpoint's + // cumulative totals (never summed across checkpoints by the bake + // script -- last checkpoint per shard is authoritative, matching + // every prior FHGAPRAW/FHJOINTRAW collection run in this project). + { + ULONG a, b, c; + for (a = 0; a < SEARCH_DEPTH_BUCKETS; a++) + { + for (b = 0; b < HANG_CP_BUCKETS; b++) + { + for (c = 0; c < GAP_CP_BUCKETS; c++) + { + for (v = 0; v < MOVE_NUM_BUCKETS; v++) + { + if (g_SuiteCounters.u64TotalJointTries[a][b][c][v] > 0) + { + Trace("FOURAXISRAW %lu %lu %lu %lu %" + COMPILER_LONGLONG_UNSIGNED_FORMAT " %" + COMPILER_LONGLONG_UNSIGNED_FORMAT "\n", + a, b, c, v, + g_SuiteCounters.u64TotalJointFH[a][b][c][v], + g_SuiteCounters.u64TotalJointTries[a][b][c][v]); + } + } + } + } + } + } + #ifdef TEST_NULL // Null-move precision/recall calibration, summed across the whole // suite (see _NullCalibBucketIndex's comment in search.c for the @@ -676,6 +1182,18 @@ Return value: FILE *p; static CHAR szLine[SMALL_STRING_LEN_CHAR]; double dSuiteStart; + ULONG uGoCount = 0; + // 2026-09-28: a wall-clock `timeout`-killed shard (e.g. a 90-minute + // sharded joint-table collection run) never reaches true end-of- + // script, so _PrintSuiteReport's single end-of-suite call would + // never fire and a killed shard would contribute zero data. Print + // the same (cumulative, non-resetting) report every + // CHECKPOINT_GO_INTERVAL positions too -- the FOURAXISRAW dump (and + // every other line in the report) always reflects the full running + // total since suite start, so the *last* checkpoint printed before + // a kill is a complete, valid snapshot on its own; a bake script + // just needs to find and use the last one, never sum across them. +#define CHECKPOINT_GO_INTERVAL (5) if (argc < 2) { @@ -696,11 +1214,21 @@ Return value: while(fgets(szLine, ARRAY_LENGTH(szLine), p)) { _ProcessSuiteLine(szLine, pos); + if (!STRNCMPI(szLine, "go", 2)) + { + uGoCount++; + if (0 == (uGoCount % CHECKPOINT_GO_INTERVAL)) + { + Trace("\nCHECKPOINT after %lu positions:\n", uGoCount); + _PrintSuiteReport(dSuiteStart); + } + } if (TRUE == g_fExitProgram) break; } fclose(p); _PrintSuiteReport(dSuiteStart); } +#undef CHECKPOINT_GO_INTERVAL } |
