summaryrefslogtreecommitdiff
path: root/src
diff options
context:
space:
mode:
authorScott Gasch <[email protected]>2026-09-25 08:44:25 -0700
committerScott Gasch <[email protected]>2026-09-25 08:44:25 -0700
commit9b28080f8b3dca4588cb2aaaf748451f45e3c626 (patch)
tree54e0aff5d6ee1520c590c7c04a5ae2f8299e7497 /src
parent66740dc89c7351c120c25788d0079055034f4942 (diff)
Two ideas:
1. Nullmove R=3 + depth/6 along with some ShouldWeDoNullmoveSearch tweaking and calibrating. 2. Probe EGTB everywhere in the search as long as the table/position is resident in memory.
Diffstat (limited to 'src')
-rw-r--r--src/CLAUDE.md76
-rwxr-xr-xsrc/chess.h102
-rwxr-xr-xsrc/eval_tune/test_vs_head.sh4
-rw-r--r--src/fathom/tbprobe.c236
-rw-r--r--src/fathom/tbprobe.h58
-rw-r--r--src/probe.c194
-rw-r--r--src/recogn.c234
-rw-r--r--src/root.c55
-rw-r--r--src/script.c137
-rw-r--r--src/search.c309
-rw-r--r--src/searchsup.c128
-rw-r--r--src/testrecogn.c273
12 files changed, 1629 insertions, 177 deletions
diff --git a/src/CLAUDE.md b/src/CLAUDE.md
index b545edd..0bfc028 100644
--- a/src/CLAUDE.md
+++ b/src/CLAUDE.md
@@ -162,13 +162,75 @@ an xboard-protocol REPL on stdin/stdout. Key commands used during development:
configure Syzygy tablebase lookup.
**Default EGTB path is hardcoded** in `main.c:297` to `/zscratch/egtb`, used
-whenever `--egtbpath` isn't passed. That directory currently has full 5-man
-WDL+DTZ Syzygy coverage but is **missing the WDL half of its 6-man set** (only
-`.rtbz`/DTZ files exist for 6-man combos) -- `init_tb()` in
-`fathom/tbprobe.c:752` requires the WDL file to exist before registering a
-combination at all, so every 6-man entry is silently skipped and the engine
-reports "up to 5 men" even though 6-man `.rtbz` files are sitting right there.
-Not a config issue; the fix is downloading the missing 6-man `.rtbw` files.
+whenever `--egtbpath` isn't passed. That directory now has **full 5-man and
+6-man WDL+DTZ Syzygy coverage** (the earlier gap -- 6-man `.rtbz`/DTZ files
+present but the `.rtbw`/WDL half missing, so `init_tb()`'s
+`fathom/tbprobe.c:752` WDL-file-required check silently skipped every 6-man
+combination and the engine reported "up to 5 men" -- has been closed by
+downloading the missing 6-man `.rtbw` files). `TB_LARGEST` should now read 6
+at startup.
+
+### Predicting whether a deep probe will be cheap (memory) vs. expensive (disk)
+
+Historically the engine only probed the EGTB shallow/near-root (`search.c`'s
+`RecognLookup` call gated its 3rd arg, `fProbeEGTB`, on
+`ctx->uPly <= (g_uIterateDepth / 2)`, and QSearch's call hardcoded `FALSE`
+-- never probed at all in qsearch) specifically to avoid stalling the tree
+search on a disk read near the leaves, where nodes are cheap and plentiful
+and a single blocking read would be ruinously expensive relatively speaking.
+
+That blanket avoidance left value on the table: once a few pieces have
+traded off near a leaf, a position is often *already* covered by a
+tablebase whose file was mapped into memory by an earlier, shallower probe
+this same search -- Fathom (`fathom/tbprobe.c`) lazily `mmap()`s + parses
+each material signature's `.rtbw`/`.rtbz` file exactly once per process,
+on first probe, behind a double-checked-locking `atomic_bool
+be->ready[type]` (`probe_table`, `tbprobe.c:1778`). Once `ready[WDL]` is
+true for a given material signature, subsequent probes of that signature
+are just memory/page-cache lookups, not disk reads -- but the old
+depth-based gate couldn't tell the difference between "warm" and "cold"
+material and refused all deep probes uniformly.
+
+Fixed by adding `tb_wdl_is_resident()` (`fathom/tbprobe.c`, declared in
+`fathom/tbprobe.h`) -- a non-blocking check that recomputes the same
+material-signature key Fathom's own `calc_key()` uses (pure per-side
+piece-type popcounts, no square/turn/ep info needed, so no board
+conversion required) and reads `be->ready[WDL]` for that signature. No
+lock, no file I/O, just a handful of popcounts + one atomic load; safe to
+call from every leaf-ish node. Wrapped as `EgtbProbeLikelyResident(ctx)`
+in `probe.c` (mirrors `ProbeEGTB`'s castling/fifty-move/cardinality
+gating, so it never claims a probe is "likely resident" that `ProbeEGTB`
+itself would refuse outright). `search.c`'s two `RecognLookup` call sites
+now pass `(ctx->uPly <= (g_uIterateDepth / 2)) || EgtbProbeLikelyResident(ctx)`
+(main search) and `EgtbProbeLikelyResident(ctx)` (QSearch, which
+previously never probed at all) instead of the old ply-only / hardcoded
+`FALSE` gates.
+
+**Caveat**: `ready[WDL] == true` only proves the file's header/index
+metadata was parsed, not that every compressed data page this specific
+position's lookup will touch is already resident in the OS page cache.
+For the small 3-5 man files that distinction is essentially moot in
+practice (the whole file tends to end up page-cached after the first
+touch); for large 6-man files it's a weaker guarantee -- a `ready`
+6-man table can still have cold pages. Not yet measured empirically
+whether this matters in practice; if deep-probe latency spikes show up
+in profiling, this is the first place to look.
+
+**Not yet done, a natural follow-up**: root-time warm-up -- proactively
+touching (probing) WDL tables for material signatures reachable within a
+few plies of likely trades from the root position, before the main
+search starts, so `ready[WDL]` is already true for most of what the deep
+search will actually encounter rather than depending on an earlier
+shallow-ply probe in the *same* search to have warmed it first.
+
+**Also worth knowing**: `ProbeEGTB` wraps the actual `tb_probe_wdl` call
+in a single process-wide spinlock (`g_uEgtbLock`, `probe.c`). Increasing
+probe frequency deep in the tree (now possible via the above) increases
+contention on that spinlock under `MP=1` multithreaded search even when
+every probe individually is memory-cheap -- not yet measured whether this
+becomes a bottleneck in practice; if throughput regresses under multiple
+threads after this change, check spinlock contention before assuming a
+disk-latency cause.
## Testing: the ECM tactical suite
diff --git a/src/chess.h b/src/chess.h
index 72289ad..ff218a9 100755
--- a/src/chess.h
+++ b/src/chess.h
@@ -941,6 +941,36 @@ MOVE_STACK;
// size the evidence-bucket counter arrays.
#define CM_EVIDENCE_BUCKETS (7)
+// TEST_NULL calibration harness (search.c's null-move precision/recall/
+// EV measurement -- see _NullCalibBucketIndex's comment in search.c for
+// the full design). Revised 2026-09-24: dropped fThreat/fSkipNull/
+// en-prise-or-trapped as bucket dimensions -- all three are already
+// validated, non-tunable binary production gates (fThreat/fSkipNull
+// gate the call to WeShouldTryNullmovePruning in search.c; en-prise/
+// trapped was shown NOT to predict false-positive risk, only recall),
+// so the calibration probe now just requires fThreat==FALSE &&
+// fSkipNull==FALSE as part of structural eligibility instead of
+// spending bucket space re-measuring them. Added a second depth
+// dimension (outer, pre-reduction uDepth, alongside the null probe's
+// own post-reduction uNullDepth) and a second material dimension
+// (mover-only piece count -- TotalNonPawnCount(pos, pos->uToMove) --
+// alongside the existing MAX(white,black) count) -- both deliberately
+// independent of the current SelectNullmoveRFactor R schedule, so a
+// cost model fit from this data stays valid if R changes.
+//
+// Per-ply, 0..15 directly plus one "16+" overflow bucket -- fine enough
+// to fit a cost-vs-depth growth curve (EBF-style) and extrapolate to
+// depths never directly observed at the sd used for a given run. Used
+// for both depth dimensions (same banding, reused).
+#define NC_DEPTH_BUCKETS (17)
+#define NC_DEFICIT_BUCKETS (6) // (iEval-iBeta) cp bands
+// Non-pawn piece count bands: 0-1,2,3-5,6+. Used for both material
+// dimensions (same banding, reused).
+#define NC_MATERIAL_BUCKETS (4)
+#define NC_TOTAL_BUCKETS \
+ (NC_DEPTH_BUCKETS * NC_DEPTH_BUCKETS * NC_DEFICIT_BUCKETS * \
+ NC_MATERIAL_BUCKETS * NC_MATERIAL_BUCKETS)
+
// ----------------------------------------------------------------------
//
// Accumulators
@@ -1018,11 +1048,35 @@ typedef struct _COUNTERS
UINT64 u64NullMoves;
UINT64 u64NullMoveSuccess;
#ifdef TEST_NULL
- UINT64 u64QuickNullSuccess;
- UINT64 u64QuickNullDeferredSuccess;
- UINT64 u64QuickNullFailures;
- UINT64 u64AvoidNullSuccess;
- UINT64 u64AvoidNullFailures;
+ // Precision/recall calibration for the null-move try/avoid
+ // decision. Indexed by _NullCalibBucketIndex() (search.c).
+ // Per cell: how often we attempted a (diagnostic, always-run)
+ // null-move probe, how often the probe itself failed high, how
+ // often the real move loop at that node *actually* failed high
+ // (ground truth, read off Search()'s own iBestScore vs iBeta at
+ // its "end:" label), and how often both agreed. From these four
+ // counts: precision = uBothFH/uNullFH, recall = uBothFH/uRealFH.
+ UINT64 u64NCAttempts[NC_TOTAL_BUCKETS];
+ UINT64 u64NCNullFH[NC_TOTAL_BUCKETS];
+ UINT64 u64NCRealFH[NC_TOTAL_BUCKETS];
+ UINT64 u64NCBothFH[NC_TOTAL_BUCKETS];
+ // Cost side of the try/skip decision: sum (not average -- divide
+ // by u64NCAttempts at report time) of the node cost of the
+ // diagnostic null probe itself, and of the immediately-following
+ // real first move's full-depth search (the "R=0" alternative),
+ // per bucket. Lets a report compute avg-nodes-under-null vs.
+ // avg-nodes-under-first-move and the try-vs-skip break-even
+ // this implies, given the bucket's own nullFH rate.
+ UINT64 u64NCNullProbeNodeSum[NC_TOTAL_BUCKETS];
+ UINT64 u64NCFirstMoveNodeSum[NC_TOTAL_BUCKETS];
+ // True total cost of however the real search actually resolved
+ // this node (one move, several moves, or production's own
+ // separate null-move success) -- node count from right after
+ // the diagnostic probe returns through this node's "end:".
+ // Strictly more honest than u64NCFirstMoveNodeSum for the
+ // try-vs-skip EV calc, which only approximates "cost of the
+ // real alternative" as move 1 alone.
+ UINT64 u64NCRealLoopNodeSum[NC_TOTAL_BUCKETS];
#endif
UINT64 u64LazyEvals;
UINT64 u64SuperLazyEvals;
@@ -1652,6 +1706,15 @@ typedef struct _GAME_OPTIONS
UINT64 u64LeftoverFH;
UINT64 u64CMEvidenceTries[CM_EVIDENCE_BUCKETS];
UINT64 u64CMEvidenceFH[CM_EVIDENCE_BUCKETS];
+#ifdef TEST_NULL
+ UINT64 u64NCAttempts[NC_TOTAL_BUCKETS];
+ UINT64 u64NCNullFH[NC_TOTAL_BUCKETS];
+ UINT64 u64NCRealFH[NC_TOTAL_BUCKETS];
+ UINT64 u64NCBothFH[NC_TOTAL_BUCKETS];
+ UINT64 u64NCNullProbeNodeSum[NC_TOTAL_BUCKETS];
+ UINT64 u64NCFirstMoveNodeSum[NC_TOTAL_BUCKETS];
+ UINT64 u64NCRealLoopNodeSum[NC_TOTAL_BUCKETS];
+#endif
CHAR szLogfile[SMALL_STRING_LEN_CHAR];
CHAR szEGTBPath[SMALL_STRING_LEN_CHAR];
CHAR szBookName[SMALL_STRING_LEN_CHAR];
@@ -4225,6 +4288,9 @@ void
TestRecognExhaustiveKNKP(void);
void
+TestRecognExhaustiveKQKP(void);
+
+void
TestRecognExhaustiveKPK(void);
void
@@ -4254,6 +4320,9 @@ ProbeEGTB(SEARCHER_THREAD_CONTEXT *ctx, SCORE *score);
FLAG
ProbeEGTBRoot(SEARCHER_THREAD_CONTEXT *ctx, MOVE *pmv, SCORE *piScore);
+FLAG
+EgtbProbeLikelyResident(SEARCHER_THREAD_CONTEXT *ctx);
+
void
InitializeEGTB(void);
@@ -4324,13 +4393,30 @@ GenerateRandomLegalSymetricPosition(POSITION *pos);
#define RECOGN_LOWER (3)
#define RECOGN_EGTB (4)
+//
+// EGTB probing mode for RecognLookup's 3rd argument -- trinary rather
+// than a plain FLAG so "probe only if we believe the page is resident"
+// is a distinct option from "always probe" and "never probe". Letting
+// RecognLookup itself decide (rather than making the caller compute
+// EgtbProbeLikelyResident up front, unconditionally, as a function
+// argument) means the residency check -- which can now involve real
+// mincore() syscalls for 6-man tables, see EgtbProbeLikelyResident's
+// own comment -- is only ever paid on the branch that's actually about
+// to decide whether to call ProbeEGTB, i.e. skipped entirely at every
+// node one of the cheap hand-written interior recognizers already
+// resolves. See CLAUDE.md's EGTB residency section.
+//
+#define EGTB_PROBE_NEVER (0)
+#define EGTB_PROBE_ALWAYS (1)
+#define EGTB_PROBE_IF_RESIDENT (2)
+
void
InitializeInteriorNodeRecognizers(void);
ULONG
RecognLookup(SEARCHER_THREAD_CONTEXT *ctx,
SCORE *piScore,
- FLAG fProbeEGTB);
+ ULONG uEgtbProbeMode);
//
// Shared, ctx-free KP+KP* "unstoppable passer" static analysis --
@@ -4368,4 +4454,8 @@ ULONG
_RecognizeKNKP(SEARCHER_THREAD_CONTEXT *ctx,
SCORE *piScore);
+ULONG
+_RecognizeKQKP(SEARCHER_THREAD_CONTEXT *ctx,
+ SCORE *piScore);
+
#endif // CHESS
diff --git a/src/eval_tune/test_vs_head.sh b/src/eval_tune/test_vs_head.sh
index e937389..d6d9fba 100755
--- a/src/eval_tune/test_vs_head.sh
+++ b/src/eval_tune/test_vs_head.sh
@@ -48,9 +48,9 @@ python3 ./match_play.py \
${HEAD} \
${CANDIDATE} \
--pgn ../../pgn/twic_filtered.pgn \
- --games 20000 \
+ --games 10000 \
--workers 12 \
- --st 1 \
+ --st 4 \
--sprt --elo0 0 --elo1 5 \
--scratch /usr/local/tmp/typhoon_match_${LABEL} \
--log ${LABEL}.log \
diff --git a/src/fathom/tbprobe.c b/src/fathom/tbprobe.c
index 5eac014..9cce02d 100644
--- a/src/fathom/tbprobe.c
+++ b/src/fathom/tbprobe.c
@@ -1550,6 +1550,24 @@ static bool init_table(struct BaseEntry *be, const char *str, int type)
be->data[type] = data;
+#ifndef _WIN32
+ if (type == WDL && be->num <= 5) {
+ // 3-5 man WDL files are small (empirically <=20MB each, <=0.4GB total
+ // across every 3-5 man signature -- see CLAUDE.md's EGTB residency
+ // section), so instead of inferring residency later from whether this
+ // header parse has run (a guess that's only reliable because these
+ // files are small enough to end up fully cached anyway), just wire
+ // the whole mapping into RAM now. Once ready[WDL] is true for one of
+ // these signatures, tb_wdl_is_resident()'s "yes" becomes a guarantee
+ // instead of a proxy -- no mincore() needed for this tier at all.
+ if (mlock(data, (size_t)be->mapping[type]) != 0) {
+ perror("mlock");
+ // Not fatal -- this signature's residency answer just reverts to
+ // being an inference (like the 6-man case) instead of a guarantee.
+ }
+ }
+#endif
+
bool split = type != DTZ && (data[4] & 0x01);
if (type == DTM)
be->dtmLossOnly = data[4] & 0x04;
@@ -1900,6 +1918,224 @@ static int probe_table(const Pos *pos, int s, int *success, const int type)
return v;
}
+// Cheap, non-blocking check for whether the WDL table covering a given
+// material signature is already mapped into memory (i.e. init_table()'s
+// one-time mmap()+header-parse has already run for this piece
+// combination, via some earlier probe_table() call this process).
+//
+// This deliberately duplicates probe_table()'s key computation and
+// tbHash[] walk instead of calling it, so that callers can ask "would a
+// probe of this position be fast?" from deep in the search tree without
+// ever taking tbMutex or touching a byte of the underlying file -- it's
+// a handful of popcounts plus one atomic_bool load.
+//
+// Returns:
+// 1 the material signature has a WDL table and it's already resident
+// (mapped + header parsed) -- a probe here should be near-free.
+// 0 the material signature has a WDL table but it hasn't been
+// touched by any probe yet this run -- a probe here will pay the
+// one-time mmap()+parse cost (and is still lazily-mapped, so may
+// also fault in fresh pages from disk for the data itself).
+// -1 no WDL table covers this material signature at all (either it's
+// outside the largest cardinality Fathom found on disk, or it's
+// KvK / otherwise not tabulated).
+//
+// NOTE: a return of 1 only guarantees the table's *header and index
+// metadata* are parsed, not that every compressed data page this
+// specific position's lookup will touch is already in the OS page
+// cache. For the small 3-5 man files that's essentially the same thing
+// in practice (the whole file ends up cached after the first touch);
+// for large 6-man files it's a weaker guarantee. See CLAUDE.md.
+int tb_wdl_is_resident(uint64_t white, uint64_t black, uint64_t queens,
+ uint64_t rooks, uint64_t bishops, uint64_t knights,
+ uint64_t pawns)
+{
+ uint64_t key = popcount(white & queens) * PRIME_WHITE_QUEEN +
+ popcount(white & rooks) * PRIME_WHITE_ROOK +
+ popcount(white & bishops) * PRIME_WHITE_BISHOP +
+ popcount(white & knights) * PRIME_WHITE_KNIGHT +
+ popcount(white & pawns) * PRIME_WHITE_PAWN +
+ popcount(black & queens) * PRIME_BLACK_QUEEN +
+ popcount(black & rooks) * PRIME_BLACK_ROOK +
+ popcount(black & bishops) * PRIME_BLACK_BISHOP +
+ popcount(black & knights) * PRIME_BLACK_KNIGHT +
+ popcount(black & pawns) * PRIME_BLACK_PAWN;
+ if (key == 0ULL) return -1; // KvK: not a tabulated material signature
+
+ int hashIdx = (int)(key >> (64 - TB_HASHBITS));
+ while (tbHash[hashIdx].key && tbHash[hashIdx].key != key)
+ hashIdx = (hashIdx + 1) & ((1 << TB_HASHBITS) - 1);
+ if (!tbHash[hashIdx].ptr ||
+ atomic_load_explicit(&tbHash[hashIdx].error, memory_order_relaxed))
+ return -1;
+
+ struct BaseEntry *be = tbHash[hashIdx].ptr;
+ return atomic_load_explicit(&be->ready[WDL], memory_order_relaxed) ? 1 : 0;
+}
+
+#ifndef _WIN32
+// Page-residency predictor for the 6-man tier, where init_table() can't
+// just mlock() the whole file (some signatures run into the hundreds of
+// MB to low GB -- see CLAUDE.md). Every dereference below is gated by a
+// mincore() check first, so a call to this function never itself blocks
+// on disk I/O: it either confirms every page a real probe_table() lookup
+// would touch is already resident (and, having just read them, returns a
+// verified "yes"), or it stops at the first non-resident page and
+// returns "no" without reading past it.
+//
+// A small direct-mapped cache remembers pages already confirmed resident
+// so repeated probes into the same hot region of a table don't re-pay a
+// mincore() syscall every time. It deliberately only caches "yes"
+// answers -- a page that mincore() reports as not-yet-resident is always
+// re-checked for real next time, so a page that warms up between calls
+// is picked up on its next probe rather than being stuck on a stale "no".
+#define TB_RESIDENCY_PAGE_SIZE 4096u
+#define TB_RESIDENCY_CACHE_BITS 12
+#define TB_RESIDENCY_CACHE_SIZE (1u << TB_RESIDENCY_CACHE_BITS)
+static atomic_uintptr_t g_ResidencyCache[TB_RESIDENCY_CACHE_SIZE];
+
+static size_t _ResidencyCacheSlot(uintptr_t pageAddr)
+{
+ uintptr_t h = (pageAddr >> 12) * 2654435761u;
+ return (size_t)(h & (TB_RESIDENCY_CACHE_SIZE - 1));
+}
+
+static int _PageIsResident(uintptr_t pageAddr)
+{
+ size_t slot = _ResidencyCacheSlot(pageAddr);
+ if (atomic_load_explicit(&g_ResidencyCache[slot], memory_order_relaxed) == pageAddr)
+ return 1;
+
+ unsigned char vec = 0;
+ if (mincore((void *)pageAddr, TB_RESIDENCY_PAGE_SIZE, (char *)&vec) != 0)
+ return 0; // shouldn't happen for a page inside a live mapping; be conservative
+
+ if (vec & 1) {
+ atomic_store_explicit(&g_ResidencyCache[slot], pageAddr, memory_order_relaxed);
+ return 1;
+ }
+ return 0;
+}
+
+static int _RangeIsResident(const void *addr, size_t len)
+{
+ uintptr_t start = (uintptr_t)addr & ~(uintptr_t)(TB_RESIDENCY_PAGE_SIZE - 1);
+ uintptr_t end = ((uintptr_t)addr + len + TB_RESIDENCY_PAGE_SIZE - 1) &
+ ~(uintptr_t)(TB_RESIDENCY_PAGE_SIZE - 1);
+ for (uintptr_t p = start; p < end; p += TB_RESIDENCY_PAGE_SIZE)
+ if (!_PageIsResident(p))
+ return 0;
+ return 1;
+}
+
+// Precise, non-blocking "would a real WDL probe of this exact position
+// hit resident pages only?" check. Unlike tb_wdl_is_resident() (which
+// only knows whether a material signature's header has been parsed at
+// all), this computes the same indexTable/sizeTable/data-block addresses
+// decompress_pairs() would touch for this specific position and confirms
+// each one is actually in the OS page cache before trusting the next.
+//
+// RETURN: 1 if a subsequent real probe is expected to be fully resident
+// (no page fault / disk read), 0 otherwise (table not loaded, material
+// untabulated, or some page on the lookup path isn't resident yet).
+int tb_wdl_probe_is_resident(uint64_t white, uint64_t black, uint64_t kings,
+ uint64_t queens, uint64_t rooks,
+ uint64_t bishops, uint64_t knights,
+ uint64_t pawns, bool turn)
+{
+ Pos pos = { white, black, kings, queens, rooks, bishops, knights, pawns,
+ 0, 0, turn };
+
+ uint64_t key = calc_key(&pos, false);
+ if (key == 0ULL)
+ return 0;
+
+ int hashIdx = (int)(key >> (64 - TB_HASHBITS));
+ while (tbHash[hashIdx].key && tbHash[hashIdx].key != key)
+ hashIdx = (hashIdx + 1) & ((1 << TB_HASHBITS) - 1);
+ if (!tbHash[hashIdx].ptr ||
+ atomic_load_explicit(&tbHash[hashIdx].error, memory_order_relaxed))
+ return 0;
+
+ struct BaseEntry *be = tbHash[hashIdx].ptr;
+ if (!atomic_load_explicit(&be->ready[WDL], memory_order_relaxed))
+ return 0;
+
+ // 3-5 man WDL files are mlock()'d whole at load time (init_table()) --
+ // ready[WDL] already IS the guarantee for this tier, no page check
+ // needed.
+ if (be->num <= 5)
+ return 1;
+
+ bool bside, flip;
+ if (!be->symmetric) {
+ flip = key != be->key;
+ bside = (pos.turn == WHITE) == flip;
+ } else {
+ flip = pos.turn != WHITE;
+ bside = false;
+ }
+
+ struct EncInfo *ei = first_ei(be, WDL);
+ int p[TB_PIECES];
+ size_t idx;
+
+ if (!be->hasPawns) {
+ ei = &ei[bside];
+ for (int i = 0; i < be->num;)
+ i = fill_squares(&pos, ei->pieces, flip, 0, p, i);
+ idx = encode_piece(p, ei, be);
+ } else {
+ int i = fill_squares(&pos, ei->pieces, flip, flip ? 0x38 : 0, p, 0);
+ int t = leading_pawn(p, be, FILE_ENC);
+ ei = &ei[t + 4 * bside];
+ while (i < be->num)
+ i = fill_squares(&pos, ei->pieces, flip, flip ? 0x38 : 0, p, i);
+ idx = encode_pawn_f(p, ei, be);
+ }
+
+ struct PairsData *d = ei->precomp;
+ if (!d->idxBits)
+ return 1; // constant-value table -- no data page to check at all
+
+ uint32_t mainIdx = (uint32_t)(idx >> d->idxBits);
+ uint8_t *idxEntry = d->indexTable + 6 * (size_t)mainIdx;
+ if (!_RangeIsResident(idxEntry, 6))
+ return 0;
+
+ uint32_t block;
+ memcpy(&block, idxEntry, sizeof(block));
+ block = from_le_u32(block);
+ uint16_t idxOffset = *(uint16_t *)(idxEntry + 4);
+
+ int litIdx = (int)((idx & (((size_t)1 << d->idxBits) - 1)) -
+ ((size_t)1 << (d->idxBits - 1)));
+ litIdx += from_le_u16(idxOffset);
+
+ if (litIdx < 0) {
+ while (litIdx < 0) {
+ uint32_t prev = block - 1;
+ if (!_RangeIsResident(&d->sizeTable[prev], sizeof(uint16_t)))
+ return 0;
+ block = prev;
+ litIdx += d->sizeTable[block] + 1;
+ }
+ } else {
+ if (!_RangeIsResident(&d->sizeTable[block], sizeof(uint16_t)))
+ return 0;
+ while (litIdx > d->sizeTable[block]) {
+ litIdx -= d->sizeTable[block] + 1;
+ block++;
+ if (!_RangeIsResident(&d->sizeTable[block], sizeof(uint16_t)))
+ return 0;
+ }
+ }
+
+ uint8_t *dataBlock = d->data + ((size_t)block << d->blockSize);
+ return _RangeIsResident(dataBlock, sizeof(uint32_t) * 2);
+}
+#endif // !_WIN32
+
static int probe_wdl_table(const Pos *pos, int *success)
{
return probe_table(pos, 0, success, WDL);
diff --git a/src/fathom/tbprobe.h b/src/fathom/tbprobe.h
index 82850e7..6f4e5e0 100644
--- a/src/fathom/tbprobe.h
+++ b/src/fathom/tbprobe.h
@@ -181,6 +181,64 @@ bool tb_init(const char *_path);
void tb_free(void);
/*
+ * Cheap, non-blocking check for whether the WDL table covering a given
+ * material signature (per-side piece-type bitboards, no square/turn/ep
+ * info needed) is already mapped into memory from an earlier probe this
+ * process -- i.e. whether a probe of a position with this material is
+ * likely to be low-latency versus pay a first-touch mmap()+parse (and
+ * possibly disk-read) cost. See the routine description in tbprobe.c for
+ * the full contract and caveats.
+ *
+ * PARAMETERS:
+ * - white, black, queens, rooks, bishops, knights, pawns:
+ * Per-side piece-type bitboards (kings/turn/ep are irrelevant to the
+ * material-signature key and are omitted).
+ *
+ * RETURN:
+ * - 1: table exists and is already resident (fast probe expected).
+ * - 0: table exists but hasn't been touched yet this run (first probe
+ * will pay the one-time mmap()+header-parse cost).
+ * - -1: no WDL table covers this material signature at all.
+ */
+int tb_wdl_is_resident(uint64_t white, uint64_t black, uint64_t queens,
+ uint64_t rooks, uint64_t bishops, uint64_t knights,
+ uint64_t pawns);
+
+/*
+ * Precise, non-blocking check for whether a WDL probe of this EXACT
+ * position (not just its material signature) would hit only pages
+ * already resident in the OS page cache. Unlike tb_wdl_is_resident()
+ * above -- which only knows whether the table's header/index metadata
+ * has been parsed at all, a weak proxy for large 6-man files -- this
+ * walks the same indexTable/sizeTable/compressed-data-block addresses a
+ * real probe would touch and confirms each one via mincore() before
+ * trusting the next, so it never blocks on disk I/O itself. Small (3-5
+ * man) WDL files are mlock()'d whole at load time (see init_table() in
+ * tbprobe.c), so for that tier this degenerates to the same
+ * header-parsed check as tb_wdl_is_resident() -- a guarantee, not a
+ * guess, for that tier specifically.
+ *
+ * PARAMETERS:
+ * - white, black, kings, queens, rooks, bishops, knights, pawns:
+ * The current position (bitboards), matching tb_probe_wdl's layout.
+ * - turn: true if White is to move.
+ *
+ * RETURN:
+ * - 1: a real probe here is expected to touch only resident pages.
+ * - 0: table not loaded, material untabulated, or some page on the
+ * lookup path isn't resident yet.
+ *
+ * Not available under _WIN32 (mincore() has no portable equivalent
+ * wired up there yet).
+ */
+#ifndef _WIN32
+int tb_wdl_probe_is_resident(uint64_t white, uint64_t black, uint64_t kings,
+ uint64_t queens, uint64_t rooks,
+ uint64_t bishops, uint64_t knights,
+ uint64_t pawns, bool turn);
+#endif
+
+/*
* Probe the Win-Draw-Loss (WDL) table.
*
* PARAMETERS:
diff --git a/src/probe.c b/src/probe.c
index 2bddf99..269a1b9 100644
--- a/src/probe.c
+++ b/src/probe.c
@@ -138,10 +138,25 @@ Routine description:
Build the set of bitboards Fathom's tb_probe_wdl wants out of our
POSITION. Fathom expects the standard tablebase square numbering
- (a1 == bit 0 ... h8 == bit 63), which is exactly what our TO64()
- macro produces -- note this is *not* the same square numbering as
- our engine's own internal BBSQUARE/COOR_TO_BB bitboards, so we
- must not use those macros here.
+ (a1 == bit 0 ... h8 == bit 63); our own internal bitboards
+ (bbPieces/bbPawns/bbOccupiedSide) instead index bit = rankTopDown*8
+ + file (rankTopDown 0 == rank 8 ... 7 == rank 1 -- see TO64's own
+ (7 - (x>>4)) flip). Those two numberings differ by exactly one
+ 64-bit byte-swap: internal bit r*8+f (byte r, offset f) lands at
+ Fathom bit (7-r)*8+f (byte 7-r, offset f) under bswap64, matching
+ TO64's formula term for term. So this can OR our own bitboards
+ together and byte-swap the result directly -- O(1) per output
+ bitboard -- instead of walking every piece one at a time through
+ CoorFromBitBoardRank8ToRank1/TO64. (2026-09-24: replaced the
+ original per-piece-popping version, which showed up as a real
+ self-play NPS/depth regression once EgtbProbeLikelyResident started
+ calling this from QSearch's hot path -- see CLAUDE.md.)
+
+ No bbPieces[color][KING] exists (deliberately -- see its own
+ comment), so the king bitboard is built from the single square in
+ cKingSq[color] instead; equivalent to bswapping a bitboard with only
+ that one bit set, since bswap64 is just a bit permutation and
+ commutes with OR.
Parameters:
@@ -154,75 +169,22 @@ Return value:
**/
{
- ULONG uColor;
- COOR c;
- UINT64 bb;
-
- *pu64White = *pu64Black = 0ULL;
- *pu64Kings = *pu64Queens = *pu64Rooks = 0ULL;
- *pu64Bishops = *pu64Knights = *pu64Pawns = 0ULL;
-
- for (uColor = BLACK; uColor <= WHITE; uColor++)
- {
- UINT64 *pu64Side = (uColor == WHITE) ? pu64White : pu64Black;
-
- {
- BITBOARD bbPawns = pos->bbPawns[uColor];
-
- while (IS_ON_BOARD(c = CoorFromBitBoardRank8ToRank1(&bbPawns)))
- {
- bb = (1ULL << (TO64(c)));
- *pu64Side |= bb;
- *pu64Pawns |= bb;
- }
- }
-
- {
- c = KingPosition(pos, uColor);
- ASSERT(IS_ON_BOARD(c));
- ASSERT(IS_KING(pos->rgSquare[c].pPiece));
- bb = (1ULL << (TO64(c)));
- *pu64Side |= bb;
- *pu64Kings |= bb;
- }
-
- {
- BITBOARD bbType;
- COOR cType;
-
- bbType = pos->bbPieces[uColor][KNIGHT];
- while (IS_ON_BOARD(cType = CoorFromBitBoardRank8ToRank1(&bbType)))
- {
- bb = (1ULL << (TO64(cType)));
- *pu64Side |= bb;
- *pu64Knights |= bb;
- }
+ UINT64 u64Kings = (1ULL << COOR_TO_BIT_NUMBER(pos->cKingSq[WHITE])) |
+ (1ULL << COOR_TO_BIT_NUMBER(pos->cKingSq[BLACK]));
- bbType = pos->bbPieces[uColor][BISHOP];
- while (IS_ON_BOARD(cType = CoorFromBitBoardRank8ToRank1(&bbType)))
- {
- bb = (1ULL << (TO64(cType)));
- *pu64Side |= bb;
- *pu64Bishops |= bb;
- }
-
- bbType = pos->bbPieces[uColor][ROOK];
- while (IS_ON_BOARD(cType = CoorFromBitBoardRank8ToRank1(&bbType)))
- {
- bb = (1ULL << (TO64(cType)));
- *pu64Side |= bb;
- *pu64Rooks |= bb;
- }
-
- bbType = pos->bbPieces[uColor][QUEEN];
- while (IS_ON_BOARD(cType = CoorFromBitBoardRank8ToRank1(&bbType)))
- {
- bb = (1ULL << (TO64(cType)));
- *pu64Side |= bb;
- *pu64Queens |= bb;
- }
- }
- }
+ *pu64White = __builtin_bswap64((UINT64)pos->bbOccupiedSide[WHITE]);
+ *pu64Black = __builtin_bswap64((UINT64)pos->bbOccupiedSide[BLACK]);
+ *pu64Kings = __builtin_bswap64(u64Kings);
+ *pu64Queens = __builtin_bswap64(
+ (UINT64)(pos->bbPieces[WHITE][QUEEN] | pos->bbPieces[BLACK][QUEEN]));
+ *pu64Rooks = __builtin_bswap64(
+ (UINT64)(pos->bbPieces[WHITE][ROOK] | pos->bbPieces[BLACK][ROOK]));
+ *pu64Bishops = __builtin_bswap64(
+ (UINT64)(pos->bbPieces[WHITE][BISHOP] | pos->bbPieces[BLACK][BISHOP]));
+ *pu64Knights = __builtin_bswap64(
+ (UINT64)(pos->bbPieces[WHITE][KNIGHT] | pos->bbPieces[BLACK][KNIGHT]));
+ *pu64Pawns = __builtin_bswap64(
+ (UINT64)(pos->bbPawns[WHITE] | pos->bbPawns[BLACK]));
}
@@ -324,6 +286,72 @@ Return value:
}
+FLAG
+EgtbProbeLikelyResident(SEARCHER_THREAD_CONTEXT *ctx)
+/**
+
+Routine description:
+
+ Cheap, non-blocking predictor for "would an EGTB probe of the
+ current position be low latency?" -- used to let deep search/qsearch
+ nodes probe the EGTB (which is normally reserved for shallow, near-
+ root plies to avoid stalling the tree search on a disk read) once
+ the specific pages this exact position's lookup needs are already
+ resident, whether from an earlier probe this run or (for 3-5 man
+ tables) because they were mlock()'d in whole at load time.
+
+ Does not itself probe the tablebase, take g_uEgtbLock, or read any
+ file data that isn't already confirmed resident first -- see
+ tb_wdl_probe_is_resident in fathom/tbprobe.c for the mincore()-gated
+ walk this performs (indexTable/sizeTable/data-block addresses for
+ 6-man tables; a single ready[] check for the mlock()'d 3-5 man
+ tier). Safe to call from every leaf-ish node without worrying about
+ search slowdown.
+
+ Same castling/fifty-move/cardinality gating as ProbeEGTB (a probe
+ that ProbeEGTB itself would refuse is never "likely resident").
+
+Parameters:
+
+ SEARCHER_THREAD_CONTEXT *ctx
+
+Return value:
+
+ FLAG -- TRUE iff a subsequent ProbeEGTB call for this exact position
+ is expected to hit already-resident tablebase data rather than pay a
+ first-touch mmap()/parse (and possibly disk-read) cost.
+
+**/
+{
+ POSITION *pos = &(ctx->sPosition);
+ UINT64 u64White, u64Black, u64Kings, u64Queens;
+ UINT64 u64Rooks, u64Bishops, u64Knights, u64Pawns;
+ ULONG wcount, bcount;
+
+ if (FALSE == g_fEgtbInitialized || 0 == TB_LARGEST)
+ {
+ return(FALSE);
+ }
+ if ((0 != pos->bvCastleInfo) || (0 != pos->uFifty))
+ {
+ return(FALSE);
+ }
+ wcount = TotalNonPawnCount(pos, WHITE) + FastCountBits(pos->bbPawns[WHITE]);
+ bcount = TotalNonPawnCount(pos, BLACK) + FastCountBits(pos->bbPawns[BLACK]);
+ if ((wcount + bcount) > TB_LARGEST)
+ {
+ return(FALSE);
+ }
+
+ _BuildFathomBitboards(pos, &u64White, &u64Black, &u64Kings, &u64Queens,
+ &u64Rooks, &u64Bishops, &u64Knights, &u64Pawns);
+ return(1 == tb_wdl_probe_is_resident(u64White, u64Black, u64Kings,
+ u64Queens, u64Rooks, u64Bishops,
+ u64Knights, u64Pawns,
+ (pos->uToMove == WHITE)));
+}
+
+
//
// Convert a Fathom tablebase square index (a1==0 ... h8==63) back into
// our own COOR encoding. Inverse of TO64().
@@ -487,8 +515,28 @@ Return value:
(mv.cTo == cTo) &&
(mv.pPromoted == pPromotes))
{
+ // Root-only: use the DTZ ply count tb_probe_root already
+ // packed into uResult (no extra probe) to report a real,
+ // ply-distance-aware mate score instead of the flat
+ // EGTB_WIN_SCORE/LOSS_SCORE used by in-search probes
+ // (ProbeEGTB/RecognLookup) -- those stay WDL-only and
+ // untouched. util.c's SCORE-to-string formatter already
+ // treats any |score| >= NMATE as a mate score and derives
+ // the move count as (INFINITY_SCORE - iScore) / 2, so
+ // INFINITY_SCORE -/+ uDtz round-trips through that
+ // unmodified as "+MATE<n>"/"-MATE<n>". Clamp defensively:
+ // a uDtz large enough to push the score below NMATE would
+ // just silently fall back to plain-decimal display, not
+ // corrupt anything, but keep it a real mate score at every
+ // legal DTZ value we could plausibly see (TB_MAX_PLY=256
+ // exceeds NMATE's 200-point margin in principle, however
+ // unlikely in practice for <=7-man tables).
+ ULONG uDtz = TB_GET_DTZ(uResult);
+ if (uDtz >= (+INFINITY_SCORE - NMATE)) uDtz = (+INFINITY_SCORE - NMATE - 1);
*pmv = mv;
- *piScore = (TB_WIN == uWdl) ? EGTB_WIN_SCORE : EGTB_LOSS_SCORE;
+ *piScore = (TB_WIN == uWdl) ?
+ (SCORE)(+INFINITY_SCORE - uDtz) :
+ (SCORE)(-INFINITY_SCORE + uDtz);
return(TRUE);
}
}
diff --git a/src/recogn.c b/src/recogn.c
index 2219e48..427297b 100644
--- a/src/recogn.c
+++ b/src/recogn.c
@@ -719,6 +719,169 @@ Return value:
}
+ULONG
+_RecognizeKQKP(IN SEARCHER_THREAD_CONTEXT *ctx,
+ IN OUT SCORE *piScore)
+/**
+
+Routine description:
+
+ Recognize KQKP positions: exactly one queen vs exactly one pawn.
+ Deliberately narrow -- only handles the classically UNCONDITIONAL
+ win subset: the pawn has NOT yet reached the rank immediately
+ before queening (rank 7 for a White pawn, rank 2 for a Black one),
+ and the queen isn't hanging to the weak king or weak pawn (see
+ below). Once the weak pawn reaches that last rank it threatens to
+ promote into a second queen -- uncontested if nothing stops it, or
+ even a capturing promotion if it lands on the strong queen's square
+ -- either of which can rebalance material into a generic (and often
+ drawn) queen-vs-queen ending, regardless of which file the pawn is
+ on. That whole family of positions returns UNRECOGNIZED and falls
+ through to search/EGTB, same as every other case this function
+ doesn't recognize.
+
+ 2026-09-24, in order of discovery (all via TestRecognExhaustiveKQKP,
+ each one found immediately after "fixing" the previous):
+
+ First hole -- ~10% wrong across the enumeration, all one shape: the
+ queen placed adjacent to the weak king, undefended, with the weak
+ side to move (e.g. K1kQ4/p7/8/8/8/8/8/8 b - - 0 0 -- White Ka8,
+ Black Kc8, White Qd8, Black Pa7, Black to move: ...Kxd8 wins the
+ queen for free). The original version of this function never
+ considered queen safety at all.
+
+ Second hole, found immediately after fixing the first (king-
+ adjacency only got the wrong rate down to ~4%, not to zero): the
+ WEAK PAWN capturing the queen while promoting, e.g.
+ k1K1q3/3P4/8/8/8/8/8/8 w - - 0 0 (Black Ka8/Qe8, White Kc8/Pd7,
+ White to move) -- dxe8=Q captures Black's queen and promotes in the
+ same move, which EGTB confirms is winning for White even though
+ White has the pawn. Fixed by also checking whether the queen sits
+ on a square the weak pawn attacks (g_PawnAttackOriginBB), not just
+ king adjacency, and by dropping the earlier "unless defended by the
+ strong king" carve-out entirely for both threats -- a pawn capture-
+ promotion is catastrophic even if recaptured (nets down to a bare-
+ kings draw at best, never the claimed win).
+
+ Third hole, found immediately after fixing the second: the original
+ rook/bishop-file-only exclusion for a pawn on the queening-adjacent
+ rank missed the exact same "uncontested promotion creates a second
+ queen" problem on every OTHER file too, e.g.
+ k1K2q2/3P4/8/8/8/8/8/8 w - - 0 0 (White's d-pawn just pushes to
+ d8=Q, an ordinary center-file pawn, and EGTB says the resulting
+ KQKQ position is a draw, not the claimed win) and
+ K1k5/Q7/8/8/8/8/1p6/8 b - - 0 0 (Black's b2 knight-file pawn simply
+ promotes THIS move, ...b1=Q, queen too far away to stop it). Fixed
+ by dropping the rook/bishop-file restriction entirely: ANY pawn on
+ the rank immediately before queening is now excluded, not just
+ rook/bishop pawns.
+
+ After all three fixes: no more confidence left in reasoning our way
+ to a boundary here without re-deriving it from EGTB -- every "safe"
+ exception considered above (defended queen, non-rook/bishop file)
+ turned out to have a counterexample the first time it was actually
+ checked. Current scope is deliberately conservative as a result.
+
+Parameters:
+
+ SEARCHER_THREAD_CONTEXT *ctx,
+ SCORE *piScore
+
+Return value:
+
+ static ULONG
+
+**/
+{
+ ULONG uStrong, uWeak;
+ COOR cPawn, cQueen, cWeakKing;
+ POSITION *pos = &ctx->sPosition;
+
+ ASSERT((TotalNonPawnCount(pos, WHITE) <= 3) &&
+ (TotalNonPawnCount(pos, BLACK) <= 3));
+ ASSERT(_NothingBut(pos, PAWN | QUEEN, WHITE));
+ ASSERT(_NothingBut(pos, PAWN | QUEEN, BLACK));
+
+ //
+ // Call the side with the queen "strong"
+ //
+ uStrong = WHITE;
+ if (TotalNonPawnCount(pos, BLACK) > 1)
+ {
+ uStrong = BLACK;
+ }
+ uWeak = FLIP(uStrong);
+ ASSERT(TotalNonPawnCount(pos, uStrong) > 1);
+ ASSERT(TotalNonPawnCount(pos, uWeak) == 1);
+
+ //
+ // Only proven for exactly one queen vs exactly one pawn.
+ //
+ if ((FastCountBits(pos->bbPieces[uStrong][QUEEN]) != 1) ||
+ (FastCountBits(pos->bbPawns[uWeak]) != 1))
+ {
+ return(UNRECOGNIZED);
+ }
+
+ //
+ // Queen safety: if the weak king can capture the queen, OR the weak
+ // pawn can capture the queen (necessarily while promoting, since
+ // the pawn has only one square left to go -- see this function's
+ // 2026-09-24 header comment for why that's catastrophic even when
+ // "defended"), this is not an unconditional win by any of the
+ // reasoning below. Bail regardless of whose move it actually is and
+ // regardless of whether the strong king defends the queen's square
+ // -- deliberately not trying to reason through defended-vs-
+ // undefended exceptions here again.
+ //
+ cQueen = CoorOfSingleBit(pos->bbPieces[uStrong][QUEEN]);
+ ASSERT(IS_ON_BOARD(cQueen));
+ cWeakKing = KingPosition(pos, uWeak);
+ if ((DISTANCE(cQueen, cWeakKing) == 1) ||
+ (0 != (g_PawnAttackOriginBB[uWeak][cQueen] & pos->bbPawns[uWeak])))
+ {
+ return(UNRECOGNIZED);
+ }
+
+ cPawn = CoorOfSingleBit(pos->bbPawns[uWeak]);
+ ASSERT(IS_ON_BOARD(cPawn));
+ ASSERT(IS_PAWN(pos->rgSquare[cPawn].pPiece));
+
+ //
+ // Any pawn on the rank immediately before queening -- not just a
+ // rook/bishop pawn -- threatens to promote uncontested next move
+ // (or, if it's the weak side's own move right now, THIS move),
+ // producing a second queen and rebalancing material regardless of
+ // file. See this function's 2026-09-24 header comment: the file-
+ // restricted version of this check missed exactly this on a
+ // knight-file pawn (K1k5/Q7/8/8/8/8/1p6/8 b - -, Black's b2 pawn,
+ // queen too far away to stop ...b1=Q) as well as the original
+ // rook/bishop-file "fortress" reasoning it was written for.
+ //
+ if (((WHITE == uWeak) && (TRUE == RANK7(cPawn))) ||
+ ((BLACK == uWeak) && (TRUE == RANK2(cPawn))))
+ {
+ return(UNRECOGNIZED);
+ }
+
+ //
+ // Every other pawn placement is an unconditional win for the queen
+ // side with best play. Material balance is already reliably very
+ // positive for the queen side here (a queen vastly outweighs a lone
+ // pawn), so it's safe to use directly as a directional bound -- we
+ // don't need (or attempt) a precise mate-distance score, just a
+ // correctly-signed one.
+ //
+ *piScore = pos->iMaterialBalance[uStrong];
+ if (pos->uToMove == uWeak)
+ {
+ *piScore *= -1;
+ return(RECOGN_UPPER);
+ }
+ return(RECOGN_LOWER);
+}
+
+
static ULONG
_RecognizeKBKP(IN SEARCHER_THREAD_CONTEXT *ctx,
IN OUT SCORE *piScore)
@@ -1912,6 +2075,27 @@ Return value:
_MakeMaterialSig(1, 0, 0, 0, 0),
_MakeMaterialSig(0, 1, 0, 0, 0));
+ // KQ+KP+ DISABLED (like KBNK above): TestRecognExhaustiveKQKP found
+ // three structurally distinct correctness holes in a row, each only
+ // surfacing once the previous one was fixed -- queen safety vs. the
+ // weak king, queen safety vs. a promoting weak pawn, and any weak
+ // pawn (not just rook/bishop-file) reaching the queening-adjacent
+ // rank threatening to rebalance material via a second queen. After
+ // those three fixes, weak-side stalemate accounted for only 1,746
+ // of the remaining 42,398 wrong positions (12.7M checked) -- the
+ // other 40,652 are a fourth, still-unidentified failure mode. Given
+ // that pattern, this "queen always beats a lone pawn" classical
+ // claim has more real exceptions across the full space of legal
+ // positions than can safely be hand-derived from memory one fix at
+ // a time. Not registered until someone re-derives the actual
+ // correct boundary (most plausibly by deriving it FROM Syzygy WDL
+ // data rather than guessing and re-testing) and re-validates via
+ // TestRecognExhaustiveKQKP.
+ //
+ // _NewRecognizer(_RecognizeKQKP,
+ // _MakeMaterialSig(1, 0, 0, 0, 0),
+ // _MakeMaterialSig(0, 0, 0, 0, 1));
+
// KB+KP+ P N B R Q
_NewRecognizer(_RecognizeKBKP,
_MakeMaterialSig(1, 0, 0, 0, 0),
@@ -1943,7 +2127,7 @@ Return value:
ULONG
RecognLookup(IN SEARCHER_THREAD_CONTEXT *ctx,
IN OUT SCORE *piScore,
- IN FLAG fProbeEGTB)
+ IN ULONG uEgtbProbeMode)
/**
Routine description:
@@ -1954,11 +2138,23 @@ Routine description:
present on the board and 2. probe on-disk EGTB files if no useful
recognizer result is found.
+ uEgtbProbeMode is trinary (EGTB_PROBE_NEVER / EGTB_PROBE_ALWAYS /
+ EGTB_PROBE_IF_RESIDENT, chess.h) rather than a plain FLAG so that
+ "probe iff we believe the page is resident" can be decided in here,
+ on only the branch that's actually about to call ProbeEGTB, instead
+ of the caller computing EgtbProbeLikelyResident unconditionally as
+ a function argument before even knowing whether an interior
+ recognizer below is about to make the question moot. This matters
+ more than it used to: EgtbProbeLikelyResident can now involve real
+ mincore() syscalls for 6-man material (see its own comment in
+ probe.c) rather than just a cheap atomic load, so deferring it here
+ directly cuts how often that runs.
+
Parameters:
SEARCHER_THREAD_CONTEXT *ctx,
SCORE *piScore,
- FLAG fProbeEGTB
+ ULONG uEgtbProbeMode
Return value:
@@ -1972,9 +2168,7 @@ Return value:
RECOGNIZER *pRec;
SCORE iScore;
- //
- // Try interior node recognizers
- //
+ // Try interior node recognizers first.
if ((TotalNonPawnCount(pos, WHITE) <= 3) &&
(TotalNonPawnCount(pos, BLACK) <= 3))
{
@@ -2017,14 +2211,12 @@ Return value:
}
}
#endif
- ASSERT(_SanityCheckRecognizers(ctx, iScore, uVal));
return(uVal);
}
}
}
}
- //
// Try EGTB probe. No piece-count gate here: ProbeEGTB already
// checks the position's piece count against the dynamic
// TB_LARGEST (set from whatever tablebase files Fathom actually
@@ -2034,17 +2226,27 @@ Return value:
// ever added alongside the 5-man set already present -- see
// CLAUDE.md).
//
- if (FALSE != fProbeEGTB)
+ // EGTB_PROBE_IF_RESIDENT's EgtbProbeLikelyResident call happens
+ // right here, not in the caller -- this is the whole point of
+ // uEgtbProbeMode being trinary instead of a plain FLAG (see this
+ // function's header comment): every node above that an interior
+ // recognizer already resolved returned before ever reaching this
+ // line, so it never pays for a residency check (which can mean
+ // real mincore() syscalls for 6-man material) it wouldn't use.
+ if (EGTB_PROBE_NEVER != uEgtbProbeMode)
{
- if (TRUE == ProbeEGTB(ctx, &iScore))
+ FLAG fShouldProbe = (EGTB_PROBE_ALWAYS == uEgtbProbeMode) ||
+ EgtbProbeLikelyResident(ctx);
+ if (TRUE == fShouldProbe)
{
- //
- // Syzygy WDL probes are win/draw/loss only -- there's no
- // mate distance to adjust for the present ply here (see
- // EGTB_WIN_SCORE / EGTB_LOSS_SCORE in probe.c).
- //
- *piScore = iScore;
- return(RECOGN_EXACT);
+ if (TRUE == ProbeEGTB(ctx, &iScore))
+ {
+ // Syzygy WDL probes are win/draw/loss only -- there's no
+ // mate distance to adjust for the present ply here (see
+ // EGTB_WIN_SCORE / EGTB_LOSS_SCORE in probe.c).
+ *piScore = iScore;
+ return(RECOGN_EXACT);
+ }
}
}
return(UNRECOGNIZED);
diff --git a/src/root.c b/src/root.c
index a8a61bf..86d8b76 100644
--- a/src/root.c
+++ b/src/root.c
@@ -501,46 +501,6 @@ Return value:
Trace("Never used endgame pruning heuristic.\n");
}
#endif
-#ifdef TEST_NULL
- n = (double)(ctx->sCounters.tree.u64AvoidNullSuccess);
- d = n + (double)(ctx->sCounters.tree.u64AvoidNullFailures);
- if (d != 0.0)
- {
- Trace("Avoid null move heuristic rate was %5.3f percent.\n"
- "Avoid null move heuristic was "
- "%"COMPILER_LONGLONG_UNSIGNED_FORMAT " / "
- "%"COMPILER_LONGLONG_UNSIGNED_FORMAT ".\n",
- ((n / d) * 100.0),
- ctx->sCounters.tree.u64AvoidNullSuccess,
- (ctx->sCounters.tree.u64AvoidNullSuccess +
- ctx->sCounters.tree.u64AvoidNullFailures));
- }
- else
- {
- Trace("Never used the avoid null move heuristic.\n");
- }
- n = (double)(ctx->sCounters.tree.u64QuickNullSuccess);
- n += (double)(ctx->sCounters.tree.u64QuickNullDeferredSuccess);
- d = n + (double)(ctx->sCounters.tree.u64QuickNullFailures);
- if (d != 0.0)
- {
- Trace("Quick null move heuristic rate was %5.3f percent.\n"
- "Quick null move heuristic rate was "
- "%"COMPILER_LONGLONG_UNSIGNED_FORMAT " ("
- "%"COMPILER_LONGLONG_UNSIGNED_FORMAT ") / "
- "%"COMPILER_LONGLONG_UNSIGNED_FORMAT ".\n",
- ((n / d) * 100.0),
- ctx->sCounters.tree.u64QuickNullSuccess,
- ctx->sCounters.tree.u64QuickNullDeferredSuccess,
- (ctx->sCounters.tree.u64QuickNullSuccess +
- ctx->sCounters.tree.u64QuickNullDeferredSuccess +
- ctx->sCounters.tree.u64QuickNullFailures));
- }
- else
- {
- Trace("Never used the quick null move heuristic.\n");
- }
-#endif
n = (double)ctx->sCounters.tree.u64BetaCutoffsOnFirstMove;
d = (double)ctx->sCounters.tree.u64BetaCutoffs + 1;
Trace("First move beta cutoff rate was %5.3f percent.\n",
@@ -1547,6 +1507,21 @@ Return value:
g_Options.u64CMEvidenceTries[u] = ctx->sCounters.tree.u64CMEvidenceTries[u];
g_Options.u64CMEvidenceFH[u] = ctx->sCounters.tree.u64CMEvidenceFH[u];
}
+#ifdef TEST_NULL
+ for (u = 0; u < NC_TOTAL_BUCKETS; u++)
+ {
+ g_Options.u64NCAttempts[u] = ctx->sCounters.tree.u64NCAttempts[u];
+ g_Options.u64NCNullFH[u] = ctx->sCounters.tree.u64NCNullFH[u];
+ g_Options.u64NCRealFH[u] = ctx->sCounters.tree.u64NCRealFH[u];
+ g_Options.u64NCBothFH[u] = ctx->sCounters.tree.u64NCBothFH[u];
+ g_Options.u64NCNullProbeNodeSum[u] =
+ ctx->sCounters.tree.u64NCNullProbeNodeSum[u];
+ g_Options.u64NCFirstMoveNodeSum[u] =
+ ctx->sCounters.tree.u64NCFirstMoveNodeSum[u];
+ g_Options.u64NCRealLoopNodeSum[u] =
+ ctx->sCounters.tree.u64NCRealLoopNodeSum[u];
+ }
+#endif
g_MoveTimer.dEndTime = SystemTimeStamp();
//
diff --git a/src/script.c b/src/script.c
index 532a0d5..f18ab7b 100644
--- a/src/script.c
+++ b/src/script.c
@@ -64,6 +64,24 @@ typedef struct _SUITE_COUNTERS
UINT64 u64TotalLeftoverFH;
UINT64 u64TotalCMEvidenceTries[CM_EVIDENCE_BUCKETS];
UINT64 u64TotalCMEvidenceFH[CM_EVIDENCE_BUCKETS];
+#ifdef TEST_NULL
+ // Null-move precision/recall calibration, summed across every
+ // position in the suite -- see _NullCalibBucketIndex's comment in
+ // search.c. Accumulated the same way as u64TotalCMEvidenceTries
+ // above (per-search counters are copied into g_Options at the end
+ // of each Think() call, then summed in here by _ProcessSuiteLine),
+ // and printed once by _PrintSuiteReport at true end-of-suite --
+ // deliberately *not* dumped after every position, which would
+ // balloon the logfile (confirmed: 191MB after ~28K of 200K
+ // positions the first time this was tried that way).
+ UINT64 u64TotalNCAttempts[NC_TOTAL_BUCKETS];
+ UINT64 u64TotalNCNullFH[NC_TOTAL_BUCKETS];
+ UINT64 u64TotalNCRealFH[NC_TOTAL_BUCKETS];
+ UINT64 u64TotalNCBothFH[NC_TOTAL_BUCKETS];
+ UINT64 u64TotalNCNullProbeNodeSum[NC_TOTAL_BUCKETS];
+ UINT64 u64TotalNCFirstMoveNodeSum[NC_TOTAL_BUCKETS];
+ UINT64 u64TotalNCRealLoopNodeSum[NC_TOTAL_BUCKETS];
+#endif
double dSigmaEBF; // sum of per-problem nodes^(1/depth)
ULONG uEBFCount; // number of problems with depth > 0
double dAverageTimeToSolution;
@@ -391,6 +409,28 @@ _ProcessSuiteLine(char *szLine, POSITION *pos)
g_Options.u64CMEvidenceFH[v];
}
}
+#ifdef TEST_NULL
+ {
+ ULONG v;
+ for (v = 0; v < NC_TOTAL_BUCKETS; v++)
+ {
+ g_SuiteCounters.u64TotalNCAttempts[v] +=
+ g_Options.u64NCAttempts[v];
+ g_SuiteCounters.u64TotalNCNullFH[v] +=
+ g_Options.u64NCNullFH[v];
+ g_SuiteCounters.u64TotalNCRealFH[v] +=
+ g_Options.u64NCRealFH[v];
+ g_SuiteCounters.u64TotalNCBothFH[v] +=
+ g_Options.u64NCBothFH[v];
+ g_SuiteCounters.u64TotalNCNullProbeNodeSum[v] +=
+ g_Options.u64NCNullProbeNodeSum[v];
+ g_SuiteCounters.u64TotalNCFirstMoveNodeSum[v] +=
+ g_Options.u64NCFirstMoveNodeSum[v];
+ g_SuiteCounters.u64TotalNCRealLoopNodeSum[v] +=
+ g_Options.u64NCRealLoopNodeSum[v];
+ }
+ }
+#endif
}
else
{
@@ -473,6 +513,103 @@ _PrintSuiteReport(double dSuiteStart)
}
}
+#ifdef TEST_NULL
+ // Null-move precision/recall calibration, summed across the whole
+ // suite (see _NullCalibBucketIndex's comment in search.c for the
+ // encoding and what each dimension means). Decode each bucket
+ // index back into its component bands for a readable line; skip
+ // empty cells so this stays legible even though the full table has
+ // NC_TOTAL_BUCKETS cells.
+ {
+ static const char *rgszDepthBand[NC_DEPTH_BUCKETS] =
+ { "0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10",
+ "11", "12", "13", "14", "15", "16+" };
+ static const char *rgszDeficitBand[NC_DEFICIT_BUCKETS] =
+ { "<-300", "-300..-100", "-100..0", "0..100", "100..300", "300+" };
+ static const char *rgszMaterialBand[NC_MATERIAL_BUCKETS] =
+ { "0-1", "2", "3-5", "6+" };
+ ULONG uIdx, uDecode, uNullDepthB, uOuterDepthB, uDeficitB;
+ ULONG uMaxMaterialB, uMoverMaterialB;
+ UINT64 u64Attempts, u64NullFH, u64RealFH, u64BothFH;
+ UINT64 u64NullProbeNodeSum, u64FirstMoveNodeSum, u64RealLoopNodeSum;
+ double dAvgNullProbeNodes, dAvgFirstMoveNodes, dAvgRealLoopNodes;
+ double dNullSuccessRate, dEVFirstMoveOnly, dEV;
+
+ Trace("\nNull-move calibration (attempts / nullFH / realFH / both / "
+ "precision / recall / nullSuccessRate / avgNullProbeNodes / "
+ "avgFirstMoveNodes / avgRealLoopNodes / EVFirstMoveOnly / "
+ "EV-per-attempt-in-nodes):\n");
+ for (uIdx = 0; uIdx < NC_TOTAL_BUCKETS; uIdx++)
+ {
+ u64Attempts = g_SuiteCounters.u64TotalNCAttempts[uIdx];
+ if (0 == u64Attempts) continue;
+ u64NullFH = g_SuiteCounters.u64TotalNCNullFH[uIdx];
+ u64RealFH = g_SuiteCounters.u64TotalNCRealFH[uIdx];
+ u64BothFH = g_SuiteCounters.u64TotalNCBothFH[uIdx];
+ u64NullProbeNodeSum = g_SuiteCounters.u64TotalNCNullProbeNodeSum[uIdx];
+ u64FirstMoveNodeSum = g_SuiteCounters.u64TotalNCFirstMoveNodeSum[uIdx];
+ u64RealLoopNodeSum = g_SuiteCounters.u64TotalNCRealLoopNodeSum[uIdx];
+
+ // Decode uIdx (innermost/fastest-varying dimension first),
+ // matching _NullCalibBucketIndex's encoding in search.c.
+ // Use a scratch copy -- uIdx itself is still the array
+ // index for the lookups above/below.
+ uDecode = uIdx;
+ uMoverMaterialB = uDecode % NC_MATERIAL_BUCKETS;
+ uDecode /= NC_MATERIAL_BUCKETS;
+ uMaxMaterialB = uDecode % NC_MATERIAL_BUCKETS;
+ uDecode /= NC_MATERIAL_BUCKETS;
+ uDeficitB = uDecode % NC_DEFICIT_BUCKETS;
+ uDecode /= NC_DEFICIT_BUCKETS;
+ uOuterDepthB = uDecode % NC_DEPTH_BUCKETS;
+ uDecode /= NC_DEPTH_BUCKETS;
+ uNullDepthB = uDecode % NC_DEPTH_BUCKETS;
+
+ dAvgNullProbeNodes = (double)u64NullProbeNodeSum / (double)u64Attempts;
+ dAvgFirstMoveNodes = (double)u64FirstMoveNodeSum / (double)u64Attempts;
+ dAvgRealLoopNodes = (double)u64RealLoopNodeSum / (double)u64Attempts;
+ dNullSuccessRate = (double)u64NullFH / (double)u64Attempts;
+ // EV of TRYING null vs. going straight to the real move loop,
+ // in nodes saved per attempt (positive = trying pays off):
+ // on the dNullSuccessRate fraction of attempts we pay the
+ // (cheap) null probe instead of the (expensive) real-loop
+ // cost; on the rest we pay the null probe *in addition* to
+ // still needing the real loop. dEV uses avgRealLoopNodes --
+ // the true total cost of however the real search actually
+ // resolved the node (one move, several, or production's own
+ // separate null success) -- while dEVFirstMoveOnly keeps the
+ // cheaper move-1-only proxy alongside it for comparison.
+ // Both assume (per the user's simplifying assumption,
+ // matched by this dataset's ~99.96% overall precision) that
+ // a null-move fail-high is correct.
+ dEVFirstMoveOnly =
+ dNullSuccessRate * (dAvgFirstMoveNodes - dAvgNullProbeNodes)
+ - (1.0 - dNullSuccessRate) * dAvgNullProbeNodes;
+ dEV = dNullSuccessRate * (dAvgRealLoopNodes - dAvgNullProbeNodes)
+ - (1.0 - dNullSuccessRate) * dAvgNullProbeNodes;
+
+ Trace(" nullDepth=%s outerDepth=%s deficit=%s "
+ "maxMaterial=%s moverMaterial=%s : "
+ "%"COMPILER_LONGLONG_UNSIGNED_FORMAT" / "
+ "%"COMPILER_LONGLONG_UNSIGNED_FORMAT" / "
+ "%"COMPILER_LONGLONG_UNSIGNED_FORMAT" / "
+ "%"COMPILER_LONGLONG_UNSIGNED_FORMAT" / "
+ "%5.3f / %5.3f / %5.3f / %10.1f / %10.1f / %10.1f / %10.1f / %10.1f\n",
+ rgszDepthBand[uNullDepthB],
+ rgszDepthBand[uOuterDepthB],
+ rgszDeficitBand[uDeficitB],
+ rgszMaterialBand[uMaxMaterialB],
+ rgszMaterialBand[uMoverMaterialB],
+ u64Attempts, u64NullFH, u64RealFH, u64BothFH,
+ (u64NullFH != 0) ? (100.0 * (double)u64BothFH / (double)u64NullFH) : 0.0,
+ (u64RealFH != 0) ? (100.0 * (double)u64BothFH / (double)u64RealFH) : 0.0,
+ 100.0 * dNullSuccessRate,
+ dAvgNullProbeNodes, dAvgFirstMoveNodes, dAvgRealLoopNodes,
+ dEVFirstMoveOnly, dEV);
+ }
+ }
+#endif
+
// Histogram stuff
if (g_SuiteCounters.uTotal > 0) {
uMax = g_SuiteCounters.uHistogram[0];
diff --git a/src/search.c b/src/search.c
index 0362701..8f4779e 100644
--- a/src/search.c
+++ b/src/search.c
@@ -104,6 +104,131 @@ _CMEvidenceBucket(ULONG uEvidence)
}
#endif
+#ifdef TEST_NULL
+//
+// Null-move try/avoid calibration harness.
+//
+// Question: given the current data (precision >99.9% almost everywhere,
+// false-positive/"null-move lie" risk concentrated by material and
+// deficit, EV strongly positive almost everywhere once restricted to
+// the safe region), can a two-stage try/skip policy do better than
+// today's flat material threshold + per-depth eval-margin table?
+//
+// Stage 1 (safety): is a null-move fail-high here even trustworthy?
+// Stage 2 (profitability/EV): given it's safe, is it worth the cost?
+//
+// fThreat, fSkipNull, and en-prise/trapped-piece state were bucketed in
+// an earlier revision of this harness (2026-09-24 run, saved under
+// nullmove_calib/run_2026-09-24_sd10_30k/) and shown to be either
+// already-validated non-tunable binary gates (fThreat/fSkipNull -- both
+// clearly reduce EV and precision/recall when set, and both already
+// gate the call to WeShouldTryNullmovePruning in Search(), so there's
+// no threshold left to calibrate) or not a safety signal at all
+// (en-prise/trapped predicts *lower recall*, not higher false-positive
+// risk -- a Stage 2, not Stage 1, concern). None of the three are
+// bucketed here; fThreat==FALSE && fSkipNull==FALSE are folded into
+// this probe's structural eligibility instead, matching production.
+//
+// This revision instead resolves two open questions from that first
+// run:
+//
+// 1. Is MAX(white,black) non-pawn piece count or the MOVER's own
+// count (TotalNonPawnCount(pos, pos->uToMove)) the better
+// Stage-1 safety predictor? Zugzwang is a property of the side
+// being given the free "pass", so mover-only is the more
+// causally-motivated signal, but the first run only measured
+// MAX(both) -- bucket both, jointly, and let the false-positive-
+// rate data settle it.
+// 2. The first run bucketed the *real* search's cost (the Stage-2 EV
+// calc's "cost of the alternative") by the null probe's own post-
+// reduction depth (uNullDepth), which is only meaningful via
+// whatever R schedule (SelectNullmoveRFactor) happened to be
+// compiled in at the time -- useless for evaluating a *different*
+// R schedule later. The real search's cost is fundamentally a
+// function of the OUTER, pre-reduction uDepth (how much real work
+// remains), not of uNullDepth at all. Bucket both depths
+// separately: uNullDepth for the probe's own cost (cost1), uDepth
+// for the real alternative's cost (cost2) -- decoupling the two
+// cost models from any particular R formula.
+//
+// Ground truth (did this node *really* fail high) is read off the same
+// iBestScore-vs-iBeta comparison Search() already computes at its
+// "end:" label for the NODE_PV/NODE_CUT/NODE_ALL surprise counters just
+// above this block; we piggyback on that rather than duplicating it.
+// The diagnostic probe is a self-contained MakeMove(NULLMOVE) -> reduced
+// Search() -> UnmakeMove, deliberately *not* reusing TryNullmovePruning,
+// so its verification-search/extension side effects (meant only for a
+// probe whose result production actually trusts) never fire here. Like
+// any double-search diagnostic, it also warms the hash/history tables
+// along its own subtree before the real move loop runs at the same
+// node -- a known, unavoidable side effect of measuring this way.
+//
+// This roughly doubles node cost at every non-PV node under TEST_NULL,
+// so it's a diagnostic-only build flag (gmake TEST_NULL=1 ...), never
+// part of the normal GENETIC=1 PERF_COUNTERS=1 MP=1 SIXTYFOUR=1 profile,
+// and should only be run against a calibration corpus at a shallow
+// fixed depth, not self-play or the full engine pools.
+//
+// Buckets are a mixed-radix index over five dimensions:
+//
+// index = nullDepthB * (NC_DEPTH_BUCKETS*NC_DEFICIT_BUCKETS*NC_MATERIAL_BUCKETS*NC_MATERIAL_BUCKETS)
+// + outerDepthB * (NC_DEFICIT_BUCKETS*NC_MATERIAL_BUCKETS*NC_MATERIAL_BUCKETS)
+// + deficitB * (NC_MATERIAL_BUCKETS*NC_MATERIAL_BUCKETS)
+// + maxMaterialB* (NC_MATERIAL_BUCKETS)
+// + moverMaterialB
+//
+// root.c's dump loop decodes this same layout when printing results.
+//
+static ULONG
+_NullCalibDepthBucket(ULONG uSomeDepth) // generic: used for both uNullDepth and uDepth
+{
+ ULONG uPlies = uSomeDepth / ONE_PLY;
+ if (uPlies >= NC_DEPTH_BUCKETS - 1) return(NC_DEPTH_BUCKETS - 1); // 16+
+ return(uPlies);
+}
+
+static ULONG
+_NullCalibDeficitBucket(SCORE iDeficit) // iEval - iBeta
+{
+ if (iDeficit < -300) return(0);
+ if (iDeficit < -100) return(1);
+ if (iDeficit < 0) return(2);
+ if (iDeficit < 100) return(3);
+ if (iDeficit < 300) return(4);
+ return(5);
+}
+
+static ULONG
+_NullCalibMaterialBucket(ULONG uPiecesPerSide) // generic: used for both MAX(both) and mover-only
+{
+ if (uPiecesPerSide <= 1) return(0);
+ if (uPiecesPerSide == 2) return(1);
+ if (uPiecesPerSide <= 5) return(2);
+ return(3);
+}
+
+static ULONG
+_NullCalibBucketIndex(ULONG uNullDepth, // probe's own post-reduction depth (cost1 axis)
+ ULONG uOuterDepth, // real node's pre-reduction depth (cost2 axis)
+ SCORE iDeficit,
+ ULONG uMaxPiecesPerSide, // MAX(white,black)
+ ULONG uMoverPiecesPerSide) // TotalNonPawnCount(pos, pos->uToMove)
+{
+ ULONG uNullDepthB = _NullCalibDepthBucket(uNullDepth);
+ ULONG uOuterDepthB = _NullCalibDepthBucket(uOuterDepth);
+ ULONG uDeficitB = _NullCalibDeficitBucket(iDeficit);
+ ULONG uMaxMaterialB = _NullCalibMaterialBucket(uMaxPiecesPerSide);
+ ULONG uMoverMaterialB = _NullCalibMaterialBucket(uMoverPiecesPerSide);
+ ULONG uIndex = uNullDepthB;
+ uIndex = uIndex * NC_DEPTH_BUCKETS + uOuterDepthB;
+ uIndex = uIndex * NC_DEFICIT_BUCKETS + uDeficitB;
+ uIndex = uIndex * NC_MATERIAL_BUCKETS + uMaxMaterialB;
+ uIndex = uIndex * NC_MATERIAL_BUCKETS + uMoverMaterialB;
+ ASSERT(uIndex < NC_TOTAL_BUCKETS);
+ return(uIndex);
+}
+#endif
+
#ifdef DEBUG
#define VERIFY_HASH_HIT \
ASSERT(IS_VALID_SCORE(iScore)); \
@@ -190,6 +315,15 @@ Search(IN SEARCHER_THREAD_CONTEXT *ctx,
FLAG fAnyMoveEFPPruned = FALSE;
FLAG fThisMoveEFPPruned = FALSE;
SCORE iMoveSee;
+#ifdef TEST_NULL
+ FLAG fNullCalibEligible = FALSE;
+ FLAG fNullCalibProbeFH = FALSE;
+ ULONG uNullCalibBucket = 0;
+ UINT64 uNullCalibProbeNodeCost = 0;
+ UINT64 uNullCalibFirstMoveNodeCost = 0;
+ UINT64 uNullCalibRealLoopNodeCost = 0;
+ UINT64 uNullCalibNodesBeforeRealLoop = 0;
+#endif
#ifdef DEBUG
ASSERT(IS_VALID_SCORE(iAlpha));
ASSERT(IS_VALID_SCORE(iBeta));
@@ -320,8 +454,18 @@ Search(IN SEARCHER_THREAD_CONTEXT *ctx,
}
// Probe interior node recognizers; allow probes of ondisk EGTB files
- // if it looks like we can get hit.
- switch(RecognLookup(ctx, &iScore, ctx->uPly <= (g_uIterateDepth / 2)))
+ // if we're shallow (near the root, where a first-touch mmap()/disk
+ // read is affordable) -- always probe there -- *or*, deeper, only
+ // if this exact position's lookup is likely resident (mincore()-
+ // gated for 6-man material; a handful of popcounts and one atomic
+ // load for 3-5 man -- see probe.c's EgtbProbeLikelyResident and
+ // CLAUDE.md). That residency check itself is deferred into
+ // RecognLookup (EGTB_PROBE_IF_RESIDENT), not computed here, so it's
+ // skipped entirely at any node a cheap interior recognizer already
+ // resolves -- see RecognLookup's own header comment.
+ switch(RecognLookup(ctx, &iScore,
+ (ctx->uPly <= (g_uIterateDepth / 2)) ?
+ EGTB_PROBE_ALWAYS : EGTB_PROBE_IF_RESIDENT))
{
case UNRECOGNIZED:
break;
@@ -362,6 +506,83 @@ Search(IN SEARCHER_THREAD_CONTEXT *ctx,
}
GENERATE_NO_MOVES;
+
+#ifdef TEST_NULL
+ // See _NullCalibBucketIndex's comment above for the full design.
+ // Structural eligibility only -- deliberately does *not* consult
+ // fSkipNull/fThreat/WeShouldTryNullmovePruning's margin table or
+ // material threshold, since those are exactly the heuristics this
+ // harness exists to evaluate. Note: like any double-search
+ // diagnostic, this probe's recursive Search() call warms the hash
+ // and history/countermove tables along its own subtree before the
+ // real move loop below runs at this same node -- a known,
+ // unavoidable side effect of measuring this way, not a bug.
+ //
+ // MUST run after GENERATE_NO_MOVES, not before: that macro is what
+ // sets ctx->sMoveStack.uBegin[ctx->uPly+1] (from uEnd[ctx->uPly]),
+ // and recursing into ctx->uPly+1 before it runs uses a stale
+ // leftover index from whatever unrelated node last occupied that
+ // ply slot -- crashed hard (MVF_MOVE_SEARCHED-already-set asserts,
+ // corrupting a real, in-flight move a few frames up) the first time
+ // this was tried with the block placed above GENERATE_NO_MOVES.
+ fNullCalibEligible = ((FALSE == pf->fAvoidNullmove) &&
+ (FALSE == pi->fInCheck) &&
+ (iBeta != +INFINITY_SCORE) &&
+ (NODE_PV != pi->eNodeType) &&
+ (FALSE == fThreat) &&
+ (FALSE == fSkipNull));
+ if (fNullCalibEligible)
+ {
+ ULONG uMaxPiecesPerSide = MAXU(TotalNonPawnCount(pos, WHITE),
+ TotalNonPawnCount(pos, BLACK));
+ ULONG uMoverPiecesPerSide = TotalNonPawnCount(pos, pos->uToMove);
+ SCORE iNullCalibScore;
+ UINT64 u64NodesBefore;
+
+ uNullCalibBucket = _NullCalibBucketIndex(uNextDepth,
+ uDepth,
+ (SCORE)(iEval - iBeta),
+ uMaxPiecesPerSide,
+ uMoverPiecesPerSide);
+ VERIFY(MakeMove(ctx, NULLMOVE));
+ pf->fAvoidNullmove = TRUE;
+ u64NodesBefore = ctx->sCounters.tree.u64TotalNodeCount;
+ iNullCalibScore = -Search(ctx, -iBeta, -iBeta + 1, uNextDepth);
+ uNullCalibProbeNodeCost = ctx->sCounters.tree.u64TotalNodeCount - u64NodesBefore;
+ pf->fAvoidNullmove = FALSE;
+ UnmakeMove(ctx, NULLMOVE);
+
+ if (!WE_SHOULD_STOP_SEARCHING)
+ {
+ // Direct increments, not INC() -- these calibration counters
+ // must not depend on PERF_COUNTERS also being defined; a
+ // TEST_NULL=1 build without PERF_COUNTERS=1 would otherwise
+ // silently collect nothing (INC() is a no-op without it).
+ fNullCalibProbeFH = (iNullCalibScore >= iBeta);
+ ctx->sCounters.tree.u64NCAttempts[uNullCalibBucket]++;
+ ctx->sCounters.tree.u64NCNullProbeNodeSum[uNullCalibBucket] +=
+ uNullCalibProbeNodeCost;
+ if (fNullCalibProbeFH)
+ {
+ ctx->sCounters.tree.u64NCNullFH[uNullCalibBucket]++;
+ }
+ }
+ else
+ {
+ fNullCalibEligible = FALSE; // don't commit ground truth either
+ }
+ // Snapshot right here, before production's own null-move logic
+ // and the real move loop below -- the delta read at "end:" is
+ // the true total cost of however this node actually gets
+ // resolved from this point on (whether that's production's own
+ // separate null success, one real move, or several).
+ if (fNullCalibEligible)
+ {
+ uNullCalibNodesBeforeRealLoop = ctx->sCounters.tree.u64TotalNodeCount;
+ }
+ }
+#endif
+
if (!fSkipNull &&
!fThreat &&
WeShouldTryNullmovePruning(ctx,
@@ -925,6 +1146,24 @@ Search(IN SEARCHER_THREAD_CONTEXT *ctx,
{
// First move, full a..b window.
ASSERT(uLegalMoves == 1);
+#ifdef TEST_NULL
+ // Cost side of the null-move calibration harness'
+ // try-vs-skip EV estimate: this is the "R=0"
+ // alternative the null probe (see above) is being
+ // weighed against -- the real first move's own
+ // full-depth subtree cost. uLegalMoves==1 guarantees
+ // this fires exactly once per node.
+ if (fNullCalibEligible)
+ {
+ UINT64 u64NodesBeforeFirstMove =
+ ctx->sCounters.tree.u64TotalNodeCount;
+ iScore = -Search(ctx, -iBeta, -iAlpha, uNextDepth);
+ uNullCalibFirstMoveNodeCost =
+ ctx->sCounters.tree.u64TotalNodeCount -
+ u64NodesBeforeFirstMove;
+ }
+ else
+#endif
iScore = -Search(ctx, -iBeta, -iAlpha, uNextDepth);
}
else
@@ -1162,6 +1401,58 @@ Search(IN SEARCHER_THREAD_CONTEXT *ctx,
}
}
#endif
+#ifdef TEST_NULL
+ // Commit the calibration record stashed at the diagnostic probe
+ // site above, now that iBestScore's fate against iBeta (this
+ // node's real, ground-truth fail-high outcome) is known.
+ if (fNullCalibEligible &&
+ (!WE_SHOULD_STOP_SEARCHING) &&
+ IS_VALID_SCORE(iBestScore))
+ {
+ // Direct increments -- see the probe site's comment on why
+ // these can't use INC().
+ FLAG fRealFH = (iBestScore >= iBeta);
+ if (fRealFH)
+ {
+ ctx->sCounters.tree.u64NCRealFH[uNullCalibBucket]++;
+ if (fNullCalibProbeFH)
+ {
+ ctx->sCounters.tree.u64NCBothFH[uNullCalibBucket]++;
+ }
+ }
+ else if (fNullCalibProbeFH)
+ {
+ // False positive: the null-move probe predicted a fail-high
+ // that the real move loop didn't confirm -- exactly the
+ // "null-move lie" case that a low-material verification
+ // search exists to catch in production. Dump a capped
+ // number of example FENs per bucket (same "capped FEN
+ // sampler" idea as eval.c's MARGINSAFETY_EXCEEDED logging)
+ // so these can be inspected by hand for genuine zugzwang
+ // our TotalNonPawnCount/ValueOfMaterialInTroubleAfterNull
+ // safety net is missing, vs. ordinary search noise.
+ static ULONG uFalsePositiveFenSamples[NC_TOTAL_BUCKETS];
+ if (uFalsePositiveFenSamples[uNullCalibBucket] < 5)
+ {
+ CHAR *pszFen = PositionToFen(pos);
+ if (NULL != pszFen)
+ {
+ uFalsePositiveFenSamples[uNullCalibBucket]++;
+ Trace("NULLCALIB_FALSE_POSITIVE bucket=%lu depth=%lu "
+ "fen=%s\n",
+ uNullCalibBucket, uDepth, pszFen);
+ SystemFreeMemory(pszFen);
+ }
+ }
+ }
+ ctx->sCounters.tree.u64NCFirstMoveNodeSum[uNullCalibBucket] +=
+ uNullCalibFirstMoveNodeCost;
+ uNullCalibRealLoopNodeCost = ctx->sCounters.tree.u64TotalNodeCount -
+ uNullCalibNodesBeforeRealLoop;
+ ctx->sCounters.tree.u64NCRealLoopNodeSum[uNullCalibBucket] +=
+ uNullCalibRealLoopNodeCost;
+ }
+#endif
DTLeaveNode(ctx, FALSE, iBestScore, mvBest);
return(iBestScore);
}
@@ -1994,9 +2285,17 @@ QSearch(IN SEARCHER_THREAD_CONTEXT *ctx,
}
DTEnterNode(ctx, 0, TRUE, iAlpha, iBeta);
- // Probe interior node recognizers; do not allow probes into ondisk
- // EGTB files since we are in qsearch.
- switch(RecognLookup(ctx, &iScore, FALSE))
+ // Probe interior node recognizers. Qsearch runs at the deepest,
+ // most performance-sensitive part of the tree, so this used to
+ // unconditionally refuse EGTB probes -- but a qsearch node with few
+ // enough pieces to be in the tablebase at all is exactly the
+ // "trades have happened near a leaf" case the EGTB is most useful
+ // for: only probe when this position's lookup is likely resident
+ // (EGTB_PROBE_IF_RESIDENT -- RecognLookup itself decides, via
+ // EgtbProbeLikelyResident, only on the branch that's actually about
+ // to call ProbeEGTB, so a node an interior recognizer resolves
+ // never pays for the residency check at all) -- see CLAUDE.md.
+ switch(RecognLookup(ctx, &iScore, EGTB_PROBE_IF_RESIDENT))
{
case UNRECOGNIZED:
break;
diff --git a/src/searchsup.c b/src/searchsup.c
index 781c702..0201219 100644
--- a/src/searchsup.c
+++ b/src/searchsup.c
@@ -1382,17 +1382,26 @@ ULONG
SelectNullmoveRFactor(SEARCHER_THREAD_CONTEXT *ctx,
INT uDepth)
{
- ULONG u = TWO_PLY;
+ ULONG u;
ULONG uMaxPiecesPerSide = MAXU(TotalNonPawnCount(&ctx->sPosition, WHITE),
TotalNonPawnCount(&ctx->sPosition, BLACK));
ASSERT(uMaxPiecesPerSide <= 16);
- if ((uDepth > 8 * ONE_PLY) ||
- ((uDepth > 6 * ONE_PLY) && (uMaxPiecesPerSide >= 3)))
+ if (uMaxPiecesPerSide < 3)
{
- u = THREE_PLY;
+ // Sparse material on the mover's side: stay with the
+ // conservative, non-growing reduction rather than risk a
+ // zugzwang-prone position getting an aggressive R. (The
+ // TotalNonPawnCount < 4 verification search in
+ // TryNullmovePruning is the other half of this safety net.)
+ u = TWO_PLY;
+ }
+ else
+ {
+ // Crafty-style growth: R = 3 + depth / 6 (in whole plies),
+ // scaled into our fractional-ply depth units.
+ u = THREE_PLY + ((uDepth / (6 * ONE_PLY)) * ONE_PLY);
}
- ASSERT((u == TWO_PLY) || (u == THREE_PLY));
return(u);
}
@@ -1403,14 +1412,82 @@ FLAG WeShouldTryNullmovePruning(SEARCHER_THREAD_CONTEXT *ctx,
SCORE iEval,
SCORE iImprovement,
ULONG uNullDepth)
+/**
+
+Routine description:
+
+ Two-stage try/skip decision, replacing the old flat >2-non-pawn-
+ piece gate plus per-depth eval-margin table. Calibrated from a
+ 30K-position, sd-10, ~9-billion-attempt null-move precision/recall/
+ EV harness (2026-09-24, two revisions -- see
+ nullmove_calib/run_2026-09-24_sd10_30k_v2/, the second and
+ authoritative one). iImprovement and uNullDepth are currently
+ unused: the Stage 1 thresholds below are flat, not depth- or
+ improvement-scaled, and Stage 2 doesn't need its own gate at all
+ (see below). Kept in the signature to avoid touching call sites.
+
+ Stage 1 (safety): is a null-move fail-high here even trustworthy?
+ Gated on the MOVER's own non-pawn piece count -- TotalNonPawnCount
+ (pos, pos->uToMove), NOT MAX(white,black) as an earlier revision of
+ this gate (and of the calibration harness that measured it) used.
+ Zugzwang is a property of the side being given the free "pass", so
+ the opponent's material shouldn't matter -- confirmed empirically:
+ (maxMaterial=6+, moverMaterial=2) measured a 0.30% false-positive
+ rate, indistinguishable from the worst low-material cells overall,
+ despite "6+" sounding safe. A materially-heavier opponent does not
+ protect a mover that is itself thin on material.
+
+ Measured false-positive rate (a null fail-high the real search
+ didn't confirm) by mover's own material x eval-vs-beta deficit:
+
+ deficit<-300 -300..-100 -100..0 0..100 100..300 300+
+ mover 0-1 pcs: 5.57% 1.35% 0.82% 1.02% 0.28% 0.12%
+ mover 2 pcs: 3.73% 0.19% 0.31% 0.32% 0.02% 0.01%
+ mover 3-5 pcs: 0.91% 0.20% 0.09% 0.002% 0.001% 0.0005%
+ mover 6+ pcs: 0.02% 0.002% 0.001% 0.0002% 0.0001% 0.0001%
+
+ 6+ pieces is safe (<=0.02%) at EVERY deficit measured, even the most
+ negative -- no deficit floor needed there at all. 2 pieces is
+ actually salvageable (0.01-0.02%, comparable to the safest cells
+ overall) once deficit >= 100, unlike a flatly-disabled <=2 rule
+ would assume -- but <=1 piece never gets safe (worst case 0.12% at
+ its *best* deficit band), so that tier is disabled outright rather
+ than deficit-gated.
+
+ Stage 2 (profitability/EV): given Stage 1 passes, is trying still
+ worth the reduced-depth probe's node cost? Checked directly against
+ the same run's EV data (EV = successRate*(realLoopCost-nullProbeCost)
+ - (1-successRate)*nullProbeCost, in nodes saved per attempt) across
+ every (depth, material) cell that survives the Stage 1 gate above:
+ EV was strongly positive everywhere measured, from low single digits
+ at shallow depth up to tens of thousands of nodes at deep probes,
+ never negative. Stage 1 is already doing essentially all the
+ discriminating work, so Stage 2 currently just returns TRUE
+ unconditionally once Stage 1 passes -- no separate EV threshold is
+ justified by this data. Revisit if a future, deeper or wider
+ calibration run finds a safe-but-unprofitable pocket this one
+ didn't sample.
+
+Parameters:
+
+ SEARCHER_THREAD_CONTEXT *ctx,
+ SCORE iAlpha,
+ SCORE iBeta,
+ SCORE iEval,
+ SCORE iImprovement,
+ ULONG uNullDepth
+
+Return value:
+
+ FLAG
+
+**/
{
- static SCORE iSkipShallowNullIfThisFarBelowAlpha[] = {
- 700, 950, 1110, 1150, 1190, 1230, 1270, 0
- };
POSITION *pos = &ctx->sPosition;
PLY_INFO *pi = &ctx->sPlyInfo[ctx->uPly];
CUMULATIVE_SEARCH_FLAGS *pf = &ctx->sSearchFlags;
- ULONG u;
+ ULONG uMoverPiecesPerSide;
+ SCORE iDeficit;
ASSERT(IS_VALID_SCORE(iAlpha) && IS_VALID_SCORE(iBeta));
ASSERT(iAlpha < iBeta);
@@ -1418,25 +1495,32 @@ FLAG WeShouldTryNullmovePruning(SEARCHER_THREAD_CONTEXT *ctx,
ASSERT(uNullDepth < MAX_PLY_PER_SEARCH * ONE_PLY);
if ((FALSE == pf->fAvoidNullmove) &&
- (TotalNonPawnCount(pos, pos->uToMove) > 2) &&
(FALSE == pi->fInCheck) &&
(iBeta != +INFINITY_SCORE) &&
(NODE_PV != pi->eNodeType))
{
- if (uNullDepth <= 6 * ONE_PLY)
+ uMoverPiecesPerSide = TotalNonPawnCount(pos, pos->uToMove);
+ iDeficit = iEval - iBeta;
+
+ // ---- Stage 1: safety ----
+ if (uMoverPiecesPerSide <= 1)
{
- u = uNullDepth / ONE_PLY;
- ASSERT(u <= 6);
- SCORE iMargin = iSkipShallowNullIfThisFarBelowAlpha[u];
- iMargin -= iImprovement; // Note: if improving, makes null more likely
- iMargin = MAX(0, iMargin); // if not, less likely.
- if ((iEval + iMargin <= iAlpha) ||
- ((iEval + iMargin / 2 <= iAlpha) &&
- (ValueOfMaterialInTroubleAfterNull(ctx, pos->uToMove))))
- {
- return FALSE;
- }
+ return FALSE; // never measured safe, even at its best deficit
+ }
+ if ((uMoverPiecesPerSide == 2) && (iDeficit < 100))
+ {
+ return FALSE;
+ }
+ if ((uMoverPiecesPerSide <= 5) && (iDeficit < -100))
+ {
+ return FALSE;
}
+ // 6+ pieces: no deficit floor -- safe everywhere measured.
+
+ // ---- Stage 2: efficiency/EV ----
+ // EV is strongly positive at every depth/material cell that
+ // survives Stage 1 (see routine comment) -- nothing further to
+ // gate on here today.
return TRUE;
}
return FALSE;
diff --git a/src/testrecogn.c b/src/testrecogn.c
index b64643e..10ee889 100644
--- a/src/testrecogn.c
+++ b/src/testrecogn.c
@@ -295,7 +295,7 @@ _TestRecognFenCases(IN OUT SEARCHER_THREAD_CONTEXT *ctx, IN OUT POSITION *pos)
}
ReInitializeSearcherContext(pos, ctx);
- uVal = RecognLookup(ctx, &iScore, FALSE);
+ uVal = RecognLookup(ctx, &iScore, EGTB_PROBE_NEVER);
if ((UNRECOGNIZED != uVal) &&
((iScore <= -NMATE) || (iScore >= NMATE)))
{
@@ -387,7 +387,7 @@ Return value:
uPawnB, uKnightB, uBishopB);
ReInitializeSearcherContext(&pos, ctx);
- uVal = RecognLookup(ctx, &iScore, FALSE);
+ uVal = RecognLookup(ctx, &iScore, EGTB_PROBE_NEVER);
if (UNRECOGNIZED == uVal)
{
uUnrecognized++;
@@ -624,6 +624,267 @@ Return value:
}
//
+// Build a KQKP (one queen, one pawn) position directly from square
+// placements. fBlackHasQueen selects which color plays the queen side
+// (cStrongKing/cWeakKing name roles, not colors, in both cases) -- see
+// TestRecognExhaustiveKQKP's header comment for why both assignments
+// are enumerated explicitly rather than assumed symmetric.
+//
+static FLAG
+_BuildKQKPPosition(OUT POSITION *pos,
+ IN COOR cStrongKing, IN COOR cWeakKing,
+ IN COOR cQueen, IN COOR cPawn,
+ IN ULONG uToMove, IN FLAG fBlackHasQueen)
+{
+ PIECE pBoard[128];
+ CHAR szFen[256];
+ CHAR *p;
+ ULONG x;
+
+ memset(pBoard, 0, sizeof(pBoard));
+ if (fBlackHasQueen)
+ {
+ pBoard[cStrongKing] = BLACK_KING;
+ pBoard[cWeakKing] = WHITE_KING;
+ pBoard[cQueen] = BLACK_QUEEN;
+ pBoard[cPawn] = WHITE_PAWN;
+ }
+ else
+ {
+ pBoard[cStrongKing] = WHITE_KING;
+ pBoard[cWeakKing] = BLACK_KING;
+ pBoard[cQueen] = WHITE_QUEEN;
+ pBoard[cPawn] = BLACK_PAWN;
+ }
+
+ memset(szFen, 0, sizeof(szFen));
+ p = szFen;
+ for (x = 0; x < 120; x++)
+ {
+ if ((x % 15 == 0) && (x != 0))
+ {
+ *p++ = '/';
+ }
+ if (!IS_ON_BOARD(x)) continue;
+ if (0 == pBoard[x])
+ {
+ *p++ = '1';
+ }
+ else
+ {
+ *p++ = *(PieceAbbrev(pBoard[x]) + 1);
+ }
+ }
+ *p++ = ' ';
+ *p++ = (WHITE == uToMove) ? 'w' : 'b';
+ strcat(p, " - - 0 1");
+
+ if (FALSE == FenToPosition(pos, szFen))
+ {
+ return(FALSE);
+ }
+ if (InCheck(pos, FLIP(pos->uToMove)))
+ {
+ return(FALSE);
+ }
+ return(TRUE);
+}
+
+//
+// Does the side to move in ctx's current position have at least one
+// legal move? Used only to classify a mismatch as "the to-move side is
+// simply stalemated" vs. something else -- not part of the recognizer
+// itself. Mirrors root.c's _DetectPreMoveTerminalStates legal-move
+// count idiom (GenerateMoves then MakeMove/UnmakeMove-filter), just
+// short-circuiting on the first legal move found instead of counting
+// them all.
+//
+static FLAG
+_ToMoveHasLegalMove(IN OUT SEARCHER_THREAD_CONTEXT *ctx)
+{
+ ULONG x;
+ MOVE mv;
+ FLAG fInCheck = InCheck(&ctx->sPosition, ctx->sPosition.uToMove);
+
+ GenerateMoves(ctx, NULLMOVE,
+ fInCheck ? GENERATE_ESCAPES : GENERATE_ALL_MOVES);
+ for (x = ctx->sMoveStack.uBegin[ctx->uPly];
+ x < ctx->sMoveStack.uEnd[ctx->uPly];
+ x++)
+ {
+ mv = ctx->sMoveStack.mvf[x].mv;
+ if (TRUE == MakeMove(ctx, mv))
+ {
+ UnmakeMove(ctx, mv);
+ return(TRUE);
+ }
+ }
+ return(FALSE);
+}
+
+void
+TestRecognExhaustiveKQKP(void)
+/**
+
+Routine description:
+
+ Exhaustively enumerate every legal KQKP (one queen, one pawn)
+ position -- not a random sample -- and verify _RecognizeKQKP's
+ verdict against direct EGTB probes wherever coverage exists. Same
+ "no counterexamples in N random samples is false confidence"
+ rationale as TestRecognExhaustiveKNKP/KPK -- doubly warranted here
+ since _RecognizeKQKP's win/UNRECOGNIZED boundary (rook/bishop pawn
+ on the square before queening) was scoped down from memory of
+ classical endgame theory rather than derived from EGTB data, and
+ this file has real precedent (_RecognizeKBKP's 2026-09-18 history)
+ for a plausible-looking classical shortcut turning out unsound once
+ fuzzed.
+
+ Learned from TestRecognExhaustiveKPK's own 2026-09-18 history:
+ fixing which COLOR plays the strong side is NOT a safe assumption
+ in this codebase, even though RecognLookup/_RecognizeKQKP determine
+ "strong side" dynamically from piece counts -- iterates both
+ fBlackHasQueen assignments explicitly rather than trusting that
+ symmetry. Unlike TestRecognExhaustiveKPK, this keeps the left/right
+ mirror-halving restriction (FILE(cStrongKing) <= D, same as
+ TestRecognExhaustiveKNKP) -- safe here because, unlike _RecognizeKPK's
+ hand-typed critical-square table, _RecognizeKQKP contains no
+ per-square lookup data at all: its only file-dependent test is
+ "FILE(cPawn) == A/C/F/H", which treats both rook files and both
+ bishop files identically by construction, so a left/right board
+ reflection cannot expose an asymmetric transcription bug the way it
+ did for KPK's table.
+
+ Not wired into TestRecogn()'s automatic startup suite (same
+ precommit_check.sh budget reasoning as KNKP/KPK) -- call this on
+ demand when validating a KQKP-area change.
+
+Parameters:
+
+ void
+
+Return value:
+
+ void
+
+**/
+{
+ SEARCHER_THREAD_CONTEXT *ctx;
+ POSITION pos;
+ COOR cStrongKing, cWeakKing, cQueen, cPawn;
+ ULONG uToMove;
+ SCORE iScore, iEgtb;
+ ULONG uVal;
+ ULONG uTotal = 0, uChecked = 0, uSkipped = 0, uWrong = 0;
+ ULONG uReportedWrong = 0;
+ ULONG uWrongStalemated = 0;
+ FLAG fStalemated;
+
+ Trace("Exhaustive KQKP (1 queen, 1 pawn) verification against "
+ "EGTB ground truth -- this takes a while...\n");
+ ctx = malloc(sizeof(SEARCHER_THREAD_CONTEXT));
+ if (NULL == ctx)
+ {
+ return;
+ }
+ InitializeSearcherContext(&pos, ctx);
+
+ FOREACH_SQUARE(cStrongKing)
+ {
+ if (!IS_ON_BOARD(cStrongKing)) continue;
+ if (FILE(cStrongKing) > D) continue;
+
+ FOREACH_SQUARE(cWeakKing)
+ {
+ if (!IS_ON_BOARD(cWeakKing)) continue;
+ if (DISTANCE(cStrongKing, cWeakKing) < 2) continue;
+
+ FOREACH_SQUARE(cQueen)
+ {
+ if (!IS_ON_BOARD(cQueen)) continue;
+ if ((cQueen == cStrongKing) || (cQueen == cWeakKing)) continue;
+
+ FOREACH_SQUARE(cPawn)
+ {
+ if (!IS_ON_BOARD(cPawn)) continue;
+ if (RANK1(cPawn) || RANK8(cPawn)) continue;
+ if ((cPawn == cStrongKing) || (cPawn == cWeakKing) ||
+ (cPawn == cQueen)) continue;
+
+ for (uToMove = BLACK; uToMove <= WHITE; uToMove++)
+ {
+ ULONG uColorPass;
+
+ for (uColorPass = 0; uColorPass < 2; uColorPass++)
+ {
+ uTotal++;
+ if (0 == (uTotal % 500000))
+ {
+ Trace(" ... %lu raw combos so far (%lu checked, "
+ "%lu wrong)\n", uTotal, uChecked, uWrong);
+ }
+ if (FALSE == _BuildKQKPPosition(&pos, cStrongKing,
+ cWeakKing, cQueen,
+ cPawn, uToMove,
+ (1 == uColorPass)))
+ {
+ continue;
+ }
+ ReInitializeSearcherContext(&pos, ctx);
+ uVal = _RecognizeKQKP(ctx, &iScore);
+ if (UNRECOGNIZED == uVal)
+ {
+ continue;
+ }
+ if (FALSE == ProbeEGTB(ctx, &iEgtb))
+ {
+ uSkipped++;
+ continue;
+ }
+ uChecked++;
+ if (FALSE == _RecognizerAgreesWithEGTB(iScore, uVal,
+ iEgtb))
+ {
+ uWrong++;
+ fStalemated =
+ (FALSE == InCheck(&ctx->sPosition,
+ ctx->sPosition.uToMove)) &&
+ (FALSE == _ToMoveHasLegalMove(ctx));
+ if (TRUE == fStalemated)
+ {
+ uWrongStalemated++;
+ }
+ if (uReportedWrong < 20)
+ {
+ Trace("MISMATCH (blackHasQueen=%lu) iScore=%d "
+ "uVal=%lu iEgtb=%d toMove=%lu "
+ "stalemated=%lu\n",
+ uColorPass, iScore, uVal, iEgtb,
+ pos.uToMove, fStalemated);
+ DumpPosition(&pos);
+ uReportedWrong++;
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+
+ {
+ ULONG uWrongOther = uWrong - uWrongStalemated;
+
+ free(ctx);
+ Trace("Exhaustive KQKP: %lu raw combos, %lu checked against EGTB, "
+ "%lu skipped (no EGTB coverage), %lu WRONG (%lu of those are "
+ "the weak side simply stalemated, %lu are something else).\n",
+ uTotal, uChecked, uSkipped, uWrong, uWrongStalemated,
+ uWrongOther);
+ }
+}
+
+//
// Build a KPK (one pawn, one lone king) position directly from square
// placements. fBlackHasPawn selects which color plays the pawn side
// (cStrongKing/cWeakKing name roles, not colors, in both cases).
@@ -845,7 +1106,7 @@ Return value:
continue;
}
ReInitializeSearcherContext(&pos, ctx);
- uVal = RecognLookup(ctx, &iScore, FALSE);
+ uVal = RecognLookup(ctx, &iScore, EGTB_PROBE_NEVER);
if (UNRECOGNIZED == uVal)
{
continue;
@@ -1227,7 +1488,7 @@ Return value:
continue;
}
ReInitializeSearcherContext(&pos, ctx);
- uVal = RecognLookup(ctx, &iScore, FALSE);
+ uVal = RecognLookup(ctx, &iScore, EGTB_PROBE_NEVER);
if (UNRECOGNIZED == uVal)
{
uUnrecognized++;
@@ -1758,7 +2019,7 @@ Return value:
_MakeRandomEndgame(&pos, uPawnA, 0, 0, uPawnB, 0, 0);
ReInitializeSearcherContext(&pos, ctx);
- uVal = RecognLookup(ctx, &iScore, FALSE);
+ uVal = RecognLookup(ctx, &iScore, EGTB_PROBE_NEVER);
if (UNRECOGNIZED == uVal)
{
uUnrecognized++;
@@ -2011,7 +2272,7 @@ Return value:
// signature routing a real search would use.
//
{
- ULONG uVal = RecognLookup(ctx, &iRecScore, FALSE);
+ ULONG uVal = RecognLookup(ctx, &iRecScore, EGTB_PROBE_NEVER);
if (UNRECOGNIZED != uVal)
{
uRecognClaimed++;