Analyze post-fix PFE memory growth and CPU peaks

Analyze the verified post-navigation-fix capture from production image commit 16e2352911449e4ee33d86addcf6a53f42075eff (capture task tsk_3483jn8une) and compare it with the pre-fix 2026-10-03 dump/investigation tsk_73fyezjdwp. New primary PFE capture at ~6h uptime: heap dump 18,586,329,088 bytes plus 60-second dotnet-sampled-thread-time + gc-verbose nettrace 93,712,354 bytes. Determine why process memory continues growing and why all four PFEs peak at roughly 500–580% CPU. Quantify managed heap, glibc arenas/free/live chunks, Bepu pools and current/detached Detour state; distinguish fixed leak, residual allocator retention and another leak/cache. Analyze CPU stacks, sample attribution, allocation hot paths and GC pauses; trace dominant paths into deployed commit. Read-only offline investigation; no production experiments, restarts or repository edits. Verified findings Memory composition and comparison - Capture-time RSS ~16.6 GiB at ~6h uptime. Managed GC: 6,138,101,760 committed / 6,120,036,816 allocated; object walk 6,111,041,293 bytes including 997,386,984 bytes Free. Thus roughly 4.76 GiB is non-Free managed object data and ~0.93 GiB managed holes. - Native ptmalloc: 217 secondary heaps / 134 arenas totaling 11,307,401,216 bytes (10.531 GiB). Only 1,491,093,312 bytes (1.389 GiB) are in-use-marked; 9,816,000,288 bytes (9.142 GiB) are free but retained (3.951 GiB top chunks, 5.191 GiB interior). The arena system_mem sum independently equals the heap_info mapping sum. - 155 heaps with <=1 MiB live retain 6.974 GiB free. Arena-head heaps alone retain 5.252 GiB free. This confirms glibc arena/high-water retention is still the dominant growth mechanism, now without orphan Detour buffers. - Pre-fix comparison: 20.074 GiB ptmalloc / 17.720 GiB free / 139 arenas / 358 heaps at ~27h versus 10.531 / 9.142 GiB / 134 / 217 post-fix at ~6h. Managed committed grew from 3.653 to 5.717 GiB because the newer capture has a larger live world/cache set; captures are not same uptime/workload, so do not infer a linear rate from their difference. - Current live managed footprint is world/entity dominated. StatPair grew 6.07m/486 MB -> 9.95m/796 MB; stat dictionary nodes 243 -> 398 MB; StatBlock 350k -> 586k; NPCs 10,053 -> 16,252; wearable instances 235k -> 413k; nanos 138k -> 262k. Representative StatPair is rooted through a current mission playfield/cell/NPC. Local locator has 46 current playfields and only four weak disposed probes; no evidence of accumulating disposed playfields. - New managed NavigationTile cache moves serialized source arrays out of unmanaged HGlobal ownership. It contains 46,424 immutable tile objects (~10.4 MB for tile/settings plus managed arrays) and preserves the record cache; this is bounded cache/world footprint, not orphan native ownership. Navigation and physics native inventory - 46 NavigationHandle objects: 45 active state=4 and one disposed state=3. All 45 sessions have owning mesh/query pairs. Validate 45 2,048-node pools, 37 discoverable 64-node pools and 42 populated tile arrays / 27,634 tile blobs / 219,525,488 bytes. Detached validated node pools: zero. Detached validated tile arrays/blobs: zero. The previous ~971 MiB Detour leak is fixed. - 46 current Bepu BufferPools own 1,194,328,064 bytes (1.112 GiB), up from 39 pools / 0.769 GiB pre-fix because more current playfields are loaded. Their 7,513 native block pointers validate against 1,194,450,640 bytes of malloc chunks. Current Bepu blocks occupy 64 heaps across 40 arenas; those heaps contain 2.232 GiB free, 0.343 GiB immediately adjacent. This proves Bepu dominates live native demand but cannot attribute every coalesced historical free chunk to Bepu. - Program still calls ThreadPool.SetMinThreads(1000,1000). Dump has 112 Thread objects and 89 worker-local queues; trace has 111 thread profiles. High thread fan-out permits many glibc arenas. This is an evidence-backed amplifier, not proof that setting minimums alone creates all threads. CPU/allocation trace - Trace duration 61.861s; managed allocation ticks estimate 8,879,046,864 bytes (~136.9 MiB/s). 1,029 GCs (~16.63/s): 730 gen0, 299 gen1, no gen2. GC event duration totals 7.17s, p50 5.47ms, p95 15.44ms, max 33.40ms. gc-verbose instrumentation also caused frequent suspension/stack-walk overhead, so total suspension and CPU percentages are not an uninstrumented baseline. - DynelCollisionPfSystem.OnNpcTick/CheckCharacterDynelProximity accounts for ~2.63 GiB (29.6% sampled allocation) via compiler DisplayClass23_0; player tick adds ~95 MB. Capturing Task.Run lambdas force a display-class allocation on every method invocation even when occupancy immediately returns. The system also scans every dynel and terminal for every ticking NPC. - WallCollisionPfSystem.CheckCollision accounts for ~896 MB (10.1%). DotProduct/CrossProduct create two float[2] arrays per call and execute repeatedly per wall segment/player tick. - CorpseManagement per-tick snapshots account for ~241 MB; LINQ Where/Union, collection snapshots and stat Values enumeration are major secondary sources. These allocations drive the measured 16.6 minor GCs/s and consume CPU. - Managed wall-clock stacks (not kernel CPU samples) show heartbeat systems as active work. Leading actionable active hotspot is FightTallyPfSystem.ReconcileIsFightingMe (3.67% inclusive / 1.56% exclusive of all thread-time samples): every character's 10-second reconcile scans all NPCs and players, quadratic at playfield scale. Dynel collision, brute-force ACG triangle rays, Bepu overlap queries and door handling follow. 56% of all samples are WaitNative because dotnet-sampled-thread-time samples waiting threads; therefore these percentages must not be presented as exact shares of the observed 500–580% container CPU. Conclusion PR #2555 fixed the concrete navigation leak, but it was not the dominant source of the monotonic RSS curve. Memory growth is primarily freed native memory retained across 134 glibc arenas after high-thread native churn, layered with a legitimate ~5 GiB managed world/cache footprint and ~1.3 GiB live native physics/navigation. CPU peaks are strongly driven by a ~137 MiB/s managed allocation storm plus quadratic/full-scan heartbeat paths, but exact core attribution needs a Linux perf/collect-linux profile. Follow-ups - tsk_pwwgzyv3kz: reduce PFE native allocator retention after playfield churn (arena cap/thread minimum/churn harness, contention and latency gates). - tsk_eqdu5zwbmf: remove heartbeat allocation storms and quadratic scans (dynel closures/spatial cadence, scalar wall math, FightTally index, snapshots, true CPU perf gate). Evidence is stored outside the repository under /home/mathias/Downloads/pfe-memory-20261004/ including heap-summary.txt, trace-summary.txt, CPU top reports and analysis/{arena-totals,pool-totals,navigation-inventory,physics-fragmentation,managed-diff,native-classification}.txt. No repository or production changes.