# qwen38 CUDA chain, up to 32 experts a forward against 128 with the brake: the prose that wrote the history (2026-10-25)
# Vast.ai instance: AMD EPYC 7443P 24-Core Processor (24 cores / 48 threads, 128 MiB L3; the container's quota 11.52 CPUs, OMP_NUM_THREADS=10) + NVIDIA RTX PRO 4000 Blackwell (24467 MiB, sm_120, PCIe 4 x16: 28.0 GB/s to the card and 558 GB/s on it, measured);
#   251.6 GiB on the host, the container's memory.max 120.7 GiB; Ubuntu 24.04.1 LTS (container nvidia/cuda:12.8.1-devel-ubuntu24.04) on Linux 6.8.0-110-generic x86_64, Vast.ai machine 59339, driver 580.142.
# Model: unsloth/Qwen3.8-Flash-Next-GGUF, UD-Q2_K_XL (Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf sha256 a4f3b21e77353999829f2f767e9ac21ce9c71d29a74f2cc9eda48c9bf23c8b86, Qwen3.8-Flash-Next-UD-Q2_K_XL-00002-of-00003.gguf sha256 2e3bf1ee7d2a04e261e9f342a2d968f696cce5941d082b0e434deb9b1edc12c6, Qwen3.8-Flash-Next-UD-Q2_K_XL-00003-of-00003.gguf sha256 ec8c106759fdf4f463039c34c0707718d7d8908d53d892bd4f002e71620803f9)
#   imported with python3 tools/gguf_import.py <the first file> <model> --no-mtp.
# v0 = origin/dev 7bbea341 (qwen38-dev); v1 = 7bbea341 + this change (qwen38-pr: the measured diff, sha256
#   391dea7bbd855bc2cc3892c5e60f7cec712a8200551ca4fc9ae5f64e35a4e5f9, differs from this commit only in comments). Both built with:
#   cd c && make -j qwen38 CUDA=1 CUDA_ARCH=native
# History: history.usage.gz beside this file (gunzip to <history.usage>, sha256 8dc23b015a8080b48b2e7ef7878f51516ac64f481bd104bba56f96a021341459),
#   written by a run of the prose prompt on another machine (134880 selections); USAGE_SAVE=0, so no run changes it.
# Each run, a new process: env COLI_CUDA=1 COLI_VULKAN=0 COLI_TEMP=0 COLI_TIMERS=1 N_NEW=128 USAGE_SAVE=0 OMP_NUM_THREADS=10 OMP_WAIT_POLICY=ACTIVE OMP_PROC_BIND=close SNAP=<model> COLI_USAGE=<history.usage> ./qwen38 128 8 <prompt>
# Prompt (sha256 cd8eba88480c681ec210d697557c5a727fd5e297bfe91c30c3e36dc9b55f5b45):
#   It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife. However little known the feelings or views of such a man may be on his first entering a neighbourhood, this truth is so well fixed in the minds of the surrounding families, that he is considered the rightful property of some one or other of their daughters. "My dear Mr. Bennet," said his lady to him one day, "have you heard that Netherfield Park is let at last?" Mr. Bennet replied that he had not. "But it is," returned she; "for Mrs. Long has just been here, and she told me all about it." Mr. Bennet made no answer.
# A warm-up run of v0 (not counted), then v0 v1 v1 v0 v0 v1 v1 v0, 60 s of rest before each. Every run's stderr
#   and stdout in full, in the order they ran. decode tok/s = (n - 1) / (n / S - TTFT), S and n from the
#   "Speed" line, TTFT from its line (the server's q38_decode_rate over the same interval).

===== A-w.err (sha256 e808763b0d8e68c1ef4bbbbe9616ea47eed1d993c78ae7d7ad8d7d32a66693db)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 150 | generating 128 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 1.59 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 22.2 GB
[place] auto: dev 0 holds 4700.6 MB of trunk (lmhead yes, 0 dnproj layers), 17.61 GB left for experts
[qtier] dev 0: budget 17.61 GB for experts (~8642 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 8642 slots of 2.09 MB (17.61 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.0s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 8642 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.5 s
[chain] qwen38: host pool, every expert (the 15934 the card leaves and its own 8642) in 49.80 GiB of page-locked RAM the device reads over the bus (18.9 s to allocate, 2.6 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 8642 slots are the chain's to fill (8642 held): up to 32 experts a forward promoted from the pool by this conversation's routes (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.4s | RSS after load: 54.26 GB
Generated (128 new tokens):
Text : 
TTFT: 1.32 s (time to first token)
[qwen38 spec gate] run: T(1) 19.1 ms, rows 2:1.23 3:1.73, rho 0.28, acceptance by position 0.31 0.67 0.50 0.50 0.50
[qwen38 lookup] run: 1.04 tokens/forward (122 forwards per 127 tokens) | acceptance 29.4% (5/17 drafts in 14 verifies) | gate on, 3 declined, 4 probes
[qwen38 timers] total: 123 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.656 s     13.464 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        0.345 s    344.753 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 122 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.311 s     10.748 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.38 GB
[qtier] resident 8642/24576 experts | uploads 8642 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (4.59 GB trunk + experts, budget 17.61 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 34.22 tok/s (3.7s for 128 tokens)
[chain] qwen38: 123 forwards, 249 frames (231370 ops, 95079 matmuls), 1656.1 ms waiting for the device, 0.0 ms of routed experts on the host, 1004.6 MiB on the device; per decode step 10.75 ms waiting, 0.00 ms of experts (122 steps)
[chain] qwen38: 5904 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 3911 slots from the host pool
[chain] qwen38: 48 prompt layer steps staged the host pool's experts (22.68 GB copied over the bus)
[chain] qwen38: 1372 experts promoted to the card (1372 residents back to the pool), 18 on their way
	Command being timed: "/workspace/bin/qwen38-dev 128 8 /workspace/prova/prompt120.txt"
	User time (seconds): 238.40
	System time (seconds): 45.78
	Percent of CPU this job got: 754%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:37.65
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57024684
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1380418
	Voluntary context switches: 172870
	Involuntary context switches: 697
	Swaps: 0
	File system inputs: 7805696
	File system outputs: 48
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== A-w.out (sha256 4c76410738a874c9ecd56f1c50da135a25822d0058b892f8fe5a41df1c3decf7)
 "Do you not want to know who has taken it?" cried his wife impatiently. "You want to tell me, and I have no objection to hearing it." This was invitation enough. "Why, my dear, you must know, Mrs. Long says that Netherfield is taken by a young man of large fortune from the north of England; that he came down on Monday in a chaise and four to see the place, and was so much delighted with it, that he agreed with Mr. Morris immediately; that he is to take possession before Michaelmas, and some of his servants are to be in the house by

===== A-1-dev.err (sha256 a48f671bbe04f13a18dbba36e3959fe55639a6a68f17fd6e4df88c0feb7ae2f4)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 150 | generating 128 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 1.59 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 22.2 GB
[place] auto: dev 0 holds 4700.6 MB of trunk (lmhead yes, 0 dnproj layers), 17.61 GB left for experts
[qtier] dev 0: budget 17.61 GB for experts (~8642 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 8642 slots of 2.09 MB (17.61 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.1s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 8642 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.4 s
[chain] qwen38: host pool, every expert (the 15934 the card leaves and its own 8642) in 49.80 GiB of page-locked RAM the device reads over the bus (18.9 s to allocate, 2.0 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 8642 slots are the chain's to fill (8642 held): up to 32 experts a forward promoted from the pool by this conversation's routes (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 2.4s | RSS after load: 54.26 GB
Generated (128 new tokens):
Text : 
TTFT: 1.32 s (time to first token)
[qwen38 spec gate] run: T(1) 19.0 ms, rows 2:1.29 3:2.00, rho 0.37, acceptance by position 0.31 0.67 0.50 0.50 0.50
[qwen38 lookup] run: 1.04 tokens/forward (122 forwards per 127 tokens) | acceptance 29.4% (5/17 drafts in 14 verifies) | gate on, 3 declined, 4 probes
[qwen38 timers] total: 123 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.686 s     13.711 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        0.344 s    344.300 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 122 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.342 s     11.001 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.39 GB
[qtier] resident 8642/24576 experts | uploads 8642 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (4.59 GB trunk + experts, budget 17.61 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 34.10 tok/s (3.8s for 128 tokens)
[chain] qwen38: 123 forwards, 249 frames (231325 ops, 95079 matmuls), 1686.4 ms waiting for the device, 0.0 ms of routed experts on the host, 1004.6 MiB on the device; per decode step 11.00 ms waiting, 0.00 ms of experts (122 steps)
[chain] qwen38: 5904 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 3931 slots from the host pool
[chain] qwen38: 48 prompt layer steps staged the host pool's experts (22.68 GB copied over the bus)
[chain] qwen38: 1346 experts promoted to the card (1346 residents back to the pool), 15 on their way
	Command being timed: "/workspace/bin/qwen38-dev 128 8 /workspace/prova/prompt120.txt"
	User time (seconds): 243.88
	System time (seconds): 47.86
	Percent of CPU this job got: 766%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:38.06
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57030452
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1381399
	Voluntary context switches: 150014
	Involuntary context switches: 661
	Swaps: 0
	File system inputs: 1760552
	File system outputs: 56
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== A-1-dev.out (sha256 4c76410738a874c9ecd56f1c50da135a25822d0058b892f8fe5a41df1c3decf7)
 "Do you not want to know who has taken it?" cried his wife impatiently. "You want to tell me, and I have no objection to hearing it." This was invitation enough. "Why, my dear, you must know, Mrs. Long says that Netherfield is taken by a young man of large fortune from the north of England; that he came down on Monday in a chaise and four to see the place, and was so much delighted with it, that he agreed with Mr. Morris immediately; that he is to take possession before Michaelmas, and some of his servants are to be in the house by

===== A-2-pr.err (sha256 85e4b706196ca73b8f18d9d5936156e906439d1d291d7598af12b105e27ff801)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 150 | generating 128 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 1.59 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 22.2 GB
[place] auto: dev 0 holds 4700.6 MB of trunk (lmhead yes, 0 dnproj layers), 17.61 GB left for experts
[qtier] dev 0: budget 17.61 GB for experts (~8642 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 8642 slots of 2.09 MB (17.61 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.2s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 8642 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.7 s
[chain] qwen38: host pool, every expert (the 15934 the card leaves and its own 8642) in 49.80 GiB of page-locked RAM the device reads over the bus (18.7 s to allocate, 2.0 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 8642 slots are the chain's to fill (8642 held): up to 128 experts a forward promoted from the pool by this conversation's routes, none while the last 64 frames found 90 % or more of their routes on the card (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.4s | RSS after load: 54.26 GB
Generated (128 new tokens):
Text : 
TTFT: 1.34 s (time to first token)
[qwen38 spec gate] run: T(1) 18.1 ms, rows 2:1.23, rho 0.30, acceptance by position 0.17 0.50 0.50 0.50 0.50
[qwen38 lookup] run: 1.00 tokens/forward (127 forwards per 127 tokens) | acceptance 0.0% (0/4 drafts in 4 verifies) | gate on, 18 declined, 2 probes
[qwen38 timers] total: 128 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.648 s     12.875 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        0.345 s    345.215 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 127 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.303 s     10.258 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.38 GB
[qtier] resident 8642/24576 experts | uploads 8642 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (4.59 GB trunk + experts, budget 17.61 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 35.00 tok/s (3.7s for 128 tokens)
[chain] qwen38: 128 forwards, 131 frames (237412 ops, 98944 matmuls), 1648.0 ms waiting for the device, 0.0 ms of routed experts on the host, 890.8 MiB on the device; per decode step 10.26 ms waiting, 0.00 ms of experts (127 steps)
[chain] qwen38: 6144 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 2876 slots from the host pool
[chain] qwen38: 48 prompt layer steps staged the host pool's experts (22.68 GB copied over the bus)
[chain] qwen38: 0 experts promoted to the card (0 residents back to the pool), 0 on their way; 128 frames braked, 97.9 % of the routes on the card
	Command being timed: "/workspace/bin/qwen38-pr 128 8 /workspace/prova/prompt120.txt"
	User time (seconds): 238.01
	System time (seconds): 47.75
	Percent of CPU this job got: 766%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:37.26
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57019564
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1378848
	Voluntary context switches: 142295
	Involuntary context switches: 641
	Swaps: 0
	File system inputs: 0
	File system outputs: 48
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== A-2-pr.out (sha256 4c76410738a874c9ecd56f1c50da135a25822d0058b892f8fe5a41df1c3decf7)
 "Do you not want to know who has taken it?" cried his wife impatiently. "You want to tell me, and I have no objection to hearing it." This was invitation enough. "Why, my dear, you must know, Mrs. Long says that Netherfield is taken by a young man of large fortune from the north of England; that he came down on Monday in a chaise and four to see the place, and was so much delighted with it, that he agreed with Mr. Morris immediately; that he is to take possession before Michaelmas, and some of his servants are to be in the house by

===== A-3-pr.err (sha256 7b4a300c7462bb0b8e0171f0d35daad9c1e041c1b53af61705476282ec6e34d0)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 150 | generating 128 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 1.59 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 22.2 GB
[place] auto: dev 0 holds 4700.6 MB of trunk (lmhead yes, 0 dnproj layers), 17.61 GB left for experts
[qtier] dev 0: budget 17.61 GB for experts (~8642 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 8642 slots of 2.09 MB (17.61 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.2s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 8642 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.7 s
[chain] qwen38: host pool, every expert (the 15934 the card leaves and its own 8642) in 49.80 GiB of page-locked RAM the device reads over the bus (18.7 s to allocate, 2.0 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 8642 slots are the chain's to fill (8642 held): up to 128 experts a forward promoted from the pool by this conversation's routes, none while the last 64 frames found 90 % or more of their routes on the card (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.4s | RSS after load: 54.25 GB
Generated (128 new tokens):
Text : 
TTFT: 1.35 s (time to first token)
[qwen38 spec gate] run: T(1) 19.0 ms, rows 2:1.31, rho 0.36, acceptance by position 0.17 0.50 0.50 0.50 0.50
[qwen38 lookup] run: 1.00 tokens/forward (127 forwards per 127 tokens) | acceptance 0.0% (0/4 drafts in 4 verifies) | gate on, 18 declined, 3 probes
[qwen38 timers] total: 128 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.779 s     13.898 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        0.345 s    344.766 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 127 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.434 s     11.293 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.38 GB
[qtier] resident 8642/24576 experts | uploads 8642 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (4.59 GB trunk + experts, budget 17.61 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 33.72 tok/s (3.8s for 128 tokens)
[chain] qwen38: 128 forwards, 131 frames (237412 ops, 98944 matmuls), 1778.9 ms waiting for the device, 0.0 ms of routed experts on the host, 890.8 MiB on the device; per decode step 11.29 ms waiting, 0.00 ms of experts (127 steps)
[chain] qwen38: 6144 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 2869 slots from the host pool
[chain] qwen38: 48 prompt layer steps staged the host pool's experts (22.68 GB copied over the bus)
[chain] qwen38: 0 experts promoted to the card (0 residents back to the pool), 0 on their way; 128 frames braked, 97.9 % of the routes on the card
	Command being timed: "/workspace/bin/qwen38-pr 128 8 /workspace/prova/prompt120.txt"
	User time (seconds): 238.73
	System time (seconds): 47.82
	Percent of CPU this job got: 767%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:37.33
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57016888
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1378451
	Voluntary context switches: 147269
	Involuntary context switches: 638
	Swaps: 0
	File system inputs: 0
	File system outputs: 56
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== A-3-pr.out (sha256 4c76410738a874c9ecd56f1c50da135a25822d0058b892f8fe5a41df1c3decf7)
 "Do you not want to know who has taken it?" cried his wife impatiently. "You want to tell me, and I have no objection to hearing it." This was invitation enough. "Why, my dear, you must know, Mrs. Long says that Netherfield is taken by a young man of large fortune from the north of England; that he came down on Monday in a chaise and four to see the place, and was so much delighted with it, that he agreed with Mr. Morris immediately; that he is to take possession before Michaelmas, and some of his servants are to be in the house by

===== A-4-dev.err (sha256 62958b54aa2e1db006289621716d4b5ef33b6fafe0231abf215903ccb14cd711)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 150 | generating 128 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 1.59 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 22.2 GB
[place] auto: dev 0 holds 4700.6 MB of trunk (lmhead yes, 0 dnproj layers), 17.61 GB left for experts
[qtier] dev 0: budget 17.61 GB for experts (~8642 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 8642 slots of 2.09 MB (17.61 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.0s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 8642 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.6 s
[chain] qwen38: host pool, every expert (the 15934 the card leaves and its own 8642) in 49.80 GiB of page-locked RAM the device reads over the bus (18.6 s to allocate, 2.0 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 8642 slots are the chain's to fill (8642 held): up to 32 experts a forward promoted from the pool by this conversation's routes (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.4s | RSS after load: 54.27 GB
Generated (128 new tokens):
Text : 
TTFT: 1.32 s (time to first token)
[qwen38 spec gate] run: T(1) 20.5 ms, rows 2:1.25, rho 0.29, acceptance by position 0.14 0.50 0.50 0.50 0.50
[qwen38 lookup] run: 1.00 tokens/forward (127 forwards per 127 tokens) | acceptance 0.0% (0/5 drafts in 5 verifies) | gate on, 17 declined, 2 probes
[qwen38 timers] total: 128 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.893 s     14.792 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        0.347 s    346.617 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 127 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.547 s     12.179 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.39 GB
[qtier] resident 8642/24576 experts | uploads 8642 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (4.59 GB trunk + experts, budget 17.61 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 32.31 tok/s (4.0s for 128 tokens)
[chain] qwen38: 128 forwards, 258 frames (240006 ops, 98944 matmuls), 1893.4 ms waiting for the device, 0.0 ms of routed experts on the host, 890.8 MiB on the device; per decode step 12.18 ms waiting, 0.00 ms of experts (127 steps)
[chain] qwen38: 6144 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 3724 slots from the host pool
[chain] qwen38: 48 prompt layer steps staged the host pool's experts (22.68 GB copied over the bus)
[chain] qwen38: 1297 experts promoted to the card (1297 residents back to the pool), 8 on their way
	Command being timed: "/workspace/bin/qwen38-dev 128 8 /workspace/prova/prompt120.txt"
	User time (seconds): 237.68
	System time (seconds): 47.81
	Percent of CPU this job got: 768%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:37.14
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57031604
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1381956
	Voluntary context switches: 145035
	Involuntary context switches: 663
	Swaps: 0
	File system inputs: 0
	File system outputs: 56
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== A-4-dev.out (sha256 4c76410738a874c9ecd56f1c50da135a25822d0058b892f8fe5a41df1c3decf7)
 "Do you not want to know who has taken it?" cried his wife impatiently. "You want to tell me, and I have no objection to hearing it." This was invitation enough. "Why, my dear, you must know, Mrs. Long says that Netherfield is taken by a young man of large fortune from the north of England; that he came down on Monday in a chaise and four to see the place, and was so much delighted with it, that he agreed with Mr. Morris immediately; that he is to take possession before Michaelmas, and some of his servants are to be in the house by

===== A-5-dev.err (sha256 2b1207ed249c79f449a9ac01758b8fd3a83a91d835d36ca6f9a33a93df0cd0d8)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 150 | generating 128 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 1.59 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 22.2 GB
[place] auto: dev 0 holds 4700.6 MB of trunk (lmhead yes, 0 dnproj layers), 17.61 GB left for experts
[qtier] dev 0: budget 17.61 GB for experts (~8642 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 8642 slots of 2.09 MB (17.61 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.1s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 8642 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.4 s
[chain] qwen38: host pool, every expert (the 15934 the card leaves and its own 8642) in 49.80 GiB of page-locked RAM the device reads over the bus (18.6 s to allocate, 2.0 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 8642 slots are the chain's to fill (8642 held): up to 32 experts a forward promoted from the pool by this conversation's routes (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.4s | RSS after load: 54.26 GB
Generated (128 new tokens):
Text : 
TTFT: 1.32 s (time to first token)
[qwen38 spec gate] run: T(1) 19.1 ms, rows 2:1.21 3:1.69, rho 0.27, acceptance by position 0.31 0.67 0.50 0.50 0.50
[qwen38 lookup] run: 1.04 tokens/forward (122 forwards per 127 tokens) | acceptance 29.4% (5/17 drafts in 14 verifies) | gate on, 3 declined, 4 probes
[qwen38 timers] total: 123 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.651 s     13.423 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        0.345 s    345.004 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 122 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.306 s     10.705 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.39 GB
[qtier] resident 8642/24576 experts | uploads 8642 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (4.59 GB trunk + experts, budget 17.61 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 34.38 tok/s (3.7s for 128 tokens)
[chain] qwen38: 123 forwards, 249 frames (231333 ops, 95079 matmuls), 1651.1 ms waiting for the device, 0.0 ms of routed experts on the host, 1004.6 MiB on the device; per decode step 10.71 ms waiting, 0.00 ms of experts (122 steps)
[chain] qwen38: 5904 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 3919 slots from the host pool
[chain] qwen38: 48 prompt layer steps staged the host pool's experts (22.68 GB copied over the bus)
[chain] qwen38: 1356 experts promoted to the card (1356 residents back to the pool), 21 on their way
	Command being timed: "/workspace/bin/qwen38-dev 128 8 /workspace/prova/prompt120.txt"
	User time (seconds): 235.86
	System time (seconds): 47.47
	Percent of CPU this job got: 770%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:36.79
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57027128
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1381041
	Voluntary context switches: 147558
	Involuntary context switches: 582
	Swaps: 0
	File system inputs: 0
	File system outputs: 56
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== A-5-dev.out (sha256 4c76410738a874c9ecd56f1c50da135a25822d0058b892f8fe5a41df1c3decf7)
 "Do you not want to know who has taken it?" cried his wife impatiently. "You want to tell me, and I have no objection to hearing it." This was invitation enough. "Why, my dear, you must know, Mrs. Long says that Netherfield is taken by a young man of large fortune from the north of England; that he came down on Monday in a chaise and four to see the place, and was so much delighted with it, that he agreed with Mr. Morris immediately; that he is to take possession before Michaelmas, and some of his servants are to be in the house by

===== A-6-pr.err (sha256 f69ac310498f700db1cfe04c9d341358d292bfc16b662ecc7883a9d2473cf878)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 150 | generating 128 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 1.59 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 22.2 GB
[place] auto: dev 0 holds 4700.6 MB of trunk (lmhead yes, 0 dnproj layers), 17.61 GB left for experts
[qtier] dev 0: budget 17.61 GB for experts (~8642 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 8642 slots of 2.09 MB (17.61 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.2s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 8642 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.5 s
[chain] qwen38: host pool, every expert (the 15934 the card leaves and its own 8642) in 49.80 GiB of page-locked RAM the device reads over the bus (18.9 s to allocate, 2.0 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 8642 slots are the chain's to fill (8642 held): up to 128 experts a forward promoted from the pool by this conversation's routes, none while the last 64 frames found 90 % or more of their routes on the card (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.4s | RSS after load: 54.27 GB
Generated (128 new tokens):
Text : 
TTFT: 1.32 s (time to first token)
[qwen38 spec gate] run: T(1) 19.5 ms, rows 2:1.25 3:2.03, rho 0.35, acceptance by position 0.31 0.67 0.50 0.50 0.50
[qwen38 lookup] run: 1.04 tokens/forward (122 forwards per 127 tokens) | acceptance 29.4% (5/17 drafts in 14 verifies) | gate on, 3 declined, 4 probes
[qwen38 timers] total: 123 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.742 s     14.163 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        0.345 s    345.302 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 122 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.397 s     11.448 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.39 GB
[qtier] resident 8642/24576 experts | uploads 8642 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (4.59 GB trunk + experts, budget 17.61 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 34.20 tok/s (3.7s for 128 tokens)
[chain] qwen38: 123 forwards, 127 frames (228650 ops, 95079 matmuls), 1742.0 ms waiting for the device, 0.0 ms of routed experts on the host, 1004.6 MiB on the device; per decode step 11.45 ms waiting, 0.00 ms of experts (122 steps)
[chain] qwen38: 5904 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 3123 slots from the host pool
[chain] qwen38: 48 prompt layer steps staged the host pool's experts (22.68 GB copied over the bus)
[chain] qwen38: 0 experts promoted to the card (0 residents back to the pool), 0 on their way; 123 frames braked, 97.7 % of the routes on the card
	Command being timed: "/workspace/bin/qwen38-pr 128 8 /workspace/prova/prompt120.txt"
	User time (seconds): 239.21
	System time (seconds): 47.54
	Percent of CPU this job got: 770%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:37.23
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57032756
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1382245
	Voluntary context switches: 125097
	Involuntary context switches: 787
	Swaps: 0
	File system inputs: 0
	File system outputs: 56
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== A-6-pr.out (sha256 4c76410738a874c9ecd56f1c50da135a25822d0058b892f8fe5a41df1c3decf7)
 "Do you not want to know who has taken it?" cried his wife impatiently. "You want to tell me, and I have no objection to hearing it." This was invitation enough. "Why, my dear, you must know, Mrs. Long says that Netherfield is taken by a young man of large fortune from the north of England; that he came down on Monday in a chaise and four to see the place, and was so much delighted with it, that he agreed with Mr. Morris immediately; that he is to take possession before Michaelmas, and some of his servants are to be in the house by

===== A-7-pr.err (sha256 87b8ae69148ce6a469753f204f185e4a3f0a770df1a5e2eddc8d33fbc2df2c63)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 150 | generating 128 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 1.59 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 22.2 GB
[place] auto: dev 0 holds 4700.6 MB of trunk (lmhead yes, 0 dnproj layers), 17.61 GB left for experts
[qtier] dev 0: budget 17.61 GB for experts (~8642 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 8642 slots of 2.09 MB (17.61 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.0s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 8642 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.7 s
[chain] qwen38: host pool, every expert (the 15934 the card leaves and its own 8642) in 49.80 GiB of page-locked RAM the device reads over the bus (18.7 s to allocate, 2.0 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 8642 slots are the chain's to fill (8642 held): up to 128 experts a forward promoted from the pool by this conversation's routes, none while the last 64 frames found 90 % or more of their routes on the card (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.4s | RSS after load: 54.26 GB
Generated (128 new tokens):
Text : 
TTFT: 1.32 s (time to first token)
[qwen38 spec gate] run: T(1) 19.3 ms, rows 2:1.24 3:1.86, rho 0.34, acceptance by position 0.31 0.67 0.50 0.50 0.50
[qwen38 lookup] run: 1.04 tokens/forward (122 forwards per 127 tokens) | acceptance 29.4% (5/17 drafts in 14 verifies) | gate on, 3 declined, 4 probes
[qwen38 timers] total: 123 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.724 s     14.017 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        0.345 s    344.620 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 122 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.379 s     11.307 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.38 GB
[qtier] resident 8642/24576 experts | uploads 8642 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (4.59 GB trunk + experts, budget 17.61 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 34.44 tok/s (3.7s for 128 tokens)
[chain] qwen38: 123 forwards, 127 frames (228650 ops, 95079 matmuls), 1724.1 ms waiting for the device, 0.0 ms of routed experts on the host, 1004.6 MiB on the device; per decode step 11.31 ms waiting, 0.00 ms of experts (122 steps)
[chain] qwen38: 5904 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 3123 slots from the host pool
[chain] qwen38: 48 prompt layer steps staged the host pool's experts (22.68 GB copied over the bus)
[chain] qwen38: 0 experts promoted to the card (0 residents back to the pool), 0 on their way; 123 frames braked, 97.7 % of the routes on the card
	Command being timed: "/workspace/bin/qwen38-pr 128 8 /workspace/prova/prompt120.txt"
	User time (seconds): 236.87
	System time (seconds): 47.74
	Percent of CPU this job got: 767%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:37.10
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57024996
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1380421
	Voluntary context switches: 138276
	Involuntary context switches: 580
	Swaps: 0
	File system inputs: 0
	File system outputs: 56
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== A-7-pr.out (sha256 4c76410738a874c9ecd56f1c50da135a25822d0058b892f8fe5a41df1c3decf7)
 "Do you not want to know who has taken it?" cried his wife impatiently. "You want to tell me, and I have no objection to hearing it." This was invitation enough. "Why, my dear, you must know, Mrs. Long says that Netherfield is taken by a young man of large fortune from the north of England; that he came down on Monday in a chaise and four to see the place, and was so much delighted with it, that he agreed with Mr. Morris immediately; that he is to take possession before Michaelmas, and some of his servants are to be in the house by

===== A-8-dev.err (sha256 342e037b3195f1aac9bb66f74b99f1796a4021a36471549637aa1cde7fb2f6d0)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 150 | generating 128 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 1.59 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 22.2 GB
[place] auto: dev 0 holds 4700.6 MB of trunk (lmhead yes, 0 dnproj layers), 17.61 GB left for experts
[qtier] dev 0: budget 17.61 GB for experts (~8642 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 8642 slots of 2.09 MB (17.61 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.0s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 8642 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.7 s
[chain] qwen38: host pool, every expert (the 15934 the card leaves and its own 8642) in 49.80 GiB of page-locked RAM the device reads over the bus (18.6 s to allocate, 2.0 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 8642 slots are the chain's to fill (8642 held): up to 32 experts a forward promoted from the pool by this conversation's routes (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.4s | RSS after load: 54.26 GB
Generated (128 new tokens):
Text : 
TTFT: 1.32 s (time to first token)
[qwen38 spec gate] run: T(1) 19.0 ms, rows 2:1.36, rho 0.40, acceptance by position 0.17 0.50 0.50 0.50 0.50
[qwen38 lookup] run: 1.00 tokens/forward (127 forwards per 127 tokens) | acceptance 0.0% (0/4 drafts in 4 verifies) | gate on, 18 declined, 4 probes
[qwen38 timers] total: 128 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.701 s     13.288 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        0.345 s    345.101 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 127 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.356 s     10.676 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.38 GB
[qtier] resident 8642/24576 experts | uploads 8642 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (4.59 GB trunk + experts, budget 17.61 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 34.00 tok/s (3.8s for 128 tokens)
[chain] qwen38: 128 forwards, 258 frames (239958 ops, 98944 matmuls), 1700.9 ms waiting for the device, 0.0 ms of routed experts on the host, 890.8 MiB on the device; per decode step 10.68 ms waiting, 0.00 ms of experts (127 steps)
[chain] qwen38: 6144 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 3685 slots from the host pool
[chain] qwen38: 48 prompt layer steps staged the host pool's experts (22.68 GB copied over the bus)
[chain] qwen38: 1280 experts promoted to the card (1280 residents back to the pool), 8 on their way
	Command being timed: "/workspace/bin/qwen38-dev 128 8 /workspace/prova/prompt120.txt"
	User time (seconds): 236.40
	System time (seconds): 47.70
	Percent of CPU this job got: 764%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:37.14
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57026108
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1380746
	Voluntary context switches: 143008
	Involuntary context switches: 575
	Swaps: 0
	File system inputs: 0
	File system outputs: 56
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== A-8-dev.out (sha256 4c76410738a874c9ecd56f1c50da135a25822d0058b892f8fe5a41df1c3decf7)
 "Do you not want to know who has taken it?" cried his wife impatiently. "You want to tell me, and I have no objection to hearing it." This was invitation enough. "Why, my dear, you must know, Mrs. Long says that Netherfield is taken by a young man of large fortune from the north of England; that he came down on Monday in a chaise and four to see the place, and was so much delighted with it, that he agreed with Mr. Morris immediately; that he is to take possession before Michaelmas, and some of his servants are to be in the house by

===== summary
v0 decode tok/s: 52.18, 48.08, 52.85, 51.95; median 52.065
v0 prompt tok/s: 113.6, 113.6, 113.6, 113.6
v1 decode tok/s: 54.81, 51.92, 52.42, 52.99; median 52.705
v1 prompt tok/s: 111.9, 111.1, 113.6, 113.6
stdout sha256 in all 8 runs: 4c76410738a874c9ecd56f1c50da135a25822d0058b892f8fe5a41df1c3decf7
