# qwen38 CUDA chain, prompt chunks of 512 rows against 2048: the 1628-token prompt, the card's allowance at 14.7 GiB (a 16 GB card) (2026-10-26)
# Vast.ai instance: AMD EPYC 7763 64-Core Processor (64 cores / 128 threads, 256 MiB L3; the container's quota 30.72 CPUs, OMP_NUM_THREADS=29) + NVIDIA RTX PRO 4000 Blackwell (24467 MiB, sm_120, PCIe 4 x16: 28.3 GB/s (28.2-28.4, the median of 5 measurements) to the card and 558 GB/s (557-560, the median of 5 measurements) on it);
#   503.5 GiB on the host, the container's memory limit 241.7 GiB; Ubuntu 24.04.1 LTS (container nvidia/cuda:12.8.1-devel-ubuntu24.04) on Linux 5.15.0-174-generic x86_64, Vast.ai, driver 595.58.03.
# Model: unsloth/Qwen3.8-Flash-Next-GGUF, UD-Q2_K_XL (Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf sha256 a4f3b21e77353999829f2f767e9ac21ce9c71d29a74f2cc9eda48c9bf23c8b86, Qwen3.8-Flash-Next-UD-Q2_K_XL-00002-of-00003.gguf sha256 2e3bf1ee7d2a04e261e9f342a2d968f696cce5941d082b0e434deb9b1edc12c6, Qwen3.8-Flash-Next-UD-Q2_K_XL-00003-of-00003.gguf sha256 ec8c106759fdf4f463039c34c0707718d7d8908d53d892bd4f002e71620803f9)
#   imported with python3 tools/gguf_import.py <the first file> <model> --no-mtp.
# v0 = origin/dev 77530b49 (qwen38-dev, 512 rows a chunk); v1 = 77530b49 + this change (qwen38-pr, 2048 rows;
#   the measured diff sha256 db97e6c2632defbf862713bbcabf092b52994b3ebd7d4422e5f286fa0a01ec00). Both built with:
#   cd c && make -j qwen38 CUDA=1 CUDA_ARCH=native
# History: history.usage.gz beside this file (gunzip to <history.usage>, sha256 8dc23b015a8080b48b2e7ef7878f51516ac64f481bd104bba56f96a021341459),
#   written by a run of the prose prompt on another machine (134880 selections); USAGE_SAVE=0, so no run changes it.
# Each run, a new process: env COLI_CUDA=1 COLI_VULKAN=0 COLI_TEMP=0 COLI_TIMERS=1 USAGE_SAVE=0 OMP_NUM_THREADS=29 OMP_WAIT_POLICY=ACTIVE OMP_PROC_BIND=close SNAP=<model> COLI_USAGE=<history.usage> N_NEW=64 CUDA_EXPERT_GB=14.7 ./qwen38 128 8 <prompt>
# Prompt (sha256 f3306e5f57ab2527735662aafb31c14e76d3b1a73cc86508b6ff020e43f4e4aa, 6599 bytes): the file prompt-1628-tokens.txt beside this file; its first 300 bytes:
#   It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife. However little known the feelings or views of such a man may be on his first entering a neighbourhood, this truth is so well fixed in the minds of the surrounding families, that he i
# A warm-up run of v0 (not counted), then v0 v1 v1 v0 v0 v1 v1 v0, 60 s of rest before each. Every run's stderr
#   and stdout in full, in the order they ran. decode tok/s = (n - 1) / (n / S - TTFT), S and n from the
#   "Speed" line; prompt tok/s = the prompt's tokens / TTFT.

===== SL-w.err (sha256 1cb872b3e23641f43d8221da504534a54f275ae414970320ea2577503d7992f0)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 1628 | generating 64 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 1.59 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 14.7 GB
[place] auto: dev 0 holds 4700.6 MB of trunk (lmhead yes, 0 dnproj layers), 10.11 GB left for experts
[qtier] dev 0: budget 10.11 GB for experts (~4961 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 4961 slots of 2.09 MB (10.11 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.2s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 4961 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.4 s
[chain] qwen38: host pool, every expert (the 19615 the card leaves and its own 4961) in 49.80 GiB of page-locked RAM the device reads over the bus (22.7 s to allocate, 1.2 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 4961 slots are the chain's to fill (4961 held): up to 32 experts a forward promoted from the pool by this conversation's routes (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.7s | RSS after load: 4.54 GB
Generated (64 new tokens):
Text : 
TTFT: 10.92 s (time to first token)
[qwen38 spec gate] run: T(1) 27.4 ms, rows 2:1.54, rho 0.52, acceptance by position 0.40 0.50 0.50 0.50 0.50
[qwen38 lookup] run: 1.02 tokens/forward (62 forwards per 63 tokens) | acceptance 33.3% (1/3 drafts in 3 verifies) | gate on, 10 declined, 1 probes
[qwen38 timers] total: 63 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        2.883 s     45.755 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.816 s   1815.730 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 62 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.067 s     17.208 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.56 GB
[qtier] resident 4961/24576 experts | uploads 4961 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (4.59 GB trunk + experts, budget 10.11 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 5.03 tok/s (12.7s for 64 tokens)
[chain] qwen38: 63 forwards, 137 frames (144152 ops, 51009 matmuls), 2882.6 ms waiting for the device, 0.0 ms of routed experts on the host, 1196.8 MiB on the device; per decode step 17.21 ms waiting, 0.00 ms of experts (62 steps)
[chain] qwen38: 3168 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 147665 slots from the host pool
[chain] qwen38: 192 prompt layer steps staged the host pool's experts (93.03 GB copied over the bus)
[chain] qwen38: 1400 experts promoted to the card (1400 residents back to the pool), 16 on their way
	Command being timed: "/workspace/bin/qwen38-dev 128 8 /workspace/prova/prompt-lungo.txt"
	User time (seconds): 1093.40
	System time (seconds): 79.06
	Percent of CPU this job got: 2061%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:56.86
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57208632
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1424058
	Voluntary context switches: 162331
	Involuntary context switches: 1169
	Swaps: 0
	File system inputs: 0
	File system outputs: 48
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== SL-w.out (sha256 c9c4093a3664c8a1e6228211f85af734b50cb81ece35ba073d339b0076deaf4c)
 But if we do not venture somebody else will; and after all, Mrs. Long and her nieces must stand their chance; and, therefore, as she will think it an act of kindness, if you decline the office, I will take it on myself." The girls stared at their father. Mrs. Bennet

===== SL-1-dev.err (sha256 2bebf3e4aaf6ab56635762501c21d49316a29ef477624cb8a616fd28687d5dd2)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 1628 | generating 64 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 1.59 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 14.7 GB
[place] auto: dev 0 holds 4700.6 MB of trunk (lmhead yes, 0 dnproj layers), 10.11 GB left for experts
[qtier] dev 0: budget 10.11 GB for experts (~4961 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 4961 slots of 2.09 MB (10.11 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.2s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 4961 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.5 s
[chain] qwen38: host pool, every expert (the 19615 the card leaves and its own 4961) in 49.80 GiB of page-locked RAM the device reads over the bus (21.7 s to allocate, 1.1 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 4961 slots are the chain's to fill (4961 held): up to 32 experts a forward promoted from the pool by this conversation's routes (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.7s | RSS after load: 4.54 GB
Generated (64 new tokens):
Text : 
TTFT: 10.98 s (time to first token)
[qwen38 spec gate] run: T(1) 27.5 ms, rows 2:1.52, rho 0.49, acceptance by position 0.40 0.50 0.50 0.50 0.50
[qwen38 lookup] run: 1.02 tokens/forward (62 forwards per 63 tokens) | acceptance 33.3% (1/3 drafts in 3 verifies) | gate on, 10 declined, 2 probes
[qwen38 timers] total: 63 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        2.882 s     45.750 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.817 s   1817.189 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 62 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.065 s     17.179 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.56 GB
[qtier] resident 4961/24576 experts | uploads 4961 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (4.59 GB trunk + experts, budget 10.11 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 5.01 tok/s (12.8s for 64 tokens)
[chain] qwen38: 63 forwards, 137 frames (144169 ops, 51009 matmuls), 2882.3 ms waiting for the device, 0.0 ms of routed experts on the host, 1196.8 MiB on the device; per decode step 17.18 ms waiting, 0.00 ms of experts (62 steps)
[chain] qwen38: 3168 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 147566 slots from the host pool
[chain] qwen38: 192 prompt layer steps staged the host pool's experts (93.02 GB copied over the bus)
[chain] qwen38: 1411 experts promoted to the card (1411 residents back to the pool), 18 on their way
	Command being timed: "/workspace/bin/qwen38-dev 128 8 /workspace/prova/prompt-lungo.txt"
	User time (seconds): 1067.31
	System time (seconds): 77.09
	Percent of CPU this job got: 2062%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.49
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57211216
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1424734
	Voluntary context switches: 164960
	Involuntary context switches: 768
	Swaps: 0
	File system inputs: 0
	File system outputs: 40
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== SL-1-dev.out (sha256 c9c4093a3664c8a1e6228211f85af734b50cb81ece35ba073d339b0076deaf4c)
 But if we do not venture somebody else will; and after all, Mrs. Long and her nieces must stand their chance; and, therefore, as she will think it an act of kindness, if you decline the office, I will take it on myself." The girls stared at their father. Mrs. Bennet

===== SL-2-pr.err (sha256 4649f9860f2dca264dd8120411b1ce1d0979fe8494290a441c77a1cfba28099e)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 1628 | generating 64 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 2.51 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 14.7 GB
[place] auto: dev 0 holds 5647.2 MB of trunk (lmhead yes, 0 dnproj layers), 9.19 GB left for experts
[qtier] dev 0: budget 9.19 GB for experts (~4508 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 4508 slots of 2.09 MB (9.19 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.2s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 4508 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.0 s
[chain] qwen38: host pool, every expert (the 20068 the card leaves and its own 4508) in 49.80 GiB of page-locked RAM the device reads over the bus (22.7 s to allocate, 1.2 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 4508 slots are the chain's to fill (4508 held): up to 32 experts a forward promoted from the pool by this conversation's routes (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.7s | RSS after load: 4.55 GB
Generated (64 new tokens):
Text : 
TTFT: 5.17 s (time to first token)
[qwen38 spec gate] run: T(1) 28.0 ms, rows 2:1.53, rho 0.50, acceptance by position 0.40 0.50 0.50 0.50 0.50
[qwen38 lookup] run: 1.02 tokens/forward (62 forwards per 63 tokens) | acceptance 33.3% (1/3 drafts in 3 verifies) | gate on, 10 declined, 2 probes
[qwen38 timers] total: 63 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.223 s     19.416 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        0.138 s    138.390 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 62 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.085 s     17.497 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.69 GB
[qtier] resident 4508/24576 experts | uploads 4508 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (5.51 GB trunk + experts, budget 9.19 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 9.15 tok/s (7.0s for 64 tokens)
[chain] qwen38: 63 forwards, 128 frames (138006 ops, 48699 matmuls), 1223.2 ms waiting for the device, 0.0 ms of routed experts on the host, 1902.9 MiB on the device; per decode step 17.50 ms waiting, 0.00 ms of experts (62 steps)
[chain] qwen38: 3024 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 162431 slots from the host pool
[chain] qwen38: 48 prompt layer steps staged the host pool's experts (23.31 GB copied over the bus)
[chain] qwen38: 1216 experts promoted to the card (1216 residents back to the pool), 12 on their way
	Command being timed: "/workspace/bin/qwen38-pr 128 8 /workspace/prova/prompt-lungo.txt"
	User time (seconds): 930.52
	System time (seconds): 75.80
	Percent of CPU this job got: 1982%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:50.76
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57349244
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1451015
	Voluntary context switches: 123765
	Involuntary context switches: 798
	Swaps: 0
	File system inputs: 0
	File system outputs: 48
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== SL-2-pr.out (sha256 c9c4093a3664c8a1e6228211f85af734b50cb81ece35ba073d339b0076deaf4c)
 But if we do not venture somebody else will; and after all, Mrs. Long and her nieces must stand their chance; and, therefore, as she will think it an act of kindness, if you decline the office, I will take it on myself." The girls stared at their father. Mrs. Bennet

===== SL-3-pr.err (sha256 a24220f6858885170f6c81a919c9df394646b8ee9437d4af0a07fdc9a3219f02)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 1628 | generating 64 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 2.51 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 14.7 GB
[place] auto: dev 0 holds 5647.2 MB of trunk (lmhead yes, 0 dnproj layers), 9.19 GB left for experts
[qtier] dev 0: budget 9.19 GB for experts (~4508 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 4508 slots of 2.09 MB (9.19 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.2s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 4508 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.0 s
[chain] qwen38: host pool, every expert (the 20068 the card leaves and its own 4508) in 49.80 GiB of page-locked RAM the device reads over the bus (21.7 s to allocate, 1.1 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 4508 slots are the chain's to fill (4508 held): up to 32 experts a forward promoted from the pool by this conversation's routes (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.7s | RSS after load: 4.54 GB
Generated (64 new tokens):
Text : 
TTFT: 5.20 s (time to first token)
[qwen38 spec gate] run: T(1) 27.9 ms, rows 2:1.55, rho 0.51, acceptance by position 0.40 0.50 0.50 0.50 0.50
[qwen38 lookup] run: 1.02 tokens/forward (62 forwards per 63 tokens) | acceptance 33.3% (1/3 drafts in 3 verifies) | gate on, 10 declined, 2 probes
[qwen38 timers] total: 63 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.222 s     19.403 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        0.140 s    139.688 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 62 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.083 s     17.463 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.68 GB
[qtier] resident 4508/24576 experts | uploads 4508 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (5.51 GB trunk + experts, budget 9.19 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 9.11 tok/s (7.0s for 64 tokens)
[chain] qwen38: 63 forwards, 128 frames (137982 ops, 48699 matmuls), 1222.4 ms waiting for the device, 0.0 ms of routed experts on the host, 1902.9 MiB on the device; per decode step 17.46 ms waiting, 0.00 ms of experts (62 steps)
[chain] qwen38: 3024 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 162419 slots from the host pool
[chain] qwen38: 48 prompt layer steps staged the host pool's experts (23.31 GB copied over the bus)
[chain] qwen38: 1203 experts promoted to the card (1203 residents back to the pool), 11 on their way
	Command being timed: "/workspace/bin/qwen38-pr 128 8 /workspace/prova/prompt-lungo.txt"
	User time (seconds): 902.66
	System time (seconds): 74.72
	Percent of CPU this job got: 1973%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:49.53
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57332488
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1446968
	Voluntary context switches: 143567
	Involuntary context switches: 855
	Swaps: 0
	File system inputs: 0
	File system outputs: 48
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== SL-3-pr.out (sha256 c9c4093a3664c8a1e6228211f85af734b50cb81ece35ba073d339b0076deaf4c)
 But if we do not venture somebody else will; and after all, Mrs. Long and her nieces must stand their chance; and, therefore, as she will think it an act of kindness, if you decline the office, I will take it on myself." The girls stared at their father. Mrs. Bennet

===== SL-4-dev.err (sha256 f446180cff3e27b9bea3bb0031681205191c0949e28ccba9dee37e10e7400636)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 1628 | generating 64 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 1.59 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 14.7 GB
[place] auto: dev 0 holds 4700.6 MB of trunk (lmhead yes, 0 dnproj layers), 10.11 GB left for experts
[qtier] dev 0: budget 10.11 GB for experts (~4961 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 4961 slots of 2.09 MB (10.11 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.2s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 4961 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.0 s
[chain] qwen38: host pool, every expert (the 19615 the card leaves and its own 4961) in 49.80 GiB of page-locked RAM the device reads over the bus (22.8 s to allocate, 1.1 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 4961 slots are the chain's to fill (4961 held): up to 32 experts a forward promoted from the pool by this conversation's routes (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.8s | RSS after load: 4.58 GB
Generated (64 new tokens):
Text : 
TTFT: 10.93 s (time to first token)
[qwen38 spec gate] run: T(1) 27.6 ms, rows 2:1.51, rho 0.49, acceptance by position 0.40 0.50 0.50 0.50 0.50
[qwen38 lookup] run: 1.02 tokens/forward (62 forwards per 63 tokens) | acceptance 33.3% (1/3 drafts in 3 verifies) | gate on, 10 declined, 2 probes
[qwen38 timers] total: 63 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        2.880 s     45.720 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.813 s   1812.793 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 62 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.068 s     17.219 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.60 GB
[qtier] resident 4961/24576 experts | uploads 4961 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (4.59 GB trunk + experts, budget 10.11 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 5.03 tok/s (12.7s for 64 tokens)
[chain] qwen38: 63 forwards, 137 frames (144165 ops, 51009 matmuls), 2880.4 ms waiting for the device, 0.0 ms of routed experts on the host, 1196.8 MiB on the device; per decode step 17.22 ms waiting, 0.00 ms of experts (62 steps)
[chain] qwen38: 3168 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 147496 slots from the host pool
[chain] qwen38: 192 prompt layer steps staged the host pool's experts (93.02 GB copied over the bus)
[chain] qwen38: 1406 experts promoted to the card (1406 residents back to the pool), 15 on their way
	Command being timed: "/workspace/bin/qwen38-dev 128 8 /workspace/prova/prompt-lungo.txt"
	User time (seconds): 1093.41
	System time (seconds): 78.07
	Percent of CPU this job got: 2082%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:56.25
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57250964
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1434332
	Voluntary context switches: 131534
	Involuntary context switches: 994
	Swaps: 0
	File system inputs: 0
	File system outputs: 48
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== SL-4-dev.out (sha256 c9c4093a3664c8a1e6228211f85af734b50cb81ece35ba073d339b0076deaf4c)
 But if we do not venture somebody else will; and after all, Mrs. Long and her nieces must stand their chance; and, therefore, as she will think it an act of kindness, if you decline the office, I will take it on myself." The girls stared at their father. Mrs. Bennet

===== SL-5-dev.err (sha256 55eaf43b43b7c42ba0952db72c4c31f2fa3fe50b1afd179aaa5293dd130c6620)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 1628 | generating 64 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 1.59 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 14.7 GB
[place] auto: dev 0 holds 4700.6 MB of trunk (lmhead yes, 0 dnproj layers), 10.11 GB left for experts
[qtier] dev 0: budget 10.11 GB for experts (~4961 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 4961 slots of 2.09 MB (10.11 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.2s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 4961 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.0 s
[chain] qwen38: host pool, every expert (the 19615 the card leaves and its own 4961) in 49.80 GiB of page-locked RAM the device reads over the bus (21.8 s to allocate, 1.2 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 4961 slots are the chain's to fill (4961 held): up to 32 experts a forward promoted from the pool by this conversation's routes (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.7s | RSS after load: 4.59 GB
Generated (64 new tokens):
Text : 
TTFT: 11.00 s (time to first token)
[qwen38 spec gate] run: T(1) 27.5 ms, rows 2:1.51, rho 0.49, acceptance by position 0.40 0.50 0.50 0.50 0.50
[qwen38 lookup] run: 1.02 tokens/forward (62 forwards per 63 tokens) | acceptance 33.3% (1/3 drafts in 3 verifies) | gate on, 10 declined, 2 probes
[qwen38 timers] total: 63 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        2.886 s     45.804 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.823 s   1823.285 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 62 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.062 s     17.135 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.61 GB
[qtier] resident 4961/24576 experts | uploads 4961 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (4.59 GB trunk + experts, budget 10.11 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 5.01 tok/s (12.8s for 64 tokens)
[chain] qwen38: 63 forwards, 137 frames (144151 ops, 51009 matmuls), 2885.6 ms waiting for the device, 0.0 ms of routed experts on the host, 1196.8 MiB on the device; per decode step 17.13 ms waiting, 0.00 ms of experts (62 steps)
[chain] qwen38: 3168 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 147596 slots from the host pool
[chain] qwen38: 192 prompt layer steps staged the host pool's experts (93.03 GB copied over the bus)
[chain] qwen38: 1399 experts promoted to the card (1399 residents back to the pool), 14 on their way
	Command being timed: "/workspace/bin/qwen38-dev 128 8 /workspace/prova/prompt-lungo.txt"
	User time (seconds): 1068.33
	System time (seconds): 76.71
	Percent of CPU this job got: 2068%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.34
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57261980
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1437061
	Voluntary context switches: 138554
	Involuntary context switches: 941
	Swaps: 0
	File system inputs: 0
	File system outputs: 56
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== SL-5-dev.out (sha256 c9c4093a3664c8a1e6228211f85af734b50cb81ece35ba073d339b0076deaf4c)
 But if we do not venture somebody else will; and after all, Mrs. Long and her nieces must stand their chance; and, therefore, as she will think it an act of kindness, if you decline the office, I will take it on myself." The girls stared at their father. Mrs. Bennet

===== SL-6-pr.err (sha256 2de24af903d9723962b940499f4e8bb74f94b06ed67e311ba5dd764685c01b2d)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 1628 | generating 64 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 2.51 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 14.7 GB
[place] auto: dev 0 holds 5647.2 MB of trunk (lmhead yes, 0 dnproj layers), 9.19 GB left for experts
[qtier] dev 0: budget 9.19 GB for experts (~4508 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 4508 slots of 2.09 MB (9.19 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.2s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 4508 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 1.9 s
[chain] qwen38: host pool, every expert (the 20068 the card leaves and its own 4508) in 49.80 GiB of page-locked RAM the device reads over the bus (22.8 s to allocate, 1.2 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 4508 slots are the chain's to fill (4508 held): up to 32 experts a forward promoted from the pool by this conversation's routes (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.8s | RSS after load: 4.55 GB
Generated (64 new tokens):
Text : 
TTFT: 5.17 s (time to first token)
[qwen38 spec gate] run: T(1) 28.0 ms, rows 2:1.52, rho 0.50, acceptance by position 0.40 0.50 0.50 0.50 0.50
[qwen38 lookup] run: 1.02 tokens/forward (62 forwards per 63 tokens) | acceptance 33.3% (1/3 drafts in 3 verifies) | gate on, 10 declined, 2 probes
[qwen38 timers] total: 63 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.221 s     19.381 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        0.139 s    138.628 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 62 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.082 s     17.457 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.69 GB
[qtier] resident 4508/24576 experts | uploads 4508 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (5.51 GB trunk + experts, budget 9.19 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 9.15 tok/s (7.0s for 64 tokens)
[chain] qwen38: 63 forwards, 128 frames (138003 ops, 48699 matmuls), 1221.0 ms waiting for the device, 0.0 ms of routed experts on the host, 1902.9 MiB on the device; per decode step 17.46 ms waiting, 0.00 ms of experts (62 steps)
[chain] qwen38: 3024 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 162433 slots from the host pool
[chain] qwen38: 48 prompt layer steps staged the host pool's experts (23.31 GB copied over the bus)
[chain] qwen38: 1216 experts promoted to the card (1216 residents back to the pool), 15 on their way
	Command being timed: "/workspace/bin/qwen38-pr 128 8 /workspace/prova/prompt-lungo.txt"
	User time (seconds): 929.18
	System time (seconds): 76.44
	Percent of CPU this job got: 1981%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:50.74
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57343900
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1449647
	Voluntary context switches: 119891
	Involuntary context switches: 961
	Swaps: 0
	File system inputs: 0
	File system outputs: 56
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== SL-6-pr.out (sha256 c9c4093a3664c8a1e6228211f85af734b50cb81ece35ba073d339b0076deaf4c)
 But if we do not venture somebody else will; and after all, Mrs. Long and her nieces must stand their chance; and, therefore, as she will think it an act of kindness, if you decline the office, I will take it on myself." The girls stared at their father. Mrs. Bennet

===== SL-7-pr.err (sha256 421f5003b78abce14636528f2d4bd2c3afdce85f54b6ffb74687a874fd3b7397)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 1628 | generating 64 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 2.51 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 14.7 GB
[place] auto: dev 0 holds 5647.2 MB of trunk (lmhead yes, 0 dnproj layers), 9.19 GB left for experts
[qtier] dev 0: budget 9.19 GB for experts (~4508 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 4508 slots of 2.09 MB (9.19 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.2s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 4508 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.4 s
[chain] qwen38: host pool, every expert (the 20068 the card leaves and its own 4508) in 49.80 GiB of page-locked RAM the device reads over the bus (21.7 s to allocate, 1.1 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 4508 slots are the chain's to fill (4508 held): up to 32 experts a forward promoted from the pool by this conversation's routes (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.7s | RSS after load: 4.54 GB
Generated (64 new tokens):
Text : 
TTFT: 5.20 s (time to first token)
[qwen38 spec gate] run: T(1) 28.1 ms, rows 2:1.52, rho 0.50, acceptance by position 0.40 0.50 0.50 0.50 0.50
[qwen38 lookup] run: 1.02 tokens/forward (62 forwards per 63 tokens) | acceptance 33.3% (1/3 drafts in 3 verifies) | gate on, 10 declined, 2 probes
[qwen38 timers] total: 63 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.221 s     19.373 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        0.139 s    138.967 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 62 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.082 s     17.444 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.68 GB
[qtier] resident 4508/24576 experts | uploads 4508 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (5.51 GB trunk + experts, budget 9.19 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 9.11 tok/s (7.0s for 64 tokens)
[chain] qwen38: 63 forwards, 128 frames (138039 ops, 48699 matmuls), 1220.5 ms waiting for the device, 0.0 ms of routed experts on the host, 1902.9 MiB on the device; per decode step 17.44 ms waiting, 0.00 ms of experts (62 steps)
[chain] qwen38: 3024 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 162438 slots from the host pool
[chain] qwen38: 48 prompt layer steps staged the host pool's experts (23.31 GB copied over the bus)
[chain] qwen38: 1233 experts promoted to the card (1233 residents back to the pool), 13 on their way
	Command being timed: "/workspace/bin/qwen38-pr 128 8 /workspace/prova/prompt-lungo.txt"
	User time (seconds): 902.64
	System time (seconds): 76.04
	Percent of CPU this job got: 1957%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:49.99
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57336116
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1447883
	Voluntary context switches: 171363
	Involuntary context switches: 853
	Swaps: 0
	File system inputs: 0
	File system outputs: 32
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== SL-7-pr.out (sha256 c9c4093a3664c8a1e6228211f85af734b50cb81ece35ba073d339b0076deaf4c)
 But if we do not venture somebody else will; and after all, Mrs. Long and her nieces must stand their chance; and, therefore, as she will think it an act of kindness, if you decline the office, I will take it on myself." The girls stared at their father. Mrs. Bennet

===== SL-8-dev.err (sha256 21c137bb0e60884b7e28b60793e7c3b0f599cf71ad9d7dc993741f82698c3412)
[gguf] /workspace/models/q38-gguf/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf: 3 file(s), 1224 tensors, read in place (native ggml types, no conversion)
== Qwen3.8-Flash-Next native text engine | cache=128/layer | CPU ==
[tok] loaded 248320 base pieces + 33 added tokens (max id 248319) from /workspace/models/q38-colibri/tokenizer.json
[enc] prompt tokens: 1628 | generating 64 new tokens
[qwen38] native text weights: prefix=model, 48 layers, PLE=1, cache=128/layer, FP8=native, BF16=native (resident matrices 3.73 GiB)
[qwen38] speculative decoding with the MTP head: off (the GGUF has none; tools/gguf_import.py --mtp names an mtp-*.gguf; Q38_MTP=0 silences this)
[qtier] qwen38: 1.59 GiB kept for the CUDA chain (its mirrors at 8192 positions, its scratch and own matrices)
[qtier] RAM holds 128 of 512 experts a layer: the tier copies each expert it takes as it passes
[qtier] COLI_GPUS unset: selecting 1 visible device(s)
[CUDA] device 0: NVIDIA RTX PRO 4000 Blackwell, 25.2 GB VRAM, sm_120
[qtier] dev 0: 23.2 GB free, allowance 14.7 GB
[place] auto: dev 0 holds 4700.6 MB of trunk (lmhead yes, 0 dnproj layers), 10.11 GB left for experts
[qtier] dev 0: budget 10.11 GB for experts (~4961 experts) after trunk
[qtier] dev 0: the GGUF experts' arena, 4961 slots of 2.09 MB (10.11 GB)
[qtier] CUDA VRAM expert tier active: 1 device(s), 2.09 MB/expert, copied as they pass
[CUDA] mode: routed experts (qwen36 VRAM tier)
[qtier] qwen38: the GGUF's experts on the CUDA tier as the file has them (RAM LRU 128/layer; experts go to VRAM as they get hot)
[qtier] qwen38 trunk: 553 of 553 offered matrices resident (int8, a GGUF's blocks as they are; 3.00 GiB) in 1.2s; the rest answers from the CPU
[qwen38] trunk: 60 matrices int8 on the CPU (0.08 GiB, 0.27 GiB of BF16 released) in 0.0s; Q38_TRUNK_CPU_INT8=0 keeps BF16
[chain] qwen38: a decode step's routed experts in the frame (the tier's residents by address, format 100 (a GGUF's blocks, by layer), group 0; COLI_CUDA_CHAIN_MOE=0: the host's step)
[chain] qwen38: 48 layers on CUDA device 0 (12 QSA), the final mixer and lm_head too; 3.37 GiB of the tier's trunk, 174.2 MiB uploaded by the chain, 9.8 MiB of parameters
[qwen38] routed experts: GGUF IQ2_XS/IQ2_XS/IQ4_NL (layer 0), 1.78 MiB per expert; cache 128/layer
[USAGE] expert history: 134880 selections (/workspace/prova/storia)
[qtier] qwen38: warm start, 4961 of the GGUF's experts in VRAM, the history's most used first (.coli_usage) -- 2.1 s
[chain] qwen38: host pool, every expert (the 19615 the card leaves and its own 4961) in 49.80 GiB of page-locked RAM the device reads over the bus (22.8 s to allocate, 1.2 s to read; COLI_CUDA_CHAIN_POOL=0 off)
[chain] qwen38: a step's pooled experts copied to the device first (16 places, 33.2 MiB)
[chain] qwen38: a prompt chunk's layer's pooled experts copied to the device while the layers before it run (246 places, 510.5 MiB; the most a layer has: 512)
[chain] qwen38: the card's 4961 slots are the chain's to fill (4961 held): up to 32 experts a forward promoted from the pool by this conversation's routes (COLI_CUDA_CHAIN_PROMOTE=0 keeps the warm start's set)
resident weights loaded in 1.7s | RSS after load: 4.60 GB
Generated (64 new tokens):
Text : 
TTFT: 10.93 s (time to first token)
[qwen38 spec gate] run: T(1) 27.5 ms, rows 2:1.53, rho 0.51, acceptance by position 0.40 0.50 0.50 0.50 0.50
[qwen38 lookup] run: 1.02 tokens/forward (62 forwards per 63 tokens) | acceptance 33.3% (1/3 drafts in 3 verifies) | gate on, 10 declined, 1 probes
[qwen38 timers] total: 63 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        2.885 s     45.797 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] prefill: 1 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.818 s   1817.590 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 timers] decode: 62 forwards
[qwen38 timers]   expert-read        0.000 s      0.000 ms/forward
[qwen38 timers]   fp8-expand         0.000 s      0.000 ms/forward
[qwen38 timers]   routed-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   shared-expert      0.000 s      0.000 ms/forward
[qwen38 timers]   resident-mm        1.068 s     17.220 ms/forward
[qwen38 timers]   deltanet           0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-index          0.000 s      0.000 ms/forward
[qwen38 timers]   qsa-attn           0.000 s      0.000 ms/forward
[qwen38 timers]   ple                0.000 s      0.000 ms/forward
[qwen38 timers]   lm-head            0.000 s      0.000 ms/forward
[qwen38 timers]   dn-recur           0.000 s      0.000 ms/forward
[qwen38 timers]   dn-snap            0.000 s      0.000 ms/forward
[qwen38 timers]   lm-draft           0.000 s      0.000 ms/forward
[qwen38 timers] architecture phases overlap resident-mm; expert-read is disk service and fp8-expand is synchronous miss work
[qwen38 expert I/O] weight-ranges=0 scale-ranges=0 coalesced-gate-up=0 prefetched=0 parallel-batches=0 resident-scales=0.00 MiB

PEAK RSS: 54.63 GB
[qtier] resident 4961/24576 experts | uploads 4961 | miss(CPU) 0 | q_skips 0
[qtier]   dev 0: hits 0 | 773 tensors, 3.09 GB VRAM used (4.59 GB trunk + experts, budget 10.11 GB)
[qtier] VRAM hit rate: 0.0 % | swaps 0 (0 in place)
Expert cache hit rate: 0.0% (hit=0 miss=0)
Speed: 5.03 tok/s (12.7s for 64 tokens)
[chain] qwen38: 63 forwards, 137 frames (144121 ops, 51009 matmuls), 2885.2 ms waiting for the device, 0.0 ms of routed experts on the host, 1196.8 MiB on the device; per decode step 17.22 ms waiting, 0.00 ms of experts (62 steps)
[chain] qwen38: 3168 layer steps with the routed experts in the frame, 0 of them hybrid (0 slots on the host in 0.0 ms, 0.0 ms waiting for the device's routes), 147526 slots from the host pool
[chain] qwen38: 192 prompt layer steps staged the host pool's experts (93.02 GB copied over the bus)
[chain] qwen38: 1388 experts promoted to the card (1388 residents back to the pool), 19 on their way
	Command being timed: "/workspace/bin/qwen38-dev 128 8 /workspace/prova/prompt-lungo.txt"
	User time (seconds): 1095.02
	System time (seconds): 77.96
	Percent of CPU this job got: 2074%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:56.55
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 57279356
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 1441400
	Voluntary context switches: 131888
	Involuntary context switches: 1653
	Swaps: 0
	File system inputs: 0
	File system outputs: 40
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0
===== SL-8-dev.out (sha256 c9c4093a3664c8a1e6228211f85af734b50cb81ece35ba073d339b0076deaf4c)
 But if we do not venture somebody else will; and after all, Mrs. Long and her nieces must stand their chance; and, therefore, as she will think it an act of kindness, if you decline the office, I will take it on myself." The girls stared at their father. Mrs. Bennet

===== summary
v0: 512 rows a chunk, 4 chunks of a 1628-token prompt, 192 prompt layer steps staged, 93.02 GiB over the bus (the median of 4 runs; the engine prints GiB as GB); ~4961 experts on the card
v0 prompt tok/s: 148.3, 148.9, 148, 148.9; median 148.6
v0 decode tok/s: 35.11, 35.12, 35.5, 35.12; median 35.12
v0 TTFT s: 10.98, 10.93, 11, 10.93; median 10.955
v0 peak RSS GB: 54.56, 54.6, 54.61, 54.63; median 54.605
v0 T(1) ms: 27.5, 27.6, 27.5, 27.5
v1: 2048 rows a chunk, 1 chunk of a 1628-token prompt, 48 prompt layer steps staged, 23.31 GiB over the bus (the median of 4 runs; the engine prints GiB as GB); ~4508 experts on the card
v1 prompt tok/s: 314.9, 313.1, 314.9, 313.1; median 314
v1 decode tok/s: 34.53, 34.52, 34.53, 34.52; median 34.525
v1 TTFT s: 5.17, 5.2, 5.17, 5.2; median 5.185
v1 peak RSS GB: 54.69, 54.68, 54.69, 54.68; median 54.685
v1 T(1) ms: 28, 27.9, 28, 28.1
stdout sha256 in all 9 runs: c9c4093a3664c8a1e6228211f85af734b50cb81ece35ba073d339b0076deaf4c
