Overview — colibri-nui

colibri-nui is a modified version of colibri (opens in new tab) by Vincenzo Fornaro. It adds Apple-GPU (Metal) acceleration for the DeepSeek V4 engine and better memory planning on machines where the CPU and GPU share memory. Everything else is colibri v2.1.0 unchanged.

Site: https://colibri.software.nuilab.org

2.1.0+nui.1 (2026-11-15)

What changed

#	Change	Where it helps	On by default
1	Correct CPU core count on Apple Silicon	Every Mac with M-series “Super” or “Performance” cores	Yes
2	Metal attention for DeepSeek V4: fused projections, weights and KV cache kept on the GPU	Apple Silicon	Yes, in a Metal build run with `COLI_METAL=1`
3	Metal output head kept resident on the GPU	Apple Silicon	Yes, same as 2
4	Routed experts computed on the GPU	Apple Silicon	No: set `COLI_METAL_V4_MOE=1`
5	One-pool memory plan for unified-memory machines	DGX Spark (GB10), Jetson, Apple Silicon	Yes, chosen automatically

Core count. colibri treated the last Apple performance tier as efficiency cores, so an 18-core M5 Max ran 6 compute threads. It now counts every compute tier and keeps the expert-loader threads aside (15 on the M5 Max). OMP_NUM_THREADS still overrides it.
Metal attention. The DeepSeek V4 attention projections, sparse attention core, and KV cache run on the GPU in two command buffers per layer, reproducing the CPU’s FP8 and BF16 rounding exactly, so output matches the CPU build token for token.
Resident head. The output projection stays registered on the GPU instead of being set up again for every token.
GPU experts (experimental). The six routed experts of each token are computed together on the GPU, with exact FP4 arithmetic. Routing, the shared expert and expert loading stay on the CPU.
Unified memory plan. On machines where the GPU shares system memory, colibri used to give the GPU tier first claim and clamp the RAM cache to what was left, even when the engine could not use that GPU share. The new plan budgets one pool, sized to what the engine can hold, and keeps a safety margin of free memory. Choose with --unified-plan auto|pooled|legacy or COLI_UNIFIED_PLAN; startup prints the plan chosen and why.

Build and run (Apple Silicon)

make -C c deepseek-v4 METAL=1
COLI_METAL=1 ./c/deepseek_v4 <model-dir> "your prompt" --max-tokens 120
COLI_METAL=1 COLI_METAL_V4_MOE=1 ./c/deepseek_v4 <model-dir> "your prompt" --max-tokens 120   # experimental GPU experts

Build from source. The prebuilt engines that coli setup can download when no compiler is present are colibri’s own and do not include these changes.

Measured results

DeepSeek V4 Flash, the same 120-token prompt, time for the whole process including start-up. Apple M5 Max, 128 GB:

#	Version	Time	Tokens/s
1	colibri 2.1.0, CPU	72.4 s	1.66
2	Correct core count, CPU	57.4 s	2.09
3	Plus Metal attention and head	50.8 s	2.36
4	Plus GPU experts (`COLI_METAL_V4_MOE=1`)	37.2 s	3.22

Rows 2 to 4 produce exactly the same tokens and logits as the CPU build.

NVIDIA DGX Spark (GB10), CUDA build, median of three fresh-process runs: 109.0 s with the old plan, 106.4 s with the one-pool plan, token-identical. Most of the remaining time on that machine is attention on the CPU, which this release does not yet move to the GPU.

Known limits

GPU experts on Metal are opt-in until they have been tested on more machines.
Attention for DeepSeek V4 on CUDA still runs on the CPU.
On the DGX Spark, the CUDA and CPU builds of colibri 2.1.0 produce different tokens from the 49th token of a long prompt onward. This predates colibri-nui and is under investigation.

Files changed

New files carry a copyright header. Modified files carry a notice at the top.

New: c/backend_metal_v4_fused.inc, c/backend_metal_v4_fused_shaders.inc, c/backend_metal_v4_moe.inc, c/backend_metal_v4_moe_shaders.inc, c/tests/probe_deepseek_v4_metal.py, c/tests/test_deepseek_v4_metal.c, c/tests/test_deepseek_v4_metal.py, c/tests/test_deepseek_v4_metal_fusion.c, c/tests/test_deepseek_v4_metal_math.mm, c/tests/test_deepseek_v4_metal_moe.mm, c/tests/test_probe_deepseek_v4_metal.py, c/tests/test_unified_launcher.py, c/tests/test_unified_plan.py, NUI-CHANGES.md
Modified: Makefile, c/Makefile, c/Makefile.deepseek-v4, c/backend_metal.h, c/backend_metal.mm, c/coli, c/deepseek_v4.c, c/doctor.py, c/omp_tune.h, c/resource_plan.py, c/version.py, c/tests/test_cli_output.py, c/tests/test_deepseek_v4_build_flags.py, c/tests/test_doctor.py, c/tests/test_family_registry.py, c/tests/test_omp_tune.c, c/tests/test_resource_plan.py, c/tests/test_v4_cli.py, c/tools/make_deepseek_v4_tiny.py, NOTICE, the version line of each README*.md and of site/index.html, and the notice below at the top of README.md

Source: NUI-CHANGES.md in colibri-nui, the same file that ships with the code.