colibri-nui

colibri-nui is a modified version of colibri (opens in new tab) , the engine that runs very large mixture-of-experts models from disk on ordinary machines. It adds Apple-GPU (Metal) acceleration for DeepSeek V4 and a better memory plan for machines where the CPU and GPU share memory. Everything else is colibri v2.1.0.

On an Apple M5 Max, DeepSeek V4 Flash generates 120 tokens in 37.2 seconds instead of 72.4, and the output is identical to the CPU build, token for token.

git clone https://github.com/iblues76/colibri-nui.git
cd colibri-nui && make -C c deepseek-v4 METAL=1
COLI_METAL=1 COLI_METAL_V4_MOE=1 ./c/deepseek_v4 <model-dir> "your prompt" --max-tokens 120

What changed, how to build it, and the measurements: the overview . Everything about using colibri itself (models, coli setup, the server) is in the colibri README (opens in new tab) .

colibri is by Vincenzo Fornaro. colibri-nui modifications and additions Copyright 2026 Francisco R. Ortega. Both are licensed under the Apache License, Version 2.0.