After a few weeks of not being able to work on Ornith 1.5 122B on both Intel Arc Pro B65 running on llama.cpp... we got new numbers 🦆
26k bench prompt - DFlash On
PP 5,572 tok/s
TG 81 tok/s (Draft 954/1784)
~57.5GB VRAM Usage between both cards
26k bench prompt - DFlash Off
PP ~6,170 tok/s (almost double vs v12)
TG ~73.5 tok/s
~55.4 VRAM Usage between both cards
Still working on the custom XE2MX quants (2,3 and 4 bit) which specifically targets the XE2 arch (B60, B65 and B70)
Hope some of this can be carried to Qwen4 (part of this work with 3.8 dense, giving about ~2,800 pp and ~42 tg, 28GB VRAM)
GitHub and Hugging Face after I move it onto current llama.cpp
6 versions later... less VRAM usage, 24% prefill and 22% decode improvement on Ornith 1.5 122B with two Intel Arc Pro B65 in llama.cpp 🦆
Same custom 3bit quant (XE2MX3), 26k prompt and 262k Q8 context with Dflash2
Prefill: 3355 tok/s
Tg: 60 tok/s
Still a WIP
1
324
6 versions later... less VRAM usage, 24% prefill and 22% decode improvement on Ornith 1.5 122B with two Intel Arc Pro B65 in llama.cpp 🦆
Same custom 3bit quant (XE2MX3), 26k prompt and 262k Q8 context with Dflash2
Prefill: 3355 tok/s
Tg: 60 tok/s
Still a WIP
Last one for tonight 🦆
Ornith 1.5 122B on two Intel Arc Pro B65
V5, no drafter: 3128 prefill, 36.8 decode
V6, DFlash2 n=3: 2705 prefill, 49.3 decode
VRAM went down because the two card scheduler was keeping 4 copies of the same scratch, now we only keep 1
Ornith 1.5 122B, same 2x Intel Arc Pro B65s, 262k contex loaded, Q8 KV, 26k benchmark prompt, running on a custom format (XE2MX3)
V2 results: 2781 prefill and 26.8 tg
V5: 3128 prefill, 36.8 decode
Still no drafter/mtp/dflash 🦆
Ornith 1.5 122B already running at 3000 prefill tok/s (same ~26k tokens prompt, 262k context loaded w/Q8 KV) on the pair of Intel B65s in llama.cpp 🦆
Ornith 1.5 122B running on 2x Intel Arc Pro B65 in llama.cpp, one of them over Thunderbolt (~6-8% performance penalty)
262k context, Q8 KV cache, 26k token benchmark prompt:
~2780 prefill tok/s
~26.8 gen tok/s (no drafter and not even tested yet tbh)
Custom 3-bit quant (3.06 bpw, ~49GB) + custom Xe2 kernels, ~61GB VRAM in use
Still a WIP, Intel B70 and B60 will be compatible
Ornith 1.5 122B running on 2x Intel Arc Pro B65 in llama.cpp, one of them over Thunderbolt (~6-8% performance penalty)
262k context, Q8 KV cache, 26k token benchmark prompt:
~2780 prefill tok/s
~26.8 gen tok/s (no drafter and not even tested yet tbh)
Custom 3-bit quant (3.06 bpw, ~49GB) + custom Xe2 kernels, ~61GB VRAM in use
Still a WIP, Intel B70 and B60 will be compatible
NVIDIA Nemotron 3.5 Bolt running under llama.cpp on a single and a pair of Intel Arc Pro B65 with custom SYCL kernels, XE2MX4 and the Liliccorr drafter
Two B65s (XE2MX4, 262k Q8 KV):
~2214 prefill / ~90 tg
One B65 (IQ2_XXS + XE2MX4, 262k Q8 KV, 16k prompt):
~2040 prefill / ~34 tg without the drafter
~1490 prefill / ~52 tg with Lilicorr
Concurrent on two B65s with drafter (8 slots with 16k Q8 KV):
~136 tg aggregate on an 8-request
Card 1 on PCIe 4.0 x4, Card 2 in a Thunderbolt 3 USB-C dock (about 5~8% slower)
B60 and B70 have not been tested but should be compatible
Code and GGUFs:
github.com/cesarsal1nas/llam…
1
1
276
Custom 4bit GGUF for Intel Xe2 under SYCL (Xe2MX4) for this llama.cpp fork
Orca Cyber 27B on a B65, ~2034 prefill / 38 tg on a 26k prompt with cheap MTP, unrotated PPL 6.96 vs 6.81 for the IQ4_XS source (this specific GGUF was requantized from IQ4_XS, not clean BF16)
1–8 concurrent requests with 16k ctx each, best total throughput ~99 tok/s at 4 requests including prefill, scaling isn't linear
Weights stay on GPU, an extra attention pack sits in RAM for prefill, fast path needs patched oneDNN, patches and instructions are in the repo
Only B65 tested here but should also work on Arc Pro B60 and B70 as both are Xe2
EXL3 support for the 4090 and B65 is still there but no optimizations were delivered as Xe2MX4 was the focus of this update
GGUF on HF, benchmarks in the README
github.com/cesarsal1nas/llam…
huggingface.co/cesarsal1nas/…
2
210












