llama.cpp build b12284 adds FP8 support to its CUDA backend for NVIDIA GPUs
llama.cpp's b12284 build, published November 11, adds OCP FP8 support to the CUDA backend, including FP8 matrix math through cuBLAS and FP8 handling for the KV cache. It builds on the FP8 tensor type added in v0.9.0 and the FP8 KV cache that arrived in b12274. The change notes include a test of a 27B model on a 32GB RTX PRO 4500 Blackwell card.
Why it matters
FP8 files need about half the memory of 16-bit ones while keeping more precision than 4-bit quantization, so NVIDIA GPU owners get a middle option between size and quality. As always, check that your GPU and CUDA version are covered before switching formats.