NVIDIA brings faster speculative decoding to llama.cpp and vLLM, plus memory-saving FP4 checkpoints
NVIDIA says new multi-token-prediction speculative decoding in llama.cpp makes agent tasks with Qwen 3.8 Flash Next up to 1.4x faster on an RTX PRO 6000 Blackwell, and vLLM changes speed up Qwen3.8 27B by up to 1.3x on DGX Spark. It also released FP4 model checkpoints built to leave memory free for context, starting with Qwen3.8-Flash-Next on a single 128GB DGX Spark or RTX PRO 6000.
Why it matters
These are free software gains on hardware you may already own, and they show up in the tools most local setups use. The FP4 checkpoints matter most for big-memory machines, where fitting the model and still having room for long context is the real limit.