Latest developments

Illustration of an open server chassis holding three plain fanless accelerator cards with blower ducts, glowing blue, with a network of connected nodes in the background

Tom's Hardware: three used V100 cards run Qwen 3.6 35B at about 100 tokens per second

Tom's Hardware reports on a homelab builder's blog benchmarks of decommissioned NVIDIA Tesla V100 16GB cards on one dual-Xeon board with llama.cpp. Three V100s ran Qwen 3.6 35B at 8-bit at about 101 tokens per second at 4K context and about 73 at 128K. An M5 Max Mac Studio, using separate benchmark data, was slightly faster at 4K and 32K, but the V100s led at 64K and 128K.

Why it matters

Old data-center cards can still run mid-size MoE models fast if you can deal with heat, noise and power: V100s ship without fans, and the builder 3D-printed a blower duct for each card. More cards add memory for bigger models, not more speed, and the Mac and V100 numbers came from different runtimes.

Source: Tom's Hardware tomshardware.com
  • gpu
  • mac

More news

All news