Eight DGX Spark boxes run a 753B model at 80 tokens a second

Nvidia has wired eight DGX Spark desktops into one machine and run a 753-billion parameter model on it. The write-up, published on 22 October by four of the company's engineers, puts GLM 5.3 from Z.ai on the cluster at NVFP4 precision and measures roughly 80 tokens per second in generation. Eight nodes at 128GB of unified memory each give 1TB in total; the model alone occupies about 470GB of it, so slightly under half the cluster is the weights sitting still.

The nodes talk over ConnectX-7 network cards using RoCE, in a ring for four of them and through a switch for eight. The stack above that is vLLM, FlashInfer and NCCL, with a Spark-specific collective backend the team calls RoCEnante that trims all-reduce latency during tensor-parallel decode and buys 18 to 21% more throughput at scale.

Training scales, decoding does not

The number worth pulling out of the post is not the 753 billion. It is the gap between two scaling curves on the same hardware.

“Decode throughput scaled from 11.86 tokens per second on a single node to 52.58 tokens per second across eight nodes, a 4.43x speedup with the RoCEnante backend against 3.05x without it.”

— Nvidia Technical Blog, 22 October 2026, https://developer.nvidia.com/blog/running-frontier-class-ai-models-across-8-nvidia-dgx-spark-nodes/

Pre-training a small model called Nanochat across the eight nodes came out at a 7.79-fold speedup, which Nvidia reports as 97.4% scaling efficiency. Inference did not behave that way. Decode throughput on Qwen3.8-27B went from 11.86 tokens per second on one node to 52.58 on eight, and that is 4.43 times the single-node figure from eight times the hardware. Work out the efficiency and it is 55%, against the 97% the training run managed. Without RoCEnante the same test gives 3.05 times, or 38%. Training fills the wire; token-by-token inference spends its life waiting on it.

What eight of them cost

Price has moved as much as the engineering. The 128GB DGX Spark launched at $3,999 in October 2025, went to $4,699 in February on memory supply, and now lists at $6,950. Eight of them is $55,600 and about 1,920W at the wall, which works out at roughly $695 for each token per second of generation on that 753B model. Against a rack of H100s that is cheap; against an API bill it is a long payback, and the thing it actually buys is that the weights never leave your desk.

The other result in the post points the same way as our earlier note that a 30B model beat Nvidia's flagship on Nvidia's own benchmark: routing 75% of requests to a smaller model through NeMo Switchyard tripled aggregate throughput while matching quality on the benchmark tasks. Two frontier-class claims in one post, and the one that will change more desks is the boring one about sending most of the work somewhere smaller.

None of this should be read as personalized investment advice. Every throughput figure comes from Nvidia's own benchmark run on its own hardware, the efficiency percentages are our arithmetic on those figures, and the $6,950 list price was checked on 24 October 2026.