Open source · NVIDIA DGX Spark · GB10

Frontier models on a stack of Sparks.

Everything you need to serve large open models on 2 to 6 DGX Sparks: the OS, the cluster tools, the engine and the weights.

  • Open source
  • OpenAI-compatible API
01 · osSpark OS 02 · clustermentat 03 · servinggr8 · vLLM 04 · modelsexlr8

Fastest decode

–tok/s

Prose decode, single stream

 

 

Longest context
–
Most throughput
–
Recipes
–

The Kindling stack

Four layers, all open source.

exlr8 Experimental

2 · 3 · 4bits per expert

Our weight format for GB10. The bit mix is picked at load time, and prefill runs on FP8 tensor cores.

kindlingai/exlr8 ↗

gr8 Experimental

43.1tok/s prose, full GLM-5.3, 6 Sparks

A from-scratch CUDA engine for full GLM-5.3 on three to six Sparks. No vLLM, PyTorch or NCCL at serve time.

Benchmarks ↓

vLLM recipes

2–6Sparks per cluster

Tuned images, overlays and launchers for each model, with the benchmark scripts in every repo.

The recipes ↓

mentat

2.5MiB of RAM, in place of Ray

Finds its peers with no config, installs as ray so vLLM runs unchanged, and puts one OpenAI-compatible endpoint in front of every model.

kindlingai/mentat ↗

model-loader Early

1download for the whole LAN

Boxes copy models from each other in 64 MiB segments, each checked against its sha256. Peers need no internet access.

kindlingai/model-loader ↗

Spark OS Beta

~4GB more memory for models

A read-only image installed beside DGX OS, with mentat built in. A bad boot falls back to DGX OS.

About Spark OS →

Talos kernel

64KiB pages, or 4 KiB

Talos Linux for Sparks that run Kubernetes: NVIDIA's 6.17 kernel, the 580 driver and GPUDirect Storage.

kindlingai/talos-dgx-kernel ↗

Recipes

Loading recipes…

Experimental · gr8 + exlr8

Full GLM-5.3 on our own engine

All 256 experts per layer, about 1.5 TB in BF16, on three to six Sparks. gr8 is a CUDA engine and exlr8 a weight format, both written for GB10.

Prose decode, single stream tok/s
31.73 Sparks
34.24 Sparks
38.05 Sparks
43.16 Sparks
TP=3TP=4TP=5TP=6
Experts2.25 bpw3.25 bpw3.25 bpw3.25 bpw
Decode, code / prose tok/s41.1 / 31.752.1 / 34.251.0 / 38.057.4 / 43.1
Aggregate, 2 / 4 streams tok/s50.6 / 63.854.0 / 69.061.8 / 76.367.6 / 85.5
Prefill, 8K prompt, cold tok/s6351,3571,5541,808
KV pool tokens307,136485,376502,848614,400

RigMark 1.0.0, low reasoning effort, booted from the Hugging Face release with default settings and GPU clocks uncapped. Mean of two fresh boots on Oct 24, 2026. One of the six boxes was capped at 2200 MHz for the TP=6 run. TP=3 uses 2.25 bpw experts, so compare it with care. Both projects change week to week: report problems on Discord or in GitHub issues.

How we measure

  1. Decode

    One stream, split by workload. Code drafts well; prose is the hard case.

  2. Prefill

    A cold prompt with no prefix-cache hits. This sets time to first token.

  3. Under load

    Total tokens per second across concurrent streams, up to saturation.

  4. Quality

    Needle recall, KL against BF16 or task scores, depending on the recipe.

Join us on Discord

Ask about a recipe, share numbers from your own cluster, or tell us what broke.