exlr8 Experimental
2 · 3 · 4bits per expert
Our weight format for GB10. The bit mix is picked at load time, and prefill runs on FP8 tensor cores.
kindlingai/exlr8 ↗Open source · NVIDIA DGX Spark · GB10
Everything you need to serve large open models on 2 to 6 DGX Sparks: the OS, the cluster tools, the engine and the weights.
Fastest decode
–tok/s
Prose decode, single stream
Four layers, all open source.
exlr8 Experimental
2 · 3 · 4bits per expert
Our weight format for GB10. The bit mix is picked at load time, and prefill runs on FP8 tensor cores.
kindlingai/exlr8 ↗Hugging Face
2exlr8 releases
Full GLM-5.3 and GLM-5.3-Flash. Each box downloads only the slices it serves.
huggingface.co/kindlingai ↗gr8 Experimental
43.1tok/s prose, full GLM-5.3, 6 Sparks
A from-scratch CUDA engine for full GLM-5.3 on three to six Sparks. No vLLM, PyTorch or NCCL at serve time.
Benchmarks ↓vLLM recipes
2–6Sparks per cluster
Tuned images, overlays and launchers for each model, with the benchmark scripts in every repo.
The recipes ↓mentat
2.5MiB of RAM, in place of Ray
Finds its peers with no config, installs as ray so vLLM runs unchanged, and puts one OpenAI-compatible endpoint in front of every model.
model-loader Early
1download for the whole LAN
Boxes copy models from each other in 64 MiB segments, each checked against its sha256. Peers need no internet access.
kindlingai/model-loader ↗Spark OS Beta
~4GB more memory for models
A read-only image installed beside DGX OS, with mentat built in. A bad boot falls back to DGX OS.
About Spark OS →Talos kernel
64KiB pages, or 4 KiB
Talos Linux for Sparks that run Kubernetes: NVIDIA's 6.17 kernel, the 580 driver and GPUDirect Storage.
kindlingai/talos-dgx-kernel ↗Loading recipes…
Experimental · gr8 + exlr8
All 256 experts per layer, about 1.5 TB in BF16, on three to six Sparks. gr8 is a CUDA engine and exlr8 a weight format, both written for GB10.
| TP=3 | TP=4 | TP=5 | TP=6 | |
|---|---|---|---|---|
| Experts | 2.25 bpw | 3.25 bpw | 3.25 bpw | 3.25 bpw |
| Decode, code / prose tok/s | 41.1 / 31.7 | 52.1 / 34.2 | 51.0 / 38.0 | 57.4 / 43.1 |
| Aggregate, 2 / 4 streams tok/s | 50.6 / 63.8 | 54.0 / 69.0 | 61.8 / 76.3 | 67.6 / 85.5 |
| Prefill, 8K prompt, cold tok/s | 635 | 1,357 | 1,554 | 1,808 |
| KV pool tokens | 307,136 | 485,376 | 502,848 | 614,400 |
RigMark 1.0.0, low reasoning effort, booted from the Hugging Face release with default settings and GPU clocks uncapped. Mean of two fresh boots on Oct 24, 2026. One of the six boxes was capped at 2200 MHz for the TP=6 run. TP=3 uses 2.25 bpw experts, so compare it with care. Both projects change week to week: report problems on Discord or in GitHub issues.
One stream, split by workload. Code drafts well; prose is the hard case.
A cold prompt with no prefix-cache hits. This sets time to first token.
Total tokens per second across concurrent streams, up to saturation.
Needle recall, KL against BF16 or task scores, depending on the recipe.
Ask about a recipe, share numbers from your own cluster, or tell us what broke.