A Local Inference Cluster of 3 Dell GB10s Running Trinity-Large-Thinking

Sentdex built a local LLM cluster from 3 Dell GB10s with 384GB of combined memory. This article breaks down the hardware setup and the logic behind running Trinity-Large-Thinking locally.

Harrison Kinsley (@Sentdex) posted a local LLM cluster on X: 3 Dell GB10 units, each with 128GB of unified memory, 384GB in total, with nodes interconnected via 200Gbps QSFP in a mesh topology.

His current results: Trinity-Large-Thinking (398B) quantized to Q4_K_M runs at 17-18 t/s. The post also mentions a run log at 17.43 t/s, taking about 13 seconds.

The author says he’ll keep observing model performance on this cluster, and also plans to use it as a coding agent, connected via Hermes.

Original Post Details