Apple's MLX can now split huge LLMs across clustered Mac Studios
A new RDMA driver over Thunderbolt 5 lets MLX split a model across several Mac Studios, running Meta's 816B-parameter Muse Spark Open at over 36 tokens per second on four M5 Ultras.
Apple’s machine learning research team published a guide on 15 October showing how MLX can now split a large language model across several Mac Studios instead of running everything on one machine. The feature relies on a low-latency RDMA driver over Thunderbolt 5 that macOS 26.2 introduced, paired with jaccl, a set of communication collectives MLX 0.30.1 added for different parallelisation schemes.
Apple tested the setup on Meta’s Muse Spark Open, an 816-billion-parameter mixture-of-experts model with 42 billion active parameters, splitting it across four Mac Studios fitted with M5 Ultra chips to reach over 36 tokens per second. On a single machine, the M5 Ultra showed up to 4.6 times faster time-to-first-token and 1.6 times faster decode speed than the previous M3 Ultra chip. A distributed run is launched like this:
mlx.launch --hostfile hostfile.json --backend jaccl --env MLX_METAL_FAST_SYNCH=1 -- \
mlx_lm.server --model meta-models/Muse-Spark-Open-816B-A42B-NVFP4 --port 8080Why it matters: this is a genuine path to self-hosting frontier-scale open models without a data centre, just a mesh of consumer-grade Macs connected over Thunderbolt. It matters most if you want to run or fine-tune a huge open-weight model privately rather than through someone else’s API.
The caveat: Apple’s own numbers show mixture-of-experts models like Muse Spark Open don’t scale across machines as cleanly as dense models do, and tensor-parallel communication overhead bites hardest at large context sizes when only a few machines are in the cluster.