# Muse Spark Open: 816B on two Mac Studios, and what the arithmetic needs > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.thesatyajit.com/articles/muse-spark-open > date: 2026-10-16 > tags: mixture-of-experts, kv-cache, apple-silicon, quantization, gguf, licensing, multimodal On 15 October Meta Superintelligence Labs released the weights of **Muse Spark Open** and called it "the most capable US open-weight model to date". The launch thread gives three specs: "816B total / 42B active parameters", a "1M token context window", and "Natively multimodal across text, image, video, audio". The next post says it "is optimized to run on two Apple M5 Ultra Mac Studios with the latest MLX framework". An hour later Unsloth posted Dynamic 2-bit GGUFs to "Run on a Mac Studio", singular. Four of those claims are about bytes and one is about a benchmark table, and all five can be checked from files. Three things first: - **Meta's own repositories are gated.** Both `meta-models` repos need manual approval. I read the same model through three ungated doors: the headers of Unsloth's GGUF build, the headers of the `mlx-community` 4-bit conversion, and the `muse_spark` modeling code already in transformers' main branch. - **The headline counts leave out the vocabulary.** Both 816B and 42B match my counts exactly once the two 202,048 x 8,192 embedding tables are taken out. - **"Two Mac Studios" is a statement about 4-bit weights.** BF16 is 1,644 GB of files and never fits. A 3-bit build fits on one. Throughout, **measured** means I computed it from a file or a run. **Reported** means the publisher's figure, not re-run. **Reasoned** means my arithmetic on the other two. ## What is public Here is what existed on Hugging Face on 16 October, with sizes summed from each repository's file list (measured): | Checkpoint | Repository | Files | Gated | |---|---|---|---| | BF16 | `meta-models/Muse-Spark-Open-816B-A42B` | 1,644.33 GB | yes | | NVFP4 | `meta-models/Muse-Spark-Open-816B-A42B-NVFP4` | 533.01 GB | yes | | FP8 | `amd/Muse-Spark-Open-816B-A42B-Quark-PTPC-FP8` | 832.75 GB | no | | MLX 4-bit | `mlx-community/Muse-Spark-Open-816B-A42B-4bit` | 532.78 GB | no | | MLX 3-bit | `mlx-community/Muse-Spark-Open-816B-A42B-3bit` | 436.14 GB | no | | GGUF, 8 quants | `unsloth/Muse-Spark-Open-816B-A42B-GGUF` | 175.64 to 505.50 GB | no | The card's hardware guidance is "at least 700GB of VRAM for NVFP4 and 2TB for BF16" (reported). There is no paper and no method diagram; the figures here are Meta's benchmark table and Unsloth's quantization charts. ## The architecture, read from the headers A GGUF file opens with its metadata and every tensor's name, shape and type, so a few HTTP range requests per shard give the whole architecture without downloading 225 GB. Unsloth's `UD-IQ2_XXS` build is six shards listing 1,429 tensors, plus 881 in `mmproj-BF16.gguf`, which carries the vision and audio encoders. What a layer looks like (measured, from shapes and metadata, cross-checked against the transformers code): - **62 layers, hidden size 8,192.** No dense layers: `mlp_layer_types` is `sparse` for all 62. - **Attention.** 128 query heads and 16 KV heads of width 64, so `attn_q` is 8,192 x 8,192 and `attn_k` and `attn_v` are 8,192 x 1,024. A fifth matrix, `attn_gate`, is another 8,192 x 8,192: the code multiplies the attention output by its sigmoid before the output projection. That is the card's "output gate". - **A latent MoE.** Each layer first projects the token from 8,192 down to 4,096 (`ffn_latent_down`), runs the experts in that 4,096-wide space, and projects back up (`ffn_latent_up`). Each of the 256 experts is a SwiGLU block whose three matrices are all 4,096 x 4,096, which is 50,331,648 parameters. There is no shared expert (`num_shared_experts: 0`). - **Routing.** Eight of 256 experts per token. Scores are sigmoids, not a softmax. A per-expert bias (`exp_probs_b`) is added only when choosing the eight, and the eight weights are then renormalised to sum to one. That is the auxiliary-loss-free balancing DeepSeek-V3 introduced. - **Residual gates.** Both residual adds are gated per channel: one learned vector `w`, with `beta = sigmoid(w / 0.3)` on the new branch and `alpha = sqrt(sigmoid(-w) x (1 + beta))` on the residual. The card does not mention them. - **A capped head.** Logits are scaled by 0.177 and then soft-capped at 20 with a tanh, the Gemma-style stabiliser. The latent projection keeps 256 experts affordable: at the full 8,192 width each would hold 100.7M parameters, about 1.6T across 62 layers (reasoned), and the down-projection is shared by all eight chosen experts. [Nemotron 3.5 Super VL](/articles/nemotron-3-5-super-vl) and [Kimi K3](/articles/kimi-k3) use the same trick; the [mixture-of-experts explainer](/architectures/mixture-of-experts) covers why routed experts are where the parameters go. Summing every tensor shape gives **819,991,575,808** parameters in the language model and **2,045,074,432** in the encoders, **822,036,650,240** in all (measured). Hugging Face's own count for Meta's gated BF16 safetensors is 822,036,529,664, which agrees to within 120,576 parameters, so the GGUF is the same model. Now the two headline numbers. Take out the input embedding and the output head, 1,655,177,216 parameters each, and the language model is **816,681,221,376**: that is the "816B". A text token runs 8 of 256 experts in each layer plus everything that is not an expert, which is **42,781,801,728** without the vocabulary tables (measured): that is the "42B", and Unsloth's guide writes it as "~42.7B active". With the output head counted it is 44.44B, and with both tables 46.09B. Both headline numbers hold, on the convention that leaves the vocabulary out. The routed experts are 798.86B of the 819.99B, 97.4% of the language model (reasoned), and idle for any one token. That is why a Mac is plausible at all. ### The encoders the card half-describes The card's "Perception Encoder" row says 1.8B parameters, 50 layers, width 1,536 and 14 x 14 patches. The `mmproj` header agrees: the vision tower is 1,851,736,576 parameters in 50 blocks (measured). It also carries something the card's architecture rows never mention: an **audio encoder** of 101,054,976 parameters, built from causal 1-D convolutions in four stages of width 256, 512, 1,024 and 2,048 with strides 16, 6, 5 and 4. One audio token therefore covers 1,920 input samples (reasoned). The sample rate is in Meta's gated processor config, so I cannot turn that into tokens per second. One catch: the `mlx-community` 4-bit conversion has no audio tensors at all (measured, from its 100 headers). On the Mac path, check that your build takes audio before relying on it. ### The tokenizer is Llama 4's The vocabulary is 202,048, and the repo carries a 3,622,230-byte file called `l4_200k_base`. Its git blob hash, `6d1ad505`, is the hash of Llama 4 Scout's `tokenizer.model`, which I downloaded and hashed (measured). In the `tokenizer.json` files the first 200,000 entries and all 439,802 merges are identical to Llama 4's; only the 2,048 special-token slots are reassigned, 20 of them to named tokens such as `<|start|>`, `<|message|>` and `<|audio|>` (measured). Muse Spark Open reads text exactly as Llama 4 did. Under the run exception for this page I loaded that tokenizer with the `tokenizers` library on a shared 4-core cloud VM and counted (measured): the licence text is 1.379 tokens per word, this site's Muse Glimmer article 1.634, and a 37 KB Python file 2.73 characters per token. A 1,048,576-token context is therefore about 760,000 words of plain prose, or about 2.9 MB of Python (reasoned). One more detail from the chat template in the GGUF metadata (measured): `reasoning_effort` switches nothing in the weights. It writes `Reasoning strength: 8`, `32`, `256` or `512` into the system prompt, and defaults to `xhigh`. ## What a million-token cache costs The layer pattern decides what "1M context" costs. `layer_types` lists 62 entries: every fourth layer starting at layer 1 is `full_attention`, the other 46 are `sliding_attention` with a 2,048-token window (measured). `layer_rope_theta` is 500,000 on every sliding layer and **0** on every global one, so the layers that see the whole context have no positional encoding. That is the design of [Muse Glimmer](/articles/muse-glimmer), Meta's 30B agent model from August, at about 27 times the size; the [RoPE explainer](/architectures/rope) covers the sliding layers' half. Per token, each layer caches a key and a value for 16 heads of width 64: 2,048 values, 4,096 bytes in BF16. Only the 16 global layers keep every token: $$ 16 \times 2 \times 16 \times 64 \times 2 \text{ bytes} = 65{,}536 \text{ bytes per token} $$ At 1,048,576 tokens that is **68.72 GB**. The 46 sliding layers stop growing at 2,048 tokens and add a fixed 0.39 GB. The whole cache is **69.1 GB** in BF16, or 34.6 GB in an 8-bit format (reasoned). If all 62 layers were global, it would be **266.3 GB**: 3.9 times more. The 3:1 pattern is the difference between a 1M context that fits beside the weights and one that does not. [Nemotron 3.5 Super VL](/articles/nemotron-3-5-super-vl) gets a million tokens into 8.6 GB with eight attention layers; the [attention and KV-cache explainer](/architectures/attention-kv) has the general arithmetic. ## Two Mac Studios: checking the arithmetic Apple's spec page lists the M5 Ultra Mac Studio with 96 GB of unified memory, configurable to 256 GB or 512 GB, and 1.2 TB/s of memory bandwidth (reported). Meta's blog says the same: "up to 512 GB per Mac Studio, so two machines can hold an entire model". I take Apple's "512GB" as 512 GiB, which is 549.76 GB, so two machines are 1,099.51 GB. Meta never says which precision runs on the two Macs. The files say it for them: - **BF16** is 1,644.33 GB, 544.8 GB more than both machines hold. It never fits. - **FP8** (AMD's checkpoint) is 832.75 GB. With a full 1M BF16 cache it needs 901.9 GB, which fits two Macs with 197.6 GB to spare. Whether MLX runs that format well is a separate question. - **4-bit** (MLX 4-bit at 532.78 GB, NVFP4 at 533.01 GB) is just under one machine's 549.76 GB, which leaves 17 GB for macOS and no cache at all. On two machines it holds the weights and the full 69.1 GB cache with 497.6 GB to spare. This is the configuration the claim describes. - **MLX 3-bit** is 436.14 GB. On **one** 512 GB Mac it holds the weights and a full 1M BF16 cache with 44.5 GB left, or with an 8-bit cache, 79.1 GB left (reasoned). So "two Mac Studios" is true and comfortable at 4-bit, and slightly conservative: a 3-bit build's bytes fit on one. Nobody has published what 3-bit costs in accuracy for this model. ### How fast, at most Decoding reads every active weight once per token, so bandwidth sets a ceiling. The `mlx-community` 4-bit build quantizes the experts of 60 layers and keeps layers 0 and 61's experts, all attention, the latent projections and the output head in BF16 (measured). That is 54.4 GB per token, 39.2 GB of it the BF16 path (reasoned). Two Macs read their halves in turn, so the ceiling is 1.2 TB/s over 54.4 GB: about **22 tokens per second** before any Thunderbolt hop or overhead (reasoned). Meta publishes no Mac throughput to check it against. Meta's NVFP4 checkpoint has the same split: Hugging Face counts 773,094,113,280 packed FP4 values in it, exactly 60 layers of 256 experts (reasoned from its dtype counts; the files are gated). ## Unsloth's 2-bit on one Mac Studio Unsloth's guide says "The 2-bit UD-IQ2_XXS quant will fit on 256GB RAM". The six shards sum to **225.57 GB** (measured), and a 256 GB Mac Studio has 274.88 GB. The vision and audio projector is another 4.10 GB in BF16. That leaves about 45 GB for the cache and macOS (reasoned), and the GPU on a Mac can use only part of unified memory unless you raise its wired limit. A 512K-token BF16 cache needs 34.7 GB and fits; the full 1M needs 69.1 GB and does not, unless the cache is 8-bit, at 34.6 GB (reasoned). So the claim holds, at half the context or with a compressed cache. "Dynamic" means the bits are uneven (measured, from tensor types): every routed expert is `IQ2_XXS`, 2.06 bits per weight and 91.3% of the file, while attention, the latent projections and the head stay at `Q6_K` or `Q8_0`. Per token that build reads about 25.1 GB, a ceiling of about 48 tokens per second at 1.2 TB/s (reasoned).
Unsloth's own charts complicate this. The guide says "quants below UD-IQ2_S do degrade for general tasks, so we generally recommend using UD-IQ2_S as the minimum quant to use for agentic tasks". `UD-IQ2_S` is **275.50 GB** (measured): 0.62 GB more than a 256 GB Mac Studio holds before macOS or a single token of cache. The quant that fits the cheaper machine is the one Unsloth tells you not to use for agents.
That step, about 86% to about 90% general top-1 agreement (read by eye), is one of the largest on the curve. For an agent emitting tool call after tool call it can be the difference between JSON that parses and JSON that does not; nobody has published tool-call accuracy per quant yet. ## "Most capable US open-weight": what it rests on
The card's table has 20 capability rows and seven comparators. Three are from US labs: Thinking Machines' [Inkling](/articles/inkling), NVIDIA's Nemotron 3 Ultra and Reflection's [Beam](/articles/reflection-beam). Mistral Large 4 is French; Qwen3.8, GLM 5.3 and Kimi K3 are Chinese. My tally of Meta's numbers: - **Against the US models, Muse Spark Open wins 18 of 20 rows.** It loses two, both to Beam: GPQA Diamond, 89.0 against 90.5, and Terminal-Bench 2.1, 76.4 against 80.1. Beam's weights were not out when [we covered it](/articles/reflection-beam) on 6 October. - **On the Artificial Analysis Intelligence Index** it scores 39 against Inkling's 26 and Nemotron 3 Ultra's 23. Our [Mistral Large 4](/articles/mistral-large-4) piece pulled every open-weight record from Artificial Analysis on 6 October: the best US one was Inkling Small at 25.7. So the claim holds, by about 13 points (reasoned on reported scores). - **Against the whole table it leads 7 of 20 rows**: JobBench, MCP-Atlas, HLE-Diamond, AA-Omniscience, ZeroBench, SimpleVQA and IntPhys2. GLM 5.3 or Kimi K3 lead the aggregate index, GDPval, every coding row and three of the six vision rows. Mistral Large 4 also beats it on GDPval-AA, DeepSWE and SWE-Atlas-QnA. The launch image is a selection. Of its 11 rows Muse Spark Open leads 6; of the 9 it leaves out, including the index, GPQA Diamond, OSWorld, Toolathlon and SWE-Atlas, it leads 1 (my tally). Nothing in it is wrong; it is the half of the table that looks best. [Nemotron 3.5 Super VL](/articles/nemotron-3-5-super-vl), the US open model this site covered the day before, is not in Meta's table; at 120B total it is a smaller class, and its card gives Terminal-Bench 2.1 as 66.70 (reported, a different harness). One hint sits in the blog's asset names: its hero image is `muse-spark-1-2-open-weights-hero-light-v1.webp` (measured, from the page source). The chart in our [Muse Wonder](/articles/muse-wonder) article puts Meta's closed Muse Spark 1.2 at 40 on the same index, one point above this model's 39. This is probably an open-weights Muse Spark 1.2 with extra safety training. That is my inference; Meta does not say it. The safety rows are where it is furthest from everyone: on StrongREJECT v2 with adaptive attacks, lower is better, it scores 0.5 against 62.9 to 86.4 for every other model with a score (reported). ## The licence, quoted The Muse Model License opens like MIT: permission "to use, copy, modify (including fine-tune), create derivative works, distill, merge, publish, distribute, sublicense, and/or sell copies". Then come the conditions. The commercial limit is section 3: > If the Licensee or any of its affiliates provide a third party with access to artificial intelligence models as a product or service, via API or a similar interface, where the third party receives outputs of such models ("**Model as a Service**"), and the aggregate gross revenue of the Licensee and its affiliates exceeds twenty (20) million US dollars (or the equivalent in other currencies) during any consecutive twelve (12) month period, the Licensee must notify Meta and enter into a separate written agreement with Meta before offering the Software or any Derivatives as part of that Model as a Service business. It then excludes "internal use, research, or any commercial activity that is not Model as a Service" and use "to power features of the Licensee's own product or service". So the limit is narrow: a company reselling the model's outputs through an API needs a deal with Meta once its whole group, not the model, earns more than twenty million dollars a year. A product that uses the model inside its own features never does. Two other conditions are broader than they look: - **Attribution.** Section 2: "Licensee shall prominently display 'Built with Muse Spark Open' in the user interface of each product or service that uses the Software or any Derivatives, or in the accompanying documentation, website, or settings menu." - **What counts as a derivative.** The same section defines Derivatives as "any AI model created, trained, modified (including fine-tuned), distilled, or otherwise improved using the Software or any outputs or results thereof". A model trained on this model's outputs is a Derivative, so it inherits the attribution and the usage policy. The card lists "Synthetic Data Generation" as a use case; this clause is the price of that. The usage policy in Exhibit A also rules out "Military, warfare, nuclear industries or applications, espionage" and "Operation of critical infrastructure, transportation technologies, or heavy machinery". Meta's summary, "limited commercial terms, which allow many developers to use the model for free", is accurate for most readers. It is not an open-source licence, and it is a step back from Muse Glimmer's Apache 2.0. ## What I could not check - **Meta's own bytes.** Everything architectural comes from the GGUF and MLX conversions and the transformers code, which agree with each other and with Hugging Face's parameter count. - **Any benchmark.** All scores are Meta's or Unsloth's; the smallest build, 175.64 GB, cannot run on this VM. - **Speed on the Macs.** Meta gives no tokens-per-second figure; my 22 is a bandwidth ceiling. - **Tokens per image or per second of audio.** Both need the gated processor config. ## The take Under the "two Mac Studios" line is an architecture built to make it true. 97.4% of the language model is routed experts that sit idle for any one token; the experts run in a half-width latent, so 256 per layer cost half what they would; and three of every four layers forget everything older than 2,048 tokens, which keeps a million-token cache at 69 GB instead of 266. Muse Glimmer was designed backwards from a 24 GB card. Muse Spark Open looks designed backwards from a terabyte of unified memory. Each claim holds with a condition attached. 816B and 42B are exact without the vocabulary tables. Two Mac Studios hold the 4-bit build with the full context; one holds the 2-bit build at half the context, in a quant Unsloth calls too lossy for agents. "Most capable US open-weight" is true by a wide margin, while the model leads 7 of its own table's 20 rows. And the licence is free for nearly everyone who is not a large API reseller, as long as they write "Built with Muse Spark Open", including under any model distilled from it. --- *Sources: Meta's [launch thread](https://x.com/AIatMeta/status/2110779754274292212), [blog](https://research.meta.ai/blog/muse-spark-open) and [model card and licence](https://huggingface.co/meta-models/Muse-Spark-Open-816B-A42B); Unsloth's [post](https://x.com/UnslothAI/status/2110789209867112779), [GGUFs](https://huggingface.co/unsloth/Muse-Spark-Open-816B-A42B-GGUF) and [guide](https://unsloth.ai/docs/models/muse-spark-open); the [mlx-community 4-bit build](https://huggingface.co/mlx-community/Muse-Spark-Open-816B-A42B-4bit); transformers' [`muse_spark` code](https://github.com/huggingface/transformers/tree/main/src/transformers/models/muse_spark) (read, not run); [Apple's Mac Studio specs](https://www.apple.com/mac-studio/specs/); [Llama 4 Scout's tokenizer](https://huggingface.co/unsloth/Llama-4-Scout-17B-16E-Instruct). Headers and file sizes read on 16 October. Figure 1 is Meta's, figures 2 and 3 Unsloth's; both interactives are mine.*