Muse Spark Open: 816B on two Mac Studios, and what the arithmetic needs

On 15 October Meta Superintelligence Labs released the weights of Muse Spark Open and called it "the most capable US open-weight model to date". The launch thread gives three specs: "816B total / 42B active parameters", a "1M token context window", and "Natively multimodal across text, image, video, audio". The next post says it "is optimized to run on two Apple M5 Ultra Mac Studios with the latest MLX framework". An hour later Unsloth posted Dynamic 2-bit GGUFs to "Run on a Mac Studio", singular.

Four of those claims are about bytes and one is about a benchmark table, and all five can be checked from files. Three things first:

• Meta's own repositories are gated. Both meta-models repos need manual approval. I read the same model through three ungated doors: the headers of Unsloth's GGUF build, the headers of the mlx-community 4-bit conversion, and the muse_spark modeling code already in transformers' main branch.
• The headline counts leave out the vocabulary. Both 816B and 42B match my counts exactly once the two 202,048 x 8,192 embedding tables are taken out.
• "Two Mac Studios" is a statement about 4-bit weights. BF16 is 1,644 GB of files and never fits. A 3-bit build fits on one.

Throughout, measured means I computed it from a file or a run. Reported means the publisher's figure, not re-run. Reasoned means my arithmetic on the other two.

What is public

Here is what existed on Hugging Face on 16 October, with sizes summed from each repository's file list (measured):

The card's hardware guidance is "at least 700GB of VRAM for NVFP4 and 2TB for BF16" (reported). There is no paper and no method diagram; the figures here are Meta's benchmark table and Unsloth's quantization charts.

The architecture, read from the headers

A GGUF file opens with its metadata and every tensor's name, shape and type, so a few HTTP range requests per shard give the whole architecture without downloading 225 GB. Unsloth's UD-IQ2_XXS build is six shards listing 1,429 tensors, plus 881 in mmproj-BF16.gguf, which carries the vision and audio encoders.

What a layer looks like (measured, from shapes and metadata, cross-checked against the transformers code):

• 62 layers, hidden size 8,192. No dense layers: mlp_layer_types is sparse for all 62.
• Attention. 128 query heads and 16 KV heads of width 64, so attn_q is 8,192 x 8,192 and attn_k and attn_v are 8,192 x 1,024. A fifth matrix, attn_gate, is another 8,192 x 8,192: the code multiplies the attention output by its sigmoid before the output projection. That is the card's "output gate".
• A latent MoE. Each layer first projects the token from 8,192 down to 4,096 (ffn_latent_down), runs the experts in that 4,096-wide space, and projects back up (ffn_latent_up). Each of the 256 experts is a SwiGLU block whose three matrices are all 4,096 x 4,096, which is 50,331,648 parameters. There is no shared expert (num_shared_experts: 0).
• Routing. Eight of 256 experts per token. Scores are sigmoids, not a softmax. A per-expert bias (exp_probs_b) is added only when choosing the eight, and the eight weights are then renormalised to sum to one. That is the auxiliary-loss-free balancing DeepSeek-V3 introduced.
• Residual gates. Both residual adds are gated per channel: one learned vector w, with beta = sigmoid(w / 0.3) on the new branch and alpha = sqrt(sigmoid(-w) x (1 + beta)) on the residual. The card does not mention them.
• A capped head. Logits are scaled by 0.177 and then soft-capped at 20 with a tanh, the Gemma-style stabiliser.

The latent projection keeps 256 experts affordable: at the full 8,192 width each would hold 100.7M parameters, about 1.6T across 62 layers (reasoned), and the down-projection is shared by all eight chosen experts. Nemotron 3.5 Super VL and Kimi K3 use the same trick; the mixture-of-experts explainer covers why routed experts are where the parameters go.

Summing every tensor shape gives 819,991,575,808 parameters in the language model and 2,045,074,432 in the encoders, 822,036,650,240 in all (measured). Hugging Face's own count for Meta's gated BF16 safetensors is 822,036,529,664, which agrees to within 120,576 parameters, so the GGUF is the same model.

Now the two headline numbers. Take out the input embedding and the output head, 1,655,177,216 parameters each, and the language model is 816,681,221,376: that is the "816B". A text token runs 8 of 256 experts in each layer plus everything that is not an expert, which is 42,781,801,728 without the vocabulary tables (measured): that is the "42B", and Unsloth's guide writes it as "~42.7B active". With the output head counted it is 44.44B, and with both tables 46.09B. Both headline numbers hold, on the convention that leaves the vocabulary out.

The routed experts are 798.86B of the 819.99B, 97.4% of the language model (reasoned), and idle for any one token. That is why a Mac is plausible at all.

The encoders the card half-describes

The card's "Perception Encoder" row says 1.8B parameters, 50 layers, width 1,536 and 14 x 14 patches. The mmproj header agrees: the vision tower is 1,851,736,576 parameters in 50 blocks (measured). It also carries something the card's architecture rows never mention: an audio encoder of 101,054,976 parameters, built from causal 1-D convolutions in four stages of width 256, 512, 1,024 and 2,048 with strides 16, 6, 5 and 4. One audio token therefore covers 1,920 input samples (reasoned). The sample rate is in Meta's gated processor config, so I cannot turn that into tokens per second.

One catch: the mlx-community 4-bit conversion has no audio tensors at all (measured, from its 100 headers). On the Mac path, check that your build takes audio before relying on it.

The tokenizer is Llama 4's

The vocabulary is 202,048, and the repo carries a 3,622,230-byte file called l4_200k_base. Its git blob hash, 6d1ad505, is the hash of Llama 4 Scout's tokenizer.model, which I downloaded and hashed (measured). In the tokenizer.json files the first 200,000 entries and all 439,802 merges are identical to Llama 4's; only the 2,048 special-token slots are reassigned, 20 of them to named tokens such as <|start|>, <|message|> and <|audio|> (measured). Muse Spark Open reads text exactly as Llama 4 did.

Under the run exception for this page I loaded that tokenizer with the tokenizers library on a shared 4-core cloud VM and counted (measured): the licence text is 1.379 tokens per word, this site's Muse Glimmer article 1.634, and a 37 KB Python file 2.73 characters per token. A 1,048,576-token context is therefore about 760,000 words of plain prose, or about 2.9 MB of Python (reasoned).

One more detail from the chat template in the GGUF metadata (measured): reasoning_effort switches nothing in the weights. It writes Reasoning strength: 8, 32, 256 or 512 into the system prompt, and defaults to xhigh.

What a million-token cache costs

The layer pattern decides what "1M context" costs. layer_types lists 62 entries: every fourth layer starting at layer 1 is full_attention, the other 46 are sliding_attention with a 2,048-token window (measured). layer_rope_theta is 500,000 on every sliding layer and 0 on every global one, so the layers that see the whole context have no positional encoding. That is the design of Muse Glimmer, Meta's 30B agent model from August, at about 27 times the size; the RoPE explainer covers the sliding layers' half.

Per token, each layer caches a key and a value for 16 heads of width 64: 2,048 values, 4,096 bytes in BF16. Only the 16 global layers keep every token:

At 1,048,576 tokens that is 68.72 GB. The 46 sliding layers stop growing at 2,048 tokens and add a fixed 0.39 GB. The whole cache is 69.1 GB in BF16, or 34.6 GB in an 8-bit format (reasoned). If all 62 layers were global, it would be 266.3 GB: 3.9 times more. The 3:1 pattern is the difference between a 1M context that fits beside the weights and one that does not. Nemotron 3.5 Super VL gets a million tokens into 8.6 GB with eight attention layers; the attention and KV-cache explainer has the general arithmetic.

Two Mac Studios: checking the arithmetic

Apple's spec page lists the M5 Ultra Mac Studio with 96 GB of unified memory, configurable to 256 GB or 512 GB, and 1.2 TB/s of memory bandwidth (reported). Meta's blog says the same: "up to 512 GB per Mac Studio, so two machines can hold an entire model". I take Apple's "512GB" as 512 GiB, which is 549.76 GB, so two machines are 1,099.51 GB.

Meta never says which precision runs on the two Macs. The files say it for them:

• BF16 is 1,644.33 GB, 544.8 GB more than both machines hold. It never fits.
• FP8 (AMD's checkpoint) is 832.75 GB. With a full 1M BF16 cache it needs 901.9 GB, which fits two Macs with 197.6 GB to spare. Whether MLX runs that format well is a separate question.
• 4-bit (MLX 4-bit at 532.78 GB, NVFP4 at 533.01 GB) is just under one machine's 549.76 GB, which leaves 17 GB for macOS and no cache at all. On two machines it holds the weights and the full 69.1 GB cache with 497.6 GB to spare. This is the configuration the claim describes.
• MLX 3-bit is 436.14 GB. On one 512 GB Mac it holds the weights and a full 1M BF16 cache with 44.5 GB left, or with an 8-bit cache, 79.1 GB left (reasoned).

So "two Mac Studios" is true and comfortable at 4-bit, and slightly conservative: a 3-bit build's bytes fit on one. Nobody has published what 3-bit costs in accuracy for this model.

How fast, at most

Decoding reads every active weight once per token, so bandwidth sets a ceiling. The mlx-community 4-bit build quantizes the experts of 60 layers and keeps layers 0 and 61's experts, all attention, the latent projections and the output head in BF16 (measured). That is 54.4 GB per token, 39.2 GB of it the BF16 path (reasoned). Two Macs read their halves in turn, so the ceiling is 1.2 TB/s over 54.4 GB: about 22 tokens per second before any Thunderbolt hop or overhead (reasoned). Meta publishes no Mac throughput to check it against.

Meta's NVFP4 checkpoint has the same split: Hugging Face counts 773,094,113,280 packed FP4 values in it, exactly 60 layers of 256 experts (reasoned from its dtype counts; the files are gated).

Unsloth's 2-bit on one Mac Studio

Unsloth's guide says "The 2-bit UD-IQ2_XXS quant will fit on 256GB RAM". The six shards sum to 225.57 GB (measured), and a 256 GB Mac Studio has 274.88 GB. The vision and audio projector is another 4.10 GB in BF16. That leaves about 45 GB for the cache and macOS (reasoned), and the GPU on a Mac can use only part of unified memory unless you raise its wired limit.

A 512K-token BF16 cache needs 34.7 GB and fits; the full 1M needs 69.1 GB and does not, unless the cache is 8-bit, at 34.6 GB (reasoned). So the claim holds, at half the context or with a compressed cache.

"Dynamic" means the bits are uneven (measured, from tensor types): every routed expert is IQ2_XXS, 2.06 bits per weight and 91.3% of the file, while attention, the latent projections and the head stay at Q6_K or Q8_0. Per token that build reads about 25.1 GB, a ceiling of about 48 tokens per second at 1.2 TB/s (reasoned).

expand

Unsloth's own charts complicate this. The guide says "quants below UD-IQ2_S do degrade for general tasks, so we generally recommend using UD-IQ2_S as the minimum quant to use for agentic tasks". UD-IQ2_S is 275.50 GB (measured): 0.62 GB more than a 256 GB Mac Studio holds before macOS or a single token of cache. The quant that fits the cheaper machine is the one Unsloth tells you not to use for agents.

expand

That step, about 86% to about 90% general top-1 agreement (read by eye), is one of the largest on the curve. For an agent emitting tool call after tool call it can be the difference between JSON that parses and JSON that does not; nobody has published tool-call accuracy per quant yet.

"Most capable US open-weight": what it rests on

expand

The card's table has 20 capability rows and seven comparators. Three are from US labs: Thinking Machines' Inkling, NVIDIA's Nemotron 3 Ultra and Reflection's Beam. Mistral Large 4 is French; Qwen3.8, GLM 5.3 and Kimi K3 are Chinese. My tally of Meta's numbers:

• Against the US models, Muse Spark Open wins 18 of 20 rows. It loses two, both to Beam: GPQA Diamond, 89.0 against 90.5, and Terminal-Bench 2.1, 76.4 against 80.1. Beam's weights were not out when we covered it on 6 October.
• On the Artificial Analysis Intelligence Index it scores 39 against Inkling's 26 and Nemotron 3 Ultra's 23. Our Mistral Large 4 piece pulled every open-weight record from Artificial Analysis on 6 October: the best US one was Inkling Small at 25.7. So the claim holds, by about 13 points (reasoned on reported scores).
• Against the whole table it leads 7 of 20 rows: JobBench, MCP-Atlas, HLE-Diamond, AA-Omniscience, ZeroBench, SimpleVQA and IntPhys2. GLM 5.3 or Kimi K3 lead the aggregate index, GDPval, every coding row and three of the six vision rows. Mistral Large 4 also beats it on GDPval-AA, DeepSWE and SWE-Atlas-QnA.

The launch image is a selection. Of its 11 rows Muse Spark Open leads 6; of the 9 it leaves out, including the index, GPQA Diamond, OSWorld, Toolathlon and SWE-Atlas, it leads 1 (my tally). Nothing in it is wrong; it is the half of the table that looks best.

Nemotron 3.5 Super VL, the US open model this site covered the day before, is not in Meta's table; at 120B total it is a smaller class, and its card gives Terminal-Bench 2.1 as 66.70 (reported, a different harness).

One hint sits in the blog's asset names: its hero image is muse-spark-1-2-open-weights-hero-light-v1.webp (measured, from the page source). The chart in our Muse Wonder article puts Meta's closed Muse Spark 1.2 at 40 on the same index, one point above this model's 39. This is probably an open-weights Muse Spark 1.2 with extra safety training. That is my inference; Meta does not say it.

The safety rows are where it is furthest from everyone: on StrongREJECT v2 with adaptive attacks, lower is better, it scores 0.5 against 62.9 to 86.4 for every other model with a score (reported).

The licence, quoted

The Muse Model License opens like MIT: permission "to use, copy, modify (including fine-tune), create derivative works, distill, merge, publish, distribute, sublicense, and/or sell copies". Then come the conditions.

The commercial limit is section 3:

If the Licensee or any of its affiliates provide a third party with access to artificial intelligence models as a product or service, via API or a similar interface, where the third party receives outputs of such models ("Model as a Service"), and the aggregate gross revenue of the Licensee and its affiliates exceeds twenty (20) million US dollars (or the equivalent in other currencies) during any consecutive twelve (12) month period, the Licensee must notify Meta and enter into a separate written agreement with Meta before offering the Software or any Derivatives as part of that Model as a Service business.

It then excludes "internal use, research, or any commercial activity that is not Model as a Service" and use "to power features of the Licensee's own product or service". So the limit is narrow: a company reselling the model's outputs through an API needs a deal with Meta once its whole group, not the model, earns more than twenty million dollars a year. A product that uses the model inside its own features never does.

Two other conditions are broader than they look:

• Attribution. Section 2: "Licensee shall prominently display 'Built with Muse Spark Open' in the user interface of each product or service that uses the Software or any Derivatives, or in the accompanying documentation, website, or settings menu."
• What counts as a derivative. The same section defines Derivatives as "any AI model created, trained, modified (including fine-tuned), distilled, or otherwise improved using the Software or any outputs or results thereof". A model trained on this model's outputs is a Derivative, so it inherits the attribution and the usage policy. The card lists "Synthetic Data Generation" as a use case; this clause is the price of that.

The usage policy in Exhibit A also rules out "Military, warfare, nuclear industries or applications, espionage" and "Operation of critical infrastructure, transportation technologies, or heavy machinery". Meta's summary, "limited commercial terms, which allow many developers to use the model for free", is accurate for most readers. It is not an open-source licence, and it is a step back from Muse Glimmer's Apache 2.0.

What I could not check

• Meta's own bytes. Everything architectural comes from the GGUF and MLX conversions and the transformers code, which agree with each other and with Hugging Face's parameter count.
• Any benchmark. All scores are Meta's or Unsloth's; the smallest build, 175.64 GB, cannot run on this VM.
• Speed on the Macs. Meta gives no tokens-per-second figure; my 22 is a bandwidth ceiling.
• Tokens per image or per second of audio. Both need the gated processor config.

The take

Under the "two Mac Studios" line is an architecture built to make it true. 97.4% of the language model is routed experts that sit idle for any one token; the experts run in a half-width latent, so 256 per layer cost half what they would; and three of every four layers forget everything older than 2,048 tokens, which keeps a million-token cache at 69 GB instead of 266. Muse Glimmer was designed backwards from a 24 GB card. Muse Spark Open looks designed backwards from a terabyte of unified memory.

Each claim holds with a condition attached. 816B and 42B are exact without the vocabulary tables. Two Mac Studios hold the 4-bit build with the full context; one holds the 2-bit build at half the context, in a quant Unsloth calls too lossy for agents. "Most capable US open-weight" is true by a wide margin, while the model leads 7 of its own table's 20 rows. And the licence is free for nearly everyone who is not a large API reseller, as long as they write "Built with Muse Spark Open", including under any model distilled from it.

Sources: Meta's launch thread, blog and model card and licence; Unsloth's post, GGUFs and guide; the mlx-community 4-bit build; transformers' muse_spark code (read, not run); Apple's Mac Studio specs; Llama 4 Scout's tokenizer. Headers and file sizes read on 16 October. Figure 1 is Meta's, figures 2 and 3 Unsloth's; both interactives are mine.