# SAM 3.1: the 7× is at 128 objects
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/sam-3-1-object-multiplex
> date: 2026-09-19
> tags: explainer, architecture, vision, inference, benchmarks
Meta released SAM 3.1 on 27 March 2026 with three different headline numbers,
on three different surfaces, for the same piece of work.
The [Hugging Face card](https://huggingface.co/facebook/sam3.1) says Object
Multiplex "delivers ~7x faster inference at 128 objects on a single H100 GPU
without sacrificing accuracy." The
[blog post](https://ai.meta.com/blog/segment-anything-model-3/) says it "doubles
the processing speed for videos with a medium number of objects, increasing
throughput from 16 to 32 frames per second on a single H100 GPU." Appendix H of
the [paper](https://arxiv.org/abs/2511.16719), posted the day after the release,
says it achieves "a 5.2× speedup at 128 objects."
None of these contradicts the others. They are three readings of one curve, and
the reason they differ by a factor of three and a half is the most interesting
thing about the release.
First, the workload itself, because the whole argument below is about what it
costs and a still frame cannot show it:
## Meta published the curve
The brief version of a scaling claim is a ratio. The honest version is a graph,
and Meta shipped one — three lines, eight measured object counts, every point
labelled.
Read the ends. At 128 objects SAM 3.1 runs at 11.5 fps against the November
release's 1.6 — **7.19×**, which is the model card's number and the 7.2×
callout on Meta's own figure. Against the middle line, SAM 3 carrying the same
batched postprocessing and `torch.compile` fusion, it is 11.5 against 2.2 —
**5.23×**, which is the paper's. The gap between those two is not accuracy or
measurement noise. It is a release that bundles an algorithm with a pile of
inference engineering, headlined on the combined figure and ablated in the
appendix.
**Receipts.** The ~7× is the SAM 3.1 release against the November 2025 SAM 3 release at 128 tracked objects. Against a SAM 3 carrying the same inference optimisations — the line the paper compares — Object Multiplex is worth 5.2×. At one object it is a 7% regression.
| objects | buckets | SAM 3 Nov 2025 fps | SAM 3 + opt fps | SAM 3.1 fps | × vs Nov 2025 | × vs SAM 3 + opt |
| ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 1 | 1 | 26.5 | 36.4 | 33.8 | 1.28× | 0.93× |
| 2 | 1 | 23.8 | 32.0 | 33.3 | 1.40× | 1.04× |
| 4 | 1 | 19.7 | 31.5 | 32.5 | 1.65× | 1.03× |
| 8 | 1 | 14.6 | 19.2 | 31.6 | 2.16× | 1.65× |
| 16 | 1 | 9.8 | 12.2 | 30.2 | 3.08× | 2.48× |
| 32 | 2 | 5.8 | 7.0 | 22.1 | 3.81× | 3.16× |
| 64 | 4 | 3.0 | 3.8 | 15.1 | 5.03× | 3.97× |
| 128 | 8 | 1.6 | 2.2 | 11.5 | 7.19× | 5.23× |
FPS on a single H100. Meta's footnote to Figure 31 states both SAM 3 and SAM 3 Multiplex are measured with optimised internal implementations; the third line ('SAM 3 (Nov 2025 Release)') is the unoptimised public baseline. The 7.2× and 3.1× callouts on the figure are Meta's own.
> method: Read off the printed data labels on Meta's own three-line FPS figure (assets/sam3.1_efficiency.png in facebookresearch/sam3, the release version of paper Figure 31). Ratios computed from those labels; bucket count is ceil(N/16) with the shipped default multiplex_count=16.
> source: https://github.com/facebookresearch/sam3/blob/main/RELEASE_SAM3p1.md
> captured: 2026-09-19
> data: https://ai.thesatyajit.com/articles/sam-3-1-object-multiplex/data/fps-curve.json (8 rows)
The blog's 2× is the same gap read at the other end. Nothing on the published
curve is exactly 16 → 32 fps; the nearest measured point is eight objects, where
SAM 3 does 14.6 and SAM 3.1 does 31.6, a 2.16× that rounds to the sentence Meta
wrote. So "medium number of objects" means about seven, and the blog chose the
part of the curve where most readers actually live. That reading is corroborated
by Meta's own November sentence about SAM 3 — that it sustains "near real-time
performance for approximately five concurrent objects," which on this curve is
somewhere around 18 fps. Both statements are describing the same handful of
objects, four months apart.
The 128 in the headline is not arbitrary either. It is the shipped default:
`max_num_objects=128` in `sam3/model/sam3_multiplex_base.py`, with the comment
*"128 objects (total across all GPUs) should be able to cover nearly all
cases."* Meta quoted the speedup at its own declared worst case. That is a
defensible place to quote from — as long as you say so, which the model card
does and the blog does not.
## What is actually shared
SAM 3 already shared the expensive part. Its detector and tracker sit on one
Perception Encoder backbone, and the frame is encoded once no matter how many
objects are in it. What was replicated was everything downstream: a memory bank
per tracked object, a memory-encode and memory-attention pass per object, a mask
decode per object. The November blog named this as a known limitation, in the
section on what the community should work on next:
"When applied to video, SAM 3 tracks every object with a SAM 2-style masklet,
which means the cost of SAM 3 inference scales linearly with the number of
objects being tracked. **Each object is processed separately, utilizing only
shared per-frame embeddings, without inter-object communication.** Incorporating
shared object-level contextual information could aid in improving efficiency and
model performance in complex scenes with many visually similar objects."
Four months later they shipped exactly that. Object Multiplex groups objects
into fixed-capacity buckets of `M` slots — `M = 16` by default — and runs the
memory path once per bucket instead of once per object, taking the pass count
from `O(N)` to `O(⌈N/M⌉)`.
The factorisation is the mechanism, and the paper states it plainly: Multiplex
splits object information into **(i) a shared spatial memory** and **(ii)
object-specific embeddings**. The shared memory is produced by encoding the
frame features together with all sixteen slots' mask signals at once, yielding
one memory representation for the whole bucket. The embeddings are what keeps
the objects apart — stored in the memory bank alongside the spatial features,
and handed to the mask decoder so shared features get attributed to the right
object.
Both halves are in the tracker, at `2345a4a`. The shared half first:
```python
# sam3/model/video_tracking_multiplex.py — _prepare_memory_conditioned_features()
B = multiplex_state.num_buckets
# B = current_vision_feats[-1].size(1) # batch size on this frame
vision_feat = current_vision_feats[-1].expand(-1, B, -1)
vision_mask = (
current_vision_masks[-1].expand(-1, B, -1)
if current_vision_masks[-1] is not None
else None
)
vision_pos_embed = current_vision_pos_embeds[-1].expand(-1, B, -1)
```
The commented-out line is the diff. `B` used to be the batch dimension, which
for a video tracker is the object dimension; now it is the bucket count. And
`.expand` on a size-1 axis is a stride-0 view, not a copy — twenty objects see
the same 5,184 × 256 frame encoding, addressed twenty times, with nothing
duplicated in memory.
The object-specific half is a matmul against a partial permutation matrix, and
it is where sixteen objects become one memory encode:
```python
# the same file — _encode_new_memory(): sixteen masks in, one memory out
mux_mask_for_mem = multiplex_state.mux(mask_for_mem).squeeze(2) # [N,1,H,W] -> [b,16,H,W]
...
embedded_conditions = multiplex_state.mux(embedded_conditions).squeeze(2)
mux_mask_for_mem = torch.cat([mux_mask_for_mem, embedded_conditions], dim=1)
...
maskmem_out = self.maskmem_backbone(pix_feat, mux_mask_for_mem, skip_mask_sigmoid=True)
```
One call to `maskmem_backbone` where SAM 3 makes sixteen. The `cat` is also
where the encoder's 32 input channels come from, and they are not what you would
guess: sixteen slot masks, plus sixteen spatial flags marking which slots were
conditioned on this frame.
The decoder is the other half of the split, and there identity costs one add:
```python
# sam3/model/multiplex_mask_decoder.py — predict_masks(): 80 queries, one pass
mask_tokens = self.mask_tokens.weight.view(
1, self.multiplex_count, self.num_mask_output_per_object, -1
).expand(B, -1, -1, -1)
mask_tokens = mask_tokens + extra_per_object_embeddings.unsqueeze(2)
mask_tokens = mask_tokens.flatten(1, 2)
tokens = torch.cat([tokens, mask_tokens], dim=1) # obj_score 16 + iou 16 + mask 48
...
hs, src = self.transformer(src, pos_src, tokens)
```
Sixteen objects, one call to `self.transformer`. The only thing that
distinguishes them is a per-object vector summed onto their mask queries.
All of that is checkable in the released code, and it checks out:
- `build_sam3_multiplex_video_model(multiplex_count: int = 16)` — the bucket
capacity, exactly as the paper says.
- The memory encoder's mask downsampler is built with
`multiplex_count=16, input_channel_multiplier=2`, and inside
`SimpleMaskDownSampler` the first convolution's input width is
`multiplex_count * input_channel_multiplier` — **32 channels**, which the
`cat` above accounts for exactly, convolved down to one 256-dim memory. SAM 3's
builder passes none of those arguments, so the same module takes **one**
channel and runs once per object. It is not quite the same stack, though: the
multiplex builder also passes `starting_out_chan=4`, shifting the whole channel
schedule one step up — `32 → 16 → 64 → 256 → 1024 → 256` against SAM 3's
`1 → 4 → 16 → 64 → 256 → 256`. Every layer's output is four times wider. That
widening is what the 7% further down is paying for.
- The mask decoder allocates `multiplex_count × 3` mask tokens plus a
16-wide `iou_token` and a 16-wide `obj_score_token` — **80 tokens per
bucket**, which is the paper's 5 × M (three mask, one IoU, one object-score,
per slot), and which is why sixteen objects' worth of mask decoding costs
barely more than one object's.
- The object embeddings arrive as `extra_per_object_embeddings`, documented in
the decoder as *"a tensor with shape b * multiplex_count * C to be added to
the mask tokens"* — added, not concatenated, as the fence above shows.
The justification for why this does not destroy information is one sentence and
worth pausing on: *"per-object memory features are largely spatially disjoint in
most frames; overlaps are sparse, so joint encoding preserves object-specific
information while avoiding redundant computation."* Sixteen objects in a frame
mostly occupy sixteen different places. Sharing the canvas costs you almost
nothing because they were never fighting over it.
The lineage is cited and it is a good one: **DataMUX** (Murahari et al., NeurIPS
2022), which multiplexes up to 20–40 unrelated inputs through one network for
11–18× throughput at a 2–4% accuracy cost. Object Multiplex is the same trick
with a much friendlier premise. DataMUX mixes inputs that have nothing to do
with each other, so interference is pure loss. Here the sixteen things being
mixed are sixteen objects in the same frame, which is why the accuracy doesn't
go down — and, on the crowded benchmarks, why it goes up.
The blog's title promises "Multiplexing **and Global Reasoning**." Outside that
title the phrase occurs once, in one blog sentence, and zero times in the
release notes, the paper, or the code. There is no module called that. What
there is is a shared memory that gives sixteen objects sight of each other,
which is a real thing and does not need a second name.
## Why the number is a slope
Here is the part that the single ratio hides. Fit a straight line to Meta's own
eight points and SAM 3's video cost is almost perfectly linear in object count:
```
SAM 3 (Nov 2025) ms/frame = 30.9 + 4.645 · N R² = 0.99967
SAM 3.1 ms/frame = 29.0 + 0.477 · N R² = 0.978
```
Two lines with **the same intercept and a slope 9.7× shallower**. The intercept
— 30.9 ms against 29.0, the same number within the noise of an eight-point fit —
is the per-frame backbone, which both versions always shared. The slope is the
memory path, and that is the only thing Object Multiplex touched.
Inside a single bucket the slope is flatter still: across 1 → 16 objects SAM 3.1
goes from 29.6 to 33.1 ms, a marginal **0.23 ms per object**, against SAM 3's
4.29 ms over the same range. Eighteen times cheaper per additional object, up to
the sixteenth. Then a second bucket opens and the line kinks.
Which means "7× faster" is not a property of the model. It is one evaluation of
a ratio between two lines, and you can make it come out at almost any value by
choosing where to stand:
| objects | speedup vs Nov 2025 | speedup vs optimized SAM 3 |
|---|---|---|
| 1 | 1.28× | **0.93× — slower** |
| 8 | 2.16× | 1.65× |
| 16 | 3.08× | 2.48× |
| 128 | 7.19× | 5.23× |
If you track a handful of objects — which, for most video work, you do — Object
Multiplex gives you somewhere between nothing and 65%, and the rest of your
gain is the `torch.compile` work.
### The 7% tax, published
At one object SAM 3.1 runs at 33.8 fps against an optimized SAM 3's 36.4. It is
**slower**, by about 7%, and the paper says so in its own words: *"At
single-object tracking, the bucketization strategy in Multiplex introduces a
modest overhead of approximately 7% in latency. However, beyond the crossover
point (≥2 objects), SAM 3 Multiplex outperforms SAM 3."*
The tax is charged in one line. A bucket is sixteen slots wide before it knows
how many objects it will hold:
```python
# sam3/model/multiplex_utils.py — MultiplexState.add_objects()
while len(object_indices) > 0:
new_bucket = [_PADDING_NUM] * self.multiplex_count # 16 slots, always
for i in range(self.allowed_bucket_capacity):
if len(object_indices) == 0:
break
new_bucket[i] = _pop_next()
self.assignments.append(new_bucket)
```
Every tensor downstream is then shaped by that width rather than by the object
count:
```python
# the same file — MultiplexState.mux(): "padding entries filled with 0"
output_shape = (self.num_buckets, self.multiplex_count) + x.shape[1:]
...
result_flat = self.mux_matrix @ x_flat # [buckets*16, N] @ [N, features]
```
`mux_matrix` carries a 1 for every filled slot and an all-zero row for every
empty one, so a single-object bucket is a `[1, 16, …]` tensor that is
fifteen-sixteenths zeros — and it is fed to a convolution stack four times wider
than SAM 3's. Counting multiply-accumulates through both constructors at the
shipped `interpol_size=[1152, 1152]`: SAM 3's mask downsampler is **1.355 GMAC
per object**, SAM 3.1's is **18.94 GMAC per bucket**. Fourteen times the
arithmetic, for one call that is supposed to cover sixteen objects.
Which means the memory *encoder* is not where multiplexing wins. At sixteen
objects it is close to a wash — 21.7 GMAC for sixteen SAM 3 calls against 18.94
for one bucket. The collapse from `O(N)` to `O(⌈N/M⌉)` is in memory attention
and the mask decoder, which genuinely go from N passes to one. At a single
object you pay the wider encoder and get none of the collapse. Seven percent.
Meta measured that and printed it on the chart, next to the number they were
selling. Credit where it is due: a release that publishes its own crossover
point is telling you how to decide against it.
## "Without sacrificing accuracy"
That phrase is doing more work than the speedup is, and the release notes are
more careful than the model card.
On video object segmentation — mask-prompted tracking, no detector involved, so
this is a clean read on the tracker change alone — SAM 3.1 improves on **six of
seven** benchmarks. Meta says "six out of seven" and prints the table, so you
can find the seventh without asking.
The seventh is **YouTube-VOS 2019 val**, which goes 89.7 → 89.3, down 0.4 G.
It is the oldest and most saturated benchmark in the set, everything above 89 on
it is noise-adjacent, and trading 0.4 there for **+2.0 on MOSEv2** — the
occlusion-heavy one where SAM 3 only manages 60.3 — is a trade any practitioner
would take. The point is not that the regression matters. It is that a release
which reports the benchmark it lost on has earned the benefit of the doubt on
the ones it won.
The video-PCS table is where "without sacrificing accuracy" strains. Same
release, different column:
| benchmark | SAM 3 | SAM 3 Multiplex | SAM 3.1 |
|---|---|---|---|
| SA-Co/VEval · YT-Temporal-1B (cgF1) | 50.8 | 53.5 | 52.9 |
| SA-Co/VEval · SA-V (pHOTA) | 58.0 | 58.4 | 58.7 |
| LVVIS test (mAP) | 36.3 | **34.2** | **34.3** |
| BURST test (HOTA) | 44.5 | **43.1** | **43.3** |
| YTVIS21 val (mAP) | 57.4 | **56.3** | **56.6** |
| OVIS val (mAP) | 60.5 | 62.3 | 61.5 |
Three of the four public benchmarks go down — LVVIS by 2.0, BURST by 1.2,
YTVIS21 by 0.8 in the shipped checkpoint. The release notes' bullet reads
*"Mixed results on SA-Co/VEval video benchmarks, with notable improvement on
YT-Temporal-1B (+2.1 cgF1)"* — but SA-Co/VEval is the column where five of that
table's six numbers improved. The mixed column is the public one, and the bullet
does not mention it. The table does, which is the part that counts. So does the
paper's prose: *"results are mixed with improvements on OVIS (+1.8 mAP) but
slight regressions on LVVIS, BURST, and YTVIS21."*
Which way they move is not random, and it points at the mechanism. The two
biggest gains anywhere in the release are OVIS on this table (+1.8) and MOSEv2
on the VOS table (+2.0). OVIS is *Occluded* Video Instance Segmentation; MOSE
is the crowded, heavily-occluded successor to DAVIS. That is exactly where
sixteen objects sharing a memory would be expected to help — a tracker that can
see its neighbours is a tracker that can tell them apart when they overlap. The
losses
are less tidy: LVVIS and BURST are the large-vocabulary benchmarks, 1.2K and
482 noun phrases against OVIS's 25, which fits a story about concept coverage —
but YTVIS21 has only 40 noun phrases and drops too, so that story is incomplete.
Six deltas is not enough to name a cause.
The paper's Table 42 also distinguishes "SAM 3 Multiplex" (same detector as
SAM 3) from "SAM 3.1 Multiplex" (a newly trained detector). The shipped SAM 3.1
is both changes at once. The VOS table in the release notes is labelled
"SAM 3.1" but carries the paper's SAM-3-detector Multiplex numbers exactly —
79.6 / 92.7 / 89.2 / 83.8 / 85.1 / 89.3 / 62.3 — which is fine, since VOS never
runs the detector, but it means the "6 of 7" claim is about multiplexing alone
and the video-PCS claim is not.
## The shift that mattered more happened in November
Object Multiplex is a performance result. The thing SAM 3 did to the field was
a task change, and it is worth restating because SAM 3.1 inherits all of it
unchanged.
SAM 1 and SAM 2 took **geometric** prompts: click a point, draw a box, hand over
a mask, get that one object back. SAM 3 added **concept** prompts — an
open-vocabulary short noun phrase, or an image exemplar — and changed the return
type. You do not get an object; you get *every* instance matching the concept,
each with a stable identity across the video. Meta call the task promptable
concept segmentation, and the benchmark they built for it, SA-Co, carries 270K
unique concepts — "over 50 times more than existing benchmarks," by their count.
The detector is DETR, and not loosely: the architecture is "broadly based on the
SAM and (M)DETR series," the object queries cross-attend to prompt-conditioned
image features exactly as in [Carion et al.
2020](https://arxiv.org/abs/2005.12872), and the first author of the DETR paper
is a core contributor on SAM 3 — the BibTeX key is `carion2025sam3segmentconcepts`.
What text prompting adds to that skeleton is a fusion encoder that conditions
the image embeddings on the prompt tokens before the queries ever see them, so
"a penguin" is not a label the model picks from a list; it is a vector the image
features have already been bent around.
The piece with no ancestor in DETR or SAM 2 is the **presence token**. Asking
each object query to decide both *what* this is and *where* it is puts two
objectives in conflict — recognition wants global context, localisation wants
local, and the two pull against each other. SAM 3 factors
them: a single learned global token predicts `p(the noun phrase is present at
all)`, each query predicts `p(I am a match | it is present)`, and the final
score is the product. That is the whole trick for keeping "a player in white"
from firing on a player in red, and it is what makes exhaustive open-vocabulary
detection work at all.
The prompts stay short, deliberately. SAM 3 does not do *"the second to last
book from the right on the top shelf"* — for that you bolt on an MLLM that
proposes noun phrases and inspects the returned masks, which Meta ship as SAM 3
Agent. Concept prompts are a vocabulary problem, not a reasoning one, and the
model is honest about which it solves.
## What a minute costs
SAM 3.1 is on the Meta Model API at **\$2.50 per 1,000 images** and **\$0.20 per
1,000 frames**. The first thing that falls out is the ratio between them: an
image costs 12.5 frames. A single image runs the full detector on one frame with
no memory to amortise; a video frame gets the tracker's shared work spread
across it. The price sheet is describing the architecture.
A minute of 30 fps video is 1,800 frames, so it is **\$0.36**. An hour is
\$21.60. Those are flat — the price does not know how many objects are in the
frame.
The compute underneath is not flat.
**Receipts.** One minute of 30 fps video costs $0.36 on the Meta Model API no matter how many objects are in it. On a rented H100 the same minute costs $0.050 at 16 objects and $0.147 at 128 — but under the November SAM 3 it cost $1.06 at 128, three times what the API charges.
| objects | API / min | SAM 3.1 on H100 / min | SAM 3 Nov on H100 / min | API ÷ SAM 3.1 | API ÷ SAM 3 Nov | break-even GPU utilisation |
| ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 1 | $0.36 | $0.050 | $0.064 | 7.2× | 5.64× | 14% |
| 2 | $0.36 | $0.051 | $0.071 | 7.1× | 5.07× | 14% |
| 4 | $0.36 | $0.052 | $0.086 | 6.9× | 4.20× | 14% |
| 8 | $0.36 | $0.053 | $0.116 | 6.7× | 3.11× | 15% |
| 16 | $0.36 | $0.056 | $0.172 | 6.4× | 2.09× | 16% |
| 32 | $0.36 | $0.076 | $0.291 | 4.7× | 1.24× | 21% |
| 64 | $0.36 | $0.112 | $0.563 | 3.2× | 0.64× | 31% |
| 128 | $0.36 | $0.147 | $1.056 | 2.4× | 0.34× | 41% |
H100 rate is $3.38/GPU-hr, the median on-demand price across 39 providers reported by getdeploying.com on 2026-09-19; the cheapest in-stock listing that day was $1.25. Compute only — no storage, egress, idle time or engineering.
> method: API: $0.20 per 1,000 frames × 1,800 frames (60 s × 30 fps). Self-host: 1,800 ÷ fps seconds of one H100, billed at the median on-demand rate. Break-even utilisation = hourly rate ÷ the API spend one busy H100-hour would replace (fps × 3600 × $0.0002).
> source: https://dev.meta.ai/docs/pricing-rate-limits
> captured: 2026-09-19
> data: https://ai.thesatyajit.com/articles/sam-3-1-object-multiplex/data/economics.json (8 rows)
At the median on-demand H100 price — \$3.38/GPU-hr across 39 providers on the day
this was written, with the cheapest in-stock listing at \$1.25 — a minute of
sixteen-object video costs **5.6¢** of rented GPU to run yourself against the
API's 36¢. Even at 128 objects, where SAM 3.1 drops to 11.5 fps, it is 14.7¢
against 36¢. Self-hosting is 2.4× to 7.2× cheaper on compute across the entire
range, and the break-even utilisation is low: keep one H100 busy more than
**14%** of the time at low object counts, or **41%** at 128, and renting it beats
the API. That comparison is compute only, though — it prices none of the
engineering, the gated download, the cold starts, or the fact that when the
hosted endpoint falls over, it is not you being paged.
The more interesting arithmetic runs the other way. Under the November model,
128 objects took 1,125 seconds of one H100 per minute of video — **\$1.06** of
rented compute for something the API sells at \$0.36. Flat per-frame pricing on a
cost curve with a 4.6 ms/object slope is a bet that nobody sends you crowded
video. Object Multiplex takes that same minute to \$0.15.
I cannot prove those two facts are connected — Meta owns its hardware, its
marginal cost is not a rental rate, and nothing in the docs caps objects per
request. But the only segmentation model on the Meta Model API is SAM 3.1:
`developer.meta.com/ai/models/sam-3-1/` resolves and `.../sam-3/` returns 404,
and the price sheet's only segmentation row is headed "SAM 3.1." Whatever the
reason, the flat per-frame price arrived with the model that made flat per-frame
pricing survivable, and not before.
## Open weights, gated door
What "open" buys you here, precisely:
- **The checkpoint**, `sam3.1_multiplex.pt`, 3.50 GB. One file, PyTorch pickle.
Not safetensors.
- **Under the SAM License** — not Apache, not MIT. Commercial use is granted,
redistribution must carry the licence forward, attribution is required in
publications, and there is a use policy that excludes ITAR-controlled
applications, weapons, nuclear and espionage.
- **Gated `manual`.** You agree to share contact information with Meta and wait
for approval. `curl` on the README returns 401.
- **The training and fine-tuning code**, the SA-Co benchmark, and the
evaluation harness — which is the part that actually lets you check any of
the numbers above, and is why this article could.
And the thing the card says in its own words: *"This repository hosts only the
SAM 3.1 model checkpoints — **there is no Hugging Face Transformers
integration**."* SAM 3 ships `model.safetensors`, `library_name: transformers`,
an `AutoModel` mapping, and is deployable to Inference Endpoints and Azure.
SAM 3.1 ships a `.pt` and the instruction to go install `facebookresearch/sam3`
from source.
You can see what that costs in the download counts. Over the same trailing
thirty days, `facebook/sam3` pulled **2,086,256** downloads and carries 100
Spaces; `facebook/sam3.1` pulled **48,249** and carries 9. The faster model has
2.3% of the slower one's traffic. Meta calls SAM 3.1 "a drop-in replacement for
SAM 3," and for anyone already on the research repo it is. For the two million
downloads a month coming through `transformers`, it is a rewrite.
The API, meanwhile, is `transformers`-free by construction: text prompt in,
boxes and pixel-precise masks out, identities preserved across frames, no
weights, no fine-tuning, no approval form. Which is the actual trade. The
weights are open and inconvenient; the API is closed and immediate; and the
release that made the open path dramatically cheaper to run is the one that made
it harder to start.
## What I'd want measured
Meta published more of its own curve than most releases do — three lines, eight
points, the crossover, the benchmark it lost, and a separate line isolating
engineering from algorithm. Sorting this article's claims by how much they rest
on that: **reported** is everything in quotation marks and every benchmark
number, all of it from Meta's own release notes, paper appendix and price sheet.
**Measured** is the file sizes, the download counts, the `multiplex_count=16`
and `max_num_objects=128` read out of the shipped code, the two mask-downsampler
MAC counts computed from their constructors, and every ratio computed from the
printed labels on Meta's figure. **Reasoned** is the two-parameter cost model,
the claim that the gains cluster on occlusion benchmarks, the reading of where
multiplexing's saving actually lives, the parameter delta, and the suggestion
that the pricing and the speedup are related. Those five are below, each with
the experiment that would kill it.
This is Meta's own figure read two ways, and the paper agrees, so the way to
overturn it is to show the middle line is wrong: take the November checkpoint,
apply only the batched postprocessing, reduced CUDA sync and `torch.compile`
changes, and re-measure at 128 objects on an H100. If it lands near 1.6 fps
rather than 2.2, the optimizations contribute nothing, and the full 7.19× is
multiplexing after all.
This is a least-squares fit to the eight labelled points on Meta's figure, not
a published decomposition. Instrument the two pipelines and time the backbone
separately from the memory path at each object count. If the intercepts differ
by more than a couple of milliseconds, or if SAM 3.1's slope is not roughly a
tenth of SAM 3's, my model of where the time goes is wrong and the bar
decomposition in the diagram above should not be trusted.
This is arithmetic over two constructors, not a profile.
`SimpleMaskDownSampler` at `interpol_size=[1152, 1152]` gives 1.355 GMAC per
object for SAM 3's `1 → 4 → 16 → 64 → 256 → 256` channel schedule and 18.94 GMAC
per bucket for SAM 3.1's `32 → 16 → 64 → 256 → 1024 → 256`; sixteen of the former
is 21.7. Profile the two pipelines per module on an H100 and if the memory
encoder turns out to be where the time goes, this is wrong. MACs are not
milliseconds, and one 32-channel convolution has very different arithmetic
intensity from sixteen 1-channel ones.
Two benchmarks is thin evidence for a mechanism story, and I have no story at
all for why YTVIS21 drops. Partition SA-Co/VEval by objects-per-clip and by
occlusion rate and evaluate both checkpoints within each bin. If the gain is
flat across crowd density, "shared context helps crowded scenes" is decoration
and something else explains MOSEv2 and OVIS.
Measured from file sizes only: `sam3.1_multiplex.pt` is 3,502,755,717 bytes
against `sam3.pt`'s 3,450,062,241, and 52.7 MB at fp32 is about 13.2M weights
on top of the 859,922,360 that SAM 3's safetensors header reports. (Its README
says 848M; the header is what the header says.) I could not open the SAM 3.1
checkpoint — the repo is gated. Anyone with access can print the state dict and
settle it in one line; if the delta is mostly pickle overhead or a changed
tokenizer table, this claim is worth nothing.
The arithmetic is public — 1,800 frames ÷ 1.6 fps × the median H100 rate is
\$1.06 against a \$0.36 fare. What is not public is Meta's internal cost, or
whether the endpoint caps objects per request. A documented per-request object
cap, a serving stack that batches across tenants, or evidence that SAM 3 was
ever offered at this same flat rate and survived it, would each take the claim
apart. The 404 on the SAM 3 API page is suggestive, not proof; pages get
retired.