Pokee-Isaac 28B: 10M tokens on one GPU, and an architecture the report never explains

Pokee AI's technical report makes two claims about Pokee-Isaac 28B. The specific one: a 28-billion-parameter, non-decoder-only model that holds retrieval fidelity across 10 million tokens, running on a single NVIDIA B200 at up to 137,200 tokens/s prefill and 335 tokens/s decode, scoring 93.3% on RULER at that length, and priced at $0.15 /$1.00 per million input/output tokens. The general one, stated right in the abstract: long-context agentic capability has been cloud-only because of infrastructure cost, which locks it out of regulated industries, the public sector, and anywhere data can't leave the building — and a model this small changes that.

The general claim is worth taking seriously. The specific one is where a close read gets uncomfortable: the report names its architecture "non-decoder-only" twice, in the abstract and the introduction, and then never says what that means anywhere in nineteen pages. Take both in turn.

"Non-decoder-only": the claim the report doesn't explain

Here is the entirety of what the report says about its own architecture. From the abstract: "Pokee-Isaac 28B is a non-decoder-only foundation model that reasons, plans, and uses tools." From the introduction: "we introduce Pokee-Isaac 28B, a 10M-token context window, non-decoder-only model engineered to operate within a compact compute budget." That's it. Those two sentences are the complete architectural disclosure. There is no architecture section, no diagram, no equation, no named mechanism, no ablation isolating what the non-decoder component contributes. The word "decoder" does not appear again anywhere else in the document.

That absence is conspicuous by comparison. Kimi K3's technical report spends its first several sections laying out Kimi Delta Attention's exact recurrence, ships the kernel that implements it, and publishes a config.json that pins down which of its 93 layers are which. KDA's decay mechanism is something a reader can derive and check independently precisely because Moonshot wrote the state-update equation down. Pokee's report has the same page budget and spends none of it there — it moves directly from the abstract to evaluation tables.

One passage nearby is the closest thing to a hint, and it complicates the claim rather than supporting it: "Some weights in Pokee-Isaac are fine-tuned from Qwen3.6-27B (Apache 2.0)." Qwen3.6-27B is, per its own citation in the same report, a conventional 27B dense model — ordinary decoder-only transformer. Isaac is a 28B dense model. The arithmetic sits right there: a ~1B-parameter gap between a decoder-only base and a model described as non-decoder-only. That is not evidence of anything specific — it could be a new module bolted onto an inherited decoder backbone, a retrieval or state component, a modified embedding or head, or something else — and the report gives no way to distinguish between those. I'm flagging the arithmetic because it's the one concrete data point available, not because it resolves the question. It doesn't.

So: what actually gives Isaac its 10M-token window and makes it "non-decoder-only" is not something this report lets a reader verify. That is a real gap, not a stylistic one — it's the single most technically interesting claim in the paper, and it's asserted rather than shown. Treat everything below as evaluation of what Isaac does, because that's what the report actually lets you check; how it does it stays a closed question.

What the retrieval numbers say

Whatever the mechanism, the report does back the context claim with two established long-context benchmarks, run against five named baselines: GPT-5.6 Luna, Gemini 3.5 Flash Lite, and Claude Haiku 4.5 (the cost-optimized tier of the three big cloud providers), plus Nemotron 3 Super 120B and Qwen 3.5 122B (open-weight models an organization can self-host). Frontier flagships — GPT-5.6 Sol, Claude Opus 5, Gemini 3.1 Pro — are explicitly excluded as costing roughly an order of magnitude more per token and addressing a different deployment envelope. That's a defensible exclusion, but worth naming: Isaac isn't compared against the actual frontier, only against the cheap tier of it.

expand

The RULER protocol is NVIDIA's own official pipeline: 10 samples per task configuration, 13 configurations at 256K and 512K, and — since common-words extraction needs a small fixed vocabulary that stops being meaningful past 1M tokens — 12 configurations from 1M onward. Isaac's own scores across the sweep are 96.9 / 96.7 / 95.0 / 95.8 / 96.7 / 93.3 at 256K/512K/1M/2M/4M/10M. Read that sequence closely and it isn't a smooth decay curve — it dips at 1M, recovers at 4M, then drops again at 10M. At 10 samples per configuration that's within the noise you'd expect, not a story about Isaac getting worse and then better with more context, but it does mean 93.3% at 10M is a fairly small-sample number, not a tight measurement.

Every other baseline falls off a cliff. GPT-5.6 Luna and Gemini 3.5 Flash Lite track Isaac closely through 512K, then hit context-overflow errors at 1M and score zero from there — they simply can't be run at that length, which the report scores as a failure rather than excusing. Claude Haiku 4.5 and Qwen 3.5 122B score zero across the entire sweep, consistent with their native windows (200K and 262K) being smaller than even the first column tested. Nemotron 3 Super 120B is the one row worth reading carefully: the 256K–512K–1M figures in the table are NVIDIA's own self-reported numbers, not Pokee's measurement — marked with a superscript s and dashed in the chart — while the 2M-onward zeros are Pokee's direct measurement. Mixing a vendor's self-reported numbers into one row of your own comparison table, clearly labeled, is honest; it's still worth noticing when you're reading the row, since it isn't measured the same way as the rest of the table.

On MRCR v2 — a harder multi-needle variant that distributes several targets through a long synthetic conversation rather than one — Isaac leads throughout: 0.607 / 0.743 / 0.500 at 256K/512K/1M (again non-monotonic — it peaks at 512K, not 256K). Gemini 3.5 Flash Lite is the closest competitor and the gap widens with length, from a 0.133 margin at 256K to 0.295 at 1M. GPT-5.6 Luna collapses to 0.050 at 1M despite scoring 95.0% on RULER at 256K — a reminder that single-needle retrieval and multi-needle disambiguation measure genuinely different failure modes, and a model can be strong at one and weak at the other.

The comparison this site already has an anchor for

The most natural comparison for a 10M-token claim is Kimi K3, the largest context window documented on this site until now: 1M tokens on a 2.78-trillion-parameter open model. Isaac's framing — implicit in the numbers, not stated by Pokee this directly — is 10× the context at roughly 1% of the parameters. That ratio is real arithmetic. It is not, however, a like-for-like measurement:

K3's 1M is what Moonshot trained it up to; Isaac's 10M is a RULER score Pokee measured at that length. Those are different kinds of number — one a training-curriculum endpoint, the other a benchmark result — and neither the report nor this piece can turn them into a single fair ratio. What the comparison can support is narrower and still notable: a 28B dense model holding measured retrieval accuracy at a context length ten times past where a 2.8T model's training stopped. Whether Isaac would still say 93% if someone ran RULER on it at 20M or 50M tokens is not something either report answers.

Why 137,200 tokens/s and 335 tokens/s are both true

The efficiency section (Table 8, on a single B200-class GPU under the RULER workload) reports time-to-first-token directly and derives prefill throughput from it — context length divided by TTFT. Decode throughput is reported separately and holds close to flat regardless of context length:

The two numbers describe different bottlenecks: prefill processes the whole prompt as one large matrix-matrix multiply and is compute-bound, so throughput scales with how much parallel work is available — which is why it actually rises with context length, from ~42K tokens/s at 1M to 137K at 10M. Decode generates one token at a time against an already-populated cache, a matrix-vector operation gated by memory bandwidth rather than arithmetic, so it doesn't get faster no matter how much context is resident — 335 tokens/s at 1M, 337 at 10M, 322 under four-way concurrency. The report draws out the one number worth remembering: a 10× jump in context costs about 3× the time-to-first-token (23.6s → 72.9s), not 10× — which is the behavior that makes a 10M window usable rather than merely addressable. A full 10M-token prefill landing its first output token at 72.9 seconds is a real number to plan around, not an abstraction.

The agentic benchmarks: where "matches or exceeds" holds and where it doesn't

The abstract's claim is specific: Isaac "matches or exceeds the strongest cost-optimized cloud systems on function calling, multi-turn interactive execution, tool orchestration, and terminal work." Four categories, four benchmarks. Worth checking each against the report's own tables, because they don't all say the same thing.

Function calling — BFCL v4. The Berkeley Function-Calling Leaderboard, programmatically scored throughout (no LLM judge), combining five components under fixed weights (0.40 agentic + 0.30 multi-turn + 0.10 live + 0.10 non-live + 0.10 hallucination) over 5,106 scored entries. Isaac leads the panel at 70.94, just ahead of GPT-5.6 Luna's 70.61. The report itself calls this "parity rather than a decisive lead," which is the right read of a 0.33-point gap — and it's a fair characterization to give credit for. This category holds up.

Multi-turn interactive execution — τ³-bench. Sierra's benchmark runs an agent against an LLM-simulated user across four domains, verified by a five-criteria rubric rather than an LLM judge's opinion of fluency. Isaac leads the four-domain average at 0.662, but that average hides two domains where it doesn't win:

Qwen 3.5 122B edges Isaac on telecom (0.947 vs. 0.912), and Gemini edges it on banking (0.203 vs. 0.186) — the domain the report itself calls "by a wide margin the hardest," where the policy an agent needs lives across 698 documents rather than the prompt. Isaac's own banking score, 18.6%, sits below the 25.5% pass@1 the report cites as the strongest previously reported result on that domain. This category holds up on average, not on every domain.

Tool orchestration — MCP-Atlas. This is the one built specifically to avoid mock tool surfaces: 500 tasks against a live 36-server sandbox of real production MCP servers (GitHub, Slack, Google Workspace, Notion, and more), scored by mean claim coverage under a shared judge. It's also the one category where the abstract's claim doesn't survive contact with the table:

Isaac places third of six, behind both GPT-5.6 Luna and Gemini 3.5 Flash Lite — two of the three cost-optimized cloud systems named in the abstract's own comparison panel. The report's honest mitigating point is efficiency, not score: Isaac reaches within 2.1 points of Gemini using 9.10 tool-call turns against Gemini's 14.99, about 60% of the trajectory length for comparable coverage. That's a genuine and worth-stating efficiency result. It is a different claim from "matches or exceeds," and on the benchmark built to be hardest to game, the report's own number doesn't back the headline phrase.

Terminal work — Terminal-Bench 2.1. An agent at a bare command line with no enumerated action space, every model driven by the same harness (Harbor 0.20.0 with Terminus-2), evaluated on the 86 text-compatible tasks of the 89-task suite:

Second of six, four tasks behind GPT-5.6 Luna, well clear of everyone else including two open-weight models an order of magnitude larger. The report is direct about this one: "Terminal-Bench is the one benchmark in this report where a cloud baseline finishes ahead of Isaac, and we report it as measured" — which is honest as far as it goes, but reads oddly next to MCP-Atlas two sections earlier, where Isaac trails not one but two cloud baselines. Calling Terminal-Bench "the one" undersells what MCP-Atlas already showed.

So: of the four categories in the headline claim, function calling holds up as genuine parity, multi-turn execution holds up on average but not on every domain, and tool orchestration and terminal work both show Isaac behind at least one of the named cost-optimized cloud systems — behind two of them on the benchmark specifically designed to be hardest to inflate. "Matches or exceeds" is a fair summary of roughly half the evidence and an optimistic gloss on the rest.

Security: safest on attacks, third on capability

Isaac is evaluated on DTAP, a red-teaming benchmark measuring attack success rate (ASR, lower is safer) and benign task success rate (BSR, higher is better) across 12 Linux-Docker domains and 6,195 judged tasks. Isaac is the safest of the six models on both direct and indirect attack rates and their combination — 35.6% combined ASR against a range of 37.9% to 66.3% for the rest — and shows the tightest balance between direct and indirect attacks (0.8 points), where models with less refusal training swing 13 to 38 points toward direct attacks specifically. On capability (BSR), though, Isaac places third: 82.5%, against 85.1% for GPT-5.6 Luna and 83.3% for Gemini 3.5 Flash Lite — thin margins, but not a win. And there's a footnote worth reading rather than skipping: "the five baselines were run under the benchmark's stock runner; Isaac was run under the Pokee harness, which is the one condition that still differs across rows." A safety benchmark is exactly the place where the evaluation harness itself matters, and this one isn't held constant.

The deployment argument, taken on its own terms

Strip away the specific benchmark rows and there's a real argument underneath this report, and it's the one I'd give the most weight to. Long-context agentic capability today is delivered almost entirely from the cloud, because the infrastructure to serve it any other way has been expensive. That forecloses the option entirely for organizations that can't send data across a boundary at all — regulated industries, public-sector deployments, on-device applications — not because of price, but because the data isn't allowed to leave. A model that holds long-context agentic capability at 28B dense parameters changes what's possible to deploy inside that boundary, independent of whether it's the best model available outside it.

The pricing table backs a narrower, more concrete version of this point better than the headline "$0.15/$1.00 beats everyone" framing does. Of the five baselines, only two — GPT-5.6 Luna and Gemini 3.5 Flash Lite — can actually be bought at the context lengths this report tests. Claude Haiku 4.5 caps at 200K, Qwen 3.5 122B at 262K, and Nemotron 3 Super 120B's public endpoints all cap at 262K despite a 1M native window — so three of five baselines simply aren't commercially available at long context, at any price. Against the two that are, Isaac is cheaper on both meters ($0.25 and$0.80 below Luna on input/output; $0.15 and$1.50 below Gemini) while covering an order of magnitude more context. That's a real and checkable comparison, distinct from the sovereignty argument, and it holds up on its own — though it's list pricing marked provisional and subject to confirmation at launch, so treat the exact numbers as directional rather than final.

The product page at console.pokee.ai fills in what the report doesn't need to say: an OpenAI-compatible endpoint at api.pokee.ai/v1/chat/completions, streaming over SSE with a background mode that survives a disconnect, and three concrete deployment tiers — a single B200-class GPU for datacenter serving, a single consumer RTX 4090 or 5090 for a private workstation, and Qualcomm or Intel Panther Lake NPU silicon for on-device edge inference. None of that page explains the architecture either — it repeats "purpose-built agentic architecture" without elaborating, which is consistent with the report rather than a missed opportunity to say more.

Whether this specific model earns that framing is exactly what the benchmark section above complicates. But the underlying argument — that long-context agentic capability has had no in-boundary path at all, not merely an expensive one — is real, underserved by the current market, and worth taking seriously as a category even while staying skeptical of any one vendor's report about their own entry into it.

Portability, and what's still thin

Beyond the B200 numbers, Pokee reports adaptation to client and edge silicon. On an Intel Arc Pro B70, their own serving stack reaches 1,087–1,500 tokens/s prefill against 305 tokens/s for stock llama.cpp on the same hardware — a 3.6–5× gain — and 58.8 tokens/s decode against 25.7, a 2.3× gain (with a note that a 90 tokens/s decode path is still "in development," meaning the shipped number is below their own internal target). On a 12-core Xe3 Panther Lake SoC, fully on-device with no discrete GPU: 150.7 tokens/s prefill, 22.84 decode. On a Snapdragon X2 Elite: 124.95 prefill, 23.54 decode. AMD support is listed as in progress.

One more benchmark result is worth flagging for provenance rather than dismissing: on something called the "Pinchbench 116-task SuperClaw suite," Isaac scores 0.9567 against 0.929 for a cloud-hosted 744B model and 0.866 for an 80B local model. Unlike RULER, MRCR, BFCL, τ³-bench, MCP-Atlas, and Terminal-Bench — every one of which is independently authored and citable — this suite has no citation, no public description, and no other appearance I could find outside this report. That doesn't make the number false. It means it can't be checked the way the rest of this report's benchmarks can, and it shouldn't carry the same weight in your own read of the model.

What I'd want before trusting this further

Take the report's own framing at face value on one thing: everything here is Pokee's measurement, on Pokee's infrastructure, reported by Pokee, with no third-party replication and no released weights or config to check independently — a gap the report is upfront about, but a gap all the same. Specific to this piece, five things I couldn't verify or that need a second source:

• The architecture. Nothing in the report or the product page says what makes Isaac "non-decoder-only" or how the 10M window is achieved. This is the biggest open question in the whole document, and it stays open here too — I'd rather say that plainly than guess at a mechanism the source doesn't support.
• The MCP-Atlas and Terminal-Bench placements, where Isaac trails one or two of the exact cost-optimized cloud systems the abstract claims it matches or exceeds.
• The DTAP harness difference — Isaac run under Pokee's own harness against five baselines run under the benchmark's stock runner, on the one evaluation most directly about safety.
• The "Pinchbench" result, which has no independent citation or description to check it against.
• Pricing and general availability. Rates are explicitly provisional, and neither the report nor the product page states a launch date or confirms the model is generally available yet.

The take

The mechanism claim doesn't survive scrutiny, because there's nothing offered to scrutinize — "non-decoder-only" is asserted twice and explained zero times, in a report that had the exact same page budget Moonshot used to write down KDA's recurrence in full. The benchmark claim survives partially: real parity on function calling, a real average lead on multi-turn tasks that hides two domain losses, and two categories — tool orchestration and terminal work, including the one benchmark built specifically to resist gaming — where Isaac trails cost-optimized cloud systems by the report's own numbers, not a critic's.

What does survive, and what I'd actually flag as the interesting part of this report, is the deployment argument underneath all of it. Long-context agentic capability being cloud-only is a real constraint today, and it genuinely does foreclose entire categories of deployment — not because of price, but because data can't leave a boundary at all. A 28B dense model that holds measured retrieval accuracy for even a fraction of a 10M-token claim, running on hardware small enough to sit in a private workstation, is a meaningfully different option than what existed before — regardless of whether this particular model, from this particular report, is the one that delivers it best. That argument deserves to be taken on its merits. This report, on its own, isn't yet the evidence that settles it.

Sources: the Pokee-Isaac 28B technical report (all benchmark tables, the efficiency and pricing profile, and the two architecture sentences quoted in full above), and console.pokee.ai/model (API details, deployment tiers, pricing display). Figure 1 here is the report's own Figure 1, cropped from the source PDF and flattened onto white for legibility in both themes — no relabeling. RULER, MRCR v2, BFCL v4, τ³-bench, MCP-Atlas, Terminal-Bench 2.1, and DTAP are each independently authored benchmarks cited in the report; none of the scores above are this site's own measurement. The context-length and prefill/decode interactives are mine, built entirely from numbers in the report's own tables — no extrapolated or simulated figures appear in either.