Jev alternatives, week five: decision models built for the shape of the decision

mdjsonmcpcopy↗ai

2026-10-15 · 22 min · ecosystem · benchmarks · calibration · kv-cache · retrieval · video-understanding · multimodal

Why read this

Notable· top 60%

Reads Jeva's and VideoJev's code, configs and headers: the 'zero flips' is guaranteed by construction, the 20x needs a full cache, and 100x is against Thinking.

• Original analysis
• Runs on a consumer GPU
• A lasting reference

·LLM architecture·Mixed licences·Practitioner roundup

How this was scored

Is it new?

2 of 3: A real new idea, method or capability

Can I trust it?

2 of 3: Measures key facts from files, code or configs

Can I run it?

2 of 3: Open code or weights with real limits

Will I understand it?

2 of 3: Mechanism from first principles with figures

Can I act on it?

2 of 3: A concrete recipe, numbers or comparison

Will it last?

2 of 3: A reference for a year or more

Does it affect many?

1 of 3: A specialist community

Only here?

2 of 3: A teardown or measurement few others did

Score 67 of 100, ranked 183 of 577 rated articles. Each question is answered 0–3 by hand, and a 3 is rare. How articles are scored

Week three was the most serious architectures with the thinnest evaluations. Week four was about where decision models run: a closed API, six open sizes, a Mac, a llama.cpp endpoint.

Week five is two arXiv papers, both posted on 13 October, asking what a decision model looks like when you design it around the decision instead of a prompt. Jeva, from the National University of Singapore, is a text model that treats its options as a set. VideoJev, from Nanjing University with JD.com and six other labs, scores moments in a video and answers to questions about it, and shares the video's cost across every question you ask.

Measured means I read it from a config, a safetensors header, the code, or a published output, or recomputed it from those; reported means the authors' figure, which I could not re-run; reasoned means my inference from the other two. Code was read, never executed. Checked on 2026-10-15.

The one idea, again

A decision model answers typed questions — choice, score, noul (yes/no) — by returning a probability for every option in one forward pass, with no tokens generated. The full definition is in what Jev actually is and why any model can be Jev.

The open ones so far sort into three readouts: a letter readout (options become letters, read the letters' next-token logits), a pointer (each option's end-token hidden state is scored against the question's, as in Kev) and a joint scorer (a trained head scores everything at once, as in clef). All three have one thing in common. The options are written into one prompt, in an order, and a causal backbone reads them left to right. Option three sees options one and two; option one sees nothing after it. Order is an input whether you meant it to be or not, which is why Jev is not deterministic could measure a 13% flip rate when the same options were shuffled.

VideoJev is a pointer, built for video. Jeva is something new on this site: a fourth family, closest to CLM, where every option is encoded alone.

Jeva: options as a set

The mechanism

Jeva-4B starts from Qwen3.5-4B. The request and each option go through the same backbone separately, each with a short template and no chat format (measured, from src/jeva/ and the paper's appendix):

Decision type: choice
Question: Which team should handle this?
State:
I was charged twice.

Option for choice: Billing
Option for choice: Technical support
Option for choice: Delivery

Each text is pooled at its last token and projected to 768 dimensions: one state vector $z_0$, and one unit vector $u_i$ per option. The option template holds no question, no state, no index and no other option, so an option's vector is the same in every request it appears in. That one decision buys two of the four properties at once.

Then one decision block lets the state read the options. Eight heads attend from the state to all options with a softmax at temperature $\tau_\ell$ (the paper's equation 3); the weighted message is added back to the state, and a SwiGLU branch refines it (equation 4). Only the state moves. The options are read, never updated. Scores are plain dot products, $s_i = z_L^\top u_i$, and a softmax at $\tau_{dec}$ turns them into the choice among the options.

Null is a separate question. A softmax over options always sums to one, so it cannot say "none of these". Jeva adds a coverage gate on the best score:

where $q$ is the softmax over options. Adding a weak option leaves $\rho$ unchanged, because it does not move the maximum. A flat softmax with a "None" option pays a $\log K$ penalty instead: the more options you list, the less likely null becomes (the paper's Proposition 2). noul reads the probability of a true option, and score is the expectation of rubric values, so all three Jev types come out of the same head.

expand

The released head is small, and the arithmetic checks (measured, from matrix_decision.py and model.py): two 2,560-to-768 projections are 3,932,160 parameters; the block's four 768-square projections 2,359,296; the SwiGLU at width 3,072 is 7,077,888; an RMSNorm 768; and five learned scalars. That is 13,370,117, the paper's figure exactly. The Hub's head.pt is 53,485,943 bytes, which is those parameters in FP32 plus a pickle wrapper (reasoned).

The rest of the Hub repo is less than it looks. backbone/ holds two shards whose SHA-256 hashes match Qwen/Qwen3.5-4B's (measured, from the Hub's LFS metadata): an unmodified copy. What Jeva trained is a rank-64 LoRA in a 519,510,856-byte adapter, merged at load, plus the head. The repo is gated, so I could not read the adapter's header.

Why a shuffle cannot move it

Reorder the options and the attention in the decision block sums the same terms in a different order; the state comes out the same; each option's score moves with its option. That is the paper's Proposition 3, and it needs no training to be true: nothing in the computation reads a slot. The tester below is a toy with hand-made vectors, not Jeva's weights. The left scorer reads cosine similarity plus an invented bonus per slot, standing in for any readout that sees an ordered list. The right one runs Jeva's equations with one block and one head.

For the double-charge request, the position-encoded toy picks Billing in 16 of the 24 orders and Account security in 8; the set-encoded toy picks Billing in all 24 (measured, from the widget's own arithmetic). Switch to a request no option fits and the coverage gate falls to 0.048, so the set-encoded toy answers "none of these"; the other one, with no way to say so, still names Delivery.

The block also does something a per-option scorer cannot: because the state reads the whole set first, swapping a third option can reverse the order of the first two (the paper's Proposition 1, a two-dimensional construction). That separates it from CLM-style late interaction on paper; how often it matters in practice is not measured (reasoned).

"Zero decision flips under tested permutations"

True, and the word "tested" is doing honest work. The test is 600 choice items from Typed Decisions, each run in its original order, reversed, and "up to three seeded random permutations", so at most five orders per item (reported, appendix C.3). Jeva records 0.00% flips and a mean total variation of $3.17 \times 10^{-8}$ between aligned distributions; its all-order accuracy is its accuracy, 76.67% (reported).

Two notes (reasoned). First, invariance is guaranteed by construction, so the test measures floating-point noise, about what FP32 sums in a different order should give. Second, the comparison row is softer than it looks. Jev-1.13.0 flips 17.00% of items in the same test, but Jev is not deterministic found that repeating an identical request already changes Jev's answers; part of any Jev flip rate is run-to-run noise, not order. And there is one documented hole: the README says "Exact ties use the first candidate", so on an exact tie the pick follows the order (measured). Ties in FP32 are rare; it is still not "permutation-invariant" without qualification.

"Handles 2,048 candidates"

Because each option is its own sequence, there is no joint context window to overflow and no fixed alphabet of letters to run out of. The paper tests this on a ToolRet-based diagnostic: 500 requests in six cohorts from 32 to 2,048 candidates, 83 to 84 queries each (reported). Over all of them Jeva gets 78.40%, against 64.00% for JevK5 run recursively; at 2,048 it gets 69.88% at a median 12.040 s per request, against JevK5's 67.47% at 28 s. Both InternDecision variants handle only 65 of 84 requests even at 32 candidates.

So "handles 2,048" holds, at twelve seconds a request on an H200. The post's companion "over 2x" speedup in median latency is that 12.040 s against JevK5's 28 s (reasoned: 2.33x). The bigger number in the same section, 7.85x, is Jeva against itself: length-aware batching of 256 rows against the old two-at-a-time encoding, 94.459 s down to 12.040 s.

"20x latency speedups via reuse"

The option memory is the third property. Since an option's vector and its per-layer keys and values depend only on the option, they can be cached. The runtime has two caches (measured, inference.py): a set cache keyed on the ordered tuple of options, and a hybrid cache that also stores options one by one, so a new combination of familiar options can be assembled from parts. The answer is never cached; the request is always encoded and the block always runs.

expand

The 20x is MLDR, a long-document retrieval set, and the measured ratio is higher than the post says: with every option of a novel set already cached, median latency falls from 2,102.0 ms to 75.1 ms, 27.98x, and the median paired speedup is 27.86x (reported). Three conditions sit under it. The options are long documents, so encoding them is nearly all of the cost. Reuse is 100%: at about 50% reuse the speedup is 1.95x. And the priming run that filled the cache is "timed separately and excluded". On Typed Decisions, with short options, the shared-bank median goes from 93.1 ms to 46.6 ms, about 2x. No decision changed across cache conditions; the largest probability difference was 0.00736 (reported).

The calculator draws a straight line between the paper's two MLDR points: a fixed 75.1 ms that never caches, plus 2,026.9 ms of option encoding that shrinks with reuse. That line predicts 1.93x at half reuse, against the measured 1.95x, so it is a fair model (reasoned). It also shows how steep the curve is. At 90% reuse the line gives 7.57x; at 95%, 11.91x; it crosses 20x only above about 98.5%. For short options the ceiling is about 2x, because the fixed part is half the cost. The 20x is real for a stable registry of long documents or tool descriptions that many requests draw from. It is not a property of decision requests in general.

The scoreboard

The paper's main table, first-argmax accuracy in percent, including a null outcome where the request has one (reported):

Jeva leads seven of the eight slices; InternDecision-4B keeps Typed Decisions by 0.4 points. Two things to hold against the big margins (reasoned). On BANKING77 and MASSIVE, some requests have their answer withheld, and the baselines, which have no null output, were given an auxiliary yes/no question ("Determine whether at least one of the following candidates correctly applies") whose probability scales their choice. That is a fair adapter, but it is the authors' protocol, not the baselines' own. And the training curriculum lists "intent recognition" and "typed decisions" among its stages without naming datasets, so I cannot tell whether the BANKING77, MASSIVE or Typed Decisions train splits were in it; nothing in the repository says. The null-detection number has the same caveat: 98.66% AUROC on 3,080 BANKING77 requests, 588 with no answer, against 93.61% for Kev and 84.27% for InternDecision-4B (reported).

On noul and score, Jeva's Brier of 0.0184 against Jev's 0.0342, and score MAE 0.2315 against 0.3744, are on 600 and 800 Typed Decisions items (reported).

Licence. The code is Apache-2.0. The gated weights have no licence field and no LICENSE file; the README says only that "Model weights and datasets retain their respective licenses" (measured). The adapter and head have no stated terms.

VideoJev: one video prefix, many decisions

The mechanism

VideoJev takes Qwen3.5-4B, and Qwen3.6-35B-A3B for the large model, and replaces the language-model head with a scorer (measured, from modeling_videojev.py and config.json). Two new tokens, <|option_end|> and <|decide|>, mark where to read: one after every candidate, one after the list. Two linear projections map those hidden states to 512 dimensions, normalise them, and score

with $u$ the decision readout, $v_i$ a candidate, and a fixed scale of 10. The final transformer layer's MLP is deleted: the code swaps the last decoder layer for an attention-only one, following an earlier finding that pre-MLP features carry similarity better. The ablation says removing it costs nothing (0.2 points better on both task means, reported).

This is the pointer family: video first, then the instruction, the query, and the candidate list in one causal sequence. Two kinds of candidate share the head:

• Answer options for multiple-choice questions, normalised by softmax.
• Time intervals for grounding. The backbone already merges frames in pairs, so each frame pair becomes one candidate, written as its timestamp text. Each gets an independent sigmoid, "is this moment part of the event?", and a span is decoded by finding the contiguous run with the largest sum of logits shifted by a fixed threshold of 0.25 (Kadane's algorithm, linear in the number of intervals).

The card's own example checks against that rule (measured, recomputed from its printed output). A 10-second clip at 2 fps gives ten interval probabilities. The fourth through ninth are above 0.83; the tenth is 0.330, which is above 0.25, so it joins the span; the third is 0.026, which does not. The decoded interval is 3.0 to 10.0 seconds, as printed.

expand

Parameter counts (measured, from the safetensors headers by range request): VideoJev-4B is 4,471,106,561 parameters in BF16. Against Qwen3.5-4B's 4,659,865,088, the 120,599,552-parameter multi-token-prediction layer is gone, so is the last layer's MLP (70,781,440 with its norm), and the head adds 2,622,465. VideoJev-35B-A3B is 33,791,740,273, of which 31,406,948,352 are experts; its head is 2,098,177. The language backbone is fully fine-tuned; the vision encoder stays frozen.

What "interaction" means

The title promises "Video Understanding and Interaction". What the paper builds is cheaper repeated questions about one video, two ways (reported, section 2.1):

• Cached inference encodes and prefills the video once, then runs each question as its own short suffix.
• Joint inference puts all the questions after one video prefix and reads every <|decide|> in a single forward pass. Later questions can attend to earlier ones (no answers are inserted), because custom masks would lose FlashAttention.

Joint is the default for every headline number, and it scored slightly better than Cached in 13 of 14 model-and-benchmark pairs. At ten questions per video, joint forward time is 0.39 s for grounding and 1.64 s for QA, against 0.85 s and 2.17 s cached (reported). That is the "interaction": asking about the same footage again costs little. No agent loop is evaluated; that is listed as future work (measured, from the paper).

One consequence of the pointer design, which the paper handles and does not measure (reasoned): option order is an input again. Each candidate sees the ones before it, and in joint mode each question sees the questions before it. Training applies a fixed random permutation to each question's options to discourage position shortcuts; inference "keep[s] the supplied order". No order-sensitivity test is reported, so there is no VideoJev number to set beside Jeva's zero.

The numbers, and where it loses

Grounding is the clean win. Mean mIoU over Charades-STA, ActivityNet Captions and QVHighlights (TimeLens-refined annotations) is 65.3 for VideoJev-4B and 67.7 for 35B-A3B, 6.7 and 10.8 points over their backbones, and 3.3 points over TimeLens2-4B, the strongest 4B baseline (reported). Gemini-2.5-Pro's mean is 60.4 in the same table. Per-query latency on all of Charades is 0.276 s and 0.336 s against the backbones' 0.763 s and 1.054 s: the 2.8x and 3.1x in the abstract.

Multiple-choice QA is closer, and the speedup needs reading:

(reported, Table 2; latency is per question on all of LVBench, one H200.)

• "Over 100x" is against Thinking mode: 129x and 100x. Against the Instruct backbones, which answer with one letter, VideoJev is 1.21x and 1.59x faster (reasoned, from the table). The real gain over Instruct is accuracy: 8.0 and 5.7 points of mean.
• It loses to Thinking on Video-MME and MLVU at both sizes, and on the 4B mean (74.3 against 74.9). It wins LVBench at both sizes.
• It loses to the closed models where they report: Gemini-3-Pro's 87.7 on Video-MME and 76.2 on LVBench, Seed2.0-Pro's 76.4 on LVBench.
• The architecture helps a little, not a lot. Fine-tuning plain Qwen3.5-4B to generate answers on exactly the same data lands 1.3 points behind on grounding mIoU and 1.9 behind on QA (reported, Table 3). Most of the gain over the backbone is the data; the head adds about two points.
• It is overconfident. Mean ECE over the four QA benchmarks is 8.65 points for 4B and 8.13 for 35B-A3B (reported). The paper says so directly and notes that Jev trains for calibration and VideoJev does not. Confidence still ranks well: on LVBench, the 591 answers at 99% confidence or more are 95.8% right, against 73.4% overall.

expand

The QA training set is "an internally curated corpus of approximately 240K multiple-choice questions over 19K videos" (reported), and it is not in the repository (measured). By the paper's own control, that is the ingredient that matters most, and you cannot inspect it.

Licence. Both checkpoints are Apache-2.0, with a LICENSE and a NOTICE crediting the Qwen bases, and the code at MCG-NJU/VideoJev is Apache-2.0 (measured).

Against the Jev definition

Both deliver the core of a System One model: a probability per option, read in one pass, nothing to parse. Neither speaks Jev's wire format (reasoned, from the code). Jeva covers choice, noul and score through a system_one() call much like Jev's, and adds the output Jev never had, an explicit null; the repository has no HTTP server, and llama.cpp's six readouts have no type for a set reader. VideoJev has choice and a new type, a time span, and no noul or score.

The bigger gap is calibration, which is Jev's whole pitch. Jeva reports Brier and AUROC but no ECE; VideoJev reports an ECE and says it is too high. Neither ships a fitted temperature.

If you are choosing this week

All reasoned:

• Routing over a big, stable catalogue (tools, policies, documents): Jeva's design is the right shape. Option vectors cache, there is no option limit short of memory, null is a real output, and order cannot move the answer. Check the licence on the weights first, and measure the speedup on your own reuse rate; below 90% it is single digits.
• Short options, a few at a time: the cache buys about 2x and the edge over Jev on Typed Decisions is 4.2 points, on the authors' run. Worth a test.
• Questions about video: VideoJev-4B for grounding is the clearest result of the week. For QA, it is Instruct latency with near-Thinking accuracy, but fit your own temperature before you threshold on its confidence.

The take

Jeva gets permutation invariance, a null answer and an option cache from one choice, encoding options apart, and proves them rather than training for them. That is why its "zero flips" is unremarkable once you read the code: it could not have been anything else. Its "20x" is the honest-but-best case of a cache, long documents fully reused. VideoJev's contribution is plainer and more immediately useful: pay for the video once, score time like you score answers, and stop generating timestamps.

Neither published a calibrated model. The architectures are moving faster than the evidence, again.

Sources, read on 2026-10-15: the @HuggingPapers post and its thread; Jeva: System One Models from First Principles (arXiv 2610.18773v1, HTML); LIQIIIII/Jeva-4B (card and file listing with LFS hashes; files gated); LiQiiiii/Jeva at f231c9a (src/jeva/_runtime/, inference.py, download.py, README, LICENSE, NOTICE); the project page; the @_akhaliq post and its replies; VideoJev: System One Model for Video Understanding and Interaction (arXiv 2610.18615v1, HTML); ZhuYuhan/VideoJev-4B and ZhuYuhan/VideoJev-35B-A3B (cards, config.json, modeling_videojev.py, validation.json, NOTICE, safetensors headers by range request); MCG-NJU/VideoJev at 9fc432c; and Qwen/Qwen3.5-4B headers for the comparison. No weights were downloaded and no third-party code was executed. Figures are reproduced for commentary from the two papers (CC BY 4.0); both interactives are original, and the permutation tester is a toy with invented vectors.