What the Jev ecosystem actually built

In the nine days after TypeSafe's September 15 launch I count at least fifteen independent projects that shipped code against Jev — a browser agent, a Mac agent, an Android agent, a Snake AI, a trading bot, a code reviewer, a per-turn model router, a log triager, a Home Assistant integration, a web search engine, an MCP server, a Rust dataset filter, and a from-scratch reimplementation of the scoring head itself. Two companion pieces on this site already covered the product (Jev's receipts, itemized) and the mechanism (Parallel constrained decoding); this one does not re-argue either. I'm not re-running the pricing arithmetic or re-deriving the seven-step decode trick — read those for that. What follows is a census: I cloned every repo named in the brief for this piece, added everything else Anil-matcha/awesome-jev-by-typesafe's own 609-line community index names as a "recent independent implementation" or a "community implementation" under its use-case map, and read the actual source — not just the README — of fourteen of them.

The roster below is exactly that: the index, plus the announcements, plus what I could clone. It is not an exhaustive census. GitHub-wide code search isn't something I can run from here, so the discovery path is other people's link collections, and a project nobody has listed yet is invisible to this piece by construction. One repo named in my brief, vlad-terin/jev-browser, returned 404 on direct git access and on every URL under its own domain — a secondhand description exists (a community search index, a third-party mirror), but I could not read its code myself, so it's flagged as unverified below rather than written up as if I had.

One thing worth settling before anything else: Jev is reachable through Vercel's AI Gateway as typesafe-ai/jev, with no waitlist mentioned anywhere on its own model page, at the same $0.042/M input price TypeSafe quotes directly. Three independent sources agree on that model ID from three different angles — Vercel's own listing, jev-mcp's provider-detection code ("With AI_GATEWAY_API_KEY set, judgments run through the Vercel AI Gateway at typesafe-ai/jev"), and jev-trader's dependency on the @ai-sdk/typesafe-ai package. That closes the access question this piece needed closed before writing another word: getting a decision back from Jev, from a fresh npm or pip project today, does not require anyone's approval.

The first real calibration evidence in this story

Both companion pieces spent real space on one open question: TypeSafe names a training method — RLCD — that nobody outside the company can run, and no published ECE, Brier score, or reliability diagram exists anywhere to check its calibration claim against. The mechanism piece's own proposal for the cheapest available fix was temperature scaling: keep every logit exactly as computed, fit one scalar that divides them before the softmax, and report ECE before and after on a held-out split.

Nine days after launch, pngwn/open-jev did exactly that, in public, with numbers anyone can check.

The model is a LoRA adapter (r=16, 30,476,800 trainable parameters of 4,236,230,656 total) over Qwen/Qwen3.5-4B-Base, trained for two epochs on 45,932 questions from a companion dataset (pngwn/system-one-decisions), with a scalar scoring head bolted on (modules_to_save: ["score", "classifier"]). It's a Choice/Score/Noul scorer in the mechanism piece's exact sense — one forward pass, softmax over the candidate rows, nothing generated. What's new is the committed metrics.json: validation accuracy, ECE, and Brier score, computed twice on the same 5,087 held-out questions across seven task categories — once on raw logits, once after dividing every logit by a temperature (T = 2.35) fit on a separate validation split.

The aggregate move is real and in the direction the mechanism piece predicted: ECE from 7.3% to 2.2%, at the cost of mean confidence dropping from 87.5% to 78.2% while accuracy sits fixed at 80.3% — because dividing every logit by the same constant cannot change which answer wins, only how sharp the winning probability looks. That's temperature scaling doing exactly the one thing it can do, measured rather than asserted. The honest complication is in the per-task breakdown: six of seven categories start overconfident at T = 1, but one global temperature is a single knob turning seven different dials, and severity gets over-corrected — its own ECE moves from 3.8% to 12.6%, worse than doing nothing. A single scalar buys a real aggregate win and can still make a specific slice of your traffic worse. That's not a flaw unique to this model; it's the actual shape of the tradeoff any single-temperature fix makes, stated here for the first time with numbers instead of a guess.

A separate write-up on the same account's demo Space measured the scorer against 3,219 real decisions about 500 held-out Gradio pull requests — kind of change, risk, review effort, breaking, security, and eight more typed questions per PR — and got 0.740 accuracy against a 0.508 majority-vote baseline, ECE 0.047, with the published temperature transferring cleanly (a refit on this new data landed at 1.867, close to the original 2.35). But the write-up reports the complication just as plainly: the untouched base model, read through letter logits with no fine-tuning at all, is just as good on the judgement questions and better above 2,000 tokens, because the fine-tune's advantage ends exactly where its 384-token training sequences do. That's the same shape openjev found for Jev itself in the product piece — an untrained model closing most of the gap to a trained one — showing up a second time, on a different model, for a different reason: not undisclosed training data, but a training-length ceiling stated in the model's own card.

# spaces/pngwn/open-jev/app.py — temperature scaling and the softmax it feeds
def distributions(qs, branches, scores):
    out = []
    for qi, q in enumerate(qs):
        logits = [s for b, s in zip(branches, scores) if b["q"] == qi]
        out.append(softmax([x / TEMPERATURE for x in logits]))
    return out
 
 
def softmax(xs):
    m = max(xs)
    es = [math.exp(x - m) for x in xs]
    s = sum(es)
    return [e / s for e in es]

TEMPERATURE here is the same 2.35 the model card reports — the Space and the model repo agree with each other, a small but real cross-check that the number wasn't mistyped in one of the two places it's published. Two more limits the model card states about itself, not flagged by me: option sets were capped at 16 during training, so a Choice with more candidates runs partly out of distribution at inference; and the license is CC-BY-NC-4.0, non-commercial, because part of the training data is real support tickets. Both are disclosed, not discovered.

The three primitives everything else is built from

Every project below sends TypeSafe exactly one of three typed questions — documented at docs.typesafe.ai/primitives and implemented identically in both official SDKs. Here is the JavaScript SDK's actual builder code, which doubles as the cleanest spec of the contract there is:

// typesafe-sdk-js/src/questions.ts
export const noul = (instructions = null, criteria) => ({
  type: "noul", instructions, criteria,
})
 
export const score = (instructions, criteria) => {
  if (!Array.isArray(criteria)) {
    throw new TypeSafeError(
      "Score criteria must be a list of descriptions indexed by score from zero, not a map.",
    )
  }
  return { type: "score", instructions, criteria }
}
 
export const choice = (instructions, criteria) => {
  if (Array.isArray(criteria)) {
    throw new TypeSafeError("Choice criteria must be a map of labels to descriptions, not a list.")
  }
  return { type: "choice", instructions, criteria }
}

Two lines worth reading twice: Score criteria must be an ordered list, because position on the list is the rubric, while Choice criteria must be a map, because a choice is a label, not a position. The SDK throws if you swap them — a small, real guardrail against a Choice silently reading as "the third option" to one caller and "billing" to another.

Where every real project actually sits

Read all fifteen against that grid and a pattern falls out that the launch pitch didn't lead with: almost nothing here is doing open-ended judgment. Every project is a cheap, fast classifier bolted onto the front of something slower or riskier — a browser step, an LLM call, a trade, a diff, a log line — buying back latency or dollars in one narrow slot. That's the pre-processing pattern, and it's a real and useful thing to have. It is a much smaller claim than "frontier decision intelligence," and the size difference is exactly what shows up once you read the code instead of the pitch. It's also worth noting what almost every README says in its own words, unprompted: "independent project," "not an official TypeSafe product," "not affiliated." Whatever this ecosystem is, it isn't TypeSafe's own account building it.

Select — browser, desktop, mobile, and one game controller

browser-use/jev-ultrafast is covered in full in the product piece — the 7.073-second flight search, the 25% latency cut, the 1,092-to-101 drop in browser protocol calls. I won't re-run those numbers here; the short version for this piece is that it's real, measured, and occupies exactly the Select slot: pick one operation and one target element, per step, from a menu code built.

awlevin/typesafe-computer-use does the same job on a Mac and is the most thoroughly engineered project in the whole roster. It never sends a screenshot to a big model — Vision OCR reads the frontmost window, the accessibility tree supplies real control labels and even off-screen controls AXPress can still reach, and a small classifier answers three Choice questions (action kind, which on-screen item, which off-screen control) per step. Splitting the decision into three questions is a deliberate design choice, stated plainly in its own docs: "every stall found while building this came from two options that meant the same thing." Its measured numbers, same screenshot and goal against Claude Opus 5: input tokens roughly equal (4,882 vs. 4,785), cost per decision $0.0002 versus $0.032 (155x cheaper), latency 0.13–0.38s versus 5.2s. Its own caveat is stated just as directly: "the big model read the event dates off the pixels and compared them unaided. The classifier needed the date parsing described below" — every piece of reasoning a frontier model gets for free has to be rebuilt here as deterministic state.

# typesafe_computer_use/decide.py — the three-Choice request, trimmed
def decide(client, goal, screen, items, history, browser, email):
    questions = {
        "kind": Choice(instructions="Which kind of action makes the most progress toward the goal right now?",
                        criteria=kind_criteria(browser, email, bool(screen.offscreen))),
        "site": Choice(instructions="If a website must be opened, which one?", criteria=site_criteria()),
    }
    if items:
        questions["item"] = Choice(instructions="If clicking an on-screen item is right, which item?",
                                    criteria=item_criteria(screen, items))
    if screen.offscreen:
        questions["offscreen"] = Choice(instructions="If activating a hidden control is right, which one?",
                                         criteria=offscreen_criteria(screen.offscreen))
    answers = client.system_one(state=base_state(goal, screen, items, history), questions=questions).answers
    return Decision(kind=answers["kind"], item=answers.get("item"), site=answers["site"],
                     offscreen=answers.get("offscreen"))

droidrun/mobile-jev is the same idea again on a real Android phone via the Mobilerun API — one request combining an operation choice with speculative per-operation targets, so unused target answers just don't execute. Its own demo is candid about scope: the published clip shows Jev opening Uber and reaching payment selection in 21 seconds over 9 actions, and the README says outright "a completed booking is not demonstrated." Its "DONE" response gets the same treatment as jev-ultrafast's: "Jev's DONE response is not independent proof of success," so the shipped demo runner re-reads the actual device state rather than trusting the model's say-so.

vinnylarouge/jevlike is the one project here training something instead of calling Jev. It's a small attention head — each option becomes a query vector, attends over the context, and a shared dot product turns the result into one logit per option, then a softmax — trained from scratch or on top of a frozen Hugging Face encoder. The same head scores text menus, Doom's seven buttons, and a five-key chess controller from the same visual encoder.

expand

Its own numbers are stated plainly and modestly: about 98% accuracy on synthetic menus, 26–29% on target-disjoint Wikispeedia next-click prediction against an 8% shuffled/random-encoder control, and — because the games are real footage, not a benchmark — the released Doom checkpoint averaged 0.60 kills and −97.50 reward over ten episodes, and the chess checkpoint went 4 wins/46 draws/0 losses against a random mover but 0 wins/2 draws/48 losses against Stockfish level 0, with the README adding "the windows were selected for activity and are not typical-play or competence claims." The README's own framing throughout is that this is "an independent starter model," not a reproduction: "TypeSafe has not published its design," and later, flatly, "we did not show equal quality with Jev or reproduce TypeSafe's private training method." That candor is worth taking at face value — and it makes an open issue on the repo worth reading past the README to find. A user (collapseindex) reports that the shuffled-context control the 8% baseline depends on rolls by one position within each batch, and because Wikispeedia rows are ordered by target article, 39.4% of "shuffled" pairs still share their real target — meaning the control is inflated by data leakage, not a clean random baseline, and the true size of the 26%-vs-shuffled gap is unsettled. As of this writing the issue has no reply. That's not a reason to dismiss the project — the honest framing throughout the README is real — but it's exactly the kind of check the house method here asks for, and it's unresolved.

sorrycc/typesafe-snake is the smallest and most honest entry in the Select column: code computes legal moves, food distance, and flood-fill reachability every tick, and Jev picks one via a single Choice question; if the answer misses the tick deadline the snake just keeps going straight. No benchmark, no accuracy claim — it's a toy, and it doesn't pretend otherwise.

vlad-terin/jev-browser is listed in the taxonomy above as unverified. What I could find secondhand describes it as "Jev-powered element selection for your agent's existing computer-use tools," later renamed jev-use; I could not clone either name, and every URL I tried under that repo returned 404 through this session's access, so I'm not writing it up as if I'd read the code.

Route — one cheap call decides which expensive path handles this turn

gargpratyush/jev-router puts Jev in front of Claude Code and OpenAI Codex themselves: one Choice per fresh user turn picks a tier (fast/balanced/strong/long), a loopback proxy forwards the CLI's own auth headers unmodified, and an explicit request like "use opus" always wins over the model's pick. It ships as a real npm package with a bundled explanation skill (/jev-explain) that renders the exact saved request/response for the last routing decision — a genuinely good piece of UX for trusting an automated router.

expand

// src/policy.mjs (via router.mjs) — one Jev call, fail-open
export async function askJev({ prompt, current, contextTokens, available }) {
  const request = {
    state: {
      request: prompt,
      session: { current_model: current, context_tokens: contextTokens },
      environment: { available_models: available },
    },
    questions: QUESTIONS,
  }
  try {
    const result = await getClient().systemOne(request, { signal: abort.signal })
    return { ...result.answers.model_tier, request, response: result, ms: Date.now() - started }
  } catch (err) {
    log(`routing failed, keeping ${current}: ${err.message}`) // never blocks the CLI
    return null
  }
}

Its own policy notes are candid about the parts that don't reduce to one clean rule: "low confidence never downgrades and caps upgrades at the balanced tier," and "large conversations refuse downgrades that would waste more prompt-cache work than they save." What's missing is any published accuracy or latency benchmark for the routing decision itself — the README shows one illustrative example (task complexity 0.82, confidence 94%) and stops there. Real, useful-sounding code; no measured number backing the thing it's actually for.

reachjalil/jevlogs runs the identical shape one layer down the stack: score every OpenTelemetry log record for diagnostic value and actionable probability before it reaches a reasoning model, and let confidently low-value, low-priority, low-probability records skip that expensive branch while staying in your existing archive untouched. Its own cost table — a $1,000/month analysis bill modeled at $129.40 once 90% of logs skip deeper analysis — carries a footnote in the same paragraph: "It is not a measured production result." The project-status table in its own README goes further: "Live Jev accuracy and production savings: Not yet independently validated for this project." That's about as explicit a non-claim as this entire roster makes, stated by the author about their own numbers, and it deserves to be read as a model for how to publish an illustrative cost calculation without dressing it up as measured.

Rank — reranking without embeddings

superagents-lab/jev-search is the one project in this roster occupying a slot nobody else does: real reranking, no embeddings, no generated answer. A Choice/Noul pass first decides sources, time window, and which of several candidate search queries best matches the request; Google, DuckDuckGo, and seven vertical engines then run concurrently; and every result gets a Noul relevance score the application sorts and groups by, streamed as newline-delimited JSON.

// src/lib/typesafe.ts — the rerank pass, batched 40 results at a time
export async function rerank(config, request, items, signal) {
  const questions = {}
  items.forEach((_, i) => {
    questions[`r${i}`] = {
      type: "noul",
      instructions: `Is \`results[${i}]\` about the subject the user asked for in \`request\`?`,
      criteria: {
        true: "The title or snippet discusses the same subject, even briefly or as one of several topics",
        false: "About something else that only shares words with the request, or is unrelated",
      },
    }
  })
  const state = { request, results: items.map((it) => ({ source: it.source, title: it.title, snippet: it.snippet })) }
  return systemOne(config, state, questions, signal)
}

Its own README is exact about what the relevance number is and isn't: "relevance percentages are model judgments, not verified accuracy." No ranking-quality benchmark is published — the engineering (concurrent multi-engine search, per-engine 15-second deadlines inside a 30-second overall budget, response caching from 10 minutes to 6 hours depending on the query's time window) is real and shipped as a live Cloudflare Worker, but whether the Noul scores actually rank better than, say, the search engines' own ordering is an unanswered question. jkudish/jev-mcp's third tool, jev_find, does the identical job as an MCP server for coding agents — rank candidate ids by meaning, no index to maintain — and is covered together with the rest of that project under Verify below, because all three of its tools ship in one package.

Classify — the trading bot, the smart home, the dataset filter

jarrodwatts/jev-trader is the one to be careful with. It watches a real Kuru MON-USDC order book on Monad and asks Jev a single Choice — buy or sell — roughly every 300ms block, then posts a real post-only limit order one tick inside the touch. With no PRIVATE_KEY set it dry-runs by default: real book, real decisions, simulated fills. MODEL=mock — a plain momentum-and-imbalance heuristic, not Jev — is the actual default even when a key is present; Jev only trades when both a private key and MODEL=jev are explicitly set.

// src/model.ts — real Jev is opt-in; mock is the default
export const createModel = (): Model => (config.model === "jev" ? new JevModel() : new MockModel())
 
export class JevModel implements Model {
  private model = typeSafeAi.evaluationModel(config.jevModelId)
  async decide(state: TradeState): Promise<Decision> {
    const r = await experimental_evaluate({ model: this.model, state, questions: QUESTIONS, maxRetries: 0 })
    const a = r.answers.direction
    return { action: a.choice as Action, probabilities: { buy: a.probabilities?.buy ?? 0, sell: a.probabilities?.sell ?? 0, hold: 0 }, upIn10: a.probabilities?.buy ?? 0, latencyMs: 0, inputTokens: r.usage?.inputTokens ?? 0 }
  }
}

What's measured, and real: the read-then-decide loop's own latency (book read p50 ~18ms, whole loop p50 ~100ms in a dry run with the mock model), and a hardcoded, deliberately narrow request path (no eth_estimateGas, a static gas price, one eth_call plus one eth_sendRawTransaction) built specifically to fit inside one ~300ms block. What's not published anywhere in the repo, and what I'm not going to imply by omission: any win rate, P&L, or profitability claim. The README states the mechanism and the dry-run default and stops there; the honest read of this project is "an experiment in fitting a decision inside a block deadline," not a trading result.

AboveColin/HA-Jev turns Home Assistant entities into Noul/Choice/Score sensors and four callable actions, with a daily token budget that halts evaluation once spent — real, tested (103 tests against a real Home Assistant with the API client swapped out) integration code, thirteen worked examples including four that pair Jev with an LLM for guardrails and cascades. Its own "Known limitations" section states the calibration question as plainly as jevlogs does: "Confidence has no published calibration evidence. Treat 0.9 as higher than 0.6 until you have measured it on your own questions." — and, separately, "Not for safety decisions. A probability with no explanation should not hold a lock, a heater, or a smoke alarm."

AkashPriyadarshii/jev-curate filters synthetic-data rows through Noul/Score gates (circular reasoning, sycophancy, unclosed code fences) in a real Rust core with PyO3 bindings and Cargo/PyPI packaging — the source is genuine, not a stub. It's also the most self-promotional README in the roster: a "Mandatory Ecosystem, Author & Social Directory" section lists the author's unrelated repos and five social profiles, and the headline throughput and cost figures ("1,500+ rows/sec," "$4.20 per 100M tokens," "700x cheaper than Claude 3.5 Sonnet") are asserted rather than shown against a reproducible harness in the repo itself. Real code, unverified numbers, presented with more marketing than anything else on this list.

pngwn/open-jev also belongs in this column — it's covered in full above, because it's the one project whose numbers I could actually check against a committed file rather than a README claim.

Verify — code review and claim-checking

devagrawal09/jev-review reviews a git diff or a whole codebase through a staged pipeline: a Noul risk matrix first, then Choice/Score file profiles, then evidence selection, mechanism classification, severity scoring, and finally conditional reviewer routing — six typed stages, each one narrowing what the next stage has to consider. The codebase enforces its own layering with a real lint rule (scripts/check-dependencies.ts fails npm run check on any upward import), which is a level of engineering discipline most projects in this roster don't bother with.

expand

No accuracy, precision, or false-positive-rate number is published anywhere in the repo, and the README says so itself, without hedging: "Findings are review prompts, not proof of a defect." That's the correct level of confidence to state for a tool with zero measured numbers behind it, and it's worth contrasting with how many launch-adjacent projects imply more than they've checked.

jkudish/jev-mcp is the one project here that spans three of the five slots in a single package: jev_verify checks claims against cited evidence (a Choice over supports/contradicts/says_nothing per claim), jev_screen judges fetched text for prompt-injection risk before an agent reads it, and jev_find — covered under Rank above — ranks candidates by meaning with no embeddings. Every call comes back in 150–500ms for a fraction of a cent, per the README, and its "real use" section reports three specific outcomes rather than a benchmark suite: a contradicted claim caught against a city ordinance at confidence 1.0, a pricing page carrying a hidden "ignore your instructions" note blocked at injection probability 0.99, and a three-way file ranking that picked the intended file at probability 1.0. Three real anecdotes with real confidence numbers is more evidence than most of this roster publishes — and it's still three anecdotes, not a suite. The README says it plainly: "this is early software."

What's actually missing: real numbers

Line up every status flag from the taxonomy above and the pattern is stark. It's sixteen rows across fifteen unique repos — jev-mcp's tools take two rows because they occupy two slots — and one of those fifteen, jev-browser, is the one I could not verify at all. Of the fourteen I could read directly: four report a real measured benchmark (jev-ultrafast, typesafe-computer-use, jevlike, pngwn/open-jev), four report one timing or anecdote and stop there, and six report real, working code with zero numbers behind the claim they're implicitly making. That's not a roster of people overselling — most of these READMEs are careful, and several (jevlogs, HA-Jev, jev-search, jev-review) go out of their way to say "not yet measured" in their own words. It's a roster of people who mostly haven't measured yet, at nine days old, which is a different and more forgivable thing.

One project makes that gap its actual subject rather than a footnote. 24601/Augustus is an agent skill, not an app — it maps classical decision-theory methods onto Choice/Score/Noul, and it ships scripts/evaluate_decisions.py, an offline evaluator that computes Brier score, a reliability curve, and a threshold/cost sweep for exactly the kind of binary decision most of this roster is quietly making without checking. It publishes no numbers of its own — it's a tool for checking someone else's — but its existence, next to pngwn/open-jev's committed metrics.json, is the second sign in nine days that calibration checking is becoming something this ecosystem does rather than something it defers to TypeSafe.

The take

Read the taxonomy again and the shape is unmistakable: browser step, desktop click, phone tap, search result, log line, git diff, order-book tick — different inputs, same move in every single case. A fast, cheap, typed classifier sits in front of something slower, riskier, or more expensive, and code keeps every threshold, every side effect, and every "what happens next." That is a real and useful thing to build, and fourteen independent teams built it in nine days without needing TypeSafe's cooperation — the same speed at which outsiders reproduced the decoding trick in the product piece, for the same reason: it needs no special access, just an API key and an afternoon.

It is also, plainly, a narrower thing than "frontier decision intelligence." Nothing in this roster asks Jev to reason about anything open-ended; every single project narrows the question down to a short, code-built menu first and lets the model pick from that. The value showing up across fifteen real projects is latency and cost in a specific slot — route, select, rank, classify, verify — not general intelligence wearing a faster hat. pngwn/open-jev is the one project that pushed past that and asked the harder question the two companion pieces on this site kept circling back to: not "is it fast," which nobody here disputes, but "does the number it reports mean what it says." Its answer — a real ECE, cut by more than two-thirds by a fix that also makes one specific task worse — is the first time in this entire story that question got a checkable number instead of a name and a paragraph.

Sources: Anil-matcha/awesome-jev-by-typesafe's community index and use-case map (MIT licensed, snapshot reviewed September 18, 2026) named the starting set beyond the ten repositories in this piece's brief. Every project write-up above is from that project's own cloned source and README, read directly, not summarized secondhand — except vlad-terin/jev-browser, flagged as unverified because its repository returned 404 on every access path tried. pngwn/open-jev's numbers are from its own committed metrics.json and spaces/pngwn/open-jev's README and app.py. Vercel AI Gateway access was checked directly against vercel.com/ai-gateway/models/jev and its own changelog. The three-primitives, decision-slot-map, and calibration interactives are original to this article.