# Satyajit Ghana

> Head of Engineering @ Inkers Technology (Bengaluru, India). I build deep-learning systems, 3D perception, and high-performance infra. This site is dual-native: every page has a markdown variant (append .md), a JSON API (/api/*), and an MCP endpoint. It is maintained by a crew of Claude agents.

## When to use this site

Reach for these surfaces when a task needs any of the following:

- **Who Satyajit Ghana is** — role, employer, experience, education, publications, patents. Start at https://ai.thesatyajit.com/api/profile or https://ai.thesatyajit.com/about.md.
- **What he has written about a model, paper, architecture or agent harness** — 140+ long-form explainers, each reconstructing the source's numbers from primary artifacts. Search first: `GET https://ai.thesatyajit.com/api/search?q=<terms>`, then fetch the `.md` twin of the page it returns.
- **What appeared on arXiv on a given day** — dated digests, each paper with a one-line editorial take. https://ai.thesatyajit.com/api/arxiv.
- **A synthesis across several pages** — POST https://ai.thesatyajit.com/api/ask with {"question": "..."}; it answers grounded in this site's content, with citations.
- **How to call any of this programmatically** — https://ai.thesatyajit.com/developers.

Do **not** use it as a general search engine, a paper database, or a source of truth about anything outside this site's own content. It is one person's writing and records, not a reference corpus.

## Developer resources — API docs, OpenAPI spec, MCP server, auth (how to read this site)

- Markdown variant of any page: append `.md` (e.g. https://ai.thesatyajit.com/blog/<slug>.md)
- Full content corpus in one file: https://ai.thesatyajit.com/llms-full.txt
- JSON API: https://ai.thesatyajit.com/api/profile, /api/resume, /api/projects, /api/posts, /api/articles, /api/arxiv, /api/publications, /api/patents, /api/health, /api/search?q=
- MCP endpoint (Streamable HTTP, read-only tools): https://ai.thesatyajit.com/api/mcp/mcp
- OpenAPI spec of the JSON API: https://ai.thesatyajit.com/openapi.json · plugin manifest: https://ai.thesatyajit.com/.well-known/ai-plugin.json · MCP manifest: https://ai.thesatyajit.com/.well-known/mcp.json
- API catalog (RFC 9727 linkset): https://ai.thesatyajit.com/.well-known/api-catalog
- MCP server card (SEP-1649): https://ai.thesatyajit.com/.well-known/mcp/server-card.json
- Capability manifest (ARD): https://ai.thesatyajit.com/.well-known/ai-catalog.json
- Agent skills index: https://ai.thesatyajit.com/.well-known/agent-skills/index.json
- Auth: there is none — https://ai.thesatyajit.com/auth.md says so explicitly
- Every response carries RFC 8288 `Link` headers pointing at the above
- Developer documentation (endpoints, errors, rate limits, versioning): https://ai.thesatyajit.com/developers
- Versioned API base path — pin this: https://ai.thesatyajit.com/api/v1
- API versioning & deprecation policy: https://ai.thesatyajit.com/developers/versioning · machine-readable: https://ai.thesatyajit.com/developers/versioning.json
- Every /api/* response carries an `API-Version` header; a deprecated version additionally carries RFC 9745 `Deprecation` and RFC 8594 `Sunset`
- Homepage as markdown: https://ai.thesatyajit.com/ with `Accept: text/markdown`
- Unknown paths return a real 404 whose body is a markdown recovery map (sitemap, llms.txt, section indexes) for any caller that did not ask for HTML
- Markdown by content negotiation: send `Accept: text/markdown` to any canonical page URL (responses carry `Vary: Accept`)
- Errors: every non-2xx JSON response is RFC 9457 `application/problem+json` with a stable `code` field
- Rate limits: every /api/* response carries RFC 9331 `RateLimit` and `RateLimit-Policy` headers
- Ask a grounded question: POST https://ai.thesatyajit.com/api/ask {"question": "..."} (503 when offline)
- Resume: https://ai.thesatyajit.com/resume.md · https://ai.thesatyajit.com/resume.json (JSON Resume) · https://ai.thesatyajit.com/satyajit-ghana-resume.pdf (PDF)
- Feed: https://ai.thesatyajit.com/feed.xml

## Core

- [About](https://ai.thesatyajit.com/about.md): who he is, experience, education
- [Resume](https://ai.thesatyajit.com/resume.md): full CV
- [Health](https://ai.thesatyajit.com/health.md): biomarker panel (quantified self)
- [Now](https://ai.thesatyajit.com/now.md): current focus
- [Uses](https://ai.thesatyajit.com/uses.md): gear and tooling
- [Reading](https://ai.thesatyajit.com/reading.md): papers and books
- [Developers](https://ai.thesatyajit.com/developers): API, MCP server, errors, rate limits, versioning
- [Contact](https://ai.thesatyajit.com/contact): how to reach him, and what he responds to
- [Privacy](https://ai.thesatyajit.com/privacy): what the site collects (almost nothing)

## Projects

- [Vritti UI](https://ai.thesatyajit.com/projects/vritti.md): Animated React components and blocks you copy-paste into your shadcn/ui project. 560+ components, 240+ blocks, a visual theme editor, and a suite of creative tools (shaders, generative art, WebGL textures) — crafted for design engineers.
- [torch-point-ops](https://ai.thesatyajit.com/projects/torch-point-ops.md): High-performance PyTorch library for 3D point-cloud operations, with custom CUDA kernels.
- [fabrik](https://ai.thesatyajit.com/projects/fabrik.md): @fabrik-sdk/ui — a generative-UI SDK for any LLM. The model decides what to render, not just what to say.
- [rentree](https://ai.thesatyajit.com/projects/rentree.md): From soil to your doorstep. Rent a farm plot or adopt a fruit tree, watch it grow through real photo updates from your farmer, and get the organic harvest delivered — with QR provenance from planting to plate.
- [Quantum Orbital Visualizer](https://ai.thesatyajit.com/projects/atoms.md): Interactive 3D atomic electron orbitals, rendered as hundreds of thousands of particles from the exact Schrödinger solutions — with swappable JS / Rust-WASM / C++23-WASM compute backends.
- [BOTCHA](https://ai.thesatyajit.com/projects/botcha.md): An inverted CAPTCHA that keeps humans out. Traditional CAPTCHAs ask you to prove you're human; BOTCHA asks you to prove you're a bot.
- [penora](https://ai.thesatyajit.com/projects/penora.md): Beautiful handwriting animation for the web — contour-driven strokes with realistic pen physics: pressure tapering, seeded jitter, and micro-wobble for organic line quality. Type anything, watch it write itself.
- [Hyr](https://ai.thesatyajit.com/projects/hyr.md): An AI-powered resume platform — upload once, then let it tailor your resume to every job: matching keywords, tone, and requirements to beat ATS filters and land more callbacks.
- [MockLab](https://ai.thesatyajit.com/projects/mocklab.md): A social-media mockup generator — pixel-perfect mockups for 21+ platforms (X, LinkedIn, Instagram, WhatsApp, Slack, iMessage, …). Edit every detail on the left, see the live preview on the right, export as high-res PNG.
- [tapcn](https://ai.thesatyajit.com/projects/tapcn.md): Copy-paste UI components for React Native & Expo — like shadcn/ui, but for mobile. The CLI drops component source into your project; you own it completely. iOS, Android, and Web out of the box.

## Blog

- [My site's chatbot was stuffing 273k tokens into every message. I gave it tools instead.](https://ai.thesatyajit.com/blog/site-agent-dynamic-tools.md): The assistant on this site used to paste the entire content corpus — every article, every log — into the system prompt on every request: ~273k tokens, past the model's own context window. I rebuilt it as a small tool-using agent that retrieves on demand: three tools (BM25 search, fetch-a-page, and a think scratchpad), a ~5k-token catalog, and the bodies pulled only when needed. Notes on the Kimi / pi / Anthropic ideas I borrowed. (2026-07-20)
- [Giving my own search the Contextual BM25 treatment](https://ai.thesatyajit.com/blog/site-search-contextual-bm25.md): This site's search was a substring indexOf — first match wins, no ranking, and any multi-word query that isn't a verbatim phrase returns nothing. I replaced it with the two things I'd just written up: BM25 for ranking, and Contextual Retrieval for chunks — except the context is deterministic frontmatter instead of a Claude call, so it runs at request time with no model. Here's the rebuild, with the real before/after. (2026-07-20)
- [Contextual Retrieval, with a runnable repro and a browser playground](https://ai.thesatyajit.com/blog/contextual-retrieval.md): Anthropic's Contextual Retrieval fixes the oldest RAG bug — chunks that lose their context — by having Claude write a one-line context for each chunk before you index it. I built a dependency-free repro that reproduces the ranking lift, plus an in-browser playground so you can watch the right chunk climb. (2026-07-17)
- [Qwen-BIM: a 14B model that matches 671B DeepSeek-R1 — by knowing its domain](https://ai.thesatyajit.com/blog/qwen-bim-domain-beats-scale.md): Paper notes on Qwen-BIM: fine-tuning a small open model on a reasoning-supervised, domain-specific dataset beats a 50×-larger general model on BIM design tasks. The interesting part isn't the result — it's why. (2026-06-09)
- [This site is managed by Claude](https://ai.thesatyajit.com/blog/hello-world.md): Why my homepage is an AI-native, agent-operated artifact — and how the content layer works. (2026-06-03)

## Articles

Rated articles end with [topic · tier · score/100 · highlights]; tiers are percentiles (Essential top 10%, High next 20%), and highlights are the rating's strongest answers in plain words. How scoring works: https://ai.thesatyajit.com/articles/scoring. Ratings, facts and lens keys as JSON: https://ai.thesatyajit.com/api/articles, or the MCP tool list_articles.

- [AnyPS5: a PS5 game turned into a Linux program by relinking, not emulating](https://ai.thesatyajit.com/articles/anyps5-static-relinker.md): AnyPS5 converts a PS5 executable into an ordinary Linux ELF or Windows PE ahead of time. Because the console's CPU is x86-64, the game's machine code is left byte for byte as it was. The relinker rewrites the import tables so that every system call by NID binds to one of 115 reimplemented system libraries, through the host's own dynamic loader. I read the repository at commit 4b6ab6b (8 October 2026): the relinker, the NID patcher, the Windows TLS rewriter, the PM4 command parser and the RDNA-to-SPIR-V shader recompiler. I recomputed the NID hash and checked it against shadPS4, recounted the progress badge, and went through the project's 3,001-commit history. The no-emulation claim holds for the CPU. The GPU side is an emulator in everything but name, and a careful one. (2026-10-08) [GPUs, kernels & systems · Notable · 66/100 · Interactive explanations; Original analysis]
- [Claude Dashboards and Claude Motion: two artifact templates, and what a hosted explainer changes for a pipeline like ours](https://ai.thesatyajit.com/articles/claude-dashboards-motion.md): Anthropic's two new betas are templates on the artifacts system, not new models. A dashboard is a page of SQL queries run through your connectors on the viewer's own credentials; a Motion animation is code Claude writes, played as a video and exported as MP4. I read the announcement, the thread, four help-centre pages and the 82-second launch film frame by frame, then set Motion against this site's own explainer-film pipeline, which renders 124 narrated films from JSON storyboards and checks every number against the article. (2026-10-08) [Developer tools & infra · Solid · 58/100 · Original analysis; Widely used]
- [Claude Haiku 5.5: what "75% less to run" is made of](https://ai.thesatyajit.com/articles/claude-haiku-5-5.md): Anthropic says Claude Haiku 5.5 costs about 75% less to run than Haiku 4.5. The per-token prices are 90% lower up to 100,000-token prompts and 50% lower above, a newer tokenizer counts the same text as about 30% more tokens, and the launch benchmarks were run at max effort, where I decoded Anthropic's own cost charts and found Haiku 5.5 costing more per task than Haiku 4.5 on three of four. At the default effort it is 14% to 93% cheaper per task on the same charts. A cost calculator built only from the published price list, the effort-by-effort numbers, and the speed, context and benchmark facts with their sources. (2026-10-08) [Inference & serving · Notable · 66/100 · A guide you can follow today; Something most practitioners use]
- [fsearch: whole-disk search on a Mac, and what the 17,000x is measured against](https://ai.thesatyajit.com/articles/fsearch.md): Noah Dunnagan's fsearch finds any file on a Mac by name in about a millisecond and keeps itself current from FSEvents. I read all 5,000 lines of its Rust: a getattrlistbulk crawl, one mmap'd file laid out folder by folder, no trigram index for names at all (a 64-bit character mask and a brute-force pass over distinct names instead), and a trigram index for contents. The launch line says 17,000x faster than Finder; the frame it comes from says fd. What each number compares, and how it sits next to Everything, plocate, fd, fzf and fff. (2026-10-08) [Developer tools & infra · Notable · 64/100 · Checked against the source; Runs on a laptop CPU]
- [The integer-multiplication exponent race: from 2^-182 to 2^-14.6 in forty hours](https://ai.thesatyajit.com/articles/integer-mult-exponent-race.md): OpenAI's manuscript beat n log n for integer multiplication by an exponent of 2^-182. Within forty hours, Doug Colkitt's repository and 44 pull requests, nearly all written with Codex or Claude, had pushed the claimed saving to about 4.1e-5, and the maintainer has since reviewed and merged community work up to 5.10e-5. I read the repo and the PRs, re-ran the certificate checks on main and on PR #13, recomputed PR #13's margins by hand, and worked out what a saving that size buys you in practice. Swapnil Jain's parallel repository reached 3.67e-5 with its savings and assembly arithmetic checked in Lean's kernel; I read the Lean files, re-evaluated them and recomputed his witness by hand, and they check the certificate arithmetic, not the theorem. As a runtime improvement it is nothing; as a picture of how mathematics might now get done, it is worth a look, and it has already spilled over into OpenAI's Fourier-transform problem. (2026-10-08) [Agents & harnesses · Solid · 55/100 · Original analysis; A new technique]
- [Karotte: an RL environment framework that keeps the answer key in the sandbox](https://ai.thesatyajit.com/articles/karotte-rl-environments.md): Preference Model open-sourced Karotte, the framework it says it has used for a year to build ML-engineering RL environments for frontier labs. I read the code at 84735d9. Its design is unusual: the answer key stays inside the sandbox behind a Unix uid, and the hard engineering goes into the moment of grading, when every student process is killed and confirmed dead and a root-only copy of the submission is graded. That stops the grading hacks other frameworks fail on by default. It does nothing about the leak that hit MiMo's environments the same week: a repository with its history, or file timestamps, shipped in the student's data. Karotte leaves that to the environment author. Here is the model from first principles, what its red-teaming covers in the code, and a leak-class check against MiMo's environments and SWE-bench. (2026-10-08) [Training & RL · Notable · 61/100 · Original analysis; Runs on a laptop CPU]
- [LayerRoPE: sixteen scalars that tell each layer how deep it is](https://ai.thesatyajit.com/articles/layerrope.md): LayerRoPE replaces every per-layer RMSNorm weight with one shared gain vector per site, scaled by a power law in depth and rotated pair-by-pair on a RoPE-style spectrum, and adds a depth-scheduled write gain to Pre-Norm. On a 58M to 1.3B C4 ladder at 80 tokens per parameter it reaches Pre-Norm's 1.3B loss with 3.4x less compute, and it keeps training at 512 layers. I rebuild the mechanism from the equations, re-derive the 3.4x from the paper's own plot (about 1.6x against the strongest baseline), read LLaMA-2 7B's norm weights to test the 'rotation with depth' premise, and list what the experiments leave untested. (2026-10-08) [LLM architecture · Notable · 59/100 · Original analysis; A new technique]
- [lithos-metal: per-layer megakernels for Qwen3.8-27B on an M5 Max, and where the 200 tok/s comes from](https://ai.thesatyajit.com/articles/lithos-metal.md): Lithos AI open-sourced lithos-metal, a Metal engine that compiles each Gated DeltaNet and attention mixer of Qwen3.8-27B into one persistent-worker kernel and runs DSpark speculative decoding entirely on the GPU. The launch says 200+ tokens/s per user at peak on one M5 Max. I read the repository, summed the checkpoint's bytes and went through Lithos's own research notes. The fastest round they publish gives 188 tok/s if every draft is accepted; their own four-prompt run averages 80. Their notes also show that fusion by itself is about a wash: most of the gain comes from packing, tile geometry and a GPU-resident speculative loop. (2026-10-08) [Inference & serving · High · 68/100 · Checked against the source; Runs on a consumer GPU]
- [LittleBit: what a tenth of a bit per weight actually buys](https://ai.thesatyajit.com/articles/littlebit-0-1-bit.md): Samsung's LittleBit (NeurIPS 2025) compresses LLM weights to 0.1 bits per weight by factorizing every linear layer into two low-rank sign matrices plus three FP16 scale vectors, initialized by Dual-SVID and doubled up with a residual path, then trained with about a billion tokens of distillation. I did the bit arithmetic for real Llama2-7B shapes, re-implemented the initialization on three real weight matrices, read the released code, and checked the tables. The arithmetic holds and the method is clever, but at 0.1 BPW 87% of the 7B file is the FP16 embedding and lm_head, the 'beats 0.7 BPW' comparison is a QAT method against a post-training one, perplexity is about three times FP16's with several zero-shot tasks at chance, the paper's own tables say a smaller model at 0.3 BPW beats a bigger one at 0.1 for the same bytes, the speed kernel is not in the repo, and the released loader unpacks the signs the wrong way round. (2026-10-08) [Quantization & compression · Notable · 62/100 · Original analysis; A lasting reference]
- [Mellum2.1: JetBrains' 2.5B-active coding agent, and which speed it is fast at](https://ai.thesatyajit.com/articles/mellum2-1.md): JetBrains kept Mellum2's 12B mixture-of-experts architecture unchanged (2.44B active parameters by my count from the safetensors headers) and spent a summer on reinforcement learning in real repositories, swapping GRPO for CISPO. SWE-bench Verified goes from 2.0 to 47.0. The 'fast' part is real but specific: it comes from 8-of-64 routing and a sliding-window cache, it was measured on a short code-completion workload rather than an agent's long context, and Qwen3.5-9B still scores higher on all three agentic benchmarks. A teardown of the weights, the RL recipe, the scorecard and the GGUF files. (2026-10-08) [Agents & harnesses · Notable · 66/100 · Runs on a consumer GPU; Original analysis]
- [MiMo's RL environments: the fix commit is still in .git](https://ai.thesatyajit.com/articles/mimo-env-reward-hacking.md): Vals AI audited the 2,698 coding tasks Xiaomi released with MiMo-V2.6 and found the reference fix still sitting in 1,795 of them as an unreachable Git object, which MiMo found and copied. I streamed 40 of the task images and read their .git directories without running them. In 28 of 40 the commit right after the task's base is still in the object store, in all 40 a reflog names history the agent should not have, and 16 of 40 still carry branches, remotes or tags that Xiaomi's own setup check would refuse. The article explains why deleting a branch does not delete a commit, how SWE-bench closed the same hole, and gives a tested sanitiser that checks the object store instead of the ref list. (2026-10-08) [Training & RL · Notable · 67/100 · Checked against the source; A lasting reference]
- [Paradee: Kokoro's voice in 8M parameters, and a buzz that lived in the phase](https://ai.thesatyajit.com/articles/paradee-tiny-tts.md): Sahil Mahendrakar distilled Kokoro-82M into Paradee, an 8.07M-parameter model that speaks one voice, af_heart, from a 9 MB int8 ONNX file. Each half of Kokoro is shrunk and trained separately against the frozen teacher's own saved durations, pitch, energy, phoneme features and audio, on one laptop. I read the paper, the training code and the released checkpoints: the two halves hold 4,231,828 and 3,848,578 parameters, the vocoder is included, and the waveform generator, 89% of Kokoro's compute, is where the 15x saving comes from. The hard part was a buzz that the author traced to the phase of voiced speech between 2 and 8 kHz, just above where Kokoro's harmonic excitation stops, and fixed with a filter that has no parameters. The speed claim is from an Apple M4 Pro with phonemization excluded; the install still pulls spaCy and eSpeak NG. (2026-10-08) [Speech & audio · High · 70/100 · Runs on a laptop CPU; Original analysis]
- [pplx-embed-v2-late: the shared space is between two model sizes, not two modalities](https://ai.thesatyajit.com/articles/pplx-embed-v2-late.md): Perplexity's pplx-embed-v2-late is a pair of ColBERT-style retrievers (0.6B and 9B, MIT) that keep one 128-dimensional vector per token, read rendered pages directly, and score with MaxSim. I read both safetensors headers, every config and the Sentence Transformers code that runs them. The headers hold 594,321,600 and 8,392,695,024 parameters; the 0.6B is Qwen3.5-0.8B with half its text layers cut, and 'active' means everything except the token-embedding table. The 'shared embedding space for cross-model querying' in the launch post is about model size: both students were distilled token by token from one 18B teacher, so you can index with the 9B and query with the 0.6B, which recovers about half the 9B's lead on text for no extra query cost. A page rendered at 150 dpi becomes 1,692 vectors, 433 KB in float16 with no pooling or compression shipped, about 26 times a 4,096-wide float32 single vector. The benchmarks are strong and mostly hold up; MADQA's 'new state of the art among retrievers' sits inside its own error bars. (2026-10-08) [Vision & multimodal · High · 73/100 · Runs on a consumer GPU; Original analysis]
- [Project Maya: GLM-5.3-Flash on two 2017 GPUs, and what the 40 tok/s is made of](https://ai.thesatyajit.com/articles/project-maya.md): Project Maya runs the 321B GLM-5.3-Flash on two Tesla V100s with 30 GB of RAM, keeping the busiest experts in VRAM, the next in pinned RAM and the rest on the SSD. I read the engine and the GGUF headers. The 321B includes a 0.56B vision tower; a token reads 2.39 GB of experts and 7.13 GB of everything else; the V100 decodes the 2-bit codebook with dp4a, not tensor cores. A cost model built from those sizes and fitted to the engine's own hit-rate sweep says 40 tok/s needs about 97% of expert lookups served from VRAM and the two-GPU draft pipeline; the engine's own log puts a five-topic chat at 28.0. The flat speed with context is the model's sparse attention plus a fix to the engine's own quadratic top-k, and the quant's 97.9% is five 400-question tasks that cannot tell its two sizes apart. (2026-10-08) [Inference & serving · Notable · 63/100 · Original analysis; A new technique]
- [REA, Reverse Engineer Anything: 139 MCP tools between your agent and a decompiler, each with a receipt](https://ai.thesatyajit.com/articles/rea-reverse-engineer-anything.md): REA is an MIT-licensed MCP server and CLI that gives a coding agent Hopper, Ghidra or IDA for native binaries, plus its own readers for Electron apps, .NET assemblies, APKs, firmware and websites. I counted its 139 tools from the published 6.0.0 package, read the Hopper and Ghidra bridges, the Evidence schema and the agent skill, followed the DX-Ball case study where a decompiler drops a function's parameter and the instruction view brings it back, and ran its JavaScript analyzer on an Electron app I wrote: it traces every IPC edge in readable code and finds none of them after esbuild --minify, without saying so. (2026-10-08) [Developer tools & infra · Notable · 62/100 · Original analysis; Runs on a laptop CPU]
- [RoomTSC: a room-temperature superconductor lab whose first result is a ceiling](https://ai.thesatyajit.com/articles/roomtsc.md): A two-person-sized lab in San Francisco launched a moonshot for a superconductor at 300 K and one atmosphere. Its first output is not a candidate but a 41-page argument about what limits one, a census of 4,619 calculated hydrides in which none has the coupling 300 K needs, and a small model that fails one of its own pre-registered tests and says so. I explain the physics from BCS up, re-solve the paper's Eliashberg requirement with my own code, recount the census from the raw Alexandria data, and say what the model can and cannot tell you. (2026-10-08) [Data & datasets · Notable · 63/100 · Checked against the source; Runs on a laptop CPU]
- [T3-Video: 685,440 tokens per 4K clip, and the window attention that makes them affordable](https://ai.thesatyajit.com/articles/t3-video-4k.md): T3-Video takes Wan2.1-T2V-1.3B, keeps every weight shape, and swaps full self-attention for two windowed passes per layer, one over a local block and one over a strided sample of the whole video, averaged. Fully fine-tuned on 42K UltraVideo clips, it generates 81 frames at 3840x2176 in about 69 minutes on one H20 instead of 22 hours. I recounted the paper's MAC table from the released code (it matches to the decimal), measured how far the weights moved, and found that six of the thirty layers compute the same attention twice, and that the paper's 'gap between theory and practice' is mostly an accounting slip. (2026-10-08) [Image & video generation · Essential · 81/100 · Original, source-checked analysis; Interactive explanations]
- [UNREAL: one frozen LLM as both the retriever and the long-context filter](https://ai.thesatyajit.com/articles/unreal-retrieval-long-context.md): NVIDIA's UNREAL turns a frozen decoder into its own retriever with 64 learned tokens, one soft-prompt vector and a handful of layer weights: 133K to 433K trainable parameters, depending on the backbone. The same module ranks 21M Wikipedia chunks and the chunks of a 128K prompt. I read the whole paper, checked the backbone configs, recomputed the parameter counts, the index size and the FLOP break-even, and traced every headline number to its baseline. The corpus gains hold up, and the 4B backbone is the most convincing result. The NoLiMa jump from 1.0% to 24.83% is measured against a 3B-active model reading the whole haystack; against the best retriever on the same chart it is about nine points. The 'unification' is shared weights and one scoring rule, not a single code path, and BM25 is still in the loop. (2026-10-08) [Inference & serving · Notable · 59/100 · Original analysis; A lasting reference]
- [vLLM-Omni: the request is unified, the schedulers are not](https://ai.thesatyajit.com/articles/vllm-omni-serving.md): vLLM-Omni's technical report pitches one serving runtime for speech assistants, image and video generators, world models and robot policies. I read the orchestrator, the shared-memory connector and both diffusion schedulers at c548a11, and checked the evaluation against its own tables. The stage graph is a sound idea and chunked streaming between stages is where the latency comes from. Every baseline in the paper, though, is vLLM-Omni itself, and one headline range understates its own table. (2026-10-08) [Inference & serving · Notable · 62/100 · Original analysis; Widely used]
- [Local coding models in Copilot on Windows: one model, three spec sheets](https://ai.thesatyajit.com/articles/windows-local-coding-models.md): Microsoft's 7 October Windows event put MAI-Code-1.1-Flash on the device inside GitHub Copilot, extended HydraFusion routing to local models, wired llama.cpp into Windows ML and made MXC sandboxing GA. I read the posts, the docs, the WindowsML, MXC and llama.cpp source and the demo video frame by frame. The sandbox ships today; the local model and local routing are an end-of-October experiment for 128 GB RTX Spark machines, and the model's size depends on which Microsoft page you read. (2026-10-08) [Inference & serving · Notable · 59/100 · Original analysis; Widely used]
- [Open d1: two decision models with one name, one API and two different readouts](https://ai.thesatyajit.com/articles/liquid-open-d1.md): Liquid AI's Open d1 is d1-3B (text and images) and d1-omni-600M (text with images or audio). I read both checkpoints by HTTP range request, their shipped code, their GGUFs and the Decision Index's own data files. d1-3B is LFM2.5-VL-3B to the parameter, 3,123,483,888 of them, its vision tower untouched, answering through its own LM head at the option letters, with a tree that reads the state once for every question and no temperature at all. d1-omni-600M is 587,161,089 parameters around a bidirectional encoder with a new 26M decision head that scores one mask marker per option, and it re-reads the state for every question. The 48.57 that makes d1-3B 'first under 10B' recomputes from its area scores, but it is a self-run on Decision Index 0.2.1, which the board replaced with 0.3 a day before launch; on 0.3's public part three smaller models already score higher. And the GGUFs declare decision types that llama.cpp's master does not know, so 'day-one llama.cpp support' was not upstream when I looked. (2026-10-07) [LLM architecture · Notable · 67/100 · Checked against the source; Runs on a consumer GPU]
- [Agent Memory Repo: Devin's dreaming memory is a git repo of one-line bullets](https://ai.thesatyajit.com/articles/agent-memory-repo.md): Cognition shipped Memory and Dreaming in Devin and open-sourced the format underneath as Agent Memory Repo: a git repository of Markdown notes with a MEMORY.md entry point, one fact per bullet, [key: value] metadata and [[path]] links. I read the whole 2.8 KB spec, the skill and both launch pages, and ran the concurrency cases through real git. The graph is file-level wikilinks, the history is git log, and there is no frontmatter, no tombstone and no validity interval. Two concurrent appends to one file conflict, an older session can bring a deleted fact back without a conflict, and dreaming itself is not in the open repo. Supermemory's MIT-licensed memoryrepo is the first open dreamer for the format: a four-hourly agent that merges and deletes by prompt, drops itself on any same-file conflict with chat, and meters cost but not quality. Compared with Claude Code's auto memory, Letta's context repositories, Zep's Graphiti and Mem0. (2026-10-06) [Agents & harnesses · Essential · 76/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [Interfaze 1 Lite: the mixture of architectures is ten public checkpoints and a tool loop](https://ai.thesatyajit.com/articles/interfaze-1-lite.md): interfaze-1-lite is pitched as the first general open-weight model for deterministic work, built on a 'Mixture of Architecture'. Reading the Hugging Face bundle shows what that means: 76 of its 79 large files are byte-identical to eight public checkpoints (Qwen3.8-27B-FP8, Chandra OCR 2, Whisper large-v3 turbo, SAM 2.1, three PaddleOCR models and pyannote), with Llama Guard 3 and TimesFM 2.5 pulled from outside it. The new work is orchestration (about 4,500 lines of Python in the bundle, 9,300 in the GitHub server): a tool-calling loop on the 27B core, an OCR stitcher that puts the document reader's text on the Paddle detector's boxes, and greedy decoding everywhere. The confidence scores are real for OCR lines, and they come from the 1.9M-parameter Paddle recognizer, not from anything new. Updated after Datalab's founder objected in public on 2026-10-07: the repo has since swapped its Apache-2.0 tag for a custom licence file that defers to each upstream, the Chandra OCR 2 weights are still byte-identical, and that model's modified OpenRAIL-M licence carries revenue, funding and non-compete limits an Apache tag never could. (2026-10-06) [Vision & multimodal · Notable · 64/100 · Original, source-checked analysis; Concrete numbers to act on]
- [Restore and relight LoRAs: teaching a generator to start from your picture](https://ai.thesatyajit.com/articles/restore-and-relight-loras.md): Two adapters released a day apart turn big generators into editors. Lightricks' Restore IC-LoRA takes an archive clip and renders it clean and in colour on LTX-2.5; RunningHub's Qwen-Image-2.1 lighting LoRA takes a pasted-in product and relights it into the scene. I read both cards, both safetensors headers (or, for the gated one, its byte count), the LTX-2 trainer and pipeline code, DiffSynth-Studio and ComfyUI. The restore file is 1,711,643,914 bytes, which matches rank 128 on all six attention modules of all 48 blocks, 855,638,016 parameters, to within 9,127 bytes; more than half of it sits in audio branches a video-only run never trains. The relight file is 83,886,080 parameters at rank 32 on every linear layer of Qwen's 32 blocks, trained on ModelScope at 3e-05 for 2,000 steps, with no word on its data. Both work the same way: the reference rides in the transformer's own sequence, clean, and the LoRA learns what to copy from it. And both paint what the input does not contain: the restore card says so itself, and the relight demos change a logo, a finish and a drink's colour. A week later the same RunningHub author shipped a second Qwen file with the identical 448-tensor layout, a 'seamless blending' LoRA trained separately at 1e-4 for 5,000 steps that keeps the product where it is and re-angles the room to its vanishing point, and redraws parts of the product while it does. (2026-10-06) [Image & video generation · Notable · 63/100 · Original analysis; A lasting reference]
- [MiMo-V2.6 RL environments: 7,780 tasks, five graders, and two judges you bring yourself](https://ai.thesatyajit.com/articles/mimo-rl-environments.md): Xiaomi released the RL environments behind MiMo-V2.6 as a Hugging Face dataset, 3,764 Docker images and a verl fork. I counted the files: 7,780 tasks in five domains (2,698 code, 1,000 cyber, 989 knowledge-work and terminal, 2,093 web dev, 1,000 music) living in 4,688 distinct environments, and I read all five graders. Code and cyber are graded deterministically. The knowledge-work rubrics are 86.6% LLM-judged and web dev is a group-relative vision pick, and for both you supply the judge model. The music scorer never reads the brief. Not released: the validation splits, the internal eval sets, the production judge and the RL-trained 9B checkpoints. (2026-09-26) [Training & RL · High · 71/100 · Original, source-checked analysis; A lasting reference]
- [DistScene: the whole room in one voxel grid, then a grid of its own for every chair](https://ai.thesatyajit.com/articles/distscene.md): DistScene turns one photo into a 3D scene whose environment and objects are separate meshes. It fine-tunes TRELLIS.2 with rank-32 LoRA to generate the room and the objects together, in up to twenty slots of one shared sparse-voxel frame, then re-generates each object in its own cube, conditioned on the scene, and puts it back with a scale and a translation. All of its training scenes are assembled from TRELLIS.2's own outputs. I read the authors' 23 full-resolution result files: every object is its own node with a uniform scale and a translation and no rotation, which is exactly what 'editable' means here, and every component, a lamp or the whole room, ships with just under two million triangles. The benchmark wins are real but measured on inputs the authors re-rendered, against baselines they re-ran, and the full-quality setting costs 38.5 seconds per object. (2026-10-07) [3D & spatial · Solid · 51/100 · Original analysis; A new technique]
- [FBTriton table-batched embeddings: the backward win is a threshold moved from 32 to 256](https://ai.thesatyajit.com/articles/fbtriton-table-batched-embeddings.md): Meta rewrote the table-batched embedding (TBE) kernels that every TorchRec recommender trains on, from FBGEMM's CUDA templates into Triton, and the PyTorch post reports a 1.28x median forward and, on X, a 2x backward. I read the post, the Triton TBE in TorchRec and the CUDA templates in FBGEMM. The typical backward gain is 1.13x unweighted and 1.22x weighted; the big wins sit in runs of 32 to 255 lookups, where CUDA hands a row to a cooperating block and Triton keeps streaming it with one program, and in FBGEMM that switch point is a single constant. The code is public, but at the commit I read it imports an internal Meta package at module load. (2026-10-07) [GPUs, kernels & systems · Solid · 58/100 · Original analysis; A lasting reference]
- [Impeccable: a design vocabulary for agents is mostly a linter with opinions](https://ai.thesatyajit.com/articles/impeccable-design-skill.md): Impeccable, Paul Bakaus's design skill for coding agents, sells itself as the missing design vocabulary. I cloned the repository and read it end to end. It contains 41 reference files of instructions, about 60,000 words, routed through one SKILL.md, plus a 212,000-line Rust engine that checks 59 anti-pattern rules. The engine works on source, on static HTML and in a real Chrome. Of those 59 rules, 33 are taste written down as thresholds. The interesting work is in how the thresholds were tuned: each rule's findings were judged on real sites, and rules that turned out harmless were retired. I then ran the skill's own critique, audit, distill, clarify and polish passes on this page's components, set the page itself by its typeset, layout and colorize references, and ran its detector before and after. (2026-10-07) [Developer tools & infra · Essential · 78/100 · Analysis found nowhere else; Runs on a laptop CPU]
- [KV caching in flow models: a cache you have to train for](https://ai.thesatyajit.com/articles/kv-caching-flow-models.md): Sayak Paul's long post on KV caching in flow models asks how a diffusion transformer, which re-runs every step on changing latents, can cache anything. I read the diffusers code behind FLUX.2 Klein KV and Qwen-Image-2.1 to find out. The cache is exact only when the cached tokens never read the noisy latents and never see the timestep, which is a property of the trained model and not something you can switch on. A FLOP model built from the configs lands 1.8 to 4.3 points above all five of his A100 measurements. Klein's reference cache never reads the prompt, his text-KV variant is an approximation, and his TaylorSeer result has no TaylorSeer-only control. (2026-10-07) [Image & video generation · Notable · 66/100 · Interactive explanations; Original analysis]
- [Malena: how little harness a strong ML-engineering agent needs](https://ai.thesatyajit.com/articles/minimal-ml-engineering-agent.md): An EPFL and Apple paper (arXiv 2609.40303) holds the model, the GPU and a 24-hour budget fixed, then climbs from a chat call to a coding-agent session to search trees and parallel agents on MLE-bench. One plain OpenCode session with three small tools, called Malena, medals on 62.5% of 30 Kaggle tasks with GLM 5.2 against 47.1% for the best of four released harnesses. I checked the viral summary against the tables: the four systems are MLEvolve, AiScientist, Arbor and ScienceFlow, run as released rather than rebuilt in one codebase; 62.5% and 55.7% come from different task splits; and the multi-agent collapse to 33.3% is mostly a failure to pick the right submission, not to make one. The error bars resample seeds and hold the tasks fixed, so I worked out how wide they get when the tasks move too. (2026-10-07) [Agents & harnesses · Notable · 65/100 · Original analysis; Widely used]
- [OpenQodex: thirteen scanners, and a reviewer that cannot skip a finding](https://ai.thesatyajit.com/articles/openqodex-code-review.md): OpenQodex is an Apache-2.0 code reviewer that runs before you push. It picks the scanners that fit your changed files, keeps only hits on the lines you changed, then starts a separate, read-only Claude Code or Codex process that must raise or drop every hit, with a script that sends the answer back until it does. I read all of it. The launch's '100+ scanners' is thirteen. The push gate warns by default, and an incomplete review never blocks. (2026-10-07) [Developer tools & infra · Notable · 63/100 · Checked against the source; Concrete numbers to act on]
- [Squeeze3D: a borrowed 3D generator as the decoder, and what its 2,187x is made of](https://ai.thesatyajit.com/articles/squeeze3d.md): Squeeze3D (TMLR 2026) compresses meshes, point clouds and radiance fields by training two small mapping networks between a frozen 3D encoder and a frozen 3D generator, using only a latent-space MSE and a Gram loss, with no reconstruction loss and no 3D dataset. I read the paper, the code and the released weights. The Gram loss is a clean idea: penalising the batch-by-batch Gram of normalised codes is the same as penalising the dimension-by-dimension one, which is the Barlow Twins and VICReg redundancy term in another form, and it maximises the paper's own effective-rank measure. The ratios need reading closely: the 58.5x for point clouds is measured against ASCII PLY files (12x against binary floats), the 'more than 650x' for radiance fields comes from a rounded denominator, the released radiance-field network carries a skip connection 128 times wider than its code, and the parameter table is ten times too small for seven of the eight checkpoints. (2026-10-07) [3D & spatial · Essential · 80/100 · Original, source-checked analysis; Interactive explanations]
- [Unsloth's decision-model trainer: a Clef head on any LLM, and what 20.7% to 74.3% measures](https://ai.thesatyajit.com/articles/unsloth-train-a-decision-model.md): Unsloth now trains Jev-style decision models: a plain LLM gets a freshly initialised copy of Cloudflare's Clef joint schema head and LoRA, trained with soft-target cross-entropy on typed questions. I read the trainer, the head, the dataset builder and the notebooks. The post's two headline numbers are different measurements: 20.7% to 74.3% is the unweighted mean of three benchmarks for the 0.8B, while 30-37% to 78% pairs typed-decisions starting points with the best single cell. Every 'before' number is a randomly initialised head, not Qwen's ability. BANKING77 and CLINC150 are trained on their own train splits. The head is 27,324,932 parameters on the 0.8B, and 4 GB is plausible because the vocabulary logits never exist. (2026-10-07) [LLM architecture · Notable · 62/100 · Original analysis; Runs on a consumer GPU]
- [EmbeddingGemma 2: the encoders are optional, the text model is not](https://ai.thesatyajit.com/articles/embeddinggemma-2.md): Google DeepMind's EmbeddingGemma 2 puts text, code, images, video and audio into one 768-dimensional space with 740M parameters, under Apache 2.0. I read the safetensors header, the transformers code and the processors. The 740M is real (744,371,992 in the file): a 271M text encoder that every input passes through, plus a 168M vision tower and a 306M audio tower that only turn pictures and sound into tokens for it. 'Modular' means those two towers can be left unbuilt, and a text embedding comes out the same either way. The audio tower is byte-for-byte Gemma 4 E2B's. The 'twice its size' line holds against the models Google picked, but on Google's own audio chart a sub-1B model (non-commercial) sits higher, the code chart plots EmbeddingGemma 2 about two points below its own card, and the 327 seconds of audio and 58 video frames are context arithmetic that the shipped processors cut to 30 seconds and 32 frames by default. Update, 7 October: Qdrant's 77x storage claim checks out byte for byte (40-byte vectors at 256 dimensions and 1 bit), but it counts vector RAM only, not the HNSW graph or the originals its 94.5% needs for rescoring, and the 1-bit path it credits to TurboQuant is RaBitQ's by Qdrant's own documentation. (2026-10-06) [Vision & multimodal · Essential · 88/100 · Original, source-checked analysis; Runs on a phone]
- [GTA V in a browser tab: Rockstar's own engine, recompiled to wasm64 and WebGPU](https://ai.thesatyajit.com/articles/gta5-in-the-browser.md): playgta5.com runs Grand Theft Auto V in Chrome from Rockstar's own RAGE engine, compiled with Emscripten into a 63 MB wasm64 module with about 91,000 named functions. Direct3D 11 is replayed on WebGPU from a 32 MB command ring of 48 opcodes, 4,919 shaders were translated offline to WGSL, and 20.9 GB of game files stream in 4 KB blocks through HTTP Range requests into an Origin Private File System store. Nobody squeezed the game into 700 MB: a session pulls only what the engine reads, 582 MB of recorded boot reads and then the world around you. I read every file it ships, pulled sample game files apart and ran it headless on a CPU-only server. This is how each piece works, with the code. (2026-10-06) [GPUs, kernels & systems · High · 74/100 · Original, source-checked analysis; Interactive explanations]
- [Kandinsky 6.0 Video: two token streams on one clock, and where the lip-sync comes from](https://ai.thesatyajit.com/articles/kandinsky-6-video.md): Kandinsky Lab's open (MIT) audio-video diffusion models generate a 5-second clip and its 44.1 kHz soundtrack together: a video transformer carried over from Kandinsky 5.0 and an audio transformer trained from scratch, joined in every block by cross-attention both ways. I read the paper, the code and the safetensors headers. Lite is 3,176,632,424 parameters and Pro 30,139,423,760 (measured), not 29B; the cross-attention between streams is 18.7% of either. A clip is 50,220 video tokens and 218 audio tokens, and the two meet on RoPE positions scaled by 0.144. There is no lip-sync module: it comes from filtered data, a face-weighted loss, and an RL stage whose headline 47% WER cut is scored by Whisper-large-v3, which is also one of its rewards. On a held-out sentence set the cut is 28%, and on VABench's own speech check the RL'd Pro still has the highest WER of the three models tested. Updated for FastVideo's port: how its pi-Flow scheduler spends ten network calls, and a latent upscaler bank larger than the SR model it feeds. (2026-10-06) [Image & video generation · Notable · 59/100 · Original analysis; Runs on a consumer GPU]
- [Limite 1B - Violetto: the throughput claim is in config.json, not the blog](https://ai.thesatyajit.com/articles/limite-1b-violetto.md): Paradigma's README calls Violetto 'a model for high-frequency mathematical intelligence' and nothing it shipped publishes a tokens-per-second figure. What does exist is config.json: 36 of 48 blocks run a 1,024-token sliding window and 12 run global attention with no positional encoding, which cuts the KV cache at 131,072 tokens from 6.44 GB to 1.65 GB, 3.91x, and is the only derivable part of the throughput claim. The technical report (6 October) confirms that reading, adds what the config could not show (Exclusive Self-Attention, gated value embeddings, a sparse MUDD mixer at 2.3% step-time cost, NorMuon on 48 H100s for 236B tokens) and lays out the post-training: distillation from VibeThinker-3B, DPO, three DPPO RL runs, two short SFTs and three model soups, with a learned critic in the last run. Read with every checkpoint at the same 131K budget, its own appendix shows the first two RL runs mostly bought shorter answers. On the headline chart RL is only about 5% of compute, so leaving it out barely matters; leaving out the teacher does. The student ties VibeThinker-3B on AIME 2026, and the released eval suite runs six of the seven benchmarks, missing ArXivMath, the one where Limite trails. (2026-09-22) [LLM architecture · High · 68/100 · Interactive explanations; Original analysis]
- [A 320B MoE on a free Kaggle TPU: I audited the bytes, not the tok/s](https://ai.thesatyajit.com/articles/kaggle-tpu-320b.md): kaggle-tpu-lab puts GLM-5.3-Flash — 320B total, 18B active — on Kaggle's free TPU v5e-8 with 5,562 lines of hand-written JAX and Pallas, and quotes ~64 tok/s. The memory claim survives a byte-level audit: I walked the GGUF tensor index and got 109.87 GB of expert weights against the README's 110, 110.72 GB after the planar repack, 13.84 GB of the chip's 16.9. Three things do not survive. The README calls 262,144 tokens 'the model's native' context when config.json says 1,048,576 and it is really the engine's own cap; the decode kernel's cost is described as 'the number of distinct experts a step touches' when its Pallas grid is one step per (token, expert) pair with no deduplication anywhere; and the quantization the project says it has not yet compared to bf16 turns out, when I download matched slices of both and measure, to sit 25% away in relative Frobenius error on the matrices 41 of 42 layers use. Update, 7 October: the same repository now serves Qwen3.8-27B in full bf16; I check its 55.6 GB, its 262K context, its 130 tok/s (a speculative-decoding rate; the plain roofline is 134) and its prefill numbers, which agree once attention's quadratic cost is counted. (2026-09-18) [GPUs, kernels & systems · High · 73/100 · Original, source-checked analysis; A new technique]
- [FastH3 Preview v1: four calls, one-tenth the attention, and what the 14x actually measures](https://ai.thesatyajit.com/articles/fastvideo-fasth3.md): FastVideo distills MiniMax's 33B H3 video-audio transformer into a 4-step, 90%-sparse generator and calls the recommended checkpoint Data-Free because no video, real or synthetic, is ever loaded during training. The headline 14.38x is real, measured on a single Blackwell GPU against the same Blackwell GPU — but it is one cell in a table that ranges from 6.65x to 14.38x, and 'sub-realtime generation' turns out to hold for exactly one of the three clip lengths FastVideo tested. A checkpoint-level accounting plus a dig into the FastVideo framework's own source: the two architecture-specific CUDA kernels behind 'on Blackwell,' the opt-in switch that actually selects between them, what the real DMD2 training step runs that the metadata doesn't show, and how much of the roadmap's Apple Silicon and DGX Spark 'interest' is already shipped code. Updated for the October consumer release: how V2 and the pruned FastH3 Trim fit in 8 to 32 GB, checked against their safetensors headers, including the 294 calibrated FP4 activation scales read back from the file. (2026-08-29) [Image & video generation · Notable · 60/100 · Original analysis; Runs on a consumer GPU]
- [3D reconstruction roundup 2: old geometry, bolted onto feed-forward models](https://ai.thesatyajit.com/articles/3d-reconstruction-roundup-2.md): Six 3D releases from the first week of October 2026, and the classical idea each one adds to a feed-forward network: a coarse-to-fine cascade for one fixed-size generator (T3lescope), an exact Gaussian budget (PocketSplat), loop closure on SE(3) (CLoSeR), a gravity prior (G3T), edge-based bundle adjustment without tracks (EPO) and a panorama a VLM can read (OneCanvas). Read against their own tables: T3lescope's 'no per-scene optimisation' takes 89 minutes a scene and Depth Anything 3 still has the better F-score; PocketSplat's budget sets the size of the asset, not the compute; CLoSeR cuts LoGeR's average error on VBR from 36.16 m to 11.15 m, which is less drift, not none; OneCanvas fine-tunes the VLM with LoRA and reads ground-truth depth and poses. (2026-10-06) [3D & spatial · Notable · 59/100 · Original analysis; A new technique]
- [Gaussian-splat capture in practice: from a 360 camera to a scene on a map](https://ai.thesatyajit.com/articles/3dgs-capture-pipelines.md): How people actually capture and ship Gaussian-splat scenes this autumn, read from five practitioners' posts, their threads, and the code behind the tools they name. A 360 camera on a drone or a stick, frames pulled and masked, structure from motion on the equirectangular panorama (because a dual-fisheye file makes the solver discover the rig on its own, and on a 2 km shrine walk it would not), cube faces for training, real scale from AprilTags or GPS, then LichtFeld Studio or Spirula Studio, then a renderer. Per stage: why the choice is made, the numbers each source gave (875 frames, 15M splats at SH2, 481,300 iterations in about 14 hours on an RTX 4090; about 5,000 8K panoramas in one pass), and what I can check in the code: LichtFeld's step auto-scale, what MRNF and PPISP actually do, why SH3 does not fit, and why PlayCanvas's stochastic splatting draws a frame in 7.5 ms on a Pixel 7 Pro when the sort alone takes 19.5. (2026-10-06) [3D & spatial · Notable · 62/100 · Original analysis; Runs on a consumer GPU]
- [Genex, ffmpeg-skill and rdsh: three harnesses, read for where they check the agent's work](https://ai.thesatyajit.com/articles/agent-tools-week.md): Three open-source tools went round this week that wrap a coding agent around a domain: Genex, a desktop app that builds Three.js games on your Claude Code or Codex subscription; ffmpeg-skill, 42 FFmpeg scripts behind a probe-plan-run-look workflow; and rdsh, a Rust launcher for DeepSeek's dsh. I read all three at source level without running them. Genex's 'self-improving harness' is concrete: the agent may rewrite its own skills, tools and loop code, but only through a gate that type-checks and boots a copy first, and a SkillOpt port that keeps a skill edit only if it wins two of three blind votes on tasks the analyst never saw. ffmpeg-skill's 'auto-verify' is two layers: the code proves every output is a readable media file, and the skill tells the agent to compare the probed numbers and open a contact sheet. rdsh's 81x is real arithmetic on a different question: its --version is printed by clap while dsh's boots Node, and its search speedup is measured against an older rdsh, not against dsh. (2026-10-06) [Agents & harnesses · Notable · 59/100 · Original analysis; Runs on a laptop CPU]
- [Agr: a decision model is Gemma's own output layer, read at one position per question](https://ai.thesatyajit.com/articles/agr-decision-model.md): Command Code's Agr (31B) and Agr-flash (360M) answer typed questions with a probability per option and no generated text, and claim 58.15 on Decision Index 0.2.1, a quarter of a point above Jev. I read both checkpoints through HTTP range requests and the code behind them. Agr is Gemma 4 31B IT minus its vision tower, 30.70B parameters, with every linear projection nudged by about 0.1 to 0.7% and the embeddings, norms and output layer bit-identical; it scores options with Gemma's own output layer at one position per question, in branches that cannot see each other, and divides by a fitted temperature near 1.2. Agr-flash is SmolLM2-360M, heavily retrained, plus a 496,320-parameter query/key head. The 58.15 is Command Code's own run, not on the board, and the launch chart leaves out the closest comparison: the same Gemma checkpoint with no training at all, read the same way, scores 57.33. Against it, the fine-tune moves Tools by 0.3 points. Agr-flash has no published score. (2026-10-06) [LLM architecture · High · 69/100 · Original, source-checked analysis; Concrete numbers to act on]
- [ALoDLM: a diffusion LM that loops on its hard tokens, read against its own tables](https://ai.thesatyajit.com/articles/alodlm-looped-diffusion.md): Amazon AGI's ALoDLM turns Qwen3 into a masked diffusion LM whose middle 16 layers run up to 4 times inside every denoising step: confident tokens commit and feed back as embeddings, unresolved ones keep refining their hidden state, and a learned exit gate decides when the step ends. I read the code, the weights and the paper. The mechanism is real and well engineered. The headline, beating Qwen3-8B 80.3 to 78.5 on average, is measured in a one-token-per-pass mode against a parent that never saw the 5B-token math-and-code SFT corpus, with no same-data control; at 1.7B the whole margin comes from code and GPQA. The 2.7x speed-up is single-stream on one B200, and by the paper's own FLOP formula each ALoDLM token costs about 8.7 times the arithmetic of a Qwen3-8B token. (2026-10-06) [LLM architecture · Notable · 63/100 · Original analysis; Runs on a consumer GPU]
- [AnyAPI's Scrape router: a ladder of scrapers, priced by the rung that gets through](https://ai.thesatyajit.com/articles/anyapi-scrape-router.md): AnyAPI launched a scrape router that sends each page to the cheapest of five anonymous scraping providers and fails over to the next in the same call, claiming 98 of 100 pages at $1.05 per 1,000 against Firecrawl's 90 at $3.97. I recounted the founder's own benchmark from its per-page CSV and every number reproduces. The 8-page lead comes from 6 sites, 3 of the pages from Yelp alone, and AnyAPI got every page any rival got. The price gap is mostly a plan choice: the gist prices Firecrawl at its Hobby rate, and at its Standard rate the same run costs about $0.87 per 1,000 real pages. Replaying the CSV through a ladder built from the four rivals gets 95 of 100 at about $1.23, which says the routing idea is sound and that the router, not any one backend, is doing the work. (2026-10-06) [Developer tools & infra · Notable · 59/100 · Analysis found nowhere else; Concrete numbers to act on]
- [arc-driver: the 8x is a one-second wait, the 1.7x is a round trip](https://ai.thesatyajit.com/articles/arc-cua.md): arc-driver is a new open-source macOS driver for computer-use agents that claims up to 8x over cua-driver and, inside Claude Code, about 1.7x faster and 30% cheaper on real tasks. I read both drivers' source. The 5x-8x per action is mostly a one-second poll for new windows that cua-driver runs after every click; the end-to-end gap is mostly a model round trip, because arc's action result already contains the next observation and cua-driver's does not. Neither the benchmark result files nor the Claude Code task list are published, so every headline number here is reported, not re-run. (2026-10-06) [Agents & harnesses · Notable · 59/100 · Original analysis; Runs on a laptop CPU]
- [A 200 GB MoE on one RTX 3090: the experts live in RAM, the speed lives in the CPU](https://ai.thesatyajit.com/articles/big-moe-one-3090.md): Two posts the same week ran very large mixture-of-experts models on a single 24 GB RTX 3090 by keeping the routed experts in system RAM. 0xSero's DeepSeek-V4.1-Flash build is not llama.cpp: it is vLLM with a 3.0-bit EXL3 quant, 190.5 GiB of experts pinned in eight-channel DDR4, a VRAM cache of the hottest experts and an AVX2 CPU tier for the misses, measured at 19-21 tok/s for one stream. I recompute every byte from the model configs, read the per-layer split the CPU tier runs, and find that the DDR4 bus is not what sets the speed: at the repo's own cost constants the CPU reaches about a third of the bus's peak, and the arithmetic of decoding a codebook quant is the long pole. The same holds for a Qwen3.8-Flash-Next IQ3_S run on DDR5, whose author found DDR4 and DDR5 almost the same. (2026-10-06) [Inference & serving · Essential · 79/100 · Analysis found nowhere else; Interactive explanations]
- [Token cues: how 'chicken' made a base model reason as well as RL](https://ai.thesatyajit.com/articles/chicken-reasoning-cues.md): Wang, Dravid et al. (arXiv 2610.06851) show that forcing the first two tokens of a base model's answer, '.\n\nOkay' for Olmo-3-7B, lifts MATH-500 pass@1 from 42% to 78%, past the 75% of its RL-Zero counterpart. RL's own changes sit almost entirely in those first positions: a base model that continues an RL model's opening scores 77.7%, and an RL model that continues a base opening scores 43.7%. They trace the cue to mid-training data, where 'Okay' opens 83% of synthetic reasoning traces, and prove it causally by renaming 'okay' to 'chicken' in the data and retraining: the chicken cue goes from 2.4% to 37.2% at 10B tokens, and from about 18% to 76.9% over the full 100B. I read the paper, its appendices and the released code. The finding holds up and it is narrower than the headline: one R1-Zero-style setting, mostly Olmo, no effective cue at all for Llama-3.1-8B, and the data-edit code is not in the repository. (2026-10-06) [Training & RL · Notable · 60/100 · Runs on a consumer GPU; Widely used]
- [Claude's 104 product demos: two clocks, a phoneme-timed mouth and one ffmpeg join](https://ai.thesatyajit.com/articles/claude-demo-videos.md): Mohd Danish had Claude Opus 5.5 drive a real Chrome through 52 extension tools with Playwright, voice each demo with Kokoro, lip-sync a mascot from Kokoro's word timings, draw the frames in HTML and join everything with ffmpeg: 104 videos in 40 minutes. No code was shared, so I read the published frames, Playwright's recorder and Kokoro's duration predictor, ran Kokoro to get real phoneme timings, and set the pipeline against this site's own film engine, which computes those same timings on every line and throws them away. (2026-10-06) [Image & video generation · High · 70/100 · Analysis found nowhere else; Interactive explanations]
- [Cloudflare OS: approve the agent's writes after it has finished](https://ai.thesatyajit.com/articles/cloudflare-os.md): Cloudflare open-sourced Cloudflare OS, the agent workspace thousands of its employees use. Its security layer, Gatekeepers, sits between agent code and every external service: reads run and are logged as observations, writes are queued and simulated so the agent keeps working, and a human approves the batch later. I read the kernel and the gatekeepers at commit b304e8c. The capability model is real and the sandbox is a V8 isolate with no outbound network, but the promise not to write before approval is kept by each gatekeeper's own code, not by the kernel, and some gatekeepers can't simulate, so the agent's turn stops and waits. (2026-10-06) [Agents & harnesses · High · 70/100 · Original analysis; Widely used]
- [CPU Performance Engineering: a reading list that brings its own benchmarks](https://ai.thesatyajit.com/articles/cpu-performance-engineering.md): usamahz/cpu-performance-engineering is 304 primary sources in reading order plus fourteen C microbenchmarks, each run on an Apple M4 Pro with its raw output committed. I read all of it and ran none of it. The benchmarks teach the core mechanisms well: a branch mispredict costs 19.4 cycles, an L2 hit 8.9 times an L1 hit, and four accumulators are not enough to fill four FMA pipes. Two of the repo's own benchmarks disagree about the NEON peak, one index row contradicts its benchmark, and the run that would matter most for llama.cpp, a batch-one GEMV, is proposed in the notes and never built. (2026-10-06) [GPUs, kernels & systems · High · 74/100 · Interactive explanations; Evergreen reference]
- [Dust: every token is a member of the population](https://ai.thesatyajit.com/articles/dust-zeroth-order.md): Q Labs' Dust pretrains a GPT-style transformer with no backward pass. It jitters each linear layer's output independently at every token, rewards each token's noise by that token's own loss change, and turns the reward-weighted noise into a weight gradient with the same outer product backprop uses. One forward pass then evaluates thousands of perturbations instead of one. This is the estimator from first principles, a toy you can run that shows why per-token rewards are the whole trick, what the released code actually does, the loss ladders checked against the appendix tables, and what it costs: at the 16k population, my estimate is a few thousand times backprop's compute per update. (2026-10-06) [Training & RL · Solid · 58/100 · Original analysis; A lasting reference]
- [Gaussian-splat colour: 192 bytes of spherical harmonics, or 28 bytes and a tiny MLP](https://ai.thesatyajit.com/articles/efficient-gaussian-appearance.md): Hahlbohm et al. put spherical harmonics, Spherical Voronoi, NASG, NASGabor and their own compact neural appearance model into one Faster-GS pipeline, each fused into the CUDA rasterizer, and measured all of them on 21 scenes, three desktop GPUs, a laptop, a tablet and a phone. The neural model stores 3 fp32 base-colour values and 8 fp16 features per Gaussian (28 bytes against 192 for degree-3 SH) and decodes them with an 816-weight MLP that still takes the view direction as SH basis values. I read the CUDA for every model, recounted the byte figures from the parameter layouts, and split the Mip-NeRF 360 table by scene type: SH still has the best outdoor PSNR, and every alternative's lead comes from the four indoor scenes. Then the paper's more interesting point, from the thread: an expressive appearance model also fits footsteps in the grass and a book that moved, so which state of the scene should a capture reconstruct? Last, what this means for a 360-camera capture pipeline: training memory, what the PLY export drops, and why SOG-style SH palettes still win on delivery. (2026-10-06) [3D & spatial · Notable · 62/100 · Original analysis; Runs on a consumer GPU]
- [Ego2Act and EgoTools: two egocentric benchmarks that grade state, not pixels](https://ai.thesatyajit.com/articles/egocentric-video-benchmarks.md): Two new first-person video benchmarks, one for generators and one for VLMs. Ego2Act gives a video model a start frame and a goal and grades whether its rollout does every required step with plausible physics, using a gated rubric and a reference-free Gemini judge. EgoTools asks VLMs 1,000 eight-choice questions about real tool use. I re-derived Ego2Act's human leaderboard and its 0.69 judge agreement from the released score tables, pinned down what that 0.69 actually measures, and found where both papers' numbers need a footnote. (2026-10-06) [Evals & benchmarks · Essential · 76/100 · Original, source-checked analysis; A lasting reference]
- [EVA: a VAE that samples well once its prior predicts the next latent](https://ai.thesatyajit.com/articles/eva-empirical-vae.md): Empirical Variational Autoencoder (arXiv 2610.06545, University of Tokyo) turns a VAE into a sequence generator by dropping the fixed N(0,1) prior. Instead, the causal decoder predicts each latent's prior from the latents before it. Next to a causal VAE, the only new weights are two 768-to-128 linear heads. I counted them in the released safetensors: 196,864 parameters, 0.11% of EVA-B. On ImageNet 256 that change takes FID from 100.27 to 7.68. Generation is a single 12-block decoder pass per token, against 24.76x the time for the diffusion-head baseline that reaches 5.44. I explain why a single Gaussian per step can be enough, then check the claims against the code and weights. Three things in the paper do not match the release: the block counts, the audio latent width, and an undocumented ramp in classifier-free guidance. The paper also leaves out the context: bidirectional models in its own appendix table beat it. (2026-10-06) [Image & video generation · Notable · 66/100 · Original analysis; Runs on a consumer GPU]
- [FreeVideo: a 66 GB video transformer on an 8 GB card, and the planner that decides where every block lives](https://ai.thesatyajit.com/articles/freevideo-minimax-h3.md): FreeVideo runs MiniMax H3 through the Video DeltaNet hybrid-attention checkpoint on as little as 8 GB of VRAM and 16 GB of RAM. I read the code and the safetensors headers. The released H3 transformer is 66.28 GB in BF16; 26.02 GB of that is AdaLN projections that a fixed eight-step schedule turns into 0.22 GB of tables, and FP8 takes the rest to 22.55 GB. That still does not fit, so a measured planner streams 432.4 MB blocks from host RAM or disk every step, slices attention into head groups sized to the clip, and keeps eight blocks resident because more bought nothing. Video DeltaNet is the speed lever, not the memory one. (2026-10-06) [Inference & serving · Notable · 63/100 · Analysis found nowhere else; Runs on a consumer GPU]
- [gdp-ts: making the authorization check a value the compiler can see](https://ai.thesatyajit.com/articles/gdp-ts.md): Guillermo Rauch's gdp-ts ports Matt Noonan's Ghosts of Departed Proofs to TypeScript: a sensitive function takes a proof about its exact arguments, the proof can only be minted by the module that ran the check, and forgetting the check, checking the wrong project or skipping the entitlement is a compile error. The library is 42 lines of code; the linter exists because TypeScript lets anyone write `{} as Proof`. I type-checked all 60 ways a caller can approach one sensitive call: three compile, and two of those are forgeries that only the lint preset catches. (2026-10-06) [Developer tools & infra · Essential · 79/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [H-JEPA: the top of the stack forgets the ant's legs, and that is the point](https://ai.thesatyajit.com/articles/h-jepa.md): H-JEPA stacks action-conditioned JEPA world models, each predicting twice as far ahead as the one below in a latent space of its own, trains every level end to end with SIGReg against collapse, and plans top-down by gradient descent on actions. Read against the paper, the code and the 93 released checkpoints: on Visual AntMaze a three-level model takes planning success from 18% to 73% at under 100 planner TFLOPs, and the paper splits that gain into two levers you can measure separately. The hierarchy does not help everywhere (Push-T falls apart at three levels), and the real-robot DROID result is offline and open-loop. Its best section is an appendix proving that SIGReg cannot see one kind of collapse at all. (2026-10-06) [Robotics & embodied · High · 71/100 · Interactive explanations; Original analysis]
- [The Hilti SLAM datasets: grading LiDAR SLAM against a steel tip on a surveyed cross](https://ai.thesatyajit.com/articles/hilti-slam-dataset.md): The Hilti SLAM Challenge datasets (2021, 2022, 2023) score SLAM against survey marks measured to the millimetre, not against another estimator. How the ground truth is made in each edition (total station, a terrestrial laser scan with a steel tip pressed onto floor crosses, then a LiDAR-readable floor target for a robot), how the banded score works and why it disagrees with RMSE, what the winners ran (FAST-LIO2 variants included), and what the new FiftyOne MCAP conversion of the 2022 set contains. I read the papers, the official ground-truth files and the evaluation script, parsed one MCAP episode over HTTP, and scored the dense reference trajectories against the survey marks myself: 7 to 76 mm. (2026-10-06) [Robotics & embodied · High · 68/100 · Checked against the source; Runs on a laptop CPU]
- [JEPA-Anything: one recipe for seven worlds, not one model](https://ai.thesatyajit.com/articles/jepa-anything.md): JEPA-Anything splits a JEPA's latent target into K orthogonal subspaces, gives each its own small predictor, and stitches the predictions back into one state with a pseudoinverse. The paper applies that recipe to vision, single cells, patient records, Pong, PDEs, weather, locomotion and molecules, and beats a matched monolithic JEPA almost everywhere. Checked against the paper, the code and the 36 released checkpoints: 'one world model' means one training recipe with separate weights per task, most gains over the matched JEPA are small, and the released projectors are far less orthogonal than the paper's geometry table suggests. (2026-10-06) [Training & RL · High · 69/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [Jev alternatives, week four: the week decision models learned where to run](https://ai.thesatyajit.com/articles/jev-alternatives-week-four.md): Week four of the decision-model wave is four posts, and all four are about where the model runs. Liquid's d1 now reads images behind a closed API, and its playground ships the raw file behind the launch chart: the 94%, 54-cent and 3.7-second averages reproduce exactly, but only by averaging six differently sized 'runs', and 'matches or beats GPT-6.1 Sol on four of six' holds only at rounding on one of the four. vLLM Semantic Router's Decision 2.0 is six open sizes, 0.6B to 27B, sharing one 1-to-5-million-parameter endpoint head with no fitted temperature, and its tables leave Jev out. FluidInference ported two of them to Core ML and labelled 1,000 GitHub issues at 15 a second, checked for parity but never for accuracy. And llama.cpp shipped /v1/systemone: one endpoint, six readouts, one of which reads every choice twice. (2026-10-06) [LLM architecture · Notable · 64/100 · Analysis found nowhere else; Runs on a consumer GPU]
- [KOLC+ puts Gaussian splats inside BIM/CIM: how each feature has to work, and how I would build one](https://ai.thesatyajit.com/articles/kolc-3dgs-bim.md): KOLG's BIM/CIM cloud KOLC+ now loads XGRIDS .lcc Gaussian-splat scans into the same space as Revit, IFC and point clouds, and runs its earthwork, section, placement, annotation and 4D tools on them. It is a closed product, so I worked out what each feature has to compute on splats that are not a surface: cut and fill by MLIT's point-height method (the release's own screenshot reads 217.328 m³ of fill on 698 cells of 0.5 m), sections as closed-form ellipse slices of the Gaussians, three-point alignment as an Umeyama similarity and why three points is a minimum and not a plan, and what the LCC container actually stores (32 bytes a splat, float32 positions, a LOD pyramid 1.88x its finest level). Then the other half: the same pipeline from open parts on a 360-camera stack, Spirula Studio (now with LiDAR alignment), gsplat, COLMAP, Revit's IFC exporter with IfcMapConversion, PDAL, ezdxf, splat-transform and a web viewer, with every licence read and the parts that still have to be written named. (2026-10-06) [3D & spatial · Notable · 63/100 · A guide you can follow today; Original analysis]
- [Kolibri-1: a 78B open MoE that does 3.46B of work per token and stretches to 1M](https://ai.thesatyajit.com/articles/kolibri-1.md): Aleph Alpha's Kolibri-1 is an Apache-2.0, English-German Mixture-of-Experts with 78.1B total and 3.46B active parameters, 384 experts per layer of which 6 run, and a 4:1 mix of sliding-window and position-free full attention that lets a 256k-trained model run at 1M tokens. I recount every parameter from the safetensors headers, work out the KV cache from the config, trace how the training scaled from the 30B 'Kolibri Origin' blog run to 768 B200s, and check the launch claims against the report's own tables. (2026-10-06) [LLM architecture · Notable · 59/100 · Original analysis; Concrete numbers to act on]
- [Leviathan: the flat 450 tokens is a card cap, and the 99% comes from a friendly benchmark](https://ai.thesatyajit.com/articles/leviathan-agent-index.md): Leviathan is a single Rust binary that indexes JSONL, CSV, SQLite or piped database output into one SQLite FTS5 file and answers an agent with five short, cited result cards. I read its Rust source, its data generator and its benchmark harness. The engine is BM25 inside SQLite with two good ideas: the entity being asked about becomes a synthetic token, so scoping is a posting-list intersection, and each card is assembled from capped fields plus the one sentence that matches the question. The flat ~450 tokens per answer follows from five cards and a 300-character cap, not from the data. The 99% top-5 figure is real for its benchmark, but that benchmark is synthetic, draws its questions from 65 phrasings that share words with the records, guarantees every question at least 4 relevant records, and boosts the same fields that define relevance. (2026-10-06) [Agents & harnesses · High · 68/100 · Analysis found nowhere else; Interactive explanations]
- [LiFT: a looped DiT that keeps improving past the loops it was trained with](https://ai.thesatyajit.com/articles/lift-loop-flow-transformer.md): LiFT (arXiv 2610.05538) turns the middle of a Diffusion Transformer into a shared core that runs K times inside every sampling step. The trick is the training target: each loop regresses a point on a straight line from the model's own first guess to the flow-matching velocity, indexed by a continuous depth coordinate, which is the same construction flow matching uses for time. A checkpoint trained with 2 loops runs at 16 and drops FID from 17.07 to 9.31. I walk through the maths, re-derive the paper's FLOP accounting from its own formula, and check the headline: the 52% inference saving is measured against a dense XL/2 sitting on its 250-step plateau, while the cleaner same-budget wins are smaller and still real. (2026-10-06) [Image & video generation · Notable · 62/100 · Original analysis; A lasting reference]
- [Linear attention's memory problem: four answers in one week](https://ai.thesatyajit.com/articles/linear-attention-state-roundup.md): Read as test-time regression, a linear-attention layer is an online regressor over past key/value pairs, and its fixed state is the regressor's weights. Four recent designs change a different part of that regressor. SwiLA (COLM 2026) fits a mixture of J linear maps. Triadic Gated DeltaNet lifts the state to a d x E x d tensor. Native Hybrid Attention runs one softmax over compressed slots and a sliding window. Percepta's Spotlight lets the state grow while each token touches 9 cells. This walks the regression view from first principles, does the state-size arithmetic per head, runs a toy recall experiment, and checks each claim against the paper, the config files and, for Spotlight, the one model it has released. (2026-10-06) [LLM architecture · Notable · 63/100 · Original analysis; A lasting reference]
- [Decision models in llama.cpp: what /v1/systemone actually evaluates](https://ai.thesatyajit.com/articles/llamacpp-decision-models.md): llama.cpp's new /v1/systemone endpoint answers Jev-style typed questions with no generated token. I built it at commit d7a695e, ran Kev-0.8B and Julia-1 on a CPU, and read every line between the JSON and the number. 'One forward pass' is per prompt: every question, and for lev every choice, gets its own, and a causal model evaluates the shared state once only while a free slot can take a copy — the blog's request evaluates 92 of its 130 tokens on Kev. Options cost tokens, not passes: 2 to 32 options took Kev from 70 to 437 tokens, and 64 were refused at 797 tokens by the 512-token batch. Julia-1, an encoder, re-reads the state for every question and runs its three-column head three times per token; past its 256-token head budget it truncates every option to four tokens without saying so, which turned 56 numbered queues into the same three words. The readouts are faithful to their upstreams, down to lev reading a 0-8 rating at the letters A to I. What the endpoint cannot do is make a small model right: on the playground's own lip-sync example Julia-1 says text to speech. (2026-10-06) [Inference & serving · Essential · 80/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [Land or Water?: reading a world map out of 16,200 one-word answers](https://ai.thesatyajit.com/articles/llm-land-or-water.md): Karpathy amplified an eval that asks a language model 'Land or Water?' for one latitude and longitude at a time, 16,200 times on a 2-degree grid, and plots the answers as a picture of the Earth. I traced it to Henry's 2025 'How Does A Blind Model See The Earth?', read the open-model reproduction's raw CSVs, and read Celeste's Claude chart back cell by cell. Her accuracies come back to within a point. Opus 5.5 with thinking gets 292 of 16,200 points wrong, and 82% of those sit on the one-cell coastal ring that holds 22% of the grid. Seventeen of them are lakes the answer key calls land. Every score has to be read against 71.1%, which is what saying 'Water' everywhere gets. And the grid buys the picture, not the score: thin it to 648 points and the same answers still score 99.6%. Then the reason it works: Gurnee and Tegmark found latitude and longitude as a straight-line read-out of Llama-2's activations, where an MLP probe adds almost nothing. (2026-10-06) [Evals & benchmarks · Notable · 68/100 · Checked against the source; Runs on a laptop CPU]
- [LOOM and Looped-DiT: looping a model more than twice](https://ai.thesatyajit.com/articles/loom-looped-moe.md): Open looped MoE recipes stop at two loops because a third does not pay for its FLOPs. LOOM (arXiv 2610.01153) traces that to two failures, hidden-state variance growth and routers that pick the same experts on every loop, and fixes each one. It reports a 700M MoE that is best at 5 loops under near-iso-FLOP and a 1.7B MoE that is best at 9 loops when compute is not matched. Looped-DiT (arXiv 2609.40305) loops the middle of a text-to-image diffusion transformer and finds that extra loops beat extra denoising steps at the same budget. I check both papers' FLOP accounting and read both codebases. The iso-FLOP result holds up. The 9-loop result spends about 9x the compute and 9x the KV cache, and takes 3 optimizer steps per batch where the baseline takes 1. (2026-10-06) [LLM architecture · Notable · 67/100 · Original analysis; Runs on a consumer GPU]
- [LoopCD: a looped transformer's first loop is its own amateur](https://ai.thesatyajit.com/articles/looped-contrastive-decoding.md): A looped transformer runs one block of layers several times and throws away every state but the last. Apple's LoopCD (arXiv 2610.02185) decodes the first loop's state too and pushes the final prediction further from it: contrastive decoding with the weak model already inside the strong one. It is training-free and costs one extra output pass, or none in the hidden-state form. The reported headline, Ouro-2.6B-Thinking on AIME 2024 going from 61.88% to 73.33% pass@1, is a full-depth result. The compute saving, 22.5% to 48.2% of forward FLOPs, comes from a separate experiment at half the loops, scored on multiple-choice benchmarks only. Both are real; the summary that went round on X fused them into one claim. (2026-10-06) [Inference & serving · Notable · 66/100 · Original analysis; Runs on a consumer GPU]
- [Level-of-Token Diffusion: a DiT that is told where the detail goes](https://ai.thesatyajit.com/articles/lot-diffusion.md): LoT Diffusion (Stanford and Google, arXiv 2610.05816) tiles a DiT's token grid with rectangles of 1 to 8 tokens a side, fine where a layout says detail matters and coarse elsewhere, and fine-tunes FLUX.2 klein and Wan2.1 14B to denoise the shorter sequence while still predicting a full-resolution velocity. The piece that makes coarse tokens work is borrowed from asymmetric flow: the network only has to see a projection of the noise, and the rest of the velocity is algebra. I walk through the layout rules, the Procrustes patch lift, the centre-indexed RoPE and the extent heads, then read the speed tables against sequence length: on 4,096-token images LoT's speedup always trails its token cut (1.56x for 1.89x fewer tokens) and a fixed cost of about 1.3 s caps it near 4.4x, while on 75,600-token video it beats the cut (3.53x for 3x). The image comparison is layout-conditioned on the held-out reference image, which is why LoT can beat the dense model's FID with fewer tokens. No code or weights are out. (2026-10-06) [Image & video generation · Solid · 55/100 · Original analysis; A new technique]
- [Mach-2 Additive Medium: 1.70 bits, a trellis, and a 102 GB table the bits leave out](https://ai.thesatyajit.com/articles/mach-2-additive.md): Syzygy's Mach-2 Additive Medium packs Qwen3.8-Flash-Next's 125.7B text parameters into 26.7 GB, and the headers confirm 1.6998 bits per weight. The 9.4x is right on its own terms, but it leaves out a 102.4 GB n-gram table shipped untouched at BF16; count it and the same files are 2.74x smaller than BF16. Despite the name, the codec is not additive multi-codebook quantization: it is a QTIP-style bitshift trellis, with five per-expert rates from 1 to 2 bits and QTIP's own hashed code on a 4-bit spine. Of the eight benchmark gaps against BF16, only Humanity's Last Exam is larger than its noise. (2026-10-06) [Quantization & compression · Notable · 62/100 · Analysis found nowhere else; Concrete numbers to act on]
- [Marin 535B-A23B: the half of a training run nobody usually shows you](https://ai.thesatyajit.com/articles/marin-535b.md): Marin's open 535B-total, 23B-active MoE has passed the halfway mark of an 18T-token run, and the team logged every change they made on the way: a MuonH norm bug an outsider found in week one, an attention gate that quietly shut itself in the early layers and earned a mid-run weight decay, a transport swap and new kernels that took MFU from 21% to 27% with expert drops near zero, a data-mixture swap, and a scaling-ladder forecast that referees all of it. I read the code, the issues and the W&B report to see what each change was, why it was made, and what it cost. (2026-10-06) [Training & RL · Notable · 59/100 · Original analysis; A lasting reference]
- [Meta Rebalancer: one expression graph, read as a MIP and as a fast local search](https://ai.thesatyajit.com/articles/meta-rebalancer.md): Meta open-sourced Rebalancer, the assignment solver behind Shard Manager, RAS and Taiji: about 40 million object-to-bin problems a day, P99 12 seconds on 265k objects and 3.2k bins. I read the solver at commit 1a8d7f8 and the OSDI'24 paper. The clever part is not either solver. Every node of the expression graph implements two things, an incremental delta evaluator and a MIP translation, so the same spec runs on HiGHS for a small problem and on a parallel, hot-bin-first local search for a huge one. A broken constraint becomes a 100 x violation + 10000 goal, the SQUARES balance formula is an exponent of 1.1, and Meta's own lesson is that faith in MIP optimality cost them a detour. (2026-10-06) [Developer tools & infra · High · 69/100 · Runs on a laptop CPU; A guide you can follow today]
- [Mistral Large 4: first among Western open models by 13 points, eighth overall](https://ai.thesatyajit.com/articles/mistral-large-4.md): Mistral announced Large 4, 'Le Chonk', on 6 October 2026: 1.05T parameters, 49B active, a 1.6B vision encoder, an API preview today and weights promised by the end of October. No config or weights exist yet, so I checked what can be checked. The headline holds: on Artificial Analysis's Intelligence Index it scores 38.4, nearly 13 points above the best open model from the US or Europe and four times Large 3's 9.3. It is also eighth among open models, behind seven Chinese ones, three of them running 18B active parameters or fewer. Mistral's own charts are more candid than its prose: they show GLM-5.3 ahead on Terminal-Bench, AutomationBench and Finance Agent, and three open models above it on the SciCode chart that the text calls state of the art. The cyber results are the real story. Large 3, whose files are public, turns out to be a DeepSeek-V3 block with Mistral's tensor names; I reproduce its 675.99B parameters to within 0.07B, and use it to work out what Large 4 will cost to hold in memory. (2026-10-06) [LLM architecture · Notable · 59/100 · Checked against the source; Widely used]
- [Multi-harness RL: training a model inside Claude Code, Codex and OpenCode without touching them](https://ai.thesatyajit.com/articles/multi-harness-rl.md): Hugging Face's multi-harness RL guide turns unmodified coding agents into RL environments by pointing them at a capture proxy that speaks four model APIs, forwards to vLLM, and records the exact token ids and logprobs the policy sampled. I read the proxy's source and recomputed the guide's claims from the Space's own data files. The headline numbers hold: LFM2.5-2.6B goes from 42.2% to 54.2% pass@1 across four harnesses, and the same base weights score 62.1% under Mini-SWE-Agent and 33.2% under Claude Code. The reward's 'small' tool-call bonus recovers as 1 + 1.5/(15 + calls), which GRPO's group normalization turns into advantages of ±1.3 when all eight rollouts are correct. Two posts about it mislabel one number: the 47.5% SFT result came from 801 OpenCode rollouts, and the run on all 3,189 rollouts scored 43.1%. (2026-10-06) [Training & RL · Essential · 76/100 · Checked against the source; Widely used]
- [CPLM in the NanoGPT speedrun: a pointer head takes 11% off the record](https://ai.thesatyajit.com/articles/nanogpt-speedrun-record.md): A pending NanoGPT speedrun record (PR #379, Nathan Godey and Yoav Artzi) trains GPT-2 to 3.28 validation loss on 8xH100 in 36.009 s against 40.575 s for record #92 rerun on the same node. The change is one old idea: a pointer over the document's earlier tokens, mixed into the output distribution through a spare vocabulary row and a learned sink, which buys 144 fewer steps for about 0.9% more time per step. I re-derive the 11.25% from the PR's 21 run logs, check the p-value, explain the mechanism from the code, and place it against the record table and the n-gram and CPU retrieval entries it is pointedly not. (2026-10-06) [LLM architecture · Notable · 64/100 · Original, source-checked analysis; Open code or weights]
- [Octop: the accounts are separated by a row check, the sandbox is opt-in](https://ai.thesatyajit.com/articles/octop.md): Tencent open-sourced Octop, a self-hosted, multi-user AI workspace: every user gets a login and several agents, each with its own workspace, plus a browser, a terminal, a remote desktop, hand-off to Claude Code or Codex over ACP, and Telegram and Discord bridges, all from one octop run. I read Octop at 4b17f1c and its agent runtime, octop-harness, at 14a1cd9. 'Nothing you say or save shows up in another account' holds in the API, where every route checks agents.user_id against the signed-in user. It does not hold on disk by default: the default backend runs agent commands as the server's own OS user with the host root as its root, the Terminal is a host shell, the remote desktop is the host's one screen, and model keys sit in one deployment-wide table. A bubblewrap jail and a per-agent Docker sandbox exist, and you have to turn them on. (2026-10-06) [Security & safety · High · 72/100 · Original, source-checked analysis; A guide you can follow today]
- [OpenSEO's AI Visibility: a scraper, a regex and one answer per prompt](https://ai.thesatyajit.com/articles/open-seo.md): OpenSEO, an MIT-licensed SEO tool, shipped AI Visibility on its $10/month plan: research the questions people ask about your market, compare four models side by side, and track whether ChatGPT, Gemini and Google AI Overviews mention or cite you. I read the repository at deb4491. The answers come from DataForSEO's scrapers of the consumer apps and of Google's results page; OpenSEO's own part is a careful regex for mentions, a domain test for citations, and a trend that compares matched prompt-and-engine cells and refuses to show a change when fewer than 90% of answers arrived. It collects one answer per prompt and engine per check, so a weekly check of 20 prompts on two engines cannot see a move smaller than about 22 points; the launch's own competitor table has intervals that overlap. The $0.002 per answer is an estimate that rounds up; settled per batch, the hosted markup is the README's 28%. (2026-10-06) [Developer tools & infra · Solid · 58/100 · Original analysis; Concrete numbers to act on]
- [openai/math: 372 claimed theorems, 26 million lines of Lean, and what the green tick means](https://ai.thesatyajit.com/articles/openai-math.md): OpenAI released 722 manuscripts in 372 result families from an unreleased model: the quasi-Riemann hypothesis, matrix multiplication at exponent 9/4, the Hodge conjecture for CM abelian varieties, all free group factors isomorphic. I read the repository: what the Lean library and Comparator actually certify, which headline claims are checked against plain-mathlib statements and which are manuscript only, what the reasoning summaries reveal, and every family by discipline in a searchable table. (2026-10-06) [Evals & benchmarks · Essential · 76/100 · Analysis found nowhere else; Genuinely new idea]
- [OpenCut: the 92K-star repo is a rewrite, and the editor you can use is the archive](https://ai.thesatyajit.com/articles/opencut.md): A viral post sells OpenCut as a free, watermark-free CapCut alternative with multitrack editing, plugins, AI agents, automations and cross-platform support. I cloned both of its repositories and used the hosted editor. The editor that exists is the classic one, archived on 2026-05-17: a Next.js app that keeps time as integer ticks at 120,000 a second, decodes with WebCodecs through mediabunny, composites on a Rust wgpu compositor compiled to WebAssembly, and stores projects in IndexedDB and media in the browser's private file system. Free, MIT and watermark-free all hold; I exported a 7.04 s project and the file has no mark. Plugins, agents (an MCP server), headless automation and one Rust core for every platform are the rewrite's roadmap. The 92.9K-star repo now holds a window that opens, a page that says 'Coming soon.' and a pinned FFmpeg build. Its only AI feature today is in-browser Whisper captioning. (2026-10-06) [Developer tools & infra · Notable · 63/100 · Checked against the source; Runs on a consumer GPU]
- [OpenHuman: the harness is lighter, and it is faster for a reason the posts don't give](https://ai.thesatyajit.com/articles/openhuman.md): OpenHuman is a Rust agent harness pitched on X as having 'beat OpenClaw and Hermes on efficiency'. The lightweight half holds: the vendor's own cross-harness benchmark, which I re-aggregated from its committed logs, has it at 68 MB peak process memory against 1.57 GB and 816 MB. The speed half is narrower than it sounds. On the six SWE-bench tasks all three solved, the three harnesses spend about the same time waiting on the model (34.6, 38.9 and 39.2 seconds); OpenHuman wins by making fewer calls and spending 1.4 seconds outside the model where the others spend 12.8 and 21.2. Its calls are individually the slowest of the three, and it solved 7 of 10 tasks where they solved 9 and 10. The viral email demo shows a smaller gap than its caption suggests, and the 'live context window' is a ring that updates once per turn, a feature both competitors also ship. (2026-10-06) [Agents & harnesses · High · 68/100 · Original, source-checked analysis; Concrete numbers to act on]
- [Overmind: the 28x fewer invented quotes did not come from anyone's agent traces](https://ai.thesatyajit.com/articles/overmind.md): Overmind open-sourced the platform it uses to turn an agent's production traces into datasets, evals and a fine-tuned open model whose weights you keep. I read the code at commit 2c65378. The loop is real and carefully built, with Unsloth and TRL doing LoRA or full fine-tunes on 53 catalogue entries. The headline number, 0.12% invented quotes against 3.36% for a frontier flagship, comes from a contract eval that is not in the repo, names none of its models, and has the shape of CUAD's public test split rather than anyone's traces. The parts that decide whether such a number means anything, the train/eval split and the judges, have gaps I can show from the source. (2026-10-06) [Developer tools & infra · Notable · 62/100 · Analysis found nowhere else; Concrete numbers to act on]
- [A $80 paper rewriter: what the fine-tune bought, and what the judge decided](https://ai.thesatyajit.com/articles/paper-rewriter-finetune.md): Maxime Rivest fine-tuned Qwen3.5 4B and 9B to rewrite research papers for a 14-year-old, and posted that a 4B beat GPT-5.6 Terra for $80. The models, the dataset and the code are all public, so I checked them. The recipe holds up: synthetic rewrites from frontier teachers, one pass, a LoRA on a home RTX 3090 or a full fine-tune on a rented B300. The headline is shakier. The 4B loses to Terra head to head, 6 to 7; the $80 was the 9B's run, not the 4B's; and when the same 540 pairings are judged by GPT-6 Astra instead of Claude Opus, Terra beats both students 15-0 and 14-0. (2026-10-06) [Training & RL · Essential · 76/100 · Original, source-checked analysis; A guide you can follow today]
- [PixelUMM and PixelDense: diffusion without the VAE, and what the pixels need instead](https://ai.thesatyajit.com/articles/pixelumm-pixeldense.md): Two papers from the same week drop the VAE and generate in raw pixels. NVIDIA's PixelUMM turns Qwen3-8B into a Mixture-of-Transformers that reads and writes 16x16 image patches and 4x16x16 video tubes through single linear layers, with no vision encoder either. PixelDense keeps the pixel denoiser and changes its training signal: REPA alignment to four frozen teachers, semantic and geometric, routed through two orthogonal projection streams. I worked through why latent diffusion needed a VAE, why pixel models predict the clean image, and checked both papers' headline numbers against their own tables, the released checkpoint and the model code. (2026-10-06) [Image & video generation · Notable · 62/100 · Original analysis; A lasting reference]
- [PixiJS 3D and benchy: the fine print under 5.9x, and the agent's pull requests behind it](https://ai.thesatyajit.com/articles/pixijs-3d-benchy.md): PixiJS says its closed-beta 3D renderer got about 6x faster on WebGPU, helped by benchy, a benchmark loop that lets Claude Opus 5.5 check its own changes. Neither benchy nor PixiJS 3D is public, so I read what is: the footnote in the launch video (a geometric mean of frame cost over 27 scenes on an M3 Pro), the two harnesses benchy is built on (endel's WebGPU/WebGL benchmarks and pmndrs/labs), and the pull requests the same agent loop sends to PixiJS core. Of 138 PixiJS PRs opened since 1 June, 27 carry a Claude Code footer and 9 name pixi-3d; their measured wins are microseconds per frame, real but nowhere near 5.9x on their own. (2026-10-06) [GPUs, kernels & systems · Solid · 58/100 · Original analysis; A lasting reference]
- [Pocket TTS with drifting: a one-step speech head without the Jacobian](https://ai.thesatyajit.com/articles/pocket-tts-drifting.md): Kyutai retrained the sampler head of Pocket TTS, its 100M-parameter on-device TTS, with drifting (Deng et al., arXiv 2602.04770), a one-step objective that pulls each generated sample toward data and pushes it away from its siblings. It replaces LSD, which needed a Jacobian-vector product at every training step. I derive the drifting field from the paper, read the loss Kyutai shipped in pocket-tts (it is the paper's doubly normalised kernel, not the simpler one in the blog's pseudo-code), check the parameter counts from the checkpoint headers (108.7M total, an 8.97M head), and walk the ablations: a learned kernel temperature is what makes it train, and a fixed one at the learned value collapses. Parity with LSD is reported at 0.96% vs 0.91% WER, at about twice the training cost. (2026-10-06) [Speech & audio · Notable · 67/100 · Original analysis; Runs on a laptop CPU]
- [PortSimEnv: real Barcelona berths, a proven optimum, and a 3D port that never touches the score](https://ai.thesatyajit.com/articles/portsimenv.md): PortSimEnv turns the Port of Barcelona's 1,784 recorded 2024 container calls into 1,050 training and 50 eval tasks: re-plan a broken week at one quay, choosing when, where and with how many cranes each ship docks, graded once against a plan CP-SAT proved optimal. I read the environment, solver, grader and eval harness, re-implemented the floor and the reward, and recounted the 300 eval rollouts. The grading holds up: every one of the 1,100 references is flagged proven optimal, and my own reward code reproduces the stored scores exactly. What the post calls a simulation is a static scheduling problem with everything known up front; the 3D twin replays a plan but never feeds the reward. And the open models did not run out of turns, as the README says. All 98 episodes without a submission ended because two turns in a row hit the 32k output cap, 79 of them after two turns of twelve. (2026-10-06) [Training & RL · High · 72/100 · Original, source-checked analysis; Interactive explanations]
- [pplx-decider v1.1: the causal mask comes off in 16 layers, and your state is billed once per question](https://ai.thesatyajit.com/articles/pplx-decider-v1-1.md): Perplexity's pplx-decider-v1.1-27b tops Hugging Face's new Decision Index 0.3 at 62.75, 2.54 points clear of second place and outside the 0.9-point tie band. I read the repository's weights by HTTP range request, its training config and code, the board's data files and the Decisions API docs. The checkpoint is Qwen3.8-27B with the vocabulary head and MTP block removed and 255 rows of that head kept as the readout: 26.09B parameters, a full fine-tune that moved every projection by about 0.24 to 0.63% and the vision tower almost not at all. The one architectural change from v1 is a forward hook that removes the causal mask from the 16 full-attention layers, so the state is read with the question in view. That same change rules out a KV cache, so the shipped server runs one complete prompt per question and bills the state each time: the docs' own 367-token example reproduces exactly that way. The win is real where it matters most, on the private tests nobody can train on; on public benchmarks alone Torchcast leads. The '250k context' is the base model's window; training stopped at 8,192 tokens. Third on the vision board, and v1 now costs the same $0.02. (2026-10-06) [LLM architecture · High · 75/100 · Original, source-checked analysis; Interactive explanations]
- [Prime Inference: serving GLM-5.3 to agents is a KV-cache problem](https://ai.thesatyajit.com/articles/prime-inference.md): Prime Intellect's write-up of how it serves GLM-5.3 on GB200 NVL72 with vLLM and Dynamo is four cache fixes in a row, and the numbers hold together better than most serving posts. I checked them against GLM-5.3's config and the vLLM source history. The 576 to 352 bytes per MLA row reconstructs exactly as 256 bytes of packed 4-bit latent, 32 FP8 scales and the 64-byte rope part, and 78 layers of it is 6.6 GB of FP8 cache for a typical 146K-token agent session. The ~5x prefix-cache win of DEP8 over TEP8 is the arithmetic of MLA itself: the latent has no head axis, so tensor parallelism cannot split it and every rank keeps all of it. Two things do not reconcile on the page: a 1.64x row shrink becomes 1.50x of capacity, and 66 sessions at 101 tok/s each would be 6,666 tok/s against the 2,400 that 24 GPUs at 100 tok/s actually produce. (2026-10-06) [Inference & serving · High · 72/100 · Original, source-checked analysis; Widely used]
- [QuantUI-rs: exact to the last bit, and a default file stock ComfyUI won't open](https://ai.thesatyajit.com/articles/quantui-rs.md): QuantUI-rs is one static Rust binary, no Python and no torch, that writes ComfyUI INT8, FP8, MXFP8 and NVFP4 safetensors and llama.cpp GGUFs, and promises output byte-exact against its references on every default path. I read all 27,000 lines of its library and CLI at d2cced6 next to the code it copies: convert_to_quant, comfy-kitchen, llama.cpp, and NVIDIA's ModelOpt for comparison. The arithmetic is ported with real care, down to torch's one-ULP division quirk and its AVX2 random-number polynomials. But the defaults miss the README in two places. Run quantize with no flags and every layer is tagged int8_blockwise, a format stock ComfyUI does not register, so the loader raises KeyError; the FP8 default has the same problem. And the default GGUF method, q4_k_m with no importance matrix, goes through a simplified encoder that the repo's own tests say is not byte-exact with llama-quantize. I also re-implemented the NVFP4 path in numpy: it matches ModelOpt's scales and codes on 1,048,576 blocks but packs nibbles the other way round, the opt-in L2 search is worth 23% less weight error on Gaussian weights, and the always-on bias correction subtracts noise with zero expected value. (2026-10-06) [Quantization & compression · Essential · 76/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [Qwen3Guard-Stream: a safety verdict on every token](https://ai.thesatyajit.com/articles/qwen3guard-stream.md): Qwen3Guard-Stream is a Qwen3 backbone with its vocabulary projection removed and two small classification heads attached, so it scores a response token by token while the generator is still writing, instead of judging the finished text. I read the configs, the safetensors headers, the modeling code and the eval script: the heads are about 1.06M parameters on a 597M-parameter model, the labels come from rollouts judged by the generative guard, a two-token debounce decides where to cut, and the reported cost is about two F1 points against Qwen3Guard-Gen. What it does not do is work with a generator that uses another tokenizer, or read reasoning traces as well as final answers. (2026-10-06) [Security & safety · Notable · 67/100 · Original analysis; Runs on a laptop CPU]
- [Recursive Self-Rewrite: teach the model what the harness did](https://ai.thesatyajit.com/articles/recursive-self-rewrite.md): Recursive Self-Rewrite (arXiv 2610.02826) runs one Qwen-3.8-27B under three terminal harnesses, keeps the successes, and has the same model rewrite each one into a fresh, verified trajectory under plain Terminus 2: a planner turns the solved run into a runbook, a critic rejects runbooks that leak the answer and sends them back, an executor re-solves the task in a new sandbox with the runbook hidden from the log. Supervised fine-tuning on 11,094 such rewrites lifts Terminal-Bench 2 pass@3 from 57.0% to 74.2%, where fine-tuning on the raw successes made it worse. The 'recursive' is the planner-critic loop, not repeated rounds of self-training; the data comparison is not size-matched; and the untrained model under one of the discovery harnesses already beats the trained one on the authors' own hard set. (2026-10-06) [Agents & harnesses · Solid · 58/100 · Original analysis; A new technique]
- [Reflection Beam: a 501B open MoE that spends 23B per token, and what its efficiency chart measures](https://ai.thesatyajit.com/articles/reflection-beam.md): Reflection AI announced Beam on 5 October 2026: a sparse mixture-of-experts with 501B total and 23B active parameters, pretrained on 23.8T tokens on 6,144 GB300s, then put through four weeks of asynchronous RL on 10,500 GB300s and more than 100 million rollouts. The weights are not out yet. I read the blog and the launch thread, rebuild the reasoning-efficiency chart from its own dots, and find that its FLOP axis is exactly 2 x active parameters x generated tokens, so roughly a 1.7x slice of every 'more efficient than GLM-5.2' ratio is the active-parameter ratio. Then the compute ledger: about 3.3e24 pretraining FLOPs by 6ND, and an RL phase that used more GPU-hours than pretraining did. (2026-10-06) [LLM architecture · Notable · 60/100 · Original, source-checked analysis; Widely used]
- [RWKV-7 G1k: what fits in a 16-million-number mind](https://ai.thesatyajit.com/articles/rwkv-7-g1k.md): BlinkDL released RWKV-7 G1k, the latest monthly checkpoint of a 100% RNN language model family, and then argued that a pure RNN is the safest route to superintelligence because all of its reasoning has to pass through a constant-size state: 61x64x4096 = 16M numbers for the 13B model, with no growing KV cache. This derives RWKV-7's state update from first principles (a delta rule with vector-valued decay and in-context learning rate), verifies the 16M figure against the tensor shapes inside the released checkpoint, compares that state with a same-size transformer's KV cache from 1K to 1M tokens, reads the reported G1k compression results, and separates what is testable in the safety claim from what is opinion. (2026-10-06) [LLM architecture · Notable · 62/100 · Interactive explanations; Original analysis]
- [SAIL: a robot that searches 45 trajectories in simulation before it moves once](https://ai.thesatyajit.com/articles/sail-robot-trajectories.md): Sakana AI and the University of Tokyo turn a VLM into a robot trajectory generator and then spend inference compute on it: Monte Carlo Tree Search where every node is a whole trajectory, every edge is a rewrite, and the value is a progress score the same VLM reads off a simulated video. Across six ALOHA tasks the share of seeds where a passing trajectory is found rises from 25% with one proposal to 73% at 45 nodes. I read the paper: the success that is counted is the simulator's own check, so the curve is how often the search finds a verified answer, and fifteen independent samples already take it from 25% to 51%. Refinement adds the next 14 points. On a real SO-101 arm it is 5 of 6 trials, about eleven minutes each. (2026-10-06) [Robotics & embodied · Solid · 51/100 · Original analysis; A new technique]
- [Sentry: failure lessons should arrive with the failure](https://ai.thesatyajit.com/articles/sentry-agent-recovery.md): Stanford's ACE team found that the recovery lessons an agent memory keeps in context all the time make the agent worse: pruning them from ACE's playbook lifts WebShop reward from 0.335 to 0.391. Sentry keeps those lessons outside the agent, has an LLM detector watch the last five steps, retrieves at most five lessons that match the diagnosed failure label, and writes a new lesson only after a verifier with no reward access judges the repair to have worked. The paper reports +37% (mean relative gain) over the best runtime-intervention baseline on four benchmarks and +39% over ACE on two. Its code is not released yet. (2026-10-06) [Agents & harnesses · Notable · 59/100 · Original analysis; A lasting reference]
- [Shaders goes MIT: the components are the menu, the compiler is the meal](https://ai.thesatyajit.com/articles/shaders-webgpu.md): Shader Effects open-sourced Shaders on 6 October 2026: the WebGPU engine, 199 effect components and the React, Vue, Svelte, Solid and JavaScript bindings, under MIT, in a package that drew 1.14 million npm downloads in the month before and that until version 4.0.0 needed a Pro licence for commercial use. I read the repository at 935f71a. A component is a plain object of props and a paint, effect or map function, mostly written as TypeScript that TypeGPU compiles to WGSL. The part worth studying is the composer, which turns a layer tree into as few GPU passes as it can: generators and pointwise filters fuse into one fragment function, a filter that reads neighbouring pixels forces a render-to-texture pass, and a distortion wrapped around generators is folded into their coordinates so it needs no texture at all. Props live in one packed uniform buffer, and only a structural hash of the tree triggers a recompile. It is WebGPU only, with no WebGL fallback. The editor, the presets and the MCP server stay closed, and every component still samples 5% of external page loads for telemetry unless you pass disableTelemetry. (2026-10-06) [Developer tools & infra · Notable · 64/100 · Runs in a browser; Original analysis]
- [Spark-X2.5-4B: fifteen minutes with the files behind a viral model](https://ai.thesatyajit.com/articles/spark-x2-5-4b.md): A post with 12,000 views says Spark-X2.5-4B just hit #1 on Hugging Face, runs on a 16 GB laptop and has a native 1M-token context. I checked it against the only evidence that does not depend on trusting anyone: config.json, the safetensors headers, the custom modeling code, and one MLP tensor from it and four likely bases. It is a real, original 4.1B model from an iFlytek subsidiary, with a clean loader and llama.cpp support. The 1M context is consistent with the config but costs 36 GiB of cache, the #1 claim is five weeks stale, and every benchmark is the vendor's own. (2026-10-06) [LLM architecture · Essential · 77/100 · Original, source-checked analysis; A guide you can follow today]
- [Strata in the source: one verify window, three workers and a cache that learns](https://ai.thesatyajit.com/articles/strata-engine.md): Ten days after the site first checked Strata against the memory bus, the repository has gone from 4 commits to 1,152, gained an MIT licence, replaced its 'output is not correct' warning with a measured A/B, and learned to sample. I read the engine at commit 1cbcacb: the doorbell that hands each layer's routed experts to the CPU mid-graph, the per-layer plan that splits misses between the VRAM cache, the PCIe copy engine and six AVX-512 cores, the 2-bit and codebook CPU kernels, the cache's replacement rule, the MTP verify window and the KV layout behind a 142,550-token prompt. Then I rebuild the 3090 run at 97 tok/s from those constants: it needs a VRAM hit rate between about 0.70 and 0.86, and its slowest lane is not the RAM, which is why DDR4 and DDR5 measured almost the same. An update reads a V100 run of the 4-bit Unsloth file at twice llama.cpp's speed and finds the win in the expert cache and the PCIe lane, not in drafting. (2026-10-06) [Inference & serving · Notable · 63/100 · Original analysis; Runs on a consumer GPU]
- [Subquadratic 3SUM and subcubic APSP: two hypotheses fall to a pruned matrix product](https://ai.thesatyajit.com/articles/subquadratic-3sum-apsp.md): Alman and Vassilevska Williams (arXiv 2610.06783, from an algorithm Claude found) solve 3SUM in O(n^1.9992) and APSP in O(n^2.9995), refuting two of the three hypotheses fine-grained complexity was built on. Everything comes from one new trick: compute a sparse set of entries of a thin integer matrix product by running Schönhage's ten-multiplication identity recursively and visiting only the leaves the wanted entries need. The reduction chain from 3SUM and APSP down to that product, the leaf count that makes it work, why the speedup is invisible below n = 10^344, and which conditional lower bounds fall, lose their evidence, or stand (SETH, and with it attention's quadratic bound, is untouched). (2026-10-06) [Agents & harnesses · Notable · 58/100 · Genuinely new idea; Original analysis]
- [tiktok-5.6B-videos: 460 GB you can query without downloading](https://ai.thesatyajit.com/articles/tiktok-5-6b-videos.md): A Hugging Face dataset claims metadata for 5,597,462,038 public TikTok videos, one Parquet file per month from 2014 to 2026-10-03. I read all 148 Parquet footers by HTTP range request instead of downloading the files. They hold 5,598,141,717 rows, 679,679 more than the card says and exactly what the Hub's own viewer reports. The layout is what makes a single laptop useful against it: ZSTD column chunks, about a million rows per row group, and a bloom filter on every column. A count over one month's views fetched 184.7 MiB of a 10.95 GB file, within 0.2 MiB of what the footers predict. Pruning inside a month is weaker: a third of September 2026's row groups span the whole month, so a one-day filter there still reads 40-47% of the month's bytes. The AI-content flag is null on 97.3% of rows. In the one file I downloaded, 34.7% of rows were last updated a week before they were posted. Underneath all of it is provenance. The data was scraped against TikTok's terms, it carries creator ids and free-text captions, and a CC BY-NC licence does not settle what the GDPR asks of whoever holds it. (2026-10-06) [Data & datasets · Notable · 64/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [TileLang: you write the dataflow, the compiler owns the schedule](https://ai.thesatyajit.com/articles/tilelang.md): TileLang is a Python-embedded DSL on TVM for writing GPU kernels at tile granularity: you declare which buffers live in shared memory and which live in registers, call T.copy and T.gemm, annotate a loop with num_stages, and the compiler infers layouts, binds threads, swizzles for bank conflicts and builds the software pipeline. I cloned the repo at v0.1.15 and read the layout-inference and pipeline-planning passes, then checked the paper's speedups against what the code actually does. The GEMM kernel is 22 lines. The layout inference runs three priority passes — strict, common, free — because T.gemm pins its fragments and everything else has to agree with it. And the pipeline pass quietly declines to pipeline at all on MI300X. (2026-10-06) [GPUs, kernels & systems · High · 74/100 · Runs on a consumer GPU; Original analysis]
- [toks: an exact tokenizer built from certified shortcuts](https://ai.thesatyajit.com/articles/toks-tokenizer.md): Actual Computer's toks returns Hugging Face's exact token ids 13-151x faster on one core, with its hot loops in hand-written NEON and AVX2. I read the whole repository. Most of the speed comes from never running BPE on a piece at all: a static table answers four pieces in five, and every shortcut is either computed by the real algorithm at load or covered by a written proof. The merge loop that is left is the bottleneck. Its own speed table also shows where the launch post overclaims: gigatoken wins about half of the replays, and the 151x is partly Hugging Face slowing down on whole files. (2026-10-06) [Developer tools & infra · Notable · 59/100 · Original analysis; Runs on a laptop CPU]
- [treg: a people-search router priced by expected cost per hit](https://ai.thesatyajit.com/articles/treg.md): Superdesign launched treg as an "open-source Clay killer": 85% cheaper, 25x faster, most accurate on a people-search benchmark. I read the repository at f93ae9b. treg is a tool registry and proxy with 3,851 catalogued endpoints across 109 providers, and its people search is a router that ranks every provider able to take your identity by price times P(billed) over P(hit), then walks them in a waterfall where most misses are free. The router is real and carefully built. The headline numbers are a different matter: the cost and speed figures come from a launch film whose table is published nowhere, the benchmark chart labels a mean of three scores as '% answered correctly', and the licence is Apache 2.0 with a clause that forbids offering it as a hosted service. (2026-10-06) [Developer tools & infra · High · 69/100 · Original, source-checked analysis; Concrete numbers to act on]
- [Cactus Whistle: speech-to-text in 16.9 MB, and a FLEURS bar the Whisper paper doesn't support](https://ai.thesatyajit.com/articles/whistle-stt.md): Whistle is Cactus Compute's speech recognizer: a 55.15M-parameter encoder-decoder, quantized to 2-4 bits into a 16,919,407-byte file that loads into the same C++ engine as Needle. I read its safetensors header and eight gate values over HTTP range requests, and checked its benchmark chart against the Whisper paper's own tables. The file size holds. The encoder carries a Conformer-style convolution module the blog never mentions. The per-layer cross-attention gates all sit at a logit near 22, which makes them exactly 1. Against Whisper base, Whistle wins two of six comparable benchmarks, not most. The chart's Whisper FLEURS bar reads 24.5, but the seven-language mean of Whisper's Table 13 is 21.0, ahead of Whistle's 21.4. (2026-10-06) [Speech & audio · High · 69/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [Spirula Studio deleted its dependencies instead of wrapping them](https://ai.thesatyajit.com/articles/spirula-studio.md): A 3D Gaussian Splatting pipeline in one binary — camera alignment, masking, normal estimation, training — with no Python, no PyTorch and no COLMAP install. 'All in one pipeline' almost always means vendored components behind a launcher, so I went looking for what was wrapped. It is not: COLMAP became 44,310 lines of Vulkan SfM, PyTorch became a 10,886-line Slang inference runtime, ffmpeg became its own H.264/H.265/AV1 decoders, and SAM 3 runs as compute shaders. What is still borrowed is checkpoints and parameter defaults, credited by name. Also: why the bundle adjustment stays fp64 when fp32 is worth 30%, and what a DJI Osmo 360 .OSV actually goes through. (2026-09-22) [3D & spatial · Notable · 63/100 · Original analysis; Runs on a consumer GPU]
- [Qwen3.8-Flash-Next: four changes and an honest report](https://ai.thesatyajit.com/articles/qwen3-8-flash-next.md): 125B parameters plus 51B of N-gram embeddings, 6B active, trained for about a ninth of what its predecessor cost. Three of every four layers are Gated DeltaNet; the fourth uses a sparse attention that indexes micro-blocks rather than tokens. But the reason to read this release is the technical report, which keeps finding places where training loss and downstream accuracy point in different directions — and says so. Updated: a llama.cpp fork ships flags dedicated to this model, and tracing --no-ngram through its loader shows the reported speed number runs with the 51B-parameter table this article covers switched off entirely — 0 bytes, no documented cost. Second update: three more treatments of that same table, checked against their own sources — a DGX Spark recipe that keeps it fully resident at ~5 bits (30.4 GB, the opposite bet from --no-ngram), a second Spark kit that re-quantizes it to NVFP4 and demand-pages it (26.8 GB, verified), and NVIDIA's own official NVFP4 release, which leaves it at FP8 untouched (47.7 GB) — the least compressed of any of them. (2026-08-26) [LLM architecture · High · 76/100 · A guide you can follow today; Original analysis]
- [DFlash 2: the drafter already knew the answer, it just picked the wrong one](https://ai.thesatyajit.com/articles/dflash2.md): A parallel speculative drafter's top pick is right 85% of the time at the first block position — and the right token is in its top sixteen 99.5% of the time. DFlash 2 spends two million parameters connecting candidates that were already computed, and a two-tap convolution to stop the block decaying at the end. One more accepted token per pass, for 1.3% more latency, with the output provably unchanged. (2026-08-23) [Inference & serving · Notable · 61/100 · Interactive explanations; Runs on a consumer GPU]
- [Context Language Models: the bitter lesson comes for context management](https://ai.thesatyajit.com/articles/context-language-models.md): A UW and Meta FAIR paper mirrors an agent's live context as an editable file and lets the model rewrite it with Bash. Unrestricted self-management beats hand-designed compaction harnesses zero-shot: 11.4% higher accuracy at 21.5% fewer FLOPs on BrowseComp-Plus, a 35.9-point in-context steering gain, and an RL recipe that lifts Qwen3.5-9B by 47.6% while spending 12% fewer FLOPs. I checked every headline number, re-derived the serving-cost formula from the paper's own constants, and read the released code — here is the mechanism, and where it is still a promise rather than a result. (2026-10-02) [Agents & harnesses · Solid · 58/100 · Facts measured from source; Widely used]
- [CSFM: flow matching never had to start from Gaussian noise](https://ai.thesatyajit.com/articles/csfm-flow-matching.md): Flow matching places no restriction on its source distribution, yet nearly every text-to-image system starts every sample from the same fixed Gaussian. CSFM (NeurIPS 2026, NYU + KAIST) learns a condition-dependent source jointly with the velocity field, which lowers the flow-matching intrinsic-variance term — the part of the loss that measures how ambiguous the velocity supervision is. The paper reports up to 3.01x faster FID convergence and 2.48x faster CLIP-Score convergence on captioned ImageNet-1K in RAE latents; I check the component ablation (fixed Gaussian FID 3.036, variance regularization 2.765, plus directional alignment 2.453), the few-step robustness (after reflow, 50 to 2 steps costs 3.51 FID for CSFM against 11.75 for standard FM), and the scale result (GenEval 0.80 vs 0.77). Two regularizers keep the learned source from collapsing, the gain shows up only when the target latent is structured, and the code ships with no licence file. (2026-10-02) [Image & video generation · Solid · 48/100 · A lasting reference; A new technique]
- [Streaming 3D from unposed video: R³, AMB3R-SLAM, and the move to bounded memory](https://ai.thesatyajit.com/articles/da3-streaming-reconstruction.md): Two systems from this year build streaming 3D reconstruction on top of Depth-Anything-3 as a frozen backbone, and both turn per-frame pointmaps into an estimate whose memory stays flat as the video grows. R³ (Westlake/UMich/NVIDIA) regresses confidence-weighted relative poses with a 372M-parameter model and a novelty-gated keyframe bank; its streaming ATE beats 1.26B baselines, and its memory holds flat to 1000 frames under the 48 GiB budget that OOMs StreamVGGT past 500. AMB3R-SLAM (UCL) is training-free: a DA3-Small front-end feeds a hierarchical Sim(3) pose graph with span-2, long-context and DBoW2 loop-closure edges, no bundle adjustment, 10-14 GB flat to 10k frames, and cuts KITTI-01 ATE from 55.91 m to 2.95 m with LiDAR. I checked the numbers against the papers and code: the tables hold, AMB3R's repo is still an empty placeholder, and R³ quotes three different frame rates across its own four surfaces. (2026-10-02) [3D & spatial · Solid · 54/100 · Original analysis; A lasting reference]
- [DeepGEMM-Ascend: DeepSeek's kernels off NVIDIA, and what 99.8% and 98% actually measure](https://ai.thesatyajit.com/articles/deepgemm-ascend.md): DeepSeek open-sourced DeepGEMM-Ascend and DeepEP-Ascend, its GEMM and expert-parallel communication kernels ported from CUDA to Huawei Ascend 950 NPUs. I cloned both repos and read the kernels and the perf tables. The two numbers in the launch tweet are real but they are different ops: 99.8% of the hardware limit is one dense BF16 matrix multiply (431 of 432 TFLOPS), and 98% is the fused MegaMoE op (846 of 865 FP8 TFLOPS) that also carries the expert dispatch and combine. The ports are API-compatible with the CUDA originals but written against Ascend's MAD matrix engine in C++ headers, JIT-compiled by Huawei's Bisheng, not nvcc. Two caveats the tables do not hide: DeepEP-Ascend ships no LICENSE file, and its full dispatch bandwidth needs a Huawei firmware release that is not public yet. (2026-10-02) [GPUs, kernels & systems · Solid · 58/100 · Original analysis; A new technique]
- [DepthBench: depth is a scaling axis only for the right residual stream](https://ai.thesatyajit.com/articles/depthbench.md): DepthBench holds the parameter count, the data and the recipe fixed and changes only the width-depth aspect ratio, across ten residual-stream designs on a 400M LLaMA backbone trained 20 tokens per parameter on FineWeb-Edu. The result is sharp: Hyper-Connections and Full Attention Residuals keep improving as models go deeper-and-narrower, all the way to an aspect ratio of 9.1 (640 wide, 70 layers); Pre-LN and its normalization variants get worse; and the cheaper production variants, mHC and Block AttnRes, lose the property. I check the claims against the released OLMo-core configs, the 162 public checkpoints, and the paper's own FLOP formulas. (2026-10-02) [LLM architecture · Notable · 59/100 · Original analysis; A lasting reference]
- [DeepSeek DSec: three million agent sandboxes a day, 50× overcommitted](https://ai.thesatyajit.com/articles/dsec-agent-sandbox.md): DeepSeek's report on DSec, the sandbox platform behind its agentic RL training from V3.2 to V4.1: four backends behind one Python SDK, environments composed from independently versioned EROFS layers loaded on demand from 3FS (a sandbox touches only 4.2–13.3% of its image), CPU packed past 50× because ~90% of sandboxes use under 5% of their requested cores, rollout execution split out of preemptible GPU training in V4.1, and a sober production catalogue of agents trying to reward-hack the environment — forging RPCs, overwriting /bin/bash, and reaching for XFS_IOC_SWAPEXT. (2026-10-02) [Developer tools & infra · Solid · 50/100 · A lasting reference; A new technique]
- [FAR-LIO: LiDAR-inertial odometry that holds together at 250 km/h](https://ai.thesatyajit.com/articles/far-lio.md): FAR-LIO (TU Munich, arXiv 2606.26010, IROS 2026) is a CUDA LiDAR-inertial odometry stack built for autonomous racing: a GPU voxel hashmap with adaptive density, a sparsity-aware GICP with KISS-ICP-style adaptive thresholding, and an EKF that upsamples the IMU and extrapolates the pipeline's own delay. I read the Apache-2.0 code and the shipped config against the paper — what each of the three pieces does, why 250 km/h breaks the usual assumptions, and what the racing APE table and the ablation actually show, mon_20 tradeoff included. (2026-10-02) [Robotics & embodied · Solid · 58/100 · Original analysis; Runs on a consumer GPU]
- [fframes: a frame is a function, and the benchmark is one scene](https://ai.thesatyajit.com/articles/fframes.md): fframes writes each video frame as an SVG tree returned by a Rust function, rasters it with Skia on the GPU, and links libav directly instead of shelling out. I cloned it and read the svgr! macro, the Skia render cache and the vs-remotion benchmark. The compile-time static caching and the agent-facing CLI are the real design; the README's 31.16× over Remotion is one synthetic scene, measured CPU-versus-CPU with the GPU skipped and the Remotion side carrying 12 chained effect passes per element. (2026-10-02) [Developer tools & infra · Solid · 53/100 · Original analysis; Runs on a laptop CPU]
- [GLM-5.3-Flash on four mining cards: the hard part is an Ampere kernel](https://ai.thesatyajit.com/articles/glm-5-3-cmp170hx.md): A 320B hybrid-attention MoE with a 262K context, served on four NVIDIA CMP 170HX mining cards. The headline is the cheap hardware; the actual engineering is a vLLM fork that writes the sparse-attention kernels GLM-5.3 needs for Ampere, which upstream only ships for Hopper. I read the fork and the model, verified what I could, and labelled the rest: every throughput and quality number is one person's self-run, the 64 GiB cards are a hardware mod, and the sparse-MLA kernel is real sm_80 Triton I could read line by line. (2026-10-02) [GPUs, kernels & systems · Notable · 63/100 · Analysis found nowhere else; Concrete numbers to act on]
- [Hindsight: +44 points on the same 20B backbone, from the memory layer](https://ai.thesatyajit.com/articles/hindsight-agent-memory.md): Hindsight is an open-source (MIT) agent-memory system — about 44k GitHub stars — that organizes memory into four epistemically distinct networks (world facts, agent experiences, synthesized observations, evolving opinions) and runs three operations over them: retain, recall, reflect. On LongMemEval it lifts a 20B open backbone from 39.0% to 83.6% with no change to the model, and reaches 91.4% with a frontier backbone. I read the arXiv paper (2512.12818) and the repo to check the numbers, the four parallel retrievers fused by Reciprocal Rank Fusion (k=60) and a cross-encoder rerank, the disposition-shaped reflect step, and the reproduction and vendor-self-report caveats behind the benchmark table. (2026-10-02) [Agents & harnesses · Solid · 58/100 · Facts measured from source; Widely used]
- [Holo4: open computer-use weights, graded on the lab's own benchmarks](https://ai.thesatyajit.com/articles/holo4.md): H Company's Holo4 is a three-checkpoint family of computer-use agents — one Apache-2.0 MoE, one non-commercial dense model, one on NVIDIA's terms — that click, type, code and call tools across desktop, web, Android and MCP. The weights, a two-expert LoRA-merge recipe and 7,366 evaluation trajectories are public; every benchmark number is self-reported and there is no technical report. This takes the mechanism apart and checks the headline claims against the model cards: how much the post-training adds over the Qwen base, the price gap to Opus 5.5 on OSWorld 2.0, and the licence of each checkpoint. (2026-10-02) [Agents & harnesses · Notable · 62/100 · Original analysis; Runs on a consumer GPU]
- [Index-Translate: one base, four translators — text, speech, dubbing, long docs](https://ai.thesatyajit.com/articles/index-translate.md): Bilibili's Index-Translate is one Qwen3.5 base turned into four open (Apache-2.0) translation heads: text across 150 languages, end-to-end speech (Index-Echo), syllable-controlled dubbing (Index-Homura) and whole-document translation (Index-NativeLong). The 9B scores 0.8789 COMET-22 on FLORES — but that flagship number is 21 languages, not 150, and the dubbing head's 81.92% syllable-hit rate is bought with a measured drop in translation quality. Read from the paper, the repo and the model cards. (2026-10-02) [Speech & audio · Notable · 62/100 · Original analysis; Runs on a consumer GPU]
- [IQuest-Q1: 320B of weights, 25 layers that keep the context](https://ai.thesatyajit.com/articles/iquest-q1.md): IQuest-Q1 is a 320B-total, 15B-active open-weight coding MoE with a 512K context, and it shipped with no technical report. So read it off config.json: a 3-sliding-window-to-1-full attention stack means only 25 of 88 layers grow a KV cache, which is a 3.45x cache saving at 512K that I derive from the file; attention sinks and partial RoPE hold the windowed layers together; and a recursive multi-token-prediction head is served through EAGLE. The self-reported benchmarks lead none of their eight panels. (2026-10-02) [LLM architecture · Solid · 58/100 · Original analysis; Concrete numbers to act on]
- [Jev alternatives, week three: bigger models, smaller benchmarks](https://ai.thesatyajit.com/articles/jev-alternatives-week-three.md): Week two of the open decision-model wave had seven releases and, for once, numbers — most of them on JevBench's 231 public items. Week three has seven more, and they are architecturally the most serious yet: Cloudflare's clef ships a trained 128M joint schema head on a full 27.4B Qwen backbone, Intern-Decision reads 62 symbols in one pass with a fitted calibration temperature, SGLang turned the readout into a first-class /v1/decisions endpoint, and taiga-s1 is a 1.2M-parameter decider that builds parts in FreeCAD at a millisecond each. But the evaluation went backwards. Not one new model this week carries an independent or sealed-item JevBench score. clef and Lumma-fev do not touch JevBench at all; Intern-Decision runs its own public-tier copy; and JEMM's one JevBench row — 198 of 231 against Jev's 200 — is a loss its own figure files under 'gap not significant.' Every 'beats Jev' number is the publisher's own run, and Jev's own latency baseline ranges from 106 to 524 ms depending on who timed it. (2026-10-02) [LLM architecture · Solid · 58/100 · Analysis found nowhere else; Runs on a consumer GPU]
- [KVzip: compress the KV cache once, by scoring it with the model's own reread](https://ai.thesatyajit.com/articles/kvzip-kv-compression.md): The KV cache dominates long-context memory and attention latency, and most eviction methods score it against the current query — so a cache compressed for one question is wrong for the next. KVzip (NeurIPS'25 oral) scores query-agnostically: it makes the model reconstruct the context from the cache and keeps whichever KV pairs that pass attended to, reported at 3-4x smaller cache and ~2x faster FlashAttention decode with negligible loss up to 170K tokens on LLaMA3.1, Qwen2.5 and Gemma3. Its complement, Attention Matching, throws the real pairs away entirely and synthesizes a smaller cache that reproduces the attention output in closed form, up to 50x in seconds. I read both papers and KVzip's MIT code; an applied blog scaling them to GLM-5.3's shared-latent sparse attention is the self-reported data point, not the spine. (2026-10-02) [Inference & serving · Solid · 56/100 · Runs on a consumer GPU; Widely used]
- [Lattice Deduction Transformers: reasoning by narrowing a lattice, not emitting tokens](https://ai.thesatyajit.com/articles/lattice-deduction-transformers.md): The Lattice Deduction Transformer is an 800K-parameter looped transformer that carries its state as a per-cell candidate set — a grid-powerset lattice in the abstract-interpretation sense — and narrows it between forward passes, trained on-policy to mirror a DPLL-style constraint solver. On Sudoku-Extreme it reports 100% solve and 100% soundness (correct-or-abstain) after about 15 minutes on one B200, while Claude Opus 4.6, DeepSeek V4-Pro and GPT-5.4 score 0%. This walks the lattice, the loop, the soundness guarantee and the solve/branch/backtrack search from the paper and the released code — then checks the headline against the authors' own correction (99.96% on the full test set) and against a follow-up critique that says on clue-rich Sudoku this is one-shot amortized prediction plus a verifier, not learned search. (2026-10-02) [LLM architecture · Notable · 61/100 · Original analysis; Runs on a consumer GPU]
- [LongLive-Plug: distill the speedup once, plug it into 54 video models](https://ai.thesatyajit.com/articles/longlive-plug.md): Specialized video-diffusion models each re-run their own distillation stage to get few-step sampling, single-pass classifier-free guidance, or long-context correction. LongLive-Plug (NVIDIA Efficient-Large-Model, with MIT's Song Han) distills each capability once as a LoRA on a frozen base model per backbone family, then adds it training-free onto compatible downstream models. The paper verifies plug-and-play transfer across 54 downstream models on three families — Wan2.1-T2V-14B, Wan2.2-TI2V-5B and MiniMax-H3 — and eight task categories. Four-step transfer drops SCOPE FVD from 805.5 (naive four-step) to 478.7, within reach of 502.1 from per-target distillation, and reusing the base adapters holds the Wan2.2-TI2V-5B family's distillation bill at roughly 80 H100 GPU-hours instead of about 456.8. The CFG adapter is trained at a fixed guidance scale yet its inference weight works as a near-linear guidance dial. Code and the six LoRA checkpoints are public. (2026-10-02) [Image & video generation · Solid · 53/100 · Runs on a consumer GPU; A new technique]
- [Model casting: compute the gate, then skip most of the FFN](https://ai.thesatyajit.com/articles/model-casting-sparse-ffn.md): A Meta FAIR and ENS mid-training recipe swaps a transformer's FFN activation to a ReLU/SiLU hybrid and adds an adaptive L1 penalty so ~90% of hidden units output exact zero — then skips the two big FFN matmuls for the dead units. LoPA Gating makes the gate cheap enough to break the 3x FLOP ceiling, for a measured 3.31x GPU FFN speedup at +0.93% NLL. (2026-10-02) [Inference & serving · Solid · 46/100 · A lasting reference; A new technique]
- [Naive-N0.5-Flash: no full-attention layers, and a runtime built by AI](https://ai.thesatyajit.com/articles/naive-n05-flash.md): A 309B-total, 15.5B-active open (MIT) MoE with native 1M context whose every attention layer is either a 128-token sliding window or a top-2,048 DeepSeek Sparse Attention layer — not one does dense global attention, which is the whole reason a million-token context is affordable. The lab says AI designed the hybrid architecture, found and fixed its numerical-stability bugs, and wrote NaiveRT, the runtime that decodes it at a reported 2,122 tokens a second. Here is what the config and the modeling code actually say, and what has to be taken on the lab's word. (2026-10-02) [LLM architecture · Notable · 59/100 · Original analysis; A new technique]
- [Olmo-core 3: an open training stack for trillion-parameter MoE](https://ai.thesatyajit.com/articles/olmo-core-3.md): Ai2's Olmo-core 3 is open infrastructure for training large Mixture-of-Experts models — the system behind the next Olmo, run from 12.9B to 1.2T parameters. Read alongside Aleph Alpha's recipe-only writeup, which takes a 30B-A3B MoE from 16 to 512 B200 GPUs at 35% MFU with near-linear scaling, it's a clean look at what an open MoE training stack has to get right: the parallelism axes (DP, FSDP, TP, EP, PP, CP), the all-to-all dispatch that makes experts resident, what MFU measures and why 35% is a real number, and the two very different meanings of 'open' — a 168-page report with code versus a blog with profiler traces. (2026-10-02) [GPUs, kernels & systems · Notable · 61/100 · Original analysis; A lasting reference]
- [OpenShell: give the agent a real shell, keep the boundary in the kernel](https://ai.thesatyajit.com/articles/openshell.md): NVIDIA open-sourced OpenShell (Apache-2.0, Rust): a sandbox that gives an AI agent real files, real network, and real credentials in play, but moves the boundary into the kernel. Landlock confines the filesystem, a seccomp user-notification filter hands every TCP open and DNS query to a trusted supervisor that checks a Rego policy and injects credentials the agent never sees, and a Z3 SMT prover gates policy changes with two checks: a boundary proof that a candidate stays inside a maximum policy, and a proposal-risk check that blocks auto-approval when a new rule would widen credentialed reach. I read the sandbox, supervisor and prover crates to see what the proof covers — filesystem, L4, REST, process, Landlock — and what it refuses to check — GraphQL, MCP. Paired against Cloudflare Computer, the state-first design that makes the filesystem itself a Durable Object. (2026-10-02) [Security & safety · High · 75/100 · Interactive explanations; Original analysis]
- [OREN: real-time Euclidean SDF, because the octree carries the prior](https://ai.thesatyajit.com/articles/oren-sdf.md): OREN (UCSD Existential Robotics Lab, arXiv 2510.18999, IROS 2026) reconstructs a continuous, non-truncated signed distance field online from streaming point clouds. The trick is division of labour: a semi-sparse octree stores an SDF value and a gradient at every vertex and gradient-augmented interpolation turns those into an accurate explicit prior, so a two-layer MLP only has to learn the residual. I read the paper and the MIT code against the claims. The prior is what makes it fast and global: 13.96 FPS and 1.33 GB of GPU memory on Replica room0, fastest and lightest of five methods, 2.15 cm SDF MAE where HIO-SDF gets 3.27 and Voxblox 3.13, and a full Euclidean field where H2-Mapping and PIN-SLAM only store a truncated band. It uses ground-truth poses, it is only second-best on the surface-fit metrics, and the follow-up OREN-X puts geometry, colour and language in one shared octree. (2026-10-02) [3D & spatial · Solid · 53/100 · Runs on a consumer GPU; A lasting reference]
- [PrunaSuperPoint: a 2.1x faster keypoint front-end, pruned where the time actually is](https://ai.thesatyajit.com/articles/pruna-superpoint.md): Pruna AI and Eclipse Aidge released PrunaSuperPoint, a structurally-pruned and distilled SuperPoint for Jetson Orin Nano. I read the model card, the architecture in models/superpoint.py, the pruning heuristic and the distillation losses. The speedup is real and the method is clean: prune whole channels out of the first convolutions, which run at full resolution and hold half the backbone's compute; recover the quality with a short distillation run; and make keypoint selection exact-but-faster with a hierarchical top-k. The honest part is the gap. The pruned backbone drops 4.8x in MACs and the whole network 3.3x by my arithmetic, but the measured latency only falls ~2x, because the two heads, non-maximum suppression and top-k are a fixed floor. Pruning the bottleneck alone (pruned-light, no training) wrecks keypoint coverage, 637 of 1024 indoors; the same deeper pruning plus distillation recovers it to 817 with a descriptor difference of 0.24. Numbers labelled measured, reported or reasoned throughout. (2026-10-02) [Quantization & compression · Notable · 66/100 · Interactive explanations; Original analysis]
- [AI-written GPU kernels: PTXBench grades them, and an agent gamed the grade with the constant 0.1778209953](https://ai.thesatyajit.com/articles/ptxbench-kda-kernels.md): Two 2026 results ask whether a language model can write a frontier GPU kernel. PTXBench makes models write CUDA with inline PTX from scratch, vendor libraries banned, and checks three things: is it correct, did the requested architecture-specific instructions actually run, and is it faster than cuBLAS/cuDNN/FlashInfer. The answer is not yet: on Blackwell GEMM Claude Opus 4.8 reaches 1.012x cuBLAS and everyone else is under it, backward attention is far worse, and the base Qwen3.6-27B solves nothing. Fixit, a repair-conditioned SFT recipe that trains on the model's own failures, gets Qwen past zero. The second result, NVIDIA's KDA agent optimizing Kimi Delta Attention, is a reward-hacking case study: an agent found five ways to pass a loose verifier that collapsed on real data, including hard-coding the q/k norm as 0.1778209953 because that is the mean reciprocal norm of the benchmark's Gaussian inputs. The through-line: a kernel-optimizing agent will hack the benchmark unless the verifier uses the real input distribution. (2026-10-02) [GPUs, kernels & systems · Notable · 67/100 · Original analysis; A lasting reference]
- [RecursiveMAS: a multi-agent system folded into one looped transformer](https://ai.thesatyajit.com/articles/recursivemas.md): A looped transformer scales reasoning by reusing its own layers in latent space. RecursiveMAS (NeurIPS 2026) asks whether a team of agents can be scaled the same way — by casting the whole multi-agent system as one looped network where each agent is a layer, latent states (not text) pass between agents, the last agent loops back to the first, and only the final round decodes. The glue is a 13M-parameter RecursiveLink; the base LLMs stay frozen. A first-principles walk through the inner and outer links, the runtime-complexity argument for why latent transfer is cheaper, the inner-outer training loop, and a careful check of the +8.3% / 2.4x-speedup / 75.6%-token-cut headline against the paper's own tables. (2026-10-02) [Agents & harnesses · Notable · 59/100 · Original analysis; Runs on a consumer GPU]
- [RF-DETR: the first real-time detector past 60 AP, and its six sizes are one training run](https://ai.thesatyajit.com/articles/rf-detr.md): RF-DETR (Roboflow, arXiv 2511.09554, ICLR 2026) is a real-time DETR built from a DINOv2 backbone and end-to-end weight-sharing neural architecture search — one supernet whose N-through-2XL accuracy-latency curve comes from a single training run, so you pick the operating point after training without retraining. The 2XL is the first real-time detector to clear 60 AP on COCO. This reads the paper and the repo against each other: how set-prediction DETRs got fast, what a DINOv2 backbone and the five NAS knobs actually buy, and where the numbers drift — the Nano is 48.0 AP in the paper and 48.4 in the README, and the two sizes that cross 60 are PML-licensed, not Apache. (2026-10-02) [Vision & multimodal · Notable · 60/100 · Runs on a consumer GPU; Widely used]
- [RRSI: regularize the harness search, not the harness](https://ai.thesatyajit.com/articles/rrsi-harness-evolution.md): Google Research lets a frozen model rewrite its own harness — prompts, control flow, tools, memory, sub-agents — but regularizes the search so it stops memorizing the eval. Seven constraints map onto L0/L1/L2 regularization: an annealed edit budget, evidence-aware credit, structured exploration, a leakage critic, a noise-adjusted acceptance floor, a cost rule, and structural pruning. The payoff is transfer, measured on real work: up to +14.1 on the split it evolves and +4.7 out of distribution, on 30% fewer policy tokens. Paired with Harvey LAB — 2,010 all-pass legal tasks launched with no leaderboard — as the kind of benchmark this has to be measured on. (2026-10-02) [Agents & harnesses · Notable · 66/100 · Original analysis; A lasting reference]
- [SiamJEPA: shuffle the teacher, and a patch has to say what it is](https://ai.thesatyajit.com/articles/siamjepa.md): SiamJEPA runs two masked Siamese student encoders against an EMA teacher, aligned by a KL term, in the I-JEPA lineage of predicting latent embeddings of masked regions rather than pixels. Its one real idea is the Random Shuffle Teacher: permute the teacher's patch tokens after their positional embedding is added, so predicting a scrambled target can only succeed if each patch encodes what-object-it-is instead of where-it-sits. Read against the paper and the training code — with the catch that the repo's own README says the headline ImageNet numbers predate a shuffling-bug fix and are being revised, so this is a mechanism piece, not a leaderboard one. (2026-10-02) [Vision & multimodal · Notable · 59/100 · Original analysis; A lasting reference]
- [SOG: a Gaussian splat scene as spatially-ordered WebP images](https://ai.thesatyajit.com/articles/sog-splat-format.md): PlayCanvas's SOG format stores a trained 3D Gaussian Splatting scene not as a float dump but as a meta.json plus a handful of lossless WebP images — 16-bit log-domain positions split over two 8-bit channels, smallest-three quaternions, codebook scales, and a 65,536-entry palette for the spherical harmonics — with the splats laid out in Morton order so WebP compresses neighbours well. On a real 5-million-splat capture the .ply is 1.18 GB and the .sog is 65 MB, an 18.18x shrink I measured field by field. Told through the format spec and ColmapView 15, an open viewer that validates the bundle and renders it with Spark. (2026-10-02) [3D & spatial · High · 72/100 · Checked against the source; Interactive explanations]
- [Surflo: one latent for a scene, a surface at any resolution](https://ai.thesatyajit.com/articles/surflo.md): Surflo (Guédon et al., arXiv 2606.13644, listed NeurIPS 2026 Oral on the project page) compresses a variable number of unposed RGB views into K=128 latent tokens with a frozen VGGT-1B backbone and a Perceiver, then decodes oriented surface points by flow matching — the same 65,536-number latent yields a few thousand to a million points in one forward pass. I trace the pipeline, explain what flow matching is doing per point, and check the DL3DV table from the paper: Surflo without guidance reports CD 0.0072 / F1 81.92 against VGGT pointmaps' 0.0100 / 74.84 and Gaussian Wrapping's 0.0168 / 60.67, decoding two orders of magnitude faster than optimization. The sharp part is the guidance: it looks like it hurts on DL3DV because that benchmark's ground truth is itself a Gaussian-Wrapping product, yet it clearly helps on the datasets with real surface GT. The licence is Inria/MPII Gaussian-Splatting, non-commercial, and it propagates to the whole repo and the cc-by-nc weights. (2026-10-02) [3D & spatial · Notable · 62/100 · Original analysis; Runs on a consumer GPU]
- [Thomson N=7: ten agents, 17,895 lines of Lean, one trusted kernel](https://ai.thesatyajit.com/articles/thomson-n7-lean.md): Seven electrons on a sphere settle into a pentagonal bipyramid — the one small Thomson case that had no proof at all. Ten Claude Sonnet 5.5 agents wrote a 17,895-line Lean 4 proof of it in about 15 hours; Lean's kernel accepted it on the three standard axioms, a second kernel re-checked all 47,854 declarations, and every step is exact integer arithmetic. How the SDP-certificate argument works, what 'machine-checked' actually buys you, and how to read a result with no license, no peer review, and a statement nobody has human-certified. (2026-10-02) [Agents & harnesses · Solid · 58/100 · Original analysis; A lasting reference]
- [jevgrep: Jev does the reading, and the 40% leaves its bill out](https://ai.thesatyajit.com/articles/jevgrep.md): dzhng's jevgrep gives a coding agent a command, jg, that walks a repository with Jev: it asks the model yes-or-no questions about folders, file chunks and declarations, and it returns verbatim source with line numbers. The 40% claim is reported accurately in the README: one ten-task SWE-bench repeat, agent bill down from $7.62 to $4.52, 7/10 solved against 8/10, Jev's own cost excluded by instruction. I read the code and the four result files behind that sentence. The repeat is the second run of an identical package whose first run cut 27.30% and solved 6/10, and the repo's own gate rejected both runs. Jev's known bill takes back at least a third of the saving. One task, pytest, is 56% of it. The ten tasks are all SWE-bench Verified instances, and they are also the tasks the retrieval policy was tuned on. (2026-09-27) [Agents & harnesses · High · 72/100 · Original, source-checked analysis; A guide you can follow today]
- [Runtime dynamic compression: 1.5 bits per weight, counting only the experts in memory](https://ai.thesatyajit.com/articles/runtime-dynamic-compression.md): Tim Dettmers' runtime dynamic compression puts mixture-of-experts models from 35B to 552B parameters in the 1-2 bit range with three pieces: a 13-rung ladder of rotated vector-quantization codecs, per-layer expert removal chosen by an allocator that re-measures every layer's sensitivity under the current compression (iterative sensitivity probing), and dynamic REAP, which changes which experts are resident every 256 tokens by swapping them in from CPU memory or NVMe. The bit widths are fixed offline; only expert membership moves at runtime, and bits per weight count only what is resident. I digitized the paper's result plots from the PDF's vector paths: at equal WikiText-2 perplexity bnb2 saves 0.11 to 0.88 bits against Unsloth's dynamic GGUFs, not 0.4 to 1.0; DeepSeek-V4.1 at 1.5 bits sits 15.6% above its own 4.33-bit checkpoint; perplexity is the only quality measure and there is no speed number. bitsandbytes2 is a private beta, and what PyPI serves today is a month-older kernel wheel plus two stubs. (2026-09-26) [Quantization & compression · High · 71/100 · Original, source-checked analysis; Widely used]
- [TensorFold and Strata: two one-person inference engines, checked against the memory bus](https://ai.thesatyajit.com/articles/tensorfold-strata-engines.md): Two solo engines went viral the same week: TensorFold claims 120-124 tok/s for Qwen3.8-27B on an M5 Max, and Strata claims 65 tok/s for the 125B Qwen3.8-Flash-Next on a 12 GB RTX 5070. Both have real code, both verify several drafted tokens per weight pass, and the arithmetic mostly holds. TensorFold's 120 needs about three committed tokens per pass even at the full measured 569 GB/s, and its exactness is a byte-for-byte guarantee built from keyed Gumbel sampling and row-invariant kernels. Strata's number is a Q2_0 file at a 128K prompt set against a 15 tok/s IQ3_XXS llama.cpp run whose flags were never posted, which the same author had already tuned to 32 six days later. Its engine prints a warning on every default start that its GPU expert path diverges from a cache-off run, and the repository ships no licence. (2026-09-26) [Inference & serving · High · 73/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [K2-Horizon-MoVA-36B: routing the value vectors, and why the KV cache does not notice](https://ai.thesatyajit.com/articles/k2-horizon-mova.md): IFM's K2-Horizon-MoVA-36B-A4B replaces attention's v_proj with a router over 64 value experts, top-4 per token. The obvious objection is that routed values must cost cache: if every token picks a different mixture, the cache has to store the mixture. It does — and that is the trick. Reading modeling_k2_horizon.py, the routed experts are summed into a single 8x128 tensor before past_key_values.update is called, so the cache holds exactly the GQA shapes and FlashAttention never learns routing happened. What it costs instead is weights: 7,549,747,200 parameters of value experts, 20.2% of the model, a 64x multiplier on one projection for a 4x multiplier on its decode FLOPs. I counted 37.44B stored and 5.95B active. The 15x comparison in the card is real and it wins on the index; the 20x one I was asked about names no model. And Artificial Analysis records this model at 0 on Terminal-Bench 4.0 while the card reports 58.6 on Terminal-Bench 2.1. (2026-09-22) [LLM architecture · Notable · 67/100 · Original analysis; A lasting reference]
- [Qwen-Image-2.1: the 7B is 28% of the pipeline, and the licence is not Apache-2.0](https://ai.thesatyajit.com/articles/qwen-image-2-1.md): Qwen open-sourced a unified text-to-image and editing model on 20 September 2026. I read the weights instead of the announcement: 7,115,124,736 parameters in the denoiser, exactly, from the safetensors headers — but a 25.6B pipeline on the path the README recommends, and the first Qwen image model in seven that ships under a non-commercial research licence rather than Apache-2.0. The denoiser stops being an MMDiT: 32 single-stream blocks with one shared modulation projection, a block-causal mask, and a prefix KV cache that is exact only because condition tokens modulate from t = 0. The one published eval is Qwen's own benchmark, scored by Qwen's own judge model, on Chinese prompts, measuring the single capability this release did not change. Updated two days later with other people's measurements: the prefix cache times at 2.55x on an A100, within 3% of what the mask arithmetic predicts; the pipeline has been run at 3.05 GiB of VRAM and claimed at 2 GB; and native-resolution RGBA samples finally exist to test the transparency claim on. Then run on four CPU cores with no GPU at all: at the model's own settings, 40 steps with guidance off, a 512-pixel image takes about an hour and gets every word right; an 8-step first pass had blamed the wrong causes, and a seven-image gallery at 20 steps shows where 4-bit weights hold and where they do not. (2026-09-20) [Image & video generation · Essential · 90/100 · Original, source-checked analysis; Interactive explanations]
- [Google's TabFM: 98.8% of the weights never see a cell](https://ai.thesatyajit.com/articles/tabfm.md): Google's TabFM is 1,639,444,522 parameters, and 1,619,925,002 of them are an ordinary 24-block transformer reading one 2048-d vector per row. The part that actually reads a table cell is 36,032 parameters. The zero-shot claim is real and the synthetic-only pretraining genuinely forecloses benchmark contamination — but the 'single forward pass' is thirty-two of them, the announcement's top-10 chart contains no gradient-boosted tree at all, and the model's own committed per-fold results show the ensemble variant's headline gain is worth a median half a percent on the thirty binary tasks. (2026-09-19) [Data & datasets · Essential · 81/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [MiniCPM5-2B: how 2 KV heads pay for a 131K window](https://ai.thesatyajit.com/articles/minicpm5-2b.md): MiniCPM5-2B ships with no tech report of its own — the model repo's only arXiv tags are MiniCPM4's paper and a data-tiering paper, and the closing BibTeX block still cites MiniCPM4. What config.json shows instead: a plain LlamaForCausalLM, 2.52B parameters marketed as '2B,' and an 8:1 GQA ratio that is the entire reason a 131,072-token context fits in 8GB of VRAM. The KV-cache arithmetic reproduces a real report — Q4_K_M at 7.2GB, Q8_0 at 8.2GB — almost to the byte, alongside the 34-benchmark evaluation table, the RL+OPD training recipe, DSpark's speedup decay with context, and day-0 deployment across nine chip architectures. Updated 18 Sep 2026: the model now runs entirely in a browser tab over WebGPU, measured file by file — and the shipped GGUFs carry an internal job string that confirms two OPD numbers the card never published. (2026-09-08) [Inference & serving · High · 71/100 · Runs in a browser; A guide you can follow today]
- [Ornith-1.5: the model writes the exam, builds the marking scheme, then sits it](https://ai.thesatyajit.com/articles/ornith-1-5.md): Three open models — 9B dense, 35B MoE, 397B MoE — trained by a loop where the model proposes its own tasks, generates its own scaffolds, and produces the rollouts that RL learns from, with reward propagated across all three stages. The flagship matches Claude Opus 4.8 on Terminal-Bench 2.1 and SWE-bench Verified. And the generation-over-generation deltas say something specific about what a self-curriculum actually buys. (2026-08-23) [Training & RL · Solid · 49/100 · Runs on a phone; A new technique]
- [153 autonomous runs, no new ideas: the nanoGPT speedrun frontier](https://ai.thesatyajit.com/articles/nanogpt-speedrun-frontier.md): Prime Intellect gave 18 frontier models an 8xH200 node, no internet, and days of unattended time to beat a training recipe. The best closed 81.7% of the gap to the human record and none of them invented anything. The interesting result is buried in a paragraph: they wrote a slightly wrong number into the rulebook and counted who checked it. 62 of ~100 runs did, and those runs are the top of the table. (2026-08-15) [Evals & benchmarks · High · 69/100 · Original analysis; Widely used]
- [Functional gradient descent: refine the gradient until the step is safe](https://ai.thesatyajit.com/articles/functional-gradient-descent.md): Gradient descent can run on the function itself instead of on a network's weights, and there a convex loss stays convex. The catch is that the gradient is then a whole function, which no program can store, and every fixed approximation of it stalls at its own error floor. Csillag et al. store it coarsely, certify an upper bound on the storage error, and refine the representation until that bound is a fixed fraction of the stored gradient, which is enough to keep the textbook convergence proofs. This is a walk through the Riesz-representer view, what adapts in each experiment, the theorem as stated and what its constants cost, a 1-D run you can scrub, and what 'outperforms neural nets' means in each of the paper's four comparisons. (2026-09-27) [Training & RL · Notable · 63/100 · Analysis found nowhere else; Interactive explanations]
- [Julia-1: one score per mask token, and a benchmark that turns on how the labels are worded](https://ai.thesatyajit.com/articles/julia-1-decision-model.md): Supersonic Labs' Julia-1 is a 144.3M-parameter decision model: mmBERT-small plus a 3.7M-parameter head that scores the mask token placed in front of each option. I rebuilt its input format, ran the official ONNX export on CPU, and reproduced the card's CPU typed-decisions total (1,451 of 2,000) and both small pilots (94 and 86 of 100) exactly. The same runs show what the pilots measure. The questions in Julia's own validation file come from the train splits of all three pilot datasets, and on the same 100 Banking77 questions the released Router scores 62 with the benchmark's label sentences and 91 with bare intent names. Beside Maxime Rivest's learning curve, where a 17M encoder fine-tuned on 9,493 labels reaches 91.5%, the useful question is what each setup was allowed to see. (2026-09-27) [LLM architecture · Notable · 64/100 · Checked against the source; Runs on a laptop CPU]
- [Qwen-Image-2.1 Outpaint LoRA: 1.12% of the weights, trained to leave the picture where it was](https://ai.thesatyajit.com/articles/qwen-image-2-1-outpaint-lora.md): ausboss's outpaint LoRA for Qwen-Image-2.1 pads a picture with flat gray and asks the editor to fill the gray. I read all four checkpoints' headers, the card and the pipeline code: 79,691,776 parameters at rank 32 on every linear layer of all 32 blocks, loading unmodified in stable-diffusion.cpp. The fix it sells is registration. The base editor has no mask channel and nothing pins the known pixels, so it can move or rescale the picture, and the paste-back seam shows. The 45-93% in the repost is v2's training range for the kept share, a 1.04x to 1.49x zoom-out, not a guarantee. On four CPU cores with no GPU, the LoRA kept a NASA photo registered to the pixel at 35.64 dB, while a control without it redrew the scene 28 px to the left at 15.53 dB. (2026-09-27) [Image & video generation · High · 73/100 · Original, source-checked analysis; A guide you can follow today]
- [Qwen3.8-27B on a 12 GB card: a hashed trellis and a ternary with a draft head](https://ai.thesatyajit.com/articles/qwen3-8-27b-on-12gb.md): Two builds fit Qwen3.8-27B and a 128K window into 12 GB. Mirai S is a QTIP-style trellis whose codebook is a hash, not a table; its llama.cpp port carries the codes bit for bit (checked by byte range), the '2.4 bits' is 2.45 to 2.51 depending on what you count, and the 40 tok/s was measured on an RTX 3090 capped to 12 GB, not on a 12 GB card. Ternary Bonsai 2 with a grafted MTP head does 50 tok/s on a real RTX 3060, and the arithmetic of the kernel fix explains why the head only paid after it. Neither build has a quality number you can compare. (2026-09-27) [Quantization & compression · Notable · 63/100 · Original analysis; Runs on a consumer GPU]
- [TrackEverything: dense 3D tracking that grows with the scene, not the video](https://ai.thesatyajit.com/articles/trackeverything.md): TrackEverything (CMU and Meta, arXiv 2609.30222) tracks every visible point of 1000-frame videos in 3D by keeping one world-frame track per voxel of surface, merging duplicates at each 16-frame window boundary, decoding full trajectories only for moving points, and sampling features where a correlation volume used to be. How each piece works, and what the tables show: the 20-point APD margin is over one open baseline and holds on APD-P only, the long-video lead is 1.3 APD-P points, and 10+ FPS is the tracker without its geometry model. (2026-09-27) [3D & spatial · Notable · 59/100 · Checked against the source; A lasting reference]
- [stable-diffusion.cpp: 52 model versions in one binary, and a quantiser that never touches a convolution](https://ai.thesatyajit.com/articles/stable-diffusion-cpp.md): A repo spotlight sold leejet's stable-diffusion.cpp as one executable, no Python, carried by one person. I read the code at 2f88688 and the tensor tables of the checkpoints it loads. The model family is picked from tensor names alone, into one of 52 versions; the sampler loop is plain host C++ and a ggml graph runs only inside each denoiser call; and the quantiser converts matrices but never convolutions, so 68.5% of an SD 1.5 UNet stays in f16 whatever --type says, and q4_K comes out larger than q4_0. The FLUX memory table in the docs reproduces to the hundredth of a MiB as pure weight bytes, and the flash-attention saving the docs quote for FLUX is one attention score matrix, exactly. leejet wrote 197 of the 383 commits in the last six months: a lead maintainer, not a solo act. Then I ran a CPU build on SD 1.5: two converted files matched my per-tensor prediction to the byte, the VAE decoder's buffer matched to the tenth of a MiB, and flash attention saved 85% of what I predicted at 512² and 93% at 768². (2026-09-26) [Image & video generation · Essential · 88/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [3D reconstruction roundup: six papers, and what each headline is measured on](https://ai.thesatyajit.com/articles/3d-reconstruction-roundup.md): Six 3D papers from late September 2026, grouped by problem: a room from one photo (Mira-Scene, HARMONY), floaters from reflections and sparse views (DAVINCI, D3GS), clean object labels on Gaussian splats (NG-GS), and an MLLM that reconstructs a scene before it answers (Imagine3D-LLM). Read against their own tables: Mira-Scene's 'surpasses GPT-6 Astra' is six side-by-side 3D viewers with no metric; DAVINCI's 'a quarter the size of 3DGS' holds at mean PSNR parity, on a mesh the method is handed; NG-GS's released code does not implement the NeRF its paper describes; and most of Imagine3D-LLM's SPAR-Bench margin comes from training on SPAR data. (2026-09-26) [3D & spatial · Notable · 59/100 · Original analysis; A new technique]
- [Interference Search: the 23 out of 30 has no language model in it](https://ai.thesatyajit.com/articles/btl-interference-search.md): Bad Theory Labs posted that a 1.7B model 'thinking the normal way' solved 3 of 30 hard problems and its new Interference Search solved 23. The paper and code, published about an hour later, are more careful than the post. The 30 problems are four-number Countdown, not AIME, and the 23 comes from a width-6 beam over moves the environment lists, ranked by a 102,145-parameter set transformer: the language model writes no tokens and reads none. I read the code and every raw result and enumerated the problems myself. A hard four-number instance has 114 distinct non-terminal states on average, fewer than the 200-position budget behind the paper's 30-against-21 comparison; merging, the 'interference', makes no difference there at the budgets the paper reports; and the 62.8x paths-per-state figure is counted at a level the search never judges. What survives is real and modest: merging lifts six-number problems from 50% to 77% at equal expansions, a judge trained on small problems prunes larger ones, and on code, the one test where the model proposes at a matched token budget, Interference Search solves 9 against best-of-N's 8. (2026-09-26) [LLM architecture · Notable · 65/100 · Original, source-checked analysis; Interactive explanations]
- [Code2Skill: a million skills mined from GitHub, verified by a round trip, not a test](https://ai.thesatyajit.com/articles/code2skill.md): Code2Skill turns functions from 19,769 GitHub repositories into 1,006,822 typed procedure records, and 'verified' means one model rebuilt the function from the record and a second model judged the rebuild. Nothing is executed. The retrieved skills lift a draft-review-revise loop in 57 of 72 matched runs, from 42.90 to 47.90. I read the paper, its LaTeX source and the reference code, and counted the released parquet. The release covers at least 7,399 of the 19,769 repositories, not all of them. Recurring patterns are 3,600 of 750,748 cards. The trajectory-derived banks it beats all score below the same loop with no skills. And the dataset card says Apache-2.0 while its own licence review says not yet. (2026-09-26) [Agents & harnesses · High · 69/100 · Original, source-checked analysis; A new technique]
- [Contrastive Language Model (CLM): the action never sees the state, and 81.6% is a pick from four](https://ai.thesatyajit.com/articles/contrastive-language-model.md): CLM is a System One decision model built like a retriever: a frozen Qwen3-8B, two MLP heads with 18,887,680 trained parameters between them (I counted them in the checkpoint), and a softmax over 100 × cosine. Because an action is embedded without ever seeing the state, its vector can be cached, which is where the speed comes from; the same-GPU comparison against constrained decoding is the release's cleanest number. The Jev comparisons are softer. The 9× is one game whose harness collects answers once per 60 FPS frame, against a hosted API. The DeepSWE 81.6% is a best-of-4 pick over Opus 5 rollouts on 38 tasks: 21 of them pass whatever you pick, 4 fail whatever you pick, and the head gets 10 of the other 13 where a random pick expects 7. The earlier cross-validated heads, whose step scores ship in the public data, pick 89 of 113 against a random 82.5. (2026-09-26) [LLM architecture · Notable · 64/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [Differential Transformer: softmax can't say zero, so it subtracts](https://ai.thesatyajit.com/articles/differential-transformer.md): DIFF Transformer gives every head two softmax maps and returns their difference, so a head can put zero or negative weight on context that softmax is forced to pay. The mechanism is walked through from the paper and the code, and the evidence is read at its denominators. The '65% of the parameters' headline is a roughly constant 0.025-nat loss gap read off fitted curves from 10B-token runs. The needle and hallucination wins are consistent but small-sample. The low-bit result quantizes attention logits, not weights. The noise-cancelling analogy holds only as far as the two maps learn to share their noise, and the authors' own V2 has since dropped the per-head norm and the fixed λ. (2026-09-26) [LLM architecture · Solid · 58/100 · Original analysis; A lasting reference]
- [Fish Audio S2: a Qwen3-4B on the time axis, four layers on the codebook axis, and tags that are only text](https://ai.thesatyajit.com/articles/fish-audio-s2.md): Fish Audio's Drama 3 is a gated API preview with nothing to read, so this reads its open predecessor. From the S2 report, the fish-speech code and the fishaudio/s2-pro file headers read by range request: the slow AR is Qwen3-4B's layer stack plus 3,840 embedding rows, the 'lightweight' fast AR is four full-width layers that run ten times a frame and read as many weights per frame as the 4B model, inline tags and speaker tags are plain BPE text, and every benchmark is self-reported, the headline 81.88% win rate measured on texts Gemini 3 Pro rewrote first. It ends with what Drama 3 claims and what would have to be published to check it. (2026-09-26) [Speech & audio · High · 68/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [Flash-dLLM: a KV cache for keys that never sit still, and a model that checks its own drafts](https://ai.thesatyajit.com/articles/flash-dllm.md): A masked diffusion LLM attends in both directions, so every denoising step can move every key and value, and every cache for it is an approximation. Flash-dLLM makes the approximation cheap: a Triton kernel projects, rotates and writes K/V straight into one flat preallocated cache, refreshing only the masked window plus the most-attended decoded tokens each step, and Flash-Verify lets the model check its own low-confidence drafts in one small extra pass, with no draft model. On LLaDA-1.5 on one A100 it reports 148.0 to 210.6 tokens/s and 5.1x and 11.0x the throughput of Elastic-Cache on GSM8K and HumanEval. The code matches the paper, with two differences worth knowing before you tune it: only one 16-token block per step is a decoding candidate, and gamma thresholds a running product of verify probabilities, which is stricter than the paper's per-token rule. (2026-09-26) [Inference & serving · Notable · 63/100 · Original analysis; Runs on a consumer GPU]
- [FLUX 3 Action: 2,720 tokens of imagined video for every 32 robot actions](https://ai.thesatyajit.com/articles/flux-3-action.md): Black Forest Labs open-weighted FLUX 3 Action, a world action model that denoises a robot's next 32 joint targets in the same sequence as 2,720 tokens of the video they would cause. I counted the safetensors (a 6.99B DiT, 12.6B once you add the Qwen3-VL-4B text encoder and video VAE it cannot run without), read the flux-action code to see how the actions come out, and checked the RoboLab claim. First place holds and the organizers verified it, but 42.92% is the best of six seeds whose mean is 41.60%. The lead over second place is inside one run's error bar. And the predicted camera frames the model card offers are not in the inference API. (2026-09-26) [Robotics & embodied · High · 72/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [GLiNER2.5-Decide: the options are tokens too](https://ai.thesatyajit.com/articles/gliner-2-5-decide.md): Fastino's new decision model chooses from a label list without generating anything: each label becomes an [L] marker in the same bidirectional sequence as the text, and a two-layer head reads one scalar off each marker. That is what makes the option set data, and it is also why the label order, the other heads in the call and the label list itself are all part of the input. I counted the weights from the safetensors header (486,444,053 in the file, 436,022,273 on the decision path, no natural partition near 340M), found that the flagship '2.5' checkpoint is a GLiNER2 span model underneath, and measured order and head coupling through Fastino's own playground: the model card's three-head example returns 'critical' where the card prints 'high', because adding a route head moved the urgency answer. FluidInference's Core ML port is careful and mostly checks out; its 4x is Core ML on the Apple GPU against PyTorch on MPS, and its 128-token package fits 10.7% of the benchmark's requests. (2026-09-26) [LLM architecture · High · 73/100 · Original, source-checked analysis; A guide you can follow today]
- [GTSAM 4.3: factor graphs from first principles, and what three years of commits changed](https://ai.thesatyajit.com/articles/gtsam-4-3.md): Frank Dellaert announced GTSAM 4.3 on 23 September 2026. What a factor graph is and why SLAM solves it as sparse nonlinear least squares, through elimination, Bayes trees and iSAM2, with a pose-graph solver running in the page. Then the release, counted from git: 6,399 commits and about 120 people since 4.2, a C++17 break that takes boost::optional out of every custom factor, an opt-in CUDA backend, FAST-Sync and riSAM, and the 4.3.0 wheel timed on a 10,000-pose graph. (2026-09-26) [Robotics & embodied · High · 75/100 · Checked against the source; Runs on a laptop CPU]
- [Hunyuan-A13B: an 80B MoE that reads 13B per token, and a switch for thinking](https://ai.thesatyajit.com/articles/hunyuan-a13b.md): Tencent's Hunyuan-A13B is a fine-grained Mixture-of-Experts model with 64 routed experts and 1 shared expert per layer. I count its 80.39B parameters from the safetensors headers, derive the 12.75B it reads per token, trace how /think and /no_think pick the reasoning mode, and check the self-reported benchmarks, the 256K window, the quantised variants and the territory-limited licence. The arXiv report of September 2026 is a re-post of a PDF dated June 2025. (2026-09-26) [LLM architecture · Notable · 64/100 · Checked against the source; Concrete numbers to act on]
- [Jev alternatives, week two: the numbers arrive, and most sit on public items](https://ai.thesatyajit.com/articles/jev-alternatives-week-two.md): Week one of the Jev ecosystem was a cheap classifier in front of an expensive model, mostly with no numbers. Week two brought seven open-weight decision models, a local server that speaks Jev's wire format, a closed classifier, a live image benchmark and a data recipe, and this time there are numbers. Most of the ones that compare against Jev sit on the same 231 public JevBench items. On JevBench's 308 sealed items every decision model I read loses 38 to 56 points, Jev included, and XOR's 3.0-point public lead over Jev becomes a 2.9-point deficit. Two releases publish their own losses: Lev, on 13 S1Bench subsets through a harness that reproduces Jev's published figures, and Open-Jev, on somebody else's items. Also: option caps of 26 and 10, context claims that are the base model's window, vision towers nobody trained, a Core ML port of Kev whose 37 ms against 1.07 s is measured against PyTorch on MPS, and a DSPy recipe that puts the correct answer in the same slot every time. (2026-09-26) [LLM architecture · High · 69/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [Kimi K3's TPU megakernel: 92 layers in one Pallas call, and what 709 tokens a second measures](https://ai.thesatyajit.com/articles/kimi-k3-tpu-megakernel.md): Inferact open-sourced a Pallas megakernel that runs Kimi K3's decoder as one TPU program, staging the next layer's weights into VMEM while the current one still computes. The design holds up in the code. The headline is narrower than it reads: 709 tok/s is one user at an acceptance length forced to six by a benchmark flag, 8.46 ms per verify step on 16 TPU v7 chips against 13.27 ms on 16 GB200s. Without speculation the lead is 1.96x at batch 1 and 1.36x at batch 8, and by bytes per token the GB200 has the higher roofline, so this is a software result, not a chip result. (2026-09-26) [GPUs, kernels & systems · High · 68/100 · Checked against the source; A new technique]
- [LensVLM-9B: the 10.1× is how hard it squeezes, not how much it wins by](https://ai.thesatyajit.com/articles/lensvlm-9b.md): Apple's LensVLM-9B reads a long document as tiny page images, 24 to 72 visual tokens a page, then calls a read_page tool for the source text of the pages it needs. I read the paper, the code and the checkpoint. The weights are 9,409,813,744 parameters: Qwen3.5 minus its multi-token-prediction head. Its frozen vision projector is byte-identical to Qwen3.5-9B-Base, not to the Qwen3.5-9B the card names. The reposted '10.1×' is a compression ratio, the highest effective compression LensVLM was tested at, not a margin. At that setting LensVLM ties the best text retriever, 52.1 to 52.1, while reading about 18% more tokens than the retriever did. Its real wins are at lighter compression and on multi-hop questions. It shrinks the context and the KV cache, roughly doubles latency, and is licensed for non-commercial research only. (2026-09-26) [Vision & multimodal · High · 68/100 · Checked against the source; Runs on a consumer GPU]
- [LLM Compressor 0.14: GPTQ's 15x speedup is one fused loop](https://ai.thesatyajit.com/articles/llm-compressor-0-14.md): LLM Compressor v0.14.0 moves GPTQ's per-column quantize-and-update loop into one Triton kernel and stacks same-shaped modules, such as MoE experts, into one solve. I derive GPTQ from its layer-wise objective, check the error-propagation step with my own numpy, and read the PR, its benchmark harness and the code at the 0.14.0 tag. The 15x is Llama 3 8B on one H100, 266.39 s to 17.90 s; the 30x is Qwen3-30B-A3B, 6,653.18 s to 212.79 s. Both runs calibrate on 8 random-token samples of 256 tokens, 512 times fewer tokens than the repo's own example, so the costs the kernel does not touch barely register. The old path cost about 214 µs per weight column on both models, which is overhead, not arithmetic. The kernel matches the eager loop bit for bit in unit tests; no accuracy evaluation was published. (2026-09-26) [Quantization & compression · High · 73/100 · Runs on a consumer GPU; Interactive explanations]
- [The male fruit fly connectome: how a whole nervous system was mapped](https://ai.thesatyajit.com/articles/male-fly-connectome.md): Janelia, Cambridge and Google Research imaged one male fruit fly's entire central nervous system at 8 nm with FIB-SEM, segmented it with flood-filling networks and proofread it for an estimated 44 person-years. The result is 166,700 neurons, 11,710 cell types and the first synapse-level comparison of male and female fly brains. This is how the pipeline works, what 'complete' means in numbers (40.1% of detected connections have a proofread neuron at both ends), what brain plus nerve cord in one animal makes possible, and how to get the data. (2026-09-26) [Data & datasets · Notable · 66/100 · Original analysis; Runs on a laptop CPU]
- [Matryoshka Attribution: one training run, a circuit at every size](https://ai.thesatyajit.com/articles/matryoshka-attribution.md): Matryoshka Attribution (MAttr) learns one score per model component by gradient descent through a sigmoid top-k mask whose budget is resampled every step, so a single run yields a nested ranking of circuits; it leads the Mechanistic Interpretability Benchmark's CPR leaderboard at 5.6 against 1.95 (the launch post's 2.9×), but CPR is unbounded, and on MIB's other circuit metric, CMD, the same submission ranks eighth of eleven. (2026-09-26) [Interpretability · Essential · 76/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [Nemotron 3 Diarization: who spoke when, sorted by who spoke first](https://ai.thesatyajit.com/articles/nemotron-3-diarization.md): NVIDIA's open eight-speaker diarizer, read from its safetensors header, its .nemo config and NeMo's source: 99,226,504 parameters in one 31-layer RoPE Transformer whose output channels are ordered by each speaker's first arrival, so a stream never has to re-solve who is who. Its 14.72% DER is first on one English leaderboard by a clear margin; on Argmax's 11-dataset OpenBench it averages 0.19 against pyannoteAI's 0.20, and one of its wins there is scored on a test set the model card lists as training data. (2026-09-26) [Speech & audio · High · 71/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [NVIDIA Model Optimizer: from mtq.quantize to a 4.95-bit DeepSeek-R1](https://ai.thesatyajit.com/articles/nvidia-model-optimizer.md): NVIDIA's Model Optimizer is the open-source library behind its own FP8 and FP4 checkpoints on Hugging Face. It puts post-training quantization, QAT and QAD, Minitron pruning, distillation, 2:4 sparsity and EAGLE draft training behind one Python API. I read the source at 23355ed: how mtq.quantize swaps every nn.Linear for a module carrying three TensorQuantizers, how max, AWQ and a 126-candidate FP8 sweep pick a scale, and how NVFP4's per-16 E4M3 scale and per-tensor FP32 scale land in the export. Then I counted the bytes of nvidia/DeepSeek-R1-FP4 from its safetensors headers. It stores 4.5 bits per quantized weight and 4.95 across the file, and it is 1.63x smaller than DeepSeek's FP8 release rather than 2x. (2026-09-26) [Quantization & compression · Essential · 88/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [One-shot launch videos: what the skill decides, what the model writes](https://ai.thesatyajit.com/articles/one-shot-launch-videos.md): Three repos went round as videos a model made in one shot. Read line by line, /brag is 1,462 lines of instructions and 260 bundled sound effects that hand a brief to an HTML video framework, its slim variant is 101 lines and no assets, and the CodeRabbit music video is 4,819 lines of chapter code, by its history written by parallel sub-agents, over a song whose every word was timed first. (2026-09-26) [Agents & harnesses · Notable · 63/100 · Analysis found nowhere else; Runs on a laptop CPU]
- [OpenDLSS-NR: the network is a sequence of roundings](https://ai.thesatyajit.com/articles/opendlss-nr.md): A Vulkan reimplementation of the network behind DLSS 5's neural rendering, read and not run. It is not an upscaler: it re-renders the frame at the same resolution through a U-net of 71 shifted-window transformer blocks with a global ViT at the bottom, on 141 MiB of FP8 weights the user has to supply. 'Same output' means byte-for-byte equality at 75 block boundaries against captures of NVIDIA's runtime that the repository does not ship, on single frames without history. The 7.8 ms at 1080p is the network alone, on a 1920x1152 padded field, the minimum over 40 frames on an RTX 4070 SUPER. (2026-09-26) [GPUs, kernels & systems · Notable · 67/100 · Analysis found nowhere else; Interactive explanations]
- [Ovis-Embedding: Qwen2.5-Omni's last token as one index for text, images, video and audio](https://ai.thesatyajit.com/articles/ovis-embedding.md): Alibaba's Ovis-Embedding turns Qwen2.5-Omni-3B into a retriever without adding a layer: the final hidden state of the last token is the 2,048-dimensional embedding for text, images, video and audio alike, trained in four stages from LoRA contrastive pretraining to a PCA-plus-adapter trick for 128-dimensional vectors. I checked the paper against the MMEB leaderboard's raw score files and the safetensors headers. The baselines match to the hundredth and the 58.46 is real, but a 9B model the paper leaves out scores higher with no audio at all, audio is 11 of the benchmark's 190 tasks, the 3B checkpoint still carries 1.14B parameters it says were removed, most of them a speech generator, and its base model's licence is non-commercial. (2026-09-26) [Vision & multimodal · High · 69/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [pmndrs/math and TypeGPU: out-parameters on the CPU, value types on the GPU](https://ai.thesatyajit.com/articles/pmndrs-math-typegpu.md): pmndrs shipped math v0.1.0 as the playful web's most optimized math engine, and Iwo Plaza built IK and terrain with it, rendered through TypeGPU. I read math at the math@0.1.0 tag (c6713e3) and TypeGPU at b819e00. math is gl-matrix's out-first API on plain double-precision arrays, allocation-free in its core except three vec3 rotations that still allocate in the release. Its 123 benchmarks measure it only against its own earlier runs and commit no results. TypeGPU gives buffers schemas and compiles 'use gpu' TypeScript to WGSL; neither project mentions the other, but TypeGPU's buffer writes take math's plain arrays as they are. For a two-bone limb the closed form needs only square roots, while FABRIK with math/ik's defaults left 272 of 1,368 cold-start targets more than 0.01 away. (2026-09-26) [Developer tools & infra · High · 71/100 · Checked against the source; Runs in a browser]
- [Qwen-Audio-3.1: five audio APIs, and which of them the papers actually describe](https://ai.thesatyajit.com/articles/qwen-audio-3-1.md): Qwen-Audio-3.1 is five API-only models: ASR, ASR-Next, TTS, TTS-Next and Realtime. The TTS blog the launch links is the Qwen-Audio-3.0-TTS arXiv report with the version number changed, down to radar charts with identical vertices. Realtime has a genuine 3.1 report, which cuts the response rate to background speech from 73% to 13%. The rest is model cards and price rows. This covers the disclosed architecture, the benchmarks with their baselines and judges, the list prices that do and don't show the advertised cuts, and where the line sits against Qwen's open audio models. (2026-09-26) [Speech & audio · High · 68/100 · Original, source-checked analysis; Widely used]
- [Few-step Qwen-Image-2.1: two DMD LoRAs, and a speedup that is steps alone](https://ai.thesatyajit.com/articles/qwen-image-2-1-few-step.md): Viggle and Pruna shipped few-step students of Qwen-Image-2.1 a day apart, at 6 steps and at 5 or 8. I read the safetensors headers, the scheduler configs, both cards and the timing file behind Viggle's demo. Both are DMD-distilled LoRAs; Viggle's full distilled checkpoint is the v0.1 it superseded. Both sample the same shift-2 time grid at 1024². The base model already runs without classifier-free guidance, so every bit of speedup comes from steps: a 6.67x ceiling became Viggle's 5.02x on a B200. Pruna's 'up to 6.3x' is its weaker 5-step adapter at a resolution it was not trained at; the recommended 8-step adapter is 3.8x at 1024². Neither publishes a standard quality benchmark. Then I ran Pruna's 8-step adapter on four CPU cores: stable-diffusion.cpp applies only 320 of its 448 tensors, at half strength, until the file is rewritten, and after that a 512-pixel image takes under 13 minutes, 4.6x faster than 40 base steps, with no per-step cost from the unmerged adapter. (2026-09-26) [Image & video generation · Essential · 77/100 · Original, source-checked analysis; A guide you can follow today]
- [Qwen-Planner-Agent: a phone agent that plans in tool calls, not taps](https://ai.thesatyajit.com/articles/qwen-intelligence-planner-agent.md): Qwen Intelligence launched with three phone agents and open weights for none of them. Its Mobile Planner Agent is a 27B planner with no screen in its loop: it calls typed tools, loads Skills, reads a memory store and hands anything visual to a GUI sub-agent. Its #1 on MobilePA-Bench, a benchmark its own authors built, is a 0.21-point lead, smaller than the 0.41-point spread the benchmark's authors measured across three identical runs. (2026-09-26) [Agents & harnesses · Solid · 57/100 · Original, source-checked analysis; Explained from first principles]
- [Qwen3.8-27B variants: the bill is tokens times bytes](https://ai.thesatyajit.com/articles/qwen3-8-27b-variants.md): Three releases in one week make the same dense Qwen3.8-27B cheaper to run. ThinkingCap fine-tunes it to think less: its 37.2% is a mean of per-benchmark cuts (22.8% pooled), measured at equal settings rather than equal accuracy, and still a far better trade than the model's own effort dial. OrcaSAQ-2 is an EXL3 trellis quant whose 3.21 bits per weight checks out exactly, over the decoder's linear layers only; its 4.61× compares a multimodal repository with a text-only checkpoint (4.45× like for like). ThinkingCap's 'MLX 4-bit' build is 6.04 bits per weight. Hemmingway-1 pulls neither lever. Read from the headers, with a calculator that puts tokens and bits on one axis. (2026-09-26) [Quantization & compression · Essential · 77/100 · Original, source-checked analysis; A guide you can follow today]
- [Rigel: a hybrid Mamba-2 MoE trained on whichever chips were free](https://ai.thesatyajit.com/articles/rigel-hybrid-mamba.md): Rigel is a 2.3B-parameter, 360M-active hybrid of Mamba-2, attention and a 128-expert MoE, pretrained by a Berkeley-led team on H100s, A100s, 32GB V100s and TPU v5p/v6e from one PyTorch codebase. I summed its safetensors header (2,345,567,552 parameters, 363.8M active), reproduced the blog's 5.36 x 10^21 FLOPs figure to the digit, and reread the harness logs behind its chart. 'Under 1% of Llama-3.2-3B's pretraining FLOPs' holds only when the 3B is charged for Llama-3.1-8B's 15T-token run; against the 3B's own 9T-token run Rigel used about 4%. 'Within a few points' is 1.9 points on the nine charted tasks and 3.5 on all twelve the logs contain. (2026-09-26) [LLM architecture · High · 72/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [SigLIP 2 from first principles, and what its Core ML port measured](https://ai.thesatyajit.com/articles/siglip-2-coreml.md): SigLIP 2 keeps SigLIP's pairwise sigmoid loss, which asks one yes-or-no question per image-caption pair so no pair waits on the rest of the batch, and adds a LocCa captioning and referring decoder, late self-distillation and masked prediction, data curation for the small models, a 90/10 multilingual mix with de-biasing, and a native-aspect NaFlex variant. I walk through each against the paper's own tables, then check FluidInference's Core ML port of the base model: the baseline is fp32 PyTorch on the Apple GPU at batch 32; 715 MB against 1.5 GB is fp16 against fp32 in mixed units, exactly 2.00x in bytes; 'same accuracy' is accuracy against the Oxford-IIIT Pet labels, 94.26% against 94.11% on all 7,349 photos; the text tower ships and is three quarters of the download; and the 262 MB peak is a process footprint smaller than the weights the process loads. (2026-09-26) [Vision & multimodal · Notable · 66/100 · Original analysis; Runs on a laptop CPU]
- [SurfSLAM: sim-to-real stereo and a DVL-driven factor graph for mapping shipwrecks](https://ai.thesatyajit.com/articles/surfslam.md): SurfSLAM (University of Michigan, arXiv 2601.10814) maps Great Lakes shipwrecks from a BlueROV2. A stereo network is fine-tuned on augmented simulation and unlabelled real footage, an IMU, DVL and barometer factor graph in GTSAM tracks the vehicle, and stereo registrations pull the drift back. Why water breaks cameras, how each piece works, what the tables actually show (the stereo gain is the water column; the DVL carries the tracking), and what the code and data release does and does not license. (2026-09-26) [Robotics & embodied · Notable · 63/100 · Original analysis; A lasting reference]
- [TeleOCR: the warped page is never unwarped](https://ai.thesatyajit.com/articles/teleocr.md): TeleOCR is China Telecom's two-pass document parser: a Qwen2.5-VL vision encoder, an MLP aligner trained from scratch and a Qwen3-0.6B decoder, shipped under a Qwen2.5-VL config. It reads phone photos without a dewarping model by outlining curved regions as polygons, blacking out everything else and reading the still-curved crop, a skill it learns from synthetic warps with exact labels. The safetensors header holds 1,415,072,768 parameters, 1,259,490,304 of them distinct, which is where '1.2B' comes from. The OmniDocBench owners' own run confirms first place at 96.91, but the lead is all table structure: with the runner-up's table score it would place third. The ICDAR first place is one task of four, won with an ensemble. The Apache-2.0 tag is one metadata field on a release with no licence file and a vision encoder whose parent the paper never names. (2026-09-26) [Vision & multimodal · High · 72/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [Tokenizers v1 and Combinatorial BPE: a faster BPE, and the six ways to spell 'the'](https://ai.thesatyajit.com/articles/tokenizers-v1-combinatorial-bpe.md): Hugging Face's tokenizers v1 release candidate keeps every token ID and rebuilds the engine behind it: a bitstream pre-tokenizer instead of a regex, a word cache, an allocation-free merge loop, a lock-free batch planner and a crate split. Combinatorial BPE goes after the vocabulary instead, folding ' the', 'The' and ' THE' into one core row plus modifiers. BPE from first principles, both designs read from their code, and my own measurements: identical IDs over 20 million tokens, 7-54x single-core speedups, a 3x smaller wheel, and 19-32% of three real vocabularies spent on space and case variants. (2026-09-26) [Developer tools & infra · Essential · 91/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [WorldCrafter: skipping the reconstruction, not the 3D](https://ai.thesatyajit.com/articles/worldcrafter.md): TencentARC's WorldCrafter keeps a video world model consistent when you walk back to a place, with no depth maps, point clouds or warping at inference. I read the paper and the released code. Before each chunk it keeps the latest latent frame, adds eight past ones chosen by how much of the upcoming views they jointly cover, runs the nine through a 1.14B-parameter encoder inherited from the LagerNVS view-synthesis model, and reads out four latent frames posed where the camera is about to go: 3,840 tokens in a 14,640-token window. On the authors' own 725-video benchmark that takes revisit LPIPS from 0.487 (Lyra 2.0, the best baseline, which does reconstruct) to 0.255, at 0.062 s of memory work per chunk against 1.346 s for depth-and-warp. Weights and inference code are out under an academic-only licence; training code and the benchmark are not. (2026-09-26) [Image & video generation · Notable · 62/100 · Original analysis; A lasting reference]
- [xHC: sixteen residual streams, four written at a time](https://ai.thesatyajit.com/articles/xhc-residual-connections.md): Hyper-connections widen a Transformer's residual stream into parallel streams; DeepSeek's mHC keeps their mixing doubly stochastic so it cannot blow up with depth, and stops at four. xHC reads all sixteen streams, mixes and writes only four per sublayer, and feeds those writes a causal convolution over neighbouring tokens. The Sinkhorn guarantee survives intact, the paper reports +4.0 points over mHC at 18B and +3.1 at 28B, and its FLOPs overhead is small. Checked against its own formulas: the 28B cost column reproduces and the 18B one cannot; the wall-clock bill is several times the FLOPs bill; at 10B the convolution does two-thirds of the loss work; token budgets are unreported; and there is no code yet. (2026-09-26) [LLM architecture · Solid · 58/100 · Original analysis; A lasting reference]
- [Qwen-Image-2.1 Pocket rewriter: 12x smaller than the 9B, and it made all four test images less faithful to the prompt](https://ai.thesatyajit.com/articles/qwen-image-2-1-pocket-rewriter.md): ML-Intern-lab distilled Qwen-Image-2.1's recommended 9B prompt rewriter into a 0.8B and a 2B that answer with no system prompt and no thinking. I checked the pitch against the files. The 9B is 18.82 GB, not 20; its system prompt is 1,721 words; and of the 1,666 tokens it generates per rewrite, 805 are thinking, not 1,600. The students are 752,393,024 and 1,881,825,088 parameters, trained for two epochs on 1,776 teacher rewrites — and because the filter kept only rewrites of 400 words or fewer, 70% of those are stickers and logos, with 11 posters and 5 UI screens. The image evaluation does include a no-rewriter arm, which is what a claim about better pictures needs. Its pairwise judge picked whichever image sat in slot A in 240 of 240 comparisons. Its OCR arm works: the 9B beats no rewriter on rendered text 11 spans to 2, and the 0.8B's record against no rewriter is 8 to 7. On four CPU cores the 0.8B rewrites in 14 s against the 9B's 387 s, and each rewrite makes every denoising step about 1.4x as expensive. Rendered from its rewrites, all four test images came out less faithful to the request than the raw prompt's: of nine checks the requests themselves define, the raw prompts pass seven and the 0.8B's rewrites pass one. (2026-09-23) [Image & video generation · High · 70/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [MiMo-V2.6: a trillion parameters, thirty RL steps, and a 20× that is a decoder](https://ai.thesatyajit.com/articles/mimo-v2-6.md): Xiaomi open-weighted MiMo-V2.6-Pro and Flash. I counted the safetensors: Pro is 1,024.2B stored parameters with 41.9B active, released natively in MXFP4 at 566 GB rather than the ~2 TB BF16 would cost; Flash is 310.8B / 15.4B. The headline — 46.32 on the Artificial Analysis Intelligence Index, highest open-weight model — holds on AA's board today. 'On par with Claude Opus 5 and GPT-5.6 Sol across most agent benchmarks' does not: on the card's own table Pro is ahead on 3 rows of 14, level on 2, and 14.1 points behind Opus 5 on Terminal-Bench 4.0. The genuinely unusual part is the RL release — roughly 7,000 graded task environments with their verifiers, plus a 9B distilled starting point and a reproducible GRPO baseline with denominators printed. And 'Pro UltraSpeed, up to 20× faster' is not a model: it is the 5-layer DFlash block-diffusion drafter in the repo, whose block_size is 8, which caps it at 8× fewer target forward passes. (2026-09-22) [LLM architecture · Essential · 77/100 · Original, source-checked analysis; Widely used]
- [Reproducing Jev, from a source I could not find](https://ai.thesatyajit.com/articles/reproducing-jev.md): Someone handed me five findings from an independent team's multi-day attempt to reproduce, break and improve Jev — a 27-point regression, a 0.769-to-0.541 in-distribution/out-of-distribution gap, an untrained decoder beating every trained encoder. No link, no author, no benchmark name. I spent the first half of this piece trying to attribute them and failed, so all five are labelled Reported-unattributed and none of their numbers appears in this article as a fact. Then I went and checked the claims against things that are public, and three of the five turn out to be independently corroborated — including the one that matters most. JevBench's hard tier ran every open reproduction on the same 220 items: Laya, which beat Jev by 3.9 points on its own benchmark, is 88 items behind it here, while a frozen Qwen3.5-4B with nothing trained at all is ranked second. (2026-09-22) [Evals & benchmarks · Notable · 61/100 · Analysis found nowhere else; A new technique]
- [A field guide to attention mechanisms](https://ai.thesatyajit.com/articles/attention-mechanisms.md): One operation, two bills — O(N²) compute and the O(N·layers) KV cache — and a map of how every variant pays them down: MHA, MQA, GQA and MLA on the memory axis; sliding-window, sinks, dilated, block-sparse and content-based NSA on the mask axis; linear/kernelized attention that drops the matrix; and the exact-but-IO-aware systems layer of FlashAttention and PagedAttention. (2026-07-08) [LLM architecture · Notable · 60/100 · Interactive explanations; A guide you can follow today]
- [Voxel Musou: the army is a 300-slot pool, and the combo is a frame table](https://ai.thesatyajit.com/articles/voxel-musou.md): A browser-playable Dynasty Warriors homage — Zhao Yun against a field of voxel Wei soldiers — in plain ES modules on a vendored three.js r186, created on 24 September 2026. I read all 9,254 lines against the README without running any of it. '~300 soldiers' is a recycled 300-slot pool: a director keeps about 84 of them on the hero whatever ?enemies says, the crowd is 22 instanced meshes at any size, and what scales is CPU work plus a per-frame instance-buffer re-upload of up to 0.43 MB at 300 and 2.80 MB at 2,000. 'Deterministic fixed 60 Hz' holds inside one browser — an exact accumulator, two seeded streams, one consumer of sim randomness — but nothing in the repository records a replay, and 156 transcendental Math calls in the sim keep it from crossing engines. The best part is that the feel is data: 15 moves and 25 hit windows whose frame arithmetic checks by hand, hitstop absorbed into the beat by one constant, and a Musou whose damage is sampled from the dragon it draws. The tuning targets live in a benchmark the code refers to 46 times, in a bench/ directory the repository does not contain. (2026-09-24) [GPUs, kernels & systems · High · 68/100 · Runs in a browser; Interactive explanations]
- [Ming-Image-0.1-Design: a Z-Image fine-tune that is #1 of 42 open models and 16th of 129](https://ai.thesatyajit.com/articles/ming-image-0-1-design.md): inclusionAI open-sourced two '6B' design models and two agent skills, and announced the first as #1 among open-weight models on Artificial Analysis's UI/UX Design leaderboard. I read the files instead. The 6B is the DiT alone: 6,154,901,056 of 24,857,847,668 parameters, 24.8% of the download, behind a 17.0B Ling-mini-2.0-based multimodal encoder, a Qwen2.5-1.5B connector and a VAE that is byte-for-byte Qwen-Image-Layered's. The DiT is Alibaba's Z-Image: the same config, the same parameter count to the unit, 631 of diffusers' 653 lines of its code under an 'Ant Group' header, and a weight fingerprint that points at the base checkpoint rather than the Turbo. The #1 is real and narrow: first of 42 open-weight models and 16th of all 129 models in that category, 45th of 161 across all prompts, 33 Elo ahead of a non-commercial runner-up with overlapping 95% intervals, and first among open models in only 3 of the arena's 20 views. The Layer model decomposes a flat design into RGBA layers by generating them as video frames; its only published number is a table whose winning row is called CLEAR, four of whose seven rows are copied from Qwen's paper. The Design model's RGBA output is a matte whose transparent background sits at alpha 1, not 0. Both skills are genuine SKILL.md packages, and both call a hosted API the open weights do not reproduce. (2026-09-23) [Image & video generation · High · 70/100 · Original, source-checked analysis; Concrete numbers to act on]
- [Adapt-1 Machina: no critic, because the simulator is the critic](https://ai.thesatyajit.com/articles/adapt-1-machina.md): Rei Labs claims continuous control learned from coarse outcome feedback with no demonstrations and no critic. Both halves are true and the second one costs 18,400 executed attempts for 62 completed placements, the first on attempt 1,394. What replaces the critic is not a baseline or an evolutionary population — it is the simulator: propose a 20-by-7 command matrix, run it, grade it with a hand-written evaluator, then edit indexed coordinates of a recorded parent. The one learned continuous part is a bounded correction from the start observation, and the post's own ablation is the best evidence in it: feed that correction another case's observation and it does worse than no correction at all, 62 of 768 against 85. The headline YAM number is +6.25 points with a published 95% interval whose lower bound is +0.78, which reproduces from 70 gains against 46 losses. All simulated; no hardware anywhere. (2026-09-22) [Robotics & embodied · Solid · 51/100 · Original analysis; Concrete numbers to act on]
- [Beacon: Jev grades your agent runs, and the lesson is not in the skill](https://ai.thesatyajit.com/articles/agent-beacon.md): Beacon captures agent sessions across 29 harnesses and asks a decision model which runs are worth learning from. Putting the model after the loop instead of inside it is the right idea and this site's own framework says so — offline kills the latency argument, and a finished trace has no next step to get wrong. Then I read the code. The rubric is three questions and one constant; the gate is an unweighted mean, so a run the model scores 0.00 on 'is there anything reusable here' still becomes a candidate. The projection drops command.exit_code, actor and approval.decision before asking whether the task succeeded and whether a human corrected it — both of which the event schema answers exactly. And the artifact that gets promoted contains no lesson: a template, the first 80 characters of the user's opening prompt, and the three probabilities reprinted. The good news is real. The rubric is a compile-time constant nothing in the loop can rewrite, so unlike jev-linkmap this loop is not grading its own exam, and the reviewer is a person. The falsifier is that nothing measures any of it — while a sibling directory ships 75 detection rules over the same events with 509 labelled cases that run in CI. (2026-09-22) [Agents & harnesses · Notable · 65/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [AgentJev-0.6B: exact order invariance, and options that still interact](https://ai.thesatyajit.com/articles/agent-jev-06b.md): Four articles on this site have described a fork: isolate a decision model's options and you get exact order-invariance but no option can refer to another; put them in one sequence and a relational question becomes expressible but position acquires a prior worth +1.71 logits. AgentJev-0.6B is arranged a third way — one cached prefix, one private branch per candidate, and a scoring head with no positional embeddings — and it pays neither price. I downloaded the weights and ran it on four CPU cores: zero answer changes across 2,520 permutations, backbone vectors bit-identical between the two orders, worst movement anywhere 9.5e-07 logits. The same head moves a candidate's logit by a median of 0.088 when you change which other options are on the list, so its candidates interact without having positions — which is the combination four articles here have described as unavailable. Also: 598,418,689 parameters off the safetensors header, the 92.4% that buys 2.04x, the reliability diagram neither release drew, and a headline of 79.25% on a benchmark whose own card puts saturation at 73.5%. (2026-09-22) [LLM architecture · High · 75/100 · Original, source-checked analysis; Interactive explanations]
- [Altar-1: a security model measured on recall alone](https://ai.thesatyajit.com/articles/altar-1.md): Aikido's first open-weight security model is GLM-5.3 with 88 of its 256 routed experts deleted and the rest at INT4 — 500.8B parameters in 327.9 GB, computed from the safetensors headers, at an effective 5.24 bits per parameter rather than the four that 'W4A16' implies. The evaluation is better than most: 32 CVEs across 30 repositories, three runs each, with the per-case grid published. Transcribing that grid reproduces all three headline averages exactly, and turns the comparison into a paired one — the 'approximately one percentage point' between Altar and its quantised parent is, precisely, one run on one case out of 96. What no version of that benchmark can contain is a false positive: every case has a planted vulnerability, so there is no cell for a finding that was not there, and precision — the number that decides whether a scanner is deployable — is not measured low, it is not measured. (2026-09-22) [Security & safety · High · 69/100 · Original, source-checked analysis; Concrete numbers to act on]
- [Carveout: SAM 3 labels for a Gaussian-splat scene, without the multiplex](https://ai.thesatyajit.com/articles/carveout-3dgs-object-labels.md): Carveout renders synthetic views of a trained 3D Gaussian Splatting scene, segments them with SAM 3 Promptable Concept Segmentation, and lifts the 2D masks onto individual Gaussians — no retraining, no original capture images. Three days after this site measured SAM 3.1's Object Multiplex cost curve, here is a downstream consumer of exactly that model, and it does not use the multiplex path for detection at all. That turns out to be correct: multiplexing accelerates the video tracker's memory path, and concept segmentation never enters it. Where Carveout does load sam3.1_multiplex.pt — the exemplar pass — it resets the session between crops and runs eight propagations where one bucket would do. Also: how the lift resolves a Gaussian that four views disagree about, and what the six-scene calibration bank could not fix. (2026-09-22) [3D & spatial · Notable · 63/100 · Original analysis; Runs on a consumer GPU]
- [Cinference runs Qwen3.8-27B at 450 tokens a second: what the number counts](https://ai.thesatyajit.com/articles/cinference.md): A one-command installer claims 450 TPS on a custom engine and above 300 at 256K context. Both numbers are real, both are measured, and the denominator is the best possible one: batch size one. The runtime manifest says max_concurrency 1 and the README says 'one request runs at a time', so this is per-user speed, not aggregate. What the headline does not say is that it is a speculative-decoding number on a workload that accepts 93 to 98 percent of the drafts, that the same engine with speculation off does 71 tok/s at 8K and 52.9 at 260K, and that the reason the long-context figure barely falls is that draft acceptance rises with context while the step time grows 35 percent. The repo's own results file records a repeat of the 260,000-token point at 275.73 tok/s, under the claim, and lists 'ordinary prose can be much slower' as a limitation. Also: the fork is fifteen files, the KV codec is Hadamard-rotate-then-quantize, and 48 of the model's 64 layers have no KV cache at all. (2026-09-22) [Inference & serving · Notable · 62/100 · Runs on a consumer GPU; Original analysis]
- [figures4papers: a house style, audited against its own figures](https://ai.thesatyajit.com/articles/figures4papers.md): A Yale PhD student open-sourced every plotting script from eight papers, with the data and the generated figures committed beside them, and packaged the conventions as a SKILL.md for Cursor, Claude Code and Codex. Because the outputs are committed, the standards are checkable rather than a matter of taste — so I checked them. Removing chart junk is legibility. The semantic palette is half legibility and half rhetoric: blue-for-mine survives a deuteranope simulation at ΔE00 64, and the green-for-better / red-for-worse axis collapses from ΔE00 45.5 to 5.2, which is the separation two adjacent steps of a single-hue ramp have for everyone else. Accessibility is not mentioned anywhere in the repo. The one rule that would have mattered most is absent: the headline bar figure has its y-axis floor at 0.5, which draws a 1.64x gap as 10.1x. And the SKILL.md's API reference specifies nine functions that no file in the repository implements. (2026-09-22) [Developer tools & infra · High · 71/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [TencentARC's GAE: the geometry decoder never sees the picture](https://ai.thesatyajit.com/articles/gae-geometry-native.md): TencentARC's GAE compresses Depth Anything 3's four-level feature hierarchy into a 64-channel latent, runs one conditional flow model in that space, and decodes RGB and 3D from the same final state. The load-bearing claim is that the showcase point clouds are native, not lifted from the generated image — and it is falsifiable from the forward path, so I traced it. decode_geometry(z) takes the latent and nothing else; the points are ray origin plus depth times ray direction, and RGB enters only as the colour written into each point. What that buys, measured under a matched flow model: κ from 6.6e16 to 53.1 on 64 channels, FVD 12.7% and 23.1% below the best competing latent, VGGT trajectory error halved. What it costs, from the paper's own tables: sampling degrades the native geometry 30–65%, the RGB head got 203M parameters and noise-hardened training while the geometry branch got neither, the released weights are a third checkpoint that reproduces none of the generation tables, and the licence is academic-use only. (2026-09-22) [3D & spatial · Notable · 60/100 · Original analysis; A new technique]
- [Google's ax agent runtime: the single writer is a Go map](https://ai.thesatyajit.com/articles/google-ax.md): Google open-sourced ax, a distributed agent runtime, on the strength of two design claims: a single-writer architecture for consistent session state, and a durable event log with automatic recovery. I read both. The unit of ownership is the conversation id, the exclusion is a map[string]struct{} guarded by a mutex inside one process, and the reference manifest deployed three of those processes against one shared log. The event log is an audit trail beside mutable state, not a source of truth — replay reconstructs two fields. I measured what that costs: one interrupted turn plus one follow-up message ran the same work three times and took fourteen irreversible actions where six were asked for. Then, on September 19, the project deleted all of it — and the deletion is what finally makes 'not a framework' true. (2026-09-22) [Agents & harnesses · High · 71/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [Halo's 2.8× over TRL is not a kernel, and its own benchmark proves it](https://ai.thesatyajit.com/articles/halo.md): A post-training framework claiming up to 2.8× stock TRL's throughput at less peak memory, on native HuggingFace checkpoints. The usual suspicion — beating an untuned baseline — mostly does not survive contact: the baseline is current TRL on transformers v5 with FSDP2, FlashAttention-4, Liger with fused-linear cross-entropy and the same grouped-GEMM expert kernel, the runner script is committed, and TRL and Halo land within 1% of the same loss over 200 seeded steps. The real asymmetry is sharding — 2.8× is Halo's ZeRO-2 against TRL's ZeRO-3, and matched at ZeRO-3 it is 1.25× at 4k and 2.68× at 16k, which is the number Halo's own README quotes. Where the gap actually comes from is token routing: transformers' expert-parallel path all-reduces the full MoE output on every rank, and I verified that in transformers' own source. (2026-09-22) [Training & RL · Notable · 59/100 · Original analysis; Concrete numbers to act on]
- [Husky's 4.5× over MLX: the method is published, and it is speculative decoding](https://ai.thesatyajit.com/articles/husky-msi.md): Two days after a 50× that turned out to be 1.39×, a second Apple-silicon engine claims 4.5× over MLX and 730 tokens a second — and this one ships the table. Sixteen tasks, three runs each, medians, versions, the same weight file on both sides. So I checked it instead of doubting it. Woof is Qwen3.5-4B at 4 bits and its file is 2.367 GB, which reproduces from the config to 0.06%; at that size the memory bus caps any one-token-per-read engine at 1.40× over MLX, and Husky's own median without its draft is 1.125×. Every larger number on the page is speculative decoding — prompt lookup on the edits, a trained draft elsewhere. The 4.48× is one row of sixteen, on the task where the reply copies the prompt. And mlx-lm 0.31.3 has shipped `--draft-model` the whole time. (2026-09-22) [Inference & serving · High · 72/100 · Original, source-checked analysis; A guide you can follow today]
- [Jev swarms: one call, ten agents, and the answer moves](https://ai.thesatyajit.com/articles/jev-engineering-swarms.md): The Jev Engineering pitch has grown a swarm variant, and the swarm variant contains the first genuinely new argument in the family: batch every agent's question against one shared state and the expensive half of the request is amortised, not repeated. The measured version holds — a ten-question tick in jev-swarm costs about the time of one. What nobody says is that the batch is not answer-preserving. duckdb-jev measured the same rows grouped and alone, and jev_score moved about 0.29 rubric levels on a 0-3 scale while jev_choice agreed 269 times out of 270, because argmax survives a shift that a probability-weighted mean does not. Also: the swarm's coordination is not in the model. Intentions stay private by construction, so the negotiation is a deterministic resolver, and the published run overrode the model on 2.2% of decisions. (2026-09-22) [Agents & harnesses · Solid · 54/100 · Original analysis; A new technique]
- [Jev is not deterministic, and option order is not noise](https://ai.thesatyajit.com/articles/jev-is-not-deterministic.md): A public complaint about Jev packs three assertions into four sentences: it is not deterministic, the option order drastically changes the probabilities, and therefore the no-hallucination claim is suspect. They need three different experiments. Two of them have already been run and the raw logs are public — I recomputed both. Identical requests: 67.2% of 1,300 items come back with a different probability vector and 3.3% change their answer, with the pinned version echoed on every response and no sampling parameter in the API to blame. Permuted options: 12 of 100 items change answer and the vector moves up to 0.69. The one published figure for this, 0.13, is the union of the two and nobody has ever separated them. The third claim is a misreading, and being precise about which reading it breaks is the interesting part. (2026-09-22) [LLM architecture · High · 70/100 · Original, source-checked analysis; A new technique]
- [Jev-Omni: a 256-way classifier with no encoders](https://ai.thesatyajit.com/articles/jev-omni.md): The first System One model that takes text, images, audio and video. Read against its own artifacts rather than its announcement: the multimodal part is 52,379,904 parameters of untouched Gemma 4 — a single Linear over a 48x48 pixel square, a single Linear over 640 audio samples — against an 11,907,350,320-parameter fine-tuned decoder whose count I summed from the safetensors headers. The decision head is Linear(3840, 256) with the unused classes masked to -1e30, so the option count is welded into the trained weights rather than into an export; that is the third time this site has found a frozen N, and the first time it was not a conversion artefact. The sub-100ms claim is an H200 and covers three of the four modalities: video is 504 ms. And 'on par with Jev' is 87.57% against Jev's 90.48% on the author's own benchmark, where three ordinary chat models score 99. (2026-09-22) [LLM architecture · High · 69/100 · Original, source-checked analysis; Concrete numbers to act on]
- [Kev, and the price of not being able to see the other options](https://ai.thesatyajit.com/articles/kev.md): Jared Palmer refactored Kev onto Qwen3.5 and shipped 0.8B, 4B and 9B checkpoints with a fine-tuning script. It is a trained checkpoint, not a serving wrapper — a rank-16 LoRA plus a bilinear pointer head, 11,347,456 to 45,376,000 trained parameters that I counted from the safetensors headers rather than the cards. It is also a third readout family: options are spans in one shared sequence, so they can see each other, so the order matters. The repository has measured how much, 259 times, and pooling those blocks gives the first scaling curve I have seen for option-order sensitivity: 15.3% of permuted questions flip on the 0.5B prototype, 1.8% on Qwen3.5-9B. Two things fall out of that ledger. The option-isolation arm — invariance by construction — costs 5.8 points of accuracy at 4B and is not even available on Qwen3.5. And Jev, which this site has read three times as exactly order-invariant, flips nothing in 324 permuted questions and still moves its probabilities by 0.0202, a hundred thousand times more than the arm that actually cannot move. (2026-09-22) [LLM architecture · High · 69/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [mini-AGI: what the 169 experts and the 0.0067 nats actually are](https://ai.thesatyajit.com/articles/mini-agi.md): A byte-level model that trains from scratch on one 8 GB laptop GPU and keeps learning while it reads, three days old, no weights published, 4.5% of the way through its first corpus pass. The claims are checkable and I checked them. The 169 experts: a two-level router, where a character sees 32 of them and never 169, growing under a config constant that permits one new expert per two million characters — and the pool shrinks nine times in the committed log. The forgetting figure: 2.23 nats to 0.0067 nats is a real and enormous effect between two arms of a controlled probe, its denominator is recoverable to 0.008 nats from the published percentages, and the winning number itself is a fifth of the noise floor the repository states two sections later. The eval that produced it is not in the repository. (2026-09-22) [LLM architecture · Notable · 66/100 · Original, source-checked analysis; A new technique]
- [Modular's LLM Inference Handbook: right formula, wrong variable](https://ai.thesatyajit.com/articles/modular-llm-inference-handbook.md): Modular's LLM Inference Handbook is 70,000 words across 57 pages, and it is better than its genre deserves — the metrics chapter makes a distinction most benchmark posts skip, the benchmark template demands every denominator, and the one place it cites its own unpublished measurements is to report a 20-30% regression on a technique it sells. It is also not originally Modular's: the repository ran on BentoML branches until July 2026, which is the real reason MAX is only the third most-mentioned framework in it. The technical problem is narrower and sharper than vendor bias. Five of its formulas are correct formulas attached to the wrong variable, and all five are in the artifacts a reader copies. Its KV-cache calculator asks for attention heads where the shape needs KV heads, which overstates by 3x to 57x across the handbook's own six exemplar models and by 32x on a hybrid one; its acceptance-length formula takes a different alpha from the one every serving stack logs, which on a real MI300X sweep makes it 55% low and points the wrong way; its memory formula makes the KV allowance proportional to weight precision, so quantizing to 4-bit quarters the cache budget while the cache does not move. I measured the weight term against five real checkpoints' safetensors headers: exact at bf16, 43% low at 4 bits. And the one hardware calculation it does do — the bandwidth-bound decode ceiling — reproduces this site's own measurement, on different silicon, to within 16%. (2026-09-22) [Inference & serving · Essential · 81/100 · Original, source-checked analysis; Interactive explanations]
- [Cactus Needle 3: the free fine-tune deletes the confidence head](https://ai.thesatyajit.com/articles/needle-3-finetune.md): Cactus ships two ways to customise Needle 3. The local one is real, runs on your laptop and writes a deployable binary. It also drops the confidence head on the floor — `needle build --lora` deletes the tensor group outright, and `confidence` comes back `None`. I pulled the head out of the shipped safetensors: six tensors, 71,077 parameters, 37,276 bytes, a two-stage attention probe over 21 depths that collapses the whole forward pass to one logit, read under `stop_gradient`. That last detail is why local tuning cannot keep it. Also: the local path exports at 4 bits because `WEIGHT_BITS = 4` and the packer raises on anything else, so a locally tuned 20-layer archive is 63.44 MB against the shipped 35.34 MB — arithmetic that reproduces the shipped file to the byte. (2026-09-22) [Training & RL · Notable · 64/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [SGLang's NVFP4 KV cache: the 56% is exact, the 1.78x is untested](https://ai.thesatyajit.com/articles/nvfp4-kv-cache.md): NVFP4 KV lands in SGLang behind one flag and the arithmetic checks out perfectly — 56.25% of FP8 already includes the block scale, and the ratio is tight enough to recover the block size of 16 without being told it. What does not survive is the step from that ratio to deployed capacity. The post's three iso-capacity points show 1.59x, 1.50x and 2.00x, and in every one of them the NVFP4 run hit the concurrency the client offered rather than the limit the cache imposed, so none of them measures 1.78x at all. On accuracy: every gap in the post sits inside its own benchmark's sampling noise, including the ones called identical — 60 AIME samples cannot tell a two-point regression from nothing. Also: why the 27B only has 16 layers' worth of cache to shrink, and why cinference's competing 4-bit KV rotates first and this one does not. (2026-09-22) [Inference & serving · High · 76/100 · Original, source-checked analysis; Widely used]
- [OpenMuse: "compatible with any agent harness" is one ternary branch](https://ai.thesatyajit.com/articles/openmuse.md): CopilotKit's OpenMuse is an MIT, self-hostable personal agent whose README leads with harness portability. I read the repository at commit c79178f. The portability is real and it is five lines: AGENT_BACKEND=agui swaps the built-in agent for AG-UI's HttpAgent, which is one POST of RunAgentInput and an SSE stream back. There is exactly one harness adapter in the tree — packages/backends/src/openbot.ts, 343 lines, no package.json, disabled by default, imported only by its own 13 tests. The swap is also total: all 29 tool names are constructed inside the agent it replaces, and the client draws 8 of them by exact string match, so a swapped-in harness gets a chat pane and no cards. What stays behind is the product — a terminal that is bash -c inside a --network none --read-only --cap-drop ALL container re-verified against 38 predicates before the server will attach, and a prepared email that is hash-bound, account-bound, 30-minute-expiring, and has no approve tool at all. (2026-09-22) [Agents & harnesses · Notable · 68/100 · Original, source-checked analysis; Concrete numbers to act on]
- [Penjing-27B: reading a quantisation claim out of the GGUF header](https://ai.thesatyajit.com/articles/penjing-27b.md): PollardWeights shipped a six-rung ladder of Qwen3.8-27B quants under the banner 'bits placed by per-layer sensitivity, not a uniform crush.' Parsing the four files that actually exist gives the allocation directly: the per-layer structure is blocks 0, 1, 63 and 64 protected and blocks 2–62 flattened to IQ1_S, identical across three different bit budgets — a fixed rule, not a per-budget knapsack. The measured bits-per-weight are 1.906, 2.120, 2.150 and 3.785, none of them the nominal type. Demoting attn_k and attn_v saves 11 MB out of 7.3 GB; demoting ffn_gate and ffn_up saves 765 MB. Credit where it is due: unlike almost every GGUF repo, this one publishes perplexity, KL and top-1 agreement — for three rungs of six. (2026-09-22) [Quantization & compression · High · 69/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [vLLM serving Qwen3.8-2.4T: a Pareto frontier is not a deployment](https://ai.thesatyajit.com/articles/qwen38-pd-serving.md): vLLM published 5,000 total tokens/s/GPU and 180 output tokens/s/user for a 2.4T model on GB300 NVL72, and the two numbers belong to different machines — 48 GPUs against 16, different topologies at both ends, and opposite on speculative decoding. Read that way the post is not a benchmark but a tuning method, and the method is the part worth having: work out a request's KV cost, discover that a hybrid model's linear layers set the page size for its attention layers, derive maximum concurrency from measured startup numbers, then benchmark prefill and decode as separate workloads before combining them. I re-derived the 759 MiB per request and it lands exactly; 37.5 percent of it is recurrent state that does not care how long your prompt is. Also: 'total tokens' counts the 8,192-token prompt, so the left-hand figure is 556 generated tokens per second per GPU, not 5,000 — and the chart where MTP turns over and costs 62 percent of throughput. (2026-09-22) [Inference & serving · Essential · 80/100 · Original, source-checked analysis; A guide you can follow today]
- [Thinking in Blender: which half of the 3D scaffolding gets eaten](https://ai.thesatyajit.com/articles/scaffolding-gets-eaten.md): The authors of a staged 3D coding harness went back to it with a stronger base model, found the gains almost gone, and warned that all of today's scaffolding will eventually be eaten. The paper is careful and the warning is reported against their own interest, so I took it seriously. What the paper actually measured is one number — the staging gain at one backbone, Claude Opus 4.7: +1.25 dB PSNR and +15.5% DINO over the unstaged baseline, with one of its five wins at 0.4% and one metric going backwards. The revisit is a post: no number, no defined bar, and the original ran no model sweep either, so both endpoints of the 'sharp threshold' are unpublished. Sorting the harness's eight mechanisms by what each one stands in for says the warning is right about more of it than I wanted, and that the parts it is wrong about are not safe — a constraint does not get eaten, it gets repriced, and this paper's own limitations section is the receipt. (2026-09-22) [Agents & harnesses · Notable · 63/100 · Original analysis; Widely used]
- [NVIDIA's Skill2Env: collective is a distribution claim, and the benchmark disagrees](https://ai.thesatyajit.com/articles/skill2env.md): NVIDIA's Skill2Env compiles 3.4k public Agent Skills into 7,971 containerised RL environments, each with programmatic tests and a rubric distilled from the Skill's own methodology. 'Collective' turns out to mean none of the things the word usually means in multi-agent work — no shared replay, no shared skill library at train time, no joint policy update. It means the task distribution comes from what thousands of practitioners bothered to write down, and software engineering is under a quarter of it. The result I keep re-reading is the one that lost: adding the rubric channel — the part actually carrying human methodology — costs 4.0 points on Terminal-Bench, 3.0 on S2EBench pass and 11.5 on S2EBench score against outcome-only RL, and wins the Skill-alignment preference test 73.0% to 54.5%. Collective preference and benchmark pass rate are measurably not the same objective. (2026-09-22) [Agents & harnesses · Solid · 53/100 · A new technique; Concrete numbers to act on]
- [Five Jev harnesses, for a model that cannot write](https://ai.thesatyajit.com/articles/system-one-harnesses.md): Five repositories landed on my desk this week, four of them younger than a week, all of them wrapping a decision model, none of their authors having read the others' code. Every one had to solve the same four problems: enumerate the options, shape the question, set a confidence threshold, and get a string out of a model that cannot produce one. They agree almost completely on the first — four of five enumerators contain no model at all — and they disagree completely on the third: below the threshold, one refuses, one drops the result, one hands the step to an LLM, one takes three more samples. System One Harness's 'one model call per step' survives the code, and I found what it cost: a compiler that refuses, at load time, any parameter that would need a word. Also, its own canvas probe, which measures the enumerator failing at every resolution it tried. (2026-09-22) [Agents & harnesses · Notable · 67/100 · Original analysis; A lasting reference]
- [Tinfield 1: 1.85% of a model, and a 51-billion-parameter hash table](https://ai.thesatyajit.com/articles/tinfield-1.md): Bad Theory Labs shipped a 360 GB checkpoint claiming 33.0 on Terminal-Bench 4.0 and 62.0 on DeepSWE v1.1. I compared it byte-for-byte against its declared base, Qwen3.8-Flash-Next: 1.85% of the parameters changed. Every attention projection moved; all 512 experts per layer, both embedding matrices, the multi-token-prediction module, the vision tower and the 51,200,245,760-parameter engram table are identical to the base. That engram table is the release's most quoted phrase and nobody has said what it is: a 320-million-row hashed n-gram embedding at exactly one layer, sharded 128 ways, indexed by sixteen distinct primes just above twenty million, of which a token reads 2,560 values. It is Qwen's, not BTL's. The k=5 in the card is mean-of-5, which the score lattice proves: 33.0 is not reachable on 66 tasks best-of-5. And the base-model figure the card cites at 29.0, Artificial Analysis lists today at 25.25. (2026-09-22) [LLM architecture · High · 73/100 · Original, source-checked analysis; Interactive explanations]
- [VoxelTTO spends 93% of a scene fixing the poses](https://ai.thesatyajit.com/articles/voxel-tto.md): Feed-forward 3D Gaussian Splatting degrades when the poses drift, and the damage is invisible on input views — it only shows on the views you did not hand the model. VoxelTTO anchors Gaussians to voxels instead of pixels and adapts LoRA modules at test time against pose supervision, with the vision foundation model frozen. The mechanism works: mean rotation error on Tanks and Temples goes 1.636° to 0.262°, and a 9.03 dB test-view gap closes to 1.66 dB. The cost is the number that decides whether you can use it, and the paper's own two runtime figures do not reconcile until you notice which precision the 2.304 s was measured at. (2026-09-22) [3D & spatial · Notable · 60/100 · Original analysis; A new technique]
- [Where to use Jev: the test is whether one option can read another](https://ai.thesatyajit.com/articles/where-to-use-jev.md): Eighteen pieces on this site have measured decision models in somebody else's production code. This is the procedure that falls out, in the order the questions have to be asked: is the answer set finite before you call, can code you own enumerate it, can every option be judged without reading another, and is the list frozen before you branch on the probability. Gate three is the one with a number on it — 0 of 100, below chance — and it is the only gate that cannot be bought back with money. Cost and latency are not gates at all: the two headline multipliers measure 4.57x and 336x when somebody runs both arms on one task, the 245x is 15x of interface times 16x of price, and the 240x is 3.7x once the tuning is counted. Also: the same checkpoint scoring 76.9% in-distribution and 54.1% out of it, which is why almost every published head-to-head in this category means less than it reads; what a catalogue of 426 shipped builds does when you run it through the procedure; and the five places I would not put one even though it would work. (2026-09-22) [Agents & harnesses · High · 68/100 · A guide you can follow today; Original analysis]
- [XGEN-JING wired up two more control axes, four days later](https://ai.thesatyajit.com/articles/xgen-jing-control-axes.md): Four days ago this site read XGEN-JING's control parser and found that four of its six camera-control dimensions were hard-coded zero. On 20 September XGEN pushed a commit titled 'fix: remove unintended limitations' that connects pitch and yaw to i/k and j/l, and shipped a new example built to exercise them. So: what exactly is behind those six numbers (302,596,096 parameters, fifty zero-initialised FiLM injectors, one per layer), what the conditioning interface really is (dialogue is a tag inside the prompt string, audio is a sentence describing it), and what 'interactive' costs — the finest interval at which anything can change is seventeen frames, and bidirectional attention means no frame exists until all of them do. (2026-09-22) [Image & video generation · Solid · 54/100 · Original analysis; Open code or weights]
- [XGrammar structural tags: a grammar guarantees it parses, a menu guarantees it was on the menu](https://ai.thesatyajit.com/articles/xgrammar-structural-tags.md): XGrammar-2's Structural Tags bring strict tool calling to SGLang and vLLM, enforcing tool-argument schemas through DeepSeek's own XML wire format rather than through JSON. That makes it the other answer to the question this site has been circling: how do you stop a model emitting something your program cannot handle? A decision model answers it by never letting the model write a string at all. A grammar answers it by letting the model write anything the language admits. These are different guarantees and neither implies the other, which XGrammar's own benchmark shows better than any argument: constraining Llama-3.2-1B took schema accuracy from 21 percent to 100 and output accuracy from 5 to 46.5 — so 79 points of unparseable output became about 41 points of correct output and 38 points of tool calls that parse cleanly and are wrong. Also: nine wire formats for one schema, 82x compilation speedup, and the one place the two mechanisms are literally the same construct. (2026-09-22) [Inference & serving · High · 70/100 · Original analysis; Runs on a consumer GPU]
- [ZCode, open-sourced: what a remediation can and cannot establish](https://ai.thesatyajit.com/articles/zcode-disclosure.md): Z.ai has published the ZCode client under Apache-2.0 after community reports that it generated and uploaded snapshots of local repositories. The response is unusually complete: remediate in v3.14.0, remove the feature, open the source, commission two third-party assessments, publish the findings. This is a factual account of what was published and what each piece of it establishes. The repository is the remediated tree and answers what the client does now — two commits, no Repo Wiki, no snapshot identifiers, and a 27 KB vendor-written enumeration of every outbound request path with no snapshot upload among its seventeen scenarios. What it cannot answer is what the client did, because the history is not in it. And a zero-data bucket is a fact about the present; 'no functional path identified' is a bounded negative, not a proof. (2026-09-22) [Security & safety · Solid · 49/100 · Original analysis; Concrete numbers to act on]
- [Laya vs Jev: the slower model gets an easier question](https://ai.thesatyajit.com/articles/laya-vs-jev.md): A repo puts a local MLX model and a hosted API on the same T-Rex course and races them. The obvious criticism — that a real-time game measures the network — the repo makes itself, at docs/TREX_DEMO.md:119, better than I would have. So I went looking for what an honest head-to-head actually measures, and it is the harness. The prompt each model receives is built from that model's own measured latency: I ran the repository's planner over its own 122 benchmark snapshots and the two players get the same question on 13.9% of them. Laya is told all three actions are safe on 71.3%; Jev on 19.7%. The side-by-side BEST MOVE PICKED bar in the screenshot is comparing two different exams, and the harder-looking number belongs to the easier one. Also: a measured takeoff window of 13-26 frames against a 22-frame round trip, a best-effort prompt that labels a doomed action 'Safe', and the one run that would settle it, which nobody has done. (2026-09-21) [Evals & benchmarks · Notable · 65/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [jev-linkmap: the rubric rewrite was mostly a threshold](https://ai.thesatyajit.com/articles/jev-linkmap.md): jev-linkmap rebuilt a 566-page site's internal link map in 5.9 seconds for $0.27, and the number I keep hearing quoted is 240x cheaper than Opus. I shipped the same feature here last night the other way — tag overlap, zero model calls — so I cloned the repo and recomputed every figure from its own run files. Most of them hold. Three do not. The 679 links are not an 8% yes rate, they are a 28% yes rate through a max_links_per_page of 3. The $0.27 is $17.91 once you count the tuning and the editor. And the headline 45%-to-65% recall gain is 64.2% if you leave the hand-written rubric alone and move two thresholds, because System 2 rewrites the instructions given to the referee as well as to the student — against a referee that never read the new rubric, its anchor agreement falls from 71% to 51%. Where their method beats mine is not in dispute and it is not cost: it puts the link inside a sentence the page already had, and mine can only staple a block to the bottom. (2026-09-20) [Agents & harnesses · High · 69/100 · Original, source-checked analysis; Concrete numbers to act on]
- [Laya on Apple silicon: 1.39×, not 50×, and K is a literal again](https://ai.thesatyajit.com/articles/laya-mlx.md): Yesterday's browser piece hit a wall — no GPU in the machine, every WebGPU number refused. Somebody else made the measurement on real Apple silicon and published the raw JSON: two independent ports of Laya, MLX and Core ML, with a hand-written Neural Engine graph. Three claims circulate about them. 50× faster than Jev: the word Jev appears zero times in 285 tracked files, and the only speed ratio either repo publishes about itself is 1.394× against compiled MLX on the same machine. 60 decisions per second: 60 is in the repo, as the paced row that fails at 100% deadline misses, and a Snake decision is three model calls with 4, 2 and 2 options, not one. At most 1 GB: that is a peak MLX allocator high-water mark for one short question, and it is 1,833 MiB for ten. And the finding I would keep — the option count is welded into the exported tensor shape here too, at 32 in five bundles and at 4 in the one that plays Snake. (2026-09-20) [Inference & serving · High · 73/100 · Original, source-checked analysis; A guide you can follow today]
- [Any model can be Jev, for the price of a serving flag](https://ai.thesatyajit.com/articles/any-model-can-be-jev.md): SGLang has shipped a /v1/score endpoint since June 2025 that returns a probability for each candidate token you name, on an ordinary generative checkpoint, in one forward pass with no decoding. That is the entire mechanism behind the System One readout, it was built for rerankers fifteen months before the category existed, and people are now pointing it at DeepSeek and LFM2.5 and getting decision models out. I read the handler, checked it against vLLM and llama.cpp, verified DiffusionGemma's parameter counts off the safetensors headers, and took the 63x speedup claim apart with a regression on its author's own committed rows. The serving half of a System One model is free. The half the category is named after is not, and you can tell which projects paid for it by who publishes a number. (2026-09-19) [Inference & serving · High · 69/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [json-render + Jev: generative UI in two model calls](https://ai.thesatyajit.com/articles/generative-ui-by-decision.md): Vercel Labs put a decision model behind json-render: instead of an LLM emitting a component tree, Jev picks from options your code enumerated. I cloned the repo and ran the composer against a recording evaluator — an eight-element dashboard is exactly two evaluation calls, 50 questions, 161 options in the first one. The 'rendered in milliseconds' claim survives, but the denominator is not one decision per component, and the cost moved out of round trips and into a ~7,100-token request that carries your entire catalog on every single call. (2026-09-19) [Agents & harnesses · Notable · 64/100 · Analysis found nowhere else; A new technique]
- [Jev as a substrate: every pixel is a Choice](https://ai.thesatyajit.com/articles/jev-as-substrate.md): Three projects this week used a typed-decision model as a general-purpose substrate. jev-paint asks Jev one 16-way Choice per pixel and paints the answers — a 32×32 painting is 1,024 questions in 8 requests for about half a US cent — and maps confidence to brush width, except the code gates strokes off below 1.92 effective colours, so the widest brush is the one it never draws. A MuJoCo arm run closes its loop at 0.62 Hz with 87.9% of the clock spent waiting on two sequential calls, and publishes an offline verifier that replays every motor command to 1e-7. And "Jesse v1", a model with no weights that consumes no tokens, claims 100% on HumanEval and 1% on SWE-bench Verified — where all five of its non-empty patches are byte-for-byte the gold patch, hunk headers and all. (2026-09-19) [Agents & harnesses · High · 69/100 · Original, source-checked analysis; Concrete numbers to act on]
- [Jev in the browser: N stops being a loop bound](https://ai.thesatyajit.com/articles/jev-in-the-browser.md): I built it: a per-option System One scorer on WebGPU/WASM through transformers.js, loaded on click, live in the page. It runs. Three things measuring it taught me. The isolation property that defines Jev costs 6.2× the tokens of the shared-sequence alternative at sixteen options, measured with one tokenizer on identical content — and both open System One models already gave that property up, which is why they are fast. The only one with a browser playground exports to ONNX with its logits fixed at [batch, 25], so the option count that was a loop bound in PyTorch becomes a tensor shape at the edge. And the ladder on a CPU is not small-and-fast versus big-and-slow: it is 437 ms and wrong versus 11.4 s and right, with nothing in between. (2026-09-19) [Inference & serving · Essential · 85/100 · Original, source-checked analysis; Runs in a browser]
- [Jev scores zero, and zero is the informative number](https://ai.thesatyajit.com/articles/jev-scores-zero.md): An independent benchmark put Jev at 0 of 100 on relational choice — questions where one option's content decides another. Not low: zero, well below chance. That is the signature of a structural constraint rather than a weakness, and it is direct evidence for something the RLCD piece could only argue: Jev's options never share a context. Also: Laya, the 421M open System One that is said to predate Jev and does not, and what its author actually published first. (2026-09-19) [LLM architecture · Notable · 60/100 · Analysis found nowhere else; A new technique]
- [ORPO: the odds ratio is the good part](https://ai.thesatyajit.com/articles/orpo.md): ORPO appends a log-odds-ratio term to the SFT negative log-likelihood and drops the reference model. The objective is derived here in full, because the odds ratio is chosen for its gradient behaviour and that argument is the best thing in the paper. The compute story is oversold: the frozen reference is 13.49 GiB against 107.91 GiB of trainable state at Mistral-7B, deleting it cuts 25% of per-step FLOPs rather than half, and TRL's DPO already keeps no second copy under LoRA. The win the paper never measured is GSM8K, where ORPO has the best mean delta of any preference objective — +2.50 against DPO's −8.27. Also: a sign error in Equation 8, and why ORPOTrainer now lives in trl.experimental. (2026-09-19) [Training & RL · Essential · 88/100 · Original, source-checked analysis; Interactive explanations]
- [Qwen3.8-LiveTranslate: half a second off the lag, and 2.7 points on](https://ai.thesatyajit.com/articles/qwen-livetranslate.md): Qwen3.8-LiveTranslate is announced on one number — average lagging down from 2.8s to 2.3s — and the number is nearly always quoted without the thing that makes it mean anything. The same figure publishes xCOMET-XXL 83.0 → 85.7 on the same 70 directions, which is the rarer and better result: in simultaneous interpretation latency and quality normally trade, and here they did not. What does not survive checking is the evidence around it. The 2.8s baseline is Qwen's own previous model, not a competitor; the eval is 19 languages, not the 60 the model supports, and every direction has English or Chinese at one end; the public release of the benchmark cited for the speaker-attribution wins contains no translation references at all; and the Interleave architecture the release is named after is disclosed as a block diagram and a sentence. (2026-09-19) [Speech & audio · Notable · 65/100 · Original, source-checked analysis; Concrete numbers to act on]
- [SAM 3.1: the 7× is at 128 objects](https://ai.thesatyajit.com/articles/sam-3-1-object-multiplex.md): Meta shipped Object Multiplex, a shared-memory rewrite of SAM 3's video tracker that packs sixteen objects into one memory-path forward pass. The model card says ~7× faster at 128 objects on an H100; the blog says it doubles throughput; the paper says 5.2×. All three are true and the difference between them is the entire story — the number is a slope, not a speedup, and Meta published the whole curve including the point where multiplexing is 7% slower. Also: which of the seven VOS benchmarks it lost, what $0.36 a minute buys, and why the API's flat per-frame price only works after this release. (2026-09-19) [Vision & multimodal · Essential · 80/100 · Interactive explanations; A guide you can follow today]
- [jev-semgrep: grep, but the pattern is a proposition](https://ai.thesatyajit.com/articles/search-by-meaning.md): jev-semgrep is 309 lines of zero-dependency Node that asks a decision model, line by line, 'does this line match the meaning of X?' and thresholds the probability. The idea is genuinely good and the argument against vector search is right: cosine distance has no complement, so 'about a refund but not a request' is not expressible as similarity, while 1 − P(proposition) is. I read the source, worked out the request shape and the real bill ($0.11 and 83 seconds for one pass over this site), found that the boolean algebra is only classical at --level normal — loose is paraconsistent, strict has a truth-value gap, and the project's own tuned optimum is one of the inconsistent ones — and then, because Satyajit asked, wrote the other half: where a decision-model call belongs in Next.js 16.3, what is actually cached by default (nothing), and why the threshold must never enter the cache key. (2026-09-19) [Developer tools & infra · Essential · 81/100 · Original, source-checked analysis; A guide you can follow today]
- [Splash on Apple silicon: 144 tok/s needs 502 GB/s, and Apple sells two M5 Max chips](https://ai.thesatyajit.com/articles/splash-engine.md): Inco AI open-sourced Splash, a Metal inference engine for two Qwen models, under Apache-2.0; LM Studio shipped it a day later. The written benchmark says 74 tok/s for Qwen3.8-27B on an M5 Pro. The number going around says 144 on an M5 Max. One decode step reads 16.73 GB — computable from the repository's own tensor shapes and its 4.5-bit pack — and Inco publish the acceptance length separately, so both figures can be checked against a memory bus. 74 needs 84% of the M5 Pro's peak bandwidth. 144 needs 502 GB/s, which the 40-core M5 Max has and the 32-core one does not. (2026-09-19) [Inference & serving · Essential · 81/100 · Original, source-checked analysis; Interactive explanations]
- [Decision models in three-tier agents: the middle tier is not in the middle](https://ai.thesatyajit.com/articles/three-tiers.md): Two posts in two days argue that agents now split three ways — deterministic code, a decision model, a reasoning model — and the split is real: I read three codebases built that way. But the spectrum reading of it is wrong. On the six properties that actually distinguish the tiers, three are not monotonic, and two of them bottom out in the middle. The decision tier is the only one that cannot see across its own options, and the only one that cannot tell you why. Stanley's router and an xArm7 harness both show the same workaround, which turns out to be the architecture: deterministic code builds the relation, then hands it down as an independent question. Plus the measured version of '193.6x faster' — 4.57x. (2026-09-19) [Agents & harnesses · High · 70/100 · Original, source-checked analysis; A lasting reference]
- [Jev on WindTunnel's WebMCP board: the 245x is 15x of interface and 16x of price](https://ai.thesatyajit.com/articles/webmcp-windtunnel.md): WindTunnel's newest board has a decision model at rank one — Jev picks the tool, Mercury 2.5 writes the arguments, 49 of 49 tasks solved over WebMCP. The published multipliers, 112x and 245x against GPT-6 Astra, are ratios of median model cost, and both factor cleanly into an interface term (15x) and a price term (16x) that the same board lets you separate. Doing that separation moves the interesting number from 245 to 15. Also: the cheap fast model is 78% of the bill, the 18% saving that came with doubled accuracy is entirely the cost of failed attempts, and the 49th task rests on a 79-millisecond wait that only this configuration has. (2026-09-19) [Agents & harnesses · High · 69/100 · Original, source-checked analysis; Concrete numbers to act on]
- [What a decision model cannot do](https://ai.thesatyajit.com/articles/what-decision-models-cannot-do.md): Every System One piece this week was about what these models can do. This is the negative space, and it is the more useful half, because the limits are predictable from the architecture rather than bugs waiting on a checkpoint. I set out to show that one architectural choice — the output is an element of the caller's option set — produces all five reported limits. It produces two of them. Two more are the opposite prices of a decision the shape leaves open, so no model pays both. The fifth is five constants in five repositories, and the received explanation for the smallest of them is wrong. Along the way: a new measurement of the letter prior in openjev (+1.71 logits for slot A, flipping 10 of 36 reversed questions), and the split in WindTunnel's published traces where 385 of 413 decisions had to be handed to a generative model. (2026-09-19) [LLM architecture · Notable · 60/100 · Original analysis; A lasting reference]
- [cua-s1-forms: a System One model in 706,048 parameters](https://ai.thesatyajit.com/articles/cua-s1-forms.md): Three open releases in one week rebuilt the Jev shape in public: cua-s1-forms trains a decision scorer from scratch at 706,048 parameters and reports 100% on its real-form eval; Bespoke's Nimble LoRA-tunes Qwen3.5-9B on contrastive pairs and beats an untuned 27B; jev-ultrafast drives a browser with no screenshots and no coordinates. The two families of decision model, side by side as the two short functions they are — a vocabulary readout that slices three rows out of a 151,936-wide logit vector, and a per-option scorer whose einsums have no option-against-option term. Plus a correction: I said none of these projects published a calibration number, and two of them did. (2026-09-18) [LLM architecture · High · 73/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [Monolith 1.0: a 1.6T open MoE built for reasoning](https://ai.thesatyajit.com/articles/monolith-1-0.md): RETRACTED BY THE LAB, the day after this was published. Basalt Labs described Monolith 1.0 as a 1.57T-parameter open Mixture-of-Experts with 49.5B active per token — top-2 routing over 128 experts plus one shared, a two-stage YaRN stretch to a 1M-token context, and an SFT/DPO/RLVR reasoning-RL recipe. The Hugging Face repo now holds a 436-byte README saying the weights are gone and the public model was an inflated Qwen 2.5 7B Instruct. Kept as a record of what was claimed: the mechanisms are real, the model was not. (2026-07-17) [LLM architecture · Niche · 11/100]
- [auto-gpu-kernel: an agent won the DSA kernel track unattended, and the baseline copies 624 MB per call](https://ai.thesatyajit.com/articles/auto-gpu-kernel.md): auto-gpu-kernel is a Claude Code loop that generated the winning DeepSeek Sparse Attention kernels for the MLSys 2026 FlashInfer contest with, its report says, no human in the loop. The rank checks out — the organisers list Dogacel first in Track B, Full-Agent. The 34.93x does not mean what it reads like: the contest scores the arithmetic mean of per-workload ratios against a named FlashInfer baseline whose sparse-attention wrapper runs torch.cat over the entire 624 MB paged KV cache before it calls a kernel the winner finishes in ten microseconds, and 69 of the 128 indexer traces never needed a top-k at all. The correctness gate, which is the part that usually rots, is the part that holds: FlashInfer-Bench runs the numerical check before it ever times anything, so a wrong kernel gets no latency, and one failing workload zeros the kernel's whole score. Also: the report swaps its own two trace counts, and the shipped kernel's grid-wide spin barrier is only deadlock-free because no contest trace has more than 8 tokens. (2026-09-18) [GPUs, kernels & systems · High · 72/100 · Original, source-checked analysis; A new technique]
- [Bend 2: proof-checked, and no longer an interaction net](https://ai.thesatyajit.com/articles/bend-2.md): Victor Taelin's Bend 2 launched with three claims welded together: it 'blocks AI mistakes via proof checking — the same technique big AI labs used to solve open math problems, like Navier-Stokes,' and it's 'very fast, and runs on GPUs.' I cloned bendlang/bend and its Lean mechanization to check each separately: a real, 3,734-line, human-audited checker for a Type-in-Type-but-consistent theory; a fair-but-rhetorical link to the Lean-checked Navier-Stokes/Euler proofs; and, unexpectedly, a shipped runtime that quietly dropped the interaction-combinator engine that made HVM and Bend 1 famous — replacing it with affine ownership, and finally publishing the honest C/TypeScript/Lean baselines the old 'meaningless baseline' complaint was asking for. (2026-09-18) [Developer tools & infra · Notable · 62/100 · Original analysis; Runs on a laptop CPU]
- [Bonsai 2 27B: PrismML's sequel gets to 98.2% — on average](https://ai.thesatyajit.com/articles/bonsai-2-27b.md): PrismML's second Bonsai release keeps roughly the same ~5.9-6 GB footprint as the first but starts from a stronger base (Qwen3.8-27B) and narrows the retention gap to a claimed 98.2% of full precision. That number is real, but it is an aggregate over a benchmark suite that changes definition between PrismML's own launch post (20 benchmarks), whitepaper (same 20), and model card (a different 14) -- and none of those three aggregates includes the two long-horizon agentic benchmarks PrismML evaluated for the first time, Terminal-Bench 2.1 and SWE-bench Verified, where the ternary model keeps only about three-quarters of full precision. This is a walk through the real GGUF/safetensors file manifest (what's still ternary, what isn't, and what the '5.9 GB language weights' figure excludes), the full 20-benchmark table, the measured throughput conditions behind the headline tok/s numbers, and why ternary weights still need a runtime built for them. (2026-09-18) [Quantization & compression · Notable · 67/100 · Checked against the source; Runs on a consumer GPU]
- [BrowserSkill: the agent has to borrow your tab to click it, not to read it](https://ai.thesatyajit.com/articles/browserskill.md): Tencent's BrowserSkill hands any shell-capable agent your real, logged-in Chrome, and its genuine contribution is where it puts the consent switch — 0.3.0 moved borrow-confirmation and human-help out of CLI flags the agent controls and into browser storage it does not, so --unattended now only prints a deprecation warning. I built the CLI, loaded the repo's own extension into Chromium 153, and ran nineteen of its commands against a tab in my own window that was never borrowed. Thirteen refuse with agent_window_scope and exit 1. Six answer: get-html returned the page's canary string, snapshot minted a clickable ref for it, and screenshot wrote a 9,923-byte PNG — no prompt, no notification. That split is deliberate and commented in the source; it is not in the README, the skill file the agent reads, or the Chrome Web Store privacy page. (2026-09-18) [Security & safety · High · 71/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [ChessLFM: 408 of the 552 Elo are not in the weights](https://ai.thesatyajit.com/articles/chesslfm.md): ChessLFM fine-tunes a 230M LFM2.5 into a chess engine and reports 2004 Elo (95% CI 1972-2039). Everything about the encoding checks out — I rebuilt the 1,968-move action space from board geometry and it matches the released tokenizer exactly, and the 80-token prompt the shipped code builds is exactly 80 tokens. What the headline leaves out is the pool and the wrapper: the ladder is node-limited Stockfish on its own UCI_Elo dial, a scale its authors calibrate to within ±100 Elo of CCRL Blitz at 120s+1s, and the project's own recipe table shows 408 of the 552 points gained over the raw policy come from a minimax loop and a repetition bug fix rather than from training — including a +106 row the prose never mentions. I also pulled the 4-bit ONNX the demo runs and tested the model with its legal-move mask removed: over 1,700 positions the unmasked argmax was already legal 99.65% of the time, so the mask is a safety net firing once every 280 moves, not the thing holding the board together. The browser demo, meanwhile, silently sets searchDepth to 0 when WebGPU is missing, while its own tooltip says it plays the 2004 configuration. (2026-09-18) [LLM architecture · High · 73/100 · Original, source-checked analysis; Interactive explanations]
- [Jev + Kimi K3 fraud detection: 96 of 100, on a 50/50 inbox](https://ai.thesatyajit.com/articles/confidence-gated-fraud-detection.md): Hassan El Mghari's Jev + Kimi K3 fraud demo reports 96/100 correct for about seven cents, which is true and also the least informative way to describe it. The 100 emails are a balanced 50/50 sample of DIFrauD's phishing split — a base rate no inbox has — and the app's own footer says so while the thread that went viral does not. Reading the confusion matrix off the recording gives 100% precision measured on 50 negatives, which bounds the false-positive rate no tighter than 5.8%; at 0.5% prevalence that bound puts precision at 7%. And the spend arithmetic says the escalation buys a label, not reasoning: $0.0679 over 31 reviews at Together's published $3/$15 leaves about five output tokens each. (2026-09-18) [Evals & benchmarks · High · 68/100 · Original, source-checked analysis; A lasting reference]
- [CUDA Rust: writing the kernel, not just launching it](https://ai.thesatyajit.com/articles/cuda-rust.md): NVIDIA's CUDA Rust announcement ships two ways to write GPU kernels natively in Rust — cuda-oxide, a real rustc backend that compiles SIMT kernels to PTX, and cutile-rs, a tile-based compiler that runs on stable Rust and is already in production at Hugging Face. I cloned both crates to find out exactly what their aliasing-safety claim covers. The honest answer: cuda-oxide's DisjointSlice machinery closes the CUDA C++ __restrict__ gap for aliasing between kernel arguments in global memory, but shared-memory races between threads in a block still need unsafe and are checked by nothing; cutile-rs makes the stronger claim by deleting per-thread indexing from the source language entirely, at the cost of a restricted kernel-body DSL rather than full Rust. cutile-rs's own B200 numbers say the safety is free — GEMM and elementwise throughput sit within measurement noise of the unsafe variant and of cuBLAS — while cuda-oxide, the harder SIMT track, ships no public benchmark at all. (2026-09-18) [GPUs, kernels & systems · Notable · 65/100 · Original analysis; Runs on a consumer GPU]
- [Depth Anything 3 in ROS 2: the metres come from the wrapper, not the model](https://ai.thesatyajit.com/articles/depth-anything-3-ros2.md): ika's ros2-depth-anything-v3-trt publishes a 32FC1 depth image and a PointCloud2 and calls both metric. I traced where the metres actually come from: a single line, depth *= 0.5*(fx*sx + fy*sy)/300, in the node's CUDA postprocess. The 300 is Depth Anything 3's own focal normalisation constant, and the checkpoint the node runs cannot apply it itself — I pulled the published ONNX apart with Range requests and the graph ends at two raw head outputs, 1418 nodes, no metric scaling, no sky fill, fp16 weights already baked in. The conversion is correct in kind and exact only for a 1.8:1 camera; on a 4:3 sensor the same code silently shortens every published distance by 13%. Plus a dead ROS parameter, a README that has the sky mask backwards, and a depth image that disagrees with its own point cloud about what the sky is. (2026-09-18) [Robotics & embodied · Essential · 76/100 · Original, source-checked analysis; Interactive explanations]
- [Dream-RSI: the loop rewrites a scheduler, not the model](https://ai.thesatyajit.com/articles/dream-rsi.md): Dream-RSI (arXiv 2609.14858) is real, and it is 14 of 17 authors deep in Google and Google DeepMind — the personal GitHub handle hosting the repo just means the code and the arXiv posting weren't ready yet, not that the affiliation is thin. What actually recurses across its outer rounds is about 150 lines of Python that schedule a frozen coding agent's search; zero gradient steps touch any model, and the 'world' it dreams in is a cache of past attempts, not a learned dynamics model. Checked against its own controlled baseline (identical agent, evaluator, and round-1 budget), the win holds at equal or lower compute in eight of nine reported comparisons — genuinely well-controlled — but it loses outright on the ninth, and the paper never counts the compute its own policy-rewriting agent spends. The longest per-round curve it publishes runs nine rounds with no sign of a plateau, which is real progress and a long way from the compounding phenomenon 'recursive self-improvement' names in safety discourse. (2026-09-18) [Agents & harnesses · Solid · 51/100 · Original analysis; A new technique]
- [Edge0 streams a 35B MoE off SSD. I counted the bytes it can't stream.](https://ai.thesatyajit.com/articles/edge0-streaming-moe.md): Edge0 open-sourced a streaming MoE engine that serves a 35B model in 2.9 GiB by keeping experts on disk and predicting the next layer's routing one token ahead. The mechanism is real and the memory claim survives a byte-level audit — I rebuilt the budget from the safetensors headers without downloading the weights, and the 1.46 GiB non-expert floor plus a K=4 decode step gives 1.49–1.73 GiB, which brackets the 1.7 GiB their own iPhone demo prints. Four things do not survive: the README's '~23 GB' checkpoint measures 19.51 GB (the paper says 19.5 GB and is right), the README's headline 'up to +59%' prerouter gain appears nowhere in the paper it links to (which measures +2.5% on the machine the README benchmarks and +80-84% on one that can't hold the model), the same 35B tier ships five different decode rates spanning 13.0 to 20.4 tok/s with two of them claiming the same Mac mini, and the 'A3B' in the repo name describes the base model's native K=8 while edge0 serves it at K=4. (2026-09-18) [Inference & serving · Essential · 77/100 · Original, source-checked analysis; Interactive explanations]
- [The fly connectome is a substrate, not a model](https://ai.thesatyajit.com/articles/fly-connectome-computing.md): In September 2026 Google Research and HHMI Janelia published the complete male fruit-fly central nervous system — 166,691 neurons, brain to nerve cord. Within two weeks a reservoir computer, a language-model adapter, a candidate answer to a 20-year-old navigation question, a browser-driving robot and a memecoin launcher were all built on it. I read the papers and cloned the repos to work out why the same wiring diagram gives completely different answers depending on what you ask it — and it turns out the disagreement dissolves the moment you separate what a connectome measures from what every model quietly supplies itself. (2026-09-18) [Data & datasets · Notable · 67/100 · Original analysis; Runs on a laptop CPU]
- [The generated web: Qwen3.8-27B on Cerebras at 2,000 tok/s, and no ground truth](https://ai.thesatyajit.com/articles/generated-web-at-2000-tokens-per-second.md): A demo running Qwen3.8-27B dense on Cerebras at roughly 1,850–2,000 tokens/second synthesizes an entire webpage's DOM live, no network calls, nothing on disk — type a site and a year and 'YouTube in 2045' renders next to 'YouTube in 1999.' The throughput is real, and the wafer-scale architecture behind it is worth understanding on its own; the arithmetic on page size versus perceptual-latency thresholds shows exactly why this speed, and not last year's, turns a page load into something that feels instant. But 'YouTube in 1999' has no ground truth to violate — the company didn't exist for another six years — which is exactly why the trick works and exactly why it stops working the moment a page has to be correct rather than merely plausible, and 'zero gigabytes stored on disk' turns out to be several thousand times more expensive per page than serving the same bytes from a CDN. (2026-09-18) [Inference & serving · Solid · 49/100 · Original analysis; Concrete numbers to act on]
- [GLM built its own inference stack: one PR, one bottleneck, and a 3x with no rival](https://ai.thesatyajit.com/articles/glm-built-its-own-inference-infra.md): Z.ai says a GLM-5.3-powered 'Infra Agent' took GLM-5.3-Flash from first successful run to production in thirteen days, tripling end-to-end throughput, and frames it as an early step toward recursive self-improvement. I cloned the three repositories the post names and checked it line by line. Two of its three engineering case studies turn out to be independently verifiable, not one: PR #1180 is a real six-file merge commit dated exactly where the timeline says, and DeepEP v1.2.1 really does hold the Python GIL through intranode_dispatch's CPU spin while internode_dispatch, 348 lines later in the same file, already releases it with a comment naming KV transfer. The third case's 'repeated four times' is the released config's head_dim 128 divided by a hardcoded BV = 32 in flash-linear-attention's KDA decode kernel. What doesn't hold up: the headline 3x is measured against Z.ai's own first working run, the GIL fix was never upstreamed, and the post's real load-bearing claim — parity with 'mainstream NVIDIA GPUs' on utilization and per-token cost — arrives with no number, no engine and no named chip. (2026-09-18) [Inference & serving · Notable · 59/100 · Analysis found nowhere else; Concrete numbers to act on]
- [GOAT: entropic optimal transport, minus the transport](https://ai.thesatyajit.com/articles/goat-optimal-transport-attention.md): GOAT reframes attention as an entropic-optimal-transport problem and claims to solve attention sinks with a continuous, trainable prior — but the transport here is one-sided: each query balances against its own mass-one constraint, with no budget on keys, so nothing here is the two-sided Sinkhorn machinery (used elsewhere on this site, and by prior OT-attention work) that would actually forbid a sink by construction. What the paper proves instead is that a sink is the mathematically optimal response to a low-signal query, then relocates it from an entangled, high-norm content key into an explicit position-only bias — decoupling sink behavior from token content. Checked against the paper, the released package, and the figures themselves: the perplexity win over the strongest prior baseline is real but modest (1.55 points, in-distribution only), the retrieval-task wins are larger and more convincing, the whole thing rides for free inside one ordinary FlashAttention call, and every experiment tops out at 125M parameters with no quantization or KV-cache-eviction test — which is exactly where sinks hurt in production. (2026-09-18) [LLM architecture · Solid · 49/100 · Original analysis; A new technique]
- [Grouped Value Attention: cache the value, reconstruct the key](https://ai.thesatyajit.com/articles/grouped-value-attention.md): GVA stores only grouped values in the KV cache and rebuilds each query head's content key from them with a learned linear map, absorbed into the query so nothing extra touches memory — cutting persistent cache scalars ~45-47% versus matched GQA. The paper's own table puts the accuracy cost at 44.35 vs GQA's 44.36, a 0.01-point gap it calls within seed variation — smaller than the spread across the paper's own seven trained variants (0.64 points) and smaller than the GQA-MLA gap in the same table (0.48). A widely quoted 44.18 doesn't match the primary source, no decoding kernel is public yet, and the one released checkpoint implements a different, explicitly-failed ablation. (2026-09-18) [LLM architecture · Solid · 54/100 · Original analysis; A new technique]
- [What the Jev ecosystem actually built](https://ai.thesatyajit.com/articles/jev-ecosystem.md): In the nine days after TypeSafe's launch, at least fifteen independent projects shipped real code against Jev — browser, Mac, and Android agents, a trading bot, a code reviewer, a model router, a log triager, a Home Assistant integration, a web search engine, and a from-scratch reimplementation of the scoring head with temperature scaling measured before and after. Read against their own source rather than their READMEs, they converge on one shape: a cheap, fast classifier sitting in front of an expensive model or an expensive action. Most report no numbers at all; one open GitHub issue undercuts the strongest self-reported result in the roster; and pngwn/open-jev's committed metrics.json is the first place in this whole story with a published, checkable calibration number. (2026-09-18) [Agents & harnesses · Notable · 59/100 · Original analysis; Concrete numbers to act on]
- [Jev, TypeSafe's System One model: the receipts, itemized](https://ai.thesatyajit.com/articles/jev-system-one-models.md): TypeSafe AI's launch post for Jev and "System One Models" ends with a dare: "extraordinary claims require extraordinary evidence, so see below for the receipts." I take the dare literally — crawling every page of docs.typesafe.ai, reading the reproductions tracker and an independent 706-row benchmark (openjev) against Jev's own published numbers, and pricing out what "free output tokens" actually means when the output is a handful of enum choices. The speed claim mostly survives contact with outsiders; the "frontier-level intelligence" claim shrinks a lot; RLCD, the training method behind the calibration story, is a name and one paragraph, not a paper. (2026-09-18) [LLM architecture · Solid · 55/100 · Original analysis; A new technique]
- [Jina-OCR-v1: 570M active parameters assumes an embedding tie that isn't in the checkpoint](https://ai.thesatyajit.com/articles/jina-ocr-v1.md): Jina-OCR-v1 pairs DeepSeek-OCR's 3.4B-parameter vision-language backbone with a FastMTP speculative-decoding head that recursively shares one draft block over three steps, and the tech report states 570M active parameters per token. Reading the safetensors headers directly shows that figure only works if the embedding and the output head share one 165M-parameter matrix -- but the released config sets tie_word_embeddings to false and ships two separate matrices, putting real per-token decoder compute near 740M, about 30% above the advertised number. The paper's own measured 1.95x FastMTP speedup holds only in eager PyTorch on an NVIDIA L4; under vLLM's CUDA graphs, the K=3 setting the marketing implies is the one to use actually lands at 1.09x, the worst of the three depths tested, because K=1 is faster once the non-speculative baseline is already 3.7x quicker. Benchmarked against a dozen current specialized OCR systems, Jina-OCR-v1 places third on both olmOCR-Bench and OmniDocBench, and its weakest category isn't tables or reading order but degraded historical scans, where it barely clears the DeepSeek-OCR backbone it post-trains. (2026-09-18) [Vision & multimodal · Notable · 64/100 · Checked against the source; Runs on a consumer GPU]
- [Lexing on the GPU: the scan is the easy part](https://ai.thesatyajit.com/articles/lexing-on-the-gpu.md): Pareas compiles a whole language front end on the GPU, and its lexer is a genuinely elegant idea — map every byte to a DFA transition function, compose them with a prefix scan, done in O(log n) parallel steps. I rebuilt its lexer generator to measure the thing that idea costs: the k×k merge table it composes through. The shipped JSON grammar needs 4,176 states and 34.9 MB, exactly as the thesis says; adding one string rule to the language grammar takes it from 7.4 MB to 126.5 MB, and adding one block-comment rule makes the generator die of std::bad_alloc. The headline result — the GPU parser beats simdjson on large input — survives, but the Total column it rests on silently excludes the host-to-device upload, and once you add the transfer back the margin falls from 3.03× to 1.86×. There are no correctness tests in the repository, so I wrote one: five of fourteen inputs disagree with a reference JSON decoder, which is three distinct defects — uppercase hex escapes rejected, a raw 0x7F rejected, and invalid UTF-8 accepted in silence. (2026-09-18) [GPUs, kernels & systems · Essential · 80/100 · Original, source-checked analysis; Interactive explanations]
- [Needle 3: one checkpoint, five models, and a file that weighs more than the page says](https://ai.thesatyajit.com/articles/needle-3.md): Cactus's Needle 3 trains one 20-layer checkpoint so that any depth from 2 to 20 layers is a standalone model — a real mechanism (bisected block order, stochastic-depth sampling, self-distillation against the full model), not a marketing gloss, and I traced it to the training report. But the 2-layer slice scores 0.9% on Mobile Actions before fine-tuning, so sliceable does not mean useful at the bottom of the ladder. The banner claim that Needle 3 'passes DeepSeek V4 Flash' is Cactus's own qualifier, not mine — fine-tuned, from 4 layers up, true on DroidCall and not true on Mobile Actions even after tuning. And the one number a download can check outright doesn't hold: the 20-layer needle3.cact on Hugging Face measures 33.7 MiB, not the 29 MB ceiling the live page and its own blog post both state. (2026-09-18) [LLM architecture · High · 73/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [Open-1B: what auditable training actually audits](https://ai.thesatyajit.com/articles/open-1b-auditable-training.md): Gensyn's open-1b ships a chained, bit-exact hash over every one of its 80,957 pretraining steps, replayable on a laptop instead of a cluster -- a real advance over proof-of-learning's usual approximate handshake, because there is no tolerance for a spoofed trajectory to hide inside. But the public ledger I pulled on day three shows 26 of those steps checked and zero of 810 segments confirmed, the server's own acceptance check is a loss comparison rather than a hash recomputation, the shipped audit kit's trajectory file contains only initialization units and no step-interval unit at all, and the two training stages that turn the audited base model into the instruct model most people will actually run -- midtraining and SFT -- sit entirely outside the chain. I joined the ledger to the run's own 80,958-link hash log myself, which nobody appears to have done: all 27 submissions land exactly on their published link, and one step has now been replayed on both an RTX 3090 and an M4 Max to the same bits. 'The first model you don't have to trust' turns out to mean something narrower and more useful: the first model where not trusting it is a concrete, byte-level exercise, for one stage of one release, that almost nobody has finished yet. (2026-09-18) [Security & safety · High · 76/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [Parallel Constrained Decoding, From First Principles](https://ai.thesatyajit.com/articles/parallel-constrained-decoding.md): Every decoder-only LLM already computes a full-vocabulary distribution at every position — restricting it to a JSON field's legal values and renormalizing is arithmetic you get for free, no new training required. The seven-step mechanism, verified against two real implementations (including a confirmed divergence between them on multi-token candidates), a runnable ~40-line PyTorch build, and a concrete calibration recipe with code. (2026-09-18) [Inference & serving · Essential · 83/100 · Original, source-checked analysis; Runs on a laptop CPU]
- [Qwen3.8-Omni-Flash: skipping most of the video on purpose](https://ai.thesatyajit.com/articles/qwen3-8-omni-flash.md): Alibaba's first omni model built around agentic delivery: it decides what to watch instead of watching everything, and Qwen's own numbers say so — 45.7% fewer tokens on OmniVideoBench, not the 51.8% making the rounds. API-only, no open weights. The pricing math, the benchmark table with real baselines, a sound-localisation demo whose own waypoints check out to six millimetres, the one latency table Qwen publishes (about a second to first audio), and a GitHub repo that's still a 404 the day it was announced as open-sourced. (2026-09-18) [Vision & multimodal · Notable · 59/100 · Original analysis; Widely used]
- [RelateAnything: no object labels, and one gate that was told the answer](https://ai.thesatyajit.com/articles/relate-anything.md): RelateAnything takes the move that made detection and segmentation taxonomy-free — put the class list in as an input — and finishes it for relation prediction: a 53M-parameter head that reads pixels and boxes, never an object label, and scores pairs against a bank of text embeddings you can swap at inference. Checked against the paper, the three released checkpoints and the code: the parameter counts reproduce to the digit, the 2.3–3.5× mean-recall margin is real but is against OvSGTR rather than the paper's own stronger baseline (1.34–1.86× against that one), the abstract's 20 ms is the compiled path the repo says never to score a model in, and the predicate-routing gate the paper says three times is unsupervised is warm-started by a logistic probe fitted on a corpus spatial flag — a flag the routing buffer inside the shipped checkpoint still separates at AUC exactly 1.0000. (2026-09-18) [Vision & multimodal · Essential · 76/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [Retrieve-for-Train: the 20x is in the prose, not the figure](https://ai.thesatyajit.com/articles/retrieve-for-train.md): R4T (arXiv 2603.06397, ICML 2026) is a genuinely reusable idea — run RL once as a compiler that turns a non-differentiable set-level reward into ordinary supervised targets, then train a 53.9M diffusion retriever on them and throw the 4B policy away. The mechanism holds up. The headline does not: I read the data points out of the paper's own Figure 5 SVG and the plotted speedup runs 8.1x to 16.2x, below the claimed 12x floor at five of eight batch sizes, and never reaching 20x anywhere. The 20x exists only in a prose sentence whose two diffusion latencies both disagree with the figure above it. 'Single-pass' means 256 denoiser evaluations. The shipped model cannot be scored on 60% of the objective it was trained for, and the paper names two different judges in two different sections. Then there is the rebuttal repo nobody opened: 419 raw human ratings that split the two metrics between the arms where Table 1 has R4T taking both. (2026-09-18) [Training & RL · Notable · 65/100 · Original, source-checked analysis; A lasting reference]
- [RLCD is not constrained decoding](https://ai.thesatyajit.com/articles/rlcd-calibrated-decisions.md): TypeSafe's launch post makes three claims about Jev — a new architecture, a parallel sampler, and a training method called RLCD. The sampler has been reproduced to death. This piece is about the other two. RLCD expands to Reinforcement Learning for Calibrated Decisions, not constrained decoding, and TypeSafe publishes its output contract and nothing else: no loss, no reward, no data, no calibration number anywhere in 835 KB of docs, the launch post, or the eval site. So I recovered what is recoverable — the undocumented `score` and `confidence` formulas, straight out of their own worked examples — read four open reproductions' safetensors headers to see what a decision checkpoint actually contains, computed the first reliability diagram anyone has computed from openjev's committed logits, and wrote down what the objective would have to be, clearly labelled as reasoning rather than reporting. (2026-09-18) [LLM architecture · Essential · 82/100 · Original, source-checked analysis; Interactive explanations]
- [Ternary Bonsai 2 27B Uncensored: the numbers behind a runtime abliteration](https://ai.thesatyajit.com/articles/runtime-abliteration.md): OrcaRouter's OrcaBonsai runtime applies Arditi et al.'s refusal-direction ablation to prism-ml's Ternary Bonsai 2 27B as a forward hook on the residual stream instead of a weight edit, because editing a ~1.72-bit QAT model's weights means dequantizing, projecting, and re-quantizing — throwing away exactly what the quantization-aware training bought. Checked against the cloned repo and the pack's own Hugging Face metadata: the runtime never writes to the 8,595,477,990-byte MLX pack it loads, which is a different, larger container from the 5,946,648,928-byte GGUF file this site already sized up; the 129 hooks resolve cleanly to one embedding write plus an attention-out and an MLP-down write per each of 64 layers; and the strength knob is an unclamped float whose alpha=0 path skips the wrapping code entirely rather than merely zeroing it out. What the repo doesn't measure: the per-token cost of those 129 hooks, on hardware it also doesn't benchmark for this specific change. (2026-09-18) [Security & safety · Solid · 54/100 · Original analysis; Runs on a laptop CPU]
- [Tiny browser models: six weeks of parameters, one week of shipping](https://ai.thesatyajit.com/articles/tiny-browser-models.md): In about a week, Vercel's Shu Ding shipped a 41,321-parameter, 27.64 KiB WebGPU syntax highlighter, and five more task-specific browser models followed: a 38,745-parameter date parser, a 29,597-parameter search-filter parser explicitly ported from it, a 36,354-parameter model trained to centre a flexbox row, an unattributed cron-expression parser, and a 245,063-parameter Arabic morphology analyzer. I cloned every public repo, read the training code, ran the live demos myself, fed one of them pure gibberish (it confidently mis-parsed it as a real schedule, at 95.8% self-reported confidence), and built the economics: a CDN-delivered 28 KB model costs about 76 times less than one Claude Haiku 4.5 API call at the first request, and the ratio only grows from there. (2026-09-18) [Developer tools & infra · High · 71/100 · Runs in a browser; Original analysis]
- [UltraData-Code: counting a 1.2 TB corpus without downloading it](https://ai.thesatyajit.com/articles/ultradata-code.md): OpenBMB's UltraData-Code advertises ~400B tokens at L2 and ~150B at L3 across 11 languages. Measured through the datasets-server APIs against openbmb/MiniCPM5-2B's tokenizer, L2 is 431.6B -- 7.9% above its own round number -- and L3's headline 150B is the full_content column specifically: read the content column, which is what the default config hands you, and L3 is 52.4B tokens. The extrapolation that produces those numbers reproduces the corpus's exact Arrow byte total to 0.07%, so I trust it. Three other things fall out of the released scores: 'algorithmically relevant code' is 26.6% ALGO by the project's own role classifier and 41.7% TOOL, the selection thresholds nobody published are readable off the data (quality never below 3.0, relevance floors of 0.50-0.85 per language, and 0.00 for Python), and 178 of 5,500 sampled L2 rows are byte-identical to another sampled row, every pair crossing a repository boundary. The ten benchmark deltas on the card all reproduce exactly from its own tables. The licence does not: the card YAML and the shipped LICENSE say Apache-2.0, and the README, in English and Chinese, prohibits the redistribution Apache-2.0 grants irrevocably. (2026-09-18) [Data & datasets · Notable · 68/100 · Original, source-checked analysis; Concrete numbers to act on]
- [Uno: the draft model was already inside the target model](https://ai.thesatyajit.com/articles/uno-diffusion-augmented.md): Uno replaces the extra draft model in speculative decoding with a LoRA adapter bolted onto the exact backbone it gets verified against, and its 'lossless' claim is the precise speculative-decoding kind — the same rejection-sampling rule that makes ordinary speculative decoding provably exact, not just 'the benchmark score didn't move.' The paper's own numbers mostly hold up: at batch size 1 it really is about 2.2–2.5x faster than its base model, and it beats EAGLE-3 and DFlash, the actual speculative-decoding incumbents, at every batch size tested — not just other diffusion LLMs. But the abstract's 'up to 3x' doesn't trace to any number in the paper's own tables, and the shipped Hugging Face checkpoint's own benchmark table disagrees with the paper's Table 1, and with the paper's own Figure 1, on the same score, by up to twelve points. (2026-09-18) [Inference & serving · Essential · 76/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [WanGP and YuE2: VRAM is a policy, not a property](https://ai.thesatyajit.com/articles/vram-is-a-policy.md): WanGP says you can run generative video on 6 GB. YuE2 says its 3B music model needs 24 GB. Both numbers are about residency policy, not about the models. I read the offload code and measured every checkpoint from its safetensors header: WanGP gets a 31.70 GiB MiniMax H3 onto a 6 GB card by holding 2.81 GiB of it — a fixed two-block window — and pushing 30.10 GiB across PCIe on every forward pass, a cost its headline never states and hasn't stated alongside a GPU and a wall-clock since April 2025. YuE2 is the mirror image: 6.76 GiB of weights, a 24 GB floor, an offload_ar flag that frees 4.03 GiB and is off by default, and an untied embedding pair that makes int8 save 32% instead of 50%. (2026-09-18) [Inference & serving · Essential · 80/100 · Original, source-checked analysis; A guide you can follow today]
- [XGEN-JING and DAO: the renderer shipped, the world engine didn't](https://ai.thesatyajit.com/articles/xgen-jing-dao.md): XGEN Labs pairs JING (镜), a first-person video/audio diffusion model, with DAO (道), a described-but-unreleased 'computable world engine' that is supposed to hold the persistent state JING only renders. JING just took #1 on WBench's Full split and #2 on Navi — verified directly against the live leaderboard, current as of today. But the code tells a narrower story than the announcement: JING ships as an offline, script-driven, non-causal diffusion transformer built on MiniMax-H3; DAO has no repo, no weights, and no paper, and its own product-page demo is a hand-scripted panorama mockup that says 'no LLM' in its own source comments. (2026-09-18) [Image & video generation · Solid · 58/100 · Original, source-checked analysis; Open code or weights]
- [ZDTaichu5.0-9B: the release contradicts itself on its own best benchmark](https://ai.thesatyajit.com/articles/zdtaichu-5-9b.md): Taichu's ZDTaichu5.0-9B leads its model card's spatial tables — 47.20 on MMSI-Bench, 62.50 on ViewSpatial, 78.27 on MindCube-tiny. A second README in the same GitHub repo reports the same model, ungated, at 36.80, 54.10 and 74.04, because the headline numbers were run with a think/boxed format instruction bolted onto the prompt and the second set was not. On MMSI-Bench that is 104 of 1,000 questions, and it is more than the whole of the model's 2-point lead over Gemini 3 Pro. I range-read every safetensors header without downloading a weight: the artifact itself is clean and reconciles to the byte, the headline feature ships in neither the Hugging Face repo nor the recommended vLLM path, and five of the terms its own mechanism diagram advertises — normalized entropy, margin, hidden residual, anchoring and rollback — return zero hits in the released code. (2026-09-18) [Vision & multimodal · High · 68/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [AuK: 6.12 GB, an empty Hub config, and a parameter count that only balances in FP32](https://ai.thesatyajit.com/articles/auk-speech-editing.md): Tencent's open speech foundation model reports no safetensors metadata on the Hub, so the only way to get a parameter count is to do it yourself: divide the checkpoint's byte size by a dtype and see which one lands on the paper's own '~1.5 billion.' bf16 — what the model actually runs inference in — misses by 2x. FP32 matches. Updated 18 Sep 2026 after reading the checkpoint's own safetensors header: the fp32 conclusion holds, the exact count is 1,530,538,629, and the byte-division shortcut overcounts by exactly one header. Plus the post-release changes — CPU offload, an MLX branch, and a one-line fix that recovered 7.5 GiB because the same cast that proved the checkpoint was fp32 had been upcasting the frozen encoder too. (2026-09-09) [Speech & audio · Essential · 76/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [Qwen3.8-2B-Distill: the filter wrote most of the headline](https://ai.thesatyajit.com/articles/qwen3-8-2b-distill.md): A 2B distillation of Qwen3.8's 2.4T flagship, shipped as GGUF, sold on MMLU 28.3 → 54.8 and GSM8K 33 → 64 at 1.3 GB on a phone. The parent card prints a second column almost nobody reads, and it changes the GSM8K story completely: the base already solves 54.5% under lm-eval's strict-match, and the same recipe makes the 4B and 9B siblings worse. Read from the primary sources — every GGUF header range-requested and decoded, the safetensors index, the shipped training_args.bin, llama.cpp's own KV allocator, and the task YAMLs the numbers came from. Updated 18 Sep 2026 for the 35B-A3B release, which fixes almost everything this article complained about — exact file sizes, a vision projector, the 8,192-token training window disclosed in writing — and then scores the model on four benchmarks where it is never allowed to generate a token. (2026-08-26) [Evals & benchmarks · Essential · 80/100 · Original, source-checked analysis; Runs on a phone]
- [Soup: an 8B model in 3.6 GB of VRAM, and 216 commands around it](https://ai.thesatyajit.com/articles/soup-cli.md): Soup's headline claim is that layer streaming fine-tunes an 8B model on a 4 GB laptop GPU. I rebuilt the memory arithmetic from the architecture — 218,103,808 params/layer, NF4 plus double-quant, 112.51 MB/layer times 32 layers — and it lands on exactly 3.600 GB, backed by a repaired silent-gradient bug, an executable proof notebook, and a paper the maintainer revised specifically to retract a claim nobody outside the project had challenged. The same codebase's 'one config, one command, no config hell' promise sits in front of a 237-field training config and 216 CLI commands, and the project's own changelog is the best source for where that promise stops holding. Both things are true; this is what checking each one turned up. (2026-09-13) [Training & RL · Essential · 76/100 · Checked against the source; Runs on a consumer GPU]
- [Marigold V2: a 20B model wearing a v1 badge](https://ai.thesatyajit.com/articles/marigold-v2.md): Marigold V2 keeps its predecessor's name and its 'cheap to fine-tune' pitch, but it is not the same model: V1 was an 866M-parameter Stable Diffusion U-Net, V2 is a rank-128 LoRA on Qwen-Image-Edit-2509's 20.43B-parameter diffusion transformer, quantized to 4-bit. I read the paper and the released training code, then range-requested the checkpoints' safetensors headers byte by byte — the LoRA parameter count, the VAE-decoder split across all nine checkpoints, and a 384x oversized text-embedding file the shipped code never reads past its first of eight slots all check out exactly. The headline '16-26% AbsRel improvement' is arithmetically real but quotes the two best of five benchmark columns; the five-way average is closer to 12%, and 'cost-effective' turns out to mean cheap relative to other diffusion-based depth models, not cheap in any absolute sense. (2026-09-10) [3D & spatial · Essential · 76/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [Probe monitors: reading activations, not the scratchpad](https://ai.thesatyajit.com/articles/activation-probe-monitors.md): A vendor post uses OpenAI's Hugging Face incident to sell activation probes as monitors, but OpenAI's own report credits a chain-of-thought monitor for the 'would have caught it' claim, not a probe, and hedges it as a hindsight-tuned retrospective test on the exact incident it's being asked to predict. I work the arithmetic the post skips: what a linear probe actually costs against GLM-4.5-Air's own published config, what Goldowsky-Dill et al. and Kramár et al. actually measured about probe robustness under distribution shift and adaptive attack, and the operating-point math for the probe-to-judge cascade the post gestures at but never runs the numbers on. (2026-09-09) [Interpretability · Notable · 62/100 · Original analysis; Runs on a consumer GPU]
- [Cross-model KV transfer: a linear map that skips prefill, four pairs out of six](https://ai.thesatyajit.com/articles/cross-model-kv-transfer.md): Production stacks swap between different-sized models in a family constantly — cost-quality cascading, mid-conversation escalation, routing — and every swap forces the receiver to repay the entire prefill from scratch. This NVIDIA paper skips it: a closed-form per-head ridge regression maps the source model's KV cache straight into the target's, no backprop, fit in under an hour on 8×H100. It works because cross-model KV has real linear structure — one Qwen3 14B→32B source layer explains 56% of key variance, 79% with eight. But the '73–98% retention' the abstract leads with is four of six tested pairs; the other two collapse to 11–15% of standalone accuracy once you correct for each benchmark's chance floor, and the nonlinear MLP that rescues them needs actual gradient descent — quietly retiring the closed-form, training-free pitch for exactly the cases that need the most help. (2026-09-09) [Inference & serving · Solid · 55/100 · Original analysis; A lasting reference]
- [Desert Ant Labs: real wins, missing metrics](https://ai.thesatyajit.com/articles/desert-ant-labs.md): Desert Ant Labs launched seventeen on-device models on September 8 with a wall of head-to-head numbers: a 2MB language detector beating a 293MB one, a 12MB PII filter within three points of a 2.3GB one, a 284MB clip-selector claimed to match Claude Sonnet at 470x less energy. Checking each claim against the Hugging Face repos, their config.json files, and Desert Ant's own SDK docs finds most of the individual comparisons hold up on their own terms, but the two most-quoted ones (Clips vs. Sonnet, Clear vs. Dolby) rest on models whose own documentation admits no quality score has been published, the energy math does not reproduce its own headline number, and "we trained the models ourselves" covers a family where at least five of seventeen repos are fine-tunes or straight format conversions of other people's open checkpoints. The one model that reads as genuinely trained from scratch — a 2MB language identifier — also ships the most complete failure-mode writeup in the release. (2026-09-09) [Developer tools & infra · Notable · 63/100 · Original, source-checked analysis; Runs on a phone]
- [FLASepformer and JAEC: the parts Alibaba shipped, and the parts it didn't](https://ai.thesatyajit.com/articles/flasepformer-jaec.md): Alibaba's Tongyi Lab put two speech front ends on ModelScope this week. The separation model's own card discloses that the released checkpoint is FLA-SepReformer-B on Libri2Mix-100 — 20.7 dB SDRi — while the headline 24.9 dB SDRi belongs to a larger, differently-trained checkpoint on a different, easier benchmark; walking the paper's own tables shows the gap is mostly the benchmark, not the architecture. JAEC's card is more upfront about its own gap: it ships the time-delay estimator and the linear filter, not the neural residual-echo suppressor a full echo canceller would add on top. The two releases also carry different licenses — one blocks the exact commercial use case the card advertises, the other doesn't. (2026-09-09) [Speech & audio · Solid · 58/100 · Original analysis; Runs on a laptop CPU]
- [oruk.ai's fly-brain speech classifier: real wiring, scrambled wiring, same score](https://ai.thesatyajit.com/articles/fly-connectome-reservoir.md): oruk.ai wired 499 real fruit-fly neurons into a speech classifier and got a catchy headline — then ran the one control that undoes it: a version with the same connections shuffled scored just as well (16.88% vs 16.84% mAP, a tie within noise). What's left over is a clean reservoir-computing result — a fixed recurrent network plus a ridge readout beats a constant baseline — and a genuinely rare thing, a team that published the number that kills its own headline. (2026-09-09) [Speech & audio · Niche · 35/100 · A lasting reference; Explained from first principles]
- [Stop looking, just guess: recomputing a GeoGuessr RL run from its raw data](https://ai.thesatyajit.com/articles/geoguessr-rl-environment.md): Adithya Kolavi's Hugging Face article trains a Qwen3.5-4B + LoRA with TRL's GRPOTrainer inside an OpenEnv GeoGuessr environment, finishing second of eleven behind Claude Sonnet 5 by turning a model that looked around for 6.7 turns into one that glances once and guesses in 1.1 — median error down from 1,226 km to 662 km. Rather than restate the numbers, I recomputed them from the article's own published JSON and dataset files: the r = -0.75 turns-vs-score correlation, the GRPO advantage-amplification math down to the multiplier, the +-1 km / no-shared-sequence contamination rule over all 690,400 train-eval pairs, and the '21 missing tasks' between the claimed pool and the two published splits, which turns out to be exactly what their own split script drops for straddling the contamination buffer. I also found a few places the numbers round more generously than the prose admits — a vocabulary figure off by 256 tokens, a 'the docs say equirectangular' claim that Mapillary's current documentation contradicts, and an internal note that touches twice as much training data as the published chapter claims. (2026-09-09) [Training & RL · Essential · 81/100 · Original, source-checked analysis; Interactive explanations]
- [SMELT: looping wins even after you pay for the FLOPs](https://ai.thesatyajit.com/articles/looped-transformers-matched-compute.md): This site's critique of virtual logic depth (arXiv 2506.18233) was that every looped-transformer comparison in the literature is parameter-matched, not compute-matched — a model that reuses a layer stack N times spends roughly N times the FLOPs of the unlooped baseline it gets shown beating. arXiv 2609.01343 (SMELT) closes that gap on MoE Transformers, narrowing hidden width and adding experts so a middle-half, twice-looped model matches an unlooped baseline's per-token FLOPs, total non-embedding parameters, and KV cache all at once, then scales the comparison to 54B non-embedding parameters. Looping still wins under matched compute — saving 6.8-18.0% of training FLOPs on the fitted frontier — real, but far more modest than the free-parameters framing implies. Two more results get checked against the same bar: the full-bandwidth transformer's (arXiv 2608.08888) own token-equivalent-compute accounting, and whether Nanbeige4.2-3B's shipped 28T-token pretrain (arXiv 2607.22083) escapes the confound or just inherits it. (2026-09-09) [LLM architecture · Solid · 54/100 · Original analysis; A lasting reference]
- [Qwen-Drive-1.0: the driving VLM that doesn't touch the VLM](https://ai.thesatyajit.com/articles/qwen-drive-1-0.md): Qwen-Drive-1.0 bolts a 1.04B-parameter flow-matching Planning Expert and a 250M-parameter BEV head onto an unmodified Qwen3.5-4B backbone -- verified in the released code down to the class name and the exact eight attention caches the planner reads. The Hub's headline 4.54B is the backbone alone; attach a planner and you're running closer to 5.6B. Its perception head is only a real 'probe' in one ablation table, not in the numbers it ships with -- and across the four planning benchmarks it reports, reinforcement learning's gains look monotonic right up until the one benchmark that actually closes the loop, where they turn into a trade. (2026-09-09) [Robotics & embodied · Notable · 63/100 · Analysis found nowhere else; Runs on a consumer GPU]
- [AIRA: the bottlenecks it named, and an audit that complicates the tease](https://ai.thesatyajit.com/articles/aira-research-agents.md): AIRA₁ (arXiv 2507.02554, July 2025) formalizes AI research agents as graph search over five operators and finds operator quality — not search sophistication — drives MLE-bench medal rate, while a 9–13 point validation/test gap quietly caps every strategy. AIRA₂ (arXiv 2603.26499, March 2026) names three bottlenecks almost word-for-word out of AIRA₁'s own Limitations section and re-measures each fix with controlled ablations. Its own April revision then audits its highest-profile new result and finds 5 of 11 claimed wins on a second benchmark were answer-key extraction, benchmark-set training, and pretrained-model contamination — worth holding next to the still-unpublished AIRA₃'s tease of 'unhackable, live tasks on the track to real RSI.' (2026-09-08) [Agents & harnesses · Solid · 52/100 · Original analysis; A lasting reference]
- [Audio8: eight checkpoints, one broken link, and a 3B model with nowhere to ship](https://ai.thesatyajit.com/articles/audio8.md): Audio8's own framing for its two-month run of on-device audio releases is that it's more than a pile of checkpoints — model design, a technical report, deployment-aware export, and ONNX/INT8/INT4/iOS-ANE variants tuned to different memory and power budgets. Enumerated through the Hugging Face API across all 12 repos, with real byte sizes and safetensors headers read directly: the 0.1B ASR model gets a full precision ladder and a Swift/Core ML iPhone SDK, but ARK-ASR-3B — the most-downloaded checkpoint in the whole family — ships with zero official quantized or edge variants, and GPA-v1.5's own promised ONNX bundle redirects to an unrelated third-party account. Both ASR-line size names undercount their own true end-to-end parameter count, by a shrinking margin the shared audio encoder explains exactly. (2026-09-08) [Speech & audio · High · 68/100 · Original, source-checked analysis; Runs on a phone]
- [Z1T: sparse transformers for a chip that samples, not multiplies — and what 100x actually measures](https://ai.thesatyajit.com/articles/extropic-z1t.md): Extropic's Z1T pairs a probabilistic sampling chip with an FPGA to run transformer-like models, leading with 'over 100x energy efficiency gains vs GPUs' while a rounder 140x circulated separately — both numbers trace to one cell in a three-row table, the row where the H100 baseline runs at 10% utilization, well below the ~40% Extropic's own citation gives for real LLM serving. A walk through what a 'sample' costs on hardware whose physical primitive is a stochastic pbit with 16 fixed neighbors rather than a dense multiply-accumulate, what survives of a transformer once its primitives are rebuilt around that constraint, why every Z1-side number in the post is a projection anchored to a different, earlier chip rather than a measurement on Z1 silicon, and what the scaling law actually is — and isn't yet. (2026-09-08) [GPUs, kernels & systems · Notable · 61/100 · Checked against the source; A new technique]
- [Ling-3.0-flash-Fin: a finance finetune, and a benchmark that grades where the numbers came from](https://ai.thesatyajit.com/articles/ling-3-0-flash-fin.md): inclusionAI's Ling-3.0-flash-Fin continues training Ling-3.0-flash on financial data under an unchanged architecture — config.json and a full safetensors shape audit across all 65 shards confirm the tensors are byte-identical to the base model. A from-scratch parameter count puts the real total at 127.49B, with the announced '124B' landing exactly on the 124.41B backbone once the ~3.07B multi-token-prediction head is excluded, and '5.1B active' reproducible from config.json alone. It ships alongside FinFIRST, a 123-task benchmark graded through 701 atomic criteria across sourcing, raw data and computation rather than final-answer matching — inspected here record by record, with the model's 82.45% source-verification score checked against the paper's own 15-model table, and the widely repeated 'Intelligence Index 38 to 41' claim checked and found unverifiable. (2026-09-08) [Evals & benchmarks · Solid · 58/100 · Original analysis; Concrete numbers to act on]
- [NextLat: a transformer graded on its own next hidden state](https://ai.thesatyajit.com/articles/next-latent-world-models.md): Next-Latent Prediction adds one auxiliary loss to ordinary next-token training: predict your own next hidden state, from the current one plus the next token, with a stop-gradient — not an EMA target, not a VQ codebook, no separate encoder anywhere — preventing collapse. Checked against the paper's own Table 1: 'compact' means a 3x lower effective-latent-rank than GPT on a Manhattan-taxi world-modeling benchmark (52.7 vs. 160.1), but MTP is a much closer second (57.7) than the headline comparison suggests, and the strongest evidence for an actual world model is a metric most readers will skip past — how often two different routes to the same place produce identical continuations. Plus what changed between the November 2025 submission and the current revision, and a paper id that circulated attached to an unrelated claim. (2026-09-08) [LLM architecture · Notable · 61/100 · Original analysis; Runs on a consumer GPU]
- [SparDA: a fourth projection that lets sparse attention prefetch its own KV cache](https://ai.thesatyajit.com/articles/sparda.md): Sparse attention cuts attention compute, but two problems survive it: the KV cache still grows with sequence length, so offloading it to CPU memory hits a PCIe wall; and the block-selection step that makes attention sparse is itself O(T²) and can dominate at long context. SparDA (NVIDIA, MIT, and co-authors now at Thinking Machines Lab / ByteDance Seed) adds a fourth per-layer projection — Forecast — that predicts next-layer's KV blocks early enough to prefetch them during this layer's compute, and collapses block-selection to one head per GQA group. On MiniCPM4.1-8B and NOSA-8B: 1.25× prefill, 1.7× decode over the offload baseline, and up to 5.3× decode throughput over non-offload sparse — three numbers measured against two different baselines, worth keeping apart. Plus the honest ceiling: the Forecast is distilled from the original selector, so it inherits whatever that selector gets wrong. (2026-09-08) [Inference & serving · Solid · 58/100 · Original analysis; A new technique]
- [tgrep: what a trigram index actually buys you](https://ai.thesatyajit.com/articles/tgrep.md): Microsoft's tgrep pre-builds a trigram index so regex search only touches files that could match, and it now powers grep inside GitHub Copilot CLI. The README says up to 52x faster than ripgrep — in fine print, 'index pre-built.' A full read of tgrep-core's Rust source: the trigram-to-regex query planner and exactly where it degrades to a full scan, the client/server architecture and file watcher, the fuzz suite, and — since neither BENCHMARKS.md nor the README does this math — the actual number of queries it takes for the one-time index build to pay for itself, computed from BENCHMARKS.md's own numbers. (2026-09-08) [Developer tools & infra · Essential · 81/100 · Analysis found nowhere else; Runs on a laptop CPU]
- [Virtual logic depth: looping buys reasoning, not knowledge](https://ai.thesatyajit.com/articles/virtual-logic-depth.md): arXiv 2506.18233 trains roughly fifty small GPT-2s plus one LoRA-tuned LLaMA-3.2-3B to ask whether reusing transformer layers decouples reasoning from knowledge. It measures knowledge as absorbed entropy on a random-token memorization task and reasoning as iGSM math accuracy, and every comparison in it is parameter-matched, never compute-matched -- a fact the paper itself never states. (2026-09-08) [LLM architecture · Solid · 53/100 · Original analysis; A lasting reference]
- [H-BAC compresses a field ViT 54.5×, and a same-size baseline nearly ties it](https://ai.thesatyajit.com/articles/vit-compression-plant-disease.md): A chilli-leaf-disease classifier gets pruned with a Hessian-curvature signal, distilled with attention maps, and quantized to INT8 — three techniques chained into one pipeline, 327 MB down to 6 MB. The paper's own tables let you check whether those three techniques' gains actually add up, and its most honest result is the one comparing the full pipeline against just training a same-size model from scratch. (2026-09-08) [Quantization & compression · Solid · 51/100 · Original analysis; Concrete numbers to act on]
- [Projecting out Whisper's hallucinations: abliteration, in reverse](https://ai.thesatyajit.com/articles/whisper-hallucination-projection.md): A low-rank subspace estimated from non-speech calibration audio, projected out of Whisper's decoder activations at inference with no retraining, takes non-speech hallucination rate from 31.31% to 2.44% averaged across three model scales and three benchmarks. It is the identical move Arditi et al. used to ablate refusal directions — a contrastive subspace found by SVD, subtracted from activations — run to remove a failure mode instead of a safeguard. The paper's own tables carry the honest cost: even the gated variant it recommends for deployment raises LibriSpeech WER by 0.33 to 4.39 points, and its own baseline table shows a stock external VAD beating it on speech quality by a wide margin. (2026-09-08) [Speech & audio · Solid · 56/100 · Widely used; A lasting reference]
- [Qwen3-30B-A3B paints in p5.brush: five judges were worth one opinion](https://ai.thesatyajit.com/articles/paint-with-code.md): Surya Narreddi and Cameron Franz fine-tuned Qwen3-30B-A3B-Thinking to paint by writing p5.brush code, judged by a model against a hand-rated reference pool. The first rubric had nine signals and plateaued at 0.65 producing the same clip-art flower every time. Four of them were the same signal, correlated 0.85–0.95 — worth 1.1 independent opinions between them — and a third of the reward had gone gradient-free by step thirty. Updated with the direct sequel: Hugging Face's open reproduction trains three otherwise-identical checkpoints that differ only in the judge-versus-preference-model split — the controlled ablation the original could only ask for. (2026-08-24) [Training & RL · High · 71/100 · Analysis found nowhere else; A lasting reference]
- [Scaling agentic RL: 365,000 environments behind one contract](https://ai.thesatyajit.com/articles/scaling-agentic-rl.md): Prime Intellect integrated 23 agentic tasksets — software-engineering, terminal, and web-search — behind a single taskset API: ~365,000 tasks with prebuilt per-task sandbox images, grading material withheld until scoring, and gold-patch/no-op-validated re-uploads. A walk through why agentic RL is bottlenecked on verified, reproducible environments; the one-contract design (verifiers v1's taskset/harness/runtime split and the Harbor format); the catalog and its counts; the validation pipeline; and the honest failure modes — reward hacks and PR-test false negatives. Updated with a second case study: Hugging Face's TRL + OpenEnv reproduction of a watercolour-painting RL environment, where the reward is a judge model and a learned preference model instead of a hidden test — OpenEnv's contract set next to Harbor's, and three published checkpoints that isolate exactly what changes when taste replaces ground truth. (2026-07-24) [Training & RL · Solid · 56/100 · Widely used; A lasting reference]
- [DART: the 15.8 FPS is a 4-class number wearing an 80-class AP](https://ai.thesatyajit.com/articles/dart-realtime-detection.md): A training-free framework that turns SAM3's single-prompt segmenter into a real-time multi-class detector — read from the paper's own tables and the repo's actual code. The architecture is genuinely clever (one backbone pass shared across every class, a batched decoder standing in for what used to be N forward passes) and the accuracy is real. The headline speed isn't measured at the headline's own class count, and the paper's own numbers say so once you read past the abstract. (2026-08-30) [Vision & multimodal · High · 72/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [GLM-5.3-Flash-Uncensored-FP8: the numbers behind an abliteration](https://ai.thesatyajit.com/articles/glm-5-3-flash-uncensored.md): orcarouter's abliterated GLM-5.3-Flash claims 320B parameters at native FP8 and refusal rates cut from the 90s into the teens on four jailbreak suites. The README is gated and unreachable, so this is checked against Hugging Face's own tensor metadata instead — where the parameter count, the FP8/BF16 split, and the layer-by-layer architecture all match the base model to the byte, and the five headline benchmark rows turn out to be one ordinary quality win sitting inside four self-reported measurements of how much safety training got removed. (2026-08-30) [Security & safety · Solid · 45/100 · Original analysis; Open code or weights]
- [HauhauCS FastMTP: a signed manifest that actually verifies](https://ai.thesatyajit.com/articles/hauhaucs-qwen-fastmtp.md): An uncensored Qwen3.8-27B GGUF quant with over a million downloads ships something almost no community release does: a signed manifest, a build-provenance file, and a llama.cpp patch. Both Ed25519 signatures verify, every declared hash matches Hugging Face's own file metadata, and the patch turns out to reuse a vocabulary-trimming mechanism this site has already covered in EAGLE-3 and DFlash. (2026-08-30) [Quantization & compression · Solid · 57/100 · Original analysis; Runs on a consumer GPU]
- [LeVJEPA: one encoder, zero collapse-prevention machinery, and what 5.6-20.8x actually measures](https://ai.thesatyajit.com/articles/levjepa.md): LeVJEPA trains a single ViT video encoder with no EMA target network, no stop-gradient, and no predictor — replacing all three with SIGReg, a distributional regularizer with a provable collapse-exclusion guarantee. Checked against the paper and the released ViT-L checkpoint: the 5.6-20.8x pretraining-compute range over V-JEPA 2 is stated up front rather than hiding a cherry-picked cell, the +7.6-point ImageNet-1K win comes paired with a real 3.2-point loss on motion benchmarks, and block-causal attention — which the shipped weights actually run — beats bidirectional outright rather than merely tying it. Plus what the Hugging Face repo's storage byte-count reveals about a mid-release weight swap, and why a 303M-parameter ViT-L ships in fp32. (2026-08-30) [Vision & multimodal · Notable · 62/100 · Original analysis; Runs on a consumer GPU]
- [Hy4 preview: frontier of five, not frontier of seven](https://ai.thesatyajit.com/articles/hy4-preview.md): Tencent's 770B/49B-active flagship, read from config.json, the safetensors headers of all 131 shards, and the model card's own 41-row benchmark appendix — not the marketing copy. The parameter math checks out exactly, IndexCache turns out to delete indexer weights outright rather than just skip computing them, and the card's own 'open-source frontier' claim survives a check that a plain 'frontier' claim wouldn't: Hy4 preview leads 5 of its 12 headline benchmarks against the open-weight field, and 1 of 12 once GPT-5.6-Sol and Claude Opus 5 are back in the picture. Updated: AngelSlim's day-zero GGUF checked against both repos' own blob listings and its recipe files — the 1.5TB-to-~200GiB size claim rounds down by a real but modest 6.8%, the sub-2-bit per-layer bit-width split (29 layers at 1.31-bit STQ1_0, 48 at 2.06-bit IQ2_XXS) is exactly what shipped, and the four-benchmark accuracy table in the promotional post has no primary source anywhere in the release. (2026-08-28) [LLM architecture · Notable · 67/100 · Checked against the source; Widely used]
- [GLM-5.3-Flash-MLX: built for MacBook Pro, verified on an H200](https://ai.thesatyajit.com/articles/glm-5-3-flash-mlx.md): Five OrcaSAQ quantizations of GLM-5.3-Flash for Apple Silicon, checked against the README's own quality tables and field notes: the smallest build — marketed as made for MacBook Pro — is specified in the same document as fitting only the single, priciest 128GB M4/M5 Max tier with a manual memory-limit workaround, was verified only on a single H200 GPU rather than any Mac, and is flagged by the vendor's own notes as unreliable for the long-code-generation work the base model is sold on. (2026-08-28) [Quantization & compression · Solid · 53/100 · Original analysis; Concrete numbers to act on]
- [PhoneLLM Alpha 1: the fine-tune is free, the cost line isn't](https://ai.thesatyajit.com/articles/phonellm-alpha-1.md): A model card read the way we read a paper: safetensors headers confirm the 3.5B-active / 30B-total MoE math to three significant figures, the PhoneBench leaderboard shows a full-parameter fine-tune buying 43.7 accuracy points on an architecture whose latency and cost are provably unchanged, and the card's own worked cost example divides $0.2232 by 88 and prints $0.00025 — ten times too low, contradicted by its own linked spreadsheet and its own leaderboard chart on the same page. (2026-08-28) [Agents & harnesses · Notable · 62/100 · Analysis found nowhere else; Concrete numbers to act on]
- [Speculative decoding on AMD GPUs: verifying the hedge](https://ai.thesatyajit.com/articles/vllm-speculative-decoding-amd.md): AMD and Embedded LLM's vLLM benchmark hedges its own headline instead of leading with a speedup number; checking their appendix tables twice confirms why — EAGLE-3 on Qwen3-8B loses to the non-speculative baseline at every tested proposal length on MATH500 (down to 0.44x), DFlash's peak ratio spans 1.10x to 2.87x across nine target models on identical MI300X hardware, and DSpark is benchmarked with its own paper's headline confidence-scheduled verifier switched off. (2026-08-28) [Inference & serving · Solid · 52/100 · Original analysis; Widely used]
- [GLM-5.3: the same base model, a month of post-training, and a cyber capability nobody planned](https://ai.thesatyajit.com/articles/glm-5-3.md): Z.ai shipped GLM-5.3 on the identical base model as GLM-5.2 — every gain is post-training. It wins 16 of 16 against its predecessor, including a 6.2x jump on Terminal-Bench 3.0, and claims open-weights SOTA for coding, which the table supports against open models and not against the closed frontier. The stranger result is cyber: state of the art at finding vulnerabilities, and less than half the throughput of GPT-5.6 Sol at chaining them. Updated: the weights are open — a config.json diff confirms the same-base-model claim at the byte level, an active-parameter count computed from all 141 safetensors shards, and a real mismatch inside Unsloth's day-zero GGUF docs between two files both called '2-bit.' (2026-08-14) [Training & RL · Solid · 57/100 · Original analysis; Widely used]
- [DiffusionOPSD: a target, not just a reward](https://ai.thesatyajit.com/articles/diffusionopsd.md): Endpoint rewards tell a diffusion model whether the final image is good, not how an intermediate denoising prediction should change. DiffusionOPSD closes that gap by walking a frozen policy's clean-output anchor up and down the reward gradient inside a bounded trust region, then fitting the current policy to the result as a plain regression target. Read from the paper's own source and the reference implementation: the exact update, the ablation that proves the win is the gradient direction and not the perturbation, and what happens to the '19 of 20' claim when you throw out the three evaluators nobody outside ByteDance can run. (2026-08-27) [Image & video generation · Notable · 62/100 · Original analysis; A new technique]
- [VoiceMem: the reply model is a QLoRA that can't touch 93% of its own base](https://ai.thesatyajit.com/articles/voicemem.md): A voice-agent memory paper read from the arXiv PDF out, alongside the safetensors headers of the QLoRA adapter it ships: a schema-routed candidate pool that keeps top-5 retrieval flat at 134ms regardless of K, a rank-32 adapter whose target regex names the routed experts but structurally cannot reach them, a training corpus whose Hugging Face repo contains zero data files, and an abstract whose printed PDF disagrees with arXiv's own indexed copy of itself by 2.3x on the one number most people will quote. (2026-08-27) [Agents & harnesses · Notable · 64/100 · Original, source-checked analysis; A new technique]
- [Breeze TTS 2: a Gemma encoder, a Qwen3 backbone, and 1.18 GiB that never runs](https://ai.thesatyajit.com/articles/breeze-tts-2.md): An open-weight TTS model that emits 200 codec tokens per second of speech and claims a 0.32 real-time factor. Read from config.json and the safetensors headers: the text encoder is Gemma-3-1B, the backbone is Qwen3-1.7B's layer stack verbatim, the codec is Qwen3-TTS's 12.5 Hz tokenizer — none of them named in the model card. Plus a gigabyte of embedding table that no code path can reach, two numbers both called TTFA that differ by 3.3x, and a #1 chart that is a twelve-model subset of a board where the model sits sixth. (2026-08-26) [Speech & audio · High · 69/100 · Original, source-checked analysis; Interactive explanations]
- [FlashAttention-3: the kernel is mostly a schedule](https://ai.thesatyajit.com/articles/flash-attention-3.md): FlashAttention-2 was an algorithm — tile the computation so the score matrix never reaches HBM. FlashAttention-3 is mostly not. It is a scheduling problem solved against one specific chip: split the warpgroups by job so the loader can hand its registers to the maths, stagger two consumer warpgroups so the tensor cores stop waiting on the exponential, and recompile the whole thing when someone asks for FP8. Read from the source, where the register constants land on exactly 65,536. (2026-08-26) [GPUs, kernels & systems · High · 68/100 · Interactive explanations; Original analysis]
- [GLM-5.3-Flash: 45 layers, 11 of them expensive](https://ai.thesatyajit.com/articles/glm-5-3-flash.md): 320B parameters, 18B active, and a stack where only eleven of forty-five layers keep a KV cache. A walk through the hybrid linear-plus-sparse attention, the IndexPool trick that pays down the tax sparse attention never advertises, the manifold-constrained hyper-connections that a rival lab ablated away the same week — and an honest scoring of Z.ai's own benchmark table, which undersells the model in one direction and oversells it in another. (2026-08-26) [LLM architecture · Notable · 66/100 · Original analysis; Widely used]
- [Needle Environments: the 0.9 is a gate, not a score](https://ai.thesatyajit.com/articles/needle-environments.md): Six hand-curated tool schemas for a 14 MB model, 192 test cases, and a claim of 90%+ on held-out production tasks. The only 0.9 in the repository is an acceptance threshold, nothing is held out, and a model that refuses everything fails zero critical cases. What is actually in there is better than the claim: eight schema-design rules for constrained decoding, each of them the residue of something that went wrong — including why the shipped smart home has a study and not an office. (2026-08-26) [Evals & benchmarks · High · 72/100 · Original, source-checked analysis; A guide you can follow today]
- [Whittle MoE 27B: the routers moved 4.5 degrees](https://ai.thesatyajit.com/articles/qwen3-8-whittle-moe.md): A dense Qwen3.8-27B was cut into 64 experts per layer and, the card says, brought back from gibberish by training the routers alone with every neuron frozen. Reading the checkpoint instead: 66.3% of the model is active per token, so no setting of k makes it sparse; the adapter that did the healing trains 337.9M parameters, of which the routers are 6.2%; and measured against the k-means matrix it was initialised with, the shipped router sits 4.50° away with its unit norm intact. The evaluation underneath the headline is better than the headline — a public, seeded, standard-library gate that reports its own failures — and on it, the same author's 16.8B pruned model beats this one. (2026-08-26) [Quantization & compression · Notable · 65/100 · Original, source-checked analysis; Runs on a consumer GPU]
- [Sana: the autoencoder does more of the work than the linear attention](https://ai.thesatyajit.com/articles/sana.md): NVIDIA's Sana is sold as a linear-attention diffusion transformer that is 100× faster than FLUX. Reading the repo at commit 5498e5b — the LiteLA kernel, the DC-AE-F32C32 configs, the Gemma-2-2B text encoder and the Sprint distillation script — the linear attention turns out to be the smaller of its two levers at every resolution the model actually ships, and the deep-compression autoencoder the larger. I reconstruct the paper's own FLOP table from the configs to within 4.2%, put the crossover at about 5000px, find the 100× is a 4096px number (it is 43× at 1024px), find the shipped 4-step Sana-Sprint schedule is not the one the paper searched for, and check the licence, which is genuinely Apache-2.0 except for the text encoder. (2026-08-26) [Image & video generation · Essential · 84/100 · Original, source-checked analysis; Interactive explanations]
- [SGLang: the tree, and the language nobody remembers](https://ai.thesatyajit.com/articles/sglang.md): 1.3 million lines, 219 model architectures, and a KV cache that is a radix tree rather than a hash map. Read from the source at e27a7fa: how lock_ref makes a prefix structurally un-evictable, how a draft tree gets verified in one forward pass, why the paper's cache-aware scheduler is no longer the default, and why jump-forward decoding — the compressed-FSM trick the paper is half-famous for — is still implemented in four backends and called by none of them. (2026-08-26) [Inference & serving · Essential · 79/100 · Original, source-checked analysis; Interactive explanations]
- [Tiel-Coder-35B-A3B: the 4-bit tier is 5.16 bits, and the only new weight is a prompt](https://ai.thesatyajit.com/articles/tiel-coder-35b-a3b.md): A GGUF-only release of a 35B-A3B coder, read from the headers out: nine tiers reconciled to the byte, an A3B arithmetic that survives contact with the tensor shapes, a dynamic recipe whose fingerprint is the same four blocks in every tier — and a differentiating artefact that turns out to be 29,157 characters of Jinja rather than a single changed weight. (2026-08-26) [Quantization & compression · High · 68/100 · Original, source-checked analysis; A guide you can follow today]
- [vLLM: what PagedAttention turned into](https://ai.thesatyajit.com/articles/vllm.md): 858,000 lines of Python, 280 model architectures, 23 attention backends of which six do not compute attention, and — new this year — a 110,000-line Rust workspace quietly replacing the serving frontend. vLLM is still described as 'PagedAttention and continuous batching', which stopped being true some time ago. A read of the v1 engine from the source: the block pool, why a prefix-cache hash has to chain its parent, the token budget that replaced the prefill/decode distinction, the swap path that got deleted, and the environment variable that exists because Python hashes strings differently in every process. (2026-08-26) [Inference & serving · Essential · 83/100 · Original, source-checked analysis; Interactive explanations]
- [FreeToken: a 753B model on one GPU, and the two bandwidths that decide everything](https://ai.thesatyajit.com/articles/freetoken.md): An edge-native MoE engine that refuses to pick an offloading strategy and measures one instead. q★ = m·B_P/B_H divides each decode step's cache misses between PCIe and the CPU using bandwidths profiled on the machine it is actually running on — and the right answer swings from 25% to 91% across ordinary consumer hardware. Two RTX 5090s with different hosts want opposite designs. (2026-08-23) [Inference & serving · Notable · 61/100 · Runs on a consumer GPU; Widely used]
- [Colibri: running a 744B model on a 25 GB machine by streaming experts from disk](https://ai.thesatyajit.com/articles/colibri.md): Colibri is a pure-C, zero-dependency engine that runs GLM-5.2 — a 744B-parameter MoE — on a consumer box with ~25 GB of RAM, by keeping only the int4 dense core resident and streaming the routed experts from disk on demand. It works because the model is extremely sparse and heavily quantized, and '25 GB of RAM' only holds if you also have ~370 GB of fast NVMe. Updated for August 2026: the ceiling has moved from 0.37 to 6.8 tok/s on six GPUs, there is now dual-SSD striping, and the project has adopted a hypothesis-and-negative-results posture that produced a sharp correction to what I wrote about speculative decoding. (2026-07-10) [Inference & serving · Solid · 56/100 · Runs on a laptop CPU; A lasting reference]
- [Apodex 1.1 and FrontierAgent: the harness, measured](https://ai.thesatyajit.com/articles/apodex-frontier-agent.md): Apodex published the same checkpoint's scores under two different harnesses, on six benchmarks. That makes the gap between the rows a rare thing — a measurement of scaffolding with the model held fixed — and across the six it accounts for around 40% of the gain from 1.0 to 1.1. On BioMysteryBench the harness moves the score twice as far as the new weights do. A look at the runtime that produces that number, and an honest read of where the model actually places. (2026-08-25) [Agents & harnesses · Solid · 53/100 · Original analysis; Concrete numbers to act on]
- [GLiNER2.5: deleting the width axis](https://ai.thesatyajit.com/articles/gliner-2-5.md): Every GLiNER before this one found entities by enumerating candidate spans — every start position paired with every allowed width — which is why they all shipped with a max_width and why a forty-word clause was not merely hard to extract but structurally invisible. GLiNER2.5 scores boundaries instead. A walk through what that one change unlocks, and an honest read of the appendix, where two checkpoints from the same release turn out to have very different stories. (2026-08-25) [LLM architecture · High · 72/100 · Runs on a laptop CPU; Original analysis]
- [Pipette: on-device speed is not a number](https://ai.thesatyajit.com/articles/pipette.md): Liquid AI and Artificial Analysis published a thousand-plus lab-verified configurations of model × quantization × runtime × device × context, and the most useful thing in it is a refusal. Two 350M models differ fourfold in how they hold up as the prompt grows. Two 8B models with identical deployment profiles reverse on task quality. A walk through what the dataset shows, why the mechanisms behind it are checkable, and the limitations Liquid put in writing. (2026-08-25) [Inference & serving · High · 70/100 · Original analysis; Runs on a phone]
- [Speculative programmatic tool calling](https://ai.thesatyajit.com/articles/spec-ptc.md): If your agent's action space is code, a turn contains a long generation followed by several slow sub-calls — and almost every harness waits for the first to finish before starting the second. Alex Zhang's sPTC launches tool calls from a half-written REPL cell by forking the interpreter and running the partial program in a shadow namespace. A walk through the mechanism, the four cases that decide what can leave early, and why the speed-up is so hard to measure. (2026-08-25) [Agents & harnesses · Solid · 56/100 · A lasting reference; A new technique]
- [Scaling laws in 2026: the line held, the recipe didn't](https://ai.thesatyajit.com/articles/scaling-laws-2026.md): The standard retelling of scaling laws ends in 2022 with 'twenty tokens per parameter' and treats it as the industry's blueprint. Models shipping now run at 200, at 60,000, at 79,000 — and that is not drift, it is a different objective. A walk from what loss actually is, through Kaplan's shallow exponents and Chinchilla's reallocation, to the two results that added a third axis to the budget: serving cost, and the samples you draw at inference. (2026-08-24) [Training & RL · Solid · 53/100 · Evergreen reference; Widely used]
- [Kimi K3: a 2.8T open model that turns compute into intelligence 2.5× better](https://ai.thesatyajit.com/articles/kimi-k3.md): Moonshot's Kimi K3 is a 2.8T-parameter, 104B-active open MoE with a 1M-token context — now with weights, code and a 47-page technical report on the table. A first-principles walk through what actually makes it new: Kimi Delta Attention, Attention Residuals, Stable LatentMoE routing 16 of 896 experts with quantile balancing, NoPE, the nine-expert post-training funnel and the reward cliffs behind it, the measured 2.5× scaling-efficiency gain over K2, and where it lands against the frontier. (2026-07-17) [LLM architecture · Notable · 67/100 · Genuinely new idea; Facts measured from source]
- [Flex-π: a robot policy that decides how much to think at deployment](https://ai.thesatyajit.com/articles/flex-pi.md): A 6B world-action model that denoises RGB, 3D pointmaps, object semantics and actions in one shared latent space — so what it reads and what it predicts become runtime flags. Fifty-six configurations from one checkpoint, VLA latency at 60 ms or full future generation at 193 ms. It repairs its own gripper across eight ordered stages in 11 of 20 rollouts; the best baseline manages it once. (2026-08-23) [Robotics & embodied · Solid · 48/100 · Runs on a consumer GPU; A new technique]
- [LFM2.5-DSpark: the best draft model in the release is the slowest one](https://ai.thesatyajit.com/articles/lfm25-dspark.md): Liquid AI ships speculative decoding for three LFM2.5 models. The 8B MoE draft accepts 6.95 tokens per pass — the highest of the three — and returns 1.18× on a MacBook, while the 1.2B dense draft accepts 5.02 and returns 2.54×. That inversion is the most useful thing in the release, and the arithmetic behind it says something structural about speculation on sparse models. (2026-08-23) [Inference & serving · Solid · 56/100 · Original analysis; Runs on a consumer GPU]
- [Matryoshka LM Suites: stop training the small models twice](https://ai.thesatyajit.com/articles/matryoshka-lm-suites.md): Every model suite ships 500M, 1.5B and 3B checkpoints trained as three separate runs. Godey and Artzi nest them into one architecture instead — 3.2B trained parameters instead of 5.2B, 36% less compute, free online distillation at every step, and a speculative decoding pair where the draft is literally the first 24 layers of its own verifier. (2026-08-23) [LLM architecture · Solid · 52/100 · Runs on a consumer GPU; A lasting reference]
- [MimiModel: a 45M LLM on a $5 chip, and 20 points that lived in the decode loop](https://ai.thesatyajit.com/articles/mimimodel.md): A single-file C engine runs Cactus's Needle 2 on an ESP32-S3 with 13.7 MB of weights that are never loaded into RAM — they stay memory-mapped in flash and the matmul reads the packed 2-bit bytes in place. The more useful result is the accuracy report: byte-identical weights, and 20.8 points of function-calling accuracy attributed row by row to three decisions in the decode loop. (2026-08-23) [Inference & serving · Notable · 66/100 · Original analysis; Runs on a phone]
- [Nar TTS: the two rewards it built and refuses to switch on](https://ai.thesatyajit.com/articles/nar-tts.md): A TTS stack that bolts Mimi speech tokens onto whatever causal LM you point it at, with a GRPO recipe, an inference-time quality gate, and an expressive control markup. Its emotion and event rewards are implemented and pinned at weight zero, with the reason written down — using the same classifier as reward and metric encourages reward hacking. That single decision tells you more about the project than the benchmarks would. (2026-08-23) [Speech & audio · Niche · 39/100 · Original analysis; A lasting reference]
- [Cactus Needle 2: the interesting number is what happens after you fine-tune it](https://ai.thesatyajit.com/articles/needle-finetune.md): A 45M-parameter tool-calling model in a 14MB binary that runs in 28MB of RAM, trained at 2 bits from pretraining onward. The base checkpoint trades wins with models five to seventy times its size. Then fine-tuning on your own tool schemas lifts it 21 to 58 points and puts it ahead of a frontier cloud model on three of four benchmarks — because your product has twelve tools, not twelve thousand. (2026-08-23) [LLM architecture · Solid · 48/100 · Runs on a laptop CPU; A new technique]
- [SenseNova-U1.5: no vision encoder, no VAE, and pixels that come back anyway](https://ai.thesatyajit.com/articles/sensenova-u15.md): A natively unified multimodal model built on NEO-unify, which removes both pre-trained bottlenecks from the usual stack — images enter as patch embeddings and leave as patch embeddings through one Mixture-of-Transformer backbone. A model with no autoencoder reconstructs COCO images within 1.09 dB of Flux's VAE, and edits well with its understanding branch entirely frozen. (2026-08-23) [Image & video generation · Solid · 48/100 · A new technique; Concrete numbers to act on]
- [Taimi-14B-Med: reading a model card properly](https://ai.thesatyajit.com/articles/taimi-14b-med.md): A medical LLM for hospital deployment and service robots, scoring 77.4% on a Chinese physician licensing exam — above the paper's human baseline. Its card also states, twice, that v0.1.0 is the base model with a rebranded configuration and the medical weights come later. Both of those things are true, and the gap between them is the most instructive thing on the page. (2026-08-23) [Evals & benchmarks · Niche · 35/100 · Runs on a consumer GPU; Concrete numbers to act on]
- [Ultra-FineWeb: what an education filter costs you, measured](https://ai.thesatyajit.com/articles/ultra-fineweb.md): OpenBMB's filtered web corpus — 1T English and 120B Chinese tokens — trained head to head against FineWeb and FineWeb-edu on identical protocol. FineWeb-edu buys ten points on ARC and gives ground back on five of nine English benchmarks. Ultra-FineWeb keeps the gains and regresses on exactly one, by 0.15 points. And the release ships the pipeline, not just the output. (2026-08-23) [Data & datasets · Solid · 46/100 · Widely used; A lasting reference]
- [How small can a verifier be? Six hundred thousand parameters](https://ai.thesatyajit.com/articles/verifier-frontier.md): A 0.63M-parameter model pretrained in two minutes verifies Countdown solutions at 0.85 — within a few points of a 7B. A 2M model beats zero-shot Gemini 2.5 Flash at faithfulness judging. Verification is flat, then on: below a task-specific size it sits at chance, above it the curve barely moves for three orders of magnitude. And the plateau is flat because every model fails the same examples. (2026-08-23) [Training & RL · Solid · 47/100 · A lasting reference; A new technique]
- [Qwen3.8, weights in hand: 98% of a 2.4T model is routed experts](https://ai.thesatyajit.com/articles/qwen3-8-open-weights.md): The Qwen3.8 collection is four repositories, two licenses and one architecture. Reading the files rather than the model card: the 95B active figure only closes if you count the untied embeddings, the FP8 checkpoint quantizes nothing but routed experts, three independent parties agree the Gated DeltaNet path is what you must not touch, reasoning_effort turns out to be two English sentences in a Jinja template, and the dense 27B beats a larger model on every agentic benchmark while losing the ones that test recall. (2026-08-15) [LLM architecture · Notable · 66/100 · A guide you can follow today; Original analysis]
- [Arcee's Open Models API: a model lab selling six models it did not build](https://ai.thesatyajit.com/articles/arcee-open-models-api.md): A one-minute product post with one genuinely interesting sentence in it. Arcee opened its API to DeepSeek, GLM, Kimi and Thinking Machines' Inkling alongside its own Trinity — explicitly so it can watch which model users pick for which task, and feed that back into Trinity. Plus the number the price list does not draw attention to: output multipliers ranging from 2x to 5x, which re-ranks the catalog once you account for what agents actually spend tokens on. (2026-08-14) [Inference & serving · Niche · 26/100 · Concrete numbers to act on]
- [DeepSeek Harness: an agent harness that refuses to send what it didn't log](https://ai.thesatyajit.com/articles/deepseek-harness.md): DeepSeek open-sourced dsh, an agent harness built on Cordis where everything — including the agent loop — is a replaceable plugin across 219 packages. The interesting engineering isn't the loop. It's a runtime check that compares every outgoing model request against a fresh replay of the session log and throws if they differ by a byte, plus the repo-wide discipline around it: 219 mandatory invariant companions, 27 verify scripts, and 686 design notes for a codebase that took 12,293 commits in 65 days. (2026-08-14) [Agents & harnesses · High · 72/100 · Analysis found nowhere else; Interactive explanations]
- [dots3-note Preview: 16B active parameters, and a critic that thinks before it scores](https://ai.thesatyajit.com/articles/dots3.md): Xiaohongshu open-sourced a 280B/16B multimodal MoE with a 512K context under Apache 2.0. The release's real contribution is TEMPO — an RL method for tasks whose rollouts run past ten hours, where the same model switches from actor to critic mid-task and spends real inference deciding how it is doing. It also scored a certified 42/42 at IMO 2026, from a branch of this model. (2026-08-14) [Training & RL · Niche · 44/100 · A new technique; Open code or weights]
- [The full-bandwidth transformer: the feedback channel is one token wide](https://ai.thesatyajit.com/articles/full-bandwidth-transformer.md): Between two decoding steps a transformer passes exactly one sampled token — log2|V| bits — and throws away the top-layer hidden state that produced it. This paper feeds that state back through a gated linear unit alongside the token embedding, keeping the architecture, KV cache and objective intact. At 1B parameters it matches models trained on 1.5x more tokens, and produces shorter reasoning traces — a gain that instruction tuning then destroys, for a reason worth understanding. (2026-08-14) [LLM architecture · Niche · 40/100 · A new technique; Explained from first principles]
- [LFM2.5-VL-3B: the release where GUI grounding appears out of nothing](https://ai.thesatyajit.com/articles/lfm2-5-vl-3b.md): Liquid AI's 3.1B vision-language model averages 69.4 across 28 benchmarks — exactly level with InternVL 3.5 4B and 0.7 behind Qwen3.5-4B, both of which are 4.7B. The headline result is buried in one row: ScreenSpot-v2 goes from 5.4 to 80.7 against the previous release, because LFM2-VL-3B could not ground UI elements at all. Plus a non-reasoning design that buys 34 ms to first token, and on-device numbers reported without the conditions to check them. (2026-08-14) [Vision & multimodal · Solid · 53/100 · Original analysis; Runs on a phone]
- [MAGI-2 Preview: 114B parameters, 6B awake, and two sparsities doing the work](https://ai.thesatyajit.com/articles/magi-2-preview.md): Sand AI's unified audio-video model activates 1 parameter in 19. Reading the published safetensors headers, the 114B total is exact and the '6B active' figure lands at 5.96B — but only once you account for the fact that the model carries three sets of modality weights and a token is only ever one modality. MoE sparsity alone gets you to 7.71B. Plus hyper-connections with four residual streams, and 64 GB of the 307 GB checkpoint that Sand AI did not train. (2026-08-14) [Image & video generation · Solid · 52/100 · Original analysis; Open code or weights]
- [MiniMax Music 3: a five-minute song is 9,000 steps of a 2.5 kbit/s code](https://ai.thesatyajit.com/articles/minimax-music3.md): MiniMax open-sourced a music model that writes complete songs — an 8B global LLM over the first RVQ codebook, a 646M local decoder for the other seven, then flow matching to a Flow-VAE. Every parameter count on the card reconciles against the bytes on disk, and the 8.584B global model is exactly Qwen3-8B with the vocabulary widened to 200,000. The repository is 57 GB because it ships the whole thing twice. (2026-08-14) [Speech & audio · Solid · 57/100 · Original analysis; Runs on a consumer GPU]
- [nac: an orchestrator that is not allowed to touch anything](https://ai.thesatyajit.com/articles/nac.md): Arcee's open-source Rust harness separates the temporary context needed to perform an action from the persistent state needed to continue a workstream. The orchestrator plans and dispatches but cannot run a command or edit a file; workers do the work and are then deleted, leaving only an episode. There is no compaction because there is nothing to compact — and the one seam they can't close, they print. (2026-08-14) [Agents & harnesses · Solid · 55/100 · A lasting reference; A new technique]
- [Breaking the softmax bottleneck: your output layer is a rank-d wall](https://ai.thesatyajit.com/articles/softmax-bottleneck.md): A 2017 paper proved something uncomfortable and easy to forget: a language model's logits are a product of two skinny matrices, so the distribution it can express has rank at most the hidden size — no matter how good the network underneath is. The fix, a mixture of softmaxes, took PTB and WikiText-2 perplexity to 47.69 and 40.68 and then never shipped, because it costs 2-3x. Nine years on the geometry is tighter than it was, and nobody re-ran the measurement. (2026-08-14) [LLM architecture · High · 68/100 · Evergreen reference; Original analysis]
- [speech-to-speech: the OpenAI Realtime API, reimplemented as four swappable parts](https://ai.thesatyajit.com/articles/speech-to-speech.md): Hugging Face's voice-agent pipeline speaks the core OpenAI Realtime event set over WebSocket and WebRTC, so an existing Realtime client swaps endpoints and keeps working — while the VAD, STT, LLM and TTS behind it become 90 combinations you choose. The clever part is Smart Turn: it starts transcription and generation before it knows the user finished talking, and uses revision numbering to make the wrong guesses invisible. (2026-08-14) [Speech & audio · Solid · 47/100 · Runs on a consumer GPU; Concrete numbers to act on]
- [Toast 1: what happens when you stop making the frontier model do the searching](https://ai.thesatyajit.com/articles/toast-1.md): Mixedbread's specialised search agent takes over the retrieval loop from a frontier model. On Harvey's legal benchmark the interesting number is the one that does not move: three retrieval stacks, 80.6M down to 23.0M tokens, and an identical task score of 55 each time — which says 57.6M of those tokens were the cost of looking, not of answering. (2026-08-14) [Agents & harnesses · Niche · 36/100]
- [WorldClaw: a 3D world generator that is really a Blender programmer](https://ai.thesatyajit.com/articles/worldclaw.md): Tencent Hunyuan's agentic open-world 3D system turns a text prompt into a structured spec, then builds the world by writing executable Blender programs — terrain as noise-and-landform code, objects as separately reconstructed meshes placed by solved transforms. The paper is candid that this only works with Claude Opus 4.8, GPT-Image-2 and Hunyuan3D in the loop, and it publishes no quantitative evaluation at all. (2026-08-14) [3D & spatial · Niche · 39/100 · A new technique; Explained from first principles]
- [Liquid time constants and gated delta rules: two literatures, one recurrence](https://ai.thesatyajit.com/articles/ltc-gated-delta.md): Liquid Time-constant Networks (2020) and Gated DeltaNet (2024) come from different fields, cite different ancestors, and use different notation. Discretize LTC's ODE with its own fused solver and the two collapse into the same object: a state that decays by an input-dependent factor per step. A full derivation of that correspondence, what the delta rule adds that pure gating cannot, and a close read of LTCAttention — a new implementation that puts a liquid time constant into an attention score instead of a recurrent state, with a three-seed experiment whose headline result may not survive a compute-matched comparison. (2026-08-10) [LLM architecture · High · 71/100 · Analysis found nowhere else; Interactive explanations]
- [Muse Glimmer: an agentic model designed backwards from a 24 GB budget](https://ai.thesatyajit.com/articles/muse-glimmer.md): Meta's 30B open-weight agent model is a distillation of Muse Spark built to run on one consumer GPU. Its architecture reads like an answer to a memory constraint rather than a capability target — 3:1 sliding-to-global attention, 16:1 GQA, NoPE on every global layer — and the 24 GB claim verifies exactly: 21.6 GB at the full 131K context, with the sliding-window pattern doing the load-bearing work. Plus a benchmark table Meta loses a third of, and a safety table where losing is the argument. (2026-08-10) [LLM architecture · Notable · 62/100 · Original analysis; Runs on a consumer GPU]
- [The Skaling law: Chinchilla assumes model size and data don't interact, and they do](https://ai.thesatyajit.com/articles/skaling-law.md): FAIR's Skaling law adds one exponent to the Chinchilla form — L = (A/Nᵅ + B/Dᵝ)^k + E — and cuts prediction error 1.5–3×. The additive law it replaces enforces a cross-derivative of exactly zero by construction, which produces a saddle-shaped bias at the corners of any grid and, more consequentially, gets the sign of the compute-optimal allocation trend wrong. Also: what the paper does to a scaling estimate I published four days ago. (2026-08-10) [Training & RL · Notable · 61/100 · Original analysis; Widely used]
- [BTL-4: reading a model card against its own weights](https://ai.thesatyajit.com/articles/btl-4.md): A 35B agentic model appeared on Hugging Face with a SWE-bench Verified number near the frontier, no technical report, no third-party evaluation, and zero downloads. Rather than take it or dismiss it, I checked it against the only evidence available — config.json, the safetensors index, and HTTP range requests into the shards. The base-model claim verifies exactly. The LiveCodeBench table is arithmetically impossible. And the weights say LoRA, which the card never does. (2026-08-06) [Evals & benchmarks · High · 75/100 · Original, source-checked analysis; A guide you can follow today]
- [Intern-S2-Mobius: a 35B model that separates memory from reasoning](https://ai.thesatyajit.com/articles/intern-s2-mobius.md): Intern-S2-Mobius is a 35B model continual-pretrained from Qwen3.5-35B on a new architecture, Mobius-v0: forty decoder layers cycle through just four shared mixture-of-experts memory banks instead of each owning one, cutting reasoning traces up to 5.0x and average inference throughput up to 4.6x against a matched Transformer baseline. It ships on Hugging Face and ModelScope under Apache-2.0 with real, ungated bf16 weights — but the only reported comparison is against its own base model, not the frontier. (2026-08-06) [LLM architecture · Solid · 57/100 · Original analysis; A new technique]
- [Maple-Preview: a 20B reasoning model where 97% of the weights are −1, 0, or +1](https://ai.thesatyajit.com/articles/maple-preview.md): DeepGrove's Maple-Preview claims ternary weights, a 5.31 GB checkpoint and 218 tok/s on a Mac mini M4. The ternary claim is true and you can verify it from the published bytes — every row of every projection matrix holds exactly three distinct values with a per-row scale. The 5.31 GB claim reconciles too, at 1.65 bits per weight. What you actually download from Hugging Face is 40.43 GB of bf16, and the shipped code does no quantization at all. (2026-08-06) [Quantization & compression · High · 72/100 · Original, source-checked analysis; A new technique]
- [Pokee-Isaac 28B: 10M tokens on one GPU, and an architecture the report never explains](https://ai.thesatyajit.com/articles/pokee-isaac-28b.md): Pokee AI's technical report claims a 10-million-token, non-decoder-only agentic model that runs on a single GPU and matches cost-optimized cloud systems — but never once explains what 'non-decoder-only' means, and its own tables show Isaac trailing two of those cloud systems on the two benchmarks that use real, unscripted tools. A close read of the numbers, and a serious look at the deployment argument underneath them. (2026-08-06) [LLM architecture · Solid · 46/100 · Original analysis]
- [Prime Agent: the interface is a Python REPL, not a tool-call schema](https://ai.thesatyajit.com/articles/prime-agent.md): Prime Intellect's open-source coding and research agent exposes exactly one model tool — a persistent IPython kernel — and treats subagents, files, shell, MCP servers, and context as things a program manipulates instead of a schema a harness parses. The schema didn't vanish, though: 21 typed host requests sit behind the kernel, most of them conditionally registered, which is capability gating in code rather than in a prompt. Plus the Continual Harness, its hard-blocked base system prompt, and what a full clone of the repository says about its real age that a shallow one cannot. (2026-08-06) [Agents & harnesses · Notable · 62/100 · Original analysis; A lasting reference]
- [Recursive Language Models: context as a variable, recursion as a function call](https://ai.thesatyajit.com/articles/recursive-language-models.md): Recursive Language Models turn a long prompt into a value a Python program can slice instead of text a model has to reread, and turn a subagent call into an ordinary recursive function instead of a special harness feature. The idea is Alex Zhang's (MIT CSAIL, October 2025, formalized with Tim Kraska and Omar Khattab as arXiv 2512.24601); Prime Intellect has since implemented it twice, once as a faithful research environment and once, more broadly, as the whole interface of Prime Agent. (2026-08-06) [Agents & harnesses · Solid · 55/100 · Widely used; A lasting reference]
- [TencentDB Agent Memory: the four-tier pyramid is really a cache-stability hierarchy](https://ai.thesatyajit.com/articles/tencentdb-agent-memory.md): Tencent's open-source agent memory system layers conversations into L0–L3 and looks, at first, like every other summarization hierarchy. Reading the source, the layers are sorted by volatility rather than abstraction: only the tiers that rarely change are allowed into the cached system prompt, and everything that moves is demoted to a tool call. That is a token-economics decision wearing a knowledge-management costume — plus a transparent OpenAI/Anthropic proxy, real RRF retrieval, an ACL with a documented privacy fix, and one benchmark number with nothing behind it. (2026-08-06) [Agents & harnesses · Notable · 61/100 · Original analysis; A lasting reference]
- [ABot-World-0: a 5B world model that wins on efficiency, not the leaderboard](https://ai.thesatyajit.com/articles/abot-world.md): Alibaba AMAP's ABot-World-0 (5B, Apache-2.0) streams a controllable, action-conditioned interactive world at 720p/16fps from a single desktop GPU. On the third-party WorldRoamBench it beats two larger open comparators, LingBot-World (14B) and HY-World 1.5 (8.3B), on every metric, but trails HappyOyster on 6 of 7 -- the honest story is efficiency, not a sweep. (2026-08-03) [Image & video generation · Niche · 44/100 · Runs on a consumer GPU; A new technique]
- [AngelSpec: specialize the drafter, share the verification budget](https://ai.thesatyajit.com/articles/angelspec.md): Tencent's production speculative-decoding system trains a different drafter per workload — MTP for chat, a block-diffusion drafter (DFly) for code and math — then treats the target model's verification depth as a shared batch resource (D-cut), reallocating it by confidence instead of by request. On live Hunyuan traffic D-cut adds up to 15.7% throughput at concurrency 64 for a 2.8% acceptance-length cost, measured on H20, not flagship H100. (2026-08-03) [Inference & serving · Niche · 37/100 · Explained from first principles]
- [AutoCompact: teaching an agent to decide when to forget](https://ai.thesatyajit.com/articles/autocompact.md): AutoCompact trains a coding agent to call its own compact() at task-phase boundaries instead of a fixed token threshold — judge-corrected SFT, then GRPO RL, on 1,052 examples. There is no paper, no code, and no arXiv listing: this is a project-page blog post with chart-shaped claims. The mechanism is worth understanding; the evidence is not yet checkable. (2026-08-03) [Agents & harnesses · Niche · 19/100]
- [A.X-K2: a sparse-attention upgrade that costs nothing, trained natively in FP8](https://ai.thesatyajit.com/articles/ax-k2.md): SK Telecom's A.X-K2 is a 688B-A33B MoE that bolts a DeepSeek-style top-k indexer onto gated MLA — Sparse Gated Attention — and publishes the ablation: LongBench moves from 62.80 to 62.99, not down. It is also trained natively in FP8 from step one, never rounded after the fact. The honest gap: BrowseComp at 9.3, worst of eight compared models. (2026-08-03) [LLM architecture · Niche · 43/100 · Open code or weights; Explained from first principles]
- [Chimera: unbundling RoPE into a diffusion Transformer that extrapolates 6× on video](https://ai.thesatyajit.com/articles/chimera-diffusion.md): Adobe's Chimera swaps full attention in a single-stream text/image/video diffusion Transformer for linear Kimi Delta Attention plus periodic Multi-head Latent Attention, drops positional embeddings entirely, and backs it with HeteroP, a per-tensor hyperparameter-transfer scheme that makes Chinchilla-style scaling laws for visual diffusion fit cleanly. Its mechanistic RoPE audit shows why that works: reassigning RoPE's three bundled inductive biases to dedicated modules buys genuine zero-shot 6× video-length extrapolation (6.5% FID degradation vs 50%+ for Wan2.1 and HunyuanVideo) — and a compute-efficiency number the abstract states as 7.3× but the paper's own Section 5.5 arithmetic gives as 6.8×. (2026-08-03) [Image & video generation · Solid · 45/100 · A lasting reference; A new technique]
- [DCFormer: an ICML oral that let attention heads borrow each other's circuits, and quietly shipped anyway](https://ai.thesatyajit.com/articles/dcformer.md): DCMHA replaces multi-head attention's fixed, independent heads with a five-branch Compose function that recombines heads per token — DCPythia-6.9B beats open Pythia-12B on Pile perplexity (5.95 vs 6.01) at roughly half the parameters. An ICML 2024 oral with under a dozen citations two years on, but real production adoption at Caiyun AI. (2026-08-03) [LLM architecture · Solid · 55/100 · Original analysis; A lasting reference]
- [Dream-Cubed: diffusion directly on Minecraft's block IDs, where inpainting comes free](https://ai.thesatyajit.com/articles/dream-cubed.md): A 280M 3D diffusion transformer trained on 2 million Minecraft chunks in native block-ID space, with no pixel or latent step in between. Because masked discrete diffusion guarantees unmasked blocks stay fixed, exact inpainting, outpainting, and user-block-conditioning fall out with zero extra machinery — something a continuous DDPM trained identically cannot do. A brand-new, uncited preprint with self-admittedly weak evaluation (FID, a 19-person study); worth it for the mechanism and the visuals, not as a settled result. (2026-08-03) [Image & video generation · Niche · 44/100 · Runs on a consumer GPU; A new technique]
- [Explorative Modeling: factor the training loop, not the generation loop](https://ai.thesatyajit.com/articles/explorative-modeling.md): Diffusion and autoregression both factor generation into many steps to handle multimodal targets. Explorative Modeling factors the training loop instead — generate K candidates, backprop only through the best — and treats the result as a third pretraining axis, substitutable for step-factored generation. The payoff: 6.2x sample efficiency on ImageNet, and single-pass end-to-end policies that match Diffusion Policy and Diffuser at 100-256x less inference compute. (2026-08-03) [Training & RL · Niche · 44/100 · A new technique; Explained from first principles]
- [Fara 1.5: an open, vision-only browser agent at 4B, 9B, and 27B](https://ai.thesatyajit.com/articles/fara-1-5.md): Microsoft's Fara1.5 family (4B/9B/27B, MIT license, built on Qwen3.5) controls a browser from screenshots alone, no DOM, no accessibility tree, and at 27B parameters it beats OpenAI Operator, Gemini 2.5 Computer Use, and Yutori Navigator n1 on Online-Mind2Web. The more interesting number is how little of that ladder the 4B model needs to be merely competitive. (2026-08-03) [Agents & harnesses · Niche · 37/100 · Runs on a consumer GPU; Widely used]
- [Language models are injective, so a KV-cache is not a summary — it's the prompt, in another basis](https://ai.thesatyajit.com/articles/injective-language-models.md): A proof that decoder-only transformer LMs are almost-surely injective — distinct prompts never collide to the same hidden state — confirmed across billions of pairwise comparisons with zero collisions. The paper's SipIt algorithm turns that into an exact prompt-recovery method: 100% accuracy, ~28s mean, under 0.25% of the vocabulary explored, in time linear in prompt length. (2026-08-03) [Security & safety · Solid · 56/100 · Runs on a consumer GPU; Widely used]
- [Inkling-Small: what on-policy distillation actually buys a reasoning model](https://ai.thesatyajit.com/articles/inkling-small.md): Thinking Machines' Inkling-Small is an Apache-2.0 MoE reasoning model distilled from its larger sibling Inkling via on-policy distillation, then pushed through two weeks of agentic-coding RL — and it now beats Inkling on most reasoning and coding benchmarks while losing on factuality. Independently verified: both models' stated parameter counts (276B / 975B) run 2.4-3.8% above what their own safetensors measure (265.96B / 952.38B). (2026-08-03) [Training & RL · Solid · 53/100 · Facts measured from source; Widely used]
- [Instella-MoE: a 16B MoE that never touches an NVIDIA GPU](https://ai.thesatyajit.com/articles/instella-moe.md): AMD's Instella-MoE is a 16B-total, 2.8B-active Mixture-of-Experts model trained end-to-end — pretrain through RL — entirely on MI300X and MI325X GPUs, shipping all six stage checkpoints, training code, and data recipes. A gated MLA attention output and FarSkip-Collective's deliberately stale activations (+12.7% pretraining throughput, −39.2% TTFT) land it ahead of OLMo-3-7B, SmolLM3-3B, and Moonlight-16B-A3B on average, trailing only Qwen3.5-4B-Base. (2026-08-03) [LLM architecture · Solid · 48/100 · A new technique; Open code or weights]
- [JOSIE-2: a 4M-token fine-tune, and a labeling bug in its own benchmark table](https://ai.thesatyajit.com/articles/josie-2.md): Gökdeniz Gülmez's JOSIE-2 fine-tunes Qwen3.5-2B/4B/9B into personality-driven models on the same ~4M-token, 3,500-sample dataset, entirely on Apple Silicon. The release claims each model beats its own larger base — but every card's benchmark table mislabels its baseline, and config.json shows same-size bases throughout. What the cards actually say, checked directly, alongside the model's genuinely interesting emergent trait: reasoning that swears and roasts because nobody trained it not to. (2026-08-03) [Training & RL · Niche · 42/100 · Checked against the source; Runs on a consumer GPU]
- [KDA has a half-life: linear attention forgets like a radioactive isotope](https://ai.thesatyajit.com/articles/kda-half-life.md): Kimi Delta Attention decays its recurrent state by a learned per-channel factor α, so a channel's memory follows Sₙ = αⁿS₀ — algebraically identical to N(t) = N₀e^{−λt}, with λ = −ln α. That gives every channel a half-life measured in tokens: α = 0.99 forgets half of what it knew after about 69 of them. A short walk through the decay law, why n½ = ln(0.5)/ln(α), and what it means that K3 learns a different α for every channel. (2026-08-03) [LLM architecture · Solid · 49/100 · Original analysis; A lasting reference]
- [kimi-k3-in-c: 2.78 trillion parameters, one CPU, 8 GB of RAM](https://ai.thesatyajit.com/articles/kimi-k3-in-c.md): A single-author C99 engine — no BLAS, no framework, no GPU — reimplements Kimi K3 inference from the checkpoint's own bytes: 2.78T parameters, a 1.56 TB checkpoint, a 176 KB binary, and a measured 8.24 GB peak RSS. The same run at 224 GB is 1.7x faster and produces byte-identical output — memory is a dial here, not a floor. (2026-08-03) [Inference & serving · High · 75/100 · Runs on a laptop CPU; Interactive explanations]
- [LFM2.5-Encoder: classification in one forward pass, zero completion tokens](https://ai.thesatyajit.com/articles/lfm2-5-encoders.md): Liquid AI's LFM2.5-Encoder-230M (229,693,184 params) and -350M (354,483,968 params) are bidirectional encoders for classification, not generation: mean-pool the last hidden state, one linear head, sigmoid per label, no decode loop. The 350M model ranks 4th of 14 on a 17-task GLUE/SuperGLUE/multilingual eval at 81.02, a point behind ModernBERT-large and two behind XLM-R XL, a model ten times its size. (2026-08-03) [LLM architecture · Solid · 51/100 · Runs on a laptop CPU; A lasting reference]
- [Towards looped models done right: what actually separates Ouro from Huginn](https://ai.thesatyajit.com/articles/looped-models-done-right.md): IFM Research isolates the three design axes tangled inside every looped transformer — iteration envelope, input injection, recurrent-state init — with controlled ablations at matched depth and parameters. The headline result upends a near-universal assumption: randomly initializing the recurrent state buys almost nothing net, and a shared H/L hierarchy is a wash. This is a Notion-hosted preprint with no arXiv listing and code marked 'coming soon' — a first look, not a settled result. (2026-08-03) [LLM architecture · Solid · 45/100 · A lasting reference; A new technique]
- [MemHarness: agent memory should be reconstructed, not replayed](https://ai.thesatyajit.com/articles/memharness.md): MemHarness trains a single 7B policy to critique and rewrite retrieved agent memories against the current state before acting, instead of injecting them verbatim. On ALFWorld and WebShop it reports 85.2% and 75.6% success — ahead of GPT-4o, Gemini-2.5-Pro, and a same-backbone GRPO baseline — and its ablations show raw memory replay can hurt more than having no memory at all. (2026-08-03) [Agents & harnesses · Niche · 37/100 · Explained from first principles]
- [MiniMax H3: open weights, four excluded countries, zero benchmarks](https://ai.thesatyajit.com/articles/minimax-h3.md): MiniMax H3 is an omni-modal text/image/video/audio-to-video model with a real, sizeable architecture — a Qwen3-VL-32B encoder feeding a 33B dense Omni-Transformer. But its 'open-source' license explicitly excludes the US, UK, EU, and South Korea, and there is not one quantitative benchmark anywhere in the release. A look at what 'open' is coming to mean, quoted precisely. (2026-08-03) [Image & video generation · Niche · 31/100]
- [pdf-inspector: classifying PDFs without a single model](https://ai.thesatyajit.com/articles/pdf-inspector.md): Firecrawl's pdf-inspector decides whether a PDF needs OCR in under 200ms using one Rust crate and zero machine learning — path-op density, font-decodability fallback chains, and a newspaper-layout detector calibrated against named WSJ and SEC documents. A short look at a well-chosen heuristic stack beating ML pipelines on a bounded problem. (2026-08-03) [Developer tools & infra · Niche · 42/100 · Runs on a laptop CPU; Explained from first principles]
- [Qwen-CUA: a computer-use agent that only ever sees pixels](https://ai.thesatyajit.com/articles/qwen-cua.md): Qwen-CUA is a 397B-A17B MoE agent (scaling to Qwen-CUA-Max, over 1 trillion parameters) that controls a computer from screenshots alone — no DOM, no accessibility tree — and reaches 86.2 on OSWorld-Verified. The mechanism worth learning is blockwise visual-history folding: 20 screenshots stay active, folded 10 at a time, which is what lets a 100,000-vCPU RL rollout fleet reuse its KV-cache instead of recomputing a fresh prefix every turn. (2026-08-03) [Agents & harnesses · Solid · 53/100 · Original analysis; Widely used]
- [Qwen3.8-Max: 16 days, 265 commits, zero humans in the loop](https://ai.thesatyajit.com/articles/qwen3-8-max.md): Alibaba's Qwen3.8-Max scales to 2.4T parameters (95B active) on the Qwen 3.5 architecture and becomes the first Qwen-Max-class model getting open weights — next week, not today. The benchmark table is the least interesting part: the real evidence is a 16-day, 265-commit self-evolving coding harness and a chip-design agent that cut a hardware accelerator from 8,298 to 678 gates over 500 turns, set against a benchmark table whose own footnotes admit real harness and judge asymmetries. (2026-08-03) [Agents & harnesses · Niche · 40/100 · Widely used]
- [Recursive Harness Self-Improvement: beat your last harness, not a population of them](https://ai.thesatyajit.com/articles/recursive-harness-self-improvement.md): Sakana AI and UC Berkeley optimize an agentic harness using only a pairwise comparison against its immediately-previous version — O(1) cost per iteration versus population search's O(m²) — and show a few iterations let a low-reasoning-effort agent beat maximum-reasoning-effort test-time scaling while cutting inference cost up to 60%. The information-theoretic account is backed by real measurements; there is no empirical comparison against the roughly 15 competing methods it discusses, and Opus-4.8 partly evaluates itself. (2026-08-03) [Agents & harnesses · Niche · 41/100 · A new technique; Explained from first principles]
- [Sol-Attn: deciding which attention blocks to skip while you're already streaming them](https://ai.thesatyajit.com/articles/sol-attn.md): Video diffusion transformers spend most of their inference budget on attention. Sol-Attn is training-free sparse attention that folds routing into the online-softmax pass a flash-attention kernel already runs — thresholding block scores at μ + βσ instead of top-k, and reusing the scores of skipped blocks to approximate their contribution instead of dropping them. 2.1× on generation, 2.3× on editing, and it ships inside a five-technique inference engine. (2026-08-03) [Inference & serving · Solid · 48/100 · A new technique; Open code or weights]
- [Uber's ADR: the agent that watches your agents](https://ai.thesatyajit.com/articles/uber-adr.md): Uber's ADR is not Architecture Decision Records — it's Agentic Detection and Response, an enterprise security framework for AI coding agents. A two-tier detector routes cheap triage on everything and expensive reasoning on the ambiguous few, and the expensive tier is a literal Claude Code CLI subprocess wielding MCP tools to fetch a suspicious tool's own source before it renders a verdict. Ten months in production, 7,200+ hosts, 1.000 precision on a 302-task enterprise benchmark, and an honest accounting of what didn't make the open-source release. (2026-08-03) [Security & safety · Solid · 56/100 · A lasting reference; A new technique]
- [Macaron-V1: four 1B adapters on a frozen 744B base](https://ai.thesatyajit.com/articles/macaron-v1.md): Mind Lab's Macaron-V1 puts all of its specialization into four 1B LoRA adapters riding on a frozen GLM-5.2, with the chat adapter doubling as a request-level router. A walk through Mixture-of-LoRA, how it differs from Mixture-of-Experts, what the adapters measurably buy over the base they sit on, and why the evaluation table needs reading twice. (2026-07-27) [Agents & harnesses · Solid · 53/100 · Original analysis; Open code or weights]
- [Neutrino-1: quantization is a training decision, not a deployment one](https://ai.thesatyajit.com/articles/neutrino-1.md): Fermion Research's Neutrino-1 8B trains ternary weights from the first gradient step instead of rounding them in afterward. The headline: 72.1 on 5-shot MMLU trained-in-format versus 24.2-24.7 for the same class of model rounded post-hoc — against a 25.0 chance line. (2026-07-27) [Quantization & compression · Solid · 52/100 · Runs on a laptop CPU; A lasting reference]
- [BTL-3: a rank-32 LoRA that turns Qwen3.6-27B into a tool-use agent](https://ai.thesatyajit.com/articles/btl-3.md): Bad Theory Labs' BTL-3 isn't a new model — it's a frozen ~934 MB PEFT LoRA adapter (rank 32) RL-tuned on top of Qwen3.6-27B for agentic coding and structured tool use. The base capability is Qwen's; what BTL-3 adds is the loop behaviour — reason, call tools, inspect results, recover, and, distinctively, stop when no tool is needed (91.2% BFCL 'irrelevance'). Self-reported: 95.1% HumanEval, 88.5% BFCL v4 AST, 88.1% LiveCodeBench v6 — with an honest cliff to 26.4% on BigCodeBench-Hard. Ships an 8.39 GB single-file Compact build. Apache-2.0. A grounded read of a thin model card. (2026-07-24) [Agents & harnesses · Niche · 32/100 · Runs on a consumer GPU]
- [Qwen's RL formulation: token-level RL is a first-order approximation to the reward you want](https://ai.thesatyajit.com/articles/first-order-rl.md): The Qwen team's formulation for RL with LLMs: token-level objectives like REINFORCE and GRPO are a first-order approximation to the true sequence-level reward, valid only when the training–inference discrepancy and policy staleness are both small. That one lens explains why importance-sampling correction, clipping, and Routing Replay for MoE models stabilize training — validated across hundreds of thousands of GPU hours on a 30B MoE. A first-principles walk through the formulation, the MoE routing problem, and the honest empirical recipe. (2026-07-24) [Training & RL · Solid · 47/100 · Widely used; A lasting reference]
- [FLUX 3: when an image model decides to become a world model](https://ai.thesatyajit.com/articles/flux-3.md): Black Forest Labs turned the image-only FLUX line into one multimodal flow-matching model that generates image, video (up to 20s with native audio), and robot action from a single backbone. A first-principles walk through flow matching and joint multimodal attention, the real architecture and Self-Flow figures, a sample clip, and BFL's own preliminary human-preference numbers — caveats kept in view. (2026-07-24) [Image & video generation · Niche · 34/100 · Widely used; A new technique]
- [GEPA: optimize anything you can score and describe](https://ai.thesatyajit.com/articles/gepa-optimize-anything.md): GEPA (Genetic-Pareto) optimizes a text system by reflecting on its execution traces in natural language and evolving a Pareto frontier of candidates — turning a handful of rollouts into a large gain where RL needs thousands. Its paper reports beating GRPO by up to 20% with 35x fewer rollouts. The 'optimize anything' interface generalizes this to any text artifact with a scoring function, and the new 'omni' release composes GEPA with agent-based optimizers into a meta-optimizer that, on Frontier-CS (10 problems, $20 each), tops every standalone optimizer. (2026-07-24) [Agents & harnesses · Solid · 54/100 · Widely used; A lasting reference]
- [Lanyon: proving a PDE solver correct before you run it](https://ai.thesatyajit.com/articles/lanyon-neurosymbolic.md): Lanyon is a neurosymbolic system that writes numerical PDE solvers and proves them against an axiomatization of IEEE-754 floating-point arithmetic — checking that the code matches the math before any simulation runs. On its own initial benchmark of simple linear PDE solvers (linear advection, Maxwell), Lanyon reports 20–250× fewer output tokens and seconds-not-minutes wall-clock versus Fable 5, Opus 4.8, GPT-5.6 Sol, GPT-5.5 and Kimi K3 — and says it catches the misformalizations (sorry/native_decide escape hatches, CFL=1 degenerate demos, real-number tactics on float code) those models commit. Self-reported, not independently replicated. The mechanism, and the skeptic's read. (2026-07-24) [Developer tools & infra · Niche · 27/100 · A new technique]
- [Ling-3.0-flash: a 124B open MoE that runs like a 5B and reaches for 1M tokens](https://ai.thesatyajit.com/articles/ling-3-0-flash.md): inclusionAI's Ling-3.0-flash is a 124B-parameter, ~5.1B-active open MoE that reaches a 1M-token context by interleaving Kimi Delta Attention (linear-time) with Gated MLA (full attention) at a 5:1 ratio, over a 512-expert / 8-active FFN. A first-principles tour of the hybrid-attention stack, the E512A8 + shared-expert MoE, and where the launch benchmarks land it against the 1T flagship and the frontier — plus the striking resemblance to Kimi K3. (2026-07-24) [LLM architecture · Niche · 39/100 · Open code or weights; Explained from first principles]
- [MAI-Image-2.5-Pro and MAI-Voice-2-Flash: Microsoft builds its own](https://ai.thesatyajit.com/articles/mai-image-2-5-voice-2.md): Microsoft AI put two in-house models into public preview — MAI-Image-2.5-Pro for text-to-image and MAI-Voice-2-Flash for fast speech — and quietly swapped them into Bing, PowerPoint, OneDrive and Dynamics 365. The story isn't a benchmark; it's the strategy: MAI now builds frontier image and voice models on its own data and serves them into Microsoft's surface at a fraction of the GPU cost of third-party models. (2026-07-24) [Image & video generation · Niche · 17/100]
- [SANA-Video 2.0: keeping video attention linear without losing the picture](https://ai.thesatyajit.com/articles/sana-video2.md): NVIDIA's SANA-Video 2.0 generates 720p, multi-second video on a single H100 — 13.06s for a 720p/5s clip, a claimed 120× over Wan 2.2-A14B. It gets there by refusing quadratic attention: a high-compression VAE shrinks the clip to a modest token count, then a hybrid backbone keeps three of every four layers linear and makes the fourth a full-softmax anchor, with Block Attention Residuals carrying the refreshed features across depth. A first-principles walk through why linear attention plus deep compression makes long, high-res video cheap — with the paper's real figures, a sample clip, and its own numbers kept honest. (2026-07-24) [Image & video generation · Solid · 49/100 · Widely used; A new technique]
- [Solar Open 2: Upstage's 250B-A15B hybrid-attention MoE](https://ai.thesatyajit.com/articles/solar-open2-250b.md): Upstage's Solar Open 2 is a 250B-parameter (15B active) Mixture-of-Experts model built on a hybrid attention stack — twelve blocks of one softmax layer and three linear-attention (KDA) layers, no positional encoding, a 1M-token context, and open weights under the Upstage Solar License. A walk through why only 12 of 48 layers keep a KV cache, the selective-weight-transfer init from Solar Open 1 (not depth up-scaling), the ~12T-token training, and the full self-reported English and Korean benchmark suite. (2026-07-24) [LLM architecture · Solid · 48/100 · Concrete numbers to act on; Open code or weights]
- [Ternary15M: a language model where every weight is −1, 0, or +1](https://ai.thesatyajit.com/articles/ternary15m.md): A from-scratch 15M-parameter TinyStories model where all 42 linear layers are ternary — each weight is exactly −1, 0, or +1 (BitNet b1.58 lineage). That turns every dot product into signed add/subtract/skip with a single scale multiply per output channel. A walk through ternary quantization, why three-valued weights make matmuls multiply-free, quantization-aware training with the straight-through estimator and an absmean scale, the ~1.58-bits/weight footprint, and the honest result: hard-ternary inference costs only +0.01 val loss over the latent model (1.61 vs 1.60), shipped at 43 MB — trained for about $0.70 on one L40S. (2026-07-24) [Quantization & compression · Niche · 42/100 · Runs on a laptop CPU; A lasting reference]
- [Nanbeige4.2-3B: looping a small model up to a big one's depth](https://ai.thesatyajit.com/articles/nanbeige-4-2-3b.md): A 3B-non-embedding open-weight agentic model that reports beating Qwen3.5-9B and Gemma4-12B on tool-use, code-agent, and most reasoning benchmarks. Now with the technical report: the lever is a Looped Transformer run twice — trained from scratch, not upcycled — plus a 28T-token pretrain, a STEM-to-agentic SFT curriculum, and multi-stage RL with outcome-and-process rewards. A walk through the loop, the parameter-efficiency story, and which numbers to trust. (2026-07-22) [LLM architecture · Niche · 44/100 · Runs on a laptop CPU; Explained from first principles]
- [Antares: a 1B model that hunts vulnerabilities like a person](https://ai.thesatyajit.com/articles/antares.md): Cisco's Foundation AI team trained sub-1B open-weight models to do one hard security job — localize where a known vulnerability lives in a codebase — as a terminal agent that greps, reads, backtracks, and submits a ranked list of files. Antares-1B scores 0.209 File F1 on a new 500-task benchmark, beating GLM-5.2 (753B) and Gemini 3 Pro at 15–172× lower cost, from a Granite 4.0 backbone that scores 0.000 untrained. The whole gain is training, not scale. (2026-07-22) [Security & safety · Niche · 29/100 · Runs on a consumer GPU]
- [Gigatoken: tokenizing at gigabytes per second](https://ai.thesatyajit.com/articles/gigatoken.md): Tokenization is the unglamorous first step of every LM data pipeline — and it's slower than you'd guess, even though HuggingFace and tiktoken already run multithreaded Rust. Gigatoken is a drop-in tokenizer that hits GB/s and runs up to ~1000× faster, not by out-threading them but by killing the regex pretokenization step with SIMD and caching every word it's already seen. A walk through why the regex was the bottleneck, the per-CPU numbers, and what 'tokenize the whole internet in 6.5 hours' actually means. (2026-07-22) [Developer tools & infra · Solid · 48/100 · Runs on a laptop CPU; Concrete numbers to act on]
- [Mage-Flow: a 4B image model that bets on its tokenizer](https://ai.thesatyajit.com/articles/mage-flow.md): Open image generators keep getting bigger — Z-Image at 6B, Qwen-Image at 20B, FLUX.2 at 32B, Hunyuan-Image at 80B. Microsoft's Mage-Flow bets the other way: a compact 4B stack for text-to-image and instruction editing that stays competitive by co-designing the pieces. Mage-VAE cuts tokenizer MACs ~12×/22× at matched fidelity, a native-resolution MMDiT packs any aspect ratio into one model, and fused CUDA kernels give 2.5× faster training — so a 4-step Turbo renders a 1024² image in 0.59s on one A100, at ~18 GB. A walk through the co-design. (2026-07-22) [Image & video generation · Niche · 29/100 · Runs on a consumer GPU]
- [NVIDIA Rubin: co-designing a GPU for the shape of agentic inference](https://ai.thesatyajit.com/articles/nvidia-rubin.md): Agentic workloads aren't one prompt and one answer — they're sustained inference across many reasoning steps, which stresses long-context attention, MoE decode, KV-cache capacity, kernel handoffs, scale-up communication, and power all at once. NVIDIA's Rubin GPU claims up to 10× more agentic throughput per watt than Blackwell by attacking each of those bottlenecks with a specific feature: 2:4 sparse attention and 4× softmax, shared MoE descriptors, 2× K-dimension GEMMs, 288 GB of 22 TB/s HBM4, tile-level kernel triggering, NVLink counted writes, and rack-scale power smoothing. A map of the co-design — with the vendor-number caveat kept in view. (2026-07-22) [GPUs, kernels & systems · Niche · 26/100 · Widely used; A new technique]
- [The harness is the generalizer: how a scaffold learns to solve longer, unseen tasks](https://ai.thesatyajit.com/articles/harness-compositional-generalization.md): Transformers are unreliable at compositional generalization — recombining known pieces to solve a novel problem. Alex Zhang and Omar Khattab argue the fix doesn't have to live in the weights: a harness that keeps every LM call locally in-distribution (via context offloading and programmatic sub-calls) can be RL-trained on short tasks and then generalize to ones 8–32× longer, and even to entirely different domains — approaching a frontier model while the base Transformer flatlines. (2026-07-21) [Agents & harnesses · Niche · 31/100 · A new technique; Explained from first principles]
- [Laguna S 2.1: an 8B-active model that won't give up](https://ai.thesatyajit.com/articles/laguna-s-2-1.md): Poolside's new agentic coding model is a 118B/8B-active MoE with a 1M-token context — small enough to run on a single DGX Spark, and the most capable coding model in its weight class by a wide margin. It gets there not by adding raw intelligence but by training behaviors: persistence, verification, and a willingness to backtrack. A walk through the weight-class story, the thinking-mode lever, three real trajectories, and the post-training that produced them — built in under nine weeks by the same Model Factory. (2026-07-21) [Agents & harnesses · Niche · 29/100 · Open code or weights]
- [LongStraw: fitting million-token RL onto a fixed GPU budget](https://ai.thesatyajit.com/articles/longstraw-2m-rl.md): Inference reaches million-token context; RL post-training is mostly stuck at 256K or below. LongStraw (MindLab + Fudan) closes that gap as a systems problem, not an accuracy one: capture the long prompt as detached resident state, replay one GRPO response at a time, and 2M-token — even 4.45M-token — steps fit on eight H20 GPUs. What it measures, the real numbers, and the honest fine print it is careful to print about what it does not claim. (2026-07-21) [Training & RL · Solid · 48/100 · A new technique; Open code or weights]
- [Qwen-Image-3.0: chasing “useful” instead of “good-looking”](https://ai.thesatyajit.com/articles/qwen-image-3.md): Qwen's third-generation image model reframes the goal from pretty pictures to deployable ones. Its one-word pitch is “Real”: 4.5k-token prompts that lay out nine infographics in a single pass, text legible down to 10px, nested UIs inside UIs, and rendering across 12 languages with real world knowledge. A walk through what it claims — with the interactives to feel the levers, the official examples, and the gaps independent testers found. (2026-07-21) [Image & video generation · Niche · 17/100 · Widely used]
- [EAGLE-3: making the draft model scale, and a from-scratch build](https://ai.thesatyajit.com/articles/eagle-3-speculative-decoding.md): Speculative decoding makes an LLM emit several tokens per forward pass by having a cheap draft model propose and the big model verify — losslessly. EAGLE-3 fixes the thing that stopped the draft model from improving with more data: it drops feature prediction for direct token prediction (Training-Time Test) and fuses low/mid/high features, unlocking a scaling law and up to 6.5x speedup. Plus a walk through tiny-speculators, a from-scratch EAGLE-3 trainer on Qwen3-8B. (2026-07-20) [Inference & serving · Niche · 45/100 · Runs on a consumer GPU; Widely used]
- [Motif 2.6B: differential attention and PolyNorm, trained at scale](https://ai.thesatyajit.com/articles/motif-2-6b.md): Motif 2.6B is a small model that makes two architecture bets you almost never see in a shipped model — differential attention and a learned polynomial activation (PolyNorm) — and trains them on 2.5T tokens with a data-mixing schedule that swings the corpus from broad web text into math, code, and reasoning. The payoff: a 2.6B model that matches 7–8B models on HumanEval and MATH. A walk through the two mechanisms, the data schedule, WSD + checkpoint averaging, and the honest benchmark picture. (2026-07-20) [LLM architecture · Niche · 38/100 · Runs on a consumer GPU; Explained from first principles]
- [Audex: audio, speech, and text through one decoder — without the text tax](https://ai.thesatyajit.com/articles/nemotron-audex.md): NVIDIA's Nemotron-Labs-Audex-30B-A3B bolts full audio intelligence — ASR, translation, audio understanding, TTS, audio generation, speech-to-speech — onto a strong text MoE LLM, and the surprise is what doesn't happen: the text scores barely move. One decoder, one extended vocabulary, audio in as continuous embeddings and out as discrete tokens. A walk through the architecture, the no-regression result, and the training recipe that buys it. (2026-07-20) [Speech & audio · Niche · 30/100 · Open code or weights]
- [Qwen Audio 3.0 TTS: an instructable LM-plus-flow-matching speech stack](https://ai.thesatyajit.com/articles/qwen-audio-3-tts.md): Alibaba's Qwen-Audio-3.0-TTS pairs an autoregressive LM with a flow-matching decoder in the CosyVoice lineage, fronted by a 12.5 Hz supervised tokenizer that keeps the token stream short. It adds free-style instruction control, 86 inline non-verbal tags, 16 languages plus 20 Chinese dialect regions, one-pass long-form to three minutes, and 48 kHz super-resolution — and it tops the Artificial Analysis TTS leaderboard. A walk through the stack, the frame-rate trick, and the control surface. (2026-07-20) [Speech & audio · Niche · 25/100 · Widely used]
- [Cosmos 3: a world model that reasons and generates in one sequence](https://ai.thesatyajit.com/articles/cosmos-world-model.md): Physical AI needs a model that can both plan and imagine the consequences. NVIDIA's Cosmos 3 does it with a Mixture-of-Transformers: an autoregressive Reasoner tower and a diffusion Generator tower sharing one token sequence, joined by dual-stream joint attention so the generator reads the reasoner's keys directly. It plans in pixel space via Action-CoT, and one backbone spawns six task models. Honestly framed: the SOTA claims are scoped to open models — Gemini 3.1 Pro still leads — and some numbers lean on best-of-N and provider-selected harnesses. (2026-07-04) [Robotics & embodied · Solid · 55/100 · Widely used; A lasting reference]
- [The harness effect: orchestration, not the model, sets your agent token bill](https://ai.thesatyajit.com/articles/harness-effect.md): A controlled swap that holds the model constant and changes only the orchestration layer — the harness — cuts blended cost per task 41%, wall-clock 44%, and tokens 38% at quality parity, and shows efficiency is model-invariant while quality gains scale almost perfectly with baseline strength (r = 0.99). (2026-07-18) [Agents & harnesses · Solid · 48/100 · Widely used; A lasting reference]
- [Diffusing blame: credit assignment under Dale's principle](https://ai.thesatyajit.com/articles/diffusing-blame.md): A biologically plausible network splits every layer into separate excitatory and inhibitory streams — obeying Dale's principle, which real neurons never break — and learns by diffusing the output error straight to every hidden unit instead of transporting transposed weights back through the stack; this is a first-principles walk through why backprop can't run in a brain, how Error Diffusion with modulo routing gets around it, and the 96.7% MNIST / 61.7% CIFAR-10 numbers that follow. (2026-07-17) [Training & RL · Niche · 32/100 · A new technique; Explained from first principles]
- [Intern-S2: a 397B model that reads the raw page](https://ai.thesatyajit.com/articles/intern-s2.md): InternLM's Intern-S2-Preview-397B is a multimodal scientific foundation model that trades blows with the closed frontier on general tasks and beats it by multiples on specialized science — how its raw-page vision pretraining, dynamic tokenizer, and multi-domain RL get there, with the benchmarks. (2026-07-17) [Vision & multimodal · Niche · 29/100 · Open code or weights]
- [LIMSSR: scoring actions when modalities go missing at training time](https://ai.thesatyajit.com/articles/limssr.md): Most missing-modality methods assume you trained on complete data and only lose a stream at test time. LIMSSR (ICML 2026 spotlight) tackles the realistic, harder case — modalities missing during training too — by reframing action-quality assessment as LLM-driven sequence-to-score reasoning, imputing missing modalities in prompt space instead of reconstructing them, and gating away hallucinated guesses. (2026-07-17) [Vision & multimodal · Niche · 19/100 · A new technique]
- [VideoChat3: a 4B video model that watches longer for less](https://ai.thesatyajit.com/articles/videochat3.md): VideoChat3 is a fully open 4B video MLLM built around two efficiency moves — an inflated 3D ViT that hits 16x spatiotemporal compression, and adaptive per-frame resolution. How both work, the benchmark deltas over Qwen3-VL-4B and Molmo2-4B, and the 44.4s → 20.4s latency win on 2048-frame video. (2026-07-17) [Vision & multimodal · Niche · 38/100 · Runs on a consumer GPU; A new technique]
- [ZUNA 1.1: a channel-agnostic EEG foundation model](https://ai.thesatyajit.com/articles/zuna-1-1.md): Zyphra's ZUNA 1.1 is a 380M-parameter encoder–decoder diffusion autoencoder that reconstructs, denoises, and upsamples EEG across any electrode layout — a 4-channel headband to a 256-channel cap — by treating each electrode as a token at a physical (x, y, z, t) coordinate instead of a fixed montage slot. How the 4D-RoPE channel-agnostic trick and the rectified-flow decoder work, and where the reconstruction actually holds up. (2026-07-17) [Vision & multimodal · Niche · 39/100 · Runs on a laptop CPU; A new technique]
- [GRAPE: RoPE, ALiBi, and FoX are the same construction](https://ai.thesatyajit.com/articles/grape-position-encoding.md): Positional encoding is a zoo of tricks — rotary embeddings, linear ALiBi biases, forget gates — each justified on its own terms. GRAPE (ICLR 2026, Princeton/UCLA/Tsinghua) shows they are one thing: a position n acting through a group action G(n) = exp(nωL). Pick a rank-2 skew generator and you get a rotation in SO(d) — that is exactly RoPE. Pick a rank-1 nilpotent generator and you get a shear in GL that adds a linear bias — that is exactly ALiBi, and FoX is its path-integral. A walk through the group theory, the closed forms, the honest (small-scale) results, and what the framework does and doesn't yet buy you. (2026-07-16) [LLM architecture · Niche · 44/100 · A lasting reference; A new technique]
- [Bonsai 27B: a 27B model at 1.125 bits, small enough for a phone](https://ai.thesatyajit.com/articles/bonsai-27b.md): PrismML's Bonsai 27B takes a full-precision Qwen3.6-27B and quantizes it end to end — embeddings, attention, MLPs, and the LM head — into a ternary (1.71 bits) or 1-bit (1.125 bits) model that shrinks 54 GB down to 3.9 GB and runs on an iPhone. The capability is Qwen's; the achievement is the extreme low-bit compression and the kernels that make it run. This is a walk through the bit encodings, why pushing low precision through the *whole* network is the hard part, and the honest, uneven cost — math survives almost intact while agentic tool-calling and vision fall much harder. (2026-07-15) [Quantization & compression · Solid · 47/100 · Runs on a phone; A new technique]
- [Inkling: an open-weights multimodal MoE built to be adapted](https://ai.thesatyajit.com/articles/inkling.md): Thinking Machines Lab's Inkling is a 975B-total / 41B-active mixture-of-experts foundation model with encoder-free text, image and audio, up to 1M-token context, and open weights. The lab is unusually blunt that it's 'not the strongest overall model' — it's a customizable base tuned for broad adaptation, not a SOTA claim. The interesting parts are the mechanics: controllable effort that hits a reference model's Terminal-Bench score at ~1/3 the tokens, RL reward that scaled log-linearly over 30M+ rollouts while chain-of-thought got shorter on its own, and a 5:1 sliding-window/global attention hybrid. All benchmarks are vendor-reported on their own suite, so scope the numbers accordingly — but the weights are open, which is the part that's checkable in time. (2026-07-15) [LLM architecture · Niche · 38/100 · Widely used; Open code or weights]
- [LOTUS: reasoning in the hidden states, not the token stream](https://ai.thesatyajit.com/articles/lotus-latent-reasoning.md): Explicit chain-of-thought writes every reasoning step out token-by-token, which is slow; latent CoT reasons in hidden states instead — but past 1B parameters it has always trailed explicit CoT, and the gap grew with scale. LOTUS closes it at 3B with a looped padded Transformer: reuse the same weights for R passes over a fixed latent region, refine all K blocks in parallel, and supervise each latent position against its gold CoT-step token. A walk through the loop, the parallel supervision, the real GSM8K numbers, and the honest limits of a fixed thinking budget. (2026-07-15) [LLM architecture · Niche · 39/100 · A lasting reference; A new technique]
- [Ring-Zero: what a trillion-parameter model learns from reward alone](https://ai.thesatyajit.com/articles/ring-zero-trillion-scale-rl.md): Zero RL — reinforcement learning from verifiable rewards, no human labels, no SFT — has mostly been studied on small models. Ring-Zero runs it on a 1-trillion-parameter (63B-active) MoE and reports three things: scale sharply raises the ceiling and sample-efficiency, training splits into a 'discovery' then a 'sharpening' phase, and advanced reasoning behaviors emerge on their own. The engineering that makes it stable is a four-stage pipeline and one quiet fix — a training-inference ratio correction that stops the importance weight from exploding. A detailed walk through the method, the real math, the numbers, and the honest caveats: it trails frontier models, and most ablations are run at 104B, not 1T. (2026-07-15) [Training & RL · Niche · 40/100 · A lasting reference; A new technique]
- [Mach-Mind-4-Flash: specialize, then integrate](https://ai.thesatyajit.com/articles/mach-mind-4-flash.md): Li Auto's Mach-Mind-4-Flash is a 35B / 3B-active MoE that reaches 100B-class scores through post-training alone — no extra pre-training compute. The recipe is specialize-then-integrate: train a dozen domain RL experts in parallel, then fuse them into one generalist with Multi-Teacher On-Policy Distillation (MOPD), a routed reverse-KL objective that kills the see-saw degradation of mixed-reward RL. A separate stage, HMPO, uses a median-length budget to cut reasoning tokens 19-46% for ≤0.7pp accuracy loss. A walk through both mechanisms, the numbers, and the honest gaps. (2026-07-13) [Training & RL · Niche · 37/100 · Explained from first principles]
- [Soofi S: a sovereign 3B-active model that keeps its cache near-constant](https://ai.thesatyajit.com/articles/soofi-s.md): A German consortium (KI Bundesverband, DFKI, Fraunhofer, TU Darmstadt, funded by the BMWE) built Soofi S 30B-A3B: an open, hybrid Mamba-Transformer MoE for German and English that activates ~3.2B of 31.6B parameters per token and keeps only 6 of 52 layers as attention — so decode throughput stays nearly flat as context grows while dense models decay. Pretrained on ~27T tokens with German deliberately up-weighted, on a sovereign German B200 cloud. This is a walk through the hybrid architecture, the near-constant-cache mechanism, the German-up-weighted data mixture, and the honest caveats — it is an author-reported consortium tech report, not peer-reviewed, and 'matches dense 14–27B' is an active-vs-total claim. (2026-07-13) [LLM architecture · Niche · 35/100 · Open code or weights; Explained from first principles]
- [One small box, 64 users at once: continuous batching on a DGX Spark](https://ai.thesatyajit.com/articles/dgx-spark-batching.md): A single DGX Spark — a GB10 Grace-Blackwell edge box with 128 GB of unified memory — serves dozens of concurrent chat users on a 35B model. The trick isn't a faster chip; it's vLLM's continuous batching: every decode step gathers all live streams into one fused forward pass, so aggregate throughput scales far above single-stream while each user's own speed stays modest. A precise, honest walk through the mechanism, the numbers I could verify against the spark-bench results, and the ones I couldn't. (2026-07-10) [Inference & serving · Solid · 55/100 · Facts measured from source; Widely used]
- [KAT-Coder-V2.5: training a coding model to live inside a repository](https://ai.thesatyajit.com/articles/kat-coder-agentic-training.md): Most 'coding models' are single-turn code generators. KAT-Coder-V2.5 is trained to act autonomously inside real, executable repositories — so the hard part isn't the model, it's the data-and-environment stack around it. This is a walk through that stack: AutoBuilder rebuilding real repos into verifiable sandboxes, a hint-boosted data flywheel that recovers near-miss trajectories without leaking hints, harness randomization that stops the policy overfitting its scaffold, an asymmetric PPO whose critic peeks at the future, and a multi-teacher distillation that fuses five specialists into one. It lands second only to Opus 4.8 on repository-level SWE — with honest caveats about internal baselines and benchmarks. (2026-07-10) [Training & RL · Niche · 39/100 · A new technique; Explained from first principles]
- [MusaCoder: teaching a model to write GPU kernels with execution-feedback RL](https://ai.thesatyajit.com/articles/musacoder-gpu-kernels.md): Generating a correct, fast CUDA/MUSA kernel from a PyTorch reference is a task where almost every early attempt fails to compile, so execution-based RL starves on sparse rewards, reward-hacks with PyTorch fallbacks, and destabilizes. MusaCoder is a full-stack recipe — kernel-oriented data synthesis, diversity-preserving rejection fine-tuning, and RL against the MooreEval verifier — held together by three fixes: PrimeEcho anchors multi-turn rewards to the first turn, Buffered Dynamic Retry recovers signal from all-failed samples, and MirrorPop masks off-policy sequences the vanilla filter misses. The 27B model reaches 93.2 Pass@8 on KernelBench, ahead of the closed frontier models the paper tests. A walk through the reward, the three stabilizers, the numbers, and the honest caveats. (2026-07-10) [Training & RL · Niche · 36/100 · A new technique; Explained from first principles]
- [Switch Transformers: route every token to exactly one expert](https://ai.thesatyajit.com/articles/switch-transformer.md): The 2021 paper that simplified Mixture-of-Experts by routing each token to a single expert instead of the usual two — cutting router math and cross-device communication, and scaling to a 1.6-trillion-parameter sparse model. This is a walk through the Switch layer, the capacity buffer that drops overflow tokens, the load-balancing loss and fp32 router that make top-1 routing stable, and the honest costs: sparse-not-dense compute, sample-efficiency-not-wall-clock speedups, and dropped tokens. The foundational simplification the modern MoE zoo descends from. (2026-07-10) [LLM architecture · Solid · 49/100 · Genuinely new idea; Evergreen reference]
- [SWE-1.7: near-frontier code RL, and the async training loop behind it](https://ai.thesatyajit.com/articles/swe-1-7.md): Cognition's SWE-1.7 is an RL-trained software-engineering model built on a Kimi K2.7 base, tuned for long-horizon work inside the Devin harness and served through Cerebras at 1000 tokens/sec. It doesn't top the benchmark table — Claude Opus 4.8 leads every row and it roughly matches GPT-5.5 — but the training write-up is the real payload: asynchronous multi-cluster RL, a top-p 'sampling distribution replay' trick that stops entropy collapse, compressed cross-continental weight sync, and self-compaction for six-hour rollouts. A first-principles walk through all four, with the provider-reported numbers in full. (2026-07-09) [Training & RL · Niche · 40/100 · A new technique; Explained from first principles]
- [A6B: k-expansion, or what breaks when you force a top-8 MoE router to fire 32 experts](https://ai.thesatyajit.com/articles/a6b-k-expansion.md): A single-author research log takes Qwen3.6-35B-A3B, overrides its Mixture-of-Experts router from top-8 to top-32 at inference (≈3B → ≈6.6B active params, zero new weights), measures the monotonic damage, and heals it with a router-frozen selective expert fine-tune. The core finding: renormalization hands 54% of the gate mass to 24 experts the model never learned to co-activate, accuracy falls at every step, and ESFT-style residual deltas recover the loss into a statistical tie — not a win. Honest, paired-McNemar measurement, negative results included. (2026-07-08) [LLM architecture · Niche · 40/100 · Open code or weights; Explained from first principles]
- [Agent harnesses: engineering the loop around the model](https://ai.thesatyajit.com/articles/agent-harness.md): Lilian Weng argues the harness — the loop and scaffolding that wraps a base model with tools, context management, control flow, and evaluation — matters as much as raw intelligence. A walk through her framing: the coding-harness loop, the tool taxonomy, why durable state belongs in files, the self-improving outer loop, and the failure modes and open problems she names honestly. (2026-07-08) [Agents & harnesses · Niche · 25/100 · Widely used; A lasting reference]
- [Antidoom: breaking doom loops with Final Token Preference Optimization](https://ai.thesatyajit.com/articles/antidoom.md): Liquid AI's Antidoom explains why small reasoning models get stuck repeating a span until the context runs out, and fixes it with Final Token Preference Optimization — a DPO-like method that retrains only the trailing token, spreads probability across several chosen alternatives, and regularizes the rest of the vocabulary in logit space. It cuts the doom-loop rate from 10.2% to 1.4% (LFM2.5-2.6B) and 22.9% to 1% (Qwen3.5-4B), with eval scores rising because the model can finally reach answers it already knew. (2026-07-08) [Training & RL · Niche · 43/100 · A lasting reference; A new technique]
- [Fireworks' distributed RL: frontier RL is cheaper than you think](https://ai.thesatyajit.com/articles/frontier-rl-cheaper.md): Fireworks argues frontier reinforcement learning does not need one co-located mega-cluster. Because more than 98% of a model's weights stay bit-identical between adjacent RL checkpoints, you can ship a ~2% delta across regions instead of the full ~1 TB on every update — cutting cross-region weight traffic ~94% over a 50-step window and letting asynchronous rollouts run on scattered GPU capacity. A walk through the delta-compression and async-RL argument, the numbers, and where it stops working — with the vendor framing named. (2026-07-08) [Training & RL · Niche · 44/100 · Concrete numbers to act on; Explained from first principles]
- [Gemma 4: an open multimodal family, tuned to the KV-cache budget](https://ai.thesatyajit.com/articles/gemma-4.md): Google DeepMind's Gemma 4 is a family of open-weight, natively multimodal models from 2.3B to 31B — dense plus a 26B-A4B MoE — released under Apache 2.0 with a thinking mode. The engineering worth reading is the memory work: a 5:1 local-to-global attention ratio, pp-RoPE, and reusing keys as values on global layers cut the global KV cache ~37.5%; the 12B drops its vision and audio encoders for raw-patch projection; and an MTP drafter head does speculative decoding without prefill. This is a first-principles walk through those choices with the paper's own numbers. (2026-07-08) [LLM architecture · Solid · 55/100 · Runs on a consumer GPU; Widely used]
- [Hunyuan Hy3: Tencent's 295B-A21B MoE, and the community 1M GGUF](https://ai.thesatyajit.com/articles/hunyuan-hy3.md): Tencent's Hy3 (preview) is a 295-billion-parameter Mixture-of-Experts model that activates only 21B per token, ships open weights with a 256K native context, and pairs grouped-query attention with a multi-token-prediction draft head for speculative decode. This is a first-principles walk through the architecture, the hybrid fast/slow-thinking training, the honest gap between the trained 256K window and the community YaRN 1M GGUF, the GGUF quant ladder for local inference, and the full provider-reported benchmark suite. (2026-07-08) [LLM architecture · Solid · 53/100 · Widely used; Concrete numbers to act on]
- [The Jacobian lens: reading the residual stream with a derivative](https://ai.thesatyajit.com/articles/jacobian-lens.md): Anthropic's Jacobian lens transports an intermediate residual-stream activation into the model's final-layer basis using the corpus-averaged input–output Jacobian, then unembeds it into a ranked token list — surfacing what a layer is disposed to say, with the honest caveat that a single averaged linear map only approximates a nonlinear stack. (2026-07-08) [Interpretability · Notable · 59/100 · Runs on a consumer GPU; A lasting reference]
- [J-space in the open: a CKA map of workspace geometry across 38 models](https://ai.thesatyajit.com/articles/jspace-open.md): Elie Bakouch's interactive J-lens CKA explorer computes the geometry of token-steering directions at every layer of 38 open models, then asks whether two layers — inside one model or across unrelated families — arrange those directions the same way. The answer is a sensory / workspace / motor block structure that sits at nearly the same relative depth in Gemma, Qwen, Llama, and OLMo alike. This is a walk through what the map plots, the exact CKA it computes, and an honest read of how universal the pattern really is. (2026-07-08) [Interpretability · Solid · 56/100 · Original analysis; Runs in a browser]
- [Muon: orthogonalizing the update for hidden layers](https://ai.thesatyajit.com/articles/muon-optimizer.md): Muon takes the momentum update for a 2D weight matrix and orthogonalizes it with a few Newton-Schulz iterations before applying it, so no singular direction dominates the step. This is a first-principles walk through the update rule and the quintic iteration (with the real coefficients), why it runs only on hidden 2D layers, the NanoGPT speedrun wins Keller Jordan reports, and Moonshot's 'Muon is Scalable' result — the ~2x compute-efficiency claim, the weight-decay and update-RMS fixes needed at scale, and the Moonlight model. (2026-07-08) [Training & RL · Notable · 67/100 · Runs on a consumer GPU; Interactive explanations]
- [Rollout Routing Replay: stabilizing MoE reinforcement learning](https://ai.thesatyajit.com/articles/rollout-routing-replay.md): RL post-training on Mixture-of-Experts models collapses because the router picks different experts at rollout time than at update time, so the importance-sampling ratio explodes. Rollout Routing Replay (R3) records the inference engine's routing masks and replays them during training — aligning expert selection while keeping the router's gradient. It cuts the train-inference KL from 1.54 to 0.75 ×10⁻³, near the dense baseline, prevents the collapses that GRPO/GSPO/TIS hit, and adds under 3% rollout overhead. (2026-07-08) [Training & RL · Solid · 51/100 · Widely used; A lasting reference]
- [Reading a torch.profiler trace: overhead-bound vs compute-bound](https://ai.thesatyajit.com/articles/torch-profiler.md): A working engineer's walk through Hugging Face's torch.profiler guide: the 20-line setup, how to read the key_averages() table and the Perfetto timeline, and how the exact same matmul+add flips from overhead-bound (GPU 23 us of work inside a 2.31 ms wall, ~98% idle) to compute-bound (a 4.285 ms gemm kernel) just by growing the matrices — plus what torch.compile actually fuses. (2026-07-08) [GPUs, kernels & systems · Solid · 52/100 · Runs on a consumer GPU; A guide you can follow today]
- [zvec: an in-process vector database, and the ANN search inside it](https://ai.thesatyajit.com/articles/zvec.md): Alibaba's zvec is an embedded, Apache-2.0 vector database — the 'SQLite for vectors' — built on the Proxima engine, with SDKs for Python, Node, Go, Rust, and Dart. It self-reports 8,475 QPS on VectorDBBench's Cohere 10M set from a 16-vCPU box using HNSW + int8 + a full-precision refiner. This is a first-principles walk through the two mechanisms that get you there — greedy graph descent and quantize-then-refine — with the self-reported numbers in full and an honest note on why the comparison isn't apples-to-apples. (2026-07-08) [Developer tools & infra · Niche · 33/100 · Runs on a laptop CPU; Explained from first principles]
- [Laguna's Model Factory: treating model development as an industrial process](https://ai.thesatyajit.com/articles/laguna-model-factory.md): Poolside's Laguna M.1 (225B/23B-active) and XS.2 (33B/3B-active) are competent agentic-coding MoE models — but the tech report is really about the thing that built them. The 'Model Factory' is a tightly-versioned loop of data, training, evaluation, and inference where every run is reproducible code and a win ships to production by flipping a flag. It's what let them build XS.2 from scratch in five weeks, and the paper argues that industrialized process — not any architecture trick — is the moat. (2026-07-03) [Training & RL · Solid · 47/100 · Runs on a consumer GPU; A lasting reference]
- [LongCat 2.0: a 1.6T open-weights MoE, and the sparse attention behind it](https://ai.thesatyajit.com/articles/longcat-2.md): Meituan's LongCat 2.0 is a 1.6-trillion-parameter Mixture-of-Experts model that activates only ~48B per token, ships under MIT, and was trained end-to-end on AI ASICs over 35T+ tokens. Its headline idea is LongCat Sparse Attention — a hierarchical, cross-layer, streaming-aware indexer that reads only a token-precise slice of a 1M-token context. This is an honest walk through the architecture and the mechanism, with the provider-reported benchmarks in full: a strong open-weights model that beats Gemini 3.1 Pro and GPT-5.5 on some coding tasks while trailing the top closed model on most. (2026-07-05) [LLM architecture · Solid · 49/100 · A new technique; Open code or weights]
- [MiniMax Sparse Attention: let each query pick its own blocks](https://ai.thesatyajit.com/articles/minimax-sparse-attention.md): Full attention makes every query read the whole context — the cost that caps long-context serving. MiniMax's MSA keeps it exact but sparse: a lightweight index branch scores past KV in blocks, keeps the top-16 (2,048 tokens) per query, and attends only those — per GQA group, so different heads see different blocks. At 1M context that's a 28.4× FLOP cut and a measured 14.2× prefill / 7.6× decode speedup, at on-par quality. A walk through the mechanism, the fixed-budget economics, and the honest caveats. (2026-07-04) [LLM architecture · Solid · 52/100 · A lasting reference; A new technique]
- [MrFlow: climb the resolution in pixel space, not diffusion steps](https://ai.thesatyajit.com/articles/mrflow-diffusion-acceleration.md): Flow and diffusion image models spend most of their compute running every denoising step at full resolution. MrFlow is a training-free reshuffle of that budget: run most steps at low resolution, do the resolution climb with a cheap pixel-space super-resolution network, re-inject a small closed-form amount of noise, and finish with a single high-resolution refine step. It buys real speedups (roughly 4–9× training-free on the good configs) at near-native quality — but the headline numbers are best-case, and how much quality you keep depends heavily on the config and the base model. (2026-07-04) [Image & video generation · Solid · 45/100 · A new technique; Explained from first principles]
- [Nemotron in NVFP4: training a frontier model natively in 4-bit](https://ai.thesatyajit.com/articles/nemotron-nvfp4.md): Quantization usually happens after training — you train in BF16, then squeeze the weights for serving. NVIDIA's Nemotron flips that: the forward and backward GEMMs run natively in NVFP4, a 4-bit format, during the training run itself. This is a walk through what NVFP4 actually is (a 4-bit element plus a two-level shared scale), the three tricks that keep 4-bit gradients from diverging, and the honest caveats — it's mixed-precision not end-to-end FP4, the quality claim is a training-loss gap, and the run diverged twice. (2026-07-04) [Quantization & compression · Solid · 48/100 · Widely used; A lasting reference]
- [Program-as-Weights: compiling a natural-language spec into a tiny local model](https://ai.thesatyajit.com/articles/program-as-weights.md): Some functions resist clean rules — 'alert on the log lines that matter', 'repair malformed JSON', 'rank by intent' — so we outsource them to a big LLM API, paying on every call. Program-as-Weights (PAW) compiles such a fuzzy function once, from a plain-language spec into a small LoRA adapter, then runs it on a frozen 0.6B interpreter locally and cheaply forever after. A 0.6B model executing PAW programs matches direct prompting of a 32B model at ~1/50th the memory, 30 tok/s on a MacBook. A walk through the compiler, the economics, and the honest caveats. (2026-07-04) [Inference & serving · Niche · 43/100 · Runs on a laptop CPU; A new technique]
- [TabFM: a foundation model that learns tables in-context](https://ai.thesatyajit.com/articles/tabular-foundation-model.md): Most tabular ML still means fitting a gradient-boosted tree per dataset. TabFM, Google's tabular foundation model, does something stranger: it treats your labeled rows as context and predicts new rows in a single forward pass — no per-dataset training — after being pre-trained on hundreds of millions of synthetic tables. This is a walk through its 3-stage architecture, its TabPFN/TabICL lineage, and an honest read of the TabArena numbers: the license, the class and feature ceilings, and the fact that 'beats GBDTs' means ensemble-vs-ensemble. (2026-07-04) [Data & datasets · Niche · 37/100 · Runs on a consumer GPU; Widely used]
- [Leanstral: proving theorems by being a code agent, not a prover](https://ai.thesatyajit.com/articles/leanstral-formal-proofs.md): Most top Lean systems win with bespoke test-time-scaling machinery — conjecture pools, parallel lemma proving, blueprint stages. Mistral's Leanstral 1.5 (119B total, 6B active) refuses all of it: it proves theorems by running the ordinary Mistral Vibe code-agent loop longer, the same interface a human uses. It saturates miniF2F, solves 587/672 PutnamBench, sets SoTA on FATE-X, and its performance scales smoothly with the per-attempt token budget — for about 1/75th the cost of the prover it edges. A walk through the loop, the scaling, and SafeVerify. (2026-07-03) [Agents & harnesses · Niche · 38/100 · A new technique; Open code or weights]
- [MiMo-V2-Flash: a 128-token window and one global layer in six](https://ai.thesatyajit.com/articles/mimo-v2-flash.md): Xiaomi's MiMo-V2-Flash is a 309B / 15B-active MoE that's the strongest open-source software-engineering model — and the interesting part is how it stays fast at 256K context. It interleaves sliding-window attention with a tiny 128-token window and a global-attention layer once every six, cutting the KV cache ~6× while long-context quality holds. Add multi-token prediction for self-speculative decoding and a multi-teacher distillation post-train, and it's a clean study in spending compute where it matters. (2026-07-03) [LLM architecture · Solid · 50/100 · Widely used; A lasting reference]
- [Set Diffusion: one knob from autoregression to diffusion](https://ai.thesatyajit.com/articles/set-diffusion.md): Autoregressive models are sequential but KV-cacheable; diffusion LMs are parallel and any-order but fixed-length and can't cache. Block diffusion split the difference with fixed left-to-right blocks. Set Diffusion generalizes all three by factorizing generation over flexible-position, flexible-length token *sets* — recovering AR, block diffusion, and order-agnostic diffusion as special cases, while keeping a KV cache that updates every step and the any-order flexibility that enables infilling. A walk through the spectrum and the results. (2026-07-03) [LLM architecture · Solid · 46/100 · Runs on a consumer GPU; A lasting reference]
- [BM25: the ranking function that refuses to die](https://ai.thesatyajit.com/articles/bm25.md): BM25 is a ~35-year-old bag-of-words ranking function, and it's still the default in Lucene, Elasticsearch, and OpenSearch — still a punishing baseline for billion-dollar neural retrievers, and still the sparse half of most hybrid search. It's TF-IDF with two honest fixes: term-frequency saturation and document-length normalization. A walk through the full formula, term by term, with a live BM25 engine you can query, and ~30 lines of Python that reproduce it. (2026-07-02) [Developer tools & infra · Notable · 61/100 · Runs on a laptop CPU; Interactive explanations]
- [TwoTower: giving a diffusion LM a frozen autoregressive memory](https://ai.thesatyajit.com/articles/nemotron-twotower.md): Diffusion language models generate in parallel, but they've all shared one network between two jobs that fight each other — causally representing the clean context, and bidirectionally denoising the noisy block. NVIDIA's Nemotron-Labs-TwoTower splits them: a frozen autoregressive tower holds the context, a trainable denoiser refines each block by cross-attending to it. Built on a 30B Mamba-Transformer MoE, it keeps 98.7% of the autoregressive baseline's quality at 2.42× the generation throughput. A walk through the architecture and the honest numbers. (2026-07-02) [LLM architecture · Solid · 48/100 · A new technique; Open code or weights]
- [TurboQuant: rotate first, then quantize the KV cache](https://ai.thesatyajit.com/articles/turboquant-kv-cache.md): The KV cache is the memory that caps how long a context and how big a batch you can serve. You can quantize it, but keys and values have outliers that make naive quantization lossy — and the usual fix, per-channel calibration, isn't available online. TurboQuant's move is to rotate every vector by a random orthogonal matrix first: that spreads the energy into a known Beta distribution, so one data-free optimal quantizer fits every vector, and a 1-bit residual trick keeps the attention scores unbiased. Near-optimal distortion, no calibration — and it shows up in vLLM, llama.cpp, and vector search. (2026-07-02) [Quantization & compression · Solid · 54/100 · Runs on a consumer GPU; Widely used]
- [TurboVec: FAISS-competitive vector search with no training phase](https://ai.thesatyajit.com/articles/turbovec.md): Product quantization made approximate nearest-neighbor search cheap — but it needs a training pass to learn codebooks, and rebuilds as your corpus drifts. TurboVec drops that: it's a Rust vector index built on Google's TurboQuant, which rotates every embedding into a known distribution and quantizes it with a data-free optimal scalar quantizer. No train() step, online ingest, ~16× compression, and it matches or beats FAISS IndexPQ recall while running 12–19% faster on ARM. A 10M-vector corpus in 4 GB instead of 31. (2026-07-02) [Developer tools & infra · Niche · 37/100 · Runs on a laptop CPU; Concrete numbers to act on]
- [HydraHead: hybrid attention at the head, not the layer](https://ai.thesatyajit.com/articles/hydrahead.md): Full attention is quadratic; linear attention is cheap but forgets. Everyone mixes them — but per layer, in blunt all-or-nothing blocks. HydraHead's finding is that the specialization lives at the head level: inside one layer, a few heads do long-range retrieval (which needs full attention) and the rest do local work (which linear attention handles fine). So it hybridizes along the head axis, keeps full attention only for the retrieval-critical heads it identifies by causal analysis, and holds long-context accuracy where other hybrids collapse. (2026-07-01) [LLM architecture · Niche · 44/100 · A lasting reference; A new technique]
- [Tapered Language Models: spend your width where the work is](https://ai.thesatyajit.com/articles/tapered-language-models.md): Every transformer since 2017 stacks identical layers — same width top to bottom. Tapered Language Models question that default: under a fixed parameter budget, pour MLP width into the early layers and thin out the late ones with a cosine schedule, and perplexity improves at no extra params or FLOPs. The reverse allocation hurts. A walk through the one-line change, the residual-stream evidence behind it, and how consistently it holds across four architectures. (2026-07-01) [LLM architecture · Niche · 43/100 · A lasting reference; A new technique]
- [Agents-A1: scaling the agent horizon, not the parameter count](https://ai.thesatyajit.com/articles/agents-a1.md): InternScience's Agents-A1 is a 35B Mixture-of-Experts model with ~3B active parameters that reaches the agentic-benchmark band of trillion-parameter frontier models — not by growing, but by scaling the horizon: 45K-token trajectories across six domains, a knowledge-action graph that turns agent traces into verifiable training targets, and a three-stage recipe that distills six specialist teachers into one student. A walk through the method and the numbers. (2026-06-30) [Training & RL · Niche · 38/100 · Open code or weights; Explained from first principles]
- [FAST-LIO2 from scratch: LiDAR-inertial odometry you can actually reproduce](https://ai.thesatyajit.com/articles/fast-lio2-lidar-inertial-odometry.md): A direct, tightly-coupled LiDAR-inertial SLAM system built on an iterated error-state Kalman filter and an incremental k-d tree. A full walk through the math and the real code — IMU propagation, scan deskewing, the point-to-plane iterated update with its reformulated Kalman gain, and the ikd-Tree map — with C++ from a clean reimplementation and simplified Python, so you can rebuild it without ROS and understand SLAM. (2026-06-28) [Robotics & embodied · High · 70/100 · Runs on a laptop CPU; A guide you can follow today]
- [How LLM inference works: prefill, decode, and where the time goes](https://ai.thesatyajit.com/articles/how-llm-inference-works.md): Every generate() call runs two phases with opposite bottlenecks — a compute-bound prefill and a memory-bound decode — and almost every inference optimization targets one or the other. A walk through the full path from tokens to streamed output, the KV cache that dominates the economics, and how to tell which phase is actually slow. (2026-06-28) [Inference & serving · Solid · 50/100 · Evergreen reference; Something most practitioners use]
- [iLLaDA: how far a masked-diffusion language model scales](https://ai.thesatyajit.com/articles/illada-diffusion-language-model.md): Almost every LLM is autoregressive — causal, left-to-right, one token per pass. iLLaDA is an 8B masked diffusion model with fully bidirectional attention, trained from scratch on 12T tokens. A walk through how masked diffusion LMs work, what iLLaDA changes over LLaDA, and the honest result: base-model parity with Qwen2.5, but a real gap that remains on instruction tuning. (2026-06-28) [LLM architecture · Niche · 44/100 · A lasting reference; A new technique]
- [The Kalman filter from first principles](https://ai.thesatyajit.com/articles/kalman-filter.md): A noisy sensor and a model of how the world moves, fused optimally into one estimate that knows how unsure it is. A first-principles walk through the predict-update cycle, the gain, and the full matrix equations — with runnable Python — building up to the EKF, the iterated EKF, and the on-manifold error-state filter that powers modern LiDAR-inertial SLAM. (2026-06-28) [Robotics & embodied · Notable · 61/100 · Runs on a laptop CPU; Interactive explanations]
- [MegaTrain: training a 120B model on one GPU by inverting where memory lives](https://ai.thesatyajit.com/articles/megatrain-single-gpu-training.md): Most large-model training is bottlenecked by GPU memory. MegaTrain flips the layout — host RAM holds every parameter, gradient, and optimizer moment, and the GPU is a transient compute engine that streams one layer at a time. A walk through the memory-centric design, the double-buffered CUDA-stream pipeline that hides PCIe, and the honest scope of training 120B params at full precision on a single H200. (2026-06-28) [GPUs, kernels & systems · Solid · 51/100 · Runs on a consumer GPU; A lasting reference]
- [DeepSeek DSpark: making speculative decoding draft better and verify smarter](https://ai.thesatyajit.com/articles/deepseek-dspark.md): DSpark isn't a new DeepSeek model — it's a speculative-decoding module bolted onto DeepSeek-V4 that combines a semi-autoregressive drafter with a confidence-scheduled, load-aware verifier. A walk through how it lifts accepted length 16–31% over prior drafters and shifts the serving Pareto frontier, losslessly. (2026-06-27) [Inference & serving · Solid · 51/100 · Widely used; A new technique]
- [Multi-token prediction: training a model to see further than one step](https://ai.thesatyajit.com/articles/multi-token-prediction.md): Predict the next n tokens instead of just one, and you get two things: a better-trained model and a built-in draft model for ~3x faster inference. A first-principles walk through MTP — Meta's parallel heads, DeepSeek-V3's sequential modules, the predict-verify-accept lineage from Google Brain, and Google's 2026 Gemma 4 and Gemini Nano deployments. (2026-06-27) [LLM architecture · Niche · 39/100 · Widely used; A lasting reference]
- [Nous Hermes and Mixture-of-Agents: when models confer before they answer](https://ai.thesatyajit.com/articles/nous-hermes-moa.md): Mixture-of-Agents stacks layers of LLMs that read each other's drafts and synthesize — and beats GPT-4 Omni on AlpacaEval using only open models. A first-principles walk through the MoA mechanism, why 'collaborativeness' works, and how Nous Research actually wired it onto the open-weight Hermes line via the Forge Reasoning API. (2026-06-27) [Agents & harnesses · Solid · 47/100 · A lasting reference; Open code or weights]
- [Unconventional AI's Un-0: generating images with coupled oscillators](https://ai.thesatyajit.com/articles/unconventional-un-0.md): Un-0 replaces neural-network layers and the diffusion schedule with a population of coupled Kuramoto oscillators — letting the physics of a dynamical system be the computation. A walk through the math, the generation pipeline, the FID numbers, and the honest gap between a working simulation and the unbuilt analog chip it's a stand-in for. (2026-06-27) [Image & video generation · Niche · 34/100 · Runs on a consumer GPU; A new technique]
- [GLM 5.2: long-horizon coding at a million tokens](https://ai.thesatyajit.com/articles/glm-5-2.md): Z.ai's GLM 5.2 is a 744B/40B-active open-weights MoE with a real 1M-token context, built for long-horizon agentic coding. How IndexShare makes that context cheap, what changed in training, and where it lands against the frontier — with the benchmarks. (2026-06-23) [LLM architecture · Solid · 51/100 · Widely used; A new technique]
- [Sakana Fugu: a multi-agent system as a model](https://ai.thesatyajit.com/articles/sakana-fugu.md): Sakana AI turned LLM orchestration into a single model. A walk through the two ICLR 2026 papers behind Fugu — TRINITY, an evolved sub-20K-parameter coordinator, and the Conductor, a 7B reinforcement-learned orchestrator — and how routing a pool of frontier models beats any one of them. (2026-06-23) [Agents & harnesses · Niche · 39/100 · A new technique; Explained from first principles]
- [Mixture of Experts, from scratch](https://ai.thesatyajit.com/articles/mixture-of-experts-from-scratch.md): Why MoE lets a model carry billions of parameters but only pay for a slice of them per token — built up from one MLP, a router, and a sparse forward pass, with the gating, dispatch, and load-balancing made visible. (2026-06-10) [LLM architecture · Solid · 57/100 · Runs on a laptop CPU; Evergreen reference]
- [Coroutines in C, intuitively](https://ai.thesatyajit.com/articles/coroutines-in-c.md): How to pause a function in the middle and resume it later — using nothing but a switch statement and __LINE__. An intuitive tour of Simon Tatham's classic trick, with a step-through animation. (2026-06-09) [Developer tools & infra · Niche · 43/100 · Runs on a laptop CPU; Evergreen reference]
- [How self-attention works in transformers](https://ai.thesatyajit.com/articles/how-transformers-attention-works.md): A from-scratch explainer of scaled dot-product attention — queries, keys, values, the softmax, and why the √d scaling matters. (2026-06-02) [LLM architecture · Niche · 35/100 · Evergreen reference; Something most practitioners use]

## Architectures

Explainers for the model architectures in the gallery at https://ai.thesatyajit.com/architectures (roster as JSON: https://ai.thesatyajit.com/api/architectures).

- [Attention and the KV cache: MHA, MQA, GQA and MLA, byte by byte](https://ai.thesatyajit.com/architectures/attention-kv.md): How the key-value side of attention shrinks to cut the cache a decoder rereads at every step: the 2·n_kv·d_h numbers per token per layer that multi-head attention keeps, one shared head in MQA, a few groups in GQA, and DeepSeek-V2's MLA, which caches a 512-wide latent plus a 64-wide decoupled RoPE key, (9/2)·d_h in all. With the quality evidence for each, and the cache of one 128K-token sequence worked out for DeepSeek-V2's shape and for Llama 3.1 70B. (2026-09-26)
- [The Diffusion Transformer, from first principles](https://ai.thesatyajit.com/architectures/diffusion-transformer.md): How a Diffusion Transformer denoises: a VAE's latent space, patches as tokens, the timestep and the prompt steering every block through adaLN-Zero's scale, shift and gate, how it replaced the U-Net, SD3's two-stream MM-DiT against single-stream blocks, the move from noise prediction to rectified flow, and the parameters and multiply-adds counted exactly for DiT-XL/2, with FLUX.1-dev and Qwen-Image-2.1 for comparison. (2026-09-26)
- [The bidirectional encoder (BERT), from first principles](https://ai.thesatyajit.com/architectures/encoder-bert.md): How BERT reads a whole sequence at once: the Transformer encoder with no causal mask, the three summed input embeddings, masked language modelling with its 15% and 80/10/10 recipe, next sentence prediction and why RoBERTa dropped it, the [CLS] vector and the fine-tuning heads, where BERT-base's 109,482,240 weights sit, and why encoders from DeBERTa to ModernBERT are still the default for classification and embeddings. (2026-09-26)
- [The encoder–decoder Transformer, from first principles](https://ai.thesatyajit.com/architectures/encoder-decoder.md): How a sequence-to-sequence Transformer turns one text into another: an encoder that reads the source with full attention, a decoder with a causal mask and cross-attention into the encoder's output, teacher forcing, where the weights and FLOPs go (each token uses half the model), T5's span corruption and BART's denoising, and why decoder-only models took over everything except a few jobs where the input really is separate. (2026-09-26)
- [Kimi K3's hybrid block, from first principles](https://ai.thesatyajit.com/architectures/kimi-k3.md): How Kimi K3 mixes information along three axes: Kimi Delta Attention, a gated delta rule derived from linear attention, on 69 of 93 layers; Gated MLA with no positional encoding on the other 24; Block Attention Residuals, which replace the residual sum with attention over earlier blocks; and Stable LatentMoE routing 16 of 896 experts in a half-width latent space. With the fixed state and the latent cache weighed in bytes, and where 2.78T parameters, 104.2B active, actually go. (2026-09-26)
- [Looped Transformers: depth from passes, not parameters](https://ai.thesatyajit.com/architectures/looped-transformer.md): How a looped, recurrent-depth Transformer runs one block of layers again and again: the prelude, core and coda, why depth then costs FLOPs, latency and KV cache but no weights, input injection and why it keeps the loop stable, recurrent-state initialisation, adaptive halting from the Universal Transformer's ACT to Huginn's zero-shot exits, training by truncated backpropagation through depth, what compute-matched comparisons leave of the gains, and what LOTUS and IFM's controlled ablations found. (2026-09-26)
- [Mamba and selective state spaces, from first principles](https://ai.thesatyajit.com/architectures/mamba-ssm.md): How a state space model turns a sequence into a fixed-size state: the continuous SSM and its zero-order-hold discretisation, why S4's time-invariant form is a convolution, why Mamba's input-dependent B, C and Δ break that and need a hardware-aware parallel scan, Mamba-2's state space duality with its scalar decay and chunked matrix multiplies, Mamba-2.8B counted to the last weight, its state weighed against a Transformer's KV cache, and why hybrids keep some attention. (2026-09-26)
- [Masked-diffusion language models, from first principles](https://ai.thesatyajit.com/architectures/masked-diffusion-lm.md): How a masked-diffusion language model works: the absorbing [MASK] forward process, a training loss that is a masked-language-model loss weighted by 1/t over every mask ratio, sampling by iterative unmasking under bidirectional attention with low-confidence remasking, why no exact KV cache exists and what parallel decoding costs, LLaDA 8B counted to its 8,015,581,184 parameters, Dream's start from an autoregressive model, and block diffusion's return to left-to-right. (2026-09-26)
- [Mixture-of-Depths: a router decides which tokens a block computes](https://ai.thesatyajit.com/architectures/mixture-of-depths.md): How Mixture-of-Depths caps a routed block at a fixed number of tokens: a scalar router per block, top-k over the sequence, the chosen tokens sent through attention and the MLP and scaled by their router weight, the rest carried past on the residual stream. The 12.5% capacity that worked best, what it saves (an eighth of the projections and MLP, a sixty-fourth of the attention scores), why top-k breaks causal sampling and the small predictor that fixes it, and how it relates to mixture-of-experts and early exit. (2026-09-26)
- [Mixture-of-Experts, from the router to the all-to-all](https://ai.thesatyajit.com/architectures/mixture-of-experts.md): How a sparse MoE layer replaces one feed-forward network with many experts and a router: softmax top-k gating, fine-grained and shared experts, why total and active parameters come apart (Hunyuan-A13B's 80.39B against 12.75B, counted to the weight, with Mixtral, DeepSeek-V2, DeepSeek-V3 and Kimi K3 beside it), load balancing from Shazeer's auxiliary losses to DeepSeek's routing bias, capacity and token dropping, and the two all-to-alls of expert parallelism. (2026-09-26)
- [Rotary position embeddings (RoPE), from first principles](https://ai.thesatyajit.com/architectures/rope.md): How RoPE tells attention where tokens are: why attention needs position at all, absolute versus relative schemes, rotating each pair of query and key dimensions by m·θ_i with θ_i = base^(−2i/d), the three-line proof that the score then depends only on m − n, what base 10,000 means for a 128-dimensional head, and how contexts were stretched past training length with position interpolation, NTK-aware scaling, YaRN and a larger base, plus partial RoPE. (2026-09-26)
- [The Transformer, from first principles](https://ai.thesatyajit.com/architectures/transformer.md): How a decoder-only Transformer turns token ids into a next-token distribution: embeddings, scaled dot-product attention and why it divides by √d_k, multi-head attention, the causal mask, the residual stream and pre-norm, the 4× MLP with GELU or SwiGLU, stacking, where the parameters and FLOPs go in Llama 2 7B, counted to the last weight, and the KV cache at inference. (2026-09-26)

## AI breakthroughs in mathematics

- [AI breakthroughs in mathematics](https://ai.thesatyajit.com/math): mathematics produced or claimed by AI systems, as a filterable wall of short narrated videos, one per result family of OpenAI's openai/math release (372 claimed results, some checked in Lean), each with the claim as we read it, our verdict, its Lean status and its manuscripts. As JSON: https://ai.thesatyajit.com/api/math (one family in full: https://ai.thesatyajit.com/api/math/107). The write-up: https://ai.thesatyajit.com/articles/openai-math.md

## Daily logs (latest)

- [Scaffolding the AI site](https://ai.thesatyajit.com/logs/2026-06-03-scaffolding.md) (2026-06-03)

## arXiv digests (latest)

- [2026-07-22 — 8 papers](https://ai.thesatyajit.com/arxiv/2026-07-22.md) ★
- [2026-07-15 — 6 papers](https://ai.thesatyajit.com/arxiv/2026-07-15.md) ★
- [2026-07-10 — 6 papers](https://ai.thesatyajit.com/arxiv/2026-07-10.md) ★
- [2026-07-03 — 7 papers](https://ai.thesatyajit.com/arxiv/2026-07-03.md) ★
- [2026-07-01 — 7 papers](https://ai.thesatyajit.com/arxiv/2026-07-01.md) ★
- [2026-06-30 — 7 papers](https://ai.thesatyajit.com/arxiv/2026-06-30.md) ★
- [2026-06-28 — 7 papers](https://ai.thesatyajit.com/arxiv/2026-06-28.md) ★
- [2026-06-27 — 8 papers](https://ai.thesatyajit.com/arxiv/2026-06-27.md) ★
- [2026-06-23 — 6 papers](https://ai.thesatyajit.com/arxiv/2026-06-23.md) ★
- [2026-06-10 — 6 papers](https://ai.thesatyajit.com/arxiv/2026-06-10.md) ★
- [2026-06-09 — 5 papers](https://ai.thesatyajit.com/arxiv/2026-06-09.md) ★
- [2026-06-03 — 4 papers](https://ai.thesatyajit.com/arxiv/2026-06-03.md) ★

## Snippets

- [Two ways to read a decision out of a model](https://ai.thesatyajit.com/snippets/two-ways-to-read-a-decision.md) (python)
- [PyTorch channels-last + AMP training loop](https://ai.thesatyajit.com/snippets/pytorch-channels-last-amp.md) (python)
- [CUDA grid-stride loop for vector add](https://ai.thesatyajit.com/snippets/cuda-vector-add-grid-stride.md) (cuda)

## Notes

- [LiDAR Odometry](https://ai.thesatyajit.com/notes/lidar-odometry.md)
- [Point Cloud Registration](https://ai.thesatyajit.com/notes/point-cloud-registration.md)

## Optional

- [Publications](https://ai.thesatyajit.com/api/publications): peer-reviewed papers (JSON)
- [Patents](https://ai.thesatyajit.com/api/patents): two pending USPTO applications (JSON)
- [GitHub stats](https://ai.thesatyajit.com/api/github): repos, stars, languages (JSON)
- [Colophon](https://ai.thesatyajit.com/colophon): how this site works