Articles

Bad Theory Labs' BTL-3 isn't a new model — it's a frozen ~934 MB PEFT LoRA adapter (rank 32) RL-tuned on top of Qwen3.6-27B for agentic coding and structured tool use. The base capability is Qwen's; what BTL-3 adds is the loop behaviour — reason, call tools, inspect results, recover, and, distinctively, stop when no tool is needed (91.2% BFCL 'irrelevance'). Self-reported: 95.1% HumanEval, 88.5% BFCL v4 AST, 88.1% LiveCodeBench v6 — with an honest cliff to 26.4% on BigCodeBench-Hard. Ships an 8.39 GB single-file Compact build. Apache-2.0. A grounded read of a thin model card.

Solid·6 min · agents · tool-use · code-generation · fine-tuning · open-weights · explainer

The Qwen team's formulation for RL with LLMs: token-level objectives like REINFORCE and GRPO are a first-order approximation to the true sequence-level reward, valid only when the training–inference discrepancy and policy staleness are both small. That one lens explains why importance-sampling correction, clipping, and Routing Replay for MoE models stabilize training — validated across hundreds of thousands of GPU hours on a 30B MoE. A first-principles walk through the formulation, the MoE routing problem, and the honest empirical recipe.

Notable·11 min · reinforcement-learning · llm · mixture-of-experts · post-training · explainer

Black Forest Labs turned the image-only FLUX line into one multimodal flow-matching model that generates image, video (up to 20s with native audio), and robot action from a single backbone. A first-principles walk through flow matching and joint multimodal attention, the real architecture and Self-Flow figures, a sample clip, and BFL's own preliminary human-preference numbers — caveats kept in view.

Essential·11 min · diffusion · image-generation · video-generation · flow-matching · black-forest-labs · explainer

GEPA (Genetic-Pareto) optimizes a text system by reflecting on its execution traces in natural language and evolving a Pareto frontier of candidates — turning a handful of rollouts into a large gain where RL needs thousands. Its paper reports beating GRPO by up to 20% with 35x fewer rollouts. The 'optimize anything' interface generalizes this to any text artifact with a scoring function, and the new 'omni' release composes GEPA with agent-based optimizers into a meta-optimizer that, on Frontier-CS (10 problems, $20 each), tops every standalone optimizer.

High·10 min · optimization · prompt-optimization · evolutionary · agents · explainer

Lanyon is a neurosymbolic system that writes numerical PDE solvers and proves them against an axiomatization of IEEE-754 floating-point arithmetic — checking that the code matches the math before any simulation runs. On its own initial benchmark of simple linear PDE solvers (linear advection, Maxwell), Lanyon reports 20–250× fewer output tokens and seconds-not-minutes wall-clock versus Fable 5, Opus 4.8, GPT-5.6 Sol, GPT-5.5 and Kimi K3 — and says it catches the misformalizations (sorry/native_decide escape hatches, CFL=1 degenerate demos, real-number tactics on float code) those models commit. Self-reported, not independently replicated. The mechanism, and the skeptic's read.

Notable·11 min · neurosymbolic · theorem-proving · pde · scientific-computing · verification · explainer

inclusionAI's Ling-3.0-flash is a 124B-parameter, ~5.1B-active open MoE that reaches a 1M-token context by interleaving Kimi Delta Attention (linear-time) with Gated MLA (full attention) at a 5:1 ratio, over a 512-expert / 8-active FFN. A first-principles tour of the hybrid-attention stack, the E512A8 + shared-expert MoE, and where the launch benchmarks land it against the 1T flagship and the frontier — plus the striking resemblance to Kimi K3.

Essential·10 min · llm · mixture-of-experts · linear-attention · hybrid-attention · open-weights · explainer

Microsoft AI put two in-house models into public preview — MAI-Image-2.5-Pro for text-to-image and MAI-Voice-2-Flash for fast speech — and quietly swapped them into Bing, PowerPoint, OneDrive and Dynamics 365. The story isn't a benchmark; it's the strategy: MAI now builds frontier image and voice models on its own data and serves them into Microsoft's surface at a fraction of the GPU cost of third-party models.

Solid·7 min · image-generation · tts · microsoft · multimodal · explainer

NVIDIA's SANA-Video 2.0 generates 720p, multi-second video on a single H100 — 13.06s for a 720p/5s clip, a claimed 120× over Wan 2.2-A14B. It gets there by refusing quadratic attention: a high-compression VAE shrinks the clip to a modest token count, then a hybrid backbone keeps three of every four layers linear and makes the fourth a full-softmax anchor, with Block Attention Residuals carrying the refreshed features across depth. A first-principles walk through why linear attention plus deep compression makes long, high-res video cheap — with the paper's real figures, a sample clip, and its own numbers kept honest.

Essential·11 min · diffusion · video-generation · linear-attention · efficient-inference · nvidia · explainer

Prime Intellect integrated 23 agentic tasksets — software-engineering, terminal, and web-search — behind a single taskset API: ~365,000 tasks with prebuilt per-task sandbox images, grading material withheld until scoring, and gold-patch/no-op-validated re-uploads. A walk through why agentic RL is bottlenecked on verified, reproducible environments; the one-contract design (verifiers v1's taskset/harness/runtime split and the Harbor format); the catalog and its counts; the validation pipeline; and the honest failure modes — reward hacks and PR-test false negatives.

Essential·13 min · reinforcement-learning · agents · environments · infrastructure · prime-intellect · explainer

Upstage's Solar Open 2 is a 250B-parameter (15B active) Mixture-of-Experts model built on a hybrid attention stack — twelve blocks of one softmax layer and three linear-attention (KDA) layers, no positional encoding, a 1M-token context, and open weights under the Upstage Solar License. A walk through why only 12 of 48 layers keep a KV cache, the selective-weight-transfer init from Solar Open 1 (not depth up-scaling), the ~12T-token training, and the full self-reported English and Korean benchmark suite.

Notable·15 min · llm · open-weights · upstage · moe · long-context

A from-scratch 15M-parameter TinyStories model where all 42 linear layers are ternary — each weight is exactly −1, 0, or +1 (BitNet b1.58 lineage). That turns every dot product into signed add/subtract/skip with a single scale multiply per output channel. A walk through ternary quantization, why three-valued weights make matmuls multiply-free, quantization-aware training with the straight-through estimator and an absmean scale, the ~1.58-bits/weight footprint, and the honest result: hard-ternary inference costs only +0.01 val loss over the latent model (1.61 vs 1.60), shipped at 43 MB — trained for about $0.70 on one L40S.

Notable·8 min · quantization · ternary · bitnet · efficiency · from-scratch · explainer

Cisco's Foundation AI team trained sub-1B open-weight models to do one hard security job — localize where a known vulnerability lives in a codebase — as a terminal agent that greps, reads, backtracks, and submits a ranked list of files. Antares-1B scores 0.209 File F1 on a new 500-task benchmark, beating GLM-5.2 (753B) and Gemini 3 Pro at 15–172× lower cost, from a Granite 4.0 backbone that scores 0.000 untrained. The whole gain is training, not scale.

High·6 min · security · ai · open-weights · agents · explainer