Diffusing blame: credit assignment under Dale's principle

Diffusing Blame asks a sharp question: can a network learn useful representations while obeying the one rule real brains never break? That rule is Dale's principle — a neuron is either excitatory or inhibitory, and every synapse it makes carries that one sign. Backpropagation quietly violates a stack of constraints like this. The paper builds a network that respects them, trains it with a rule called Error Diffusion, and asks how far it gets. The answer: 96.7% on MNIST, a 61.7% baseline on CIFAR-10, and reinforcement-learning agents that hold their own against a backprop-free baseline — all while enforcing Dale's principle strictly. This is a walk through why that is hard, what the rule actually does, and the numbers, honestly labelled.

Why backprop can't run in a brain

Backpropagation is the reason deep nets learn, and it is also the reason nobody thinks the brain runs it. Look at the backward pass. To update layer $\ell$, backprop needs the error signal $\delta_\ell$, and it computes it by pulling the layer above's error back through the transposed forward weights:

Read that literally as a circuit and three problems fall out.

• Weight transport. The backward pass uses $W_{\ell+1}^{\top}$ — the same forward weights, transposed. A synapse would have to read the value of the forward synapse it feeds and reuse it, exactly, on the way back. There is no known biological mechanism for a synapse to know its partner's weight.
• Sign symmetry. Because it is the same weight, the feedback carries the same sign as the forward connection. Forward and backward paths are locked together.
• A separate error channel. The $\delta$'s have to travel back through a network that is distinct from the forward one, layer by layer, without disturbing the forward activations.

Dale's principle makes all of this worse. In cortex a neuron's outgoing synapses are uniformly excitatory or uniformly inhibitory — the sign belongs to the neuron, not the synapse. A weight in a standard net has no such loyalty: one unit can push one target up and pull another down in the same step. Toggle between the two regimes and click a source neuron to see its fan-out:

Enforcing Dale's principle means the sign is frozen per neuron, so a net has to split into separate excitatory (E) and inhibitory (I) populations and coordinate them. That changes credit assignment at the root: you can no longer flip a weight's sign to fix an error, and the tidy $W^{\top}$ feedback path is off the table anyway. Prior biologically plausible rules — feedback alignment and its kin — dodge weight transport by sending the error back through a fixed random matrix $B$ instead of $W^{\top}$. That helps, but it trades one implausible object (the transpose) for another (a dedicated random feedback matrix), and historically these rules stall out past MNIST. The paper wants neither the transpose nor the random matrix.

Dale's principle, in the weights

The architecture is dual-stream. Each layer carries a positive activation vector $\mathbf{p}$ and a negative one $\mathbf{n}$, and there are four weight matrices between consecutive layers — the within-stream pair $W_{pp}, W_{nn}$ and the cross-stream pair $W_{np}, W_{pn}$:

The trick is in the signs. Every learnable weight is constrained non-negative, $W_{\bullet\bullet} \ge 0$ element-wise, and the minus signs in front of the cross-stream terms are hardcoded into the wiring, not learned. So the E stream always excites and the I stream always inhibits, structurally — Dale's principle holds by construction, and gradient descent can never sneak a sign flip past it. A readout that needs a signed output just subtracts the streams: $\hat{y} = y^{+} - y^{-}$.

This is a real cost. A standard dense layer is one matrix; this is four non-negative matrices with a fixed sign pattern, and the optimizer has to move the whole coordinated E/I system in lockstep. The question the paper answers is whether a learning rule can drive that system without any of backprop's illegal moves.

Diffusing the error instead of transporting it

Error Diffusion's answer: don't route the error back through the layers at all. Route it directly to every layer. Take the output error $S$ (shape $B \times C$ for a batch of $B$ over $C$ classes) and broadcast it to the hidden units through a fixed routing matrix $M$, then form each layer's local update from presynaptic activity and the postsynaptic nonlinearity's derivative:

$R$ is the routed error drive, $U_p$ scales it by the local activation slope $\phi'$, and the weight change is an outer product of presynaptic activations $A_p$ with $U_p$. No $W^{\top}$ appears anywhere. No random feedback matrix appears either — the routing $M$ is a fixed, structured broadcast, not a learned or random projection.

The original Error Diffusion was defined for binary classification. To go past that, the paper adds modulo error routing: hidden unit $i$ is assigned to output channel

and learns from that channel's error $s_{r(i)}$. It is coarse — several hidden units share a channel, and unit $C$ wraps back to channel 0 — but it is deterministic, transport-free, and enough to spread class-specific blame across a wide hidden layer. Step through the forward pass, the diffusion, and the update, and flip between backprop and Error Diffusion to see the paths diverge:

The contrast is the whole point. Backprop's blame crawls back one layer at a time, each hop paying the $W^{\top}$ transport tax. Error Diffusion drops the error onto every hidden unit at once and lets each layer compute a local update. That is what makes it plausible — and also what makes it approximate, since a modulo-routed broadcast is a much blunter credit signal than the exact gradient.

expand

Three fixes that turn it into a learner

Error Diffusion out of the box does not learn much — the seed configuration lands at 50.4% on MNIST and 11.6% on CIFAR-10 (barely above chance on ten classes). Three domain-specific fixes close most of the gap.

Layer-specific sigmoid widths. The activation is a temperature-controlled sigmoid,

with a per-layer width $\alpha_i$. Why it matters: the update is scaled by $\phi'$, and a standard sigmoid's derivative is tiny once units saturate. The paper measures a 25x attenuation of the surrogate gradient from the output down to the first hidden layer, so the early layers barely move. Widening the sigmoid (larger $\alpha$) keeps the derivative alive deeper in the stack. Their CIFAR-10 setup uses $\alpha = 3.0$ for convolutional layers and $\alpha = 6.0$ for fully connected ones; MNIST uses $\alpha = 6.0$ throughout.

Batch-centered class error. Instead of feeding raw one-vs-all errors, the class error is centered across the batch,

so every class's error signal is zero-mean over the mini-batch. This removes a constant per-class bias that would otherwise push all units in a channel the same way regardless of the input.

Asymmetric E/I initialization. The excitatory matrices $W_{pp}, W_{nn}$ are scaled up by $1.5\times$ at init and the inhibitory $W_{np}, W_{pn}$ scaled down by $0.5\times$, a starting excitation-to-inhibition ratio of roughly 3:1. That gives the network net-positive drive to begin with, and the paper shows the ratio relaxes toward a biological-like balance as training proceeds.

What it scores

On the standard benchmarks, the constrained network learns — not to backprop's level, but well past chance, and well past unconstrained biologically plausible baselines that stall on MNIST. Direct Feedback Alignment (DFA), the backprop-free baseline that still uses a random feedback matrix, sits a few points ahead as the reference ceiling:

expand

The most interesting result is not the headline accuracy — it is what the ablations reveal. Remove each fix and measure the accuracy drop, and the ranking reverses between the two datasets:

On MNIST the whole model lives or dies by the sigmoid widths — pull them and accuracy craters by 71 points, while the batch-centering does almost nothing. On CIFAR-10 it is the exact opposite: batch-centered class error is load-bearing (−47.9), and the widths matter far less. Same architecture, same rule — but the credit-assignment bottleneck is task-dependent, and a single-benchmark evaluation would have hidden that entirely. That is the paper's sharpest point: which fix carries the model is a property of the task, not the method.

Into RL: ED-PPO

Classification is the easy setting — the error signal is a clean label. To test the rule where credit assignment is genuinely hard, the paper drops Error Diffusion into PPO, replacing the backprop gradient through the hidden layers of both the policy and value networks (the PPO objective still supplies the output-level error). Policy errors route to hidden units by action channel; value errors broadcast to all units. On Brax continuous control, ED-PPO is competitive with DFA and, on HalfCheetah, clears backprop:

That HalfCheetah result — ED-PPO at 5494±691, essentially matching DFA-PPO's 5581±359 and beating backprop's 3520±485 — is the strongest single number in the paper, but it does not generalize cleanly. On Humanoid, ED-PPO (6670±2592) trails backprop (8478); on the open-ended exploration task Craftax, ED-PPO edges out DFA-PPO (19.8±1.5 return) but sits below BP-PPO. The honest read is "competitive with the backprop-free baseline, still short of backprop on the hardest tasks" — which is exactly what the abstract claims, and worth stating plainly rather than cherry-picking HalfCheetah.

The take

The idea is clean and the framing is honest. Real neural circuits obey constraints backprop ignores — a synapse can't read its partner's weight (no weight transport), and a neuron can't flip signs synapse by synapse (Dale's principle). Build a network that respects both, and credit assignment stops looking like a transpose and starts looking like a broadcast: Error Diffusion drops the output error straight onto every hidden layer, routes it by a modulo rule, and updates each layer locally. Three fixes — wider per-layer sigmoids to fight a 25x gradient decay, batch-centered class error, and a 3:1 excitation-to-inhibition initialization — are what turn a 50%-on-MNIST seed into a 96.7% learner and a 61.7% CIFAR-10 baseline. None of that is state of the art, and the paper doesn't pretend otherwise. What it earns is a real claim: you can learn representations under the brain's actual wiring rules, the gap to backprop is a few points rather than a chasm, and — the part I'll remember — which trick matters most depends on the task, a bottleneck you only see if you test on more than one benchmark.

Built on Y. Yamada, L. Grillotti, R. Charakorn, S. Risi, D. Ha and R. T. Lange, Diffusing Blame: Task-Dependent Credit Assignment in Biologically Plausible Dual-Stream Networks (arXiv 2606.31700, 2026). Figures 1 and 2 are reproduced from the paper for commentary. The DaleNetwork and ErrorDiffusion widgets are my own illustrations of the mechanism, not measured traces; all accuracies, returns, and ablation deltas are author-reported and I have not independently reproduced them.