SteadySplats: teach the Gaussians to agree, and a one-sample splat renderer stops boiling

The post that sent me here is Sameh Khamis's: "I have SteadySplats implemented in the browser! Saw this tweet yesterday - had it working today, couldn't wait for the code release! Training is unaffected - new variance term is practically free. Rendering is at a solid 240 fps throughout, bigger difference with denser views." The tweet he saw is Radiance Fields', whose reply links the paper, arXiv 2610.05576, and the project page.

It is easy to read "steady" and "variance term" as a cure for popping, the flash you see when a sorted splat renderer reorders two overlapping Gaussians as the camera moves. That is not what this paper fixes. SteadySplats renders without sorting at all, so there is no order to flip. What it fixes is the price you pay for not sorting: noise. This piece is about where that noise comes from, the one-line training term that removes most of it, and the resampling filter that cleans up the rest. I checked the variance term on the authors' own released model.

Numbers are labelled measured (I computed them from a file or a run of my own code), reported (the paper's or a poster's figure, not re-run) or reasoned (my arithmetic on the other two).

Sorted, popping, or noisy

A 3DGS renderer colours a pixel by walking the Gaussians that cover it front to back. Gaussian $i$ has colour $c_i$ and, at this pixel, an alpha $\alpha_i$. Its weight is its alpha times the light that got past everything in front of it:

That needs an order. The original renderer sorts Gaussians once per tile by the depth of their centres, which is cheap but wrong whenever two Gaussians overlap in depth: nudge the camera, the centre order flips, and a pixel's colour jumps. That jump is popping. StopThePop removes it by sorting per pixel by the depth of each Gaussian's peak response, at a cost in speed. The sort itself is the other cost: millions of keys, every frame, which is the frame-time dial on phones.

Stochastic transparency throws the sort away. Draw every fragment opaque with depth testing on, but keep fragment $i$ only if a random number $u_i$ falls below $\alpha_i$. Fragment $i$ is the one you see when it survives and everything in front of it does not, which happens with probability $\alpha_i \prod_{k}(1-\alpha_k) = \alpha_i T_i$, exactly its blending weight. The z-buffer does the ordering for free. This is the StochasticSplats estimator the paper builds on; Blender 5.3 does the same thing with rays in Cycles, and PlayCanvas ships it as a WebGPU flag.

expand

One sample is one Gaussian's colour, not the blend. The expected value is right; any single frame is wrong by however far that Gaussian's colour sits from the blend. The paper writes the root-mean-square error of an $N$-sample estimate as

At one sample per pixel and uncorrelated noise between frames, this is not popping; it is a boiling, sparkling image. The usual answer is to raise $N$ by accumulating frames, which ghosts when the camera moves and undoes the speed you bought by not sorting. The paper's insight is to go after the other factor.

The variance term

Nothing in standard 3DGS training cares about $\operatorname{Var}[c_r]$. The photometric loss only sees $C$. So the optimiser is free to hit a pixel's olive green by stacking a pale yellow Gaussian, a near-black one and a pinkish one at half opacity, and it often does: semi-transparent "fuzz" layered over a surface is how 3DGS buys fine detail and view-dependent effects. A sorted renderer cannot tell that pixel from one made of four olive Gaussians. A stochastic renderer can, because every sample is one of those colours.

SteadySplats adds the variance to the loss (Eq. 7), averaged over the $H \times W$ pixels of the training view:

$P_{x,y}(i) = \alpha_i T_i$ is the blending weight the rasteriser already computes. The paper swaps the rendered colour $C$ for the ground-truth pixel $C^{\text{gt}}$, which is cheaper and, for a converged model, the same thing. The effect is that every Gaussian covering a pixel is pulled towards that pixel's colour.

expand

Here is that pixel as a toy. The slider walks between two fits I made up that blend to the same colour; the strip shows what a one-sample renderer would put on screen in successive frames.

The plain fit has a variance of 0.2885, which at one sample per pixel is a PSNR of about 10.2 dB against the sorted colour; the regularised fit has 0.0041, about 28.6 dB (reasoned, from the toy's own numbers, which are illustrative). Going from 1 to 64 samples buys $10\log_{10}64 \approx 18$ dB (reasoned). The variance term buys more than that here without drawing a single extra sample, which is the whole pitch.

Why it is cheap to train

Sameh says the term is "practically free", and the structure says why (reasoned). The forward pass already walks each pixel's Gaussians and has $P(i)$ and $c_i$ in registers; $\mathcal{L}_{\text{var}}$ adds one more running sum. The paper's gradients (its Section 13) are $\partial\mathcal{L}/\partial c_i = 2P(i)(c_i - C^{\text{gt}})$, and for $\alpha_i$ a term in $T_i$ minus a suffix sum over the Gaussians behind $i$. The 3DGS backward pass already walks back to front carrying exactly such a suffix sum for the colour gradient, so this is one more accumulator per pixel, not a new pass. I could not measure the overhead: the released training logs cover only the regularised second half of training, with no baseline run on the same machine to compare against.

What it does to the model, measured

The authors publish their $\lambda_{\text{var}} = 1$ models for seven Mip-NeRF 360 scenes, with the training config and TensorBoard logs. The original 3DGS bonsai model by Kerbl is in the dylanebert/3dgs dataset (revision 703d19c, the same file the Blender article sampled). Both use the same COLMAP frame: camera 0 in each cameras.json sits at the same position to 15 digits. I parsed both PLYs in full (measured):

The paper reports that the term "naturally acts as a binarization penalty on opacity" (Figure 11). On bonsai that is not subtle: the typical Gaussian goes from mostly transparent to nearly opaque.

To measure the variance itself I wrote a small CPU reference of 3DGS compositing in NumPy: project each Gaussian with the EWA Jacobian and the 0.3-pixel dilation, evaluate degree-3 spherical harmonics, sort each pixel's fragments by depth, composite front to back with the 3DGS early exit at transmittance $10^{-4}$, and accumulate $\sum_i P(i)\lVert c_i - C\rVert^2$ per pixel, counting the black background as a sample with whatever mass is left. It also draws one stochastic sample per pixel from $P$. I rendered test camera 0 (DSCF5565) at the training resolution, 1559 x 1039 pixels, on a shared 4-core VM, CPU only (measured):

expand

Variance down by a factor of 10.7 is 10.3 dB at one sample per pixel (reasoned). The predicted and simulated PSNRs agree, which is the check that the variance formula and the sampler are the same thing. The occluder count matters for the next section: SteadySplats' resampling weights need the Gaussians in front of a sample, and they only stay cheap when there are few. The term cut the expected occluders from 14.7 to 10.2, the direction the paper's Figure 16 shows. It also cut the number of fragments blended before early exit, from 48.6 to 41.8, where the paper's Figure 16 has that number going up with $\lambda_{\text{var}}$; my baseline is Kerbl's original model rather than the authors' own $\lambda_{\text{var}} = 0$ run, and this is one view of one scene, so I would not read much into it. Also not comparable: the paper's variance axis uses the ground-truth colour, which adds the model's own error, while mine uses the rendered colour, so mine is smaller by construction.

A caution on the bright specks in the original model's one-sample image. Many are Gaussians whose colours are far outside the blend: bright, mostly hidden layers that a sorted renderer attenuates to nothing and a stochastic one occasionally shows in full. The paper notes the term "reduces the number of bright primitives, which are often floater artifacts" (its Figure 15).

The rest of the pipeline

The variance term changes the model. Two more pieces change the renderer.

History-based spatial resampling. Neighbouring pixels see nearly the same Gaussians with nearly the same weights. So pixel $(x,y)$ also uses the sample its four neighbours drew, weighted by how likely that sample is at $(x,y)$ compared with where it was drawn: $w_{n\to x,y}(r) = P_{x,y}(r) / P_n(r)$, then normalised (self-normalised importance sampling). A sample from across an edge gets a small weight and does not blur it; a plain 3x3 blur would. The ratio splits into the Gaussian's own alpha at the two pixels, which is easy, and the change in what occludes it, which needs to know what is in front, which is exactly what a sort-free renderer does not have. The trick: keep a small per-pixel buffer of the $K$ distinct Gaussians sampled there recently, with their depths, and take the occlusion product over only those. With the variance term on, the few dominant occluders are the ones most likely to have been sampled, so a small buffer catches them.

expand

Temporal reuse as an EMA. The same weighting extended over past frames would need every old sample. The paper proves (Theorem 1) that if every history sample decays by a shared factor per frame, the full spatio-temporal estimator collapses to a two-number recursion per pixel; at steady state it is the familiar exponential moving average of temporal anti-aliasing, $\hat C^t = \delta\,\hat C^{t-1} + (1-\delta)\,\hat C^{t,\text{spatial}}$, with $\bar\delta = 0.70$ in the evaluation (reported). StochasticSplats' cumulative average and Gaussian Point Splatting's fixed EMA fall out as special cases. Instead of clamping the history colour as TAA does, which clips legitimate Monte Carlo tails, SteadySplats lowers $\delta$ smoothly when the reprojected history is far from the current $3\times3$ neighbourhood. Samples are drawn from spatio-temporal blue noise, offset per primitive, rather than white noise.

Results, against what

The comparison is to the other sort-free renderers, StochasticSplats and Gaussian Point Splatting, with sorted 3DGS as ground truth. StopThePop is in the related work, not in the tables: it is a sorted renderer and does not have this noise. All of the following is reported.

expand

On a camera-path benchmark of 21 trajectories and 3,152 frames over 7 Mip-NeRF 360 scenes, rendered at 1 sample per pixel and scored against sorted rendering of the same path (Table 1):

tPSNR is PSNR on frame-to-frame differences, the flicker metric. 31.437 minus 18.336 is 13.1 dB, which is the abstract's "13 dB" (reasoned). The ablation (Table 2) isolates the training term: $\mathcal{L}_{\text{var}}$ alone takes average PSNR from 18.336 to 25.561, a 7.2 dB gain (reasoned), and tPSNR from 15.235 to 22.423, more than spatial reuse alone (24.774) or temporal reuse alone (25.227). All three together are best. My bonsai frame, with the variance term and no reuse at all, sits at 26.7 dB, the same regime as that 25.561 (reasoned; one view of one scene, not their benchmark).

Speed is on an RTX 5090 in the authors' Vulkan renderer. Their Figure 7 puts plain stochastic rendering near 0.56 ms a frame and sorted blending near 1.11 ms, with spatial reuse rising slowly from about 0.7 ms and staying under the sorted line until the buffer size $K$ reaches 14, where shared memory fills and the cost jumps (reported, read off the plot). They use $K = 4$. Their "excess quality factor", the multiple of samples resampling is worth at equal time, peaks at 3.5 at low sample counts. And on convergence: with $|\mathcal N| = 4$, matching the PSNR of 31 samples at $K = 16$ takes 256 samples at $K = 1$.

expand

What it costs

Peak quality. The paper says so in its limitations: $\mathcal{L}_{\text{var}}$ "dilutes the training signal", giving "a slight decrease in peak quality metrics", and there is a floor near $\operatorname{Var} \approx 0.01$ below which quality collapses. Its Figure 5(a) plots deterministic PSNR against variance for each scene; the bonsai curve starts near 31.9 dB at $\lambda_{\text{var}} = 0$ (reported, read off the plot). The released bonsai model's own log records a test PSNR of 30.68 dB at iteration 30,000 (measured, from the published TensorBoard file). That is about 1.2 dB below the plot's unregularised point (reasoned), more than "slight", although the plot labels only its first and last points, so I cannot tell which of its points the released model corresponds to. The other released logs (measured):

Every config trains at the Mip-NeRF 360 convention, indoor scenes at half resolution and outdoor at quarter. The iteration times are the authors' GPU, not mine, and with no baseline log they say nothing about the term's overhead.

Bias. Plain stochastic transparency is unbiased. Resampling is not: the self-normalised estimator has $O(1/N)$ bias, the occlusion product misses occluders that were never sampled into the buffer (worst at silhouettes and thin structures), and a neighbour across a 16x16 tile border can borrow a Gaussian the tile never rasterised, which the paper bounds at alpha below 1/256. The paper still reports convergence to sorted 3DGS at an average L1 error under $10^{-4}$.

A changed model. The variance term is a training-time change, so you cannot apply it to a splat you already have. The renderer side (resampling, EMA, blue noise) works on any model, but its occlusion approximation leans on the model having few occluders, which is what the training term provides.

Sameh's browser version

There is no code from the authors yet; the project page says "Code coming soon". Sameh Khamis had not published his implementation either when I checked, so everything here is reported, read from his post and its 106-second video.

expand

The video shows a WebGPU app called splats that trains as well as renders. The capture is "SmallCity (full block)" with 2,500,000 trainable Gaussians, 40,000 updates and MCMC densification, plus a bilateral grid and pose optimisation. "Variance loss" is a checkbox with weight 1.00, the paper's $\lambda_{\text{var}}$; "Stochastic rendering" is a separate toggle in the viewer, at 0.75 render scale, 1230 x 500 pixels. So his "training is unaffected" is about cost; the model is trained with the term, as the paper requires.

On the 240 fps: the readout sits at exactly 240.0 FPS whenever training is not running and drops to the 20s to 40s while it is (37.7 FPS at step 11,700). A frame rate that flat and that round is most likely a 240 Hz display's vsync cap, not the renderer's throughput (reasoned). A reply asked exactly this and was not answered; he did say the GPU is a 3090 Ti. "Bigger difference with denser views" is not explained in the thread, so I will not guess.

Takeaways

• SteadySplats is about noise, not popping. The renderer it improves never sorts, so it never pops; it boils. If popping is your problem, the answer is still a per-pixel sort like StopThePop, or going sort-free and paying in noise.
• Most of the win is in the model, not the renderer. One loss term, already computable from the forward pass, pushes every Gaussian on a ray towards the pixel's colour. On bonsai it cut the measured per-pixel variance by a factor of 10.7 and turned the median Gaussian nearly opaque.
• It is not free in quality. The released bonsai logs 30.68 dB, and the paper itself names a variance floor and a peak-quality trade.
• Resampling is ReSTIR's idea in splat clothing, made cheap because the trained model has few occluders, and its temporal half is an EMA you already have.

What I could not check: I did not run the authors' renderer (unreleased) or reproduce any table; my compositor is an independent reference with no reuse, so it measures the training term only. Figure 7 and Figure 5(a) values are read off plots. Sameh's implementation is known only from his video.

Related: the sort-free trick and its noise on phones are in Gaussian-splat capture in practice; the ray-traced version is in Blender 5.3's Gaussian splats; what the opacity of a splat does for cleaning a feed-forward scene is in InfiniSplat v2; how splats are shipped to browsers at all is in the SOG format; and a lidar-fused capture you might render this way is in Hesai Kosmo.