~/satyajit

Spirula Studio deleted its dependencies instead of wrapping them

mdjsonmcp

2026-09-22 · 34 min · 3d · gaussian-splatting · performance · cuda

A 2:03 narrated explainer, drawn in code. Every number and picture in it is this article's own; the sources are below.

› transcript

Hi, I'm Hollyhock! Spirula Studio turns raw video into Gaussian splats in one binary. Here's how, without wrapping anyone's tools. The idea: instead of wrapping COLMAP, PyTorch and ffmpeg, Spirula rewrote each one on Vulkan. It borrows only checkpoints and parameter defaults. A 360 video comes in. Spirula's own decoders pull the frames on the GPU. No ffmpeg. SAM 3 masks the frames, running as compute shaders on the project's own runtime. No PyTorch. Structure from motion aligns the cameras, porting COLMAP's algorithms and defaults, not its code. It still writes COLMAP's format, so any stage can be swapped back to bisect a failure. Then training turns the aligned cameras into Gaussian splats. Here it runs, in the project's own reel: fisheye capture, masking, then structure from motion. The timings on screen are the project's claims, not measured here. Most pipelines cut a panorama into pinhole faces. Spirula treats the two fisheye lenses as one rig. Once a lens is calibrated, every calibrated lens's matches go into one pool. One RANSAC fit solves the whole frame's pose, scored against every lens. Then the file's own sensors fix scale and up. Gravity solved to nine point eight two. fp32 would save twenty-five to thirty-five percent of the stage. But atomics add in arbitrary order, and that last-bit noise flips decisions. In fp64 the same noise never crosses a threshold. Every run scores ninety-six point one. So: splat pipelines die at the seams. Spirula is one stack, each part checked against what it replaced. To recap: one Vulkan stack, a 360 camera solved as a rig, and fp64 so every run repeats. Every source is in the full article. I'm Hollyhock. Bye!

Why read this

Notabletop 60%

Checks whether a one-binary splat pipeline wraps or rewrites COLMAP, PyTorch and ffmpeg (it rewrites), why BA stays fp64, and how the new LiDAR path works.

  • Original analysis
  • Runs on a consumer GPU
  • A new technique

3D & spatialGPL-3.0Practitioner tool

How this was scored
Is it new?
2 of 3: A real new idea, method or capability
Can I trust it?
2 of 3: Measures key facts from files, code or configs
Can I run it?
2 of 3: Open code or weights with real limits
Will I understand it?
2 of 3: Mechanism from first principles with figures
Can I act on it?
2 of 3: A concrete recipe, numbers or comparison
Will it last?
1 of 3: Relevant for months
Does it affect many?
1 of 3: A specialist community
Only here?
2 of 3: A teardown or measurement few others did

Score 64 of 100, ranked 261 of 666 rated articles. Each question is answered 0–3 by hand, and a 3 is rare. How articles are scored

When a project describes itself as "one self-contained binary" covering camera alignment through to 3D Gaussian Splatting training, my first move is to open the CMake and count the FetchContent_Declare calls. That is where the seams usually are: a launcher, a config schema, and six other people's repositories doing the work.

Spirula Studio has two. glfw and imgui. That is the entire external fetch list.

An excerpt (frames 72–135 of 212) from the project's own hero animation, at its native 5 fps. The captions and the figures in them are the project's claims, not my measurements — I have not run the binary. The source is silent. (Spirula Studio, spirula.studio hero animation; GPLv3, licence and asset credits committed beside the clip.)

This matters beyond tidiness. Every Gaussian-splatting pipeline I have had to put into production died at the same place: not at the training, but at the seams. A COLMAP that has to be built with the right CUDA arch. A PyTorch wheel that fights the driver. A masking script that pulls a second CUDA stack in beside the first. An ffmpeg that is on the machine but wrong. The model was never the hard part; the bill of materials was.

So the interesting question about Spirula Studio is not whether it trains good splats. It is whether "no Python, no PyTorch, no separate COLMAP install" is a packaging claim or a rewrite.

It is a rewrite

what "no Python, no COLMAP" costs82,074 lines
dependency removedwhat replaced itlines
colmap (subprocess)
src/sfm/
still borrowed: COLMAP's behaviour and parameter defaults, as the reference
44,310
PyTorch + CUDA (inference)
src/nn/
still borrowed: nothing — Vulkan + Slang, its own tensor and op set
10,886
lang-segment-anything
src/sam/
still borrowed: Meta's SAM 2.1 / SAM 3 checkpoints, fetched on first use
7,962
ffmpeg (frame extraction)
src/video/
still borrowed: nothing — VK_KHR_video_decode plus its own demuxers
9,145
onnxruntime (features)
src/aliked/ + src/loma/
still borrowed: COLMAP's and LoMa's own ONNX exports, parsed in process
5,712
torch.hub + mmcv (depth)
src/moge/ + src/metric3d/
still borrowed: onnx-community and Ruicheng ONNX exports of MoGe-2 / Metric3D v2
4,059

What is left of Python in the repository is 39 files under reference/, kept as the behavioural reference the ports were checked against, plus the build's own codegen. Nothing in the shipped binary runs them.

Read the module READMEs and the same sentence appears six times in slightly different words: "with no onnxruntime, no PyTorch and no converter." Each module says what it replaced and why:

The checkpoints are the honest exception and the project is precise about it. ALIKED's weights are "COLMAP's, fetched from COLMAP's release URL and parsed in process. We host nothing and convert nothing." The ONNX reader is nn/io/Onnx.cpp, a ~400-line varint walk of protobuf that reads initializers only — the graph structure is hard-coded in C++, so nothing has to understand an operator or a shape rule. The point of that is stated plainly: parity. Both sides run the same bytes, so a difference against colmap feature_extractor --FeatureExtraction.type ALIKED_N16ROT is yours and can be bisected, "which is exactly how the two bugs below were found."

That is the tell of a real port rather than a reimplementation-by-vibes. The modules report their parity as a number: MoGe-2's forward pass matches onnxruntime on the same checkpoint to a relative L2 of 3.8e-6, Metric3D v2's to 1.5e-3, and LoMa's three networks each match on the same bytes.

The -DSS_ENABLE_PATENTED=ON line

Buried in the build instructions is the most grown-up licensing decision I have seen in a hobby-scale repository:

Nine thousand lines of decoder, written, tested, and fenced behind a flag that defaults to off, with the fallback wired and the user told why. The upside of turning it on is stated too — roughly 15× faster frame extraction, a 127-second 1080p30 clip in ten seconds, masking that rides along on the same device pass, and no ffmpeg to install.

Where the numbers are

The feature list is the marketing surface: cross-vendor via Vulkan, one strategy combining MCMC/IGS+/MRNF, 10 million SH3 Gaussians in 8 GB of VRAM under quantized training, native 360 and equirectangular support, a modified bilateral grid and PPISP for exposure and white balance. I have not run the binary and have no measurement of my own to put against any of it.

What I can read is the engineering documentation, and there the numbers have the shape of things someone actually measured and then had to live with. My favourite is the one that argues against a speedup.

bundle adjustment · three identical runs, 379 imagesAUC@10
9092949698run 196.196.5run 296.192.3run 396.191.5fp64mean 96.1spread 0.0fp32mean 93.4spread 5.0Same capture, same flags, same machine — only the scalar differs.fp32 would be worth 25–35% of the mapping stage.It is not taken: --ba-real-coarse float exists, and is not the default.

The mapping stage spends most of its time in bundle adjustment, and most of those solves are provisional — a growth-phase refinement followed by more growth, a merge-tree level followed by another level. Running them in fp32 is worth 25–35% of the stage, and the README does the arithmetic for why: "on solves large enough to be arithmetic-bound, halving the bytes every kernel moves is worth ~2×, and below a few dozen images the scalar makes no difference at all because those solves are bounded by dispatch count instead."

It is fp64 anyway. The reason is the one that only shows up if you run the same job three times:

The README reports three identical runs of a 379-image capture: 96.1 AUC@10 every time in fp64, and 96.5 / 92.3 / 91.5 in fp32. Noisier and 2.7 points worse on average. The flag to take the speed anyway exists, --ba-real-coarse float, and the default does not use it.

This is a familiar bruise. Non-determinism from atomic ordering is the thing that makes a CUDA pipeline impossible to debug — you fix a bug, the number moves, and you cannot tell whether you fixed it. Paying 30% to keep a pipeline bit-reproducible is the right trade, and almost nobody makes it, because the 30% is visible in a benchmark and the reproducibility is not.

The second measurement in that family is about an abandoned design rather than a scalar. An earlier "manage" loop repaired a model by growing it without a bound: "on a 5356-image capture it spent 65 minutes to merge three models and recover 119 images, and ended with three near-copies of one reconstruction." That is in the README, describing code that has since been replaced, under the switch that still turns it off.

The 360 path

The claim I most wanted to check is the one about 360 cameras, because it is where most pipelines cheat. The cheap version is: undistort the panorama into six or ten pinhole faces, hand those to COLMAP, and call it support. Spirula does not do that.

one .OSV file → a levelled, metric reconstructionno undistortion step
  1. container

    ISO-BMFF picked by content, not extension; the file's two video tracks are enumerated, never merged — a 360 camera's two fisheye lenses.

    one output folder per track

  2. decode

    VK_KHR_video_decode on the same device as everything else; the H.264/H.265/AV1 bitstream parsing and reference-picture state are Spirula's own.

    127 s of 1080p30 in 10 s · ~15× vs shelling out to ffmpeg

  3. rig

    kind: dual-fisheye — the second lens is the first turned 180° about the image vertical, and the baseline between them is its only translation (refined as axial).

    Insta360 X, Osmo 360 and a PortalCam all calibrate within 0.8–1.4° of that rotation; the Osmo's baseline lands within a millimetre of the axis

  4. matching

    --rig-pairs extends every verified frame pair to the other lens: cam0–cam0 brings cam1–cam1, cam0–cam1 brings cam1–cam0. A second, smaller verification pass over what the first confirmed.

    72–95% of rig-mates verify on the dual fisheyes measured; 6% on a .360's narrow views, which is why it stops there

  5. registration

    Once a member is calibrated the FRAME is what PnP estimates: every calibrated lens's 2D–3D correspondences go into one pool, one LO-RANSAC over all of it.

    a hypothesis explaining one lens and contradicting the other nine loses to one that explains the frame

  6. gauge

    The file's own sensors fix scale and up. The Osmo 360 carries an attitude stream but no raw gyro, so pre-integration runs from the attitude instead; the IMU-to-lens rotation is calibrated from the reconstruction itself.

    gravity solves to 9.82 m/s² within 0.3° of up on a 118 s walk; the whole sensor fit takes 0.3 s

  7. record

    sparse/N/gauge.txt states oriented and metric and which source settled each. A gauge a sensor settled is never overwritten by the guess it replaced.

    scale and orientation uncertainties are printed and never gated on — measured against GPS they under-state the real error 3.9–4.5×

The whole thing hangs on treating the two lenses as a rig with known geometry rather than as two unrelated cameras. kind: dual-fisheye says the first two members are back-to-back lenses of one camera: the second turned 180° about the image vertical, the baseline between them its only translation. The repository reports how well that nominal geometry holds — Insta360 X, DJI Osmo 360 and a PortalCam's two fisheyes "all calibrate within 0.8–1.4 degrees of that rotation, so it is refined; the Osmo and the PortalCam, measured against something metric, put the baseline within a millimetre of the axis" — and then uses it from the seed pair on, instead of waiting for each lens to register independently.

Once a member is calibrated, registration changes shape: "a candidate whose frame has no lens placed yet brings the whole frame, and the frame — not a lens — is what the PnP estimates: every calibrated member's 2D–3D correspondences go into one pool, and one LO-RANSAC over that pool solves for the frame's pose… scored against all the members, so a hypothesis that explains one lens and contradicts the other nine loses to one that explains the frame."

The rig-mate extension is the part with the most useful negative result. Extending a verified pair to the other lens verifies 72–95% of the time on dual fisheyes and 6% on a .360's ten narrow views — so it is enabled for the first and stopped for the second, with the measurement printed rather than a heuristic asserted.

And the metric gauge is where the Osmo 360 gets special-cased by its own deficiency rather than by name. Scale and up normally come from the accelerometer through pre-integration and from the GPS log. What is missing is refused by its own uncertainty rather than by a rule: "a camera that only pans gets up and no scale, a stale phone fix gets no GPS, a file with an attitude stream but no raw gyro (the Osmo 360) pre-integrates from the attitude instead." The check that it worked is gravity: 9.82 m/s² within 0.3° of up on a 118-second walk, and the whole fit takes 0.3 s.

Then the uncertainties get printed and not enforced, for a reason worth quoting at every team that has ever shipped a quality gate:

What I would want before putting it in a pipeline

The gap is comparison. The SfM README's own unstarted list names it twice: "what is left is a scored comparison of the three frontends on a public dataset — nothing here says which to reach for", and, separately, "parity benchmarking on ETH3D / IMC." So the port is verified against onnxruntime and COLMAP for agreement, which is the harder and more useful check, but nothing says whether the finished reconstruction is as good as COLMAP's on a dataset both have seen.

The support surface is also one person's. The README says so — "developed and maintained almost entirely by one person" — and the git history says the same: 50 commits in the public tree, the most recent three on the day before I wrote this, all titled "render: misc UI/UX fixes". Against that, 471,999 lines of C++, Slang and CUDA across src/, of which roughly 286,000 is hand-written once the generated kernel instantiations and the translation catalogues are taken out.

harry7557558/spirula-studio@af0de16 · snapshot 2026-10-11
tracked files
1,744
license
GPL-3.0
branch
master
tests
164 files
source
30.1 MB
commit date
2026-10-11
source by language
C++10.3 MB(492)CUDA9.8 MB(437)C9.0 MB(468)Python581.9 kB(55)JavaScript353.5 kB(18)Shell71.9 kB(21)HTML70.4 kB(3)

by size of tracked source at this commit, file counts in brackets; docs, data and vendored trees excluded

local clone, 2026-10-11 at af0de16 — branch, commit, commitDate, fileCount, hasTests, languages, license, licenseFile, shallow, testFileCount

shallow clone: counts describe the pinned tree, not the history

The Spirula Studio desktop application. A dark ImGui interface with a left panel of training parameters and a large viewport showing a Gaussian-splat reconstruction of an outdoor scene, with a training progress readout and camera frustums drawn over the splats.
The GUI. The same TrainerCore drives this and `spirula train` on the command line; the dataset parsers, the viewer server and the training driver were each once implemented twice, in Python and C++, and the Python halves were deleted after a parity gate proved the pairs agreed. (Spirula Studio, assets/screenshot.png; GPLv3, licence committed beside the image.)

The other thing I would want is a licence read before adopting. Spirula Studio is GPLv3, not Apache or MIT, which is a different conversation from most of the splatting ecosystem — gsplat is Apache-2.0, nerfstudio is Apache-2.0. If you are shipping a product, that binary is a subprocess boundary you have to think about, and -DSS_ENABLE_PATENTED=ON is a second one.

None of which changes the thing I keep coming back to. The reason a splat pipeline is painful in production is that it is six stacks pretending to be one. This is one stack. Somebody sat down and wrote the SfM, the inference runtime, the video decoders and the segmentation model as compute shaders, checked each against the thing it replaced to six decimal places, and then refused the 30% speedup that would have made the result irreproducible.

Update: v2026.10.6

Two weeks after I wrote this, release v2026.10.6 shipped. Its changelog leads with three things: a region of interest for training, faster depth and normal maps for 360 cameras, and LiDAR scans as an input. I cloned the tag (commit b9c8042) and diffed it against v2026.9.20, the last tag before this article. That is 86 commits and 776 files changed, +143,316 / −8,341 lines (measured, git diff --shortstat). Two details from the log matter for the "one person" caveat above. Harry Chen wrote 72 of the 86 commits, and eight other people wrote the other 14 (measured). The LiDAR feature, the largest single commit at +10,604 lines over 54 files, came from an outside contributor (PR #101), with Harry Chen as co-author.

The release notes call LiDAR support "experimental, tested on limited public datasets". I still have not run the binary. Everything below comes from reading the code and the notes committed with it.

LiDAR: what it accepts

On the dataset screen, a laser scan is an input like photos and videos, and a dataset can take several. The formats are:

The committed note draws the whole path as one diagram:

scan.e57 ──► photographs + their scan poses ─┐   (anchors.json)
scan.las ──► views rendered from where the   ├─► SfM ─► spirula lidar ─► sparse/0 in the scan frame
             scanner stood (no photographs)  │          (fit, merge,        + depths/ normals/
photos / videos ─────────────────────────────┘           seeds, tracks)

(Spirula Studio, docs/notes/lidar-alignment.md, "The pipeline".)

LiDAR: how it gets into the reconstruction

The scan does not replace structure from motion. It re-frames what SfM produces. Four steps do the work.

  1. Anchors. An anchor is an image whose pose in the scan's frame is already known. Anchors come from the scanner's own photographs, or, for a scan with none, from views of the cloud rendered where the scanner stood: E57 station poses, a PLY camera, or one position every 2 m along a .trajectory.las.
  2. SfM runs as usual over everything. The exception is a dataset whose only images are the scanner's own photographs. Then the scanner's poses are used directly and no SfM runs.
  3. A similarity fit per model. For each SfM model with anchors, the fit finds a rotation, a translation and a scale. Rotation is the chordal mean of the anchors' relative rotations. Scale and translation come from RANSAC over pairs of anchors, then least squares. Then a point-to-plane ICP pulls the model's points onto the scan, thinned to 2M points, with a 7-parameter Gauss–Newton step. ICP is refused if it moves the anchors more than 5% of their spread. The note records a design that was tried and dropped: fitting anchor centres alone (Umeyama) left the rotation 1.9° off on a capture that walked nearly in a line, because the rotation about that line is free. Fitting rotations first got 0.16° (reported).
  4. Seeds, depth and normals from the scan. The scan is voxel-thinned to 500k points by default. For every image, the scan is rendered into a 1600 px depth map through that image's real lens, distortion included. A seed counts as seen by an image when it lies on that depth to within 3% + 5 cm. Each seed keeps 12 of the images that see it as its COLMAP track, so the trainer's initial cloud is the laser's points, not SfM's. SfM's own points survive only where the scan has nothing, such as sky or anything past the scanner's range. The same renders are written to depths/ (16-bit millimetres) and normals/, in place of the monocular MoGe or Metric3D step.

The geometry the scan contributes is metric: the output model's gauge.txt says metric 1. The release notes put it as "recover metric scale and compute depth and normal maps".

LiDAR: how training uses it

This part is easy to over-read, so here is what the trainer actually does with the scan's three products.

The note's own training check agrees that this is no quality jump yet. On its FJD capture, 7k steps with every 8th image held out, the scan's seeds, normals and depth gave held-out PSNR 17.27 and SSIM 0.533. The same images on the unaligned SfM model and its own points gave 17.37 and 0.526. That is "within this capture's run-to-run noise (about 1 dB)" (reported). The alignment numbers are stronger. On ETH3D courtyard, a merged FARO scan with no photographs, the aligned model lands 3.0 cm / 0.11° from ground truth (reported). What LiDAR buys today is a metric, gravity-aligned frame and a dense seed cloud. It does not yet buy better pixels.

The note is just as frank about what is missing. A model with no anchor is lost. On a NavVis capture, the largest SfM model (1,046 images) had no anchor and was dropped (reported). Using the scanner's poses as priors inside bundle adjustment, which the note calls "the popular request", is listed as not done yet.

Region of interest

The trainer could already keep a run to a region of space (--roi-region), but the region had to come from a partition or a hand-written JSON. What is new is an editor, app/gui/RoiEditor.cpp at 2,271 lines (measured). In it you build a region from boxes, ellipsoids, cylinders and extruded outlines. Each shape is added, cut or intersected, applied in order from top to bottom, so "everything except the crowd in the corner" takes one shape. The editor has starting points ("cylinder around the subject" finds where the cameras' optical axes come closest). The file it saves, <dataset>/roi/<name>.json, is picked up by the trainer automatically.

The region does not hard-crop the scene. Splats outside it count for roi_outside_weight (default 1e-4) when densification picks where to add or move splats, and roi_mask_pixels (default on) masks each training image to what it shows of the region. Coordinates are written at full double precision, because a geo-referenced model sits millions of units from its origin, where %.9g would round them to centimetres.

"Faster depth and normal maps for 360"

This one is a change of sizing, not a faster kernel (commit a82d926). A 360 frame is too distorted for a depth network, so it is split into overlapping pinhole faces: a front face plus a ring of seven or eight, depending on the lens. Before this release each face was sized at the source frame's pixel density, so a high-resolution fisheye ran roughly as many pixels as it was shot with, in pieces. Now each face is sized to the written map's density, capped at --max-size (default 1064), and raised to at least 1,200 patches (MoGe's lowest token budget) where the frame has the pixels. In the README's words, a high-resolution 360 capture "costs what its map does, not what its sensor does". The old sizing is still available as --face-res source.

How much faster depends on the camera, and the release gives no number. The README points to a "Face size" section for the measurement, and no section by that name exists at this tag. The README's own vit-large timing shows why the lever is large: 260 ms at 616 px, 810 ms at 1064 px and 2,200 ms at 1600 px (reported). Cost grows faster than pixel count, so shrinking faces cuts time by more than it cuts pixels (reasoned). The same commit also blanks any pixel that any face called sky (MoGe's mask), so a wrong normal never reaches the loss.

No screenshots ship with this release: the GitHub release has binaries and a changelog only, and the tag adds no images to the repository. The figure earlier in this article is still the current GUI apart from the redesigned home screen.

October 11: the first scans from outside

Four days after the release, the Japanese account Duckbill Studio posted the first third-party LiDAR results I have seen. Translated, the post says they merged terrestrial-laser-scan E57 files with video and photos in Spirula Studio, got "scale and position constraints", and generated normal and depth maps. They used two rigs: a FARO Focus scan with DJI Osmo 360 video, and a Leica BLK360 scan with Sony A1 stills.

A top-down view of a coloured point cloud of a long, winding outdoor path through vegetation, with small orange camera frustums strung along the path. A counter in the corner reads 2146986 points.
FARO Focus E57 with DJI Osmo 360 video: the video's cameras, in orange, placed along the scanned path (Duckbill Studio, post on X, first image).
An oblique view of a dense green point cloud of trees around a round, two-tiered structure. Orange camera frustums form two rings around the structure and a long looping trail across the ground in front of it.
Leica BLK360 E57 with Sony A1 photographs: two rings of stills around the structure and a walking loop in front of it, all in the scan's frame (Duckbill Studio, post on X, second image).

Neither pairing appears in the project's own measurements. In docs/notes/lidar-alignment.md, the FARO data is ETH3D's merged scan with no photographs, the BLK360 is tested alone, and the 360 video is an Insta360 X5 .insv, not an Osmo .OSV. So this is new evidence that the path works on other hardware. It comes as screenshots only, with no residuals or held-out scores, so it shows the alignment succeeded and not how well it did.

"Scale and position constraints" should be read the way the code does it. The reconstruction is still pure structure from motion. Afterwards, the whole model is moved onto the scan by the similarity fit and the point-to-plane ICP described above, and given metric scale. Nothing constrains the camera poses while bundle adjustment runs. Using the scanner's poses as priors inside BA is still under "Not done yet" in the note at af0de16, the tip of the public tree on October 11.

The follow-up post is the useful part. Translated: the initial point cloud for the splats is generated from the E57, things go wrong when the photos and the scan don't cover the same area, and the initial points for the distant background, outside the scan's range, disappear too.

A top-down view of a sparse grey point cloud of paths and building edges, with a dense chain of orange spheres tracing two routes along the paths and loops off them.
From the follow-up post about coverage. The orange spheres are camera positions along the walked routes, and the dark areas around them are where the scan has few points (Duckbill Studio, follow-up post on X).

The code explains why. In LidarDataset.cpp, an SfM point survives the merge only if most of the images that observe it see no scan depth at all at that pixel. Each image's depth map is rendered from a draw of up to 40M scan points (kDrawCap). The vote asks whether the map has any depth there (d > 0), not whether that depth agrees with the SfM point's. A terrestrial scanner reaches far but thinly. Its sparse distant returns still put some depth into many background pixels, so the background's SfM points can lose the vote. Meanwhile the seeds that replace them are thinned to 500k by default, and only a few of those land far out. That combination is what removes a distant tree line. I am reading this from the code and the post, not from their data. The CLI has two knobs for it: --points all keeps every scan point as a seed, and --no-gap-points goes the other way and drops the SfM points entirely.

The code has hardly moved since the release. Through af0de16 there are 31 more commits, and the only ones that touch the LiDAR path are small. The XGRIDS importer now finds LCC Studio's nCore_data/ layout as well as developer_data/, the seed cloud's double-precision PLY write moved to an enum, and a header gained an include. The others cover GPS in SfM, RoMa v2 dense point clouds, a per-input colour LUT for video, AMD subgroup tuning, and held-out evaluation after GUI training. I read both posts through the fxtwitter mirror (the translations are mine), took the images from it, and diffed v2026.10.6 against af0de16; I still have not run the binary.

Update: training on a PS5

On October 13, Gradeeterna posted a 51-second video of Spirula Studio "training Gaussian splats on a PS5… running natively on the console's AMD GPU via the new PS5 Vulkan port", on a capture of Impasse Saint Claude in Paris shot with an Insta360. A reply links the two repositories. Neither names the other: at d6d0d85, Spirula's tip on October 14, no file mentions the PS5, and the driver never mentions Spirula (measured, git grep). The controller-driven front end in the video is in neither public tree.

A dark menu screen headed 'Spirula Studio, Gaussian splatting on PlayStation 5 Demo'. Under 'Train a dataset (30,000 steps, up to 3M splats, half resolution)' are ColmapData, 520 images, and ImpStClaude, 486 images. Under 'View a trained model' are four saved models, the largest at step 20000 with 3999k splats. A 'Custom' entry shows a train command line read from args.txt.
The console front end: two datasets to train and four saved models to view. Everything on screen is the poster's, not mine (Gradeeterna, video in post on X, frame at 0:03).

What the port is

PS5_Vulkan, by Mihawk-99 under GPL-3.0-or-later, holds two Vulkan drivers. The first, ps5vk, is the author's own Mesa-derived driver: 23 C files, 20,677 lines (measured), reporting Vulkan 1.1. The one that matters here is the second, Mesa's RADV, the stock Linux driver for AMD GPUs, built from the author's Mesa 26.2.0 fork with a new window-system layer for the console. RADV, its NIR passes and the ACO shader compiler stay as upstream ships them except where the console differs. The PS5 winsys takes GPU memory from the console's direct-memory pool. It copies each command stream into the graphics ring through sceAgcDriverSubmitDcb, the same call AnyPS5 reimplements in the other direction, and the CPU waits on a sequence number each submission writes.

The repository's findings describe the GPU as a hybrid. Its fixed-function blocks behave like GC 10.1.3, the family of AMD's BC-250 board, while shaders ACO compiles for GFX10.3 (RDNA 2) run. The winsys presents a Navi21 and switches off what this chip lacks: variable-rate shading, per-primitive mesh outputs, hardware task-shader dispatch. Mesh and task shaders are rebuilt as compute passes. Compute and transfer queue families exist, but they are served from the graphics ring, so there is no real async compute yet.

RADV reports Vulkan 1.4. The run log records the second full run of the pinned Khronos CTS (vulkan-cts-1.4.6.2) ending on September 29 with 2,919,757 cases: 1,569,390 pass, 0 fail, 1,350,306 not supported, 60 quality warnings, and one crash the author caused by closing the test title mid-batch (reported, docs/RADV_PHASE.md). Not supported is 46% of the run (measured from those figures), and docs/CTS_GAPS.md gives every gap the port caused an owner. The README is careful about the word: conformanceVersion stays 0.0.0.0, the project is not a Khronos Adopter, and a clean run on one console is not conformance.

Running any of it requires a modified console. That is all this article will say about it.

Why Spirula fits

Spirula's trainer has two interchangeable backends, CUDA and Vulkan + Slang, and the SfM, inference and video modules in the ledger above are Vulkan only. On an AMD console only Vulkan applies, and what it asks of a device is short. VulkanContext.cpp refuses a GPU without Vulkan 1.2, bufferDeviceAddress and timeline semaphores, or with compute subgroups wider than 64 lanes; everything else is a shader variant picked at load time (read from the code at d6d0d85). That rules out ps5vk at 1.1 and admits RADV at 1.4, so the video ran on RADV (reasoned).

One variant matters. Spirula's own comment says native fp32 buffer atomic add starts at RDNA 3, and that RADV trains correctly on older GPUs through compare-and-swap loops. On this RDNA 2 shader core the trainer should be running that emulation, not the .atomicadd blobs (reasoned).

Memory is not the constraint. The port measured a 12 GiB direct-memory pool (reported), and the demo caps training at 3M splats. A Gaussian at SH degree 3 is 59 fp32 parameters, 236 bytes. With a gradient and two Adam moments it is about 944 bytes, so 3M splats is about 2.8 GB before images and sort buffers (reasoned), with no need for the quantized training behind the feature list's 10M in 8 GB. The GPU has 36 compute units, the public figure the port's notes keep. Where the console should hurt is the CAS-loop atomics, and fp64 if anyone runs the bundle adjustment there, since RDNA 2 runs fp64 at a small fraction of its fp32 rate (reasoned). The video does not show SfM; the menu starts from datasets that already exist.

What the video shows

The post's text carries no numbers. The on-screen HUD does, and everything in this section is read off its frames (reported).

A rendered street view of a narrow Paris lane with cream-coloured buildings, bicycles and motorbikes parked on the left and a hedge-topped wall on the right. A banner across the top reads 'Spirula Studio, ImpStClaude, Step 22,048 / 30,000, 3,000,000 splats, PSNR 34.8, ETA 29 min, 8 fps'.
Training on the console at step 22,048 of 30,000. The PSNR is the trainer's running figure, not a held-out score (Gradeeterna, video in post on X, frame at 0:25).

The run trains at half resolution on 486 images. At step 22,048 of 30,000 it holds 3,000,000 splats with 29 minutes left. The clip is edited, and the ETA reads 25 minutes at step 51 and 61 at step 931, so it is an estimate that moves; the video gives no wall-clock time for a run. A finished model, ImpStClaude-2, is then flown through at 2,999,961 splats, SH degree 3, 60 fps. That figure is probably the display's cap, not the renderer's limit (reasoned).

The driver's architecture, licence and CTS log are what its repository says at 252bf91 (10 October); I read it and ran nothing. That training works on the console, and how fast, rests on one 51-second edited video.

16 October: v2026.10.16

Release v2026.10.16 is the PS5 section's tree with the version bumped: against d6d0d85 it changes two lines (measured). Since v2026.10.6 it is 43 commits and +55,074 / −4,967 lines, 28 of them by Harry Chen (measured). The release page returned 403 to my sandbox, so this comes from the commits and their notes. I ran nothing.

SfM memory (8ea0744). Data structures changed, not matching: matches.bin comes out byte for byte the same. Descriptors are memory-mapped from the feature files rather than copied to the heap. Pair selection keeps each image's best partners in a bounded heap, n x k edges where it held all n^2/2 pairs twice. A verified match costs 8 bytes in its pair list and 8 in the correspondence graph, down from 12 and 16, and past 256 MB both go to a spill file mapped back, so the OS pages them out instead of the run being killed. The bundle adjustment's Jacobian is split across up to four buffers: one buffer past NVIDIA's 4 GB binding limit was not refused, its reads wrapped, and large solves silently failed. A solve that still does not fit stores Jacobians at fp32 and keeps accumulating in fp64. On a 4000-image dual-fisheye capture, peak heap fell from 11.4 GB to 5.1 GB, and fp32 Jacobians took VRAM from 6764 MB to 4027 MB. A 12,580-image Osmo 360 walk, once killed at 28 GB of heap, now registers whole in 1 h 31 (all reported, RTX 5070, src/sfm/README.md).

Log footage (abdc29f8, PR #168). The LUT is applied at frame extraction, so SfM, masking and training all see Rec.709 frames. On the built-in Vulkan decoder it runs in the GPU colour conversion, before rounding to 8 bits, so a 10-bit log clip keeps its precision; the ffmpeg fallback applies it to 8-bit RGB. Auto detection reads only the file's metadata, never colour tags: DJI's ColorMode in its djmd protobuf (2 is D-Log, 19 D-Log M), GoPro's PTCL, Sony's CaptureGammaEquation, Panasonic's CaptureGamma. Insta360's I-Log marker has not been seen, so .insv is chosen by hand. Each built-in LUT is the maker's published curve plus a 129-number fitted look, within 0.23 mean / 1.0 p99 8-bit levels of DJI's own D-Log M LUT (reported, docs/notes/log-luts.md). It matters because SIFT keeps a keypoint only when its difference-of-Gaussians response clears a fixed 0.02 / 3, and log footage is flat, so fewer clear it; a trainer fed log frames would also learn grey splats (reasoned).

GPS (01fe940, d6d0d85). The fit's inlier radius was a fixed 5 m; it now comes from the fixes' own scatter, so RTK is judged in centimetres and a phone in metres. The bundle adjustment's position prior follows: its sigma is floored at 0.6 of that radius, not 3 m. The gate on each new registration widens with distance r from the fitted positions, by sqrt(1 + (r/R)^2) for R their spread, and a far-off pose is refused once, not for good. Fixes at exactly 0, 0, what a receiver without a fix writes, are dropped (read from the code).

Duckbill Studio, in the thread (translation mine): "It picked D-Log M automatically: some log videos are detected and the LUT is applied at frame extraction (log .osv is easier now; for .insv I chose the LUT by hand)… The big loop closed in one go (I haven't A/B-tested, so I can't say it was the improvement)." One anecdote, no control. The widening gate is a plausible cause, not a shown one (reasoned).

A top-down sparse point cloud on a dark background with a long chain of orange camera markers tracing one closed loop with several lobes. A counter in the corner reads 40818 points.
The loop that closed on v2026.10.16. Whether the GPS changes closed it is untested, by the poster's own account (Duckbill Studio, second image in post on X).

What would change my mind

5 claims above, and what would falsify each

  1. Spirula Studio wraps almost nothing: its only external build-time fetches are glfw and imgui, and the components it replaced — COLMAP, PyTorch, ffmpeg, onnxruntime — were rewritten rather than vendored.

    Checked by grepping FetchContent_Declare and GIT_REPOSITORY across CMakeLists.txt and cmake/*.cmake at the tip of the public tree, and by reading the six module READMEs. A vendored third-party tree under a path I did not search, a submodule, or a runtime dlopen of a system library doing the real work would overturn it. Note the scope: this is a claim about code, not about weights — the checkpoints are all somebody else's and the project says so.

  2. Keeping bundle adjustment in fp64 buys bit-reproducibility, and fp32 costs 2.7 points of mean AUC@10 with a five-point spread across identical runs.

    Three runs of one 379-image capture, reported by the project, not by me. Three runs is thin. Run the same capture ten times at each scalar on a different GPU and vendor: if fp32's spread collapses, the atomic-ordering story is specific to one driver's scheduling and the default is over-cautious. If fp64 ever varies at all, the reproducibility claim is wrong outright.

  3. The dual-fisheye rig path is real 360 support — the two lenses are reconstructed as a rig with known relative geometry, not undistorted into pinhole faces first.

    Read from src/sfm/README.md and src/sfm/core/Rig.h's documentation, plus the .insv/.OSV multi-track handling in DatasetPrep.cpp. If the frames are in fact resampled to pinhole faces before matching — app/GeometryWarp.h does exactly that for the depth models, so the machinery exists — then the rig is operating on rectified crops and the distinction I am drawing is thinner than I claim. Running spirula sfm auto on a real .OSV with --rig logging on would settle it in one run.

  4. Nothing establishes that Spirula Studio's finished reconstructions are as accurate as COLMAP's; the verification is parity of components, not of results.

    An absence claim, and the project asserts it too — item 17 of its own unstarted list is "parity benchmarking on ETH3D / IMC", and item 15 says "nothing here says which [frontend] to reach for". A published ETH3D or IMC table, from the author or anyone else, kills this immediately, and I would like it to.

  5. The hero animation's captions — 5,374 images in 25 minutes on an RTX 5070 Laptop GPU, 10 M SH3 Gaussians in 8 GB — are the project's claims and are unverified here.

    This one is trivially falsifiable in the right direction: anyone with the binary and one of those captures can reproduce or fail to reproduce it. I am flagging it because the video in this article is evidence of what the software looks like running, not evidence that the numbers burned into its captions are right, and those two are easy to conflate.

Cite this article

For attribution, please use the following reference or BibTeX:

Satyajit Ghana, "Spirula Studio deleted its dependencies instead of wrapping them", ai.thesatyajit.com, September 2026.

bibtex
@misc{ghana2026spirulastudio,
  author = {Satyajit Ghana},
  title  = {Spirula Studio deleted its dependencies instead of wrapping them},
  url    = {https://ai.thesatyajit.com/articles/spirula-studio},
  year   = {2026}
}
share