# Spirula Studio deleted its dependencies instead of wrapping them
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/spirula-studio
> date: 2026-09-22
> tags: 3d, gaussian-splatting, performance, cuda
When a project describes itself as "one self-contained binary" covering camera
alignment through to 3D Gaussian Splatting training, my first move is to open
the CMake and count the `FetchContent_Declare` calls. That is where the seams
usually are: a launcher, a config schema, and six other people's repositories
doing the work.
[Spirula Studio](https://github.com/harry7557558/spirula-studio) has two.
`glfw` and `imgui`. That is the entire external fetch list.
**Updated 2026-10-06** for release v2026.10.6, which adds LiDAR scans as an
input, a region-of-interest editor and faster depth and normal maps for 360
cameras. What changed, and how the LiDAR path works in the code, is in
[Update: v2026.10.6](#update-v2026106) at the end.
This matters beyond tidiness. Every Gaussian-splatting pipeline I have had to
put into production died at the same place: not at the training, but at the
seams. A COLMAP that has to be built with the right CUDA arch. A PyTorch wheel
that fights the driver. A masking script that pulls a second CUDA stack in
beside the first. An `ffmpeg` that is on the machine but wrong. The model was
never the hard part; the bill of materials was.
So the interesting question about Spirula Studio is not whether it trains good
splats. It is whether "no Python, no PyTorch, no separate COLMAP install" is a
packaging claim or a rewrite.
## It is a rewrite
Read the module READMEs and the same sentence appears six times in slightly
different words: *"with no onnxruntime, no PyTorch and no converter."* Each
module says what it replaced and why:
- **`src/sfm/`** — *"A standalone SfM pipeline — images in, a COLMAP
`sparse/` model out — with a GPU compute backend (Vulkan + Slang) and no
heavy dependencies… It exists to replace the `colmap` subprocess."* The rule
it sets for itself is the one that makes this defensible: *"Port algorithms,
not code. COLMAP is the reference for behaviour and parameter defaults; the
implementation is ours, against our own data structures."* COLMAP's on-disk
formats stay the interchange format precisely so any one stage can be swapped
back for COLMAP's to bisect a failure.
- **`src/nn/`** — a general inference layer: its own Vulkan runtime, a device
tensor, an op set, host image I/O. *"It knows nothing about any model."* No
computation graph, deliberately: *"a module's forward pass reads as
straight-line C++ next to its PyTorch reference, and the arena's scoping is
the whole memory strategy."*
- **`src/sam/`** — SAM 2 / 2.1 / 3 on that layer, a Vulkan + Slang port of
[sam3.cpp](https://github.com/PABannier/sam3.cpp), itself a ggml port of
Meta's SAM 3. It exists *"to replace the Python/PyTorch subprocess the GUI
used to shell out to for masking."*
- **`src/video/`** — frame extraction without ffmpeg, on
`VK_KHR_video_decode_*`, with the containers, bitstream parsing and
reference-picture management written here because *"a hardware decoder does
entropy decoding, motion compensation and filtering. It does **not** track
reference-picture state, and Vulkan makes that the application's job."*
The checkpoints are the honest exception and the project is precise about it.
ALIKED's weights are *"COLMAP's, fetched from COLMAP's release URL and parsed
in process. We host nothing and convert nothing."* The ONNX reader is
`nn/io/Onnx.cpp`, a ~400-line varint walk of protobuf that reads *initializers
only* — the graph structure is hard-coded in C++, so nothing has to understand
an operator or a shape rule. The point of that is stated plainly: **parity**.
Both sides run the same bytes, so a difference against
`colmap feature_extractor --FeatureExtraction.type ALIKED_N16ROT` is yours and
can be bisected, "which is exactly how the two bugs below were found."
That is the tell of a real port rather than a reimplementation-by-vibes. The
modules report their parity as a number: MoGe-2's forward pass matches
onnxruntime on the same checkpoint to a relative L2 of **3.8e-6**, Metric3D
v2's to **1.5e-3**, and LoMa's three networks each match on the same bytes.
## The `-DSS_ENABLE_PATENTED=ON` line
Buried in the build instructions is the most grown-up licensing decision I have
seen in a hobby-scale repository:
"**Compiled only with `-DSS_ENABLE_PATENTED=ON`, which is OFF by default.** The
H.264 / H.265 / AV1 bitstream parsers are the one piece of this repository
carrying third-party patent exposure, and this is a GPLv3 tree. With the option
off, `src/video/` is neither compiled nor linked and every consumer falls back
to an external ffmpeg: the GUI says why in the dataset panel, and
`spirula sam video` / `extract` say so and exit."
Nine thousand lines of decoder, written, tested, and fenced behind a flag that
defaults to off, with the fallback wired and the user told why. The upside of
turning it on is stated too — roughly 15× faster frame extraction, a 127-second
1080p30 clip in ten seconds, masking that rides along on the same device pass,
and no ffmpeg to install.
## Where the numbers are
The feature list is the marketing surface: cross-vendor via Vulkan, one
strategy combining MCMC/IGS+/MRNF, 10 million SH3 Gaussians in 8 GB of VRAM
under quantized training, native 360 and equirectangular support, a modified
bilateral grid and PPISP for exposure and white balance. I have not run the
binary and have no measurement of my own to put against any of it.
What I can read is the engineering documentation, and there the numbers have
the shape of things someone actually measured and then had to live with. My
favourite is the one that argues *against* a speedup.
The mapping stage spends most of its time in bundle adjustment, and most of
those solves are provisional — a growth-phase refinement followed by more
growth, a merge-tree level followed by another level. Running them in fp32 is
worth 25–35% of the stage, and the README does the arithmetic for why:
*"on solves large enough to be arithmetic-bound, halving the bytes every kernel
moves is worth ~2×, and below a few dozen images the scalar makes no difference
at all because those solves are bounded by dispatch count instead."*
It is fp64 anyway. The reason is the one that only shows up if you run the same
job three times:
"The Schur and Jacobian kernels accumulate with floating-point atomics, whose
execution order is arbitrary, so no two solves agree in their last bits. At
fp64 that perturbation is ~1e-16 and never crosses a decision threshold — the
whole pipeline is reproducible run to run. At fp32 it is ~1e-7 and crosses them
constantly, and LM's accept/reject plus the mapper's filters turn a last-bit
difference into a different reconstruction."
The README reports three identical runs of a 379-image capture: **96.1 AUC@10
every time** in fp64, and **96.5 / 92.3 / 91.5** in fp32. Noisier *and* 2.7
points worse on average. The flag to take the speed anyway exists, `--ba-real-coarse float`,
and the default does not use it.
This is a familiar bruise. Non-determinism from atomic ordering is the thing
that makes a CUDA pipeline impossible to debug — you fix a bug, the number
moves, and you cannot tell whether you fixed it. Paying 30% to keep a pipeline
bit-reproducible is the right trade, and almost nobody makes it, because the
30% is visible in a benchmark and the reproducibility is not.
The second measurement in that family is about an abandoned design rather than
a scalar. An earlier "manage" loop repaired a model by growing it without a
bound: *"on a 5356-image capture it spent 65 minutes to merge three models and
recover 119 images, and ended with three near-copies of one reconstruction."*
That is in the README, describing code that has since been replaced, under the
switch that still turns it off.
## The 360 path
The claim I most wanted to check is the one about 360 cameras, because it is
where most pipelines cheat. The cheap version is: undistort the panorama into
six or ten pinhole faces, hand those to COLMAP, and call it support. Spirula
does not do that.
The whole thing hangs on treating the two lenses as a **rig** with known
geometry rather than as two unrelated cameras. `kind: dual-fisheye` says the
first two members are back-to-back lenses of one camera: the second turned 180°
about the image vertical, the baseline between them its only translation. The
repository reports how well that nominal geometry holds — Insta360 X, DJI Osmo
360 and a PortalCam's two fisheyes *"all calibrate within 0.8–1.4 degrees of
that rotation, so it is refined; the Osmo and the PortalCam, measured against
something metric, put the baseline within a millimetre of the axis"* — and then
uses it from the seed pair on, instead of waiting for each lens to register
independently.
Once a member is calibrated, registration changes shape: *"a candidate whose
frame has no lens placed yet brings the whole frame, and the frame — not a lens
— is what the PnP estimates: every calibrated member's 2D–3D correspondences go
into one pool, and one LO-RANSAC over that pool solves for the frame's pose…
scored against **all** the members, so a hypothesis that explains one lens and
contradicts the other nine loses to one that explains the frame."*
The rig-mate extension is the part with the most useful negative result.
Extending a verified pair to the other lens verifies **72–95%** of the time on
dual fisheyes and **6%** on a `.360`'s ten narrow views — so it is enabled for
the first and stopped for the second, with the measurement printed rather than
a heuristic asserted.
And the metric gauge is where the Osmo 360 gets special-cased by its own
deficiency rather than by name. Scale and up normally come from the
accelerometer through pre-integration and from the GPS log. What is missing is
refused *by its own uncertainty* rather than by a rule: *"a camera that only
pans gets up and no scale, a stale phone fix gets no GPS, a file with an
attitude stream but no raw gyro (the Osmo 360) pre-integrates from the attitude
instead."* The check that it worked is gravity: **9.82 m/s² within 0.3° of up**
on a 118-second walk, and the whole fit takes 0.3 s.
Then the uncertainties get printed and not enforced, for a reason worth
quoting at every team that has ever shipped a quality gate:
"Measured against a reference whose error is correlated — GPS drift — they
under-state the real error by 3.9–4.5×: on one flight a 2% gate on them passed
a 3.6% scale error. **A gate that passes what it exists to catch is worse than
no gate**, so they are printed as the lower bounds they are."
## What I would want before putting it in a pipeline
The gap is comparison. The SfM README's own unstarted list names it twice:
*"what is left is a scored comparison of the three frontends on a public dataset
— nothing here says which to reach for"*, and, separately, *"parity
benchmarking on ETH3D / IMC."* So the port is verified against onnxruntime and
COLMAP for *agreement*, which is the harder and more useful check, but nothing
says whether the finished reconstruction is as good as COLMAP's on a dataset
both have seen.
The support surface is also one person's. The README says so — *"developed and
maintained almost entirely by one person"* — and the git history says the same:
50 commits in the public tree, the most recent three on the day before I wrote
this, all titled "render: misc UI/UX fixes". Against that, 471,999 lines of
C++, Slang and CUDA across `src/`, of which roughly 286,000 is hand-written once
the generated kernel instantiations and the translation catalogues are taken
out.
The other thing I would want is a licence read before adopting. Spirula Studio
is **GPLv3**, not Apache or MIT, which is a different conversation from most of
the splatting ecosystem — gsplat is Apache-2.0, nerfstudio is Apache-2.0. If
you are shipping a product, that binary is a subprocess boundary you have to
think about, and `-DSS_ENABLE_PATENTED=ON` is a second one.
None of which changes the thing I keep coming back to. The reason a splat
pipeline is painful in production is that it is six stacks pretending to be
one. This is one stack. Somebody sat down and wrote the SfM, the inference
runtime, the video decoders and the segmentation model as compute shaders,
checked each against the thing it replaced to six decimal places, and then
refused the 30% speedup that would have made the result irreproducible.
## Update: v2026.10.6
Two weeks after I wrote this, release v2026.10.6 shipped. Its changelog leads
with three things: a region of interest for training, faster depth and normal
maps for 360 cameras, and LiDAR scans as an input. I cloned the tag (commit
`b9c8042`) and diffed it against `v2026.9.20`, the last tag before this
article. That is 86 commits and 776 files changed, +143,316 / −8,341 lines
(measured, `git diff --shortstat`). Two details from the log matter for the
"one person" caveat above. Harry Chen wrote 72 of the 86 commits, and eight
other people wrote the other 14 (measured). The LiDAR feature, the largest
single commit at +10,604 lines over 54 files, came from an outside contributor
(PR #101), with Harry Chen as co-author.
The release notes call LiDAR support *"experimental, tested on limited public
datasets"*. I still have not run the binary. Everything below comes from
reading the code and the notes committed with it.
### LiDAR: what it accepts
On the dataset screen, a laser scan is an input like photos and videos, and a
dataset can take several. The formats are:
- **E57**, read by `data/E57Reader.cpp`, 481 lines with *"no libE57Format and
no Xerces behind it"*. It walks the file's 1020-of-1024-byte pages and the
bit-packed point streams by hand. A file that names any codec except
`bitPackCodec`, the only one the standard defines, is refused.
- **LAS** 1.0–1.4, point formats 0–10, uncompressed. **LAZ is refused** with
a message. That is the dependency-deletion habit again: decompressing LAZ
would mean taking on LASzip.
- **PLY**, ASCII or binary, streamed. A `camera` element (PCL, ETH3D, NavVis)
gives the scanner's position.
- **XGRIDS** LCC Studio exports. Their `raw/` folder is already a COLMAP
dataset in the LAS file's frame, so it is kept as it is (`--mode keep`).
The committed note draws the whole path as one diagram:
```text
scan.e57 ──► photographs + their scan poses ─┐ (anchors.json)
scan.las ──► views rendered from where the ├─► SfM ─► spirula lidar ─► sparse/0 in the scan frame
scanner stood (no photographs) │ (fit, merge, + depths/ normals/
photos / videos ─────────────────────────────┘ seeds, tracks)
```
*(Spirula Studio, `docs/notes/lidar-alignment.md`, "The pipeline".)*
### LiDAR: how it gets into the reconstruction
The scan does not replace structure from motion. It **re-frames** what SfM
produces. Four steps do the work.
1. **Anchors.** An anchor is an image whose pose in the scan's frame is
already known. Anchors come from the scanner's own photographs, or, for a
scan with none, from views of the cloud *rendered* where the scanner stood:
E57 station poses, a PLY `camera`, or one position every 2 m along a
`.trajectory.las`.
2. **SfM runs as usual** over everything. The exception is a dataset whose
only images are the scanner's own photographs. Then the scanner's poses are
used directly and no SfM runs.
3. **A similarity fit per model.** For each SfM model with anchors, the fit
finds a rotation, a translation and a scale. Rotation is the chordal mean
of the anchors' relative rotations. Scale and translation come from RANSAC
over pairs of anchors, then least squares. Then a point-to-plane ICP pulls
the model's points onto the scan, thinned to 2M points, with a 7-parameter
Gauss–Newton step. ICP is **refused** if it moves the anchors more than 5%
of their spread. The note records a design that was tried and dropped:
fitting anchor centres alone (Umeyama) left the rotation 1.9° off on a
capture that walked nearly in a line, because the rotation about that line
is free. Fitting rotations first got 0.16° (reported).
4. **Seeds, depth and normals from the scan.** The scan is voxel-thinned to
500k points by default. For every image, the scan is rendered into a
1600 px depth map through that image's real lens, distortion included. A
seed counts as seen by an image when it lies on that depth to within 3% +
5 cm. Each seed keeps 12 of the images that see it as its COLMAP track, so
the trainer's initial cloud is the **laser's points**, not SfM's. SfM's own
points survive only where the scan has nothing, such as sky or anything
past the scanner's range. The same renders are written to `depths/` (16-bit
millimetres) and `normals/`, **in place of** the monocular MoGe or Metric3D
step.
The geometry the scan contributes is **metric**: the output model's
`gauge.txt` says `metric 1`. The release notes put it as *"recover metric
scale and compute depth and normal maps"*.
### LiDAR: how training uses it
This part is easy to over-read, so here is what the trainer actually does
with the scan's three products.
- **The seeds** are used by default. They are the initial Gaussians.
- **The normal maps** are used by default. `normal_supervision_weight` is
0.01 in `config/TrainConfig.h`.
- **The depth maps are not used** unless you ask. `depth_supervision_weight`
defaults to **0**, no preset in the tree sets it, and `TrainerCore` loads
depths only when that weight is above zero. When it is on, the loss in
`shaders/per_pixel_losses.slang` is a **Pearson correlation** between the
rendered depth and the reference. A correlation ignores scale and offset.
So inside the loss, a laser's millimetres count for no more than a
monocular network's relative depth. The metric part of LiDAR reaches the
splats through the frame and the seeds, not through the depth loss
(reasoned, from the code).
The note's own training check agrees that this is no quality jump yet. On its
FJD capture, 7k steps with every 8th image held out, the scan's seeds, normals
and depth gave held-out PSNR **17.27** and SSIM **0.533**. The same images on
the unaligned SfM model and its own points gave **17.37** and **0.526**. That
is *"within this capture's run-to-run noise (about 1 dB)"* (reported). The
alignment numbers are stronger. On ETH3D courtyard, a merged FARO scan with
no photographs, the aligned model lands **3.0 cm / 0.11°** from ground truth
(reported). What LiDAR buys today is a metric, gravity-aligned frame and a
dense seed cloud. It does not yet buy better pixels.
The note is just as frank about what is missing. **A model with no anchor is
lost.** On a NavVis capture, the largest SfM model (1,046 images) had no
anchor and was dropped (reported). Using the scanner's poses as priors inside
bundle adjustment, which the note calls *"the popular request"*, is listed
as not done yet.
### Region of interest
The trainer could already keep a run to a region of space (`--roi-region`),
but the region had to come from a partition or a hand-written JSON. What is
new is an editor, `app/gui/RoiEditor.cpp` at 2,271 lines (measured). In it you
build a region from boxes, ellipsoids, cylinders and extruded outlines. Each
shape is **added**, **cut** or **intersected**, applied in order from top to
bottom, so "everything except the crowd in the corner" takes one shape. The
editor has starting points ("cylinder around the subject" finds where the
cameras' optical axes come closest). The file it saves, `/roi/.json`,
is picked up by the trainer automatically.
The region does not hard-crop the scene. Splats outside it count for
`roi_outside_weight` (default **1e-4**) when densification picks where to add
or move splats, and `roi_mask_pixels` (default on) masks each training image
to what it shows of the region. Coordinates are written at full double
precision, because a geo-referenced model sits millions of units from its
origin, where `%.9g` would round them to centimetres.
### "Faster depth and normal maps for 360"
This one is a change of sizing, not a faster kernel (commit `a82d926`). A 360
frame is too distorted for a depth network, so it is split into overlapping
pinhole **faces**: a front face plus a ring of seven or eight, depending on
the lens. Before this
release each face was sized at the *source frame's* pixel density, so a
high-resolution fisheye ran roughly as many pixels as it was shot with, in
pieces. Now each face is sized to the *written map's* density, capped at
`--max-size` (default 1064), and raised to at least 1,200 patches (MoGe's
lowest token budget) where the frame has the pixels. In the README's words, a
high-resolution 360 capture *"costs what its map does, not what its sensor
does"*. The old sizing is still available as `--face-res source`.
How much faster depends on the camera, and the release gives **no number**.
The README points to a "Face size" section for the measurement, and no section
by that name exists at this tag. The README's own vit-large timing shows why
the lever is large: 260 ms at 616 px, 810 ms at 1064 px and 2,200 ms at 1600 px
(reported). Cost grows faster than pixel count, so shrinking faces cuts time
by more than it cuts pixels (reasoned). The same commit also blanks any pixel
that *any* face called sky (MoGe's mask), so a wrong normal never reaches the
loss.
No screenshots ship with this release: the GitHub release has binaries and a
changelog only, and the tag adds no images to the repository. The figure
earlier in this article is still the current GUI apart from the redesigned home
screen.
### October 11: the first scans from outside
Four days after the release, the Japanese account Duckbill Studio
[posted](https://x.com/DuckbillStudio/status/2108836147971686783) the first
third-party LiDAR results I have seen. Translated, the post says they merged
terrestrial-laser-scan E57 files with video and photos in Spirula Studio, got
"scale and position constraints", and generated normal and depth maps. They
used two rigs: a FARO Focus scan with DJI Osmo 360 video, and a Leica BLK360
scan with Sony A1 stills.
Neither pairing appears in the project's own measurements. In
`docs/notes/lidar-alignment.md`, the FARO data is ETH3D's merged scan with no
photographs, the BLK360 is tested alone, and the 360 video is an Insta360 X5
`.insv`, not an Osmo `.OSV`. So this is new evidence that the path works on
other hardware. It comes as screenshots only, with no residuals or held-out
scores, so it shows the alignment succeeded and not how well it did.
"Scale and position constraints" should be read the way the code does it. The
reconstruction is still pure structure from motion. Afterwards, the whole model
is moved onto the scan by the similarity fit and the point-to-plane ICP
described above, and given metric scale. Nothing constrains the camera poses
while bundle adjustment runs. Using the scanner's poses as priors inside BA is
still under "Not done yet" in the note at `af0de16`, the tip of the public tree
on October 11.
The follow-up post is the useful part. Translated: the initial point cloud for
the splats is generated from the E57, things go wrong when the photos and the
scan don't cover the same area, and the initial points for the distant
background, outside the scan's range, disappear too.
The code explains why. In `LidarDataset.cpp`, an SfM point survives the merge
only if most of the images that observe it see *no scan depth at all* at that
pixel. Each image's depth map is rendered from a draw of up to 40M scan points
(`kDrawCap`). The vote asks whether the map has any depth there (`d > 0`), not
whether that depth agrees with the SfM point's. A terrestrial scanner reaches
far but thinly. Its sparse distant returns still put some depth into many
background pixels, so the background's SfM points can lose the vote. Meanwhile
the seeds that replace them are thinned to 500k by default, and only a few of
those land far out. That combination is what removes a distant tree line. I am reading
this from the code and the post, not from their data. The CLI has two knobs
for it: `--points all` keeps every scan point as a seed, and `--no-gap-points`
goes the other way and drops the SfM points entirely.
The code has hardly moved since the release. Through `af0de16` there are 31
more commits, and the only ones that touch the LiDAR path are small. The
XGRIDS importer now finds LCC Studio's `nCore_data/` layout as well as
`developer_data/`, the seed cloud's double-precision PLY write moved to an
enum, and a header gained an include. The others cover GPS in SfM, RoMa v2
dense point clouds, a per-input colour LUT for video, AMD subgroup tuning, and
held-out evaluation after GUI training. I read both posts through the fxtwitter mirror
(the translations are mine), took the images from it, and diffed
`v2026.10.6` against `af0de16`; I still have not run the binary.
Checked by grepping `FetchContent_Declare` and `GIT_REPOSITORY` across
`CMakeLists.txt` and `cmake/*.cmake` at the tip of the public tree, and by
reading the six module READMEs. A vendored third-party tree under a path I did
not search, a submodule, or a runtime `dlopen` of a system library doing the
real work would overturn it. Note the scope: this is a claim about *code*, not
about weights — the checkpoints are all somebody else's and the project says so.
Three runs of one 379-image capture, reported by the project, not by me. Three
runs is thin. Run the same capture ten times at each scalar on a different GPU
and vendor: if fp32's spread collapses, the atomic-ordering story is specific to
one driver's scheduling and the default is over-cautious. If fp64 ever varies at
all, the reproducibility claim is wrong outright.
Read from `src/sfm/README.md` and `src/sfm/core/Rig.h`'s documentation, plus
the `.insv`/`.OSV` multi-track handling in `DatasetPrep.cpp`. If the frames are
in fact resampled to pinhole faces before matching — `app/GeometryWarp.h` does
exactly that for the depth models, so the machinery exists — then the rig is
operating on rectified crops and the distinction I am drawing is thinner than I
claim. Running `spirula sfm auto` on a real `.OSV` with `--rig` logging on
would settle it in one run.
An absence claim, and the project asserts it too — item 17 of its own unstarted
list is "parity benchmarking on ETH3D / IMC", and item 15 says "nothing here
says which [frontend] to reach for". A published ETH3D or IMC table, from the
author or anyone else, kills this immediately, and I would like it to.
This one is trivially falsifiable in the right direction: anyone with the
binary and one of those captures can reproduce or fail to reproduce it. I am
flagging it because the video in this article is evidence of what the software
looks like running, not evidence that the numbers burned into its captions are
right, and those two are easy to conflate.