PS5LM: a 27B model on a PlayStation 5's GPU, checked against its memory bus

mdjsonmcpcopy↗ai

2026-10-15 · 15 min · llama-cpp · gguf · quantization · on-device · inference · gpu · qwen · linear-attention

Why read this

Notable· top 60%

The PS5 Qwen 27B claim checked: 2-bit not 4-bit, bytes per token from the GGUF tensor tables, the heap planner redone, and why the CPU run was the SSD.

• Original analysis
• Widely used
• A new technique

·Inference & serving·Runs on a consumer GPU·GPL-3.0-or-later·Practitioner teardown

How this was scored

Is it new?

2 of 3: A real new idea, method or capability

Can I trust it?

2 of 3: Measures key facts from files, code or configs

Can I run it?

1 of 3: API-only, gated or restrictive licence

Will I understand it?

2 of 3: Mechanism from first principles with figures

Can I act on it?

2 of 3: A concrete recipe, numbers or comparison

Will it last?

1 of 3: Relevant for months

Does it affect many?

2 of 3: A widely used model, tool or lab release

Only here?

2 of 3: A teardown or measurement few others did

Score 63 of 100, ranked 255 of 577 rated articles. Each question is answered 0–3 by hand, and a 3 is rare. How articles are scored

Two posts on X on 14 October said the same thing in different registers. One said it is possible to run "Qwen 3.8 27b @ ~20 tok/s on a jailbroken PS5's GPU" and linked cobanov/PS5LM. The other added that "the same model on the PS5's CPU did about one token a minute" and linked abhi245y/PS5LM. Two repositories with the same name, two numbers, and a gap between them of about 1,200x.

I cloned both, read them, and did not run anything. Running any of it needs a modified console; that is all this article says about that. What follows is the software stack, how a 27B model fits into a games console's memory, and whether 20 tokens a second is what the memory bus allows.

Every number is labelled: measured (I computed it from a file), reported (the project's figure, not re-run) or reasoned (my arithmetic on the other two).

Which repository is which

cobanov/PS5LM, by Mert Cobanov, is the original. GitHub lists it as a source repository created on 1 October; abhi245y/PS5LM is a fork created on 9 October (both reported by the GitHub API). The fork's README says so in its second paragraph: the llama.cpp port, the first payload build, its model library and the research notes are Cobanov's, and the fork adds the native GPU app, renamed PS5LM Plus on 13 October. The fork offered that app back upstream as pull request #12, and at the commit I cloned, f2ee6ba from 15 October, the original's head is the merge of that pull request.

So both posts point at the same work, at two stages:

The original's README was written for the CPU build and still leads with it ("small models for now (up to 1.6 GB)"), with one paragraph on the GPU app. That is why one reply under the first post said the project runs on the CPU only: it was reading the top of a README that had moved on. The fork's README is the one with the 27B table.

One reply also linked brysontang/iiama2.c. That is Karpathy's llama2.c ported to a Nintendo Wii, running TinyStories-sized models; it shares a joke with PS5LM and nothing else.

The stack, bottom to top

PS5LM does not write GPU kernels. Its README makes that the point: earlier console work, ProsperoAI in particular, hand-writes kernels for one model and one quantization at a time, while PS5LM ports llama.cpp whole so that every architecture and every GGUF quant comes along. The layers are:

1. The GPU driver. PS5_Vulkan by Mihawk-99: Mesa's RADV with a PS5 window-system layer that takes GPU memory from the console's direct-memory pool and submits command streams through the console's own graphics interface. I covered it in the 14 October update to the Spirula Studio article, where the same port trains Gaussian splats: it reports Vulkan 1.4 and presents the chip as a Navi21. PS5LM uses exactly that port; its build script links libvulkan_radeon.ps5.a from PS5_Vulkan's RADV build (read from scripts/build-app.sh).
2. No Vulkan loader. The app links RADV in whole and defines vkGetInstanceProcAddr itself, forwarding to the driver's ICD entry point, so ggml's dispatcher finds every command without a loader (ps5/app/main.cpp).
3. One heap. RADV normally splits an APU's memory into a device-local and a host-visible heap. The app sets radv_enable_unified_heap_on_apu=true before anything starts, and RADV then reports one device heap of 11.44 GiB instead of 8 GiB (reported, docs/CONSOLE.md).
4. The inference engine. Stock llama.cpp at 552f18f, built with -DGGML_VULKAN=ON and AVX2 for the CPU side, plus small patches for the console's libc and threading. llama-server runs on its own thread and serves the chat page and an OpenAI-compatible API on port 8081.
5. The app around it. A dashboard on the TV drawn by a separate homebrew UI toolkit through a GL port, in the same process as RADV. The two Mesa builds share 4,399 symbol names, so the GL stack is linked with only its API visible (reported).

What is missing matters for speed. PS5LM's research notes say RADV on this chip has what ggml-vulkan requires (Vulkan 1.2, fp16, 16-bit storage, subgroups, a 4 GiB storage-buffer range) but no cooperative matrix and no accelerated integer dot product, so matrix products take ggml's fp16 shaders. That is the generic path, not the fast one. Keep it in mind for the speed section.

This is the same answer the AnyPS5 relinker reaches from the other side. AnyPS5 turns a PS5 game into a Linux program and has to emulate the GPU command stream; PS5_Vulkan keeps the console and swaps the graphics API on top of it. Both meet at the console's command submission.

Why a 27B fits at all

The console has 16 GB of GDDR6 shared by its CPU and GPU, and the system keeps its share. PS5LM measured what a program actually gets, two ways.

A background payload, which is how the first CPU build ran, gets a malloc ceiling of 6.25 GiB and no direct memory at all, and holding about 6 GiB froze the home screen (reported). An installed app gets the console's 12 GiB of direct memory, which RADV turns into that 11.44 GiB heap. With the dashboard up, 11.01 GiB is free for the model (reported, from the app's logs as the repo quotes them).

Qwen3.8-27B is a dense model with a hybrid layout: 64 layers, of which every fourth is full attention and the other 48 are Gated DeltaNet layers, a linear attention with a fixed-size recurrent state (measured, from the GGUF metadata: block_count 64, full_attention_interval 4). That layout is why long context is cheap here. Only 16 layers keep a KV cache, with 4 KV heads of 256 dimensions each, so a token of context costs

which is 64 KiB at f16, 34 KiB at q8_0 and 18 KiB at q4_0 (reasoned, from the metadata). The DeltaNet state is a constant 0.146 GiB in f32 whatever the context (reasoned, with the planner's own formula). If this is new, the state-space architecture page covers why a recurrent layer has no cache that grows.

The weights are the problem. Unsloth publishes the GGUFs PS5LM uses, and the file sizes are (measured, from the Hugging Face listing):

Nothing at 3 bits or above fits. The PS5 runs this model at 2 bits, and the posts did not say so. That is not a footnote: one reply under the first post argued that a 27B at IQ2_XXS does worse than the 9B sibling at Q6. I have no evaluation to settle that, and neither repository contains one. It is the question to ask before calling this "a real 27B".

The planner, redone

The app chooses context length and cache type for you. ps5/app/model_plan.cpp adds four terms and checks them against the free memory:

It walks contexts from long to short, capped at 64k by default, tries q8_0 then q4_0 at each length (f16 only up to 8k), and takes the first that fits. The state is counted twice, once on its own and once inside the reserve; whether that is deliberate headroom the code does not say.

I redid it by hand against the screenshots. For UD-IQ2_XXS at 64k with a q8_0 cache: 6.767 + 2.125 + 0.146 + 0.896 = 9.93 GiB, and the app's own panel says "64k context, q8_0 cache: 9.9 of 11.0 GiB" (reasoned, then matched to the reported figure). For UD-Q2_K_XL the 64k options both miss (11.32 GiB at q4_0), 32k at q8_0 misses too, and 32k at q4_0 fits at 10.76 GiB. That is the row in the README's table, "32k, q4_0 cache" (reasoned, matched). The planner's comment says Q2_K_XL did run at 64k with 11.38 GiB free, which is the same arithmetic without the dashboard.

expand

Is 20 tokens a second what the bus allows?

At batch 1, generating a token reads every weight once. Arithmetic is cheap next to that, so the ceiling is

where $B$ is memory bandwidth and $R$ the bytes read per token. The PS5's GDDR6 peaks at 448 GB/s (reported; it is Sony's figure, which the project's notes repeat, with 576 GB/s on the Pro).

$R$ is not quite the file size. The token-embedding table is read one row per token, and the multi-token-prediction block is not used by this llama-server configuration. I read the tensor table from the header of each GGUF with HTTP range requests, took each tensor's size from its offset, and subtracted those two (measured):

The ceilings are reasoned. The project's own roadmap estimates "around 45 tok/s" for Q2_K_XL from the file size, which agrees.

The obvious back-of-envelope assumes a 4-bit model: about 16 GB a token, a ceiling near 28 tok/s, and 20 tok/s at an impressive 70% of peak. That is not what ran. The reported runs are:

• UD-IQ2_XXS, 64k context, q8_0 cache: 20.7 tok/s (reported, README table). On the dashboard screenshot: 20.9 now, average 20.4, minimum 14.9, maximum 21.2 (reported).
• UD-Q2_K_XL, 32k context, q4_0 cache: 20.3 tok/s, and a 7,775-token answer at 19.7 tok/s (reported).

Against the ceilings, the IQ2_XXS run uses 32% of peak bandwidth, about 142 GB/s, and the Q2_K_XL run uses 42%, about 190 GB/s (reasoned). Plausible, then, and comfortably so. Twenty tokens a second at 2 bits is not a sign of anything exotic.

expand

The interesting part: both quants run at the same speed

UD-Q2_K_XL reads 37% more bytes per token than UD-IQ2_XXS (measured), and decodes at 20.3 tok/s against 20.7 (reported). If memory bandwidth were the binding limit, the bigger file would be about 27% slower. It is 2% slower (reasoned). So at these sizes the PS5 is not bandwidth-bound; something else costs about 48 ms per token.

I can name candidates, not measure them:

• Dequantization on the generic path. Both files are mostly i-quants (measured, from the tensor types: IQ2_XXS and IQ1_S lead the smaller file, IQ3_XXS, IQ2_S and IQ3_S the larger). i-quants decode through codebook lookups, and with no integer dot product on this driver ggml takes its fp16 shaders. That is ALU and latency work that does not shrink with the file.
• Dispatch overhead. The IQ2_XXS file has 851 tensors (measured). One token is a long chain of small matrix-vector dispatches, through a driver that serves every queue from the one graphics ring (the port's own notes, as I read them for the Spirula article).
• The DeltaNet layers. 48 recurrent layers do a sequential state update per token that a pure attention model does not.

The dashboard reads 100% GPU busy and 86 W for the SoC, 4.1 J per token (reported), which fits a GPU that is working, not one waiting on memory. The honest summary: the reported speed is consistent with the hardware, sits at a third to two fifths of the bandwidth ceiling, and the remaining gap is compute or overhead that a profile would have to split.

Two outside reference points, both reported by others and neither re-run: PS5LM's research notes quote ProsperoAI's hand-written kernels running Qwen3.5 9B at Q4_0 (6.37 GiB) at about 30 tok/s, which is at most about 205 GB/s; and the PS5LM Plus table has Llama 3.2 3B at Q4_K_M (1.9 GiB) at 124 tok/s, about 253 GB/s, or 56% of peak (reasoned). Generic llama.cpp on a Vulkan port lands in the same range as hand-written kernels. That, more than the 27B, is the result worth noting.

For comparison with machines people buy for this, the widget's other rows use published peak bandwidths (reported): an RTX 4090 at 1,008 GB/s would put the same 2-bit file's ceiling at 147 tok/s, and a DGX Spark at 273 GB/s at 40 (reasoned). The PS5's advantage is not speed. It is that the GPU and CPU share 16 GB at 448 GB/s in a box that already exists. The TensorFold and Strata comparison runs the same model on Macs and an RTX 5070 against the same kind of roofline.

The 9 tok/s in the early notes

docs/CONSOLE.md, written on 9 October, reports 21 tok/s on a short prompt and 9 tok/s after the web UI's 371-token prompt. A 371-token KV cache on 16 layers is about 12 MiB at q8_0 (reasoned), far too little to halve the speed by bandwidth. The later README table, with a 7,775-token answer at 19.7 tok/s, does not show the drop. Neither repository says what changed between the two; I would not read anything into the 9 until someone does.

Prompt processing is the real weak spot: 35 to 50 tok/s for the 27B (reported), because without cooperative-matrix support the batched matrix products run on the fp16 shader path too. The project's own coding-agent notes say a 20,000-token prompt takes several minutes before the first word.

One token a minute was the SSD, not the CPU

The second post's comparison, "the same model on the PS5's CPU did about one token a minute", reads as a CPU benchmark. It is not one, and the project says so. The roadmap's entry for 9 October:

UD-IQ1_S (5.77 GiB) in a payload, mapped from /data and paged in from the SSD, so about one token a minute

So it was not the same model either: a 1-bit quant, not the 2-bit one on the GPU. A background program on the console gets at most about 6 GiB of locked memory and stalls the system near that, so the 5.77 GiB file was memory-mapped and the system paged it in from storage as each layer was touched. Reading about 6 GB from disk per token at one token a minute is roughly 100 MB/s of page-fault traffic (reasoned), the speed of small random reads, not of the console's SSD streaming.

How fast would the CPU be with the model in RAM? The project has the data to estimate it. With weights locked in memory, Qwen3.5 2B ran at 8.5 to 9.2 tok/s on three threads at 1.4 GB resident (reported), about 12.6 GB/s effective (reasoned). The same rate over the 27B's 6.84 GB at IQ2_XXS gives about 2 tok/s, and the roadmap's own expectation before the run was "1 to 3 tok/s". Those are a reasoned guess and a stated expectation, not measurements. Either way, the CPU-to-GPU gap is about 10x, not 1,200x. The other 120x was paging.

The console's CPU side has its own caveat. A background program is pinned to five logical CPUs shared with system services, and busy-waiting on all five shut the console down twice (reported), which is why PS5LM's first patch caps llama.cpp's default thread count at the allowed CPUs minus two.

What I could not check

• No logs or benchmark files are in either repository. Every speed above is from a README, docs/CONSOLE.md, docs/ROADMAP.md or a dashboard screenshot. There is no llama-bench output.
• Quality at 2 bits. Untested here, unmeasured there.
• Where the 48 ms per token goes. Bandwidth explains about a third of it (reasoned); the rest needs a GPU profile.
• The PS5 Pro. The notes list 576 GB/s for it; PS5LM reports no run on one, and the port's own measurements were mostly on a Pro, a different console from PS5LM's test machine.

Takeaway

The claim survives with one correction. A dense 27B decodes at about 20 tok/s on a PS5's GPU through stock llama.cpp and a Vulkan port of Mesa's RADV, inside an 11.44 GiB heap, at 2 bits, with room left for a 64k context because only 16 of its 64 layers keep a KV cache. The bus would allow about 65 tok/s at that size; the console uses about a third of it. The one-token-a-minute CPU figure is a paging artefact from a 1-bit run that never had the memory to hold its weights.

The more durable result is the stack: ggml-vulkan needed no new kernels to reach the same neighbourhood as hand-written ones on this GPU.