NightRun Boots a Local LLM on a Pi 5 With No OS

What If the Operating System Is the Overhead?

NightRun is an experimental open-source project that boots a machine directly into a local large language model from a USB stick or microSD card, with no Linux kernel and no operating system underneath it. It is a UEFI application written in Rust, and it exists to answer a specific question that anyone who has tried to run a model on a memory-constrained board has asked at least once: how much RAM would be available if nothing else were running at all?

• Runs as a UEFI application on 64-bit x86_64 PCs with Secure Boot disabled, and on the Raspberry Pi 5 via USB or microSD
• Loads quantized models of 1.3–2.4 GB directly into RAM at boot, verifying CRC-32 checksums as it goes
• Supports Llama 3.2 1B (4 GB RAM), Llama 3.2 3B and Granite 4.1 3B (6 GB), and Qwen3 4B Instruct 2507 (8 GB)
• Roughly 20 tokens/second on an 8-core x86 QEMU VM and 3.0 tokens/second on a Raspberry Pi 5; MIT licensed on GitHub

How Does It Run Without an Operating System?

By leaning entirely on UEFI Boot Services. Firmware already provides primitives for input, display, storage access, and device control before any kernel loads — that is how a bootloader draws a menu and reads a disk. NightRun stays inside that environment rather than handing off to a kernel, using those services for its console and its model loading.

The detail I find most satisfying is what happens after the weights are in memory. Once the model is loaded, NightRun *seals* storage: any subsequent attempt to touch the disk triggers a hard fault. That is not a safety feature bolted on afterward — it is a statement about the execution model. There is no swap, no page cache, no background process that might decide to write something. The machine holds one model in RAM and does one job.

Is 3 Tokens Per Second Actually Useful?

Not for interactive chat, and the project does not pretend otherwise. Three tokens per second on a Pi 5 is roughly a word every half second, which is fine for a short answer and tedious for anything longer. The 20 tokens/second figure on an 8-core x86 VM is considerably more usable.

But throughput is not really the point of NightRun, and judging it on tokens per second misses what is interesting. The point is the elimination of everything between the hardware and the model. For readers of our mini computer coverage who have spent an afternoon trimming a Linux install to claw back a few hundred megabytes for a larger quantization, this is that instinct taken to its logical end. On a 4 GB board, the RAM a distribution consumes is not a rounding error — it is the difference between a 1B and a 3B model.

Where Would You Actually Use This?

A few places suggest themselves. A single-purpose appliance that does nothing but run one model has a dramatically smaller attack surface with no OS, no network stack, and sealed storage. An air-gapped machine for sensitive work benefits from the same properties. And as a teaching artifact it is unusually clear: the whole path from firmware to token is in one Rust codebase you can read.

The model list is chosen sensibly for the memory budgets involved. Llama 3.2 1B fits in 4 GB, Llama 3.2 3B and Granite 4.1 3B want 6 GB, and Qwen3 4B Instruct 2507 needs 8 GB. Anyone deciding which quantization to pair with which board will find our guide to GGUF, AWQ, and MLX formats useful background, and the best mini PC for local LLMs guide covers what the x86 side of this looks like with more headroom.

The Practical Caveats

Secure Boot has to be off on x86, which is a real consideration on any machine that is not dedicated to this. The project is explicitly experimental. And the boot-time CRC-32 verification is an integrity check rather than a signature check — it catches a corrupted download, not a substituted model.

None of that diminishes what NightRun demonstrates. The code is MIT licensed and on GitHub, which means the interesting parts — the UEFI-native inference loop, the storage sealing, the memory layout — are there to be read and borrowed. Sometimes the value of a project is the proof that a thing is possible, and this is one of those.

Sources: CNX Software — July 30, 2026; NIGHTRUN on GitHub — July 2026.