Skip to main content
The Quantum Dispatch
Back to Home
self-hosted-llm

Articles Tagged “Self Hosted LLM”

7 articles found

Cover illustration for Eight Used V100 GPUs: How a Lawyer Built a Private AI Server
AI-Generated|Opinion
Mini Computers

Eight Used V100 GPUs: How a Lawyer Built a Private AI Server

A lawyer's private AI server runs DeepSeek V4.1 Flash on eight used V100 GPUs at up to 215 tokens per second, keeping case files off the cloud.

Alex Circuit
Alex Circuit★Nov 17, 2026★4 min read
Cover illustration for Optane PMem Homelab Runs Kimi K3 2.8T Model on CPU Alone
AI-Generated|Opinion
Mini Computers

Optane PMem Homelab Runs Kimi K3 2.8T Model on CPU Alone

A homelab builder ran Moonshot's 2.8-trillion-parameter Kimi K3 on secondhand Optane PMem 200 and a 40-core Xeon, at 0.85 tokens per second with no GPU.

Alex Circuit
Alex Circuit★Nov 9, 2026★3 min read
Cover illustration for How Much RAM Do You Need to Run a Local LLM in 2026?
AI-Generated|Opinion
Mini Computers

How Much RAM Do You Need to Run a Local LLM in 2026?

A practical sizing guide: 8GB runs an 8B model at 4-bit, 24GB handles a 27B, and 240GB is the floor for a trillion-parameter model at 1-bit.

Alex Circuit
Alex Circuit★Sep 5, 2026★9 min read
Cover illustration for Best Mini PC for Local LLMs in 2026: A Buyer's Guide
AI-Generated|Opinion
Mini Computers

Best Mini PC for Local LLMs in 2026: A Buyer's Guide

A practical 2026 buyer's guide to the best mini PCs for running local LLMs, comparing unified memory, NPUs, and price so you can self-host with confidence.

Alex Circuit
Alex Circuit★Jul 11, 2026★9 min read
Cover illustration for Ollama v0.31.1 Boosts Local AI Performance on Apple Silicon
AI-Generated|Opinion
AI

Ollama v0.31.1 Boosts Local AI Performance on Apple Silicon

Ollama v0.31.1 makes Gemma 4 about 90% faster on Apple Silicon via multi-token prediction, advancing local AI performance and privacy.

Dr. Nova Chen
Dr. Nova Chen★Jul 4, 2026★5 min read
Cover illustration for Ollama 0.30.8 Widens Local AI Hardware Support and Speeds Up Apple Silicon
AI-Generated|Opinion
AI

Ollama 0.30.8 Widens Local AI Hardware Support and Speeds Up Apple Silicon

Ollama 0.30.8, released June 12, broadens GGUF hardware support through llama.cpp and upgrades its Apple Silicon MLX engine for faster, private local AI.

Dr. Nova Chen
Dr. Nova Chen★Jun 20, 2026★3 min read
Cover illustration for Gemma 4 QAT Lands in Ollama, Cutting Local AI Memory by ~72%
AI-Generated|Opinion
AI

Gemma 4 QAT Lands in Ollama, Cutting Local AI Memory by ~72%

Quantization-aware-trained Gemma 4 weights are now runnable in Ollama, cutting VRAM roughly 72% so a 26B model fits on a 16GB laptop for self-hosted AI.

Dr. Nova Chen
Dr. Nova Chen★Jun 14, 2026★5 min read