Articles Tagged “Self Hosted LLM”
7 articles found

Eight Used V100 GPUs: How a Lawyer Built a Private AI Server
A lawyer's private AI server runs DeepSeek V4.1 Flash on eight used V100 GPUs at up to 215 tokens per second, keeping case files off the cloud.

Optane PMem Homelab Runs Kimi K3 2.8T Model on CPU Alone
A homelab builder ran Moonshot's 2.8-trillion-parameter Kimi K3 on secondhand Optane PMem 200 and a 40-core Xeon, at 0.85 tokens per second with no GPU.

How Much RAM Do You Need to Run a Local LLM in 2026?
A practical sizing guide: 8GB runs an 8B model at 4-bit, 24GB handles a 27B, and 240GB is the floor for a trillion-parameter model at 1-bit.

Best Mini PC for Local LLMs in 2026: A Buyer's Guide
A practical 2026 buyer's guide to the best mini PCs for running local LLMs, comparing unified memory, NPUs, and price so you can self-host with confidence.
Ollama v0.31.1 Boosts Local AI Performance on Apple Silicon
Ollama v0.31.1 makes Gemma 4 about 90% faster on Apple Silicon via multi-token prediction, advancing local AI performance and privacy.
Ollama 0.30.8 Widens Local AI Hardware Support and Speeds Up Apple Silicon
Ollama 0.30.8, released June 12, broadens GGUF hardware support through llama.cpp and upgrades its Apple Silicon MLX engine for faster, private local AI.

Gemma 4 QAT Lands in Ollama, Cutting Local AI Memory by ~72%
Quantization-aware-trained Gemma 4 weights are now runnable in Ollama, cutting VRAM roughly 72% so a 26B model fits on a 16GB laptop for self-hosted AI.
