Skip to main content
The Quantum Dispatch
Back to Home
Cover illustration for Eight Used V100 GPUs: How a Lawyer Built a Private AI Server

Eight Used V100 GPUs: How a Lawyer Built a Private AI Server

A lawyer's private AI server runs DeepSeek V4.1 Flash on eight used V100 GPUs at up to 215 tokens per second, keeping case files off the cloud.

Alex Circuit
Alex Circuit★Nov 17, 2026★4 min read

The cheapest fast VRAM in the world might be sitting in a data center liquidation pile. A South Carolina trial lawyer proved the point this month with a private AI server built from eight used Nvidia Tesla V100 GPUs, a 2017-era data center card, that now runs the open-weight DeepSeek V4.1 Flash model entirely inside his office. The goal is simple: client case files never leave the building.

  • GPUs: eight Tesla V100-SXM2-32GB cards on two four-card NVLink boards, bought at about $2,400 per board
  • Speed: up to 215 tokens per second for code and up to 118 tokens per second for legal writing in the original build
  • Latest upgrade: about 80% of the model moved from 2-bit to near-NVFP4 4-bit precision, system RAM raised to 1TB, plus an RTX 3090 and a 524K context
  • Software: a published llama.cpp serving recipe and bug fix from the builder's PLI Labs

What Hardware Is Inside the V100 AI Server?

Tom's Hardware broke down the first version on November 9. The eight V100-SXM2-32GB cards each carry 32GB of HBM2 memory at 900 GB/s, and they plug into SXM2 mezzanine sockets rather than PCIe slots. Each four-card baseboard links its GPUs over NVLink for 128GB of pooled memory per board, while the two boards talk to each other over PCIe through four PEX switch cards and a nest of SlimSAS cables.

The host is a Gigabyte Threadripper Pro board with a 12-core 3945WX processor. Two power supplies totaling 2,800 W sit on separate ordinary office circuits, which matters for anyone hoping to copy this without an electrician. The build was not glamorous: the original post included one burnt power connector.

How Does DeepSeek V4.1 Flash Fit on Old GPUs?

DeepSeek V4.1 Flash is a mixture-of-experts model, so only a slice of its weights does the work for each token. In the first build, the model was compressed mostly to 2 bits, still totaling 248.8GB, with every expert held in GPU memory and roughly 110GB of lookup tables parked in system RAM. The builder split the model's 40 layers in half across the two boards, so only one hand-off per step crosses the slower PCIe link.

The November 14 update pushed quality higher. Moving about 80% of the work to near-NVFP4 precision, Nvidia's 4-bit floating-point format, cost a little speed: code output now lands at 150 to 180 tokens per second instead of 160 to 215. In exchange the server gained 1TB of RAM, a newer RTX 3090, a 524K context window, and the ability to handle more than one request at a time. The crashes that plagued the earlier build turned out to be a bad PSU, not the PCIe setup.

Why This Local LLM Build Matters

This is a textbook case for self-hosted LLM hardware. The server does first-pass work on case files, reading folders of records, extracting facts, dates and numbers, summarizing them and drafting routine documents, and every output gets a human review before it goes anywhere. The builder estimates half or more of his AI work can now skip frontier cloud models.

He has also shared the hard-won lessons. His PLI Labs Hugging Face repository documents a working llama.cpp recipe for DeepSeek V4 Flash on eight V100s, which it describes as the first published Volta datapoint for the model, along with a fix for a quantized KV-cache bug that produced confident gibberish. That fits neatly with the momentum behind llama.cpp's stable v1.0 release.

Is a Used V100 Build Right for Your Homelab?

Go in with eyes open. Used V100 SXM2 32GB cards have climbed to about $900 on eBay, more than double what this builder paid, and Nvidia's R580 driver branch is the last to support Volta. Power and cooling are real costs. If you want a quieter, warrantied route to big local models, compare it against options like the DGX Spark in our mini computer coverage. But for raw VRAM per dollar, retired data center silicon still has plenty of life left.

Sources: Tom's Hardware — November 9, 2026; Tom's Hardware — November 14, 2026; PLI Labs on Hugging Face — project repository.

More Mini Computers Stories