Which LLM Can Your VPS Run? 2026 Hardware Guide: RAM, Model Size, Multiple Users

Anyone who wants to self-host an open language model like Gemma 4 31B for several concurrent users soon hits the hardware question. Two other questions come first:

1. Which model do you need? Gemma 4 also comes as the Mixture-of-Experts variant 26B A4B, which activates 3.8 of 25.2 billion parameters per token (Gemma 4 model card, as of Nov 8, 2026).
2. Do you need your own hardware? Whether buying or renting fits depends on utilization, power and runtime. RackDiff currently does not list GPU servers.

This guide sorts models by file size and RAM tier. It shows how Ollama, llama.cpp server and vLLM differ with multiple users and which cost factors matter when buying or renting.

The short answer

Details below.

What determines whether a model runs on your server?

Three factors:

1. RAM: weights + context + system

The memory needed for the model weights is shown by the file size in Ollama's tag directory, e.g. 5.2 GB for qwen3:8b (ollama.com, as of Nov 8, 2026) and 4.9 GB for llama3.1:8b in q4_K_M (ollama.com, as of Nov 8, 2026). Context (KV cache) and system come on top; 16 GB leaves more room for longer contexts and other containers.

RAM needs scale with OLLAMA_NUM_PARALLEL × OLLAMA_CONTEXT_LENGTH; the default context length is 4096 tokens (Ollama FAQ, as of Nov 8, 2026).

2. CPU: cores and instruction sets

What sets dedicated vCPUs apart from shared ones and which plans offer them is shown in the VPS comparison.

llama.cpp supports AVX, AVX2, AVX512 and AMX on x86 (llama.cpp, as of Nov 8, 2026). For CPU operation, vLLM lists avx512f as recommended and avx2 with reduced features (vLLM docs, as of Nov 8, 2026). lscpu shows which flags your server has.

3. Quantization: Q4_K_M and Q8_0

Q4_K_M variants are smaller than Q8_0, e.g. 5.2 instead of 8.9 GB for qwen3:8b (ollama.com, as of Nov 8, 2026) and 18 instead of 28 GB for Gemma 4 26B A4B (ollama.com, as of Nov 8, 2026). llama.cpp supports quantization from 1.5 to 8 bits (llama.cpp, as of Nov 8, 2026). More on quantization in the coding assistant guide.

Mixture-of-Experts: many parameters, few of them active

Complementing the Ollama setup guide: a Mixture-of-Experts (MoE) model has many parameters but activates only part of them per token (Gemma 4 model card, as of Nov 8, 2026).

One example is the Gemma 4 family:

Architecture, parameters and context per the Gemma 4 model card, file sizes per ollama.com, as of Nov 8, 2026.

Gemma 4 26B A4B has 25.2 billion parameters and activates 3.8 billion per token; per the model card it runs almost as fast as a 4B model (Gemma 4 model card, as of Nov 8, 2026). The whole model still sits in memory, 18 GB in q4_K_M (ollama.com, as of Nov 8, 2026).

Also MoE are Qwen3-30B-A3B with 30.5 billion parameters, 3.3 billion activated (Qwen3 model card, as of Nov 8, 2026), and gpt-oss-20b with 21 billion parameters, 3.6 billion active, able to run within 16 GB of memory per its model card (gpt-oss model card, as of Nov 8, 2026). For a CPU server you can shortlist MoE variants; they need RAM for all parameters.

The dense Gemma 4 31B (30.7 billion parameters) takes 20 GB in q4_K_M (Gemma 4 model card, as of Nov 8, 2026; ollama.com, as of Nov 8, 2026).

Model matrix: which model size fits which RAM tier?

File sizes per ollama.com, tag directories llama3.2, llama3.1, llama3.3, gemma4, qwen3, gpt-oss, mistral and mistral-small, as of Nov 8, 2026.

Plans with 32 GB RAM are shown in the VPS comparison and the dedicated comparison, with prices incl. VAT and their date.

Multiple concurrent users: Ollama, llama.cpp server and vLLM

How many requests are processed at once depends on the hardware and also on the serving software and its settings.

Ollama by default processes one request per model at a time (OLLAMA_NUM_PARALLEL, default 1) and queues others (OLLAMA_MAX_QUEUE, default 512) (Ollama FAQ, as of Nov 8, 2026). More parallel requests raise RAM needs (OLLAMA_NUM_PARALLEL × OLLAMA_CONTEXT_LENGTH).

vLLM lists continuous batching of incoming requests and PagedAttention as features (vLLM docs, as of Nov 8, 2026). The llama.cpp server also offers parallel decoding for multiple users and continuous batching, on by default (llama.cpp server README, as of Nov 8, 2026).

What you can adjust:

• Ollama: raise OLLAMA_NUM_PARALLEL and plan RAM per parallel request (Ollama FAQ, as of Nov 8, 2026).
• llama.cpp server: number of slots via -np/--parallel, automatic by default (llama.cpp server README, as of Nov 8, 2026).
• vLLM: on CPU, the docs list the data types FP32, FP16 and BF16 plus the quantizations AWQ, GPTQ and INT8 W8A8 (vLLM docs, as of Nov 8, 2026).

On a CPU server, all active requests share the same cores and memory. The Gemma 4 and Qwen3 model cards give no VRAM figures for GPU use; RackDiff currently does not list GPU servers.

Each parallel request needs its own context memory. Ollama can quantize the KV cache with OLLAMA_KV_CACHE_TYPE (q8_0 about half the memory of f16) when Flash Attention is on (OLLAMA_FLASH_ATTENTION=1) (Ollama FAQ, as of Nov 8, 2026).

Buy or rent? Cost factors for your own LLM hardware

Running it yourself adds power: work out the components' rated power draw with your electricity tariff and the planned runtime.

Add replacement after failures, hardware depreciation and whether future models will fit into the same memory.

Buying and renting differ in utilization, runtime and operation. Minimum term and notice period are listed with each plan; prices incl. VAT are shown in the VPS comparison and the dedicated comparison.

The model choice also shifts memory needs: the 26B A4B takes 18 GB in q4_K_M versus 20 GB for the dense 31B (ollama.com, as of Nov 8, 2026) and activates 3.8 instead of 30.7 billion parameters per token (Gemma 4 model card, as of Nov 8, 2026).

Example setups by scenario

Trying it out, one user. A VPS with 8 GB RAM (VPS comparison) with Ollama and Llama 3.2 3B or Gemma 4 E2B (sizes in the model matrix) for personal chats and text summaries. Spare RAM can run other services such as Vaultwarden on the same server. Limits: short contexts, one user.

One user: chat, n8n workflows, coding assistant. VPS with 16 GB RAM (VPS comparison), Ollama, Gemma 4 12B or Qwen3 14B. Setup in the Ollama guide.

Small team, not all at once. Server with 32 GB RAM (VPS or dedicated server), llama.cpp server with several slots or Ollama with a higher OLLAMA_NUM_PARALLEL, plus Gemma 4 26B A4B or Qwen3-30B-A3B. Concurrent requests share the compute.

Many concurrent users. vLLM lists support for NVIDIA and AMD GPUs (vLLM docs, as of Nov 8, 2026); the model cards give no VRAM figures. RackDiff currently does not list GPU servers.

Privacy as a motive. When run locally, Ollama sees neither prompts nor data; this doesn't apply to Ollama cloud models (Ollama FAQ, as of Nov 8, 2026). The server provider stays involved; Art. 28 GDPR governs what to agree with it (GDPR, as of Nov 8, 2026). More in VPS in Germany and GDPR (in German).

Conclusion

Choose the model first and check MoE variants, then estimate concurrent load and tune the serving software, then pick the RAM tier. The VPS comparison and the dedicated comparison show what such servers cost, with prices incl. VAT.

More LLM and self-hosting guides

Ollama auf VPS installieren

Lokale KI mit Open WebUI

Lesen