Ollama Model Too Slow — 5 Optimizations
If your Ollama model generates fewer than 5 tokens per second inside OpenClaw, you're most likely running on CPU without GPU acceleration, using a full-precision (F16) model that doesn't fit in VRAM, or hitting a context window that's unnecessarily large. Each optimization below targets one of these root causes — apply them in order for the fastest cumulative improvement. On a Mac Mini M4 with gpu_layers: -1 and Q4_K_M quantization, throughput typically jumps from 4-6 tok/s to 80-120 tok/s.
🔍 First: Diagnose Your Baseline
Look for two things in the output: the 'device' column in 'ollama ps' — if it shows 'CPU' instead of 'GPU' or 'Metal', no hardware acceleration is active. And the 'eval rate' line from --verbose — anything below 10 tok/s on Apple Silicon or below 20 tok/s on a discrete NVIDIA GPU indicates a configuration problem, not a hardware limitation. Note the number before continuing to the fixes.
✅ 5 Optimizations (apply in order)
By default, if Ollama can't confirm all layers fit in VRAM, it offloads some to CPU. Setting gpu_layers: -1 forces all layers to GPU — if they don't all fit, Ollama warns you instead of silently degrading performance. This single change is responsible for the majority of performance gains on Apple Silicon and NVIDIA systems.
Attention computation scales quadratically with context length. Most OpenClaw conversations use fewer than 1,500 tokens of actual context, so the default 4096 window is often wasted. Halve it unless you're doing long document analysis. For a heartbeat agent that just sends brief check-ins, 512 is enough.
Q4_K_M is the recommended quantization for most use cases: it's 40-50% smaller than Q8 and loses less than 1% benchmark performance. Q2_K is even faster but noticeably worse at reasoning and instruction following — avoid it for agent tasks that require multi-step logic.
Repeated identical prompts — daily greetings, heartbeat pings, static FAQ queries — can be served from cache at near-zero latency. Cache entries persist across restarts if you mount the Ollama data directory as a Docker volume. Most useful for scheduled agents that send the same prompt multiple times per day.
Route intent-classified 'simple' queries to a 2-3B model. OpenClaw's routing config lets you specify a fast model for greetings, status checks, and brief responses — reserving the large model for tasks that actually need deep reasoning. A Gemma 2B can handle most conversational exchanges at twice the speed of a 7B.
Cold start: "model not found" on the first request after a reboot
Slow first responses and outright ‘model not found’ errors after a reboot are the same problem seen from two angles: the model is not resident in memory yet. These fixes make the first request behave like the tenth.
✅ Fix 1 — Increase Ollama Request Timeout in OpenClaw
# In openclaw.yaml, increase Ollama timeout
providers:
- id: local-ollama
type: ollama
baseUrl: http://localhost:11434
timeout: 120000 # 120 seconds (default: 30)
coldStartTimeout: 300000 # 5 min for first load✅ Fix 2 — Pre-warm Ollama Before Gateway Start
# Add to your startup script before openclaw gateway start ollama run llama3.1:8b-q4_K_M '' --nowordwrap & sleep 30 # wait for model to load openclaw gateway start
✅ Fix 3 — Disable Model Unloading (Keep Model Always Loaded)
# Keep model loaded indefinitely (uses more RAM)
{
"ai": {
"keep_alive": -1
}
}
# Or set a long timeout
{
"ai": {
"keep_alive": "24h"
}
}