Hermes 3 locally: requirements, installation and complete 2026 guide

1. The quick answer

If you are looking at hermes 3 local and are in a hurry: the 8B runs on any GPU with 6+ GB, the 70B needs 24 GB as a minimum and the 405B is for clusters. Use the VRAM calculator before downloading anything.

2. What Hermes 3 is and why it is not just another Llama

Hermes 3 is the third generation of models from Nous Research, an open source lab that specializes in high-quality finetunes. The Hermes 3 series is a full finetune of Llama 3.1 — same parameters, same architecture, same VRAM footprint. What changes is what comes out of the model when you talk to it.

Nous Research trained Hermes 3 with three clear goals: native structured function-calling, agent capabilities with <scratchpad>-style chain-of-thought reasoning, and neutral alignment — the model does not refuse legitimate requests or inject unnecessary disclaimers like Llama 3.1-Instruct does. It still has judgment, but it is not paternalistic.

Why the finetune matters

Llama 3.1 base is great as a foundation model, but the official Instruct version (trained by Meta) is heavily aligned toward rejecting anything ambiguous. For roleplay, fiction, coding assistance without babysitting or technical analysis on frontier topics, the official Instruct causes constant friction. Hermes 3 removes that behavior while keeping the technical capability intact.

All variants keep the 128K-token context window from Llama 3.1. That is ~100 pages of text — enough to pass an entire codebase, long logs or full books.

3. 8B, 70B and 405B variants: exact VRAM

The numbers below are the real ones from the model loaded with llama.cpp — they include KV cache overhead and base activations. Since Hermes 3 shares architecture with Llama 3.1, requirements are identical to the base model.

4. Hermes 3 vs Llama 3.1 base: when to pick each

Hermes 3 and Llama 3.1-Instruct have exactly the same VRAM footprint and speed — they are the same base model. The choice is about behavior, not hardware. If you already have Llama 3.1 installed, you can try Hermes 3 without changing a single GPU. For more background on Llama specifically, read our Llama vs Mistral comparison.

Rule of thumb: if you are going to build an agent, integrate tool use, or do anything creative, Hermes 3. If you just want a basic chatbot with official Meta branding, Llama 3.1-Instruct. Both take exactly the same space on disk and VRAM.

5. Install with Ollama (step by step)

Ollama is the most direct way to run Hermes 3 locally. It handles download, quantization and a REST API with no configuration. Total time from zero: 5 minutes plus the model download (~5 GB for 8B Q4).

Step 1: Install Ollama

# Linux / macOS — one line
curl -fsSL https://ollama.ai/install.sh | sh

# Windows — download the installer from https://ollama.ai/download

Step 2: Download Hermes 3

# Hermes 3 8B — 4.8 GB VRAM Q4 — any GPU ≥6 GB
ollama pull hermes3:8b

# Hermes 3 70B — 40 GB VRAM Q4 — 2× 3090 or 4090 + offload
ollama pull hermes3:70b

Step 3: Run

# Interactive chat
ollama run hermes3:8b

# Direct query from terminal
ollama run hermes3:8b "Design an agent that reads my calendar and summarizes my day"

# REST API (automatic on localhost:11434)
curl http://localhost:11434/api/generate -d '{
  "model": "hermes3:8b",
  "prompt": "Hello Hermes",
  "stream": false
}'

6. HuggingFace and llama.cpp (alternative)

If you need the original raw weights (for further fine-tuning, vLLM, or custom quantizations), Hermes 3 models live on HuggingFace under the NousResearch account:

• NousResearch/Hermes-3-Llama-3.1-8B — original FP16 weights
• NousResearch/Hermes-3-Llama-3.1-8B-GGUF — already converted to GGUF for llama.cpp
• NousResearch/Hermes-3-Llama-3.1-70B — 70B version
• NousResearch/Hermes-3-Llama-3.1-405B — frontier version

# Download with huggingface-cli
pip install huggingface_hub
huggingface-cli download NousResearch/Hermes-3-Llama-3.1-8B-GGUF \
  Hermes-3-Llama-3.1-8B.Q4_K_M.gguf --local-dir ./models

# Run with llama.cpp
./llama-server -m ./models/Hermes-3-Llama-3.1-8B.Q4_K_M.gguf \
  -c 32768 --host 0.0.0.0 --port 8080

Use llama.cpp if you want full control over quantization (Q4_K_M, Q5_K_S, IQ4_XS), per-layer offloading, or integration into custom stacks. For 99% of users, Ollama is faster to get running.

7. Function-calling and agent capabilities

This is the killer feature of Hermes 3 over Llama 3.1 base. Nous Research trained the model with structured function-calling datasets, ReAct-style reasoning, and parallel tool use. JSON output is consistent — no need for brittle regex to parse it.

Function-calling format

<tool_call>
{ "name": "get_weather", "arguments": { "city": "Madrid" } }
</tool_call>

<scratchpad>
The user is asking for the weather in Madrid. I need to call get_weather
before answering.
</scratchpad>

The model emits <tool_call> blocks with valid JSON, and <scratchpad> blocks for internal reasoning. Integrates well with MCP (Model Context Protocol), LangChain, LlamaIndex and custom agent stacks.

Agent capabilities

• Parallel tool use: can call several functions in a single turn
• Chain-of-thought reasoning: <scratchpad> to plan before acting
• Native JSON mode: structured output without temperature hacks
• Long system prompts: follows complex instructions without degrading
• Self-correction: if a tool call fails, it adjusts and retries

To build a real local agent with Hermes 3, the recommended combo is: Ollama (serving) + LangChain or LlamaIndex (orchestration) + your custom tools exposed as Python functions. With an RTX 4070 Super you get sub-second latency for most tool calls.

8. Real-world use cases

9. Benchmarks and expected speed

Speeds estimated from each GPU's memory bandwidth and the model size. Since Hermes 3 shares architecture with Llama 3.1, tokens/sec are essentially identical to Llama 3.1.

To compare options and reason about your specific case, also see our guide on DeepSeek R1 locally, which covers reasoning with similar hardware.

Recommended hardware for Hermes 3

Four GPUs that cover the three real scenarios: budget entry for 8B, sweet spot for 8B with headroom, premium for 8B in Q8, and workstation for 70B.

€269

budget Amazon Prime

RTX 3060 12GB

4.8 (1,400 reviews)

Pros

• 12 GB VRAM — Hermes 3 8B Q8 fits comfortably
• Unbeatable entry price (~€269)
• Ideal to start with local agents

Cons

• Bandwidth lower than Ada
• Not enough for 70B

Check RTX 3060 on Amazon

€399

mid Amazon Prime

RTX 4060 Ti 8GB

4.6 (640 reviews)

Pros

• Ada Lovelace — efficiency per watt
• Hermes 3 8B Q4 with headroom
• Good price/performance balance

Cons

• 8 GB limit Q8 of the 8B
• Long context pressures VRAM

Check RTX 4060 Ti on Amazon

€499

mid Amazon Prime

RTX 4070 Super 12GB

4.7 (520 reviews)

Pros

• Best pick for Hermes 3 8B in 2026
• Q8 comfortable + 32K context easy
• ~75 tok/s in Q4

Cons

• Price above RTX 3060
• 12 GB tight for 8B + 128K context

Check RTX 4070 Super on Amazon

€1799

pro Amazon Prime

RTX 4090 24GB

4.8 (1,200 reviews)

Pros

• 24 GB VRAM — only consumer GPU for Hermes 3 70B with offload
• Ada Lovelace flagship
• Peak speed on 8B (~110 tok/s)

Cons

• Flagship price (~€1,799)
• 450W typical draw

Check RTX 4090 on Amazon

As an Amazon Associate we earn from qualifying purchases. This does not affect our recommendations.

10. Frequently asked questions

Sources

• NousResearch/Hermes-3-Llama-3.1-8B on HuggingFace
• Hermes 3 in Ollama Library
• Nous Research — Hermes 3 technical report