Hermes Agent with Ollama: Run a Free AI Agent Locally (No API Keys)
Why Run Hermes Agent Locally?
Hermes Agent, built by Nous Research, is an open-source autonomous AI assistant that connects to a model endpoint, executes real tasks on your machine, and improves over time through memory and learned skills. When you run it locally, your data never leaves your hardware — no API keys, no cloud bills, and no privacy trade-offs.
For Indian developers and finance professionals building internal tools — whether that's an automated report generator pulling from NSE data or a personal portfolio analyser running on your office workstation in Bengaluru — a fully local agent is not just a privacy win. It eliminates recurring API costs that can run into thousands of rupees per month when scaled across a team.
This post is for the pure-local path. If you want to use OpenAI's API or connect Hermes to Telegram, see the cloud-assisted guide. If you're deciding whether to build Hermes at all (vs. using a hosted alternative), see the Hermes vs OpenClaw comparison.
Prerequisites
Before starting, confirm your machine meets these requirements:
- Operating System: Linux (Ubuntu 22.04+) or macOS. Windows users should use WSL2.
- RAM: Minimum 16 GB (32 GB recommended for larger models).
- VRAM: At least 12 GB for running quantized 7B–13B models. For 20B+ models, 24 GB+ is strongly recommended.
- Python: 3.10 or later.
- Ollama: Latest version installed (ollama.com).
Step 1: Update Ollama to the Latest Version
Ollama is the easiest way to serve local LLMs. Ensure you're running the latest version to avoid compatibility issues with Hermes:
curl -fsSL https://ollama.com/install.sh | sh
On macOS, you can also download the Ollama desktop app directly. Verify the installation:
ollama --version
Step 2: Install Hermes Agent
Hermes Agent provides a one-line installer. Run it directly from the official Nous Research repository:
curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash
Alternatively, if you prefer pulling directly through Ollama:
ollama launch hermes
This command downloads the model and launches the interactive agent session automatically.
Step 3: Choose Your Local Model
Hermes Agent is model-agnostic — it works with any LLM served through an OpenAI-compatible endpoint. Here are practical model recommendations based on your hardware:
- 12–16 GB VRAM:
ollama pull qwen2.5:7b or ollama pull gemma3:4b. These handle general-purpose agent tasks like file management, web searches, and code generation.
- 24 GB+ VRAM:
ollama pull qwen2.5:32b or ollama pull gemma3:27b. Recommended for complex multi-step reasoning and longer context windows.
- Apple Silicon (M2/M3/M4): Ollama handles Metal GPU acceleration natively. A 32 GB MacBook Pro can comfortably run 14B–30B parameter models.
For Indian users experimenting with local AI on a budget, a machine with an RTX 4070 Ti Super (16 GB VRAM) — available for around ₹65,000–₹75,000 on Flipkart — is a solid entry point for running 7B–13B models smoothly.
[IMAGE: A simple hardware-fit table or chart showing model size vs RAM/VRAM requirements, with the 3 tiers above visualized]
Step 4: Configure Hermes to Use Your Local Model
After installation, Hermes needs to know which model endpoint to talk to. By default, Ollama serves on http://localhost:11434.
Create or edit the Hermes configuration file:
hermes config init
This generates a config file (typically at ~/.hermes/config.yaml). Set your provider to Ollama:
provider:
name: ollama
base_url: http://localhost:11434/v1
model: qwen2.5:7b
api_key: not-needed
If you're using Unsloth instead of Ollama (useful for fine-tuned models or GGUF formats), start the Unsloth server first:
unsloth run --model unsloth/Qwen3-8B-GGUF --port 8000
Then point Hermes to the Unsloth endpoint:
provider:
name: openai-compatible
base_url: http://localhost:8000/v1
model: unsloth/Qwen3-8B-GGUF
api_key: not-needed
Step 5: Run Hermes Agent
Launch the interactive terminal session:
hermes
You'll see the Hermes prompt. Try a simple local task to verify everything works:
> Find the largest file in my home directory and show its size.
Hermes will use its built-in file system tools to execute this task autonomously. You should see it reason through the steps, run shell commands, and return the result — all without leaving your machine.
Tools and Capabilities (Built-In)
Out of the box, Hermes Agent supports:
- File system access: Read, write, edit, and search files on your machine.
- Shell execution: Run any terminal command.
- Web search: Search the internet and scrape web pages.
- Code interpreter: Execute Python code in a sandboxed environment.
- Memory: Persist learned facts and preferences across sessions.
- Skills: Install community skills from the Hermes skill registry for specialised tasks.
Install a skill:
hermes skill install github:NousResearch/hermes-skills/finance
Troubleshooting Common Issues
- "Connection refused" errors: Ensure Ollama is running:
ollama serve. Check that port 11434 is not blocked.
- Model too slow: Use a more aggressively quantized version (Q4_K_M or Q3_K_M GGUF). Reduce context length in Ollama's Modelfile.
- Out of memory: Switch to a smaller model or add swap space. On Linux:
sudo fallocate -l 16G /swapfile && sudo chmod 600 /swapfile && sudo mkswap /swapfile && sudo swapon /swapfile.
- Hermes not responding: Run
hermes doctor to diagnose configuration issues and check for updates.
Quick Reference Cheat Sheet
# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh
# Install Hermes Agent
curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash
# Pull a model
ollama pull qwen2.5:7b
# Initialize config
hermes config init
# Run interactive session
hermes
# Check for updates
hermes doctor
# Switch model
hermes config set provider.model gemma3:4b
What to Do Next
You now have a fully local Hermes Agent running with no API keys and no monthly bill. Two natural next steps depending on what you're building:
Sources
Last updated: 3 July 2026. Setup steps verified against Hermes Agent v0.x and Ollama v0.5+.
📌 Editorial Disclosure & Verification
Researched and drafted with PyBankers AI research pipelines; fact-checked, edited, and verified by the PyBankers editorial team. This report is for educational and informational purposes only and does not constitute financial, legal, or investment advice. Verify critical details independently.