Run RHO AI on a Local Model (Ollama)

RHO AI normally talks to a cloud model with your own API key. It can also run entirely on your machine through Ollama — private, offline, and free per message. Wiring it up takes a manual settings edit and about five minutes.

What to expect

Works well locally Better left to cloud models
Searching and asking about your notes Authoring interactive documents (charts, quizzes)
Summaries and short analyses Long, complex multi-step tasks
Driving the app ("open X", "switch theme") Web search (no built-in search — add Tavily/Brave, or skip it)

An 8B-class model handles the everyday loop surprisingly well. Interactive-document authoring usually needs a frontier model — local attempts will be caught by Rho's validator and bounced back with errors, which small models don't always recover from.

Step 1 — Install Ollama and pull a model

  1. Install Ollama from ollama.com/download and make sure it's running (it serves on localhost:11434).
  2. Pull a model. A good starting point:
ollama pull qwen3:8b

~5 GB download. Rule of thumb: an 8B model wants ≥ 8 GB of free RAM and runs best on a GPU; bigger models write nicer prose but need proportionally more. Verify with ollama list.

Step 2 — Quit Rho MD

Close the app completely before editing its settings file — it saves settings on its own schedule and could overwrite your edit.

Step 3 — Edit plugins.json

Find the file:

OS Path
Linux ~/.local/share/md.rho.app/plugins.json
Windows %APPDATA%\md.rho.app\plugins.json

It's a flat JSON object. Add (or change) this entry — the {"value": …} wrapper matters:

"rho.ai:providerId": { "value": "ollama" }

Optionally pin the model too (otherwise Rho uses its default local model):

"rho.ai:model": { "value": "qwen3:8b" }

Keep the JSON valid: watch the commas, and don't touch other entries.

Step 4 — Relaunch and verify

Start Rho MD and open Settings → RHO AI. The Provider now shows Ollama (local) — no API key needed. Click Test connection; then ask RHO AI something in the chat panel.

Rho's local path is pre-tuned for you: it requests a 16k context window (so the assistant's full instructions survive) and disables "thinking" streams (so reasoning models answer directly instead of sitting silent).

Switching back

Just pick any cloud provider in the Provider dropdown — that works normally. To return to Ollama later, edit plugins.json again (it isn't offered in the dropdown).

Troubleshooting

Symptom Fix
"Cannot reach local Ollama" Ollama isn't running — launch the Ollama app, or run ollama serve.
Errors mentioning the model name The model isn't pulled, or the Model field doesn't match a tag from ollama list exactly.
Long pause before the first word Normal on first message — the model is loading into memory. Subsequent replies start faster.
Very slow or low-quality replies The model is too big for your hardware. Try qwen3:8b, close other heavy apps, or return to a cloud provider.
Provider shows Claude after your edit The JSON edit didn't stick — check the {"value": "ollama"} wrapper and that the app was closed while you edited.
Open in Rho MD →