LLM Providers

NeuralCleave ships with 19 provider adapters, configured under [models] in config.toml. The active provider is chosen automatically per request (primary → fallback → fast → local, based on task complexity), or forced to one provider at runtime via POST /api/v1/settings/model.

Supported providers

Provider	Config key	Env var fallback
Anthropic	`anthropic_api_key`	`ANTHROPIC_API_KEY`
Google Gemini	`gemini_api_key`	`GEMINI_API_KEY`
OpenAI	`openai_api_key`	`OPENAI_API_KEY`
DeepSeek	`deepseek_api_key`	`DEEPSEEK_API_KEY`
Ollama (local)	`ollama_base_url`	—
Mistral AI	`mistral_api_key`	`MISTRAL_API_KEY`
xAI Grok	`xai_api_key`	`XAI_API_KEY`
Cohere	`cohere_api_key`	`COHERE_API_KEY`
Moonshot / Kimi	`moonshot_api_key`	`MOONSHOT_API_KEY`
Zhipu / GLM	`zhipuai_api_key`	`ZHIPUAI_API_KEY`
Qwen (Alibaba)	`dashscope_api_key`	`DASHSCOPE_API_KEY`
ERNIE (Baidu)	`qianfan_api_key`	`QIANFAN_API_KEY`
Doubao (ByteDance)	`ark_api_key`	`ARK_API_KEY`
OpenRouter	`openrouter_api_key`	`OPENROUTER_API_KEY`
Azure OpenAI	`azure_api_key` + `azure_endpoint`	`AZURE_OPENAI_API_KEY` / `AZURE_OPENAI_ENDPOINT`
Amazon Bedrock	`bedrock_region` (credentials via AWS SDK chain)	`AWS_REGION`
Groq	`groq_api_key`	`GROQ_API_KEY`
Together AI	`together_api_key`	`TOGETHER_API_KEY`
Fireworks AI	`fireworks_api_key`	`FIREWORKS_API_KEY`

Every key also accepts ENV:VAR_NAME (reads an environment variable) or op://vault/item/field (resolves via the 1Password CLI) instead of a plaintext value.

Configuration

[models]
primary = "claude-opus-4-8"
fallback = "gemini-2.5-flash"
fast = "gemini-2.5-flash"
local = "ollama/llama3.2:1b"

anthropic_api_key = "op://Private/Anthropic/api_key"
gemini_api_key = "ENV:GEMINI_API_KEY"
groq_api_key = "ENV:GROQ_API_KEY"

primary/fallback/fast/local are model IDs, not provider names — the provider is inferred from the ID’s prefix (claude-* → Anthropic, gemini-* → Google, grok-* → xAI, ollama/* → Ollama, openrouter/* → OpenRouter, etc.).

Ollama (fully local)

[models]
local = "ollama/llama3.2:1b"
ollama_base_url = "http://localhost:11434"

No API key required. Pull the model first: ollama pull llama3.2:1b.

Checking provider status

neuralcleave models list                # configured or not — no network calls
neuralcleave models status --live       # also probes reachability for supported providers

Live probing only works for providers with a documented, stable check (OpenAI-compatible GET /v1/models for OpenAI/Mistral/xAI/Moonshot/OpenRouter/Groq/Together/Fireworks/DeepSeek, Ollama’s GET /api/tags) — every other provider reports “not supported” rather than a guessed result. See API Reference → Overview for the matching GET /api/v1/models route.

Switching providers at runtime

curl -X POST http://localhost:7432/api/v1/settings/model \
  -H "Content-Type: application/json" \
  -d '{"provider": "anthropic"}'

Pass "provider": null to restore automatic task-based routing. The switch takes effect immediately for the next request — no restart needed.

Reasoning effort (“thinking”)

A normalized off|low|medium|high|xhigh|max ladder maps to each provider’s native reasoning control — Anthropic’s extended thinking (with a scaled token budget), xAI/OpenRouter’s reasoning_effort field, and Ollama’s think field (collapsed to a boolean: off → false, anything else → true, since not every Ollama model accepts the 3-tier string form). Every other provider ignores it — notably DeepSeek, which has no per-request reasoning-effort field at all; its only reasoning lever is which model you configure (deepseek-chat vs. deepseek-reasoner).

# In any channel, the Web UI chat, or the voice assistant:
/think high
/think          # show current level

# Or via REST:
curl -X POST http://localhost:7432/api/v1/settings/model \
  -H "Content-Type: application/json" \
  -d '{"thinking": "high"}'

Applies uniformly whether the reply is generated by ModelRouter.generate() (the 32 channel adapters) or generate_stream() (the Web UI and voice assistant) — both read the same /think level.

Cost tracking

Token counts and estimated USD cost (from a static per-model pricing table) are tracked automatically per generation:

neuralcleave usage

Unpriced providers (OpenRouter, Azure, Bedrock, or any unrecognized model) are omitted from the cost total rather than reported as $0.00 — see Observability.

Privacy mode

With privacy_mode = true, NeuralCleave forces every request to Ollama regardless of the configured provider. Combined with local voice (faster-whisper + Kokoro) and local embeddings (all-MiniLM-L6-v2), zero bytes leave your machine. See Privacy Mode.

Getting Started

Core Concepts

Voice

Operations

Changelog

Supported providers

Configuration

Ollama (fully local)

Checking provider status

Switching providers at runtime

Reasoning effort (“thinking”)

Cost tracking

Privacy mode

​Supported providers

​Configuration

​Ollama (fully local)

​Checking provider status

​Switching providers at runtime

​Reasoning effort (“thinking”)

​Cost tracking

​Privacy mode

Supported providers

Configuration

Ollama (fully local)

Checking provider status

Switching providers at runtime

Reasoning effort (“thinking”)

Cost tracking

Privacy mode