LLM Providers

NeuralCleave ships 13 production-ready providers — including all major Western and Chinese models — with task-aware automatic routing, extended thinking support, and a privacy mode that forces everything local.

All providers

Provider	Config alias	Env var	Base URL	Protocol
Anthropic	`anthropic`, `claude`	`ANTHROPIC_API_KEY`	api.anthropic.com	Anthropic SDK
OpenAI	`openai`, `gpt`	`OPENAI_API_KEY`	api.openai.com	OpenAI-compat SSE
Google Gemini	`gemini`, `google`	`GEMINI_API_KEY`	generativelanguage.googleapis.com	Gemini REST
DeepSeek	`deepseek`	`DEEPSEEK_API_KEY`	api.deepseek.com	OpenAI-compat SSE
Ollama	`ollama`, `local`	—	localhost:11434	Ollama REST
Mistral AI	`mistral`	`MISTRAL_API_KEY`	api.mistral.ai	OpenAI-compat SSE
xAI Grok	`xai`, `grok`	`XAI_API_KEY`	api.x.ai	OpenAI-compat SSE
Cohere	`cohere`	`COHERE_API_KEY`	api.cohere.com	Cohere v2 chat
Moonshot / Kimi	`moonshot`, `kimi`	`MOONSHOT_API_KEY`	api.moonshot.cn	OpenAI-compat SSE
Zhipu GLM	`zhipu`, `glm`	`ZHIPUAI_API_KEY`	open.bigmodel.cn	OpenAI-compat SSE
Alibaba Qwen	`qwen`, `alibaba`, `dashscope`	`DASHSCOPE_API_KEY`	dashscope.aliyuncs.com	OpenAI-compat SSE
Baidu ERNIE	`ernie`, `baidu`	`QIANFAN_API_KEY`	qianfan.baidubce.com	Qianfan REST
ByteDance Doubao	`doubao`, `bytedance`, `ark`	`ARK_API_KEY`	ark.cn-beijing.volces.com	OpenAI-compat SSE

Configuring providers

Set your API key in ~/.neuralcleave/config.toml. Use ENV: prefix to read from an environment variable instead of hardcoding:

toml

[models]
primary_provider = "anthropic"
primary_model    = "claude-sonnet-4-6"

# Hardcoded key (not recommended for shared configs)
anthropic_api_key = "sk-ant-..."

# Preferred: resolve from environment at runtime
openai_api_key    = "ENV:OPENAI_API_KEY"
gemini_api_key    = "ENV:GEMINI_API_KEY"
deepseek_api_key  = "ENV:DEEPSEEK_API_KEY"
mistral_api_key   = "ENV:MISTRAL_API_KEY"
xai_api_key       = "ENV:XAI_API_KEY"
cohere_api_key    = "ENV:COHERE_API_KEY"
moonshot_api_key  = "ENV:MOONSHOT_API_KEY"
zhipuai_api_key   = "ENV:ZHIPUAI_API_KEY"
dashscope_api_key = "ENV:DASHSCOPE_API_KEY"
qianfan_api_key   = "ENV:QIANFAN_API_KEY"
ark_api_key       = "ENV:ARK_API_KEY"

Task-aware routing

The ModelRouter selects the best available model based on the detected task type. Ten task types are defined — see Architecture → ModelRouter for the full mapping table.

Force a specific provider at runtime:

bash

# Switch primary model via REST
curl -X POST http://localhost:7432/api/v1/settings/model \
  -H "Content-Type: application/json" \
  -d '{"provider": "gemini", "model": "gemini-2.0-flash"}'

Or in config with forced_provider:

toml

[models]
forced_provider = "deepseek"   # accepts alias: "kimi", "baidu", "bytedance", etc.

Anthropic Claude

NeuralCleave uses the official anthropic Python SDK. Recommended models:

Model	Best for
`claude-sonnet-4-6`	General use (default)
`claude-opus-4-8`	Complex reasoning, research
`claude-haiku-4-5-20251001`	Fast, low-cost inference

Extended thinking (Anthropic)

Extended thinking activates Claude's reasoning process before answering. NeuralCleave exposes it via config and per-request:

toml

[models]
extended_thinking        = true
thinking_budget_tokens   = 8000   # tokens allocated to reasoning chain

When active, temperature is forced to 1.0 (Anthropic requirement) and the reasoning chain is streamed as a separate thinking field in WebSocket events.

ℹ

Extended thinking is only available on Anthropic models. The flag is silently ignored for all other providers.

Privacy mode (Ollama-only)

Privacy mode routes all inference — regardless of task type — to the local Ollama instance. No data leaves your machine.

bash

# Enable via REST
curl -X POST http://localhost:7432/api/v1/settings/privacy \
  -d '{"enabled": true}'

# Or via CLI
neuralcleave settings privacy --on

toml

[models]
ollama_url   = "http://localhost:11434"
ollama_model = "llama3.2"   # any model pulled locally

OpenAI

Uses the OpenAI-compatible SSE streaming protocol. Tested models: gpt-4o, gpt-4o-mini, gpt-4-turbo, o1-preview, o3-mini.

Google Gemini

Uses Google's REST API directly. Tested models: gemini-2.0-flash, gemini-2.0-flash-lite, gemini-1.5-pro.

DeepSeek

OpenAI-compatible. Tested models: deepseek-chat, deepseek-reasoner. Excellent price/performance for code tasks.

Mistral AI

OpenAI-compatible. Tested models: mistral-large-latest, mistral-small-latest, codestral-latest.

xAI Grok

OpenAI-compatible (api.x.ai). Tested models: grok-3, grok-3-fast. Used for complex_reasoning routing.

Cohere Command-R

Uses Cohere's v2 chat API (not OpenAI-compatible — NeuralCleave implements a bespoke response parser). Tested models: command-r-plus-08-2024, command-r-08-2024.

Moonshot / Kimi

OpenAI-compatible (api.moonshot.cn). Alias: kimi. Tested models: moonshot-v1-8k, moonshot-v1-32k.

Zhipu GLM

OpenAI-compatible (open.bigmodel.cn). Alias: glm. Tested models: glm-4-flash, glm-4-air. Fast and cheap — used for intent_extraction and cheap_inference tasks by default.

Alibaba Qwen / DashScope

OpenAI-compatible (dashscope.aliyuncs.com). Aliases: alibaba, dashscope. Tested models: qwen-max, qwen-turbo, qwen-long. Used for code_generation tasks by default.

Baidu ERNIE / Qianfan

Uses Baidu's Qianfan API. Alias: baidu. Tested models: ERNIE-4.0-8K, ERNIE-Speed-128K.

ByteDance Doubao / Ark

OpenAI-compatible (ark.cn-beijing.volces.com). Aliases: bytedance, ark. Tested models: doubao-lite-32k, doubao-pro-32k. Used for cheap_inference by default.