Architecture

One daemon, four cooperating systems: a FastAPI gateway, an agent runtime pipeline, a 3-tier memory store, and a plugin registry. Local-first — runs on your own machine or any server with no external dependencies beyond the LLM API.

System overview

When a message arrives from any channel adapter, it flows through this pipeline:

Channel Adapter  →  Gateway (FastAPI)  →  AgentRuntime
                                              │
                        ┌─────────────────────┤
                        │                     │
                   MemoryManager         ModelRouter
                        │                     │
                  Redis (hot)         ┌── Anthropic
                  Qdrant (vec)        ├── OpenAI
                  SQLite (long)       ├── Gemini
                        │             └── … 10 more
                        └─────────────────────┤
                                              │
                                     ReflectionEngine
                                     (quality score + retry)
                                              │
                                         Response  →  Channel Adapter

Gateway

The gateway is a FastAPI + uvicorn application, default port 7432.

• 41 REST endpoints under /api/v1/ — chat, memory, plugins, hub, orchestrator, canvas, voice, settings, metrics
• 2 WebSocket endpoints — /ws/chat (streaming) and /ws/canvas (live canvas)
• Full OpenAPI schema via FastAPI's built-in /docs and /redoc
• Optional X-API-Key middleware on all /api/* routes; /health and /ws/* are always exempt
• Built in lifespan context manager wires up: PluginRegistry, HeartbeatScheduler, CanvasRenderer, PushManager, MemoryManager

Process model

NeuralCleave is a single-process daemon. Each channel adapter runs as an asyncio task within the same event loop — no separate worker processes. The gateway's --background flag detaches with subprocess.Popen, writing a PID to ~/.neuralcleave/gateway.pid.

Agent runtime pipeline

Each inbound message passes through four stages:

1. Intent extraction

A lightweight call to a cheap model (cheap_inference task type — e.g. GLM-4-flash or Doubao-lite) parses the message into a structured intent. This is used for tool dispatch and routing decisions.

2. Tool execution

If the intent triggers a tool, it executes before the LLM call:

• WebSearchTool — DuckDuckGo Instant Answer + SearXNG fallback
• FileOpsTool — 10 file operations (read/write/append/list/delete/move/copy/mkdir/stat/search)
• ShellTool — allowlist-sandboxed subprocess; shell=False always; 50 KB cap
• BrowserTool — headless Chromium via Playwright; 10 actions; domain allowlist
• CanvasTool — 9 LLM-callable canvas actions; WebSocket broadcast
• WriteSkillTool / ListSkillsTool / DeleteSkillTool — dynamic skill authoring
• Any tool registered by a loaded plugin

3. ModelRouter

The router selects the optimal model based on task type, complexity, and provider availability. Ten task types are defined:

Auto-complexity detection — keyword scan + word-count threshold upgrades simple tasks to complex_reasoning automatically.

Privacy mode — POST /api/v1/settings/privacy forces all inference to local Ollama regardless of task type.

4. ReflectionEngine

After the model returns, the response is scored on 4 dimensions (0–100 each):

• Relevance — does the response address what was asked?
• Completeness — is anything material missing?
• Accuracy — are facts plausible given the context?
• Tone — is the register appropriate for the channel?

If the composite score falls below the configured threshold (default 70), the engine re-prompts the model with explicit improvement instructions. It accepts the retry only if the new score is higher than the original — preventing infinite loops. Maximum 1 retry per message. Scores are published to the generation_quality_score Prometheus histogram.

Memory system

Three storage tiers, each with a distinct role:

• Redis — hot tier. Per-session key-value store with a 1-hour TTL. Holds the current conversation context, rolling message window, and active tool state. Falls back to in-memory dict if Redis is unavailable.
• Qdrant — semantic tier. ANN index (cosine similarity) for vector search over past conversations and documents. MD5 dedup prevents re-indexing identical content. Falls back gracefully if Qdrant is unavailable.
• SQLite — long-term tier. Importance-scored rows (0.0–1.0), tags, full-text search, session archiver. Stored at ~/.neuralcleave/memory.db.

Full details in Memory System.

Plugin system

NeuralCleave uses Python's standard PEP 451 entry-points (importlib.metadata) for plugin discovery. A plugin package registers itself under the neuralcleave.plugins group in its pyproject.toml:

[project.entry-points."neuralcleave.plugins"]
my-plugin = "my_plugin.plugin:MyPlugin"

PluginRegistry discovers all registered plugins on gateway startup, calls on_load(), and wires their tools and channel adapters into the running system. Plugins can be hot-reloaded (POST /api/v1/plugins/{name}/reload) without restarting the gateway.

See Plugin SDK for the full authoring guide.

Multi-agent orchestrator

AgentOrchestrator routes tasks to named sub-agents (nodes), each with its own model override, task-type filter, channel pattern, keyword filter, and private memory namespace.

Routing priority order:

1. Filter to nodes where can_handle(task) returns True
2. Use the designated fallback node if no match
3. Raise NoEligibleNodeError if still none
4. Pick the highest-priority eligible node
5. Round-robin tie-break per task_type

Each node owns a private MemoryNamespaceStore (ordered LRU, configurable max_entries). Nodes can share a namespace by setting the same memory_namespace in their config.

Heartbeat scheduler

HeartbeatScheduler is a background asyncio task that fires registered handlers on a cron or interval schedule — without any external cron dependency. It is wired into the FastAPI lifespan and accessible as app.state.scheduler.

Supports 5-field cron expressions (* * * * *), */n step syntax, ranges, and comma-separated lists. Handlers can send outbound messages on any registered channel adapter — this is how NeuralCleave implements proactive / autonomous behaviour.

Canvas (A2UI)

CanvasRenderer maintains a ring buffer of up to 200 blocks (text, markdown, code, image, table, chart, html) and broadcasts state changes over WebSocket to all connected viewers. The LLM calls CanvasTool to render content; a live HTML page at /canvas renders it in-browser using the Canvas API — no CDN dependencies.

Configuration path

All user data lives under ~/.neuralcleave/: