Testing

NeuralCleave has 5,064 tests across unit and integration suites, all running under pytest with pytest-asyncio. Tests live in tests/ and cover every core subsystem.

Running the test suite

bash

# Install test dependencies
pip install -e ".[dev]"

# Run the full suite
pytest

# Run with verbose output
pytest -v

# Run a specific test file
pytest tests/unit/test_memory.py -v

# Run tests matching a keyword
pytest -k "telegram" -v

# Run with coverage report
pytest --cov=backend --cov-report=term-missing

ℹ

All async tests use pytest-asyncio with asyncio_mode = "auto" in pyproject.toml, so no @pytest.mark.asyncio decorator is needed on individual tests.

Coverage by module

Module / area	Tests	What's covered
Gateway (FastAPI routes)	~620	All 41 REST endpoints, auth middleware, WebSocket protocol
Memory system	~480	Redis session, Qdrant semantic, SQLite long-term, compaction, archiver, namespaces
LLM providers	~390	All 13 providers: success paths, error handling, retries, extended thinking, privacy mode
Channels	~640	All 32 adapters: inbound parsing, outbound send, auth, reconnect
ReflectionEngine	~180	4 dimensions, score thresholds, retry logic, pass-through when score ≥ 80
Orchestrator	~260	Node registration, routing algorithm, priority, disable, namespace isolation
Plugin SDK	~310	Tool execution, Plugin lifecycle, entry-point discovery, hot-reload, ChannelAdapter base
Hub	~220	Package install/uninstall, enable/disable, PackageScanner, registry search
Canvas	~140	Block rendering, WebSocket broadcast, state retrieval, clear
Voice	~190	Whisper STT (mocked), TTS provider fallback chain, wake-word trigger logic
CLI	~210	All subcommands with mocked gateway: start, stop, status, init, chat, plugins, hub, voice
Config	~180	TOML parsing, ENV: resolution, validation, NEURALCLEAVE_CONFIG override
Observability	~144	Metric increments, JsonFormatter output, ContextLogger field emission
HeartbeatScheduler	~100	Task registration, cron trigger, disable flag

Test file structure

text

tests/
├── unit/
│   ├── test_memory.py
│   ├── test_llm_providers.py
│   ├── test_channels.py          ← one file per adapter group
│   ├── test_reflection.py
│   ├── test_orchestrator.py
│   ├── test_config.py
│   ├── test_config_extended.py
│   ├── test_gateway_auth.py
│   ├── test_orchestrator_routing_extended.py
│   └── test_update_checker_edge_cases.py
├── integration/
│   ├── test_gateway_routes.py    ← real FastAPI TestClient
│   ├── test_memory_integration.py
│   └── test_plugin_lifecycle.py
└── conftest.py                   ← shared fixtures

Writing a new test

Tests use standard pytest conventions. For async gateway tests, use httpx.AsyncClient with the FastAPI app:

python

import pytest
from httpx import AsyncClient, ASGITransport
from backend.gateway.app import create_app

@pytest.fixture
async def client():
    app = create_app()
    async with AsyncClient(
        transport=ASGITransport(app=app), base_url="http://test"
    ) as c:
        yield c

async def test_health(client):
    resp = await client.get("/health")
    assert resp.status_code == 200
    assert resp.json()["status"] == "ok"

async def test_chat_returns_response(client):
    resp = await client.post("/api/v1/chat", json={
        "message": "Hello",
        "session_id": "test-session",
    })
    assert resp.status_code == 200
    data = resp.json()
    assert "response" in data
    assert "session_id" in data

For unit-testing a channel adapter:

python

from unittest.mock import AsyncMock, patch
from backend.channels.telegram import TelegramAdapter

async def test_telegram_inbound_parsed():
    adapter = TelegramAdapter(bot_token="fake-token")
    raw_update = {
        "message": {
            "from": {"id": 12345},
            "text": "Hello bot",
            "chat": {"id": 12345},
        }
    }
    msg = adapter._parse_update(raw_update)
    assert msg.channel == "telegram"
    assert msg.text == "Hello bot"
    assert msg.sender_id == "12345"

CI integration

GitHub Actions runs three parallel test jobs on every push and pull request:

Job	Workflow file	What it runs
Core tests	`.github/workflows/ci.yml`	`pytest tests/unit/ tests/integration/`
Plugin tests	`.github/workflows/plugins.yml`	Installs each example plugin, runs its test suite
Docs deploy	`.github/workflows/deploy-docs.yml`	Deploys `docs-site/` to GitHub Pages on main push

💡

LLM provider tests are mocked by default — no real API keys required for CI. The fixture in conftest.py patches all provider HTTP clients with pre-recorded responses.