Skip to content

feat(controller): add agent tool catalogue - #776

Draft
elikoga wants to merge 8 commits into
masterfrom
feat/ai-agent-tools
Draft

feat(controller): add agent tool catalogue#776
elikoga wants to merge 8 commits into
masterfrom
feat/ai-agent-tools

Conversation

@elikoga

@elikoga elikoga commented Jul 13, 2026

Copy link
Copy Markdown
Member

Summary

  • add a provider-neutral, schema-discoverable Python tool catalogue for Thymis controller operations
  • register tools with a decorator, keeping schemas and descriptions next to their implementations instead of maintaining a separate hand-written registry
  • cover configuration, repository, tasks, secrets, artifacts, logs, fleet, controller settings, and deployment actions
  • add live device telemetry tools for hardware, connection history, metrics, and recent error logs
  • add run_device_command, which queues a root command on a connected device through the existing relay/task infrastructure
  • add targeted patch_configuration_field using RFC 6901 JSON Pointer mutations; it preserves unrelated configuration fields and shares the normal connected-device type safety gate
  • add repository visibility tools for remote branches, tags, flake prefetch checks, and API-access checks
  • add a bottom-right Thymis Assistant window using AI SDK UI streaming, Markdown responses, tool activity, safe dashboard navigation, entity links, and optional VNC screenshots
  • add Kiosk i3 session controls: inspect outputs, read the active i3 configuration to discover its exact Chromium store path, inspect current-boot display journals, restart display-manager.service, and run explicitly requested graphical commands as thymiskiosk
  • treat every controller as production: diagnostics may be proactive, but deployments and every other mutation require an explicit operator request in the current conversation
  • preserve the existing signed-in controller API boundary: the assistant cannot access secrets, issue generic root commands, or delete records

Local assistant setup

The controller runs normally without an LLM. To enable the assistant locally:

cd controller
cp .env.example .env
export OPENROUTER_API_KEY=''
uv run uvicorn thymis_controller.main:app --reload

.env.example sets THYMIS_AGENT_MODEL=openrouter:openai/gpt-4.1-mini. Use a model that supports tool calls and image input. The provider key intentionally remains a shell environment variable rather than a repository or .env secret. Sign in at http://localhost:8000, then open Thymis Assistant in the bottom-right corner.

Verification

  • uv run --python python3.13 pytest tests/test_agent_tools.py tests/routes/test_state_device_type_gate.py tests/routes/test_switch_config_api.py
  • uv run --python python3.13 pytest tests/test_agent_tools.py tests/test_agent_runtime.py tests/routes/test_agent_chat_api.py — 16 passed
  • uv run --python python3.13 pytest tests/test_agent_tools.py tests/test_agent_runtime.py — 17 passed
  • uv run --python python3.13 pytest tests/test_agent_runtime.py — 8 passed
  • uvx ruff check --target-version py312 thymis_controller/agent_runtime.py thymis_controller/agent_tools/arguments.py thymis_controller/agent_tools/toolbox.py tests/test_agent_runtime.py tests/test_agent_tools.py
  • nix develop .#forNpmTesting -c env THYMIS_DEV_SHELL=true UVICORN_PORT=8002 npx playwright test tests/agent-chat.spec.ts --workers=1 --retries=0 — 6 passed
  • npm run build
  • commit hooks: Python formatting and npm run check, format, lint in ./frontend

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant