Skip to content

README.md

Latest commit

 

History

History
237 lines (179 loc) · 5.54 KB

README.md

File metadata and controls

237 lines (179 loc) · 5.54 KB
Raw

Hermes Burn Tool Router

A tiny Rust/Burn neural router for Hermes Agent control-path selection.

This repo demonstrates a fast pre-LLM classifier for Hermes-style agents. Instead of spending a full frontier-model call to decide “which tool family does this message need?”, the router predicts the likely route locally in sub-millisecond time and emits a safe hint that Hermes can use to narrow or prioritize toolsets.

user message
   │
   ▼
MD5 hash embedding (384 dim)
   │
   ▼
170K-param Burn model
   │
   ▼
Hermes route/category + confidence
   │
   ▼
optional enabled_toolsets hint, with fallback to full surface

Current Hermes status

This router is not currently wired into live Hermes Agent by default. Today Hermes still builds tool schemas from its normal toolsets.py / model_tools.py path.

That is intentional for now: routing should start as an advisory sidecar, not a hard gate. A wrong hard route can hide the correct tool from the main LLM. The safe progression is:

  1. observe-only logging;
  2. high-confidence toolset hints;
  3. narrow first-pass routing with fallback to full tool surface.

See docs/hermes-integration.md for the opt-in integration sketch.

Why this exists

Agent systems are increasingly bottlenecked by routing overhead, context bloat, and unnecessary model calls. A small local model can handle cheap classification so the expensive LLM is reserved for actual reasoning.

This project is intentionally hackable:

  • Rust/Burn inference binary;
  • PyTorch training/export script;
  • SafeTensors model artifact;
  • parity vectors to verify Rust output against PyTorch;
  • Hermes-specific route categories, including CLI/control commands and modern toolsets.

Categories

The refreshed model predicts 19 categories aligned with current Hermes toolsets:

  • terminal
  • file
  • web
  • x_search
  • browser
  • memory
  • skills
  • delegation
  • media_generation
  • media_analysis
  • messaging
  • cron
  • hermes_cli
  • todo
  • smart_home
  • kanban
  • social_platforms
  • productivity
  • computer_use

hermes_cli covers commands like:

  • hermes config
  • hermes tools enable web
  • hermes skills list
  • hermes gateway restart
  • hermes cron list
  • /model
  • /reasoning high
  • /fast
  • /commands
  • /restart
  • /platforms

Model

MD5 hash embedding (384 dim)
→ Linear(384→256)
→ GELU
→ LayerNorm
→ Linear(256→256)
→ GELU
→ LayerNorm
→ Linear(256→19)
→ Softmax

Current metadata:

  • Parameters: 170,259 (~170K)
  • Validation accuracy: 100.0% on refreshed synthetic Hermes routing patterns
  • Rust/Burn parity: max local diff around 0.0000134
  • Batched throughput on Apple Metal: ~0.60µs/item at batch 1024 in local benchmark
  • Single-request predict includes model/GPU overhead and is usually ~600–750µs after warmup in local CLI tests

Build

git clone https://github.com/ivanontech/hermes-burn-tool-router.git
cd hermes-burn-tool-router
cargo build --release

Predict

./target/release/hermes-burn-tool-router predict \
  "hermes tools enable web" \
  tool_router.safetensors

Expected category:

"hermes_cli"

X/Twitter example:

./target/release/hermes-burn-tool-router predict \
  "search X for trending Base coins" \
  tool_router.safetensors

Expected category:

"x_search"

File-path example:

./target/release/hermes-burn-tool-router predict \
  "read /tmp/foo.txt" \
  tool_router.safetensors

Expected category:

"file"

Route hint shim

scripts/route_hint.py "search X for trending Base coins"

Example output:

{
  "category": "x_search",
  "confidence": 1.0,
  "enabled_toolsets": ["x_search"],
  "mode": "narrow"
}

The shim only narrows toolsets when confidence is above the threshold. Otherwise it returns fallback_full_surface.

Verify Burn/Rust parity against PyTorch test vectors

./target/release/hermes-burn-tool-router \
  tool_router.safetensors \
  test_vectors.safetensors

Important expected line:

[VERIFY] ✅ Matches PyTorch

Retrain

python3 train_tool_router.py

Retraining regenerates:

  • tool_router.safetensors
  • test_vectors.safetensors
  • tool_router_meta.json

Files

src/main.rs                 Burn/Rust inference binary
train_tool_router.py        PyTorch training/export script
scripts/route_hint.py       optional Hermes enabled_toolsets hint shim
docs/hermes-integration.md  safe opt-in integration sketch
tool_router.safetensors     exported model weights
test_vectors.safetensors    parity vectors for Rust verification
tool_router_meta.json       categories, toolsets, and training metadata
Cargo.toml                  Rust dependencies

Verification

python3 train_tool_router.py
cargo fmt --check
cargo check
cargo build --release
./target/release/hermes-burn-tool-router tool_router.safetensors test_vectors.safetensors
scripts/route_hint.py "search X for trending Base coins"

Public safety

This release intentionally includes no API keys, .env files, local Hermes config, auth files, session logs, private datasets, wallet material, or build artifacts. The included model/data artifacts are public demo artifacts for the router.

Related

License

MIT