Token use and report templates

[lingfish]

Is your feature request related to a problem?
One issue/question, and a suggestion.

Describe the solution you'd like

1. I know MCP is token heavy, but wow... I think a first initial query racked up over 100k token use. Am I missing something?
2. I would appreciate more and clearer doco on report generation, and what report templates are all about.

[hermanekt]

Hi Jason, thanks for the report! To help us investigate the token usage, could you share a few details:

1. MCP server version - which version are you running?
2. Zabbix version - which Zabbix version is the server pointing to?
3. Compact mode - do you have ZABBIX_COMPACT_MODE enabled? It significantly reduces token usage by stripping verbose fields from API responses.
4. Which tool/query triggered the 100k tokens? - was it a single tool call (e.g. host_get with no filter) or accumulated over a conversation?
5. Client - Claude Desktop, Claude Code, or something else?

Regarding report templates: this part is currently in a beta / first-concept state. The reasoning behind it is that LLMs aren't really able to produce consistent reports on their own, so we built a template system inside the MCP that handles the reporting logic deterministically. Templates are currently HTML-based.

We're internally experimenting with a few concepts to extend this area with better UI/management and an easier way to create templates, but for now what's shipped is intentionally minimal and we're collecting feedback. We'll definitely expand the documentation, including a worked example of how to author your own template. If there's a specific use case you have in mind, let us know - it helps us prioritize.

[hermanekt]

Hi Jason,

Following up on the documentation request - we just released v1.17 which includes:

• A new dedicated reporting guide: docs/REPORTING.md - covers architecture, all 4 built-in templates with their data sources, the report_generate tool API, branding, custom template authoring (admin portal and by hand), the full list of Jinja2 context variables exposed per report type, the CSS classes provided by base.html, a worked end-to-end example ("Problems Summary"), and the current limitations / roadmap.
• A new "PDF Reports (beta)" section in the README with a built-in template overview, example prompts, and a custom-template snippet.
• A documented [report_templates.*] example in config.example.toml.

There is also one small breaking change worth mentioning: in v1.16 the admin portal wrote custom HTML templates to /var/log/zabbix-mcp/templates/, which was an oversight (config files do not belong in a log directory). From v1.17 custom templates live in /etc/zabbix-mcp/templates/. The installer migrates existing files automatically - no manual action needed.

How to upgrade

cd zabbix-mcp-server
git fetch origin && git reset --hard origin/main
sudo ./deploy/install.sh update

The update will:

1. Pull v1.17 source.
2. Reinstall the package into the venv.
3. Run migrate_report_templates - creates /etc/zabbix-mcp/templates/ with zabbix-mcp:zabbix-mcp ownership and mode 0750, moves every *.html from the old location (preserving timestamps, mode 0640), rewrites template_file paths in [report_templates.*] sections of your config.toml via tomlkit (preserving comments and formatting), and removes the now-empty old directory.
4. Restart the service.

The migration is idempotent and safe to re-run. If you have no custom templates, it is a no-op.

Re: the original token usage question - we are still interested in the details from the previous comment (MCP server version, Zabbix version, whether compact_output is enabled, which tool/query triggered the 100k tokens, and the client). Token reduction is on our radar but we need to know which call shape is the worst offender to prioritize.

Release notes: https://github.com/initMAX/zabbix-mcp-server/releases/tag/v1.17

[lingfish]

Hi Jason, thanks for the report! To help us investigate the token usage, could you share a few details:

Yeah sorry, a tad lazy of me!

MCP server version - which version are you running?

v1.16 (in podman-compose)

Zabbix version - which Zabbix version is the server pointing to?

Zabbix 7.4.5 (full VM)

Compact mode - do you have ZABBIX_COMPACT_MODE enabled? It significantly reduces token usage by stripping verbose fields from API responses.

I see zero mention of this as a env var or config item, including in this repo; I'm going to assume you mean "Compact Output - Return only key fields from get methods (reduces LLM token usage)" in the admin UI, and if so, that's on (default).

Which tool/query triggered the 100k tokens? - was it a single tool call (e.g. host_get with no filter) or accumulated over a conversation?

Fresh session, first and only command was Show me all current problems

Context 104,621 tokens

I have 23 current problems in Zabbix, some of which are from disabled triggers (and Zabbix, being Zabbix, hasn't closed them off, and hides them in the UI, so your MCP server is useful in that respect)

8. Client - Claude Desktop, Claude Code, or something else?

OpenCode (using "MiniMax M2.5 Free OpenCode Zen" model, not that that should matter).

Regarding report templates: this part is currently in a beta / first-concept state. The reasoning behind it is that LLMs aren't really able to produce consistent reports on their own, so we built a template system inside the MCP that handles the reporting logic deterministically. Templates are currently HTML-based.

Yeah, nice. Might have to peek at how you're doing that for my own knowledge.

We're internally experimenting with a few concepts to extend this area with better UI/management and an easier way to create templates, but for now what's shipped is intentionally minimal and we're collecting feedback.

Yeah, ok, I just don't get how you expect feedback when it isn't even clear how to use it ;) (but I see you've done another comment update.)

[lingfish]

How to upgrade

cd zabbix-mcp-server
git fetch origin && git reset --hard origin/main
sudo ./deploy/install.sh update

Thanks for this detail, appreciated. I use podman though, so I don't use the deploy/install.sh script (I'm personally not a fan of running scripts for installs, especially as root, without some containerisation/separation.)

(I think that brings up another piece of feedback from me too, the script stuff is prickly.)

[hermanekt]

Hi Jason, thanks again for the detailed report. Your feedback turned into a full v1.18 release that shipped just now: https://github.com/initMAX/zabbix-mcp-server/releases/tag/v1.18

Before the release notes, a correction and an explanation.

The 100k tokens mystery

First, a correction to my earlier reply: I wrote ZABBIX_COMPACT_MODE but the actual option is compact_output (in [server] of config.toml, or the "Compact Output" checkbox in the admin portal). You already have it on, so that is not the problem. Apologies for the misleading name.

The real cause of your 104k tokens is almost certainly not the response - it is the tool catalog. With the default config, the MCP server exposes ~232 Zabbix API tools, and MCP clients load all of their schemas into the system prompt at the start of every session. Each Zabbix .get method has 20-40 optional filter fields in its JSON schema, so the serialized tool definition typically runs 400-500 tokens per tool. 232 tools * ~450 tokens = ~100k tokens of tool catalog before your first prompt even reaches the model.

compact_output only shrinks the response payload of each .get call. It does not reduce the tool catalog size. My previous answer was incomplete on this point.

The fix: filter the tool list aggressively

Add a tools allowlist (or disabled_tools denylist) to [server] in config.toml. Both accept either group names or individual API prefixes. For your "show me current problems" workflow, I would go straight to a narrow individual-prefix list rather than a whole group:

[server]
# Tight allowlist for problem triage / host inspection
tools = ["host", "hostgroup", "problem", "trigger", "event", "item"]

That gets you to ~15 tools, which is roughly 6-7k tokens of catalog instead of 100k. You still have problem_get, trigger_get, host_get, event_get, item_get, and the related create/update/delete variants, which is everything you need for "what is broken in my Zabbix right now" conversations.

If you want a slightly broader set (e.g. also templates and dashboards), add them individually:

tools = ["host", "hostgroup", "problem", "trigger", "event", "item",
         "template", "dashboard", "maintenance"]

Group names are also accepted as shortcuts, but they pull in more than you probably want:

Group	Tool count (approx)	Notes
monitoring	~31	host, item, trigger, problem, event, history, trend, graph, sla, discovery, httptest, ...
alerts	~16	action, alert, mediatype, script
data_collection	~107	template, templategroup, templatedashboard, valuemap, dashboard - biggest group by far
users	~30	user, usergroup, userdirectory, usermacro, token, role, mfa
administration	~39	settings, housekeeping, authentication, maintenance, map, proxy, ...
extensions	~9	graph_render, anomaly_detect, capacity_forecast, report_generate, ...

Default (nothing set) = all groups loaded = ~232 tools = the ~100k tokens you saw. The same filtering mechanism is exposed per-token via [tokens.*].scopes, so you can give each client a different slice. Lifting this guidance into the README with measured per-group token numbers is on my short-term TODO.

Report templates

Fair point on the feedback loop - you can't give feedback on something you can't figure out how to use. The reporting feature is explicitly a beta in our heads as well; v1.16 was the first-concept drop, v1.17 finally shipped the authoring guide that should have been there from day one, and v1.18 fixes the container-side rough edges that made it effectively unusable outside host installs (more on that below).

The main entry point is the "PDF Reports (beta)" section in the README, which links down to docs/REPORTING.md for the full authoring guide: a worked "Problems Summary" example, the Jinja2 context variables exposed per report type, and the CSS classes available in base.html. We consider the UI, the context shape, the template layout, and the storage path all fair game to change based on user feedback. If after reading the doc you still find parts confusing or missing, please tell me which parts.

v1.18 release - container parity with install.sh

Digging into your "scripting stuff is prickly" comment, I mapped what install.sh does for host installs that container deployments were silently missing. It turned out to be more than the one issue I first spotted, so this release fixes all three:

1. Container admin portal first-run bootstrap

This one surfaced from reading your feedback carefully. install.sh has a setup_admin step that auto-generates a random admin password, scrypt-hashes it, and writes [admin.users.admin] into config.toml before printing the credentials in a green terminal box. Container deployments never ran the installer, so a fresh container started from config.example.toml had admin.enabled = true but no [admin.users.*] entries. The admin portal was reachable, but every login attempt silently failed and the operator was completely locked out unless they manually hashed a password and edited the mounted config. Exactly the "scripting stuff" you were pushing back on.

Fixed by a new zabbix_mcp.admin_bootstrap module that runs at server startup. If the admin portal is enabled and no users exist, it generates a 16-character cryptographically random password, scrypt-hashes it with the same format as install.sh and admin/auth.py, writes [admin.users.admin] into config.toml via tomlkit (preserving comments and formatting), and prints the credentials prominently to stderr + the logger:

======================================================================
ADMIN PORTAL FIRST-RUN BOOTSTRAP

  Username: admin
  Password: xJIO1HL7dxtZzg1X

CHANGE THIS PASSWORD on first login via the admin portal
(/ -> user menu -> change password) or by editing
[admin.users.admin] in /etc/zabbix-mcp/config.toml directly.
======================================================================

Container operators now just run podman-compose up -d && podman logs zabbix-mcp-server | grep -A5 BOOTSTRAP and they have the password. Idempotent, non-fatal, a no-op on host installs (where install.sh already wrote the user) and on subsequent restarts.

2. Container report templates now persist across restarts

v1.17 moved custom templates from /var/log/zabbix-mcp/templates/ to /etc/zabbix-mcp/templates/ (which was correct - config files do not belong in a log directory), but the container setup never caught up. Dockerfile did not create the new directory, docker-compose.yml did not mount it as a persistent volume, and the legacy-to-current migration only ran in deploy/install.sh (which containers do not use). A custom template created via the admin portal inside a container would be written to a non-persistent path and lost on the next restart, and a v1.16 -> v1.17 container upgrade would strand existing templates in the old logs volume forever.

Fixed: Dockerfile now creates /etc/zabbix-mcp/templates/ with correct ownership and mode, docker-compose.yml mounts it as a named volume, and a new zabbix_mcp.template_migration Python module ports the bash migration and runs it at server startup. It moves any legacy *.html files to the new location (skipping any that already exist), rewrites template_file paths in [report_templates.*] sections via tomlkit (preserving comments and formatting), and removes the legacy directory if it ends up empty. Idempotent, non-fatal, no-op on fresh installs.

3. New --check-config CLI flag

install.sh test-config had no container equivalent - podman users had no way to validate config.toml without starting the container and reading the crash in the logs. v1.18 adds a --check-config flag that does the same offline validation without starting the server:

podman run --rm \
  -v ./config.toml:/etc/zabbix-mcp/config.toml:ro \
  ghcr.io/initmax/zabbix-mcp-server:latest \
  --config /etc/zabbix-mcp/config.toml --check-config

Reports TOML syntax errors, semantic errors (wrong types, invalid values), missing file, and permission errors with clear messages. Exits 0 on success, 1 on any validation failure. Loads through the exact same code path the server uses, so anything that passes here will start without config complaints.

Podman upgrade path (v1.16 / v1.17 -> v1.18)

For your podman-compose setup:

cd zabbix-mcp-server
git pull
podman-compose down
podman-compose up -d --build

The migration logic is fully automatic and runs once on the next startup. No manual file moves, no hash recomputation, no editing config.toml by hand. Your existing admin user (if any) is preserved - the bootstrap only runs when there are zero users, so it will not overwrite what you already have. Same for custom templates: the migration only runs when legacy files exist at the old path, and it skips any file that already exists at the new location.

The long-term direction is still a pre-built image on GHCR so you can drop the git pull entirely, but that is still ahead of us.

Test coverage

Before tagging the release I ran the full matrix on a local Docker host:

• 18 installer distro tests (tests/installer/run_all.sh) - RHEL 8/9/10, AlmaLinux 8/9/10, Oracle 8/9/10, Fedora, Amazon 2023, openSUSE 15, Ubuntu 22.04/24.04, Debian 12/13, minimal python3.10-slim. All pass (including rejections where the distro only ships Python 3.9).
• Fresh container install: bootstrap banner visible in docker logs, POST /login with captured credentials returns 303 + session cookie, GET / returns 200 + correct dashboard.
• Container restart idempotency: no new password generated on second start, hash unchanged.
• Container upgrade v1.17 -> v1.18 with an existing admin user: no bootstrap banner, admin hash preserved, login with original password works.
• Container upgrade v1.16 -> v1.18 with legacy custom templates + existing admin user: template_file path rewritten, HTML files moved to new persistent volume, admin hash preserved, login works.
• Host install.sh install (Ubuntu 24 in a container): green ASCII box shown with URL, username, password.
• Host install + server start: Python bootstrap sees the existing user and is a no-op (no double bootstrap).
• Host install.sh update (v1.17 baseline -> v1.18) with user customizations: custom port 18080 # my custom port, [tokens.my_custom] section, and user comments all preserved verbatim. Admin hash preserved.
• --check-config: valid, broken TOML, semantic error, missing file - all return the right exit code and clear message.

While setting this up I also caught an unrelated pre-existing bug in tests/installer/run_all.sh where the ((PASS++)) bash post-increment was aborting the test loop after the first distro under set -e. Fixed in the same release.

A bit of context on where this is headed

One thing worth mentioning: MCP is a strategic direction for us across multiple products at initMAX, not just this server. We also have an existing ChatGPT widget product that we plan to extend with MCP support, and the same patterns being worked out in this repo (tool filtering for context budget, reporting templates, the auth model) will feed directly into that widget. So the feedback you are giving on the server maps onto a broader roadmap - concretely, your "232 tools is too much for one session" observation is exactly the kind of thing we need to solve cleanly here before layering it into the widget.

Summary of action items

• Fix container templates persistence (v1.18)
• Fix container admin portal first-run bootstrap (v1.18)
• Add --check-config CLI flag for offline config validation in containers (v1.18)
• Fix tests/installer/run_all.sh bash gotcha so the full distro matrix runs to completion (v1.18)
• Document "tool filtering for token budget" prominently in README with measured per-group token numbers
• Publish a pre-built container image to GHCR so the container upgrade path does not require git pull

Thanks again for the detailed report. This turned out to surface three real container-side gaps plus a dormant test tooling bug, so it was well worth the dig.

[lingfish]

Hi Tomas, wow, this was in-depth, love it!

The 100k tokens mystery

First, a correction to my earlier reply: I wrote ZABBIX_COMPACT_MODE
Apologies for the misleading name.

All good, no need to apologise.

The real cause of your 104k tokens is almost certainly not the response - it is the tool catalog.

Indeed it was.

The fix: filter the tool list aggressively

Add a tools allowlist (or disabled_tools denylist) to [server] in config.toml.

Yep, so I went with:

tools = ["host", "hostgroup", "problem", "trigger", "event", "item", "template"]

And then, first/same prompt as above, I see 31,688 tokens. Much better. And unsurprisingly faster.

If you want a slightly broader set (e.g. also templates and dashboards), add them individually:

Thanks again for the detail here. Just re-checked the README and I think it'd be worth calling out this "default" all tools token impact near the top.

See below for another question from me on how to make Zabbix template updates easier.

Report templates

We consider the UI, the context shape, the template layout, and the storage path all fair game to change based on user feedback. If after reading the doc you still find parts confusing or missing, please tell me which parts.

Will have a look at this more over the next few days.

v1.18 release - container parity with install.sh

Digging into your "scripting stuff is prickly" comment, I mapped what install.sh does for host installs that container deployments were silently missing. It turned out to be more than the one issue I first spotted, so this release fixes all three:

Nice, much appreciated. Seems to have gone without a hitch (except of course, git refusing to pull when docker-compose.yml is modified locally).

1. Container admin portal first-run bootstrap

This one surfaced from reading your feedback carefully. install.sh has a setup_admin step that auto-generates a random admin password, scrypt-hashes it, and writes [admin.users.admin] into config.toml before printing the credentials in a green terminal box. Container deployments never ran the installer, so a fresh container started from config.example.toml had admin.enabled = true but no [admin.users.*] entries. The admin portal was reachable, but every login attempt silently failed and the operator was completely locked out unless they manually hashed a password and edited the mounted config. Exactly the "scripting stuff" you were pushing back on.

Yup, I neglected to mention this one as well and had to manually setup the hash. As mine was already setup, I haven't tested your improvements but thanks for bringing parity.

Container operators now just run podman-compose up -d && podman logs zabbix-mcp-server | grep -A5 BOOTSTRAP and they have the password. Idempotent, non-fatal, a no-op on host installs (where install.sh already wrote the user) and on subsequent restarts.

👏

2. Container report templates now persist across restarts

👍

3. New --check-config CLI flag

Nice.

Podman upgrade path (v1.16 / v1.17 -> v1.18)

For your podman-compose setup:

cd zabbix-mcp-server
git pull
podman-compose down
podman-compose up -d --build

Yep, worked fine but as mentioned above, a git stash was suggested by git of course. Par for the course there.

The long-term direction is still a pre-built image on GHCR so you can drop the git pull entirely, but that is still ahead of us.

👍

A bit of context on where this is headed

[ ... ] So the feedback you are giving on the server maps onto a broader roadmap - concretely, your "232 tools is too much for one session" observation is exactly the kind of thing we need to solve cleanly here before layering it into the widget.

Lovely, glad to have helped a little.

Thanks again for the detailed report. This turned out to surface three real container-side gaps plus a dormant test tooling bug, so it was well worth the dig.

And thanks for your detailed reply and work.

Now... whilst attempting to make an older Zabbix template for lm-sensors a bit more modern, I went into battle with LLMs and Zabbix picky YAML (this actually has nothing to do with your project; I've had a lot of trouble generating valid templates with basically any model).

My question: rather than what I did (because I'm cautious of "read/write" modes etc), I to-and-fro'd with the model, import -> error -> LLM chat -> rinse repeat... it would have been nicer to have your MCP do the API uploads, get the error right away and continue the cycle. What do you suggest as a config to lock down to mostly RO except allowing MCP write access to create/update templates?

I would obviously need read_only = false, but then what else to allow template CRUD, but RO for the rest? Perhaps a good example for the doco too?

[hermanekt]

Following up with v1.19 (just released) which addresses your template editing question directly.

What's new in v1.19 for your workflow

response_max_chars is now configurable - the 50k character truncation limit that would have broken configuration_export on any non-trivial template is now a config option:

[server]
response_max_chars = 200000   # default: 50000, min: 5000

Also available in admin portal (Settings -> MCP Server -> Response Max Chars). At 200k you get most medium-sized templates in full; 500k covers the large built-in ones.

Additionally, string results (YAML/XML from export tools) now return partial content with a truncation note instead of the old useless {"_error": "Result too large to return"} summary. So even if a template exceeds the limit, you get the first N characters of actual YAML plus a note telling you to increase the limit.

Release: https://github.com/initMAX/zabbix-mcp-server/releases/tag/v1.19

Template editing - practical approach

For the lm-sensors template workflow you described, you don't necessarily need a full YAML round-trip. Three options depending on what you're doing:

A) Direct API calls (no YAML needed): Zabbix treats templates as hosts in the API. You can item_create / trigger_create / discoveryrule_create directly with the template ID as hostid. The LLM makes small, targeted calls - no 100KB YAML generation needed.

"Add a new item system.cpu.util[,idle] to template 10001 with change-per-second preprocessing"
-> LLM calls item_create(hostid="10001", name="CPU idle", key_="system.cpu.util[,idle]", ...)

B) Import user-provided YAML via MCP: If you have the YAML already (from Zabbix UI export, a file, or a previous conversation), paste it into the chat and the LLM forwards it to configuration_import. There is no input size limit on our side - the full YAML goes through as a tool parameter. The Zabbix error comes back in the response, the LLM fixes it, re-imports. This is your import-error-fix loop and it works today.

C) Export + modify + re-import (with increased limit): Set response_max_chars = 200000 or higher. configuration_export returns the template YAML (up to the limit), the LLM modifies it, configuration_import sends it back. Works for templates that fit within the configured limit.

Config for this

[server]
response_max_chars = 200000
tools = ["template", "configuration", "host", "hostgroup", "item", "trigger", "discoveryrule"]

One caveat on scoping: the MCP tool catalog sent at session start is the same for all tokens (filtered by [server].tools, not by per-token scopes). Token scopes are enforced at call time - unauthorized calls return an error, but the LLM still sees the tool listed. Per-session dynamic tool filtering is on the roadmap.

There is also a new "Token Budget" section in the README with per-group tool counts and config examples, based on your earlier feedback.