Feature(#3816): Make sure MCP pod image is up to date#4331
Open
mchugunov wants to merge 37 commits intoarchestra-ai:mainfrom
Open
Feature(#3816): Make sure MCP pod image is up to date#4331mchugunov wants to merge 37 commits intoarchestra-ai:mainfrom
mchugunov wants to merge 37 commits intoarchestra-ai:mainfrom
Conversation
…rt after mcp server image update
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds automatic freshness checks for installed local MCP servers.
In scope:
User-facing changes
When editing a local MCP server, user has an option to either enable or disable if the server will be part of the freshness check, as well as to opt in or out of automatic reinstalls once checking task resolves a newer version of the image.
By default, checks and auto-reinstall are off for already existing servers, and on for all the newly created.
In the MCP server configuration modal, user sees information about the check update status, last success / fail, last error, last reinstallation timestamp, and both running and the newest available digests.
HTTP API exposed: new PATCH /api/mcp-server/:id is exposed to support toggle updates
Backend flow
A new task type
check_mcp_image_updatesis introduced and it handles detection and processing of the incoming updates. By default it runs every 15 minutes, yet the interval can be configured via environment variable.Only servers that satisfy the following conditions are considered as eligible for freshness check: a server must be local docker-enabled one, the "check for image updates" toggle must be on for this server, it must be in running (installed) state, not marked for manual reinstall, not disabled or in pending or error state.
Freshness checks are safeguarded by database-backed per-server locks. Once a lock is acquired, system spins up a probe pod following current Kubernetes settings (same namespace, scheduling constraints, service account, image pull secrets, etc. as the original MCP server). No matter the result of the probe instantiation, it's cleaned up unconditionally. If the probe manages to start, the system either marks the server as "update available", or proceeds with automatic reinstallation, depending on the "reinstall automatically" toggle value.
Automatic reinstall is not considered complete until a follow-up task checks that rollout managed to start the image with target digest. Depending on the outcome of the follow-up the state may become either
up_to_date,rollout_failed, or remain inreinstallingif reinstall still goes on and should be checked again after an exponential backoff timeout.State machine for the update state

Flowchart of the update process
