Detect accessory image changes during boot by gmautner · Pull Request #1808 · basecamp/kamal

gmautner · 2026-03-21T15:51:58Z

Summary

kamal accessory boot currently silently skips accessories when a container already exists, regardless of whether the configured image has changed. This makes it impossible for CI/CD pipelines to detect stale accessories — the only option is to unconditionally reboot all accessories on every deploy, causing unnecessary downtime.

This PR adds image change detection to accessory boot, following the same pattern established for proxy boot (which checks the running proxy version and raises when outdated).

When the image has changed:

Accessory `db` image has changed (postgres:16 → postgres:17),
run `kamal accessory reboot db` to update

When the image matches: the accessory is skipped as before (no-op).

This enables CI/CD pipelines to handle accessory updates the same way they handle proxy updates:

kamal accessory boot all || kamal accessory reboot all -y

Changes

lib/kamal/commands/accessory.rb — Added running_image method (mirrors proxy's version method)
lib/kamal/cli/accessory.rb — Modified boot to compare running image against configured image; raises when they differ
test/cli/accessory_test.rb — Added tests for image changed (raises) and same image (skips)

Test plan

bin/test test/cli/accessory_test.rb — 34 runs, 180 assertions, 0 failures
bin/test test/cli/proxy_test.rb — 36 runs, 376 assertions, 0 failures
bin/test test/commands/accessory_test.rb test/commands/proxy_test.rb — all pass
Existing boot, boot all, reboot tests unaffected

Currently `kamal accessory boot` silently skips an accessory when a container already exists, regardless of whether the configured image has changed. This makes it impossible for CI/CD pipelines to detect stale accessories — the only option is to unconditionally reboot all accessories on every deploy, causing unnecessary downtime. This follows the same pattern established for kamal-proxy in the `proxy boot` command, which checks the running proxy version and raises an error when it is outdated, telling the user to run `kamal proxy reboot`. With this change, `accessory boot` inspects the running container's image and compares it to the configured image. When they differ, it raises an error with an actionable message: Accessory `db` image has changed (postgres:16 → postgres:17), run `kamal accessory reboot db` to update This enables CI/CD pipelines to handle accessory updates the same way they handle proxy updates: kamal accessory boot all || kamal accessory reboot all -y When the image has not changed, the accessory is skipped as before. Signed-off-by: Gilberto Mautner <gilberto.mautner@locaweb.com.br>

Copilot

Pull request overview

This PR makes kamal accessory boot detect when an existing accessory container’s configured image has changed, and raises with a clear instruction to reboot instead of silently no-op’ing. This aligns accessory behavior with the existing proxy boot “outdated version” detection pattern to better support CI/CD flows.

Changes:

Added a command helper to retrieve an accessory container’s configured image.
Updated kamal accessory boot to compare the running container image vs configured image and raise when they differ.
Added CLI tests for “image changed” (raises) and “same image” (skips).

Tip

If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
`lib/kamal/commands/accessory.rb`	Adds `running_image` command to fetch `.Config.Image` via `docker inspect`.
`lib/kamal/cli/accessory.rb`	Compares existing container image to configured image during `boot` and raises on mismatch.
`test/cli/accessory_test.rb`	Adds coverage for changed-image error and same-image skip behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-21T15:54:54Z

lib/kamal/cli/accessory.rb

+            if capture_with_info(*accessory.info(all: true, quiet: true)).strip.presence
+              running_image = capture_with_info(*accessory.running_image).strip.presence


The container existence check uses docker ps -a -q --filter label=service=..., but the follow-up docker inspect targets service_name (container name). If a container matches the label but is not named exactly service_name (e.g., renamed/manual container), docker inspect will fail and abort boot. Consider capturing the container id(s) from accessory.info(...) and inspecting that id (or first id) instead, so the inspect target matches the existence filter.

Suggested change

if capture_with_info(*accessory.info(all: true, quiet: true)).strip.presence

running_image = capture_with_info(*accessory.running_image).strip.presence

container_ids = capture_with_info(*accessory.info(all: true, quiet: true)).strip

if container_ids.present?

container_id = container_ids.split(/\s+/).first

running_image = capture_with_info("docker", "inspect", "--format", "{{.Config.Image}}", container_id).strip.presence

Copilot · 2026-03-21T15:54:54Z

lib/kamal/cli/accessory.rb

+            if capture_with_info(*accessory.info(all: true, quiet: true)).strip.presence
+              running_image = capture_with_info(*accessory.running_image).strip.presence
+              if running_image && running_image != accessory.image
+                raise "Accessory `#{name}` image has changed (#{running_image} → #{accessory.image}), run `kamal accessory reboot #{name}` to update"


The raised "image has changed" error doesn't include which host triggered the mismatch. Since accessories can run on multiple hosts, this can be hard to act on when only some hosts are stale. Consider including host in the message (or collecting all mismatching hosts/images and raising once after the on(hosts) block).

Suggested change

raise "Accessory `#{name}` image has changed (#{running_image} → #{accessory.image}), run `kamal accessory reboot #{name}` to update"

raise "Accessory `#{name}` image has changed on host #{host} (#{running_image} → #{accessory.image}), run `kamal accessory reboot #{name}` to update"

Copilot · 2026-03-21T15:54:54Z

lib/kamal/cli/accessory.rb


          if booted_hosts.any?
-            say "Skipping booting `#{name}` on #{booted_hosts.sort.join(", ")}, a container already exists", :yellow
+            say "Skipping booting `#{name}` on #{booted_hosts.sort.join(", ")}, already running with the correct image", :yellow


accessory.info(all: true, ...) uses docker ps -a, so it will match stopped containers as well as running ones. The skip message says "already running with the correct image", which can be inaccurate when a container exists but is not running; consider wording that reflects "container already exists" (or switch the existence check to running-only if that’s the intent).

Suggested change

say "Skipping booting `#{name}` on #{booted_hosts.sort.join(", ")}, already running with the correct image", :yellow

say "Skipping booting `#{name}` on #{booted_hosts.sort.join(", ")}, container already exists with the correct image", :yellow

Copilot · 2026-03-21T15:54:55Z

test/cli/accessory_test.rb

+    Thread.report_on_exception = false
+
+    SSHKit::Backend::Abstract.any_instance.stubs(:capture_with_info)
+      .with(:docker, :ps, "-a", "-q", "--filter", "label=service=app-mysql")
+      .returns("abc123")
+
+    SSHKit::Backend::Abstract.any_instance.stubs(:capture_with_info)
+      .with(:docker, :inspect, "app-mysql", "--format '{{.Config.Image}}'")
+      .returns("private.registry/mysql:5.6")
+
+    exception = assert_raises do
+      run_command("boot", "mysql")
+    end
+
+    assert_includes exception.message, "Accessory `mysql` image has changed (private.registry/mysql:5.6 → private.registry/mysql:5.7)"
+    assert_includes exception.message, "run `kamal accessory reboot mysql` to update"
+  ensure
+    Thread.report_on_exception = true
+  end


This test mutates the global Thread.report_on_exception and then forces it back to true in ensure. To avoid leaking global state across the suite (especially if another test sets it differently), save the original value and restore that value in the ensure block instead of always setting true.

- Include host in the image mismatch error message, so multi-host accessories are easier to debug - Change skip message from "already running" to "container already exists" since `docker ps -a` also matches stopped containers Signed-off-by: Gilberto Mautner <gilberto.mautner@locaweb.com.br>

Copilot AI review requested due to automatic review settings March 21, 2026 15:52

Copilot started reviewing on behalf of gmautner March 21, 2026 15:52 View session

Copilot AI reviewed Mar 21, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Detect accessory image changes during boot#1808

Detect accessory image changes during boot#1808
gmautner wants to merge 2 commits intobasecamp:mainfrom
gmautner:accessory-boot-image-change-detection

gmautner commented Mar 21, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 21, 2026

Uh oh!

Copilot AI Mar 21, 2026

Uh oh!

Copilot AI Mar 21, 2026

Uh oh!

Copilot AI Mar 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		if capture_with_info(*accessory.info(all: true, quiet: true)).strip.presence
		running_image = capture_with_info(*accessory.running_image).strip.presence

-            if capture_with_info(*accessory.info(all: true, quiet: true)).strip.presence
-              running_image = capture_with_info(*accessory.running_image).strip.presence
+            container_ids = capture_with_info(*accessory.info(all: true, quiet: true)).strip
+            if container_ids.present?
+              container_id = container_ids.split(/\s+/).first
+              running_image = capture_with_info("docker", "inspect", "--format", "{{.Config.Image}}", container_id).strip.presence

	raise "Accessory `#{name}` image has changed (#{running_image} → #{accessory.image}), run `kamal accessory reboot #{name}` to update"
	raise "Accessory `#{name}` image has changed on host #{host} (#{running_image} → #{accessory.image}), run `kamal accessory reboot #{name}` to update"

	say "Skipping booting `#{name}` on #{booted_hosts.sort.join(", ")}, already running with the correct image", :yellow
	say "Skipping booting `#{name}` on #{booted_hosts.sort.join(", ")}, container already exists with the correct image", :yellow

Conversation

gmautner commented Mar 21, 2026

Summary

Changes

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants