Skip to content

fix(controller): ensure progress when canary > desired replicas#4619

Open
andrewjamesbrown wants to merge 2 commits intoargoproj:masterfrom
andrewjamesbrown:ajb/min_available_canary_stall
Open

fix(controller): ensure progress when canary > desired replicas#4619
andrewjamesbrown wants to merge 2 commits intoargoproj:masterfrom
andrewjamesbrown:ajb/min_available_canary_stall

Conversation

@andrewjamesbrown
Copy link
Contributor

@andrewjamesbrown andrewjamesbrown commented Feb 13, 2026

We've experienced a stall in our canary process when an HPA is configured with 1 replica, but we set minPodsPerReplicaSet: 2 in the rollout. Turns out that we check whether the canary replicaset is not equal to the desired, but don't handle the case where it is greater than the desired; which should also be treated as success.

We have these logs that repeat ad infinutum:

Started syncing rollout
Found 1 TrafficRouting Reconcilers
Reconciling TrafficRouting with type 'Istio'
DestinationRule my-service subset updated (canary: 6d64947786, stable: 6f6f4bdc4c)
No Steps remain in the canary steps
No status changes. Skipping patch
Queueing up Rollout for a progress check now
Reconciliation completed

Our rollout:

% k get rollout
NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
my-service               1         3         2            3           465d
% k get rs
NAME                                DESIRED   CURRENT   READY   AGE
my-service-6d64947786               2         2         2       5h36m
my-service-6f6f4bdc4c               1         1         0       3d3h

Checklist:

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this is a chore.
  • The title of the PR is (a) conventional with a list of types and scopes found here, (b) states what changed, and (c) suffixes the related issues number. E.g. "fix(controller): Updates such and such. Fixes #1234".
  • I've signed my commits with DCO
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My builds are green. Try syncing with master if they are not.
  • My organization is added to USERS.md.

@andrewjamesbrown andrewjamesbrown force-pushed the ajb/min_available_canary_stall branch from d096a8d to e3a59e2 Compare February 13, 2026 19:52
@andrewjamesbrown andrewjamesbrown changed the title fix(sync): ensure progress when canary > desired replicas fix(controller): ensure progress when canary > desired replicas Feb 13, 2026
Signed-off-by: Andrew Brown <andrew.brown@wealthsimple.com>
@andrewjamesbrown andrewjamesbrown force-pushed the ajb/min_available_canary_stall branch from e3a59e2 to 5066615 Compare February 13, 2026 20:11
Signed-off-by: Andrew Brown <andrew.brown@wealthsimple.com>
@sonarqubecloud
Copy link

@github-actions
Copy link
Contributor

Published E2E Test Results

  4 files    4 suites   3h 30m 21s ⏱️
120 tests 110 ✅  7 💤 3 ❌
484 runs  453 ✅ 28 💤 3 ❌

For more details on these failures, see this check.

Results for commit bb3a682.

@github-actions
Copy link
Contributor

Published Unit Test Results

2 395 tests   2 395 ✅  3m 4s ⏱️
  129 suites      0 💤
    1 files        0 ❌

Results for commit bb3a682.

@codecov
Copy link

codecov bot commented Feb 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.41%. Comparing base (348260a) to head (bb3a682).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4619      +/-   ##
==========================================
+ Coverage   84.40%   84.41%   +0.01%     
==========================================
  Files         164      164              
  Lines       18855    18855              
==========================================
+ Hits        15914    15916       +2     
+ Misses       2077     2076       -1     
+ Partials      864      863       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant