Skip to content

runner: isolate template errors instead of aborting all templates [fj4WqyCCw3C5ShR1RfB7MoBPTpkRrBFYP1uT35g3MvT] - #2163

Open
waterWang wants to merge 1 commit into
hashicorp:mainfrom
waterWang:fix/template-error-isolation-v2
Open

runner: isolate template errors instead of aborting all templates [fj4WqyCCw3C5ShR1RfB7MoBPTpkRrBFYP1uT35g3MvT]#2163
waterWang wants to merge 1 commit into
hashicorp:mainfrom
waterWang:fix/template-error-isolation-v2

Conversation

@waterWang

Copy link
Copy Markdown

Description

When one template in a multi-template Vault Agent configuration encounters a fatal error (e.g., an exec {} template function fails), the consul-template runner's Run() method returns immediately after the first failure, skipping all remaining templates.

This means:

  1. Template A with a failing exec {} block causes templates B, C, D to be skipped
  2. The runner stops and sends the error to ErrCh
  3. On restart, ALL templates are re-processed from scratch
  4. The failing template fails again, creating an infinite loop

Fix

Collect template errors and continue processing the remaining templates:

  • When a template has a fatal error, log it, add to the error list, and continue to the next template
  • If at least one template rendered successfully, return nil (don't stop the runner — the failed template will be retried on the next dependency change)
  • If ALL templates failed, return an error so the caller can handle it

Changes

manager/runner.go — 20 lines added, 1 deleted

Related issue

Fixes hashicorp/vault#32035

When one template in a multi-template configuration encounters a fatal
error (ErrFatal=true), the Run() method currently returns immediately,
skipping all remaining templates. This causes the entire runner to stop
after the first failed template, and all previously successful templates
are re-processed on restart.

Fix: collect template errors and continue processing the remaining
templates. At the end of Run(), if at least one template rendered
successfully, log a warning and return nil (don't stop the runner).
If all templates failed, return an error so the caller can decide
whether to stop or continue.

Fixes hashicorp/vault#32035
@waterWang
waterWang requested review from a team as code owners July 29, 2026 05:48
@hashicorp-cla-app

Copy link
Copy Markdown

CLA assistant check

Thank you for your submission! We require that all contributors sign our Contributor License Agreement ("CLA") before we can accept the contribution. Read and sign the agreement

Learn more about why HashiCorp requires a CLA and what the CLA includes


waterWang seems not to be a GitHub user.
You need a GitHub account to be able to sign the CLA.
If you have already a GitHub account, please add the email address used for this commit to your account.

Have you signed the CLA already but the status is still pending? Recheck it.

1 similar comment
@hashicorp-cla-app

Copy link
Copy Markdown

CLA assistant check

Thank you for your submission! We require that all contributors sign our Contributor License Agreement ("CLA") before we can accept the contribution. Read and sign the agreement

Learn more about why HashiCorp requires a CLA and what the CLA includes


waterWang seems not to be a GitHub user.
You need a GitHub account to be able to sign the CLA.
If you have already a GitHub account, please add the email address used for this commit to your account.

Have you signed the CLA already but the status is still pending? Recheck it.

@tgross tgross left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No one can review this till the CLA is signed . I'd ask reviewers to consider how this impacts other downstream projects like Nomad template blocks. Especially given the bug reported in Vault this is trying to fix was not acknowledged by the Vault team so maybe they intend for this behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Vault Agent aborts evaluation of all temlates and reschedules all if one fails

2 participants