Don't spam config poller cache miss retries#2071
Draft
ogtownsend wants to merge 1 commit into
Draft
Conversation
f5b6d07 to
0ca190d
Compare
0ca190d to
4d05090
Compare
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
NOPs reported excessive error log spam from
processSourceChainConfigResults()when a source chain'sGetSourceChainConfigRPC call fails persistently (e.g., due to an RPC misconfiguration like gas limit below intrinsic gas).Root cause:
GetAllConfigsLegacyfetches source chain configs and receives some failure,processSourceChainConfigResultslogs the error and skips that chain and it's never added to the config poller cache.configPollerV2.GetOfframpSourceChainConfigs(), any chain not in the cache is treated as a "cache miss," which triggers an inline call tobatchRefreshChainAndSourceConfigs.asyncObserver.sync()(ObserveOffRampNextSeqNumsandObserveLatestOnRampSeqNums) both hit this path on every tick, multiplying the log volume.Fix:
pkg/reader/config_poller_v2.go:attemptedSourceChainsmap tochainCacheto track which source chains have been included in at least one fetch attempt.GetOfframpSourceChainConfigs: A chain missing from the cache is only treated as a "cache miss" (triggering an inline fetch) if it has never been attempted. Chains that were previously attempted but failed are not re-fetched inlinerefreshAllKnownChains) will still retry them on its regular schedule.batchRefreshChainAndSourceConfigs: After each fetch, all requested source chain selectors are recorded inattemptedSourceChainsand sourceChainRefresh is always updated, regardless of whether individual chains succeeded.pkg/chainaccessor/config_processors.go:Errorlevel creates excessive noise