Skip to content

retry: TpsBudget over-allocates retries for fractional retry_percent < 1.0 #857

@meng-xu-cs

Description

@meng-xu-cs

Summary

TpsBudget::new truncates 1.0 / retry_percent when retry_percent <= 1.0:

let (deposit_amount, withdraw_amount) = if retry_percent == 0.0 {
    (0, 1)
} else if retry_percent <= 1.0 {
    (1, (1.0 / retry_percent) as isize)
} else {
    (1000, (1000.0 / retry_percent) as isize)
};

For fractional values below 1.0 whose reciprocal is not an integer, this rounds the withdrawal cost down and permits more retries than configured.

For example, retry_percent = 0.6 yields (deposit_amount, withdraw_amount) = (1, 1), so every deposit authorizes one full retry (100%), not 0.6 retries (60%).

Reproduction

use std::time::Duration;
use tower::retry::budget::{Budget, TpsBudget};

#[test]
fn fractional_retry_percent_below_one_overissues_budget() {
    let budget = TpsBudget::new(Duration::from_secs(1), 0, 0.6);

    for _ in 0..10 {
        budget.deposit();
    }

    let allowed = (0..10).filter(|_| budget.withdraw()).count();

    assert!(
        allowed <= 6,
        "10 deposits at retry_percent=0.6 should not allow more than 6 retries, got {allowed}"
    );
}

This silently over-allocates retry budget for many fractional values below 1.0, which undermines the budget's role in limiting retry amplification during failures.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions