Skip to content

[Feature Request] Two separate resources on the same limiter? Or ability to release resources early. #189

@AndreFCruz

Description

@AndreFCruz

Hi,
I've been trying to use pyrate-limiter to limit calls to the OpenAI API according to two rate-limited resources: tokens per minute, and requests per minute.

As far as I can tell, after reading the documentation, I need two separate limiters. Something like:

self._rpm_limiter.try_acquire("api_calls", weight=1)
self._tpm_limiter.try_acquire("tokens", weight=token_count)

However, I think acquiring each resource sequentially can create a race condition with other threads: We may acquire the 1st resource and use it up, and then block when acquiring the 2nd resource.

It would be great if it was possible to either:

  • Acquire different resources at the same time;
  • Or release a resource early after a try_acquire: in this case, we would acquire the 1st resource, and if we can't acquire the 2nd resource we would then release the 1st resource and retry;

Is there a way to do this?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions