Streaming inactivity timeout incorrectly aborts after total timeout ( @langchain/openai": "^1.0.0-alpha.1" ) #9088
Description
Checked other resources
- This is a feature request, not a bug report or usage question.
- I added a clear and descriptive title that summarizes the feature request.
- I used the GitHub search to find a similar feature request and didn't find it.
- I checked the LangChain documentation and API reference to see if this feature already exists.
- This is not related to the langchain-community package.
Feature Description
When using the streaming API, the stream is aborted after X seconds total, regardless of whether chunks are still arriving. I need an inactivity timeout: the stream should abort only if no chunk arrives for X seconds, not after X seconds total request time.
in this example
await llm.invoke( messages, { timeout: 3000 });the output is something like this:
node usage.js
Quantum computing is a new type of computing that uses the principles of quantum mechanics to process information in a fundamentally different way from classical computers. Here’s a simple breakdown:
1. **Qubits vs. Bits**:
- Classical computers use **bits**, which can be either 0 or 1.
- Quantum computers use **qubits** (quantum bits), which can be 0, 1, or both at the same time. This is called **superposition**.
2. **Superposition**:
Imagine a coin spinning in the air—it’s neither fully heads nor tails until it lands. Qubits work similarly, existing in multiple states simultaneously, allowing quantum computers to process many possibilities at once.
3. **Entanglement**:
Qubits can be **entangled**, meaning the state of one qubit instantly affects the state of another, no matter how far apart they are. This creates powerful connections that speed up calculations.
4. **Parallel Processing**:
Because of superposition, quantum computers can explore many solutions toAll providers failed: The operation was aborted due to timeoutthe connection gets closed after 3 second. the message in above log:
The operation was aborted due to timeout
Use Case
it helps for load balancing between providers and ....
Proposed Solution
add an option:
await llm.invoke( messages, { stream_timeout: 3000 });Alternatives Considered
No response
Additional Context
No response
Metadata
Metadata
Assignees
Labels
No labels