Add Net HTTP persistent adapter compatibility

[madwork]

Explanation

The idea is to allow connection reuse with net http persistent adapter. It should also improve memory footprint by lowering the number of Faraday::Connection objects and quoting net http persistent project description:

Using persistent HTTP connections can dramatically increase the speed of HTTP. Creating a new HTTP connection for every request involves an extra TCP round-trip and causes TCP congestion avoidance negotiation to start over.

Benchmark

The graph below display usage of the gem with the latest version (before) versus this branch (after) for the exact same job. It illustrates that it drastically reduces the number of open connections and port usage.

Usage example

gem "faraday", "~> 2.7", ">= 2.7.11"
gem "faraday-net_http_persistent", "~> 2.1"

openai_client ||= OpenAI::Client.new do |faraday|
  faraday.adapter :net_http_persistent, pool_size: 5 do |http|
    http.idle_timeout = 30
  end
end
thread = openai_client.threads.create
openai_client.messages.create(
  thread_id: thread["id"],
  parameters: {
    role: "user",
    content: "content"
  }
)
run = openai_client.runs.create(
  thread_id: thread["id"],
  parameters: {
    assistant_id: "assistant_id"
  }
)
loop do
  response = openai_client.runs.retrieve(
    id: run["id"],
    thread_id: thread["id"]
  )

  # Begin polling
  case response["status"]
  when "queued", "in_progress"
    sleep 1 # Wait 1s and poll again
  when "completed"
    # TODO
  else
    raise
  end
end

All Submissions:

• Have you followed the guidelines in our Contributing document?
• Have you checked to ensure there aren't other open Pull Requests for the same update/change?
• Have you added an explanation of what your changes do and why you'd like us to include them?

[alexrudall]

Thanks for this @madwork -- apologies for the delay in review. This is an important PR and I appreciate your work on this. The code is readable and carefully written which I really appreciate.

Questions

• Does this work universally for all endpoints?
• Does this work with streaming? I believe users use streaming much more than they poll.
• What affects the optimal pool size?
• Is there any reason this can't be the default? Happy to make a breaking change if it provides an significant improvement to all users.

Requests

• Could you add the given example to the README?
• I believe changing from conn to connection is a breaking change - happy to do this in a separate PR but if it's possible to make this PR not breaking that would be great.
• Could you share instructions for generating that graph? That will really help me & others in future testing.