Skip to content

Commit de94b35

Browse files
Peng Chenmeta-codesync[bot]
authored andcommitted
comms/uniflow/tcp: refuse a bind() on a transport that is already bound
Summary: bind() guarded only shutdown_, then set state_ = Initialized unconditionally. Initialized is the one state connect() admits, so a second bind() re-armed that gate on a live transport and nothing rejected the second connect() that follows. Two things go wrong on that path. bind() itself calls servers_.clear(), dropping the listener out from under an established connection. Then the second connect() reaches establishLanes(), whose first act is lanes_.clear() -- and a TcpLane owns its reader and sender std::threads, so that destroys two joinable threads. ~std::thread on a joinable thread calls std::terminate unconditionally. The lane's Conn is destroyed in the same sweep while both threads are still using it, so there is a use-after-free behind the terminate as well. This needs caller misuse: the Transport contract is bind-then-connect once, and nothing in tree re-binds -- MultiTransport::bind() fails the whole bind when a transport returns empty info, and Uniflow::establishConnection() binds each freshly created transport once. But the failure mode is a process-wide abort with no diagnostic, and on AMD this transport shares a MultiTransport with RDMA, so it would take down transports that did nothing wrong. The refusal costs three lines next to the shutdown_ check that is already there. Error is treated as terminal here too. A bind() that failed leaves state_ = Error, and this guard therefore refuses a retry. Nothing retries today, and a transport whose listener never came up should not be revived by a second attempt against the same host_. Differential Revision: D117884613
1 parent 981a7ce commit de94b35

2 files changed

Lines changed: 49 additions & 0 deletions

File tree

comms/uniflow/transport/tcp/TcpTransport.cpp

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -328,6 +328,18 @@ TransportInfo TcpTransport::bind() {
328328
UNIFLOW_LOG_ERROR("TcpTransport::bind: transport is already shut down");
329329
return TransportInfo{};
330330
}
331+
// Refuse a re-bind rather than re-arming Initialized, which is the only state
332+
// connect() admits. Re-arming lets a second connect() reach
333+
// establishLanes(), whose first act is to clear lanes_ -- destroying the live
334+
// lanes' joinable reader/sender threads, and ~std::thread on a joinable
335+
// thread calls std::terminate. The servers_.clear() below would already have
336+
// dropped the listener out from under the current connection. Error is
337+
// terminal here too: nothing retries a failed bind, and a transport that
338+
// never came up should not be revived.
339+
if (state_ != TransportState::Disconnected) {
340+
UNIFLOW_LOG_ERROR("TcpTransport::bind: transport is already bound");
341+
return TransportInfo{};
342+
}
331343
// One listener per device when striping, otherwise a single listener on host_
332344
// with egress left to the routing table. Each device's listener binds that
333345
// device's own address *and* sets SO_BINDTODEVICE, because accepted sockets

comms/uniflow/transport/tcp/tests/unit/TcpTransportConnectTest.cpp

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
#include <cstdint>
99
#include <cstring>
1010
#include <memory>
11+
#include <thread>
1112
#include <vector>
1213

1314
#include "comms/uniflow/executor/ScopedEventBaseThread.h"
@@ -192,6 +193,42 @@ TEST_F(TcpTransportConnectTest, BindAfterShutdownIsRefused) {
192193
<< "bind() must not re-open a shut-down transport";
193194
}
194195

196+
// bind() re-arms the Initialized state that connect() gates on, so without an
197+
// already-bound guard a second bind() lets a second connect() through on a live
198+
// transport. establishLanes() clears lanes_ as its first act, which destroys
199+
// the current lanes' joinable reader/sender threads -- and ~std::thread on a
200+
// joinable thread calls std::terminate, taking down every other transport in
201+
// the process with it.
202+
TEST_F(TcpTransportConnectTest, BindAfterConnectIsRefused) {
203+
auto peerEvb =
204+
std::make_unique<ScopedEventBaseThread>("tcp-connect-test-peer");
205+
auto peer = std::make_unique<TcpTransport>(
206+
/*deviceId=*/-1,
207+
peerEvb->getEventBase(),
208+
registry_,
209+
controller::TcpSocketConfig{},
210+
/*host=*/"127.0.0.1");
211+
212+
const TransportInfo selfInfo = transport_->bind();
213+
const TransportInfo peerInfo = peer->bind();
214+
ASSERT_FALSE(selfInfo.empty());
215+
ASSERT_FALSE(peerInfo.empty());
216+
// Each side dials or accepts by endpoint order, so both connects have to be
217+
// in flight at once for the handshake to complete.
218+
std::thread dialer([&]() { (void)peer->connect(selfInfo); });
219+
const Status status = transport_->connect(peerInfo);
220+
dialer.join();
221+
ASSERT_FALSE(status.hasError()) << status.error().message();
222+
ASSERT_EQ(transport_->state(), TransportState::Connected);
223+
224+
EXPECT_TRUE(transport_->bind().empty())
225+
<< "bind() must not re-arm a transport that is already connected";
226+
EXPECT_EQ(transport_->state(), TransportState::Connected)
227+
<< "a refused bind() must leave the live connection intact";
228+
229+
peer->shutdown();
230+
}
231+
195232
// The factory's topology blob used to be a default-constructed *addressing*
196233
// struct, so it advertised 127.0.0.1:0 rather than a real endpoint and carried
197234
// no version. canConnect() therefore validated nothing, and a wire-format

0 commit comments

Comments
 (0)