[runtime] Add Support for Thread-Per-Core

[patrick-ogrady]

While benchmarking, we should be sure to experiment with TPC (via Glommio).

[andresilva]

We need to be a bit more tactical than what glommio/monoio do. Those frameworks assume requests can be sharded at ingress and then stay on a single core for the entire pipeline. A typical example is a web request: it comes in, gets routed by some key (for example, username), hits the database, does some application logic, and renders a response. In that model, thread-per-core works well because ownership is decided up front and most of the work remains local to that shard.

That model does not generally work for us. For example, how would you shard the simplex voter? Even in cases like p2p, where the obvious sharding key would be the peer, there are still inherently global concerns (e.g. broadcast), which break the assumption that work can remain isolated within a single shard.

That said, we should probably support pinning dedicated tasks to a core, and then determine which tasks actually benefit from that model, which is not entirely obvious to me. We also do not want the split to be too eager: futures that interact frequently should be co-located, otherwise we risk paying heavily in cross-core communication, cache misses, and synchronization overhead. For example, you probably would not want the simplex voter and the batcher to live on different threads.

I suspect co-locating interacting futures is more important than pinning threads to specific cores. Locality between tasks matters more than core affinity in most cases.

The annoying part is doing this generically. Once you move away from a pure fixed model, the runtime has to become topology-aware to some extent, or at least aware of how much parallelism is actually available on the machine.