CTA-scope CQ acquire fence for prims: -4.6% small-message sendrecv latency

[goelayu]

Summary:
On Blackwell, the acquire fence DOCA runs after observing a completion lowers
to CCTL.IVALL -- a whole-L1 invalidate -- when its scope is SYS, and to a
NOP when it is CTA. prims fires the SYS form on every retired completion,
discarding the entire L1 each time.

prims hits that fence on two paths, not one: after observing a completion,
and again inside reserve_wq_slots -> wait_until_slot_available, which polls
the same CQ on every posting operation once the SQ has wrapped. The posting
path turns out to be the larger of the two (-3.1% of the -4.6%).

This adds an acquire_scope template parameter to
doca_gpu_dev_verbs_poll_one_cq_at, doca_gpu_dev_verbs_poll_cq_at,
doca_gpu_dev_verbs_wait, doca_gpu_dev_verbs_wait_until_slot_available and
doca_gpu_dev_verbs_reserve_wq_slots, defaulting to SYNC_SCOPE_SYS. prims
passes SYNC_SCOPE_CTA via kCqAcquireScope. Every other caller is
unchanged.

Note reserve_wq_slots also accepts GPU_CODE_OPT_SKIP_AVAILABILITY_CHECK,
which would skip the poll outright. Deliberately not used: that check exists to
stop us overrunning the SQ, and removing a safety check is a different risk
class from narrowing a fence scope. Scoping gets the same saving.

WHY NOT UPSTREAM'S MACRO -- upstream solves this with
DOCA_GPUNETIO_VERBS_EXP_NIC_FENCE_ACQUIRE_CTA, defined to 1 in GPUNetIO
v3.0.0+, closed SDK 3.4.0112 and NCCL 4.1.0. Adopting it here would be a
silent stale-data bug in stable ncclx:

• ncclx/v2_30/def_build.bzl:224,514 depend on :doca_gpunetio_dl, which
  propagates -DDOCA_VERBS_USE_META_THIRD_PARTY=1.
• That macro makes gin_gdaki.h:28 include this vendored tree rather than
  ncclx's own bundled DOCA copy.
• v2_30 GIN issues doca_gpu_dev_verbs_get (RDMA READ) at gin_gdaki.h:266
  and waits via poll_one_cq_at/doca_gpu_dev_verbs_wait (:305,308), NOT
  get_wait -- so upstream's compensating get_wait fence never fires for
  it, and the caller then reads the READ destination with ordinary loads.

A template parameter with a SYS default cannot be mis-scoped: any call site
that does not opt in keeps the strong fence, even inside a TU that also uses
prims. An earlier revision of this diff used the macro and was exactly the bug
described above; it was caught in review before submission.

Reviewed By: zhiyongww, snarayankh

Differential Revision: D118532592

[meta-codesync]

@goelayu has exported this pull request. If you are a Meta employee, you can view the originating Diff in D118532592.

[meta-codesync]

This pull request has been merged in 408e097.