You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: fern/fern/pages/api/nixl_connect/README.mdx
+14-14Lines changed: 14 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,7 +28,7 @@ For additional information, please read this [GPUDirect RDMA](https://docs.nvidi
28
28
import dynamo.nixl_connect
29
29
```
30
30
31
-
All operations using the NIXL Connect library begin with the [`Connector`](connector) class and the type of operation required.
31
+
All operations using the NIXL Connect library begin with the [`Connector`](/additional-resources/api-reference/nixl-connect/connector) class and the type of operation required.
32
32
There are four types of supported operations:
33
33
34
34
1.**Register local readable memory**:
@@ -71,9 +71,9 @@ sequenceDiagram
71
71
72
72
### Generic Example
73
73
74
-
In the diagram below, Local creates a [`WritableOperation`](writable_operation) intended to receive data from Remote.
74
+
In the diagram below, Local creates a [`WritableOperation`](/additional-resources/api-reference/nixl-connect/writable-operation) intended to receive data from Remote.
75
75
Local then sends metadata about the requested operation to Remote.
76
-
Remote then uses the metadata to create a [`WriteOperation`](write_operation) which will perform the GPU Direct RDMA memory transfer, when available, from Remote's GPU memory to Local's GPU memory.
76
+
Remote then uses the metadata to create a [`WriteOperation`](/additional-resources/api-reference/nixl-connect/write-operation) which will perform the GPU Direct RDMA memory transfer, when available, from Remote's GPU memory to Local's GPU memory.
77
77
78
78
```mermaid
79
79
---
@@ -94,7 +94,7 @@ When RDMA isn't available, the NIXL data transfer will still complete using non-
94
94
95
95
### Multimodal Example
96
96
97
-
In the case of the [Dynamo Multimodal Disaggregated Example](/additional-resources/multimodal-details/vllm):
97
+
In the case of the [Dynamo Multimodal Disaggregated Example](/additional-resources/multimodal-details/v-llm):
98
98
99
99
1. The HTTP frontend accepts a text prompt and a URL to an image.
100
100
@@ -146,24 +146,24 @@ The KV Cache transfer between Decode Worker and Prefill Worker utilizes a differ
146
146
#### Code Examples
147
147
148
148
See [MultimodalPDWorkerHandler](https://github.com/ai-dynamo/dynamo/tree/main/components/src/dynamo/vllm/multimodal_handlers/worker_handler.py) or [MultimodalDecodeWorkerHandler](https://github.com/ai-dynamo/dynamo/tree/main/components/src/dynamo/vllm/multimodal_handlers/worker_handler.py) from our Multimodal example,
149
-
for how they coordinate directly with the Encode Worker by creating a [`WritableOperation`](writable_operation),
149
+
for how they coordinate directly with the Encode Worker by creating a [`WritableOperation`](/additional-resources/api-reference/nixl-connect/writable-operation),
150
150
sending the operation's metadata via Dynamo's round-robin dispatcher, and awaiting the operation for completion before making use of the transferred data.
151
151
152
152
See [MultimodalEncodeWorkerHandler](https://github.com/ai-dynamo/dynamo/tree/main/components/src/dynamo/vllm/multimodal_handlers/encode_worker_handler.py) from our Multimodal example,
153
-
for how the resulting embeddings are registered with the NIXL subsystem by creating a [`Descriptor`](descriptor),
154
-
a [`WriteOperation`](write_operation) is created using the metadata provided by the requesting worker,
153
+
for how the resulting embeddings are registered with the NIXL subsystem by creating a [`Descriptor`](/additional-resources/api-reference/nixl-connect/descriptor),
154
+
a [`WriteOperation`](/additional-resources/api-reference/nixl-connect/write-operation) is created using the metadata provided by the requesting worker,
155
155
and the worker awaits for the data transfer to complete for yielding a response.
Copy file name to clipboardExpand all lines: fern/fern/pages/api/nixl_connect/connector.mdx
+23-23Lines changed: 23 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,11 +17,11 @@ The connector provides two methods of moving data between workers:
17
17
18
18
- Preparing local memory to be read by a remote worker.
19
19
20
-
In both cases, local memory is registered with the NIXL-based I/O subsystem via the [`Descriptor`](descriptor) class and provided to the connector.
20
+
In both cases, local memory is registered with the NIXL-based I/O subsystem via the [`Descriptor`](/additional-resources/api-reference/nixl-connect/descriptor) class and provided to the connector.
21
21
When RDMA is available, the connector then configures the RDMA subsystem to expose the memory for the requested operation and returns an operation control object;
22
22
otherwise the connector will select the best available RDMA alternative.
23
-
The operation control object, either a [`ReadableOperation`](readable_operation) or a [`WritableOperation`](writable_operation),
24
-
provides NIXL metadata ([RdmaMetadata](rdma_metadata)) via its `.metadata()` method, functionality to query the operation's current state, as well as the ability to cancel the operation prior to its completion.
23
+
The operation control object, either a [`ReadableOperation`](/additional-resources/api-reference/nixl-connect/readable-operation) or a [`WritableOperation`](/additional-resources/api-reference/nixl-connect/writable-operation),
24
+
provides NIXL metadata ([RdmaMetadata](/additional-resources/api-reference/nixl-connect/rdma-metadata)) via its `.metadata()` method, functionality to query the operation's current state, as well as the ability to cancel the operation prior to its completion.
25
25
26
26
The NIXL metadata must be provided to the remote worker expected to complete the operation.
27
27
The metadata contains required information (identifiers, keys, etc.) which enables the remote worker to interact with the provided memory.
@@ -41,8 +41,8 @@ This data provides direct memory access between workers, and should be considere
41
41
```
42
42
43
43
<Calloutintent="success">
44
-
See [`ReadOperation`](read_operation#example-usage), [`ReadableOperation`](readable_operation#example-usage),
45
-
[`WritableOperation`](writable_operation#example-usage), and [`WriteOperation`](write_operation#example-usage)
44
+
See [`ReadOperation`](/additional-resources/api-reference/nixl-connect/read-operation#example-usage), [`ReadableOperation`](/additional-resources/api-reference/nixl-connect/readable-operation#example-usage),
45
+
[`WritableOperation`](/additional-resources/api-reference/nixl-connect/writable-operation#example-usage), and [`WriteOperation`](/additional-resources/api-reference/nixl-connect/write-operation#example-usage)
46
46
for additional examples.
47
47
</Callout>
48
48
@@ -59,9 +59,9 @@ async def begin_read(
59
59
) -> ReadOperation:
60
60
```
61
61
62
-
Creates a [`ReadOperation`](read_operation) for transferring data from a remote worker.
62
+
Creates a [`ReadOperation`](/additional-resources/api-reference/nixl-connect/read-operation) for transferring data from a remote worker.
63
63
64
-
To create the operation, the serialized request from a remote worker's [`ReadableOperation`](readable_operation)
64
+
To create the operation, the serialized request from a remote worker's [`ReadableOperation`](/additional-resources/api-reference/nixl-connect/readable-operation)
65
65
along with a matching set of local memory descriptors which reference memory intended to receive data from the remote worker
66
66
must be provided.
67
67
The serialized request must be transferred from the remote to the local worker via a secondary channel, most likely HTTP or TCP+NATS.
@@ -71,7 +71,7 @@ Once created, data transfer will begin immediately.
71
71
Disposal of the object will instruct the NIXL subsystem to cancel the operation,
72
72
therefore the operation should be awaited until completed unless cancellation is intended.
73
73
74
-
Use [`.wait_for_completion()`](read_operation#wait_for_completion) to block the caller until the operation has completed or encountered an error.
74
+
Use [`.wait_for_completion()`](/additional-resources/api-reference/nixl-connect/read-operation#wait_for_completion) to block the caller until the operation has completed or encountered an error.
75
75
76
76
### `begin_write`
77
77
@@ -83,9 +83,9 @@ async def begin_write(
83
83
) -> WriteOperation:
84
84
```
85
85
86
-
Creates a [`WriteOperation`](write_operation) for transferring data to a remote worker.
86
+
Creates a [`WriteOperation`](/additional-resources/api-reference/nixl-connect/write-operation) for transferring data to a remote worker.
87
87
88
-
To create the operation, the serialized request from a remote worker's [`WritableOperation`](writable_operation)
88
+
To create the operation, the serialized request from a remote worker's [`WritableOperation`](/additional-resources/api-reference/nixl-connect/writable-operation)
89
89
along with a matching set of local memory descriptors which reference memory to be transferred to the remote worker
90
90
must be provided.
91
91
The serialized request must be transferred from the remote to the local worker via a secondary channel, most likely HTTP or TCP+NATS.
@@ -95,7 +95,7 @@ Once created, data transfer will begin immediately.
95
95
Disposal of the object will instruct the NIXL subsystem to cancel the operation,
96
96
therefore the operation should be awaited until completed unless cancellation is intended.
97
97
98
-
Use [`.wait_for_completion()`](write_operation#wait_for_completion) to block the caller until the operation has completed or encountered an error.
98
+
Use [`.wait_for_completion()`](/additional-resources/api-reference/nixl-connect/write-operation#wait_for_completion) to block the caller until the operation has completed or encountered an error.
99
99
100
100
### `create_readable`
101
101
@@ -106,7 +106,7 @@ async def create_readable(
106
106
) -> ReadableOperation:
107
107
```
108
108
109
-
Creates a [`ReadableOperation`](readable_operation) for transferring data to a remote worker.
109
+
Creates a [`ReadableOperation`](/additional-resources/api-reference/nixl-connect/readable-operation) for transferring data to a remote worker.
110
110
111
111
To create the operation, a set of local memory descriptors must be provided that reference memory intended to be transferred to a remote worker.
112
112
Once created, the memory referenced by the provided descriptors becomes immediately readable by a remote worker with the necessary metadata.
@@ -116,7 +116,7 @@ Once acquired, the metadata needs to be provided to a remote worker via a second
116
116
Disposal of the object will instruct the NIXL subsystem to cancel the operation,
117
117
therefore the operation should be awaited until completed unless cancellation is intended.
118
118
119
-
Use [`.wait_for_completion()`](readable_operation#wait_for_completion) to block the caller until the operation has completed or encountered an error.
119
+
Use [`.wait_for_completion()`](/additional-resources/api-reference/nixl-connect/readable-operation#wait_for_completion) to block the caller until the operation has completed or encountered an error.
120
120
121
121
### `create_writable`
122
122
@@ -127,7 +127,7 @@ async def create_writable(
127
127
) -> WritableOperation:
128
128
```
129
129
130
-
Creates a [`WritableOperation`](writable_operation) for transferring data from a remote worker.
130
+
Creates a [`WritableOperation`](/additional-resources/api-reference/nixl-connect/writable-operation) for transferring data from a remote worker.
131
131
132
132
To create the operation, a set of local memory descriptors must be provided which reference memory intended to receive data from a remote worker.
133
133
Once created, the memory referenced by the provided descriptors becomes immediately writable by a remote worker with the necessary metadata.
@@ -137,7 +137,7 @@ Once acquired, the metadata needs to be provided to a remote worker via a second
137
137
Disposal of the object will instruct the NIXL subsystem to cancel the operation,
138
138
therefore the operation should be awaited until completed unless cancellation is intended.
139
139
140
-
Use [`.wait_for_completion()`](writable_operation#wait_for_completion) to block the caller until the operation has completed or encountered an error.
140
+
Use [`.wait_for_completion()`](/additional-resources/api-reference/nixl-connect/writable-operation#wait_for_completion) to block the caller until the operation has completed or encountered an error.
141
141
142
142
143
143
## Properties
@@ -172,11 +172,11 @@ Gets the Dynamo component name used by the connector.
0 commit comments