Skip to content

Support GPU & DRF #433

@k82cn

Description

@k82cn

Motivation

Currently, the faire share policy only support cpu and memory. When integrating flame with SGLang/vLLM/Pytorch, the GPU resources are requried. This feature is used to support GPU resources in Flame for inference and training workload.

Function Specification

  1. The executor manager will report GPU resources accordingly
  2. The session manager will support both faire-share by slots and DRF policy
  3. The SDK can request GPU resources

Solutions

  1. Support GPU in slots parsing, so the faire-share policy will support GPU
  2. Introduce a new DRF policy for GPU

Additional context

N/A

Metadata

Metadata

Assignees

Labels

kind/featureNew feature or requestpriority/p1High priorityrferequest for enhancement

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions