[RFC] AMD support for DGL

[jamesETsmith]

🚀 Feature Proposal RFC

Adding support for AMD’s CDNA GPUs to DGL.

Motivation/Goals

• Enable DGL for AMD hardware
• Make DGL accessible to more users
• Enable users to easily switch between NVIDIA or AMD depending on their needs
• There has been interest in AMD support in DGL for years
  • Previous implementations of HIP/ROCm support: [RFC] Adding AMD GPU support via HIP/ROCM #7838
  • User requests for AMD support:
    • [Roadmap] 0.7 Release Plan #2888
    • Can DGL support AMD GPU with ROCM ? #2659

Pitch

Our primary hipification strategy was the following:

• Adding a header file (what we called a header shim) that used preprocessor statements to replace cuda calls with their corresponding rocm ones
• Where that wasn’t possible, I tried to use namespace aliases and the like to the header include sections of source files
• As a last resort, we used macros in the source code, but this was only necessary in a handful of cases

Our aim was to limit source code changes where possible and try to contain most changes to the #include section of source files.

Alternatives

Several contributors to this work explored using source code generation and pre-build hipification (using scripts to hipify the source code). See this DGL fork and this RFC posted to DGL several years ago.

We felt that it was important that individual users/developers could clone the source code and not have to hipfy them, but otherwise our changes are similar to those in the DGL fork mentioned above. @GMNGeoffrey who was the primary maintainer of that fork and author of the previous RFC was huge help during this work.

Additional Context

• PR for hipification of Graphbolt and HugeCTR: [FEATURE] Hipification of Graphbolt via Header Shims ROCm/dgl#3
• https://rocm.blogs.amd.com/artificial-intelligence/dgl-in-depth/README.html

@Rhett-Ying and @BarclayII, I'm happy to happy to elaborate on any points here or even set up an offline meeting if that would be preferred. Please let us know what you think. If there are other developers, I should tag or loop into this discussion, please let me know.

All the best,
James