Include drivers + runtime to run AI-accelerated compute workloads on GPU hardware from AMD, Intel, and Nvidia by default in Home Assistant Operating System?

[Hedda]

Will you please consider including drivers + runtime to run AI-accelerated compute workloads on GPU hardware from AMD (ROCm), Intel (OpenCL/Level Zero/oneAPI), and Nvidia (CUDA) by default in future Home Assistant Operating System distro version so can provide compute API support for GPUs?

I think most relevant is perhaps these new experimental Speech-to-Text apps/addons from the Open Home Foundation voice lead, such as example NeMo's Canary which is a very large model that requires lots of RAM or a GPU:

• https://github.com/OHF-Voice/apps-experimental/

There are of course also those of us who would like be be able to run smaller local LLMs inside Home Assistant Operating System too for conversation agent via a local voice assistant.

• https://www.home-assistant.io/voice_control/assist_create_open_ai_personality/

  • https://github.com/allenporter/home-assistant-datasets/tree/main/reports

Then there are other use case as well, such as example OpenCV and Tensorflow for real-time AI object detection for use with videos/images from ex. Frigate NVR, which are also things that can be hardware accelerated using modern GPU hardware.

• https://frigate.video/

The point is that using only the system CPU is often not enough for use cases like these and it would therefor be very nice if those who can afford it as are interested in this could just buy a computer with a power GPU and simply install Home Assistant Operating System directly on bare metal and have apps/addons access compute API support for GPU without the user having to hack in drivers and such.