Skip to content

Add optional Vulkan hardware acceleration for H.264 transcoding#19

Open
khamilowicz wants to merge 4 commits into
masterfrom
add-vk-support-claude
Open

Add optional Vulkan hardware acceleration for H.264 transcoding#19
khamilowicz wants to merge 4 commits into
masterfrom
add-vk-support-claude

Conversation

@khamilowicz
Copy link
Copy Markdown
Contributor

@khamilowicz khamilowicz commented May 6, 2026

  • Adds a native_acceleration: :never | :if_available option to Membrane.Transcoder that routes H.264 decode/encode through Membrane.VKVideo.Decoder / Membrane.VKVideo.Encoder when Vulkan hardware is present.
  • Adds Membrane.Transcoder.vulkan_available?/0 — a public helper for callers that need to check GPU availability before building a pipeline.
  • Handles NV12 pixel-format conversion around the Vulkan path (VK decoder outputs NV12; SWScale converters are inserted automatically when the surrounding formats differ).
  • Keeps membrane_vk_video_plugin as an optional dependency — software fallback is unchanged when the dep is absent or native_acceleration: :never.

Test plan

  • Run existing integration tests: mix test
  • Run new Vulkan integration tests on a machine with a Vulkan-capable GPU: mix test --include vulkan
  • Verify the updated examples/vp8_to_h264.exs produces valid output with and without native_acceleration: :if_available
  • Confirm vulkan_available?/0 returns false on a machine without membrane_vk_video_plugin installed

🤖 Generated with Claude Code

Comment thread mix.exs Outdated
System.get_env("MEMBRANE_VK_VIDEO") == "1" or
match?(
{_, 0},
System.cmd("pkg-config", ["--exists", "vulkan"], stderr_to_stdout: true)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's kind of non-trivial to figure out how to determine if vulkan video extensions are available :D
There is no requirement to use pkg-config but also plain existence of vulkan pkg-config entry does not indicate that Vulkan Video Extensions are available (it's gets even worse - Vulkan Video Extensions might be present, but there might not be H264 encoding and decoding capabilities).
I think we should uncodintionally add an optional dependency to membrane_vk_video_plugin and rely on fact that user will install it only after checking that particular capabilities are delivered.

Comment thread test/integration_test.exs
module when is_atom(module) ->
assert_sink_stream_format(pid, :sink, received_format)
assert received_format.__struct__ == module
end
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to assert on output file content as well

Comment thread lib/transcoder.ex Outdated
Comment on lines +185 to +186
Use this to guard code that sets `native_acceleration: :if_available`, for example
to print a diagnostic or pick a different pipeline branch.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I understand this part :D

Comment thread lib/transcoder.ex Outdated
state.output_stream_format,
state.transcoding_policy
state.transcoding_policy,
state.use_vulkan?
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would use some other name for that flag than use_vulkan? - sth like use_hardware_acceleration?

@khamilowicz khamilowicz force-pushed the add-vk-support-claude branch 3 times, most recently from b46e513 to 3b5b3c1 Compare May 8, 2026 13:01
@khamilowicz khamilowicz force-pushed the add-vk-support-claude branch from fc53441 to d7589cc Compare May 8, 2026 13:15
@khamilowicz khamilowicz requested a review from varsill May 11, 2026 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add elements from membrane_vk_video_plugin to Membrane.Transcoder

2 participants