Add example: deploy Faster-Whisper as a custom model on a SageMaker Triton MME - #4888
Add example: deploy Faster-Whisper as a custom model on a SageMaker Triton MME#4888eldadcohen1 wants to merge 2 commits into
Conversation
…riton MME Ports the multi-model-endpoint Triton custom-model (Faster-Whisper-Large-v3) example into the new repository structure under deploy_and_monitor/, updates the CI badge URLs and kernelspec to match sibling examples, and registers the notebook in the Deploy and Monitor toctree. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
|
@mollyheamazon can you please review and approve |
|
The notebook uses Another note: This notebook uses raw boto3 calls rather than the SageMaker Python SDK. It might be a better fit for awsdocs/aws-doc-sdk-examples (which hosts boto3 examples) |
Addresses review feedback on aws#4888: config.pbtxt uses backend: "python" but no cell wrote the required model.py. Adds a %%writefile cell that creates model.py in the version directory, loading Faster-Whisper and returning the transcript, so the example is reproducible end-to-end. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Thanks @mollyheamazon! 1. Missing 2. boto3 vs SageMaker SDK - the raw |
|
@jam-jee , @mollyheamazon fixed you comments, can you please take a look. |
|
@mollyheamazon any thoughts? |
Adds an example showing how to deploy a pre-trained Faster-Whisper-Large-v3 model from Hugging Face as a custom model on an Amazon SageMaker multi-model endpoint (MME) backed by Triton, including building a custom Triton container and running audio-transcription inference.
Description of changes:
deploy_and_monitor/sm-triton_mme_faster_whisper/(notebook,README.md, and a sample audio file), following the current repository structure andsm-triton_*naming convention.index.rst.conda_python3) match the sibling Triton examples.This supersedes #4608 / #4887, which targeted the legacy
mainbranch and the oldsagemaker-triton/layout. This PR targets the current default branch and the new folder structure.Testing is done:
Tested on AWS SageMaker.
Merge Checklist
black-nb -l 100 {path}/{notebook-name}.ipynbBy submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.