Add support for latest-generation Google Cloud machine families#6841
Open
sosoihd wants to merge 1 commit intonextflow-io:masterfrom
Open
Add support for latest-generation Google Cloud machine families#6841sosoihd wants to merge 1 commit intonextflow-io:masterfrom
sosoihd wants to merge 1 commit intonextflow-io:masterfrom
Conversation
refactor(google-batch): introduce 'HYPERDISK_ONLY_FAMILIES' constant to identify machine types that only support Hyperdisk disk types. feat(google-batch): automatically configure 'hyperdisk-balanced' as boot disk type for Hyperdisk-only machine families. fix(google-batch): correctly calculate local SSD size for 'c4', 'c4a', and 'c4d' machine types. test(google-batch): add unit tests for 'isHyperdiskOnly' method in 'GoogleBatchMachineTypeSelector'. test(google-batch): add integration tests to verify Hyperdisk boot disk configuration in 'GoogleBatchTaskHandler'. Signed-off-by: Sofiane Ihaddadene <sosoihd@hotmail.fr>
✅ Deploy Preview for nextflow-docs-staging canceled.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add support for latest-generation Google Cloud machine families #6614
Problem
1. Missing support for latest-generation machine families
Google Cloud has introduced several new general-purpose machine families that are not currently supported by Nextflow:
These families offer significant improvements in:
Without this support, users cannot leverage:
2. Inability to specify boot disk type
Currently, Nextflow only allows configuring the boot disk size via
google.batch.bootDiskSize, but not the disk type. This creates several issues:Compatibility problems:
pd-balanceddisks (the Google Cloud default)hyperdisk-balancedorpd-ssdPerformance optimization:
pd-ssd(higher IOPS)pd-standard(lower cost)Reference: Google Cloud Disk Types Documentation
Solution
This PR addresses both issues with a comprehensive solution:
1. Add support for latest-generation machine families
Machine type recognition:
-lssdsuffixTesting:
Changes
GoogleBatchMachineTypeSelector.groovyHYPERDISK_ONLY_FAMILIESlist,isHyperdiskOnly()method, C4/C4A/C4D-lssdhandlingGoogleBatchTaskHandler.groovyhyperdisk-balancedboot disk for Hyperdisk-only familiesGoogleBatchMachineTypeSelectorTest.groovyisHyperdiskOnly(), C4/C4A/C4D local SSDGoogleBatchTaskHandlerTest.groovyTests
References