It seems like the default nodeSelector and tolerations added to all step pods (via METAFLOW_KUBERNETES_TOLERATIONS and METAFLOW_KUBERNETES_NODE_SELECTOR), aren't added to the argo workflows error-msg-capture-hook pod, as well.
This makes it so this pod can't re-use the same node from the previous steps.
Is this intended since this is a lightweight pod that should run in a slimmer node?
https://github.com/Netflix/metaflow/blob/master/metaflow/plugins/argo/argo_workflows.py#L3272
https://github.com/Netflix/metaflow/blob/master/metaflow/plugins/argo/argo_workflows.py#L2850-L2853
It seems like the default
nodeSelectorandtolerationsadded to all step pods (via METAFLOW_KUBERNETES_TOLERATIONS and METAFLOW_KUBERNETES_NODE_SELECTOR), aren't added to the argo workflows error-msg-capture-hook pod, as well.This makes it so this pod can't re-use the same node from the previous steps.
Is this intended since this is a lightweight pod that should run in a slimmer node?
https://github.com/Netflix/metaflow/blob/master/metaflow/plugins/argo/argo_workflows.py#L3272
https://github.com/Netflix/metaflow/blob/master/metaflow/plugins/argo/argo_workflows.py#L2850-L2853