[HELM][GCS] Found issue in helm template related to GCS Storage in airbyte-workload-api-server volume

[Jérôme Bourgeais (jrmbrgs)]

Hello Airbyte !

Thx you for the OSS product you build !
I think I found a small bug in a helm deployment chart.

Context

According to the Helm External Storage implementation guide.
We can define a secret containing the service account json key, as defined in the Integration storage doc like :

apiVersion: v1
kind: Secret
metadata:
  name: airbyte-config-secrets
type: Opaque
stringData:
 gcp.json: |
    {
      "type": "service_account",
      ...
    }

Then referencing the previously created secret in the values.yaml file as mentioned in the documentation

global:
  storage:
    type: "GCS"
    storageSecretName: airbyte-config-secrets
    bucket:
      log: airbyte-bucket
      state: airbyte-bucket
      workloadOutput: airbyte-bucket
    gcs:
      projectId: <project-id>

Later, during the deployment, this secret is used to populate the gcs-log-creds-volume volume for at least the airbyte-workload-api-server and airbyte-server charts

Issue

While this is working for airbyte-server charts I've found an issue w/ airbyte-workload-api-server.
The airbyte-workload-api-server deployment fails because of this err :

MountVolume.SetUp failed for volume "gcs-log-creds-volume" : secret "airbyte-gcs-log-creds" not found

At the end, the secret you've provided in your values.yaml is not used, the default one is kept instead

Code

In airbyte-server chart deployment file

volumes:
    {{- if eq .Values.global.deploymentMode "oss" }}
      {{- if eq (lower (default "" .Values.global.storage.type)) "gcs" }}
      - name: gcs-log-creds-volume
        secret:
          secretName: {{ ternary (printf "%s-gcs-log-creds" ( .Release.Name )) .Values.global.storage.storageSecretName (not ((.Values.global.storage).storageSecretName)) }}
      {{- end }}

While in airbyte-workload-api-server chart deployment file

volumes:
      {{- if and (eq .Values.global.deploymentMode "oss") (eq (lower (default "" .Values.global.storage.type)) "gcs") }}
      - name: gcs-log-creds-volume
        secret:
          secretName: {{ ternary (printf "%s-gcs-log-creds" ( .Release.Name )) (.Values.global.credVolumeOverride) (eq .Values.global.deploymentMode "oss") }}
      {{- end }}

It sounds the helm ternary function should looks like the one in airbyte-server to use the .Values.global.storage.storageSecretName if it has been provided in the values.yaml

...
secretName: {{ ternary (printf "%s-gcs-log-creds" ( .Release.Name )) .Values.global.storage.storageSecretName (not ((.Values.global.storage).storageSecretName)) }}
...

Regarding the small fix it might represent I was wondering if it was worth sending a PR.

[Airbyte Support Bot (airbyte-support-bot)]

Hi! This discussion hasn't had activity in over a year. Is this still relevant to you? If so, please reply and we'll keep it open. If we don't hear back within 14 days, we'll close this as part of our housekeeping. You can always reopen or create a new discussion later. Thanks!

---

Devin session

[Kévin (kev-datams)]

Keep it open