-
Notifications
You must be signed in to change notification settings - Fork 176
Restore fails with x509: certificate signed by unknown authority when using custom CA bundle with MinIO storage (PSMDB Operator 1.22.0) #2257
Description
Report
Summary
Backups complete successfully using a MinIO-compatible storage with a custom CA bundle (self-signed / internal CA), but restore operations fail during the backup validation phase with a classic TLS certificate verification error:
tls: failed to verify certificate: x509: certificate signed by unknown authority
The error occurs when PBM tries to perform a HEAD request on the .pbm.json metadata file in storage.
This strongly suggests that the caBundle setting defined in the PerconaServerMongoDB.spec.backup.storages section is not being propagated / used by the restore process (or by the PBM agent during the pre-restore validation step).
Affected version
- Percona Server for MongoDB Operator: 1.22.0
Current configuration (relevant excerpt)
apiVersion: psmdb.percona.com/v1
kind: PerconaServerMongoDB
spec:
backup:
enabled: true
storages:
main:
type: minio
minio:
bucket: backup-bucket
credentialsSecret: my-cluster-name-backup-minio
endpointUrl: my-s3-compatible-storage.example.com # HTTPS endpoint
prefix: backup
region: us-east-1
secure: true
caBundle:
name: single-root-ca-certificate
key: single-root-ca-certificate.crt
main: true
tasks:
- name: daily
enabled: true
schedule: "25 * * * *"
storageName: main
compressionType: gzip
retention:
type: count
count: 7
deleteFromStorage: trueThe corresponding PerconaServerMongoDBBackup CR status shows the backup completed successfully and also contains the same caBundle configuration:
status:
minio:
bucket: backup-bucket
caBundle:
key: single-root-ca-certificate.crt
name: single-root-ca-certificate
endpointUrl: my-s3-compatible-storage.example.com
prefix: backup
region: us-east-1
secure: true
state: readyObserved behavior during restore
apiVersion: psmdb.percona.com/v1
kind: PerconaServerMongoDBRestore
metadata:
name: my-cluster-restore
spec:
clusterName: my-cluster-psmdb
backupName: cron-my-cluster--20260226094000-dz85m
status:
state: error
error: >-
failed to validate backup: validate backup in storage: get backup metadata
from storage: get from store: list with parts for mw source reader: fetching pbm
file parts base for 2026-02-26T09:40:00Z.pbm.json: get using minio: Head
"https://my-s3-compatible-storage.example.com/backup-bucket/backup/2026-02-26T09%3A40%3A00Z.pbm.json":
tls: failed to verify certificate: x509: certificate signed by unknown authorityExpected behavior
The restore process should respect the same caBundle configuration that was successfully used during backup → it should trust the custom CA and be able to read the metadata file.
Suspected root cause
The caBundle is correctly applied during backup operations (PBM agent writes to storage successfully), but not applied (or not mounted/used) during restore validation or when the operator/PBM agent reads back the backup metadata.
Thanks for looking into this!
More about the problem
Backups complete successfully using a MinIO-compatible storage with a custom CA bundle (self-signed / internal CA), but restore operations fail during the backup validation phase with a classic TLS certificate verification error.
Steps to reproduce
- Deploy a PSMDB cluster (sharded or replica set) using Operator ≥ 1.22.0
- Configure a MinIO-compatible S3 storage with HTTPS + custom/internal CA
- Add
caBundlesecret reference in the storage configuration - Create & complete a logical backup (succeeds)
- Create a
PerconaServerMongoDBRestoreCR pointing to that backup (fails)
Versions
- Kubernetes v1.33.1
- Operator percona/percona-server-mongodb-operator:1.22.0
- Database: percona/percona-server-mongodb:8.0.19-7
- Backup Agent: percona/percona-backup-mongodb:2.12.0
Anything else?
No response