[BUG] BackupRepo PreCheck Error for Aliyun OSS: BucketAlreadyExists

[shanshanying]

To Reproduce

1. Install KB 1.0.1-beta 6 (datasafed version 0.2.1)
2. Create BackupRepository

kbcli backuprepo create kb-oss \
  --provider oss \
  --region <ERGION> \
  --bucket  <BUCKET_NAME> \
  --access-key-id <AK> \
  --secret-access-key <SK>  

3. BackupRepo PreCheck Job failed

[shanshanying]

To workaround at the moment we should use --no-check-bucket options

kbcli backuprepo create kb-oss \
  --provider oss \
  --region <ERGION> \
  --bucket  <BUCKET_NAME> \
  --access-key-id <AK> \
  --secret-access-key <SK>  \
  --no-check-bucket

[zjx20]

This is the issue introduced by upgrading datasafed to version 0.2.1 in #9510, which upgraded the rclone dependency and caused some behavioral changes.

rclone always tries to create a bucket upon startup to ensure its existence. If the bucket already exists, s3 will report an error, but in previous versions, rclone would ignore this error and assume the bucket belongs to you. Later, rclone added a feature (rclone/rclone@f56ea2b) that checks the s3 return value to determine if the bucket truly belongs to you, and reports an error if it does not.

However, different s3 providers return different error codes in this situation. Some return AlreadyOwnedByYou, while others return BucketAlreadyExists. Thus, rclone added an option use_already_exists. When this option is true, it assumes the provider supports the AlreadyOwnedByYou error code, so BucketAlreadyExists indicates the bucket belongs to someone else and should trigger an error. rclone sets use_already_exists=true by default for OSS, assuming that if the bucket exists and belongs to you, OSS will return 200 Ok, but this is not the actual case.

In #9555, we set use_already_exists=false for OSS to resolve this issue.