fix: RetainUntilDate in the correct format - #1890
Conversation
|
who is returning this error? which server @gribouille |
|
OVHcloud s3 object storage: https://www.ovhcloud.com/en/public-cloud/object-storage/ The problem is similar to: minio/minio#15741 |
AWS S3 runs fine, this looks like OVH Cloud bug. |
|
And has been for years. I don't think we can take this PR. It looks like server implementation bug. |
This is not the same problem as here. Server preserves meta is iso9601 format. But it is okay to send it either in iso8601 or rfc3339 |
|
|
||
| if !opts.RetainUntilDate.IsZero() { | ||
| header.Set("X-Amz-Object-Lock-Retain-Until-Date", opts.RetainUntilDate.Format(time.RFC3339)) | ||
| header.Set(amzLockRetainUntil, opts.RetainUntilDate.Format(iso8601TimeFormat)) |
There was a problem hiding this comment.
The trailing Z in iso8601TimeFormat is a literal, so this writes the time's local wall clock stamped as UTC — a RetainUntilDate carrying a zone offset is sent as a different instant, and for west-of-UTC zones the object becomes deletable earlier than requested. Verified against a live MinIO server: a +02:00 value came back with retention two hours later than asked; the time.RFC3339 form this replaces encoded offsets correctly, and master applied the same .UTC() normalization to this exact layout in #2135.
| header.Set(amzLockRetainUntil, opts.RetainUntilDate.Format(iso8601TimeFormat)) | |
| header.Set(amzLockRetainUntil, opts.RetainUntilDate.UTC().Format(iso8601TimeFormat)) |
| if opts.Mode != RetentionMode("") && !opts.RetainUntilDate.IsZero() { | ||
| header.Set(amzLockMode, opts.Mode.String()) | ||
| header.Set(amzLockRetainUntil, opts.RetainUntilDate.Format(time.RFC3339)) | ||
| header.Set(amzLockRetainUntil, opts.RetainUntilDate.Format(iso8601TimeFormat)) |
There was a problem hiding this comment.
Same as api-put-object.go:161 — convert to UTC before formatting with the literal-Z layout.
| header.Set(amzLockRetainUntil, opts.RetainUntilDate.Format(iso8601TimeFormat)) | |
| header.Set(amzLockRetainUntil, opts.RetainUntilDate.UTC().Format(iso8601TimeFormat)) |
| const ( | ||
| signV4Algorithm = "AWS4-HMAC-SHA256" | ||
| iso8601DateFormat = "20060102T150405Z" | ||
| iso8601TimeFormat = "2006-01-02T15:04:05.000Z" | ||
| ) |
There was a problem hiding this comment.
This block is documented as signature-related, but the new layout is used only by the object-lock header code — a reader tracing signing finds an unrelated constant, and a reader of the object-lock path never looks here. Splitting it out also gives it a place to record that the trailing Z is a literal, so values must be converted to UTC before formatting. The same layout string already exists in this package as expirationDateFormat (post-policy.go:31), so pointing both call sites at one shared constant is an option too — kept separate here to match the MinIO server's own iso8601TimeFormat name; your call.
| const ( | |
| signV4Algorithm = "AWS4-HMAC-SHA256" | |
| iso8601DateFormat = "20060102T150405Z" | |
| iso8601TimeFormat = "2006-01-02T15:04:05.000Z" | |
| ) | |
| const ( | |
| signV4Algorithm = "AWS4-HMAC-SHA256" | |
| iso8601DateFormat = "20060102T150405Z" | |
| ) | |
| // iso8601TimeFormat is the millisecond-precision ISO 8601 layout used for | |
| // S3 object-lock headers. The trailing 'Z' is a literal, so values must be | |
| // converted to UTC before formatting. | |
| const iso8601TimeFormat = "2006-01-02T15:04:05.000Z" |
| @@ -158,7 +158,7 @@ func (opts PutObjectOptions) Header() (header http.Header) { | |||
| } | |||
|
|
|||
| if !opts.RetainUntilDate.IsZero() { | |||
There was a problem hiding this comment.
Nothing in the tree asserts the header value these options produce, which is how a silent instant shift passes every existing test. This table test — appended to api-put-object_test.go, with "time" added to its imports — fails on the current branch for the two offset cases and passes once the .UTC() fixes above are applied.
func TestRetainUntilDateHeader(t *testing.T) {
testCases := []struct {
name string
nsec int
loc *time.Location
want string
}{
{
name: "utc",
nsec: 123456789,
loc: time.UTC,
want: "2030-01-02T12:00:00.123Z",
},
{
name: "utc-zero-fraction",
nsec: 0,
loc: time.UTC,
want: "2030-01-02T12:00:00.000Z",
},
{
// 12:00 at +05:30 is 06:30 UTC. The header layout ends in a
// literal 'Z', so the value must be normalized to UTC first.
name: "positive-offset",
nsec: 123456789,
loc: time.FixedZone("UTC+05:30", 5*3600+1800),
want: "2030-01-02T06:30:00.123Z",
},
{
// 12:00 at -08:00 is 20:00 UTC. Getting this wrong shortens
// the retention period, making the object deletable early.
name: "negative-offset",
nsec: 123456789,
loc: time.FixedZone("UTC-08:00", -8*3600),
want: "2030-01-02T20:00:00.123Z",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
retainUntil := time.Date(2030, time.January, 2, 12, 0, 0, tc.nsec, tc.loc)
putHdr := PutObjectOptions{
Mode: Governance,
RetainUntilDate: retainUntil,
}.Header()
if got := putHdr.Get(amzLockRetainUntil); got != tc.want {
t.Errorf("PutObjectOptions.Header()[%s] = %q, want %q", amzLockRetainUntil, got, tc.want)
}
copyHdr := http.Header{}
CopyDestOptions{
Bucket: "bucket",
Object: "object",
Mode: Governance,
RetainUntilDate: retainUntil,
}.Marshal(copyHdr)
if got := copyHdr.Get(amzLockRetainUntil); got != tc.want {
t.Errorf("CopyDestOptions.Marshal()[%s] = %q, want %q", amzLockRetainUntil, got, tc.want)
}
})
}
}|
Proposed title: Proposed description: Some S3-compatible services reject the `X-Amz-Object-Lock-Retain-Until-Date` header produced by `time.RFC3339`, which renders whole seconds without a fraction and non-UTC times with a numeric zone offset. OVHcloud Object Storage answers `The retain until date must be provided in ISO 8601 format`; AWS S3 and MinIO accept both forms.
This changes `PutObjectOptions` and `CopyDestOptions` to send the millisecond-precision UTC form `2006-01-02T15:04:05.000Z` — the same layout the MinIO server itself uses. The value is converted to UTC before formatting because the trailing `Z` in that layout is a literal. The new form is sent to every endpoint. MinIO was verified live to accept it, and AWS S3 is expected to, since AWS's own SDKs emit fractional forms of the same layout — so this widens compatibility without changing what compliant servers store.The current description says RFC3339 is used "instead of the ISO 8601 format", but One dependency: the sentence about converting to UTC describes the branch with the two inline One more gap worth recording while describing scope: |
RetainUntilDateuses thetime.RFC3339format instead of the ISO 8601 format.This problem generates the error
The retain until date must be provided in ISO 8601 format.