Fix ListBucket in s3bolt backend to honor page.Marker and page.MaxKeys

[Aliexe-code]

Fixes #66

The s3bolt backend now properly supports pagination with the marker parameter and maxKeys limit, matching the behavior of the s3mem backend.

Changes:

• Remove ErrInternalPageNotImplemented error for non-empty pages
• Implement page.Marker support using c.Seek() to jump to the marker
• Implement page.MaxKeys support with proper IsTruncated and NextMarker
• Add comprehensive tests for marker-based pagination

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 51.70%. Comparing base (ebf3e50) to head (48b337d).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #114      +/-   ##
==========================================
+ Coverage   49.35%   51.70%   +2.34%     
==========================================
  Files          47       47              
  Lines        5155     4000    -1155     
==========================================
- Hits         2544     2068     -476     
+ Misses       2299     1615     -684     
- Partials      312      317       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[johannesboyne]

@Aliexe-code - thanks a lot for the contribution! Just some minor remarks - see comments.

For testing, these tests could be added

package s3bolt

import (
    "os"
    "strings"
    "testing"

    "github.com/johannesboyne/gofakes3"
)

func mkBackend(t *testing.T, objs []string) (*Backend, func()) {
    t.Helper()
    f, err := os.CreateTemp("", "pr114-*.db")
    if err != nil { t.Fatal(err) }
    _ = f.Close()
    b, err := NewFile(f.Name())
    if err != nil { t.Fatal(err) }
    if err := b.CreateBucket("b"); err != nil { t.Fatal(err) }
    for _, o := range objs {
        if _, err := b.PutObject("b", o, nil, strings.NewReader(o), int64(len(o)), nil); err != nil { t.Fatal(err) }
    }
    return b, func(){ _ = os.Remove(f.Name()) }
}

func TestRepro_NextMarkerEmptyWithDelimiter(t *testing.T) {
    b, done := mkBackend(t, []string{"a/1", "a/2", "a/3", "b/1"})
    defer done()
    p := &gofakes3.Prefix{HasDelimiter: true, Delimiter: "/"}
    out, err := b.ListBucket("b", p, gofakes3.ListBucketPage{MaxKeys:1})
    if err != nil { t.Fatal(err) }
    if !out.IsTruncated { t.Fatalf("expected truncated") }
    if out.NextMarker == "" { t.Fatalf("expected non-empty next marker, got empty") }
}

func TestRepro_TruncatedShouldRespectPrefixFilter(t *testing.T) {
    b, done := mkBackend(t, []string{"x/1", "x/2", "y/1"})
    defer done()
    p := &gofakes3.Prefix{HasPrefix:true, Prefix:"x/"}
    out, err := b.ListBucket("b", p, gofakes3.ListBucketPage{MaxKeys:2})
    if err != nil { t.Fatal(err) }
    if out.IsTruncated { t.Fatalf("expected not truncated when no more matching keys") }
}

func TestRepro_DuplicateCommonPrefixCountsTowardMaxKeys(t *testing.T) {
    b, done := mkBackend(t, []string{"a/1", "a/2", "a/3", "b/1"})
    defer done()
    p := &gofakes3.Prefix{HasDelimiter:true, Delimiter:"/"}
    out, err := b.ListBucket("b", p, gofakes3.ListBucketPage{MaxKeys:2})
    if err != nil { t.Fatal(err) }
    if len(out.CommonPrefixes) != 2 {
        t.Fatalf("expected 2 common prefixes (a/, b/), got %d", len(out.CommonPrefixes))
    }
}

[johannesboyne]

IsTruncated is computed by checking whether any later key exists (c.Next()), not whether any later key matches the requested prefix/delimiter filter. So, I assume paginated calls can return IsTruncated=true even when there are no more matching results.

[johannesboyne]

When a page is truncated on CommonPrefixes entries, NextMarker can be empty because lastKey is only set for object contents, not for common-prefix matches, so clients cannot reliably request the next page (especially for delimiter-based listing).

[johannesboyne]

cnt increments for every matching key, including duplicates that collapse into the same CommonPrefix. This violates each rolled-up prefix counts once behavior and causes early truncation.

[sonarqubecloud]

Quality Gate passed

Issues
6 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud