Fix: RestAPIHeaderLinkPaginator broken for non-'items' response keys

[baconcheese113]

Summary

Fixes two critical bugs in RestAPIHeaderLinkPaginator that have prevented Link header pagination from working with APIs that don't use the items response key. These bugs have existed since the paginator was introduced in PR #36 (July 2023).

Bugs Fixed

Bug #1: Early exit when response lacks 'items' key (pagination.py)

Location: RestAPIHeaderLinkPaginator.get_next_url() (lines 170-180)

Problem: The method was hardcoded to check for an items key in the response body. If not found, it would return None immediately, completely ignoring the Link headers.

Impact: Pagination failed for any API that doesn't use items as the response key, including:

• GitHub Copilot API (seats key)
• Many other REST APIs with different response structures

Fix: Moved the response body parsing inside the conditional block that only applies when using the fake_since workaround. Now properly follows RFC 8288 Link headers regardless of response body structure.

Bug #2: Malformed URL from ParseResult (streams.py)

Location: _get_url_params_header_link() (line ~478)

Problem: Calling str() on a ParseResult object created malformed URL strings with nested ParseResult representations instead of properly extracted query strings.

Fix: Added proper ParseResult handling using hasattr() and .query property, with backward-compatible fallback to str() for other types.

Testing

• Added 2 tests in test_streams.py validating both new functionality and backward compatibility
• All 9 existing tests continue to pass
• Tested with real GitHub Copilot API: successfully extracts all 379 records across 4 pages (previously only extracted first 100)

Backward Compatibility

✅ Non-breaking change - Existing implementations using items key continue to work exactly as before. These fixes only enable previously broken use cases.

Related Issues

• Addresses issues similar to Pagination isn't working as expected #25, Understanding pagination  #28, Paginate until no records or limit #41 about pagination problems
• Enables the documented GitHub API example in README that was previously broken