refactor(provider): remove dependency on probe-lab/go-libdht@v0.4.0

[lidel]

Warning

Opening this to kick-off discussion – we need to make decision here before Kubo 0.39 (ipfs/kubo#10946) ships by end of November.
@guillaumemichel @gammazero @aschmahmann for visibility/feedback. >
TBD if we do this. We may work with probe-lab to share maintenance rights with shipyard instead, as mid-term solution.

Removes external dependency on github.com/probe-lab/go-libdht@v0.4.0 by importing required Kademlia utilities into provider/internal/kad/.

Motivation

The provider sweep system relies on key abstractions and XOR trie structures from go-libdht. With sweep becoming the default provider strategy in Kubo 0.39, this external dependency creates maintenance risk for critical infrastructure.

Rationale & Alternative

@guillaumemichel still had maintenance permissions from the time probelab and shipyard were the same entitty, but other shipyard members are not able to do the same.

Internalizing ~1,800 LOC of stable, well-tested code gives us full control without upstream breaking change risk (low) and potential ciritcal bugfix releases being blocked by lack of maintenance permissions (high). Alternative would be granting shipyard team maintenance rights to https://github.com/probe-lab/go-libdht.

Changes

Imported to provider/internal/kad/

• kad/key.go - Key[K] interface (Kademlia key abstraction)
• kad/key/ - bit256, bitstr implementations + utilities
• kad/trie/ - XOR trie data structure
• kad/kadtest/ - test utilities (required by trie tests)

Not Imported

• kad/triert/ - routing table implementation (unused, probably used only by probe-lab/zikade)

Imports Updated

-"github.com/probe-lab/go-libdht/kad"
+"github.com/libp2p/go-libp2p-kad-dht/provider/internal/kad"

Git History

Full commit history from go-libdht@v0.4.0 preserved using git subtree add. Original authors and timestamps maintained for git blame and git log.

Using subtree ensures the history is exactly the same, but we could rewrite history if we want things like " #2 " to link to proper issue/pr in correct repository.

TODO

• import with subtree at tmp/go-libdht
• move used code to provider/internal/kad, remove the rest
• update imports
• Testing
  • All tests pass (go test ./...)
  • No public API changes

License

go-libdht uses Permissive License Stack (Apache-2.0 OR MIT), compatible with go-libp2p-kad-dht's MIT license.

[gammazero]

No problem with this PR, LGTM.

Too bad we will give up the change history as there were many changes made to this code as part of the sweeping reprovide work. If there is any particularly informative change history (as identified by @guillaumemichel) from the old repo, then that can be made into documentation in the new repo. No need to block waiting for that.

[lidel]

Triage notes:

• @gammazero identified that I may need to redo this PR to also keep commit history and code of kad/triert (XOR-trie based routing table #572 (comment))
• @cewood is attending meetup soon, he will ask ProbeLab about adding more Shipyard maintainers to probe-lab/go-libdht repo as a short-term maintenance solution
  • If that happens, we will park this PR until @guillaumemichel is back, and then decide if we do this PR or move both kad-dht and go-libdht to boxo/dht/kad|amino
  • If that does not happen, we need to merge this PR to be able to ship bugfixes in sweeping provider code

[dennis-tra]

@lidel @gammazero I've just made you admins for the probe-lab/go-libdht 👍

[lidel]

Thank you @dennis-tra <3 This solves the problem at hand.

Closing this for now – we may reuse this strategy in the future to consolidate repos if/when kad-dht has to be moved out of libp2p (entire IPFS-specific provide code lives here now), but that is a topic for 2026.