chore: [generate_all.sh] wire in fastly, oracle-oci and driftnet generators - #375
Merged
Merged
Conversation
…t-net/lots-project + re-enable publicdns/tranco/majestic-million/gptbot Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0121gk7vaCXX9bMri7tC9XVd
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Four lines in
generate_all.sh, wiring in generators that exist intools/but were never called from it:Why
generate_all.shhad drifted out of sync withtools/.generate-fastly.py,generate-oracle-oci.py,generate-cloudflare-top-domains.pyandgenerate-driftnet.pywere absent from it entirely, yet thefastlyandoracle-ocilists are current — someone has been running them by hand. That is the same failure mode that letcheck-host-netandlots-projectgo stale for years.Every added line was run first
A generator that fails breaks the whole batch (
set -e), so nothing was wired in unverified:generate-oracle-oci.pyoracle-oci793 → 798generate-fastly.pyfastly21 entriesgenerate-driftnet.pygenerate-cloudflare-top-domains.pyKeyError: 'CLOUDFLARE_API_TOKEN'— needs a Cloudflare Radar API tokenDeliberately not wired in, with reasons:
generate-akamai.py— its only sourceapi.bgpview.iois NXDOMAIN, so it cannot run at all; adding it to aset -ebatch would break the nightly run.generate_alexa.py,generate-google.py,generate-stackpath.py,generate-windows-binary-hashes.py— already disabled with documented reasons.Scope
This PR touches
generate_all.shonly, and only the four lines above. Re-enablinggenerate_tranco.py,generate_majestic-million.py,generate-publicdns.py,generate-gptbot.py,generate-check-host-net.pyandgenerate-lots-project.pyis deliberately left to the separate PRs that fix and verify each of those generators, so this one does not conflict with them.Verification
bash -n generate_all.shpasses; each enabled generator was executed successfully before its line was added.🤖 Generated with Claude Code
https://claude.ai/code/session_0121gk7vaCXX9bMri7tC9XVd