Skip to content

README.md

Latest commit

 

History

History
200 lines (146 loc) · 6.29 KB

README.md

File metadata and controls

200 lines (146 loc) · 6.29 KB

redis2github

Syncs IP ban lists from Redis / Valkey / BunkerWeb to a GitHub file on a schedule. Use the GitHub file as a firewall blacklist — it accumulates IPs over time and never overwrites existing entries.

What it does

  • Reads banned IPs from Redis (set, zset, list, hash, string) or from bwcli bans (BunkerWeb)
  • Merges them with whatever is already in the GitHub file (accumulate-only, never overwrite)
  • Commits the result to GitHub on a configurable interval
  • Runs as a background daemon that survives reboots via systemd
  • Encrypts credentials at rest with Fernet (AES-128)

Requirements

  • Python 3.8+
  • redis, requests, cryptography pip packages (auto-installed on first run)
  • A GitHub personal access token with repo scope (or contents:write for fine-grained tokens)
  • Redis / Valkey / Sentinel accessible from the host

Installation

git clone <repo>
cd redis2github
sudo bash install.sh

The installer copies redis2github.py to /usr/local/lib/ and creates a wrapper at /usr/local/bin/redis2github.

Quick start

redis2github setup            # configure Redis + GitHub credentials
redis2github sync             # test: push IPs now
redis2github start            # start background daemon
redis2github status           # confirm it is running
redis2github install-service  # make it survive reboots
sudo loginctl enable-linger $USER

BunkerWeb (bwcli) setup

Why sudo is needed

bwcli bans requires root. The daemon runs without a terminal, so sudo cannot ask for a password interactively. You must grant passwordless sudo for bwcli once.

Fix (run once on the server)

Find where bwcli is installed:

which bwcli

Add a sudoers rule (replace path and username if different):

# For user ubuntu with bwcli at /usr/bin/bwcli:
echo "ubuntu ALL=(ALL) NOPASSWD: /usr/bin/bwcli" | sudo tee /etc/sudoers.d/redis2github-bwcli
sudo chmod 440 /etc/sudoers.d/redis2github-bwcli

Verify it works without a password prompt:

sudo bwcli bans

Expected log output after a working sync

[INFO] [job:job1] bwcli returned 47 IPs (attempt 1)
[INFO] [job:job1] 'spam-ips.txt': 19 existing + 28 new = 47 total
[INFO] [job:job1] Committed 47 IPs → 'spam-ips.txt'

Diagnosing bwcli issues

redis2github test-bwcli --sudo
Log message Cause Fix
sudo: a password is required No sudoers rule Add /etc/sudoers.d/redis2github-bwcli (see above)
bwcli returned 0 IPs Output not captured Tool uses a pseudo-TTY internally — update to latest version
bwcli not found in PATH Wrong path Check which bwcli and use --container if in Docker
bwcli bans timed out Slow API response Increase bwcli_timeout via redis2github edit-job

Commands

Command Description
setup Interactive wizard — Redis, GitHub token, first job
setup-analysis Configure automatic IP blacklisting from request reports
config Show current config (secrets masked)
add-job Add a sync job
edit-job Edit an existing job
remove-job Remove a job
start Start the background daemon (auto-backgrounds from a terminal)
stop Stop the daemon
status Show daemon status, jobs, last sync times
sync Run all jobs once immediately
run-job [ID] Run one specific job immediately
analyze Run pattern analysis once
test-bwcli Debug bwcli — shows raw output and parsed IPs
list-keys List Redis keys
list-jobs List configured jobs
logs [-f] Show log file (-f to follow live like tail -f)
install-service Install systemd user service for auto-start on boot
update --url URL Download and install an updated version

Daemon and reboot survival

redis2github start auto-backgrounds when called from a terminal. The daemon saves last_sync after every job run, so if it restarts it resumes from where it left off rather than starting a full new interval.

To survive reboots:

redis2github install-service
sudo loginctl enable-linger $USER

This creates a systemd user service (~/.config/systemd/user/redis2github.service) with Restart=always. The service runs as the correct user even if invoked via sudo.

Useful systemd commands:

systemctl --user status redis2github
systemctl --user restart redis2github
journalctl --user -u redis2github -f

Source modes

When adding a job (add-job) you choose one of three source modes:

Mode What it reads Use case
value Value of one Redis key (set/zset/list/hash/string) Generic Redis ban sets
key_names Scans keys matching a glob, extracts IPs from key names BunkerWeb bad-behavior counters (metrics:badbehavior_counter_ip_*)
bwcli Runs bwcli bans, parses the full ban list BunkerWeb active bans

The bwcli mode can optionally also merge a Redis key in the same job.

Pattern analysis

Enable with setup-analysis. The analyser reads a Redis key containing HTTP request reports and blacklists IPs that hit the same suspicious pattern ≥ N times (default 30).

Supported report formats: json_list, hash, zset, list, string.

Detected patterns: admin panels, PHP probes, .env/.git access, path traversal, CMS probes, webshells, SQL injection, code injection, info disclosure, auth probes, LFI, XSS.

File locations

Path Purpose
~/.config/redis2github/config.json Encrypted config and job definitions
~/.config/redis2github/.fernet.key Encryption key (chmod 600, never share)
~/.config/redis2github/redis2github.log Log file
~/.config/redis2github/daemon.pid PID of running daemon
~/.config/systemd/user/redis2github.service systemd unit (created by install-service)
/usr/local/lib/redis2github.py Installed script
/usr/local/bin/redis2github Shell wrapper
/etc/sudoers.d/redis2github-bwcli Passwordless sudo rule for bwcli

Updating

sudo cp redis2github.py /usr/local/lib/redis2github.py
redis2github stop
redis2github start

Existing jobs and config are preserved — no reconfiguration needed.

If using systemd:

sudo cp redis2github.py /usr/local/lib/redis2github.py
systemctl --user restart redis2github