WebDev Backup Tool

A backup solution for web development projects: multiple source directories, full/incremental/differential backups, compression, integrity verification, and optional cloud (planned).

Quick Start (clone and run in 2 minutes)

Clone, run install (sets permissions, creates dirs, checks tools), run tests, edit config, then backup.

git clone https://github.com/ICJIA/backup-webdev.git
cd backup-webdev
./install.sh                    # Sets chmod +x, creates logs/test, checks tools (decline alias/security prompts if you prefer)
./run-tests.sh                  # Verify it works (48 tests, no config needed)

Edit config.sh — set your source and destination paths (see Configuration):

# Defaults: $HOME and $HOME/backups — edit to narrow scope (e.g. $HOME/Developer, $HOME/projects)

Then run:

./backup.sh --dry-run            # Smoke test (no writes)
./webdev-backup.sh               # Interactive menu — press Enter for Quick Backup
./backup.sh --quick              # Direct quick backup (no menu)

Optional: Run ./setup-alias.sh to add a webback alias. This is a run-from-repo tool — no system-wide install.

Features

• Backup: Multi-directory, full/incremental/differential, quick backup (default from menu), compression (gzip/pigz). node_modules is never included in backups to save space; you can rebuild dependencies after restore with yarn install or npm install. Each .tar.gz is self-contained: extracting creates a single top-level folder with everything inside. Quick backup shows live per-folder progress (same dashboard as interactive).
• Restore: With integrity validation (tar + SHA256); optional --skip-verify
• Reporting: HTML reports, email, charts (gnuplot), backup comparison
• Security: Permissions, secrets handling, security audit script
• Platforms: macOS, Linux, WSL2 (Bash 3.2+; paths differ by platform)

Requirements

• Bash 3.2+ (macOS-compatible)
• tar, gzip (required); pigz, gnuplot, AWS CLI (optional)

Usage

• Menu: ./webdev-backup.sh — option 1 (Quick Backup) is the default; press Enter to run it.
• Quick backup: ./backup.sh --quick — uses config defaults, shows dashboard progress; or ./backup.sh --silent for minimal output.
• With options: ./backup.sh --incremental --verify, ./backup.sh --source ~/projects, ./backup.sh --sources ~/a,~/b --verify
• Restore: ./restore.sh --latest, ./restore.sh --project myproject --dest ~/restored
• Compare: ./compare-backups.sh --backup1 latest --backup2 latest (optional --project, --format json, --only-changes)
• Logs: ./webdev-backup.sh --list-log (all log files), --list-log --short (paths only), --list-log-errors (error/warning lines only)

Testing

Tests run with no config (RUNNING_TESTS uses test/ and test/backup_out).

./run-tests.sh              # Unit + integration + security audit
./run-tests.sh --unit       # Unit only (38 tests)
./run-tests.sh --integration # Integration only (10 tests)
./run-tests.sh --security   # Security audit only

Coverage (48 tests):

Category	Tests	What's covered
Unit (38)	format_size (0 B through GB), sanitize_input (basic + strict), validate_path (traversal, injection, empty, relative, absolute), detect_os, get_os_version_display, get_file_size_bytes (file + missing), calculate_checksum (consistency + SHA256 length), check_required_tools (present + missing), format_time (seconds/minutes/hours), capitalize, verify_directory (exists + missing), find_projects, verify_backup (valid + corrupted archive), is_safe_config_literal checks, and sanitize_cron_backup_options allow/reject cases	Core utility functions, edge cases, and error paths
Integration (10)	Backup dry-run, full backup, backup-file existence, node_modules exclusion, archive integrity (verify_backup), source-file presence in archive, .env and lockfiles included, restore dry-run, incremental backup, config with RUNNING_TESTS	End-to-end backup/restore workflow and configuration
Security	File permissions, sensitive files in git, hardcoded credentials, eval usage, temp file handling	Static analysis of common vulnerabilities

You can also run ./test-backup.sh (with --quick, --unit, or --integration) or npm test.

Platform verification (macOS, Linux, WSL2): The same test suite and scripts run on all three. Ubuntu is the primary Linux target.

Platform	How to verify
macOS	./run-tests.sh (all 48 tests), ./backup.sh --quick --dry-run, ./verify-implementation.sh
Linux (Ubuntu)	Same commands; uses GNU tools (stat -c, date -d, etc.)
WSL2	Same as Linux; use /mnt/c, /mnt/d etc. for Windows drives (see Paths by platform below)

• Compatibility check: ./verify-implementation.sh confirms config, first-run, and OS-specific paths (e.g. get_file_size_bytes, run_with_timeout, tar flags). It reports macOS or Linux and suggests next steps.
• Quick smoke test: ./backup.sh --quick --dry-run shows the backup dashboard and a simulated run without writing files.

The app uses uname -s to choose the right commands (e.g. BSD vs GNU find, stat vs du, date -r vs date -d).

Exclusions and what’s included

• node_modules – Excluded from all backups (full, incremental, differential, and quick) to save space. Dependencies are easy to rebuild after restore: run yarn install or npm install in each project directory. Exclusion is verified by the integration test suite.

• Always included (so you can reconstruct the app after restore): .env (and common variants like .env.local), yarn.lock, and npm lockfiles (package-lock.json, npm-shrinkwrap.json) are not excluded. With these in the backup, a restore plus yarn install or npm install reproduces the same dependency tree and environment.

Configuration

Edit config.sh. Sensible defaults ($HOME, $HOME/backups) are pre-configured; edit to narrow scope (e.g. $HOME/Developer) or use different paths.

• DEFAULT_SOURCE_DIRS – array of directories to back up
• DEFAULT_BACKUP_DIR – backup destination
• DEFAULT_VERIFY_BACKUP – true (default) or false. When true, verification is on by default; interactive and Quick Backup prompts use this as the default ([Y/n] or [y/N]).
• Cloud: Leave DEFAULT_CLOUD_PROVIDER blank (reserved for later)

Paths by platform:

Platform	Internal / home	External / mounted
macOS	$HOME, $HOME/Developer	/Volumes/VolumeName/...
Linux	$HOME, /home/user/...	/mnt/..., /media/...
WSL2	$HOME	/mnt/c, /mnt/e, etc. (no /media/)

First run may show FIRST TIME SETUP (config path and prompt). Current config is shown when you run the menu or interactive backup.

Verify your config with a dry run — no writes: no directories, no log files, no archives, no verification. Pure simulation:

./backup.sh --dry-run          # From the repo directory
./backup.sh --dry-run --silent # Same, but minimal output
webback --dry-run              # Works with the alias too

All command-line options pass through to the alias. Dry-run only checks that sources exist, the destination (or its parent) is writable, and projects are detected; it does not create or modify any files.

Security

1. ./secure-permissions.sh – set strict permissions (755 scripts, 640 config, 750 dirs)
2. ./secure-secrets.sh – set up credential storage
3. ./security-audit.sh – checks permissions, sensitive files in git, hardcoded credentials, eval, temp files (also runs with ./run-tests.sh)

Pull Request Checklist

Use this checklist before opening or merging a PR:

• Ran ./run-tests.sh locally and all suites passed.
• Ran ./security-audit.sh and it reported no issues.
• Verified on at least one target platform (macOS or Ubuntu/WSL2).
• If shell scripts changed, ensured bash -n passes.
• Updated README.md or docs for any behavior/CLI changes.
• Confirmed no secrets were added (especially secrets.sh, .env, credentials).

CI (.github/workflows/ci.yml) enforces syntax checks and full tests on macOS and Ubuntu.

Contributing

Before opening a PR, run the local quality gate:

./run-tests.sh && ./security-audit.sh

Recommended when changing shell scripts:

find . -type f -name "*.sh" -exec bash -n {} \;

If your change affects platform-specific behavior, validate at least one of:

• macOS
• Ubuntu Linux (or WSL2 Ubuntu)

What's hardened:

• All temp files use mktemp (no /tmp/ predictable paths in active code)
• umask 027 in utils, encryption, and setup scripts
• Tar extraction uses --no-same-owner on Linux to prevent root-owned restored files
• Archive traversal checks (absolute paths, ../) before extraction
• Mail credentials wiped with shred/dd after use
• secure-permissions.sh sets chmod 755 scripts, 640 config, 750 dirs (not 777)

Full details: docs/SECURITY_REVIEW.md

Scripts (root)

Script	Purpose
webdev-backup.sh	Main menu
backup.sh, restore.sh, quick-backup.sh	Backup and restore
prune-backups.sh	Prune old backups (keep 5 latest or delete one by one)
config.sh, utils.sh, fs.sh, ui.sh	Config and shared modules
utils-os.sh, utils-time.sh, utils-path.sh, utils-format.sh	Utils submodules (sourced by utils.sh)
parse-backup-args.sh	CLI parsing for backup.sh
configure-cron.sh	Cron schedules (uses mktemp)
compare-backups.sh	Compare two backups
run-tests.sh, test-backup.sh	Tests (unit + integration + security)
test/test-cron.sh, test/test-tar-compatibility.sh	Cron and tar compatibility tests (npm run test:cron, test:tar)
check-config.sh, security-audit.sh, secure-*.sh	Config and security
list-logs.sh	List log files: --short for paths only; --errors or --list-log-errors for error/warning lines only
cleanup-backup-files.sh	Organize backup dir: npm run organize, organize:all, organize:dry

Legacy code: archive/src.legacy/. More docs: docs/, test/README.md.

Adding a source directory

• Menu: Manage Source Directories → Add
• Or edit config.sh (DEFAULT_SOURCE_DIRS) or use --sources path1,path2

Troubleshooting

Issue	What to do
Permission denied	chmod +x *.sh or ./secure-permissions.sh
Path not found	./check-config.sh, ./dirs-status.sh, edit config.sh
Tests fail	./run-tests.sh (no config needed); ensure scripts executable
Backup full	./cleanup.sh --days 30 or change destination
Restore validation fails	Try another backup with ./restore.sh --list; or --skip-verify if sure
Config issues	./check-config.sh, edit config.sh, try ./backup.sh --dry-run

Logs: logs/backup_history.log, logs/failed_backups.log. List all: ./webdev-backup.sh --list-log (or ./list-logs.sh); add --short for paths only. Only errors/warnings: ./webdev-backup.sh --list-log-errors. Diagnostics: ./debug-backup.sh, ./security-audit.sh.

NPM scripts

Common: npm start, npm test, npm run backup, npm run backup:quick, npm run backup:dry, npm run restore, npm run cron, npm run cleanup. Run npm run to list all.

More

• Architecture: docs/ARCHITECTURE.md
• Security review: docs/SECURITY_REVIEW.md
• Changelog: CHANGELOG.md
• License: MIT (see LICENSE)