Restructure into src/ layout; harden correctness/security/reliability; add themed CLI, test suite, and CI#32
Open
n0connect wants to merge 7 commits into
Open
Restructure into src/ layout; harden correctness/security/reliability; add themed CLI, test suite, and CI#32n0connect wants to merge 7 commits into
n0connect wants to merge 7 commits into
Conversation
…correctness bugs Introduces the package (errors hierarchy, logging_config, net_utils, sip_packet, threadpool shared by SIP-NES/SIP-ENUM - fixing F2's print interleaving and F4's busy-wait), one run() per module (nes/enum/das), and F1's fix (SIP-DAS reporting success on silently-failed sends). Legacy mr.sip.py/sip_packet.py/utilities.py flat scripts are removed now that src/cli.py dispatches everything. Moves the package to a src/ layout (from an initial mrsip/ layout) and drops pip console-script packaging in favor of the free/open-source pentest-tool convention: clone and run in place as `python3 mr.sip.py ...`. Consolidates badges/logo/screenshots into assets/, and ip_list.txt/from.txt into output/. Correctness/RFC fixes: - F3: SIP-NES --mt=register/subscribe produced invalid SIP-URIs (blanked to_user instead of pairing it with from_user, per RFC 3261 §10.2/§10.3). - F7: CANCEL template's CSeq used the wrong method (RFC 3261 §9.1). - F8: SIP-DAS re-read all four wordlist files from disk every loop iteration. - F9: promisc() shelled out to Linux-only ip-link syntax on macOS/BSD. - F11: dead SIP status-code class constants removed. - F12: unpinned dependencies (broke against a newer Scapy with no warning). - F13: sip_packet.client_port was a class attribute, not an instance one, so every packet in a process's lifetime shared the same source port. - F14: SIP-DAS's -m (manual spoof) mode couldn't parse its own documented ip_list.txt input format.
…estyle README; bump to 1.2.0 - F19: narrow the overly broad `except Exception` in sip_packet.generate_packet() to the actual OSError/UnicodeDecodeError failure modes; anything else (a real bug like TypeError/AttributeError) now propagates as a traceback instead of being misreported as a packet send failure. - F20: SIP-NES's default from/to wordlists (sized for SIP-ENUM/SIP-DAS) were cross-producted for non-register/subscribe message types, so the tool's simplest documented invocation queued up to 81,000,000 requests against a single host. Now sends a single generic probe per target when --from/--to are left at their default path. - F21: unclosed '<' in the From header of bye.message/cancel.message/ sp-invite.message (present since the 2017 initial commit) made ;tag= part of the URI instead of a header parameter, invalid per RFC 3261 §25.1. Verified live: sp-invite now gets a real 401 instead of being silently dropped by the parser. Long-term maintainability: - pytest suite (network-free): IP/CIDR validation, packet templating, CLI parsing, the thread pool, and a static structural check on every .message template. - GitHub Actions CI: ruff + pytest + --help smoke test on every push/PR, across Python 3.9/3.11/3.13. - CHANGELOG.md documenting the full F1-F21 hardening pass. - Wire up the previously-dead __version__ constant to a real --version flag; bump to 1.2.0. Cleanup: - Resolve and remove To-Do.md (SIP-ASP integration is Pro-only, out of scope for this public repo; every other item addressed). - Fix file permissions (only mr.sip.py needs to be executable); move requirements-dev.txt into tests/. - Restyle README.md: centered header, badges, table of contents, tables replacing bullet-list parameter docs, real markdown links, and a Development section pointing at tests/CI/CHANGELOG.md - presentation only, no content rewritten.
…noise, and prompt safety Found and fixed during a full manual test pass against a live Docker/ Asterisk target: - Progress bar was a hand-rolled \r implementation that didn't reliably overwrite its own line outside a raw TTY. Replaced with tqdm (new pinned dependency, >=4.60,<5). - Scapy's send(pkt, iface=conf.iface) passed a dead parameter (confirmed against scapy's own source: L3 send() ignores it and warns), and its default verbose=True spammed "Sent 1 packets." per packet, breaking the progress bar. Removed the dead kwarg, set verbose=False. - prompt_yes_no() - the "N packages will be generated, continue?" gate F20 is about - defaulted to "yes" when stdin was unavailable (EOFError), silently allowing a large cross-product run through with no human confirmation. Now defaults to "no", matching the fail-safe already used for unrecognized typed answers. - A missing --from/--to/--su/--ua/--il wordlist file raised a raw FileNotFoundError in all three modules; now a clean CLI error, caught centrally in cli.main(). - SIP-DAS's -r/-s/-m (IP spoofing) silently did nothing when combined with -l (socket mode can't spoof); now warns instead of pretending to succeed. - The y/n confirmation prompt required pressing Enter after y/n (input() is line-buffered). Now reads a single raw keypress via termios/tty on POSIX, in cbreak mode (raw mode was tried first and produced doubled/ misaligned output on a real terminal - cbreak only disables line buffering, leaving echo and output post-processing alone). A non-TTY stdin still refuses to proceed, same fail-safe as above - note `echo y | mr.sip.py ...` no longer auto-confirms, since a pipe isn't a real TTY.
… (1.3.0) Adds a centralized theme.py driving level-colored console logging (with a new FOUND level replacing the old "[+]" convention), a themed block-style banner, boxed run summaries, a TqdmLoggingHandler so logs stop corrupting the live progress bar, and a SIP-ENUM pre-check that warns when a target looks like it's blanket-rejecting every request (e.g. modern PJSIP). Consolidates wordlist/ip-list reading, the bulk-confirmation prompt, and IP-octet validation into single shared implementations across nes.py/enum.py/das.py, caches .message template reads, and fixes a second round of real bugs found via a from-scratch full-source audit: a printResult crash on malformed SIP responses, a --tc=0 hang, an ENUM single-username off-by-one, an implicit logger.found() import-order dependency, DAS's -m/-s validation gaps and promiscuous-mode restoration on unexpected exceptions, and (for real this time) the tty.setraw()/setcbreak() y/n-prompt bug that an earlier amend of this same commit had silently reintroduced after it was already fixed and pushed. 122 tests, ruff clean. See CHANGELOG.md for the full breakdown.
n0connect
changed the title
…(1.3.1 + 1.4.0) Property-based fuzzing (hypothesis) against every function that parses untrusted input found and fixed 5 real crash bugs: check_ip_address's CIDR split, fill_packet_data's surrogate handling, console/file logging encode errors, read_lines' UnicodeDecodeError, and --dp's missing port range check. On top of that, requested directly by the operator using the tool day-to-day: -c 0 now floods indefinitely (hping3/nping convention, instead of the previous degenerate 0-packet no-op), a new --pps flag throttles the flood to a chosen rate, and Ctrl+C no longer throws away the run summary in either SIP-DAS's flood loop or threadpool's shared worker pool (SIP-NES/SIP-ENUM) - previously the most common way to stop a run also discarded the one thing the instrumentation exists to show. 141 tests total (was 128), ruff-clean.
Leftover from local live-testing against the QEMU lab VM, not part of the curated example data this file is meant to hold.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this PR
This repo hasn't had a commit since 2020-12-12 (~5.6 years). This PR is a
modular refactor plus several rounds of correctness/security/reliability
hardening on top of v1.1.0, done as part of an internal security-tooling
internship project. Full technical writeup for every fix — RFC citation
where applicable, root cause, live-target evidence — is in CHANGELOG.md
(
1.2.0,1.3.0,1.3.1, and1.4.0sections).No CLI behavior is removed and no existing flag changes meaning. Three
new flags were added (
--mtu,--version,--pps). Below is a per-modulesummary of what actually changed observably, so the scope is clear without
reading the full changelog.
Functional impact by module
python3 mr.sip.py ...)pip installstep.--mtu/--version/--ppsadded, nothing removed.--tc/--thread-countand--dp/--destination-portnow reject invalid values (0/negative, out-of-range port) at parse time instead of hanging the run forever or crashing deep inside a scan with a raw traceback.--mt=register/subscribesent an invalid Request-URI (RFC 3261 §10.2/§10.3), silently dropped by the target — now correct. F20: default--from/--towordlists caused up to 81M queued requests per host on the tool's own simplest documented invocation — now sends one probe per target unless you explicitly pass a wordlist (explicit wordlists keep the original full cross-product behavior). A crash on single-host + multi-entry wordlist (UnboundLocalError) is also fixed. Ctrl+C during a bulk scan now still prints the "N live IP address(es) found" summary for whatever completed before the interrupt, instead of throwing it away.bind(("0.0.0.0", 0))+getsockname()) instead of a pre-picked one, eliminatingAddress already in usefailures under high-frequency flood runs. F14:-mmanual-spoof mode couldn't parse its own documentedip_list.txtinput. F21: an unclosed<in theFrom:header ofbye/cancel/sp-invitetemplates (present since the 2017 initial commit) made requests invalid per RFC 3261 §25.1 — confirmed via target-side PJSIP syntax-error logs, now fixed.-mwithout--il, and-son an interface with no netmask, now fail with a clean error instead of a raw traceback or a silent no-op; promiscuous mode is now guaranteed to be turned back off (try/finally) even if an unrelated exception interrupts the run. New:--mtufragmentation flag;-c 0now means flood indefinitely (hping3/npingconvention — previously a degenerate no-op that sent zero packets); new--ppsflag throttles the flood to a chosen packets/sec rate instead of always sending as fast as possible; Ctrl+C mid-flood now still prints the final summary panel (packets/sec, sent/failed counts) for whatever was sent before the interrupt, instead of discarding it.[ INFO ]/[ FOUND ]/[ WARN ]/[ ERROR ], auto-disabled whenNO_COLORis set or output isn't a real terminal), each module's final result is a bordered summary panel, and logs are routed throughtqdm.write()so they no longer corrupt the live progress bar's line. The y/n confirmation prompt's single-keypress reader had a real bug (terminal garbling fromtty.setraw(), despite an earlier commit's message claiming it was already fixed totty.setcbreak()) — now genuinely fixed and verified over a real controlling terminal. Console/file logging can no longer itself crash on an unencodable character (e.g. a lone Unicode surrogate from invalid-UTF-8 CLI input) — now gracefully escaped instead of raising.output/ip_list.txt,output/from.txt)output/.src/package instead of three flat scripts, but it's still invoked the exact same way, still nopip installstep — matches the existing free/open-source pentest-tool convention.Everything else (concurrency fixes F2/F4, the CRLF-injection fix, F7-F9,
F11, F12, F15-F19, a
printResult()crash on malformed SIP responses, animplicit
logger.found()import-order dependency, a round of DRYconsolidation across
nes.py/enum.py/das.py, and a dedicatedproperty-based fuzz-testing pass that found and fixed 5 further
input-parsing crash bugs — CIDR parsing, SIP packet templating on
surrogate/invalid-UTF-8 input, and wordlist file decoding) is either
invisible-to-the-user reliability work or internal cleanup — see
CHANGELOG.md for the complete list.
Long-term maintainability
covering every deterministic piece of logic — IP/CIDR validation, SIP
packet templating (incl. the CRLF-injection fix), CLI parsing, the thread
pool, colored-logging/theme behavior, DAS flood-control behavior
(
-c 0,--pps, Ctrl+C summaries), and a static structural check onevery
.messagetemplate that independently catches F21.hypothesis,tests/test_fuzz.py) against every function that parses untrustedinput — CLI argparse validators, SIP packet templating, SIP response
parsing, and wordlist file reading — found and fixed 5 real crash bugs;
see CHANGELOG.md's
1.3.1section for the full list.ruff+pytest+--helpsmoke test) acrossPython 3.9/3.11/3.13.
rufflint pass: zero unused-import/unused-variable/undefined-namefindings across the whole tree; fixed real exception-chaining issues
plus cosmetic modernization, across every round.
review) were done on the
1.3.0pass specifically to catch anything thefirst pass missed — see CHANGELOG.md's
1.3.0section for the completelist of what each round found and fixed.
Test plan
pytest— 141 passedruff check src/ tests/ mr.sip.py— cleanpython3 mr.sip.py --help— smoke testa QEMU/Trixbox CE (classic chan_sip) lab — see CHANGELOG.md's Testing
section for the full methodology and a scenario-by-scenario
baseline-vs-current comparison table (commit
bdd98advs. thisbranch, same live target).
burst against the Docker target — zero bind failures.
controlling terminal (
setsid/TIOCSCTTY) — a single keypress nowrenders correctly with no doubling.
vulnerability highlight against the Docker target.
hypothesis, 200-3000 examples perfunction) against every input-parsing function, plus live
extreme-input testing against both lab targets — see CHANGELOG.md's
1.3.1section.-c 0(infinite flood),--pps(rate throttling tothe exact requested packets/sec), and Ctrl+C mid-flood still
printing the summary panel before exiting — all against a real
socket send.
See CHANGELOG.md for the complete fix list (F1-F21 and the
1.3.0/1.3.1/1.4.0follow-up rounds), RFC citations, and reproductionnotes.