Skip to content

Commit 9537e1d

Browse files
committed
chore: release v0.2.0 Performance & Precision Update
1 parent 4fef6eb commit 9537e1d

12 files changed

Lines changed: 804 additions & 596 deletions

File tree

Cargo.lock

Lines changed: 184 additions & 285 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "rustywoof"
3-
version = "0.1.9"
3+
version = "0.2.0"
44
rust-version = "1.94"
55
edition = "2024"
66
description = "A blazing fast, relentless secret scanner & supply chain defensive tool. Written in Rust"
@@ -70,13 +70,15 @@ serde_json = "1.0.149"
7070
# For fast substring searching in large text data
7171
aho-corasick = "1.1.4"
7272

73-
# For parsing and writing YAML files (like configuration or manifest files)
74-
# Note: Changed `serde-yaml` to `serde-yml`
75-
serde_yml = "0.0.12"
76-
7773
# For self-update functionality
7874
self_update = { version = "0.44.0", features = ["archive-tar", "archive-zip"] }
7975

76+
# For blazing fast, zero-copy memory-mapped file reading
77+
memmap2 = "0.9.10"
78+
79+
# For serializing and deserializing Rust data structures
80+
serde_norway = "0.9.42"
81+
8082
[profile.release]
8183
# Maximum performance optimization
8284
opt-level = 3

Changelog.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Release v0.2.0 - Performance & Precision Update - 04/06/2026
2+
3+
Rustywoof `v0.2.0` is a massive architectural upgrade focused on enterprise-grade performance, lockfile auditing accuracy, and improved developer experience. We've replaced heavy heap allocations with zero-copy memory mapping, moved to a lock-free concurrency model, and hardened our supply chain parsers.
4+
5+
## Features & Developer Experience
6+
* **Configurable Cryptographic Sensitivity:** Added `min_entropy` to `.woof.toml`. Teams can now tune the Shannon entropy threshold to silence false positives (like dummy test tokens) while maintaining strict guards for production secrets.
7+
* **Robust `requirements.txt` Parsing:** Replaced basic string splitting with a resilient Regex extractor. The engine now flawlessly handles environment markers, version ranges (`>=`, `~=`), and inline comments.
8+
* **Graceful Background Updates:** Resolved the "Ghost Update" race condition. The background self-updater now synchronizes properly with the CLI execution thread, ensuring developers are reliably notified of new engine updates without hanging the terminal.
9+
10+
## Performance Optimizations
11+
* **Zero-Copy Memory Mapping:** Replaced `fs::read_to_string` with OS-level memory mapping (`memmap2`). The Aho-Corasick automaton now scans raw `&[u8]` byte slices directly, drastically reducing RAM allocation and CPU overhead on large repositories.
12+
* **Lock-Free Concurrency:** Eliminated the Mutex bottleneck (`Arc<Mutex<Vec>>`) that throttled multi-threaded execution. Findings are now transmitted instantly via a multi-producer single-consumer (`mpsc`) channel, vastly improving parallel scan times.
13+
* **Binary Fast-Fail Heuristic:** The scanner now inspects the first 512 bytes of a file for null (`\0`) characters, instantly skipping compiled binaries and assets before attempting expensive UTF-8 validation.
14+
* **Structured Deserialization:** Migrated lockfile parsing away from arbitrary DOM trees (`serde_json::Value`). By using strict Rust structs and ignoring unused fields, `package-lock.json` and `pnpm-lock.yaml` parsing times and memory footprint have been slashed.
15+
16+
## Bug Fixes
17+
* **Git Hook Pathing Lockout:** Fixed a critical issue where the generated pre-commit hook would permanently block Git commits if the user did not have `woof` installed globally in their `PATH`. The hook now features a graceful fallback.
18+
* **Directory Exclusion Traversals:** Corrected the implementation of the `ignore` crate. The scanner now correctly respects string paths defined in `.woof.toml` via `OverrideBuilder`, preventing unbounded traversals into `node_modules/` or `target/`.
19+
* **Yarn Audit Blindspot:** Patched an audit/remediate mismatch. The engine now correctly parses `yarn.lock` during the OSV threat intelligence sweep.
20+
* **pnpm Peer Dependency Parsing:** Fixed a string-splitting bug in the `pnpm-lock.yaml` parser that incorrectly captured peer dependencies as part of the SemVer string (e.g., `18.2.0(react@18.2.0)`), which previously caused OSV API query rejections.

README.md

Lines changed: 109 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,74 +1,158 @@
1-
# 🐕 Rustywoof (woof)
1+
# Rustywoof
22

3-
> **Enterprise-grade secret scanner and supply chain watchdog.**
3+
[![Version](https://img.shields.io/badge/version-v0.2.0-blue.svg)](https://github.com/ianramy/rustywoof/releases)
4+
[![Language](https://img.shields.io/badge/language-Rust-orange.svg)](https://www.rust-lang.org/)
5+
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
6+
[![Category](https://img.shields.io/badge/category-Cybersecurity-red.svg)]()
7+
[![Integration](https://img.shields.io/badge/integration-OSV_Database-purple.svg)](https://osv.dev/)
8+
[![Status](https://img.shields.io/badge/status-Open_Source-brightgreen.svg)]()
49

5-
Rustywoof is a blazing-fast, memory-safe CLI tool designed to detect exposed cryptographic credentials and compromised dependencies before they breach your perimeter. Built in Rust, it utilizes an `O(n)` Aho-Corasick automaton to scan thousands of files in milliseconds without crashing your CI/CD pipelines.
10+
**Enterprise-grade perimeter defense, secret scanner, and supply chain watchdog.**
611

7-
## ✨ Features
12+
Rustywoof is a high-performance, memory-safe command-line engine designed to detect exposed cryptographic credentials and compromised dependencies before they breach your system perimeter. Built entirely in Rust, it is engineered for strict enterprise CI/CD environments, utilizing zero-copy memory mapping and an O(n) Aho-Corasick automaton to analyze thousands of files in milliseconds without exhausting system resources.
813

9-
- **High-Speed Regex Pre-filtering:** Uses Aho-Corasick for simultaneous, multi-pattern literal prefix matching.
10-
- **OOM-Immune Engine:** Strict file-size constraints and binary bypassing ensure stability on massive monorepos.
11-
- **Live Threat Intelligence:** Queries the Open Source Vulnerability (OSV) API using JSON batching to audit Node, Rust, and Python lockfiles in under 500ms.
12-
- **Proactive Perimeter Defense:** Automatically detects un-tracked `.env` files and manages Git `pre-commit` hooks.
13-
- **Mathematical Context:** Calculates the Shannon Entropy of leaked credentials to reduce false positives.
14+
---
1415

15-
## 🚀 Installation
16+
## Security & Trust Posture
17+
18+
We recognize that running a security binary locally requires absolute trust. Rustywoof is built with strict operational transparency:
19+
20+
* **No Telemetry or Exfiltration:** The engine does not track user data, usage analytics, or exfiltrate source code.
21+
22+
* **Local Execution:** Secret scanning operations (Aho-Corasick pre-filtering and Regex validation) are executed entirely on your local machine.
23+
24+
* **Verifiable Outbound Traffic:** The only outbound network requests made by Rustywoof are standard HTTPS POST requests directed exclusively to [https://api.osv.dev](https://api.osv.dev) (Google's Open Source Vulnerability database) during the audit or patrol commands.
25+
26+
* **Open Source Verifiability:** The complete source code is available for independent audit. We encourage security researchers and engineers to compile the binary directly from the source.
27+
28+
---
29+
30+
## Core Capabilities
31+
32+
### 1. High-Performance Secret Detection
33+
34+
* **Aho-Corasick Automaton:** Utilizes simultaneous, multi-pattern literal prefix matching to filter text in O(n) time before invoking expensive Regex evaluations.
35+
36+
* **Zero-Copy Architecture:** Employs OS-level memory mapping (mmap) to read files directly as byte slices, drastically reducing RAM allocation.
37+
38+
* **Binary Fast-Fail Heuristic:** Instantly identifies and bypasses compiled binaries by evaluating the initial byte headers, preventing Out-Of-Memory (OOM) crashes on large artifacts.
39+
40+
* **Cryptographic Context:** Calculates the Shannon Entropy of matched strings to confidently differentiate between genuine cryptographic keys and high-entropy plaintext, heavily reducing false positives.
41+
42+
### 2. Supply Chain Threat Intelligence
43+
44+
* **OSV Batch Querying:** Parses multiple lockfiles (Cargo, npm, pnpm, yarn, Poetry, pip) and audits them against the Open Source Vulnerability (OSV) database. (Note: This specific feature requires an active internet connection to query the upstream database).
45+
* **Local Threat Caching:** To prevent network bottlenecks during repeated CI/CD runs, Rustywoof caches OSV API responses locally with a 12-hour Time-To-Live (TTL).
46+
* **Automated Remediation:** Capable of spawning shell processes to force native package managers to update a compromised dependency to a secure target version.
47+
48+
### 3. Proactive Perimeter Defense
49+
50+
* **Staged-Only Git Hooks:** Manages pre-commit hooks that strictly scan newly staged files (git diff --cached), ensuring rapid execution without scanning the entire repository.
51+
* **Environment Guarding:** Automatically detects untracked .env files and interactively deploys strict .gitignore configurations to prevent accidental credential leakage.
52+
53+
---
54+
55+
## Installation
1656

1757
### Option 1: Quick Install (macOS / Linux)
1858

59+
To install Rustywoof on macOS or Linux, run the following command in your terminal:
60+
1961
```bash
20-
curl --proto '=https' --tlsv1.2 -LsSf [https://github.com/ianramy/rustywoof/releases/latest/download/installer.sh](https://github.com/ianramy/rustywoof/releases/latest/download/installer.sh) | sh
62+
curl -sSL https://ianramy.co.ke/rustywoof/installer.sh | sh
2163
```
2264

23-
### Option 2: Via Cargo (Rust developers)
65+
### Option 2: Windows (PowerShell)
66+
67+
To install Rustywoof on Windows, run the following command in your PowerShell terminal:
68+
69+
```bash
70+
irm https://ianramy.co.ke/rustywoof/installer.ps1 | iex
71+
```
72+
73+
### Option 3: Compile from Source (Via Cargo)
74+
75+
To compile the engine from source, use Cargo, Rust's package manager.
2476

2577
```bash
2678
cargo install rustywoof
79+
```
2780

28-
# or
81+
> [!NOTE]
82+
> For the highest level of trust, use `cargo binstall` to compile the engine from source.
2983
84+
```bash
3085
cargo binstall rustywoof
3186
```
3287

33-
### Option 3: Windows (PowerShell)
88+
89+
### Option 4: Node.js Package Managers
90+
91+
To install Rustywoof via Node.js package managers, use the following commands:
3492

3593
```bash
36-
irm [https://github.com/ianramy/rustywoof/releases/latest/download/installer.ps1](https://github.com/ianramy/rustywoof/releases/latest/download/installer.ps1) | iex
94+
npm install -g @ianramy/rustywoof
3795
```
3896

39-
## 🛠️ Usage
97+
```bash
98+
pnpm install -g @ianramy/rustywoof
99+
```
40100

41-
Rustywoof is designed to be simple and devastatingly effective.
101+
```bash
102+
yarn global add @ianramy/rustywoof
103+
```
42104

43105
```bash
44-
# Execute a full perimeter sweep (Code + Dependencies)
106+
bun install -g @ianramy/rustywoof
107+
```
108+
109+
---
110+
111+
## Usage Guide
112+
113+
Rustywoof is designed to provide maximum visibility with minimal configuration.
114+
115+
```bash
116+
# Execute a full perimeter sweep (Code Secrets + Dependency Threats)
45117
woof patrol
46118

47-
# Scan a specific directory for secrets
119+
# Scan a specific directory or file for exposed secrets
48120
woof scan ./src
49121

50-
# Audit lockfiles against the OSV vulnerability database
122+
# Audit project lockfiles against the OSV vulnerability database
51123
woof audit
52124

53125
# Force a package manager to remediate a compromised dependency
54126
woof remediate axios 1.7.4
55127

56-
# Deploy the pre-commit guard to block leaked secrets
128+
# Deploy the Watchdog pre-commit guard to block leaked secrets natively in Git
57129
woof hook install
58130
```
59131

60-
## ⚙️ Configuration
132+
---
133+
134+
## Configuration
61135

62-
Run `woof init` to generate a `.woof.toml` file in your project root. You can add custom domain-specific regex rules and ignore paths here.
136+
Run `woof init` to generate a `.woof.toml` configuration file in your project root. This allows teams to define domain-specific rules, suppress false positives via entropy tuning, and manage scan perimeters.
63137

64138
```toml
139+
# Directories to bypass during sweeps
65140
ignore_paths = ["tests/", "node_modules/", "target/"]
66141

142+
# Adjust the Shannon entropy threshold (lower values increase sensitivity)
143+
min_entropy = 3.0
144+
145+
# Define proprietary token formats
67146
[[custom_rules]]
68147
name = "Internal API Token"
69148
pattern = "api_prod_[a-zA-Z0-9]{32}"
70149
```
71150

72-
## 🛡️ License
151+
> [!NOTE]
152+
> Developers can also use inline suppression directives like `// woof:ignore-next-line` directly in their source code to bypass specific flagged strings.
153+
154+
---
155+
156+
## License
73157

74-
This project is licensed under the [MIT License](LICENSE) - see the LICENSE file for details.
158+
This software is distributed under the [MIT License](LICENSE). See the LICENSE file in the repository for full details.

src/config.rs

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,17 @@
11
// src/config.rs
22

33
use crate::error::SystemError;
4-
use miette::{Result, WrapErr, IntoDiagnostic};
4+
use miette::{IntoDiagnostic, Result, WrapErr};
55
use serde::{Deserialize, Serialize};
66
use std::fs;
77
use std::path::Path;
88

9-
#[derive(Serialize, Deserialize, Default, Debug)]
9+
#[derive(Serialize, Deserialize, Debug)]
1010
pub struct Config {
1111
pub ignore_paths: Vec<String>,
1212
pub custom_rules: Vec<CustomRule>,
13+
#[serde(default = "default_entropy")]
14+
pub min_entropy: f32,
1315
}
1416

1517
#[derive(Serialize, Deserialize, Debug)]
@@ -18,6 +20,20 @@ pub struct CustomRule {
1820
pub pattern: String,
1921
}
2022

23+
fn default_entropy() -> f32 {
24+
3.0
25+
}
26+
27+
impl Default for Config {
28+
fn default() -> Self {
29+
Config {
30+
ignore_paths: vec![],
31+
custom_rules: vec![],
32+
min_entropy: default_entropy(),
33+
}
34+
}
35+
}
36+
2137
pub fn load_config() -> Result<Config> {
2238
let config_path = Path::new(".woof.toml");
2339

@@ -40,9 +56,11 @@ pub fn init_config() -> Result<()> {
4056
"target/".to_string(),
4157
],
4258
custom_rules: vec![],
59+
min_entropy: 3.0,
4360
};
4461

45-
let toml_content = toml::to_string_pretty(&config).map_err(SystemError::ConfigSerializeFailed)?;
62+
let toml_content =
63+
toml::to_string_pretty(&config).map_err(SystemError::ConfigSerializeFailed)?;
4664

4765
fs::write(".woof.toml", toml_content)
4866
.map_err(SystemError::ConfigWriteFailed)

src/git.rs

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,14 @@ use std::path::Path;
99

1010
const PRE_COMMIT_HOOK: &str = r#"#!/bin/sh
1111
# Watchdog Perimeter Defense Hook
12+
13+
# [FIX] Check if woof is in the PATH before enforcing the hook
14+
if ! command -v woof >/dev/null 2>&1; then
15+
echo "[WARN] Watchdog (woof) is not installed or not in PATH."
16+
echo "[WARN] Skipping perimeter evaluation. Please install Rustywoof to secure commits."
17+
exit 0
18+
fi
19+
1220
echo "[INFO] Watchdog evaluating commit perimeter..."
1321
woof check .
1422
if [ $? -ne 0 ]; then

src/main.rs

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
use clap::{Parser, Subcommand};
44
use miette::Result;
55
use std::process;
6+
use std::time::Duration;
67

78
mod config;
89
mod detector;
@@ -161,7 +162,7 @@ fn main() -> Result<()> {
161162

162163
// 2. Check if the background thread found an update
163164
if !matches!(cli.command, Commands::Update) {
164-
if let Ok(Some(new_version)) = update_receiver.try_recv() {
165+
if let Ok(Some(new_version)) = update_receiver.recv_timeout(Duration::from_millis(150)) {
165166
println!(
166167
"\n\x1b[33m[NOTICE]\x1b[0m A new engine update (v{}) is available! Run `woof update` to update.",
167168
new_version

0 commit comments

Comments
 (0)