Add input validation and security hardening

[dandye]

Summary

LogStory currently lacks comprehensive input validation and security measures. As the project matures, we need to implement proper security practices to protect against malicious inputs and ensure safe operation.

Current Security Gaps

• No input validation: User-provided regex patterns and configurations aren't validated
• Regex injection risks: Malicious patterns could cause ReDoS attacks
• No secrets scanning: Credentials might accidentally be committed
• Error information leakage: Errors might expose sensitive system information
• No rate limiting: No protection against resource exhaustion

Proposed Security Improvements

Phase 1: Input Validation

• Add validation for regex patterns (complexity, safety checks)
• Validate YAML configuration structure before processing
• Sanitize file paths to prevent directory traversal
• Add input length limits and character restrictions
• Validate timestamp format strings for safety

Phase 2: Regex Security

• Implement regex complexity analysis to prevent ReDoS
• Add timeout mechanisms for regex operations
• Create allowlist of safe regex patterns
• Add warnings for potentially dangerous regex constructs

Phase 3: Secrets Protection

• Add secrets scanning to CI/CD pipeline using detect-secrets
• Create .gitignore rules for common credential files
• Implement secure credential handling guidelines
• Add runtime checks to prevent logging sensitive data

Phase 4: Error Handling Security

• Implement secure error messages that don't leak system info
• Add structured logging with security event tracking
• Create sanitized error responses for API interactions
• Implement proper exception handling hierarchy

Phase 5: Operational Security

• Add resource usage limits (memory, processing time)
• Implement file size limits for log processing
• Add audit logging for sensitive operations
• Create security policy documentation (SECURITY.md)

Example Security Validations

def validate_regex_pattern(pattern: str) -> bool:
    \"\"\"Validate regex pattern for safety and complexity.\"\"\"
    # Check for catastrophic backtracking patterns
    dangerous_patterns = [
        r'(.*?)*',  # Nested quantifiers
        r'(a+)+',   # Exponential complexity
        r'(a|a)*',  # Alternation with redundancy
    ]
    
    # Complexity limits
    if len(pattern) > MAX_PATTERN_LENGTH:
        raise ValueError(\"Regex pattern too long\")
    
    # Test compilation and basic safety
    try:
        compiled = re.compile(pattern)
        # Test with timeout
        signal.alarm(REGEX_TIMEOUT)
        compiled.search(\"test\" * 1000)
        signal.alarm(0)
    except re.error as e:
        raise ValueError(f\"Invalid regex pattern: {e}\")
    
    return True

Security Testing

• Add security-focused test cases for all validation functions
• Create fuzzing tests for input parsing
• Test with malicious regex patterns and inputs
• Validate error handling doesn't leak information
• Test resource exhaustion scenarios

Benefits

• Protection from attacks: Prevent ReDoS, injection, and other attacks
• Secure by default: Safe operation even with untrusted inputs
• Audit compliance: Proper logging and security controls
• User confidence: Professional security practices build trust
• Incident prevention: Catch security issues before they become problems

Acceptance Criteria

• All user inputs are validated before processing
• Regex patterns are analyzed for safety
• No credentials can be accidentally committed
• Error messages don't leak sensitive information
• Security policy is documented and followed
• CI pipeline includes security scanning

🤖 Generated with Claude Code