Changelog

Added: New features
Changed: Changes in existing functionality
Deprecated: Features that will be removed in future versions
Removed: Removed features
Fixed: Bug fixes
Security: Security improvements

All notable changes to the AI News WebCrawler project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

Initial release of AI News WebCrawler
Multi-source scraping support (RSS, Reddit, API)
Tiered source organization (Tier 1-3)
Intelligent scoring algorithm (0-10 scale)
Automated scheduling system
SQLite database for article storage
Duplicate detection using URL normalization and title similarity
Daily report generation (HTML and JSON)
CLI interface with rich output support
Comprehensive logging system
Robots.txt compliance
Rate limiting and retry logic
User-Agent rotation
Support for 15 news sources:
- Tier 1: The Verge, TechCrunch, Ars Technica
- Tier 2: Wired, MIT Technology Review, The Guardian, Reuters, BBC
- Tier 3: Hacker News, Reddit (r/MachineLearning, r/artificial, r/singularity), IEEE Spectrum, VentureBeat, ZDNet
Scrapers for: TechCrunch, VentureBeat, The Verge, AI News, Hacker News
Keyword-based filtering (60+ high-value keywords, 40+ medium-value keywords)
Scoring components:
- Keyword matching (0-3.0 points)
- Source authority (0-2.0 points)
- Engagement metrics (0-2.0 points)
- Recency scoring (0-1.5 points)
- Title quality analysis (0-1.5 points)
Documentation:
- Comprehensive README.md
- Architecture documentation
- API reference
- Guide for adding new sources
- Changelog

For detailed information about changes, see commit history or release notes.