Infosec News Reader

A comprehensive RSS feed aggregator and news reader specifically designed for cybersecurity professionals. This application automatically fetches, analyzes, and summarizes security news from leading infosec sources, providing AI-powered relevance scoring and threat analysis.

🚀 Features

Core Functionality

• RSS Feed Aggregation: Automatically fetches news from 15+ leading infosec sources
• AI-Powered Analysis: Uses OpenAI to generate summaries and threat analysis
• Relevance Scoring: Intelligent filtering based on cybersecurity keywords
• Real-time Updates: Scheduled feed updates every 30 minutes (configurable)
• Modern Web Interface: Responsive, intuitive dashboard

News Sources

• The Hacker News
• SecurityWeek
• ThreatPost
• Dark Reading
• Bleeping Computer
• KrebsOnSecurity
• Schneier on Security
• CSO Online
• SC Magazine
• Infosecurity Magazine
• Help Net Security
• Security Affairs
• The Record
• CyberScoop
• Cyber Security Hub

Advanced Features

• Smart Filtering: Filter by source, relevance score, and date
• Search Functionality: Full-text search across all articles
• Threat Analysis: AI-powered severity assessment and recommendations
• Keyword Management: Customizable relevance keywords
• Database Management: Cleanup old articles and manage storage
• Feed Health Monitoring: Test and monitor RSS feed accessibility

📋 Prerequisites

• Python 3.8 or higher
• OpenAI API key (optional, for AI features)
• Modern web browser

🛠️ Installation

1. Clone the repository

   git clone <repository-url>
   cd Infosec_New_Reader

2. Create a virtual environment

   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install dependencies

   pip install -r requirements.txt

4. Set up environment variables Create a .env file in the project root:

   # OpenAI API Configuration (optional)
   OPENAI_API_KEY=your_openai_api_key_here
   
   # Flask Configuration
   FLASK_SECRET_KEY=your_secret_key_here
   FLASK_DEBUG=True

5. Initialize the database The database will be automatically created when you first run the application.

🚀 Usage

Starting the Application

1. Activate the virtual environment

   source venv/bin/activate  # On Windows: venv\Scripts\activate

2. Run the application

   python app.py

3. Access the web interface Open your browser and navigate to http://localhost:5000

First Run

On first run, the application will:

• Create the SQLite database
• Perform an initial RSS feed fetch
• Start the scheduled update service

Manual Feed Update

You can manually trigger RSS feed updates by:

• Clicking the "Update Feeds" button in the navigation
• Using the API endpoint: POST /api/update

📊 Dashboard Overview

Home Page (/)

• Quick statistics overview
• Latest news preview
• Quick action buttons
• Popular keywords display

Dashboard (/dashboard)

• Comprehensive news feed with filtering
• Relevance score indicators
• Source filtering
• Article details modal

Search (/search)

• Full-text search across all articles
• Advanced filtering options
• Search result highlighting
• Search tips and suggestions

Sources (/sources)

• RSS feed status monitoring
• Feed health testing
• Source-specific article viewing
• Feed accessibility reports

Settings (/settings)

• Application configuration
• Database management
• Keyword customization
• API configuration

🔧 Configuration

RSS Feeds

Edit config.py to add or modify RSS feeds:

RSS_FEEDS = {
    "Source Name": "https://source.com/rss/feed",
    # Add more feeds here
}

Relevance Keywords

Customize keywords in config.py:

RELEVANCE_KEYWORDS = [
    "vulnerability", "exploit", "breach",
    # Add your custom keywords
]

Update Intervals

Modify the update frequency in config.py:

UPDATE_INTERVAL_MINUTES = 30  # Change as needed

🔌 API Endpoints

News Endpoints

• GET /api/news - Get latest news articles
• GET /api/search?q=<query> - Search articles
• GET /api/news?source=<source> - Get articles by source

Management Endpoints

• POST /api/update - Manual feed update
• GET /api/statistics - System statistics
• GET /api/sources - Available sources
• GET /api/feeds/status - Feed status
• GET /api/feeds/test - Test feed accessibility
• GET /api/keywords - Relevance keywords

🗄️ Database Schema

The application uses SQLite with the following tables:

Articles Table

• id: Primary key
• title: Article title
• link: Article URL
• description: Article description
• content: Full article content
• summary: AI-generated summary
• source: RSS feed source
• published_date: Publication date
• relevance_score: Calculated relevance (0-10)
• keywords_found: JSON array of matched keywords

Feeds Table

• id: Primary key
• name: Feed name
• url: RSS feed URL
• last_fetch: Last fetch timestamp
• status: Feed status (active/inactive)

🤖 AI Integration

OpenAI Features

• Article Summarization: Generate concise summaries
• Threat Analysis: Assess severity and provide recommendations
• Content Classification: Categorize articles by threat type

Fallback Mode

If OpenAI API is not configured, the application will:

• Use simple text extraction for summaries
• Calculate relevance based on keyword matching
• Provide basic threat categorization

🔒 Security Considerations

• API keys are stored in environment variables
• Database is local and not exposed externally
• Input validation on all API endpoints
• Rate limiting on RSS feed requests
• Secure headers and CORS configuration

🧪 Testing

Test RSS Feeds

python -c "from news_agent import news_agent; print(news_agent.test_feeds())"

Test Database

python -c "from database import NewsDatabase; db = NewsDatabase(); print(db.get_article_count())"

Manual Feed Update

python -c "from news_agent import news_agent; print(news_agent.manual_update())"

📈 Performance

Optimization Features

• Database indexing on frequently queried fields
• Pagination for large result sets
• Caching of feed status
• Efficient keyword matching
• Background processing for AI operations

Monitoring

• Feed health monitoring
• Database size tracking
• Update frequency monitoring
• Error logging and reporting

🐛 Troubleshooting

Common Issues

1. RSS Feeds Not Updating

   • Check internet connectivity
   • Verify RSS feed URLs are accessible
   • Check feed status in Sources page

2. AI Features Not Working

   • Verify OpenAI API key is set
   • Check API key permissions
   • Review error logs

3. Database Issues

   • Check file permissions for database directory
   • Verify SQLite is installed
   • Check available disk space

4. Web Interface Not Loading

   • Verify Flask is running
   • Check port 5000 is available
   • Review browser console for errors

Logs

Application logs are printed to the console. For production deployment, consider using a proper logging framework.

🔄 Updates and Maintenance

Regular Maintenance

• Monitor database size
• Clean up old articles periodically
• Update RSS feed URLs if needed
• Review and update relevance keywords

Backup

• Regular database backups recommended
• Export configuration files
• Backup custom keywords and settings

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• RSS feed sources for providing security news
• OpenAI for AI capabilities
• Bootstrap for the UI framework
• Python community for excellent libraries

📞 Support

For issues and questions:

1. Check the troubleshooting section
2. Review the logs
3. Create an issue on GitHub
4. Check the documentation

---

Note: This application is designed for educational and professional use. Always verify information from multiple sources and follow your organization's security policies.