Add URL security scanning to prevent inappropriate content#13
Open
Pjv93 wants to merge 45 commits intoaws-samples:mainfrom
Open
Add URL security scanning to prevent inappropriate content#13Pjv93 wants to merge 45 commits intoaws-samples:mainfrom
Pjv93 wants to merge 45 commits intoaws-samples:mainfrom
Conversation
- Automatically reverts commits containing inappropriate URLs - Creates descriptive revert commit message - Enhanced Slack notifications for revert actions - Provides immediate protection against malicious content
- Detects http://, https://, protocol-relative (//), and domain-only URLs - Scans all file types (not just .md/.html) for comprehensive coverage - Parallel processing for improved performance with 140+ repositories - Smart filtering to avoid false positives - Maintains auto-revert and Slack notification functionality
- Two-layer security: Google Safe Browsing + content analysis - Detects malware, phishing, and inappropriate content - Fast batch checking with Google's threat database - Fallback to content analysis for non-malware violations - Maintains auto-revert and Slack notification functionality - Free tier: 10,000 requests/day (perfect for 140+ repositories)
- Remove incorrect -m flag usage - Use --message flag for proper commit message formatting - Ensures auto-revert functionality works correctly
- Check URLs for inappropriate keywords before attempting to access content - Blocks URLs like 'badsite.com/porn-content' immediately - Maintains educational context filtering - Provides faster detection without network requests
- Use single-line commit message to avoid git parsing issues - Maintains essential information about security violation - Ensures auto-revert functionality works properly
✅ URL pattern detection: Blocks inappropriate keywords in URLs ✅ Content analysis: Scans accessible URL content ✅ Google Safe Browsing: API integration ready ✅ Auto-revert: Automatically removes malicious commits ✅ Slack notifications: Alerts team of security violations ✅ Comprehensive URL extraction: Handles all URL formats
- Show detailed logs of what content is being retrieved - Increase content analysis from 5KB to 50KB - Display response status, content length, title, and preview - Better error handling - don't mark failed requests as clean - More verbose output to debug security scanning issues
- Use simple -m flag instead of --message - Avoid special characters that cause git parsing issues - Ensures auto-revert works reliably
- Remove custom message flags that cause parsing issues - Use git's default revert message format - Should fix auto-revert functionality
This reverts commit 0b02451.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds automated URL content scanning with automatic commit reverting to detect and block inappropriate links that could be added via URL hijacking or malicious commits.
Problem Solved
Features Added
How Auto-Revert Works
Testing
Security Benefits
This ensures all modernization workshops created from this template have automatic security protection with immediate remediation of inappropriate content.