-
Notifications
You must be signed in to change notification settings - Fork 60
[RUM-13802][CHORE] Add workflow for Github issue notifications to Slack #1120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
|
|
||
| name: Github Issue Notification | ||
|
|
||
| on: | ||
| # # When a new issue is opened | ||
| # issues: | ||
| # types: | ||
| # - opened | ||
|
|
||
| # When manually triggering the workflow | ||
| workflow_dispatch: | ||
| inputs: | ||
| issue_number: | ||
| description: 'Issue number to process' | ||
| required: true | ||
| type: number | ||
|
|
||
| jobs: | ||
| process_issue: | ||
| runs-on: ubuntu-latest | ||
| environment: protected-dev-env | ||
|
|
||
| steps: | ||
| - name: Checkout Repository | ||
| uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 | ||
|
|
||
| - name: Set up Python 3.x | ||
| uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0 | ||
| with: | ||
| python-version: '3.x' | ||
|
|
||
| - name: Install Dependencies | ||
| run: | | ||
| cd tools/issue_handler | ||
| pip install -r requirements.txt | ||
|
|
||
| - name: Set Issue Number | ||
| run: | | ||
| if [ "${{ github.event_name }}" = "issues" ]; then | ||
| echo "GITHUB_EVENT_NUMBER=${{ github.event.issue.number }}" >> $GITHUB_ENV | ||
| else | ||
| echo "GITHUB_EVENT_NUMBER=${{ github.event.inputs.issue_number }}" >> $GITHUB_ENV | ||
| fi | ||
|
|
||
| - name: Run the Processing Script | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| OPENAI_TOKEN: ${{ secrets.OPENAI_TOKEN }} | ||
| OPENAI_SYSTEM_PROMPT: ${{ vars.OPENAI_SYSTEM_PROMPT }} | ||
| SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }} | ||
| SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }} | ||
| GITHUB_REPOSITORY: ${{ github.repository }} | ||
| run: | | ||
| cd tools/issue_handler | ||
| # Add timeout to prevent hanging processes (5 minutes) | ||
| timeout 300 python -m src.analyze_issue $GITHUB_EVENT_NUMBER | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| # Python | ||
| __pycache__/ | ||
| *.pyc | ||
|
|
||
| # Virtual environment | ||
| venv/ | ||
|
|
||
| # Environment variables | ||
| .env | ||
| secrets.env |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,191 @@ | ||
| # GitHub Issue Handler | ||
|
|
||
| Automated GitHub issue analyzer that uses OpenAI to analyze new issues and posts summaries to Slack. | ||
|
|
||
| ## Features | ||
|
|
||
| - 🔍 Fetches GitHub issue details via API | ||
| - 🤖 Analyzes issues using OpenAI | ||
| - 💬 Posts analysis to Slack | ||
| - 🔄 Runs automatically on new issues via GitHub Actions | ||
| - 🛠️ Can be run manually for specific issues | ||
|
|
||
| ## Setup | ||
|
|
||
| ### 1. Create Virtual Environment | ||
|
|
||
| ```bash | ||
| # Create a virtual environment | ||
| python3 -m venv venv | ||
|
|
||
| # Activate the virtual environment | ||
| source venv/bin/activate | ||
| ``` | ||
|
|
||
| ### 2. Install Dependencies | ||
|
|
||
| ```bash | ||
| pip install -r requirements.txt | ||
| ``` | ||
|
|
||
| ### 3. Configure Environment | ||
|
|
||
| First, create your local environment file: | ||
| ```bash | ||
| # Create .env file from template | ||
| ./setup_env.sh | ||
| ``` | ||
|
|
||
| This creates a `.env` file that you will need to fill with the required tokens and optional configuration. The file includes: | ||
|
|
||
| **Required variables:** | ||
| - `GITHUB_TOKEN` - GitHub token with repo access | ||
| - `OPENAI_TOKEN` - OpenAI API token | ||
| - `OPENAI_SYSTEM_PROMPT` - Prompt for OpenAI to analyze issues | ||
| - `SLACK_WEBHOOK_URL` - Slack webhook URL for posting notifications | ||
| - `SLACK_CHANNEL_ID` - Slack channel ID | ||
| - `GITHUB_REPOSITORY` - Repository in format `owner/repo` | ||
|
|
||
| **Optional variables** (override defaults): | ||
| - `OPENAI_MODEL` - Model to use (default: `chatgpt-4o-latest`) | ||
| - `OPENAI_TEMPERATURE` - Response creativity 0.0-1.0 (default: `0.4`) | ||
| - `OPENAI_MAX_RESPONSE_TOKENS` - Max response length (default: `500`) | ||
|
|
||
| ## Usage | ||
|
|
||
| ### Run Manually | ||
|
|
||
| Activate your virtual environment and analyze a specific issue: | ||
| ```bash | ||
| python -m src.analyze_issue ISSUE_NUMBER | ||
| ``` | ||
|
|
||
| Example: | ||
| ```bash | ||
| python -m src.analyze_issue 1234 | ||
| ``` | ||
|
|
||
| ### GitHub Action | ||
|
|
||
| The tool runs: | ||
| 1. Automatically when a new issue is opened | ||
| 2. Manually via workflow dispatch with an issue number | ||
|
|
||
| **Required GitHub Secrets** (configured in protected environment): | ||
| - `OPENAI_TOKEN` - OpenAI API key | ||
| - `SLACK_WEBHOOK_URL` - Slack webhook URL | ||
| - `SLACK_CHANNEL_ID` - Slack channel ID (for reference, not currently used in code) | ||
|
|
||
| **Required GitHub Variables** (configured in protected environment): | ||
| - `OPENAI_SYSTEM_PROMPT` - OpenAI analysis prompt (stored as variable for easier updates) | ||
|
|
||
| **Optional GitHub Variables** (override defaults if needed): | ||
| - `OPENAI_MODEL` - Model to use (default: `chatgpt-4o-latest`) | ||
| - `OPENAI_TEMPERATURE` - Response creativity 0.0-1.0 (default: `0.4`) | ||
| - `OPENAI_MAX_RESPONSE_TOKENS` - Max response length (default: `500`) | ||
|
|
||
| **Automatically Provided**: | ||
| - `GITHUB_TOKEN` - Provided by GitHub Actions | ||
| - `GITHUB_REPOSITORY` - Repository name (e.g., `DataDog/dd-sdk-ios`) | ||
|
|
||
| ## Output | ||
|
|
||
| For each issue, the tool does the following: | ||
| 1. Analyze the issue using OpenAI | ||
| 2. Post a message to Slack containing: | ||
| - GitHub issue notification | ||
| - Analysis summary | ||
| - Suggested response | ||
| - Confidence level | ||
|
|
||
| ## Development | ||
|
|
||
| ### Project structure | ||
|
|
||
| - Source code is in `src/` | ||
| - Tests are in `tests/` | ||
| - Environment variables are managed via `.env` | ||
|
|
||
| ### Architecture | ||
|
|
||
| Main Components: | ||
| - analyze_issue.py - Main entry point that orchestrates the workflow | ||
| - github_handler.py - Fetches GitHub issue details via API | ||
| - openai_handler.py - Analyzes issues using OpenAI | ||
| - slack_handler.py - Posts notifications and analysis to Slack | ||
|
|
||
| ### Workflow | ||
|
|
||
| - GitHub issue is opened → triggers GitHub Action | ||
| - Fetches issue details from GitHub API | ||
| - Analyzes issue with OpenAI using a custom prompt | ||
| - Posts Github issue notification and analysis on Slack | ||
|
|
||
| ## Security | ||
|
|
||
| ### Protection Mechanisms | ||
|
|
||
| **Content Limits** | ||
| - GitHub issue content: Configurable limit (default 4,000 characters) | ||
| - OpenAI responses: Configurable token limit (default 500 tokens) | ||
| - Slack messages: Configurable character limits (default 2,000-3,000 characters) | ||
| - GitHub Action timeout: 5 minutes | ||
|
|
||
| **Input Sanitization** | ||
| - Removes HTML comments and system instructions | ||
| - Filters prompt injection attempts | ||
| - Validates issue numbers (must be integers) | ||
|
|
||
| **Output Sanitization** | ||
| - Removes markdown links and URLs from AI-generated content | ||
| - Strips HTML tags and suspicious patterns | ||
| - Filters script-like content before posting to Slack | ||
|
|
||
| **Dependencies** | ||
| - All Python dependencies pinned to exact versions | ||
| - Dependabot configured for automated security updates | ||
| - Third-party GitHub Actions pinned to commit SHAs | ||
|
|
||
| ### Best Practices | ||
|
|
||
| - Never commit `.env` files (git-ignored by default) | ||
| - Store tokens in GitHub Secrets | ||
| - Store prompts in GitHub Variables (for easier updates) | ||
| - Use protected environments for workflow execution | ||
|
|
||
| ## Running Tests | ||
|
|
||
| Make sure your virtual environment is activated before running tests. | ||
|
|
||
| ### Unit Tests | ||
|
|
||
| Run unit tests (no API calls required): | ||
|
|
||
| ```bash | ||
| pytest tests/ | ||
| ``` | ||
|
|
||
| Or using make: | ||
| ```bash | ||
| make issue-handler-test | ||
| ``` | ||
|
|
||
| ### Integration Tests | ||
|
|
||
| These tests make real API calls. Ensure your `.env` file is configured before running: | ||
|
|
||
| ```bash | ||
| # Test full workflow with a real issue | ||
| PYTHONPATH=. python integration_tests/test_analysis.py --issue 1234 | ||
|
|
||
| # Test GitHub API connectivity | ||
| PYTHONPATH=. python integration_tests/test_real_issue.py --issue 1 | ||
|
|
||
| # Test Slack webhook | ||
| PYTHONPATH=. python test_slack_webhook.py | ||
| ``` | ||
|
|
||
| Or use make: | ||
| ```bash | ||
| make issue-handler-integration-test | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| # ----------------------------------------------------------- | ||
| # Unless explicitly stated otherwise all files in this repository are licensed under the Apache License Version 2.0. | ||
| # This product includes software developed at Datadog (https://www.datadoghq.com/). | ||
| # Copyright 2019-Present Datadog, Inc. | ||
| # ----------------------------------------------------------- | ||
|
|
||
| """ | ||
| Test script for analyzing real GitHub issues. | ||
| """ | ||
| import os | ||
| import argparse | ||
| import sys | ||
| from pathlib import Path | ||
| from dotenv import load_dotenv | ||
|
|
||
| # Try to load environment variables from .env file | ||
| env_path = Path(__file__).parent.parent / '.env' | ||
| if env_path.exists(): | ||
| load_dotenv(env_path) | ||
|
|
||
| # Add src directory to Python path | ||
| src_dir = Path(__file__).parent.parent / "src" | ||
| sys.path.append(str(src_dir)) | ||
|
|
||
| from src.github_handler import create_github_handler | ||
| from src.openai_handler import create_openai_handler | ||
|
|
||
| def parse_args(): | ||
| parser = argparse.ArgumentParser(description='Test GitHub issue analysis') | ||
| parser.add_argument('--issue', type=int, required=True, | ||
| help='Issue number to analyze') | ||
| return parser.parse_args() | ||
|
|
||
| def main(): | ||
| args = parse_args() | ||
| try: | ||
| # First fetch the issue | ||
| github = create_github_handler() | ||
| issue = github.get_issue(args.issue) | ||
| if not issue: | ||
| print(f"\nIssue #{args.issue} not found") | ||
| return | ||
|
|
||
| print(f"\nAnalyzing issue #{args.issue}: {issue.title}") | ||
|
|
||
| # Then analyze it | ||
| openai = create_openai_handler() | ||
| analysis = openai.analyze_issue(issue) | ||
|
|
||
| # Print results | ||
| print("\nAnalysis Results:") | ||
| print(f"\nSummary:") | ||
| print(analysis.summary) | ||
|
|
||
| print(f"\nSuggested Response:") | ||
| print(analysis.suggested_response) | ||
|
|
||
| print(f"\nConfidence Level: {analysis.confidence_level}") | ||
|
|
||
| except Exception as e: | ||
| print(f"\nError: {str(e)}") | ||
|
|
||
| if __name__ == "__main__": | ||
| main() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| # ----------------------------------------------------------- | ||
| # Unless explicitly stated otherwise all files in this repository are licensed under the Apache License Version 2.0. | ||
| # This product includes software developed at Datadog (https://www.datadoghq.com/). | ||
| # Copyright 2019-Present Datadog, Inc. | ||
| # ----------------------------------------------------------- | ||
|
|
||
| """ | ||
| Integration test for processing actual GitHub issues. | ||
| """ | ||
| import argparse | ||
| import os | ||
| import sys | ||
| from pathlib import Path | ||
| from dotenv import load_dotenv | ||
| from src.github_handler import create_github_handler | ||
|
|
||
| # Try to load environment variables from .env file | ||
| env_path = Path(__file__).parent.parent / '.env' | ||
| if env_path.exists(): | ||
| load_dotenv(env_path) | ||
|
|
||
| # Add src directory to Python path | ||
| src_dir = Path(__file__).parent.parent / "src" | ||
| sys.path.append(str(src_dir)) | ||
|
|
||
| def parse_args(): | ||
| parser = argparse.ArgumentParser(description='Test GitHub issue fetching') | ||
| parser.add_argument('--issue', type=int, default=1, | ||
| help='Issue number to fetch (default: 1)') | ||
| return parser.parse_args() | ||
|
|
||
| def main(): | ||
| args = parse_args() | ||
| try: | ||
| if not os.environ.get("GITHUB_TOKEN"): | ||
| print("\nError: GITHUB_TOKEN environment variable is not set.") | ||
| print("Please set it using:") | ||
| print(" export GITHUB_TOKEN='your-token'") | ||
| return | ||
|
|
||
| # Create handler using environment variables (GITHUB_REPOSITORY should be set in .env) | ||
| handler = create_github_handler() | ||
|
|
||
| print(f"\nFetching issue #{args.issue} from {os.environ.get('GITHUB_REPOSITORY', 'unknown')}...") | ||
|
|
||
| issue = handler.get_issue(args.issue) | ||
| if issue: | ||
| print("\nSuccessfully fetched issue:") | ||
| print(f"Title: {issue.title}") | ||
| print(f"Created by: {issue.user}") | ||
| print(f"URL: {issue.html_url}") | ||
| print(f"\nBody preview: {issue.body[:200]}...") | ||
| else: | ||
| print(f"\nIssue #{args.issue} not found") | ||
|
|
||
| except Exception as e: | ||
| print(f"\nError: {str(e)}") | ||
| if "GITHUB_REPOSITORY" in str(e): | ||
| print("\nMake sure GITHUB_REPOSITORY is set in your .env file (e.g., 'DataDog/dd-sdk-ios')") | ||
|
|
||
| if __name__ == "__main__": | ||
| main() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| [tool:pytest] | ||
| testpaths = tests | ||
| addopts = -v --tb=short |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Automatic runs of this action are initially disabled until proper testing has been performed.