Skip to content

Commit 9e21262

Browse files
committed
Enhance community review system with two submission methods
Updated CONTRIBUTING.md and README.md to clarify the community review process. Introduced two methods for submitting reviews: an in-dashboard review form and JSON file submissions. The JSON format requirements are detailed, and the download functionality for reviews has been improved in script.js to ensure a single review object is downloaded. This enhances user experience and streamlines the review process.
1 parent 32c1728 commit 9e21262

File tree

7 files changed

+365
-22
lines changed

7 files changed

+365
-22
lines changed
Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
name: Process JSON Reviews
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
paths:
7+
- 'reviews/*.json'
8+
paths-ignore:
9+
- 'reviews/processed/**'
10+
workflow_dispatch: # Allow manual trigger
11+
12+
permissions:
13+
contents: write
14+
issues: write
15+
16+
jobs:
17+
process-reviews:
18+
runs-on: ubuntu-latest
19+
20+
steps:
21+
- name: Checkout repository
22+
uses: actions/checkout@v4
23+
with:
24+
fetch-depth: 0
25+
26+
- name: Set up Python
27+
uses: actions/setup-python@v5
28+
with:
29+
python-version: '3.11'
30+
31+
- name: Install dependencies
32+
run: |
33+
python -m pip install --upgrade pip
34+
pip install PyGithub>=2.1.0
35+
36+
- name: Process JSON reviews and create Issues
37+
env:
38+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
39+
run: |
40+
python scripts/process_json_reviews.py
41+
42+
- name: Commit and push processed files
43+
run: |
44+
git config --local user.email "[email protected]"
45+
git config --local user.name "GitHub Action"
46+
git remote set-url origin https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}.git
47+
git add reviews/processed/
48+
if git diff --staged --quiet; then
49+
echo "No processed files to commit"
50+
else
51+
git commit -m "Move processed JSON reviews to processed/ directory [skip ci]"
52+
git push origin HEAD:main
53+
fi
54+

CONTRIBUTING.md

Lines changed: 48 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,9 @@ Thank you for your interest in contributing! This repository uses a community re
44

55
## Submitting Reviews
66

7-
Reviews help ensure the accuracy and reliability of graph analysis results. Anyone can submit a review using GitHub Issues.
7+
Reviews help ensure the accuracy and reliability of graph analysis results. There are two ways to submit reviews:
88

9-
### Quick Start
9+
### Method 1: In-Dashboard Review Form (Recommended for Quick Reviews)
1010

1111
1. **Visit the Dashboard**
1212
- Open the HTML dashboard: https://singularitynet-archive.github.io/Graph-Python-scripts/
@@ -17,16 +17,55 @@ Reviews help ensure the accuracy and reliability of graph analysis results. Anyo
1717
- Review the results displayed
1818

1919
3. **Submit a Review**
20-
- Click the **"Review This Analysis"** button at the top of the section
21-
- This opens a GitHub Issue template pre-filled with the method name
20+
- Scroll to the bottom of the analysis tab to find the review form
2221
- Fill out the form:
2322
- **Rating**: Select "Correct", "Needs Review", or "Incorrect"
24-
- **Comments**: Provide your feedback, observations, or concerns
23+
- **Comments**: Provide your feedback, observations, or concerns (required)
24+
- **Your Name** (optional): Your name or identifier
2525
- **Suggestions** (optional): Include specific improvements, patches, or corrections
26-
27-
4. **Submit the Issue**
28-
- Your review will be automatically tagged with the `review` label
29-
- It will be collected by the nightly audit script and displayed in the Audit tab
26+
- Click "Submit Review"
27+
- Optionally download the review as JSON using the "Download Review as JSON" button
28+
29+
4. **Reviews are stored locally** in your browser and will appear in the Audit tab immediately.
30+
31+
### Method 2: JSON File Submission (For Programmatic Submissions)
32+
33+
You can also submit reviews by committing JSON files to the repository:
34+
35+
1. **Create a Review JSON File**
36+
- Download a review from the dashboard (uses the correct format)
37+
- Or create a JSON file with the following structure:
38+
```json
39+
{
40+
"id": "unique-review-id",
41+
"method": "coattendance",
42+
"rating": "correct",
43+
"comment": "Your review comment here",
44+
"reviewer": "Your Name",
45+
"suggestions": "Optional suggestions",
46+
"file": "docs/index.html",
47+
"timestamp": "2025-01-15T10:00:00.000Z"
48+
}
49+
```
50+
51+
2. **Submit the JSON File**
52+
- Commit the JSON file to the `/reviews/` directory in the repository
53+
- Push to the main branch
54+
55+
3. **Automatic Processing**
56+
- A GitHub Action will automatically:
57+
- Validate the JSON structure
58+
- Create a GitHub Issue from your review
59+
- Move the processed file to `/reviews/processed/`
60+
61+
**JSON File Requirements:**
62+
- File must be valid JSON with `.json` extension
63+
- Must include required fields: `method`, `rating`, `comment`, `timestamp`
64+
- `method` must be one of: `coattendance`, `field-degree`, `path-structure`, `centrality`, `clustering`, `components`
65+
- `rating` must be one of: `correct`, `needs-review`, `incorrect`
66+
- `comment` cannot be empty
67+
68+
**Note:** Invalid JSON files will be skipped with error messages logged. Only successfully processed files are moved to `/reviews/processed/`.
3069

3170
### What to Review
3271

README.md

Lines changed: 13 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -64,18 +64,21 @@ Notes: Scripts run headlessly and save files to disk; images can be opened via y
6464

6565
## Community Review System
6666

67-
This repository includes a community review system to validate graph analysis results. You can review analysis results and provide feedback through GitHub Issues.
67+
This repository includes a community review system to validate graph analysis results. You can submit reviews in two ways:
6868

69-
### How to Review
69+
### Review Methods
7070

71-
1. Visit the [HTML dashboard](https://singularitynet-archive.github.io/Graph-Python-scripts/) (or open `docs/index.html` locally)
72-
2. Navigate to any analysis tab (Co-attendance Degree, Field Degree, Path Structure, etc.)
73-
3. Click the **"Review This Analysis"** button at the top of the section
74-
4. Fill out the review form with:
75-
- Your rating (Correct / Needs Review / Incorrect)
76-
- Comments and feedback
77-
- Optional suggestions for improvements
78-
5. Submit the issue - it will be automatically tagged with the `review` label
71+
**Method 1: In-Dashboard Form**
72+
- Visit the [HTML dashboard](https://singularitynet-archive.github.io/Graph-Python-scripts/)
73+
- Navigate to any analysis tab and scroll to the review form at the bottom
74+
- Submit reviews directly in your browser (stored locally)
75+
- Optionally download reviews as JSON files
76+
77+
**Method 2: JSON File Submission**
78+
- Commit JSON review files to the `/reviews/` directory
79+
- GitHub Action automatically creates GitHub Issues from JSON files
80+
- Processed files are moved to `/reviews/processed/`
81+
- See [CONTRIBUTING.md](CONTRIBUTING.md) for JSON format specification
7982

8083
### View Review Results
8184

Scripts/process_json_reviews.py

Lines changed: 224 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,224 @@
1+
#!/usr/bin/env python3
2+
"""
3+
Process JSON review files and create GitHub Issues.
4+
5+
Reads JSON review files from /reviews/ directory, validates them,
6+
creates GitHub Issues, and moves processed files to /reviews/processed/.
7+
"""
8+
9+
import json
10+
import os
11+
import shutil
12+
import sys
13+
from pathlib import Path
14+
from typing import Dict, Any, Optional, List, Tuple
15+
from datetime import datetime
16+
17+
try:
18+
from github import Github
19+
except ImportError:
20+
print("Error: PyGithub not installed. Run: pip install PyGithub", file=sys.stderr)
21+
sys.exit(1)
22+
23+
24+
# Repository configuration
25+
REPO_OWNER = "SingularityNET-Archive"
26+
REPO_NAME = "Graph-Python-scripts"
27+
REVIEWS_DIR = Path("reviews")
28+
PROCESSED_DIR = Path("reviews/processed")
29+
30+
# Method names to validate
31+
METHODS = [
32+
"coattendance",
33+
"field-degree",
34+
"path-structure",
35+
"centrality",
36+
"clustering",
37+
"components",
38+
]
39+
40+
# Valid ratings
41+
VALID_RATINGS = ["correct", "needs-review", "incorrect"]
42+
43+
# Required fields in review JSON
44+
REQUIRED_FIELDS = ["method", "rating", "comment", "timestamp"]
45+
46+
47+
def validate_review(review_data: Dict[str, Any]) -> Tuple[bool, Optional[str]]:
48+
"""Validate review JSON structure."""
49+
# Check required fields
50+
for field in REQUIRED_FIELDS:
51+
if field not in review_data:
52+
return False, f"Missing required field: {field}"
53+
54+
# Validate method
55+
if review_data["method"] not in METHODS:
56+
return False, f"Invalid method: {review_data['method']}. Must be one of {METHODS}"
57+
58+
# Validate rating
59+
if review_data["rating"] not in VALID_RATINGS:
60+
return False, f"Invalid rating: {review_data['rating']}. Must be one of {VALID_RATINGS}"
61+
62+
# Validate comment is not empty
63+
if not review_data.get("comment", "").strip():
64+
return False, "Comment field cannot be empty"
65+
66+
return True, None
67+
68+
69+
def format_issue_body(review_data: Dict[str, Any]) -> str:
70+
"""Format review data as GitHub Issue body."""
71+
body_lines = [
72+
"## Analysis Method",
73+
f"**Method:** {review_data['method']}",
74+
"",
75+
f"**File:** {review_data.get('file', 'docs/index.html')}",
76+
"",
77+
"---",
78+
"",
79+
"## Rating",
80+
"",
81+
f"- [x] {review_data['rating'].replace('-', ' ').title()}",
82+
"",
83+
"## Comments",
84+
"",
85+
review_data.get("comment", ""),
86+
]
87+
88+
if review_data.get("suggestions"):
89+
body_lines.extend([
90+
"",
91+
"---",
92+
"",
93+
"## Suggestions",
94+
"",
95+
review_data["suggestions"],
96+
])
97+
98+
if review_data.get("reviewer"):
99+
body_lines.extend([
100+
"",
101+
"---",
102+
"",
103+
f"**Reviewed by:** {review_data['reviewer']}",
104+
f"**Review ID:** {review_data.get('id', 'unknown')}",
105+
f"**Submitted:** {review_data.get('timestamp', 'unknown')}",
106+
])
107+
108+
return "\n".join(body_lines)
109+
110+
111+
def create_github_issue(repo: Any, review_data: Dict[str, Any]) -> Optional[int]:
112+
"""Create a GitHub Issue from review data."""
113+
try:
114+
title = f"[Review] {review_data['method'].replace('-', ' ').title()}"
115+
body = format_issue_body(review_data)
116+
117+
# Labels to apply
118+
labels = ["review", review_data["rating"]]
119+
120+
# Create issue
121+
issue = repo.create_issue(
122+
title=title,
123+
body=body,
124+
labels=labels
125+
)
126+
127+
print(f" ✓ Created issue #{issue.number}: {title}")
128+
return issue.number
129+
130+
except Exception as e:
131+
print(f" ✗ Error creating issue: {e}", file=sys.stderr)
132+
return None
133+
134+
135+
def process_json_file(file_path: Path, repo: Any) -> Tuple[bool, Optional[str]]:
136+
"""Process a single JSON review file."""
137+
try:
138+
# Read and parse JSON
139+
with open(file_path, 'r') as f:
140+
review_data = json.load(f)
141+
142+
# Validate review
143+
is_valid, error_msg = validate_review(review_data)
144+
if not is_valid:
145+
return False, error_msg
146+
147+
# Create GitHub Issue
148+
issue_number = create_github_issue(repo, review_data)
149+
if issue_number is None:
150+
return False, "Failed to create GitHub Issue"
151+
152+
# Move file to processed directory
153+
processed_path = PROCESSED_DIR / file_path.name
154+
os.makedirs(PROCESSED_DIR, exist_ok=True)
155+
shutil.move(str(file_path), str(processed_path))
156+
157+
return True, None
158+
159+
except json.JSONDecodeError as e:
160+
return False, f"Invalid JSON: {e}"
161+
except Exception as e:
162+
return False, f"Error processing file: {e}"
163+
164+
165+
def main() -> None:
166+
"""Main function to process JSON review files."""
167+
# Get GitHub token from environment
168+
github_token = os.environ.get("GITHUB_TOKEN")
169+
if not github_token:
170+
print("Error: GITHUB_TOKEN environment variable not set", file=sys.stderr)
171+
print("Note: For GitHub Actions, this is automatically set as GITHUB_TOKEN", file=sys.stderr)
172+
sys.exit(1)
173+
174+
# Initialize GitHub client
175+
g = Github(github_token)
176+
repo = g.get_repo(f"{REPO_OWNER}/{REPO_NAME}")
177+
178+
# Ensure directories exist
179+
os.makedirs(REVIEWS_DIR, exist_ok=True)
180+
os.makedirs(PROCESSED_DIR, exist_ok=True)
181+
182+
# Find all JSON files in reviews directory (not in processed)
183+
json_files = list(REVIEWS_DIR.glob("*.json"))
184+
185+
if not json_files:
186+
print("No JSON review files found in reviews/ directory")
187+
return
188+
189+
print(f"Found {len(json_files)} JSON review file(s) to process")
190+
191+
# Process each file
192+
processed_count = 0
193+
failed_count = 0
194+
failed_files = []
195+
196+
for json_file in json_files:
197+
print(f"\nProcessing: {json_file.name}")
198+
success, error_msg = process_json_file(json_file, repo)
199+
200+
if success:
201+
processed_count += 1
202+
else:
203+
failed_count += 1
204+
failed_files.append((json_file.name, error_msg))
205+
print(f" ✗ Failed: {error_msg}")
206+
207+
# Summary
208+
print(f"\n{'='*50}")
209+
print(f"Processing Summary:")
210+
print(f" ✓ Successfully processed: {processed_count}")
211+
print(f" ✗ Failed: {failed_count}")
212+
213+
if failed_files:
214+
print(f"\nFailed files:")
215+
for filename, error in failed_files:
216+
print(f" - {filename}: {error}")
217+
218+
if failed_count > 0:
219+
sys.exit(1)
220+
221+
222+
if __name__ == "__main__":
223+
main()
224+

0 commit comments

Comments
 (0)