Implement community review and audit system

Add GitHub Issues-based review system for validating graph analysis results:
- Review buttons in dashboard linking to issue template
- Python audit script to collect reviews via GitHub API
- GitHub Action for nightly review aggregation
- Audit tab displaying trust scores and review comments
- Documentation for contributors and reviewers

Enables community validation of analysis results with automated metrics display.

Author: stephen-rowan

.github/ISSUE_TEMPLATE/analysis_review.yml

@@ -0,0 +1,67 @@
+name: Analysis Review
+description: Review and provide feedback on graph analysis results
+title: "[Review] "
+labels: ["review"]
+body:
+  - type: dropdown
+    id: method
+    attributes:
+      label: Analysis Method
+      description: Which analysis method are you reviewing?
+      options:
+        - coattendance
+        - field-degree
+        - path-structure
+        - centrality
+        - clustering
+        - components
+      default: 0
+    validations:
+      required: true
+
+  - type: radio
+    id: rating
+    attributes:
+      label: Rating
+      description: How would you rate the accuracy of this analysis?
+      options:
+        - label: Correct
+          value: correct
+        - label: Needs Review
+          value: needs-review
+        - label: Incorrect
+          value: incorrect
+    validations:
+      required: true
+
+  - type: textarea
+    id: comment
+    attributes:
+      label: Comments
+      description: Please provide your feedback, observations, or concerns about this analysis
+      placeholder: |
+        Example:
+        - The results look accurate based on my understanding
+        - I noticed an issue with...
+        - Suggestion for improvement...
+    validations:
+      required: true
+
+  - type: textarea
+    id: suggestions
+    attributes:
+      label: Suggestions (Optional)
+      description: If you have specific suggestions, patches, or JSON corrections, please provide them here
+      placeholder: |
+        Optional JSON/patch suggestions:
+        - Suggested code changes
+        - Data corrections
+        - Algorithm improvements
+
+  - type: input
+    id: file
+    attributes:
+      label: File Reference (Auto-filled)
+      description: The file being reviewed (will be auto-filled from the review button)
+      placeholder: docs/index.html
+      value: docs/index.html

.github/workflows/review_audit.yml

@@ -0,0 +1,45 @@
+name: Review Audit Collection
+
+on:
+  schedule:
+    # Run nightly at 2 AM UTC
+    - cron: '0 2 * * *'
+  workflow_dispatch:  # Allow manual trigger
+
+jobs:
+  collect-reviews:
+    runs-on: ubuntu-latest
+    
+    permissions:
+      contents: write
+      issues: read
+    
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.x'
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install PyGithub>=2.1.0
+
+      - name: Run audit script
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          python scripts/audit_reviews.py
+
+      - name: Commit and push if changed
+        run: |
+          git config --local user.email "[email protected]"
+          git config --local user.name "GitHub Action"
+          git add docs/audit/reviews.json
+          git diff --staged --quiet || (git commit -m "Auto-update audit reviews [skip ci]" && git push)
+

CONTRIBUTING.md

@@ -0,0 +1,74 @@
+# Contributing to Graph-Python-scripts
+
+Thank you for your interest in contributing! This repository uses a community review system to validate graph analysis results.
+
+## Submitting Reviews
+
+Reviews help ensure the accuracy and reliability of graph analysis results. Anyone can submit a review using GitHub Issues.
+
+### Quick Start
+
+1. **Visit the Dashboard**
+   - Open the HTML dashboard: https://singularitynet-archive.github.io/Graph-Python-scripts/
+   - Or open `docs/index.html` locally in your browser
+
+2. **Navigate to an Analysis**
+   - Click on any analysis tab (e.g., "Co-attendance Degree", "Field Degree")
+   - Review the results displayed
+
+3. **Submit a Review**
+   - Click the **"Review This Analysis"** button at the top of the section
+   - This opens a GitHub Issue template pre-filled with the method name
+   - Fill out the form:
+     - **Rating**: Select "Correct", "Needs Review", or "Incorrect"
+     - **Comments**: Provide your feedback, observations, or concerns
+     - **Suggestions** (optional): Include specific improvements, patches, or corrections
+
+4. **Submit the Issue**
+   - Your review will be automatically tagged with the `review` label
+   - It will be collected by the nightly audit script and displayed in the Audit tab
+
+### What to Review
+
+Focus on:
+- **Accuracy**: Do the results match your understanding of the data?
+- **Completeness**: Are important patterns or relationships captured?
+- **Clarity**: Are the visualizations and tables clear and interpretable?
+- **Methodology**: Are the analysis methods appropriate for the questions being answered?
+
+### Review Labels
+
+When you submit a review, you should select one rating label:
+- `correct` - The analysis results appear accurate
+- `needs-review` - The analysis may have issues that require investigation
+- `incorrect` - The analysis results appear incorrect or misleading
+
+### Review Best Practices
+
+- **Be Specific**: Include details about what you observed or what concerns you
+- **Be Constructive**: If reporting issues, suggest how they might be addressed
+- **Reference Data**: If possible, cite specific examples from the data
+- **Be Respectful**: Maintain a respectful and collaborative tone
+
+### Viewing Review Statistics
+
+Check the **"Audit"** tab in the dashboard to see:
+- Trust scores aggregated across all reviews
+- Distribution of ratings
+- All review comments from the community
+
+## Code Contributions
+
+If you want to contribute code improvements:
+
+1. Fork the repository
+2. Create a feature branch
+3. Make your changes
+4. Test thoroughly
+5. Submit a pull request with a clear description
+
+## Questions?
+
+- Check [REVIEWING.md](REVIEWING.md) for details about how the audit system works
+- Open an issue for questions or discussions
+

Graph Analysis/unified_analysis.py

@@ -340,6 +340,12 @@ def ensure_iterable_records(data: Any) -> List[Any]:
 
 # ---------------- HTML Report Writer ----------------
 
+def _review_button(method_name: str) -> str:
+    """Generate HTML for a review button linking to GitHub Issues."""
+    url = f"https://github.com/SingularityNET-Archive/Graph-Python-scripts/issues/new?template=analysis_review.yml&method={method_name}&file=docs/index.html"
+    return f'<a href="{url}" class="review-button" target="_blank">Review This Analysis</a>'
+
+
 def write_html_report(
     output_file: str,
     timestamp: str,
@@ -394,6 +400,7 @@ def write_html_report(
             <button class="tab-button" onclick="showTab('centrality')">Centrality</button>
             <button class="tab-button" onclick="showTab('clustering')">Clustering</button>
             <button class="tab-button" onclick="showTab('components')">Components</button>
+            <button class="tab-button" onclick="showTab('audit')">Audit</button>
         </div>
 
         <div class="tab-content">
@@ -411,6 +418,7 @@ def write_html_report(
             <!-- Co-attendance Degree Tab -->
             <div id="coattendance" class="tab-pane">
                 <h2>Degree (Co-attendance) Analysis</h2>
+                """ + _review_button("coattendance") + """
                 <p class="explanation">People are connected if they attend the same meeting; a person's degree is how many unique people they co-attended with.</p>
                 
                 <h3>Interactive Network Visualization</h3>
@@ -452,6 +460,7 @@ def write_html_report(
             <!-- Field Degree Tab -->
             <div id="field-degree" class="tab-pane">
                 <h2>JSON Field Degree Analysis</h2>
+                """ + _review_button("field-degree") + """
                 <p class="explanation">Fields are connected when they appear together inside the same JSON object; a field's degree is the number of distinct fields it co-occurs with.</p>
                 
                 <h3>Top Fields by Degree</h3>
@@ -485,6 +494,7 @@ def write_html_report(
             <!-- Path Structure Tab -->
             <div id="path-structure" class="tab-pane">
                 <h2>JSON Path Structure Analysis</h2>
+                """ + _review_button("path-structure") + """
                 <p class="explanation">Each JSON path represents a unique nested route (keys/array indices); depth shows how deeply information is nested.</p>
                 
                 <ul class="summary-list">
@@ -518,6 +528,7 @@ def write_html_report(
             <!-- Centrality Tab -->
             <div id="centrality" class="tab-pane">
                 <h2>Field Centrality (Co-occurrence)</h2>
+                """ + _review_button("centrality") + """
                 <p class="explanation">Centrality scores highlight fields that are well-connected (degree), act as bridges (betweenness), are close to others (closeness), or connect to other influential fields (eigenvector).</p>
                 
                 <table>
@@ -543,6 +554,7 @@ def write_html_report(
             <!-- Clustering Tab -->
             <div id="clustering" class="tab-pane">
                 <h2>Clustering (Field Co-occurrence Graph)</h2>
+                """ + _review_button("clustering") + """
                 <p class="explanation">Clustering measures how tightly a field's neighbors are connected to each other (higher means more triads).</p>
                 
                 <p><strong>Average Clustering Coefficient:</strong> """)
@@ -567,6 +579,7 @@ def write_html_report(
             <!-- Connected Components Tab -->
             <div id="components" class="tab-pane">
                 <h2>Connected Components (Field Co-occurrence Graph)</h2>
+                """ + _review_button("components") + """
                 <p class="explanation">Components are groups of fields that are all reachable from each other; multiple components suggest separate substructures.</p>
                 
                 <ul class="summary-list">
@@ -581,6 +594,55 @@ def write_html_report(
             f.write(f"                    <li>{n}</li>\n")
         f.write("""                </ul>
             </div>
+
+            <!-- Audit Tab -->
+            <div id="audit" class="tab-pane">
+                <h2>Review Audit</h2>
+                <p class="explanation">Community review scores and feedback for each analysis method.</p>
+                
+                <div id="audit-loading" style="text-align: center; padding: 20px;">
+                    <p>Loading audit data...</p>
+                </div>
+                
+                <div id="audit-content" style="display: none;">
+                    <p id="audit-last-updated" class="explanation" style="font-weight: bold; margin-bottom: 20px;"></p>
+                    
+                    <h3>Trust Scores by Method</h3>
+                    <p class="explanation">Percentage of reviews rated as "Correct" for each analysis method.</p>
+                    <table id="trust-scores-table">
+                        <thead>
+                            <tr><th>Method</th><th>Trust Score</th><th>Total Reviews</th><th>Correct</th><th>Incorrect</th><th>Needs Review</th></tr>
+                        </thead>
+                        <tbody id="trust-scores-body">
+                        </tbody>
+                    </table>
+                    
+                    <h3>Rating Distribution</h3>
+                    <p class="explanation">Overall distribution of review ratings across all methods.</p>
+                    <div id="rating-chart-container" style="margin: 20px 0; text-align: center;">
+                        <canvas id="rating-chart" width="400" height="400"></canvas>
+                    </div>
+                    
+                    <h3>Review Comments</h3>
+                    <p class="explanation">All review comments submitted by the community.</p>
+                    <table id="review-comments-table">
+                        <thead>
+                            <tr><th>Method</th><th>Rating</th><th>Comment</th><th>Author</th><th>Date</th><th>Issue</th></tr>
+                        </thead>
+                        <tbody id="review-comments-body">
+                        </tbody>
+                    </table>
+                    
+                    <p style="margin-top: 20px;">
+                        <a href="audit/reviews.json" target="_blank" class="review-button">View Raw JSON Data</a>
+                    </p>
+                </div>
+                
+                <div id="audit-error" style="display: none; color: #d73a49; padding: 20px; background-color: #ffeef0; border-radius: 6px;">
+                    <p><strong>Error loading audit data:</strong> <span id="audit-error-message"></span></p>
+                    <p>This may be because no reviews have been submitted yet, or the data file doesn't exist.</p>
+                </div>
+            </div>
         </div>
     </div>
 
@@ -607,6 +669,7 @@ def write_html_report(
             ], ensure_ascii=False) + """
         };
     </script>
+    <script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/chart.umd.min.js"></script>
     <script src="script.js"></script>
 </body>
 </html>

README.md

@@ -58,5 +58,30 @@ All scripts read from a shared public JSON:
 - `graph.png`, `graph2.png` — rendered graphs
 - `Scripts/all_workgroups_graph_sanitized.gexf` — Gephi import
 - Markdown reports in `reports/`
+- HTML dashboard at `docs/index.html` (GitHub Pages)
 
 Notes: Scripts run headlessly and save files to disk; images can be opened via your OS default viewer.
+
+## Community Review System
+
+This repository includes a community review system to validate graph analysis results. You can review analysis results and provide feedback through GitHub Issues.
+
+### How to Review
+
+1. Visit the [HTML dashboard](https://singularitynet-archive.github.io/Graph-Python-scripts/) (or open `docs/index.html` locally)
+2. Navigate to any analysis tab (Co-attendance Degree, Field Degree, Path Structure, etc.)
+3. Click the **"Review This Analysis"** button at the top of the section
+4. Fill out the review form with:
+   - Your rating (Correct / Needs Review / Incorrect)
+   - Comments and feedback
+   - Optional suggestions for improvements
+5. Submit the issue - it will be automatically tagged with the `review` label
+
+### View Review Results
+
+- Click on the **"Audit"** tab in the dashboard to see:
+  - Trust scores for each analysis method
+  - Rating distribution charts
+  - All review comments from the community
+
+Reviews are collected nightly via GitHub Actions and displayed in the audit dashboard. See [CONTRIBUTING.md](CONTRIBUTING.md) and [REVIEWING.md](REVIEWING.md) for more details.