feat: core/ lacks unit tests despite dedicated test directories — untested changes to the intelligence pipeline risk silently breaking confidence scoring and LLM prompt construction.

[divyanshim27]

💡 Problem Statement

The project structure documents three test directories:
├── tests/
│ ├── unit/
│ ├── integration/
│ └── e2e/

However, these directories are empty. The existing tests/ folder contains no test files for any of the core intelligence modules. With 102 open issues and 44 PRs flowing through, contributors are modifying the trust_scorer.py, consequence_sim.py, and context_engine.py with no automated safety net.

Specific risk: If trust_scorer.py returns an incorrect confidence value (e.g., always returns 1.0), Execra will display high-confidence guidance for incorrect suggestions — directly undermining the project's core promise of reliable guidance. This failure would be invisible without tests.

Proposed Fix

Implement a foundational test suite covering the core intelligence layer:

# tests/unit/test_trust_scorer.py
import pytest
from core.intelligence.trust_scorer import TrustScorer

class TestTrustScorer:
    def setup_method(self):
        self.scorer = TrustScorer()

    def test_high_confidence_sources_score_above_80(self):
        """When LLM, rule engine, and execution trace all agree, score should be >80."""
        result = self.scorer.score(
            llm_suggestion="Add null check before line 42",
            rule_engine_result={"valid": True, "rule": "null_check_required"},
            execution_trace={"error_pattern": "NullPointerException at line 42"},
        )
        assert result.confidence >= 80
        assert result.sources  # Must list sources

    def test_conflicting_sources_score_below_60(self):
        """When LLM and rule engine disagree, score should be <60 and uncertainty flagged."""
        result = self.scorer.score(
            llm_suggestion="Remove the for-loop",
            rule_engine_result={"valid": False, "rule": "loop_required_for_iteration"},
            execution_trace=None,
        )
        assert result.confidence < 60
        assert result.uncertainty_flagged is True

    def test_score_includes_reasoning_string(self):
        result = self.scorer.score(
            llm_suggestion="Use context manager",
            rule_engine_result={"valid": True, "rule": "resource_management"},
            execution_trace=None,
        )
        assert isinstance(result.reasoning, str)
        assert len(result.reasoning) > 10

    def test_confidence_is_bounded_0_to_100(self):
        result = self.scorer.score("any suggestion", {}, None)
        assert 0 <= result.confidence <= 100

# tests/unit/test_context_engine.py
from core.intelligence.context_engine import ContextEngine

class TestContextEngine:
    def test_new_session_has_empty_context(self):
        engine = ContextEngine()
        assert engine.get_current_task() is None

    def test_task_detection_from_screen_content(self):
        engine = ContextEngine()
        engine.update_from_screen("def fibonacci(n):\n    # TODO: implement")
        task = engine.get_current_task()
        assert task is not None
        assert task.domain == "digital"
        assert "python" in task.inferred_language.lower()

    def test_context_persists_across_frames(self):
        engine = ContextEngine()
        engine.update_from_screen("import pandas as pd")
        engine.update_from_screen("df = pd.read_csv('data.csv')")
        task = engine.get_current_task()
        # Should remember pandas context across frames
        assert task is not None

# .github/workflows/tests.yml
name: Tests
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: '3.10' }
      - run: pip install -r requirements.txt -r requirements-dev.txt
      - run: python -m pytest tests/unit/ -v --tb=short
      - run: python -m pytest tests/unit/ --cov=core --cov-report=term-missing

Files to Create

File	Description
tests/unit/test_trust_scorer.py	Unit tests for confidence scoring
tests/unit/test_context_engine.py	Unit tests for task detection and session memory
tests/unit/test_screen_capture.py	Unit tests for delta detection logic
tests/unit/test_llm_client.py	Unit tests for provider fallback logic (mocked)
.github/workflows/tests.yml	CI workflow to run tests on every PR

Suggested labels: enhancement, testing, good first issue, ci/cd

I would like to work on this. Could you please assign it to me?

[divyanshim27]

Hey @sahoo-tech ,
Please assign it to me under GSSoC 2026.
/claim /assign /claim-assignment