Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -165,5 +165,40 @@ outputs/
logs/
nohup.out

# Repository media policy: do not commit images, videos, or asset/media folders.
# Use external hosting, release artifacts, or approved storage and link from docs.
asset/
assets/
image/
images/
img/
media/
video/
videos/
*.avif
*.bmp
*.gif
*.heic
*.heif
*.icns
*.ico
*.jpeg
*.jpg
*.png
*.svg
*.tif
*.tiff
*.webp
*.avi
*.flv
*.m4v
*.mkv
*.mov
*.mp4
*.mpeg
*.mpg
*.webm
*.wmv

# Use-cases: exclude lock files to keep the repo lean
use-cases/**/package-lock.json
8 changes: 8 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,14 @@ repos:
- id: detect-private-key
- id: check-merge-conflict

- repo: local
hooks:
- id: no-repo-assets
name: block committed images, videos, and asset directories
entry: python3 scripts/check_repo_assets.py
language: system
pass_filenames: false

- repo: https://github.com/jorisroovers/gitlint
rev: v0.19.1
hooks:
Expand Down
12 changes: 7 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,11 @@ Use the [feature request template](https://github.com/EverMind-AI/EverOS/issues/

1. Create a branch from `main`.
2. Keep the change scoped to one purpose.
3. Run `make ci` locally before requesting review.
4. Use a Conventional Commit title, such as `fix(search): guard empty profile`.
5. Open a pull request to `main` and fill out the PR template.
3. Do not commit images, videos, or asset/media directories. Use external
hosting, release artifacts, or approved storage and link from docs.
4. Run `make ci` locally before requesting review.
5. Use a Conventional Commit title, such as `fix(search): guard empty profile`.
6. Open a pull request to `main` and fill out the PR template.

By submitting a pull request, you agree that your contribution is licensed under
the project's [Apache-2.0](LICENSE) license.
Expand Down Expand Up @@ -90,7 +92,7 @@ git clone https://github.com/EverMind-AI/EverOS.git
cd EverOS
make install # deps + pre-commit hooks (one-stop dev setup)
everos init # write ./.env, then fill in the API key slots
make ci # verify: lint + unit + integration
make ci # verify: lint + unit + integration + package
```

### Code style
Expand All @@ -110,7 +112,7 @@ Highlights:

```bash
make format # ruff fix + format
make lint # ruff check + import-linter
make lint # ruff check + import-linter + hard repo hygiene gates
```

### Branch strategy
Expand Down
11 changes: 9 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
.PHONY: help install install-deps lint docs-check check-commits check-cjk check-datetime openapi check-openapi format test integration package cov ci clean
.PHONY: help install install-deps lint docs-check check-commits check-assets check-cjk check-datetime openapi check-openapi format test integration package cov ci clean

help:
@echo "Targets:"
@echo " install Install deps + pre-commit hooks (full dev setup)"
@echo " install-deps Install deps only (uv sync --frozen, used by CI)"
@echo " lint ruff (check + format-check) + import-linter + datetime discipline + openapi drift"
@echo " lint ruff + import-linter + repo asset/media + datetime discipline + openapi drift"
@echo " docs-check Validate Markdown links, use-case banners, and issue template YAML"
@echo " check-commits Validate Conventional Commit subjects for a git range"
@echo " check-assets Block committed images, videos, and asset/media directories"
@echo " check-cjk Scan for CJK outside the language-policy allowlist (advisory)"
@echo " check-datetime Scan for code that bypasses component/utils/datetime (HARD gate, run via lint)"
@echo " openapi Regenerate docs/openapi.json from the FastAPI app"
Expand Down Expand Up @@ -34,6 +35,7 @@ lint:
uv run ruff check src tests
uv run ruff format --check src tests
uv run lint-imports
uv run python scripts/check_repo_assets.py
uv run python scripts/check_datetime_discipline.py
uv run python scripts/dump_openapi.py --check

Expand All @@ -44,6 +46,11 @@ docs-check:
check-commits:
python3 scripts/check_commit_messages.py $(RANGE)

# Repository media hygiene gate. Images/videos belong in external hosting,
# release artifacts, or other approved storage, then linked from docs.
check-assets:
uv run python scripts/check_repo_assets.py

# Advisory CJK scan (see .claude/rules/language-policy.md). Deliberately NOT
# wired into `lint` / `ci`: the policy is enforced by review and the rules
# doc, not a hard gate. Run on demand when touching potentially-CJK files.
Expand Down
14 changes: 10 additions & 4 deletions docs/engineering.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,13 @@ Reasons this is documented separately:
│ │ ├ check-yaml / check-toml │ │
│ │ ├ check-added-large-files (≥1MB warn) │ │
│ │ ├ detect-private-key │ │
│ │ ├ no committed images/videos/assets │ │
│ │ └ gitlint (commit-msg stage) │ │
│ │ │ │
│ │ ruff lint + format │ │
│ │ (replaces black / isort / flake8) │ │
│ │ import-linter DDD layer-direction enforcement │ │
│ │ repo asset gate blocks images/videos/assets in git │ │
│ │ pytest unit / integration │ │
│ │ │ │
│ └────────────────────────────────────────────────────────────┘ │
Expand All @@ -94,6 +96,7 @@ Reasons this is documented separately:
│ ┌─ CI/CD (GitHub Actions) ───────────────────────────────────┐ │
│ │ │ │
│ │ CI: .github/workflows/ci.yml lint / test / integ │ │
│ │ / package build │ │
│ │ Docs: .github/workflows/docs.yml Markdown + YAML check │ │
│ │ Gates invoke Makefile targets; the Makefile is the │ │
│ │ single source of truth for commands. │ │
Expand Down Expand Up @@ -204,19 +207,21 @@ Stage 2: pre-commit (triggered by `git commit`)
├ check-yaml, check-toml
├ check-added-large-files (≥1MB)
├ detect-private-key
├ no-repo-assets (rejects images/videos/assets in git)
└ gitlint (commit-msg stage; rejects malformed messages)

Stage 3: local `make ci` (manual, before push)
├ make lint (ruff check + ruff format --check + import-linter)
├ make lint (ruff + import-linter + repo hygiene gates)
├ make test (pytest tests/unit)
└ make integration (pytest tests/integration)
├ make integration (pytest tests/integration)
└ make package (sdist/wheel build + import smoke test)

Stage 4: CI (GitHub Actions, push + PR triggered)
└ re-runs the same `make lint / test / integration` targets
└ re-runs the same `make lint / test / integration / package` targets

Expand Down Expand Up @@ -272,7 +277,7 @@ dev = ["ruff", "pytest", "pytest-asyncio", "pytest-cov",
make help list all targets
make install uv sync --frozen
make format ruff fix + format
make lint ruff + import-linter + datetime discipline + openapi drift
make lint ruff + import-linter + repo asset/media + datetime discipline + openapi drift
make test pytest tests/unit
make integration pytest tests/integration
make package build sdist/wheel + smoke-test wheel import
Expand Down Expand Up @@ -338,6 +343,7 @@ Every key has a sensible default except the `API_KEY` fields, which you fill in.
|---|---|---|
| Lint | `make lint` (ruff check + ruff format --check) | any error |
| Layer direction | `make lint` (lint-imports inside) | layer violation |
| Repository media | `make lint` (check_repo_assets.py) | images/videos/assets committed |
| Datetime discipline | `make lint` (check_datetime_discipline.py) | bypasses helper module |
| OpenAPI drift | `make lint` (dump_openapi.py --check) | schema ≠ committed openapi.json |
| Unit | `make test` (pytest tests/unit) | any failure |
Expand Down
122 changes: 122 additions & 0 deletions scripts/check_repo_assets.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
"""Block committed image/video files and asset-style directories."""

from __future__ import annotations

import subprocess
import sys
from dataclasses import dataclass
from pathlib import PurePosixPath
from typing import Iterable

BLOCKED_DIR_NAMES = frozenset(
{
"asset",
"assets",
"image",
"images",
"img",
"media",
"video",
"videos",
}
)
IMAGE_EXTENSIONS = frozenset(
{
".avif",
".bmp",
".gif",
".heic",
".heif",
".icns",
".ico",
".jpeg",
".jpg",
".png",
".svg",
".tif",
".tiff",
".webp",
}
)
VIDEO_EXTENSIONS = frozenset(
{
".avi",
".flv",
".m4v",
".mkv",
".mov",
".mp4",
".mpeg",
".mpg",
".webm",
".wmv",
}
)


@dataclass(frozen=True)
class Violation:
path: str
reason: str


def _normalise_path(path: str) -> PurePosixPath:
return PurePosixPath(path.replace("\\", "/"))


def _violation_reason(path: str) -> str | None:
posix_path = _normalise_path(path)
lower_parts = tuple(part.lower() for part in posix_path.parts)
if any(part in BLOCKED_DIR_NAMES for part in lower_parts):
return "asset/media directory"

suffix = posix_path.suffix.lower()
if suffix in IMAGE_EXTENSIONS:
return "image file"
if suffix in VIDEO_EXTENSIONS:
return "video file"
return None


def find_violations(paths: Iterable[str]) -> list[Violation]:
violations: list[Violation] = []
for path in paths:
reason = _violation_reason(path)
if reason is not None:
violations.append(Violation(path=path, reason=reason))
return violations


def _tracked_paths() -> list[str]:
result = subprocess.run(
["git", "ls-files", "-z"],
check=True,
stdout=subprocess.PIPE,
text=False,
)
return [
raw.decode("utf-8")
for raw in result.stdout.split(b"\0")
if raw
]


def main() -> int:
violations = find_violations(_tracked_paths())
if not violations:
print("Repository asset/media check passed.")
return 0

print(
"Repository asset/media check failed.\n"
"Do not commit images, videos, or asset/media directories. "
"Host visual media externally, in release artifacts, or another "
"approved storage location, then link to it from docs.\n"
)
for violation in violations:
print(f"- {violation.path}: {violation.reason}")
return 1


if __name__ == "__main__":
raise SystemExit(main())
17 changes: 14 additions & 3 deletions tests/unit/test_memory/test_cascade/test_scanner_unit.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import pytest

from everos.core.persistence import MemoryRoot
from everos.memory.cascade import scanner as scanner_module
from everos.memory.cascade.scanner import CascadeScanner, _collect_scan_inputs


Expand Down Expand Up @@ -112,16 +113,26 @@ async def test_run_loop_swallows_scan_exception(
scanner = CascadeScanner(mr, scan_interval_seconds=0.05)

call_count = {"n": 0}
second_scan = asyncio.Event()
logged_errors: list[str] = []

async def fake_scan() -> list: # type: ignore[type-arg]
call_count["n"] += 1
if call_count["n"] == 1:
raise RuntimeError("simulated scanner failure")
second_scan.set()
return []

def fake_exception(_event: str, *, error: str) -> None:
logged_errors.append(error)

monkeypatch.setattr(scanner, "scan_once", fake_scan)
monkeypatch.setattr(scanner_module.logger, "exception", fake_exception)
await scanner.start()
# Let the loop iterate at least twice (interval is 50ms).
await asyncio.sleep(0.2)
await scanner.stop()
try:
await asyncio.wait_for(second_scan.wait(), timeout=1.0)
finally:
await scanner.stop()

assert logged_errors == ["simulated scanner failure"]
assert call_count["n"] >= 2 # second call ran despite first throwing
Loading
Loading