Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
125 commits
Select commit Hold shift + click to select a range
6b372ee
Merge pull request #71 from wg-lux/erc
maxhild Jan 15, 2026
e38f63b
Merge pull request #73 from wg-lux/erc
maxhild Jan 16, 2026
c69a9ba
Merge pull request #74 from wg-lux/erc
maxhild Mar 2, 2026
fdd17e6
Migration overrides and django integration with dtypes
maxhild Mar 25, 2026
eca3519
test setup for hub works
maxhild Mar 26, 2026
3ea350a
Added update to endoreg db and filewatcher
maxhild Mar 27, 2026
78c3c9b
overrides
maxhild Mar 27, 2026
f6009a5
overrides removed
maxhild Mar 27, 2026
5c1b2f0
Revert "overrides removed"
maxhild Mar 27, 2026
f3584df
overrides reapplied, made compatible with upstream squash
maxhild Mar 27, 2026
0e7b798
overrides reapplied, made compatible with upstream squash
maxhild Mar 27, 2026
6cbb3fe
Convert lx-terminology-editor from submodule to regular directory
maxhild Mar 27, 2026
7f9c4de
explicit remove ninja and add django ninja
maxhild Mar 27, 2026
22270d7
Dont set working dir to data dir
maxhild Mar 27, 2026
0d4a31d
Added pydantic-settings
maxhild Mar 27, 2026
e83a21d
New static file permissions
maxhild Mar 27, 2026
5255b9c
added german translation and lx data model api route
maxhild Mar 27, 2026
03c3482
0.0.1 - dead code removed
maxhild Mar 30, 2026
a6d0795
Application settings, encrypted streaming
maxhild Mar 30, 2026
fda21e8
encrypted directory setup
maxhild Apr 1, 2026
c92d14b
encrypted directory setup
maxhild Apr 1, 2026
03a7aeb
encrypted directory setup
maxhild Apr 1, 2026
626e4c0
encrypted directory setup
maxhild Apr 1, 2026
b9c54ea
encrypted directory setup
maxhild Apr 2, 2026
7a6de30
remove devenv env setup
maxhild Apr 2, 2026
abb8112
merge conflicting migrations
maxhild Apr 2, 2026
5b868dd
merge conflicting migrations
maxhild Apr 2, 2026
6323196
migrations fixed and xorg support
maxhild Apr 7, 2026
90e1532
repair plaintext copies
maxhild Apr 7, 2026
483439c
Activating encryption
maxhild Apr 7, 2026
7347f1a
Actual preservation of storage settings and env reading
maxhild Apr 8, 2026
b90f205
documenting watcher
maxhild Apr 8, 2026
d09d751
Center Key usage, hub export is ready for usage
maxhild Apr 9, 2026
c276cdd
0.1.1 - Simplification of Hub setup and Frame annotation
maxhild Apr 10, 2026
3d98dcd
0.9.4.5 - critical migration bug resolved idempotently in script
maxhild Apr 10, 2026
99574c6
0.1.2 - UX niceties, stream hopefully back
maxhild Apr 10, 2026
efee238
storage pressure addressing
maxhild Apr 13, 2026
3c32875
Atomic file operations
maxhild Apr 14, 2026
20a8390
filewatcher repeat safety
maxhild Apr 16, 2026
9098cdc
Further centralizing paths and file operations to shared helpers
maxhild Apr 21, 2026
baf339b
Frontend loaded pdf url for video
maxhild Apr 22, 2026
ddfb27b
V 0.2.8 Filewatcher issues with new cryptsetup hopefully resolved
maxhild Apr 22, 2026
4da4d40
V 0.3.1 longer timeout for videos and guard on video annotation so vi…
maxhild Apr 23, 2026
27aac05
V 0.3.2 - safe ffmpeg timeout, wiring through from the
maxhild Apr 23, 2026
4667e56
V 0.3.6 - Working state
maxhild Apr 28, 2026
443c26a
checks
Maddonix Apr 29, 2026
7da3d2c
checks updated
Maddonix Apr 29, 2026
ea46358
lx-annotate full application
maxhild Apr 29, 2026
fadee6a
Merge remote-tracking branch 'refs/remotes/origin/main'
maxhild Apr 29, 2026
6d51e59
lx-annotate full application
maxhild Apr 29, 2026
0d58387
encrypted streaming is now lazy and file field based
maxhild Apr 29, 2026
3a6b118
Add bounded filewatcher drain mode
maxhild Apr 30, 2026
08a4e61
0.4.0 - Various optimizations of frontend and filewatcher to make it …
maxhild Apr 30, 2026
0eacee9
Skip streamable artifacts during payload repair
maxhild Apr 30, 2026
ecfcc5f
Update generated documentation artifacts
maxhild Apr 30, 2026
aba8210
Pipeline and Frame Annotation debug
maxhild May 5, 2026
aa84228
Media integrity check to allow backfill
maxhild May 6, 2026
219d5a7
built docs assets
maxhild May 6, 2026
4b01526
Add annotation and validation workflow updates
maxhild May 7, 2026
5b6d9b6
Run pre-commit hooks through devenv
maxhild May 7, 2026
ebd2ef6
feat(ai): add dataset manifest settings page
maxhild May 8, 2026
1309362
feat(settings): export ai datasets by id
maxhild May 8, 2026
62e31a7
feat(training): add phi detector controls
maxhild May 8, 2026
a7c8455
chore(frontend): refresh generated store declarations
maxhild May 8, 2026
09396da
New explicit state models utilized in hub export
maxhild May 11, 2026
af19321
Filewatcher made threadsafe by importing signals first
maxhild May 11, 2026
a4a9f18
Add hub export k-anonymity summary
maxhild May 13, 2026
087f273
Show hub export k-anonymity metric
maxhild May 13, 2026
c7c6547
fix(hub-export): harden worker execution and privacy reporting
maxhild May 13, 2026
737fe4c
feat(ai-datasets): add dataset creation flow
maxhild May 13, 2026
5ca75cd
test(system): cover worker-thread imports
maxhild May 13, 2026
c4fe89f
fix(reporting): accept camelCase draft payloads
maxhild May 13, 2026
bcf736c
fix(annotation): normalize queue and segment payload state
maxhild May 13, 2026
8d02bfd
fix(anonymization): surface cleanup state and phi review links
maxhild May 13, 2026
c4c6348
feat(reporting): refresh the report editor workspace
maxhild May 13, 2026
a23dc54
docs: rebuild generated site artifacts
maxhild May 13, 2026
dbdbd49
Task safety in endoreg_db
maxhild May 13, 2026
b3ac831
Task safety in endoreg_db
maxhild May 13, 2026
4e30160
fix(stats): apply progress bar min height as style
maxhild May 18, 2026
e833986
chore(deps): bump endoreg-db to 0.9.9.6
maxhild May 18, 2026
ed3c8e1
feat(api): expose runtime quarantine overview
maxhild May 18, 2026
ab08d1d
feat(anonymization): show quarantined media in overview
maxhild May 18, 2026
8ecdbf0
feat(annotation): preserve dataset context for frame tasks
maxhild May 18, 2026
223bdfb
feat(video): track segment dataset and cleanup status
maxhild May 18, 2026
67605d9
feat(ai-datasets): attach annotations and scope training
maxhild May 18, 2026
5e5f6f2
feat(reporting): add report export page
maxhild May 18, 2026
c28e07e
docs(video): document segment job monitoring
maxhild May 18, 2026
200df58
chore(frontend): refresh generated declarations
maxhild May 18, 2026
a33a4fb
fix(reporting): type timeline href safely
maxhild May 18, 2026
dddb160
chore(frontend): refresh generated artifacts
maxhild May 18, 2026
bef3542
CUDA optimizations, new LX-Annotate with better jobs and monitoring b…
maxhild May 18, 2026
8592ded
chore(deploy): move deployment examples
maxhild May 18, 2026
abb2013
feat(api): add explicit endoreg and dtypes mounts
maxhild May 18, 2026
550a97c
docs(api): document frontend API contract
maxhild May 18, 2026
49f3426
style(app): improve sidebar toggle controls
maxhild May 18, 2026
68ac3ca
feat(stats): summarize annotation progress
maxhild May 18, 2026
0a410de
copy(anonymization): clarify import status labels
maxhild May 18, 2026
c6445e8
docs(frontend): clarify watcher import guide
maxhild May 18, 2026
e46bc2e
chore(deps): bump endoreg-db
maxhild May 18, 2026
bc349a0
copy(quarantine): localize quarantine reason
maxhild May 18, 2026
0d97ec8
chore(config): tidy secretspec sections
maxhild May 18, 2026
09c5bab
style(tests): apply ruff formatting
maxhild May 18, 2026
a0486fb
style(tests): format base api mount test
maxhild May 18, 2026
a36e31f
test(api): update frontend mount contract
maxhild May 18, 2026
59bc59b
test(frontend): update api helper and import label expectations
maxhild May 18, 2026
ed735b0
chore(frontend): refresh generated artifacts
maxhild May 18, 2026
864853a
Docs
maxhild May 18, 2026
1b22a63
Endoreg media operations now run using different workers
maxhild May 19, 2026
e04142d
feat(frontend): add anonymization metrics page
maxhild May 19, 2026
ad40fcd
feat(frontend): expose metrics in sidebar workflow
maxhild May 19, 2026
d28692c
feat(frontend): capture no-more-names confirmation
maxhild May 19, 2026
e17d106
style(frontend): tidy anonymization overview spacing
maxhild May 19, 2026
c78ecd8
chore(release): bump package metadata to 0.6.0
maxhild May 19, 2026
720cae7
migrations for endoreg db
maxhild May 20, 2026
748051d
nix package for lx-annotate
maxhild May 20, 2026
7feec46
migration made more compatible, stale test removed
maxhild May 29, 2026
061ce78
migration 13 made more compatible, stale test removed
maxhild May 29, 2026
3fee564
endoreg_db is strict pyright clean and uses dtypes contracts
maxhild Jun 12, 2026
f04e913
test extension, new migration of linked classifications to videos
maxhild Jun 30, 2026
809f8bf
test extension, new migration of linked classifications to videos
maxhild Jun 30, 2026
557d8ac
test extension, new migration of linked classifications to videos
maxhild Jun 30, 2026
46d525f
Stream with nginx back in env vars
maxhild Jun 30, 2026
25f4b4d
Version 0.9.20 - Large file operations ported to rust
maxhild Jul 1, 2026
b398555
raise default fps
Maddonix Jul 2, 2026
781824e
Raised the streamable migration limit
Maddonix Jul 2, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
Empty file added .codex
Empty file.
3 changes: 1 addition & 2 deletions .envrc
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
#!/usr/bin/env bash

# Original devenv direnvrc sourcing
eval "$(devenv direnvrc)"

use devenv
121 changes: 113 additions & 8 deletions .github/workflows/ci-cd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: CI/CD
on:
push:
branches:
- '**'
- "**"
pull_request:
workflow_dispatch:

Expand All @@ -14,6 +14,50 @@ concurrency:
group: ci-cd-${{ github.ref }}
cancel-in-progress: true

env:
PYTHON_VERSION: "3.12"
NODE_VERSION: "22"
DJANGO_SETTINGS_MODULE: lx_annotate.settings.settings_test
DJANGO_SETTINGS_MODULE_PRODUCTION: ${{ vars.DJANGO_SETTINGS_MODULE_PRODUCTION || 'lx_annotate.settings.settings_prod' }}
DJANGO_ENV: ${{ vars.DJANGO_ENV || 'production' }}
DJANGO_DEBUG: ${{ vars.DJANGO_DEBUG || 'False' }}
DJANGO_STATIC_ROOT: ${{ github.workspace }}/staticfiles
DJANGO_HOST: ${{ vars.DJANGO_HOST || '127.0.0.1' }}
DJANGO_PORT: ${{ vars.DJANGO_PORT || '8000' }}
DJANGO_ALLOWED_HOSTS: ${{ vars.DJANGO_ALLOWED_HOSTS || 'localhost,127.0.0.1' }}
DJANGO_CSRF_TRUSTED_ORIGINS: ${{ vars.DJANGO_CSRF_TRUSTED_ORIGINS || 'http://127.0.0.1' }}
DJANGO_CORS_ALLOWED_ORIGINS: ${{ vars.DJANGO_CORS_ALLOWED_ORIGINS || 'http://127.0.0.1' }}
DJANGO_DB_ENGINE: ${{ vars.DJANGO_DB_ENGINE || 'django.db.backends.sqlite3' }}
DJANGO_DB_NAME: ${{ vars.DJANGO_DB_NAME || ':memory:' }}
DJANGO_DB_USER: ${{ vars.DJANGO_DB_USER || 'lx_annotate' }}
DJANGO_DB_HOST: ${{ vars.DJANGO_DB_HOST || 'localhost' }}
DJANGO_DB_PORT: ${{ vars.DJANGO_DB_PORT || '5432' }}
DJANGO_SECRET_KEY: ${{ secrets.DJANGO_SECRET_KEY }}
DJANGO_DB_PASSWORD: ${{ secrets.DJANGO_DB_PASSWORD }}
OIDC_RP_CLIENT_ID: ${{ vars.OIDC_RP_CLIENT_ID || 'lx-annotate-frontend' }}
OIDC_RP_CLIENT_SECRET: ${{ secrets.OIDC_RP_CLIENT_SECRET }}
HTTP_PROTOCOL: ${{ vars.HTTP_PROTOCOL || 'https' }}
BASE_URL: ${{ vars.BASE_URL || 'http://127.0.0.1:8000' }}
STATIC_URL: ${{ vars.STATIC_URL || '/static/' }}
MEDIA_URL: ${{ vars.MEDIA_URL || '/media/' }}
TIME_ZONE: ${{ vars.TIME_ZONE || 'UTC' }}
CENTER_NAME: ${{ vars.CENTER_NAME || 'LX Annotate' }}
WORKING_DIR: ${{ github.workspace }}
HOME_DIR: ${{ github.workspace }}
STORAGE_DIR: ${{ github.workspace }}/storage
EXPORT_OUTPUT_DIR: ${{ github.workspace }}/exports
ASSET_DIR: ${{ github.workspace }}/assets
ENFORCE_AUTH: ${{ vars.ENFORCE_AUTH || '0' }}
RUN_VIDEO_TESTS: ${{ vars.RUN_VIDEO_TESTS || '0' }}
SKIP_EXPENSIVE_TESTS: ${{ vars.SKIP_EXPENSIVE_TESTS || '1' }}
VITE_API_BASE_URL: ${{ vars.VITE_API_BASE_URL || 'http://127.0.0.1:8000' }}
VITE_API_PREFIX: ${{ vars.VITE_API_PREFIX || 'api/' }}
VITE_FINDINGS_BACKEND: ${{ vars.VITE_FINDINGS_BACKEND || 'endoreg' }}
VITE_ENABLE_DEBUG: ${{ vars.VITE_ENABLE_DEBUG || 'false' }}
VITE_KEYCLOAK_URL: ${{ vars.VITE_KEYCLOAK_URL || 'https://keycloak.endo-reg.net' }}
VITE_KEYCLOAK_REALM: ${{ vars.VITE_KEYCLOAK_REALM || 'lx-realm' }}
VITE_KEYCLOAK_CLIENT_ID: ${{ vars.VITE_KEYCLOAK_CLIENT_ID || 'lx-annotate-frontend' }}

jobs:
backend-tests:
name: Backend Tests
Expand All @@ -29,17 +73,20 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
python-version: ${{ env.PYTHON_VERSION }}

- name: Set up uv
uses: astral-sh/setup-uv@v5

- name: Install runtime system dependencies
run: |
sudo apt-get update
sudo apt-get install -y ffmpeg tesseract-ocr tesseract-ocr-deu libgl1 libglib2.0-0 libxcb1

- name: Install backend dependencies
run: uv sync --dev
run: uv sync --frozen --extra dev

- name: Run backend tests
env:
DJANGO_SETTINGS_MODULE: lx_annotate.settings.settings_test
run: uv run pytest

frontend-tests:
Expand All @@ -53,23 +100,81 @@ jobs:
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
node-version: ${{ env.NODE_VERSION }}
cache: npm
cache-dependency-path: frontend/package-lock.json

- name: Install frontend dependencies
run: npm --prefix frontend ci

- name: Run frontend unit tests
run: npm --prefix frontend run test:unit -- --run

- name: Run frontend type-check
run: npm --prefix frontend run type-check

build-deployment:
name: Build Deployment Package
runs-on: ubuntu-latest
needs:
- backend-tests
- frontend-tests

steps:
- name: Check out repository
uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: recursive

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}

- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
cache: npm
cache-dependency-path: frontend/package-lock.json

- name: Set up uv
uses: astral-sh/setup-uv@v5

- name: Install runtime system dependencies
run: |
sudo apt-get update
sudo apt-get install -y ffmpeg tesseract-ocr tesseract-ocr-deu libgl1 libglib2.0-0 libxcb1

- name: Install backend dependencies
run: uv sync --frozen --extra dev

- name: Install frontend dependencies
run: npm --prefix frontend ci

- name: Build frontend staticfiles
run: npm --prefix frontend run build

- name: Verify Vite manifest
run: test -f staticfiles/.vite/manifest.json

- name: Build hatchling distribution
run: uv run python -m build

- name: Upload deployment artifacts
uses: actions/upload-artifact@v4
with:
name: lx-annotate-deployment-${{ github.sha }}
path: |
dist/*
staticfiles/**

push-to-prototype:
name: Push To prototype
runs-on: ubuntu-latest
needs:
- backend-tests
- frontend-tests
- build-deployment
if: github.event_name == 'push' && github.ref_name != 'prototype' && github.actor != 'github-actions[bot]'

steps:
Expand Down
Loading
Loading