Skip to content

feat: updates #23

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: updates #23

wants to merge 1 commit into from

Conversation

cofin
Copy link
Member

@cofin cofin commented May 11, 2025

Description

Closes

Summary by Sourcery

Restructure and modularize application configuration, settings, and service layers, introduce new schema and service modules, and add comprehensive deployment tooling for Docker and Kubernetes environments.

New Features:

  • Add modular service and schema layers for accounts, teams, roles, and tags.
  • Introduce new configuration and environment variable utilities for flexible settings management.
  • Implement new Docker and Kubernetes deployment scripts and templates, including compose files and k8s manifests.
  • Add support for object storage and scratch volumes in deployment environments.

Enhancements:

  • Refactor settings and configuration into dedicated modules for improved maintainability.
  • Update service and controller patterns to use new dependency and provider utilities.
  • Improve environment variable parsing and type handling for configuration values.
  • Enhance database and logging configuration for better flexibility and observability.

Build:

  • Update Dockerfiles and compose files for both development and production workflows.
  • Add scripts and templates for Kubernetes deployment automation.

Documentation:

  • Add and update docstrings and inline documentation for new modules and configuration patterns.

Tests:

  • Add and update unit tests for new dependency and service provider utilities.

Copy link

sourcery-ai bot commented May 11, 2025

Reviewer's Guide

This pull request introduces a major refactor and modularization of the application's configuration, service, and schema layers, as well as significant improvements to environment variable handling, Docker/Kubernetes deployment, and code organization. The changes include moving and restructuring settings/configuration, introducing new service and schema modules, updating dependency injection patterns, enhancing environment variable parsing, and adding new deployment scripts and templates.

Sequence Diagram for System Health Check

sequenceDiagram
    actor Client
    participant SystemController
    participant Database

    Client->>SystemController: GET /health
    activate SystemController
    SystemController->>Database: Execute query (e.g., SELECT 1)
    activate Database
    Database-->>SystemController: Query Result (Success/Failure)
    deactivate Database
    alt Connection Successful
        SystemController-->>Client: HTTP 200 OK (SystemHealth: database_status='online')
    else Connection Failed
        SystemController-->>Client: HTTP 500 Internal Server Error (SystemHealth: database_status='offline')
    end
    deactivate SystemController
Loading

Updated Class Diagram for Settings Configuration

classDiagram
    class Settings {
        +app: AppSettings
        +db: DatabaseSettings
        +vite: ViteSettings
        +server: ServerSettings
        +saq: SaqSettings
        +log: LogSettings
        +storage: StorageSettings
        +from_env(dotenv_filename: str) Settings
    }
    note for Settings "RedisSettings removed, StorageSettings added.
from_env() logic updated to use dotenv and load individual settings components."

    class AppSettings {
        +NAME: str
        +VERSION: str
        +CONTACT_NAME: str
        +CONTACT_EMAIL: str
        +URL: str
        +DEBUG: bool
        +SECRET_KEY: str
        +JWT_ENCRYPTION_ALGORITHM: str
        +ALLOWED_CORS_ORIGINS: list[str] | str
        +CSRF_COOKIE_NAME: str
        +CSRF_HEADER_NAME: str
        +CSRF_COOKIE_SECURE: bool
        +STATIC_DIR: Path
        +STATIC_URL: str
        +BASE_URL: str | None
        +DEV_MODE: bool
        +ENABLE_INSTRUMENTATION: bool
        +GOOGLE_OAUTH2_CLIENT_ID: str
        +GOOGLE_OAUTH2_CLIENT_SECRET: str
        +ENV_SECRETS: str
        -OPENTELEMETRY_ENABLED: bool (removed)
        -LOGFIRE_ENABLED: bool (removed)
        -GITHUB_OAUTH2_CLIENT_ID: str (removed)
        -GITHUB_OAUTH2_CLIENT_SECRET: str (removed)
    }
    note for AppSettings "Fields now use get_env from app.utils.env for defaults."

    class DatabaseSettings {
        +ECHO: bool
        +ECHO_POOL: bool
        +POOL_DISABLED: bool
        +POOL_MAX_OVERFLOW: int
        +POOL_SIZE: int
        +POOL_TIMEOUT: int
        +POOL_RECYCLE: int
        +POOL_PRE_PING: bool
        +URL: str
        +MIGRATION_CONFIG: str
        +MIGRATION_PATH: str
        +MIGRATION_DDL_VERSION_TABLE: str
        +FIXTURE_PATH: str
        +get_engine() AsyncEngine
    }
    note for DatabaseSettings "URL default changed to Postgres.
get_engine() now uses app.utils.engine_factory.create_sqlalchemy_engine.
Fields now use get_env for defaults."

    class ViteSettings {
        +DEV_MODE: bool
        +USE_SERVER_LIFESPAN: bool
        +HOST: str
        +PORT: int
        +HOT_RELOAD: bool
        +ENABLE_REACT_HELPERS: bool
        +BUNDLE_DIR: Path
        +RESOURCE_DIR: Path
        +TEMPLATE_DIR: Path
        +ASSET_URL: str
    }
    note for ViteSettings "Default paths for BUNDLE_DIR, RESOURCE_DIR, ASSET_URL changed.
Fields now use get_env for defaults."

    class ServerSettings {
        +APP_LOC: str
        +HOST: str
        +PORT: int
        +KEEPALIVE: int
        +RELOAD: bool
        +RELOAD_DIRS: list[str]
        -HTTP_WORKERS: int | None (removed)
    }
    note for ServerSettings "Fields now use get_env for defaults."

    class SaqSettings {
        +PROCESSES: int
        +CONCURRENCY: int
        +WEB_ENABLED: bool
        +USE_SERVER_LIFESPAN: bool
    }
    note for SaqSettings "Fields now use get_env for defaults."

    class LogSettings {
        +EXCLUDE_PATHS: str
        +INCLUDE_COMPRESSED_BODY: bool
        +LEVEL: int
        +OBFUSCATE_COOKIES: set[str]
        +OBFUSCATE_HEADERS: set[str]
        +REQUEST_FIELDS: list[RequestExtractorField]
        +RESPONSE_FIELDS: list[ResponseExtractorField]
        +SAQ_LEVEL: int
        +SQLALCHEMY_LEVEL: int
        +ASGI_ACCESS_LEVEL: int
        +ASGI_ERROR_LEVEL: int
        -HTTP_EVENT: str (removed)
        -JOB_FIELDS: list[str] (removed)
        -WORKER_EVENT: str (removed)
        -UVICORN_ACCESS_LEVEL: int (removed)
        -UVICORN_ERROR_LEVEL: int (removed)
    }
    note for LogSettings "Significantly refactored. Fields now use get_env for defaults."

    class StorageSettings {
        <<new>>
        +PUBLIC_STORAGE_KEY: str
        +PUBLIC_STORAGE_URI: str
        +PUBLIC_STORAGE_OPTIONS: dict[str, Any]
        +PRIVATE_STORAGE_KEY: str
        +PRIVATE_STORAGE_URI: str
        +PRIVATE_STORAGE_OPTIONS: dict[str, Any]
    }
    note for StorageSettings "New class for storage configurations.
Fields use get_env for defaults."

    Settings *-- AppSettings
    Settings *-- DatabaseSettings
    Settings *-- ViteSettings
    Settings *-- ServerSettings
    Settings *-- SaqSettings
    Settings *-- LogSettings
    Settings *-- StorageSettings

    class env_utils {
      <<utility>>
      +get_env(key, default, type_hint) Callable
      +get_config_val(key, default, type_hint) any
    }
    AppSettings ..> env_utils : uses
    DatabaseSettings ..> env_utils : uses
    ViteSettings ..> env_utils : uses
    ServerSettings ..> env_utils : uses
    SaqSettings ..> env_utils : uses
    LogSettings ..> env_utils : uses
    StorageSettings ..> env_utils : uses
Loading

Class Diagram for New SystemController

classDiagram
    class SystemController {
        <<Controller>>
        +tags: list[str]
        +check_system_health(db_session: AsyncSession) Response[SystemHealth]
    }
    class SystemHealth {
        <<Schema>>
        +database_status: Literal['online', 'offline']
    }
    SystemController ..> SystemHealth : returns
    SystemController ..> AsyncSession : uses
Loading

Class Diagram for Renamed UserOAuthAccount Model

classDiagram
    class UserOauthAccount {
        <<model, old name>>
        # Attributes...
    }
    note for UserOauthAccount "This class was renamed."
    class UserOAuthAccount {
        <<model, new name>>
        +id: UUID (PK)
        +user_id: UUID (FK)
        +oauth_name: str
        +access_token: str
        +expires_at: int | None
        +refresh_token: str | None
        +account_id: str
        +account_email: str
        +created_at: DateTimeUTC
        +updated_at: DateTimeUTC
        +user: User (Relationship)
    }
    class User {
        <<model>>
        +oauth_accounts: list[UserOAuthAccount]
    }
    User "1" -- "0..*" UserOAuthAccount : has
Loading

File-Level Changes

Change Details Files
Refactored and modularized application configuration and settings management.
  • Moved settings logic from app/lib/settings.py to app/config/_settings.py and related modules.
  • Introduced app/config/_app.py and app/config/_utils.py for structured config and environment variable parsing.
  • Updated all imports and usage to reference new config locations.
app/lib/settings.py
app/config/_settings.py
app/config/_app.py
app/config/_utils.py
app/config.py
app/lib/log.py
app/lib/otel.py
app/db/utils.py
tests/conftest.py
tests/unit/lib/test_settings.py
Introduced new service and schema modules for users, teams, roles, tags, and related entities.
  • Added app/services/ with modular service classes for users, teams, roles, tags, OAuth accounts, etc.
  • Added app/schemas/ with structured msgspec-based and pydantic schemas for all major entities.
  • Updated controllers to use new service and schema modules.
app/services/_users.py
app/services/_teams.py
app/services/_roles.py
app/services/_tags.py
app/services/_user_roles.py
app/services/_user_oauth_accounts.py
app/services/_team_files.py
app/services/_team_invitations.py
app/services/_team_members.py
app/services/__init__.py
app/schemas/accounts.py
app/schemas/roles.py
app/schemas/tags.py
app/schemas/teams.py
app/schemas/base.py
app/schemas/system.py
app/schemas/__init__.py
Reorganized and updated dependency injection and provider patterns.
  • Replaced legacy dependency modules with new provider functions in app/domain/accounts/deps.py and app/server/deps.py.
  • Updated all controller dependencies to use new provider patterns and service factories.
  • Removed old dependency modules.
app/domain/accounts/deps.py
app/server/deps.py
app/domain/accounts/controllers.py
app/domain/teams/controllers.py
app/domain/tags/controllers.py
app/domain/accounts/dependencies.py
app/domain/tags/dependencies.py
app/domain/teams/dependencies.py
app/lib/dependencies.py
Enhanced environment variable and configuration parsing.
  • Introduced app/utils/env.py and improved get_env/get_config_val for robust type-safe env parsing.
  • Updated all settings and config classes to use new environment variable parsing functions.
app/utils/env.py
app/config/_settings.py
app/config/_utils.py
Added new Docker and Kubernetes deployment scripts and templates.
  • Added tools/deploy/docker/ and tools/deploy/k8s/ with Dockerfiles, docker-compose files, Kubernetes manifests, and deployment scripts.
  • Replaced old Dockerfiles and deployment scripts with new modular, environment-aware versions.
tools/deploy/docker/Dockerfile
tools/deploy/docker/Dockerfile.dev
tools/deploy/docker/dev/Dockerfile
tools/deploy/docker/docker-compose.yml
tools/deploy/docker/docker-compose.override.yml
tools/deploy/docker/docker-compose.infra.yml
tools/deploy/docker/minio-config.sh
tools/deploy/k8s/deploy.py
tools/deploy/k8s/templates/
Refactored and modularized API route/controller structure.
  • Moved and split API controllers into app/server/routes/ for access, user, team, tag, role, system, etc.
  • Updated imports and registration in application core.
app/server/routes/access.py
app/server/routes/user.py
app/server/routes/team.py
app/server/routes/tag.py
app/server/routes/roles.py
app/server/routes/system.py
app/server/routes/user_role.py
app/server/routes/team_member.py
app/server/routes/team_invitation.py
app/server/routes/__init__.py
Updated and improved database models and migrations.
  • Added TeamFile model and related relationships.
  • Updated Alembic migration scripts for new/changed models and indexes.
  • Refactored model imports and init.py organization.
app/db/models/team_file.py
app/db/models/__init__.py
app/db/models/team.py
app/db/models/tag.py
app/db/models/role.py
app/db/models/user.py
app/db/models/oauth_account.py
app/db/migrations/versions/2025-05-04_initial_0b6fd6502dbe.py
app/db/migrations/env.py
app/db/migrations/script.py.mako
Removed obsolete or replaced modules and tests.
  • Deleted old dependency, repository, and test modules that are now superseded by new structure.
app/domain/accounts/dependencies.py
app/domain/accounts/repositories.py
app/domain/tags/dependencies.py
app/domain/teams/dependencies.py
app/lib/dependencies.py
deploy/docker/Dockerfile.dev
deploy/docker/Dockerfile.distroless
tests/unit/lib/test_dependencies.py
Miscellaneous improvements and fixes.
  • Improved serialization utilities (app/utils/serialization.py).
  • Added DTO utilities (app/utils/dto.py).
  • Updated helper/test utilities and fixed minor bugs.
app/utils/serialization.py
app/utils/dto.py
tests/helpers.py
tests/unit/lib/test_deps.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've reviewed this pull request using the Sourcery rules engine. If you would also like our AI-powered code review then let us know.

Returns:
The ISO formatted datetime string.
"""
dt = dt.replace(tzinfo=datetime.UTC) if not dt.tzinfo else dt.astimezone(datetime.UTC)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (code-quality): Swap if/else branches of if expression to remove negation (swap-if-expression)

Suggested change
dt = dt.replace(tzinfo=datetime.UTC) if not dt.tzinfo else dt.astimezone(datetime.UTC)
dt = dt.astimezone(datetime.UTC) if dt.tzinfo else dt.replace(tzinfo=datetime.UTC)


ExplanationNegated conditions are more difficult to read than positive ones, so it is best
to avoid them where we can. By swapping the if and else conditions around we
can invert the condition and make it positive.

msg = f"Internal error: List parsing requested for key '{key}' but no item constructor determined."
raise RuntimeError(msg)

if parse_as_list and item_constructor:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (code-quality): We've found these issues:

Suggested change
if parse_as_list and item_constructor:
if parse_as_list:


Explanation
The quality score for this function is below the quality threshold of 25%.
This score is a combination of the method length, cognitive complexity and working memory.

How can you solve this?

It might be worth refactoring this function to make it shorter and more readable.

  • Reduce the function length by extracting pieces of functionality out into
    their own functions. This is the most important thing you can do - ideally a
    function should be less than 10 lines.
  • Reduce nesting, perhaps by introducing guard clauses to return early.
  • Ensure that variables are tightly scoped, so that code using related concepts
    sits together within the function rather than being scattered.

Comment on lines +229 to +230
for item in items:
constructed_list.append(item_constructor(item.strip()))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (code-quality): Replace a for append loop with list extend (for-append-to-extend)

Suggested change
for item in items:
constructed_list.append(item_constructor(item.strip()))
constructed_list.extend(item_constructor(item.strip()) for item in items)

Comment on lines +170 to +180
console.print("[bold red]Error executing kubectl command:[/bold red]")
console.print(f"Command: {' '.join(e.cmd)}")
console.print(f"Return Code: {e.returncode}")
console.print(f"Stderr:\n{e.stderr}")
console.print(f"Stdout:\n{e.stdout}")
# Re-raise the error if check was True, otherwise just report
if check:
raise
# Explicitly return the error object if check is False. Type checker might complain.
# Consider a more robust error handling strategy if needed.
return e # type: ignore
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (code-quality): Extract code out into function (extract-method)

Comment on lines +315 to +317
# Validate required secrets
missing = [key for key in REQUIRED_SECRETS if key not in os.environ]
if missing:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (code-quality): We've found these issues:

Suggested change
# Validate required secrets
missing = [key for key in REQUIRED_SECRETS if key not in os.environ]
if missing:
if missing := [key for key in REQUIRED_SECRETS if key not in os.environ]:


Explanation
The quality score for this function is below the quality threshold of 25%.
This score is a combination of the method length, cognitive complexity and working memory.

How can you solve this?

It might be worth refactoring this function to make it shorter and more readable.

  • Reduce the function length by extracting pieces of functionality out into
    their own functions. This is the most important thing you can do - ideally a
    function should be less than 10 lines.
  • Reduce nesting, perhaps by introducing guard clauses to return early.
  • Ensure that variables are tightly scoped, so that code using related concepts
    sits together within the function rather than being scattered.

Comment on lines +451 to +453
# Validate required secrets (for delete, warn if missing but do not exit)
missing = [key for key in REQUIRED_SECRETS if key not in os.environ]
if missing:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (code-quality): Use named expression to simplify assignment and conditional (use-named-expression)

Suggested change
# Validate required secrets (for delete, warn if missing but do not exit)
missing = [key for key in REQUIRED_SECRETS if key not in os.environ]
if missing:
if missing := [key for key in REQUIRED_SECRETS if key not in os.environ]:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant