release: 2026.04.1#77
Merged
Merged
Conversation
Defines scope, source precedence, per-backend report structure, and index for auditing 11 backend settings classes against their authoritative driver/Ibis/vendor specs. Report-only; fixes handled in separate per-backend cycles. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
13 tasks: one per backend (11) + pyiceberg_rest + index summary fill-in. Each task is self-contained with concrete file paths, source URLs tagged by precedence, and report schema references. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Descriptor-based backend registry with typed AuthSpec discriminated union and optional per-backend adapter for composite driver mappings. Consumes audit findings (docs/superpowers/specs/2026-04-15-settings-audit/) to guide parameter tiering and bug fixes carried into the refactor. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
22-task plan covering Phases 1-3 of the registry redesign: scaffolding (ConnectionProfile + descriptor + AuthSpec union + registry), per-backend migrations in cheap->hard order carrying audit fixes, and consumer-side call-site updates. Phase 4 (residual per-backend audit sweeps) is intentionally deferred to follow-up plans. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add the core primitive data structures for the settings registry refactor: - MISSING sentinel for required (no-default) fields - ParameterSpec: frozen dataclass describing one backend field - BackendDescriptor: frozen dataclass describing a complete backend These are plain frozen dataclasses with no pydantic or runtime behavior, establishing the foundation for descriptor-driven settings configuration. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Remove unused `field` import from descriptor.py - Remove unused `Literal` and `Optional` imports from test file - Tighten pytest.raises(Exception) to FrozenInstanceError for freeze tests - Replace tautology test_parameter_spec_tier_must_be_valid with: - test_parameter_spec_accepts_advanced_tier (positive test) - test_missing_sentinel_is_falsy_and_singleton (MISSING behavior test) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Add 11 auth variants covering all authentication patterns needed across database backends (none, password, token, oauth2, service account, IAM, Windows, Azure AD, Kerberos, certificate). Each variant is a frozen pydantic model with a discriminator kind field, enabling clean type-safe union composition in downstream connection profiles. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Drop kind: str from AuthSpec base (fixes Pyright override warnings) - Add frozen + extra=forbid contract tests - Remove unused AuthSpec import in test file Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Document _iam empty-dict return (ambient AWS credentials) - Add OAuth2 token-wins, OAuth2 empty, IAM empty regression tests Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Introduces ConnectionProfile, a pydantic v2 base class that uses __pydantic_init_subclass__ to materialize BackendDescriptor parameters and auth_modes into validated pydantic fields at subclass definition time, wiring up to_driver_kwargs / to_connection_string generically. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Clarify to_driver_kwargs adapter contract (adapter owns the pipeline) - URL-encode username/password in to_connection_string - Document the MountainAshBaseSettings setattr bypass - Add tests: adapter-replaces, adapter-fresh, transform, URL-encoded password - Source adapter via __dict__ lookup (fixes Pyright call-site arity) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Rewrite SQLiteAuthSettings as a two-line shell subclass of ConnectionProfile - Replace old BaseDBAuthSettings inheritance with descriptor-driven config - Add SQLITE_DESCRIPTOR with auth, parameters (DATABASE, TYPE_MAP), and ibis mapping - Register SQLITE_DESCRIPTOR via @register decorator - Add comprehensive test suite: test_minimal_construction, test_database_memory, test_to_driver_kwargs_memory, test_to_driver_kwargs_none_database_dropped, test_type_map_optional - Verify 10 parametric descriptor invariants for sqlite pass via registry This is the template migration for all 11 subsequent backends (Tasks 8-18). Legacy consumer tests in tests/test_unit/databases/ fail as expected; Task 20 updates all call sites to use the new ConnectionProfile API.
…ixes - Rewrite mssql.py with MSSQLDriver/MSSQLEncryption StrEnum and registry decorator - Add mssql.py adapter with comprehensive auth dispatch (Password/Windows/AzureAD) - Instance-name folding (host\instance), encryption flags, MARS support - Test audit fixes: AZURE_MANAGED_IDENTITY/MSI_ENDPOINT, args['server'] KeyError - 5 new specific tests + 10 invariant tests (all passing, no __setattr__ override needed) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…egion regex Task 17: Redshift settings migration complete. Replaced BaseDBAuthSettings with descriptor-driven ConnectionProfile. Added RedshiftSSLMode StrEnum (default VERIFY_FULL). Widened region regex to accept GovCloud (us-gov-west-1). Widened role ARN regex to accept non-commercial partitions (arn:aws-us-gov:, arn:aws-cn:). Implemented IAMAuth/PasswordAuth adapter that properly routes AWS credentials and session tokens. Plumbed CLUSTER_READ_ONLY and WORKGROUP_NAME. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Task 20 of the settings-registry refactor. All call sites outside settings/ now use the new ConnectionProfile API: to_driver_kwargs(), to_connection_string(), and provider_type. Key changes: - core/connection.py: BaseDBAuthSettings → ConnectionProfile, db_provider_type → provider_type, get_connection_kwargs delegates to to_driver_kwargs() with legacy fallback - backends/ibis/connection.py: provider_type rename (12 subclasses), new connect_default code path uses ibis.<dialect>.connect(**driver_kwargs) for ConnectionProfile instances, filters empty lists (e.g. extensions=[]) that ibis rejects - backends/iceberg/connection.py: provider_type rename, connect_default uses to_driver_kwargs() for ConnectionProfile instances - core/factories/settings_factory.py: auto-injects auth=NoAuth() for NoAuth-only ConnectionProfile subclasses in from_backend_type() - All test files: BaseDBAuthSettings → ConnectionProfile imports, auth=NoAuth() added to all SettingsParameters.create() kwargs that call connect() All 483 unit tests pass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The validator field on ParameterSpec was declared but never consumed by ConnectionProfile.__pydantic_init_subclass__. Now wired via pydantic v2's Annotated[type, AfterValidator(fn)] pattern. This enables descriptor-level validation (rejection) without needing explicit @field_validator(check_fields=False) on each shell class. Backends like DuckDB, BigQuery, and Redshift that already declared validator= on their ParameterSpecs now get automatic validation. Note: due to MountainAshBaseSettings' setattr-bypass path, validators work for rejection but not for value transformation — the raw kwarg overwrites the validated value post-construction. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
9 tasks: rewire mountainash-data to import descriptor/profile/auth machinery from mountainash-settings.profiles + mountainash-settings.auth, replace module-level REGISTRY with per-domain DATABASES_REGISTRY, preserve external API byte-for-byte, collapse test_descriptors_invariants to the one-line shared helper. Depends on Phase 1 (profiles-scaffolding) being merged and released. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
refactor(settings): descriptor-based registry — 12 backends migrated
Replace the fat local ConnectionProfile/registry/descriptor/auth stack with thin wrappers over mountainash_settings.profiles primitives. Deletes the local descriptor.py (BackendDescriptor, ParameterSpec, MISSING) and the entire auth/ sub-package; those now live in the upstream mountainash-settings library. Introduces DATABASES_REGISTRY (Registry instance) and updates register/ get_descriptor/get_settings_class to delegate to it. Adds a _RegistryDictView backwards-compat alias for existing REGISTRY imports. Intentionally breaks imports across the 12 per-backend files — Task 3 fixes them. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… auth - Adapter files: `from mountainash_data.core.settings.auth import` → `from mountainash_settings.auth import`; rename `_default_driver_kwargs()` → `_default_kwargs()` and `_auth_to_driver_kwargs()` → `_auth_kwargs()` across 7 adapter files. - Create `settings/descriptor.py` with typed `BackendDescriptor` subclass (extends `ProfileDescriptor` with `default_port`, `connection_string_scheme`, `ibis_dialect`, `rides_on` fields) so per-backend files keep typed field access. - Per-backend files: import now routes through `.descriptor` → `mountainash_settings.profiles`. - `settings/auth/` compatibility shim package: re-exports all auth primitives from `mountainash_settings.auth` so existing `from mountainash_data.core.settings.auth import X` call sites continue to work. - `registry.py`: expose `_snapshot_for_tests` / `_reset_for_tests` module-level wrappers delegating to `DATABASES_REGISTRY`. - `settings/__init__.py`: auth re-exports now pull from `mountainash_settings.auth` directly. Result: 234 passed, 1 skipped (test_adapter_replaces_pipeline_output uses old `_default_driver_kwargs()` name in test adapter — Task 7 territory). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Remove test_auth.py and test_auth_dispatch.py (coverage moved to mountainash-settings). Rewrite descriptor/profile/registry tests to cover only data-specific behaviour (BackendDescriptor fields, to_driver_kwargs(), to_connection_string(), DATABASES_REGISTRY wrapper). Fixes 1 failing test that called the removed _default_driver_kwargs(). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…otion Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat(settings): Phase 2 — migrate to mountainash-settings.profiles + auth
The override coerced MODE strings to PySparkMode enum because the pre-fix parent __init__ called update_settings_from_dict() which used raw setattr, bypassing pydantic validators. As of mountainash-settings v26.4.1, MountainAshBaseSettings sets model_config["validate_assignment"] = True and the redundant update_settings_from_dict call in __init__ is gone. Every setattr on an instance — including in __init__, SettingsManager runtime overrides, and apply_runtime_overrides — now runs the field's pydantic validator pipeline, which handles StrEnum coercion natively. Verified in dev env with mountainash-settings 26.4.2: both PySparkAuthSettings(MODE='batch', ...) and s.MODE = 'streaming' produce PySparkMode enum instances without this override. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Zero imports in src/ or tests/. Superseded by ConnectionProfile.to_connection_string() on the new descriptor pattern. Flagged in the mountainash-data legacy-cleanup backlog as safe-to-delete. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
chore(settings): retire PySpark __setattr__ override and orphaned templates.py
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Production release cut from
develop. First production bump since the 2026.04.0 RC.Changes since last main merge (2026.04.0 RC)
feat/settings-registry— descriptor + registry + ConnectionProfile refactorfeat/profiles-migration— rewire imports tomountainash-settings.profiles+auth(upstream primitives from settings-promotion Phase 1)chore/legacy-cleanup— retirePySparkAuthSettings.__setattr__override (redundant after settings v26.4.1validate_assignment=True) + delete orphanedcore/settings/templates.pyDo not publish this release until
mountainash-settingsPR #36 (v26.4.2 hotfix) merges and is released. Pre-v26.4.2,import mountainash_settingscrashes in any venv without pytest installed (module-topimport pytestinprofiles/invariants.py). Everymountainash_dataproduction deploy imports settings transitively, so the hotfix must ship first or prod deploys break on import.Test plan
mountainash-settingsv26.4.2 merged + taggedhatch run test:test tests/test_unit/green on this branchPost-merge
v2026.04.1🤖 Generated with Claude Code