Skip to content

fix(database): migrate valid_start_mjd / valid_end_mjd columns in caltables (fixes #102) - #130

Open
jakobtfaber wants to merge 1 commit into
mainfrom
devin/1784115901-issue-102
Open

fix(database): migrate valid_start_mjd / valid_end_mjd columns in caltables (fixes #102)#130
jakobtfaber wants to merge 1 commit into
mainfrom
devin/1784115901-issue-102

Conversation

@jakobtfaber

Copy link
Copy Markdown
Contributor

Summary

The caltables schema in dsa110_continuum/database/unified.py has included valid_start_mjd / valid_end_mjd in the CREATE TABLE for new databases, but the migration function _migrate_caltables_schema only added the provenance text columns (source_ms_path, solver_command, etc.). Existing pipeline.sqlite3 databases created before those columns therefore raised no such column: valid_start_mjd when the pipeline tried to register generated calibration tables (#102).

Changes

  • dsa110_continuum/database/unified.py: add valid_start_mjd and valid_end_mjd to the new_columns list in _migrate_caltables_schema, guarded by the existing PRAGMA table_info probe. The CREATE TABLE and idx_caltables_valid index code are unchanged; the index is still created in ensure_calibration_database after the migration.

Verification

  • make test-cloud PYTHON=/home/ubuntu/.venv-dsa110-312/bin/python passes (200 passed).
  • ruff check dsa110_continuum/database/unified.py reports five pre-existing style/name issues in the file that are unrelated to this change (I001, F821 at query_df, and W291/W293 on migration SQL strings); I did not bulk-fix them per repo guidance.
  • H17 validation pending: confirm that a real pipeline.sqlite3 with the old caltables schema successfully runs _migrate_caltables_schema and caltable registration no longer logs no such column: valid_start_mjd.

Closes #102.

…tables (fixes #102)

Co-Authored-By: Jakob Faber <jfaber@caltech.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Live pipeline.sqlite3 caltables table predates valid_start_mjd — provenance tracking dead for generated tables

1 participant