Skip to content

Add logging to file with MLflow integration #141

@loren-ac

Description

@loren-ac

Description

Currently, logs only output to stdout. We should add the ability to log to a file
and automatically save that log file to MLflow for better debugging and analysis.

Context

When runs encounter errors or need debugging, having a complete log file saved in
MLflow would be valuable for post-mortem analysis and tracking training history.

Requirements

  1. Configure Python's logging library to write to both stdout and a file

    • Research the logging library's configuration for multiple handlers
    • Add a file handler in addition to the existing stream handler
  2. Add log file path configuration

    • Option 1: Add a log_file_path parameter to the run configuration
    • Option 2: Use a default path convention (e.g.,
      logs/{experiment_name}/{run_name}.log)
  3. Integrate with simplexity run manager

    • Ensure log file is saved to MLflow in the cleanup/teardown phase
    • Include log file upload even when runs fail/error out
    • Log file should be available as an MLflow artifact

Acceptance Criteria

  • Logs are written to both stdout and a file simultaneously
  • Log file path is configurable (either via config or using sensible defaults)
  • Log files are automatically uploaded to MLflow as artifacts
  • Log files are captured and uploaded even when runs fail with errors
  • Documentation updated to explain log file configuration options

Notes

The log file path needs to be known by the run manager's cleanup function to ensure
upload happens reliably.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions