shutil.SameFileError: '/geotunes/tune-tasks/geotune-cwtestft/config-train.yaml' and '/geotunes/tune-tasks/geotune-cwtestft/config-train.yaml' are the same file

[romeokienzler]

Describe the issue
shutil.SameFileError: '/geotunes/tune-tasks/geotune-cwtestft/config-train.yaml' and '/geotunes/tune-tasks/geotune-cwtestft/config-train.yaml' are the same file

/geotunes/tune-tasks/geotune-cwtestft/epoch=2-v1.ckpt
INFO:albumentations.check_version:A new version of Albumentations is available: 2.0.5 (you have 1.4.10). Upgrade using: pip install --upgrade albumentations
INFO: Seed set to 0
INFO:lightning.fabric.utilities.seed:Seed set to 0
/opt/miniforge/lib/python3.12/site-packages/terratorch/models/prithvi_model_factory.py:75: UserWarning: PrithviModelFactory is deprecated. Please switch to EncoderDecoderFactory.
warnings.warn("PrithviModelFactory is deprecated. Please switch to EncoderDecoderFactory.", stacklevel=1)
INFO:timm.models._builder:Loading pretrained weights from file (/terratorch/gfm_models/prithvi_swin_B/epoch-100-loss-0.0923.pt)
INFO:timm.models._helpers:Loaded from checkpoint '/terratorch/gfm_models/prithvi_swin_B/epoch-100-loss-0.0923.pt'
INFO:timm.models._builder:Missing keys (stages.0.norm.weight, stages.0.norm.bias, stages.1.norm.weight, stages.1.norm.bias, stages.2.norm.weight, stages.2.norm.bias, stages.3.norm.weight, stages.3.norm.bias) discovered while loading pretrained weights. This is expected if model is being adapted.
WARNING:timm.models._builder:Unexpected keys (mask_token, norm.weight, norm.bias, stages.0.blocks.1.attn_mask, stages.1.blocks.1.attn_mask, stages.2.blocks.1.attn_mask, stages.2.blocks.3.attn_mask, stages.2.blocks.5.attn_mask, stages.2.blocks.7.attn_mask, stages.2.blocks.9.attn_mask, stages.2.blocks.11.attn_mask, stages.2.blocks.13.attn_mask, stages.2.blocks.15.attn_mask, stages.2.blocks.17.attn_mask) found while loading pretrained weights. This may be expected if model is being adapted.
/opt/miniforge/lib/python3.12/site-packages/lightning/pytorch/cli.py:681: SemanticSegmentationTask.configure_optimizers will be overridden by MyLightningCLI.configure_optimizers.
INFO: Using 16bit Automatic Mixed Precision (AMP)
INFO:lightning.pytorch.utilities.rank_zero:Using 16bit Automatic Mixed Precision (AMP)
INFO: GPU available: True (cuda), used: True
INFO:lightning.pytorch.utilities.rank_zero:GPU available: True (cuda), used: True
INFO: TPU available: False, using: 0 TPU cores
INFO:lightning.pytorch.utilities.rank_zero:TPU available: False, using: 0 TPU cores
INFO: HPU available: False, using: 0 HPUs
INFO:lightning.pytorch.utilities.rank_zero:HPU available: False, using: 0 HPUs
/opt/miniforge/lib/python3.12/site-packages/urllib3/connectionpool.py:1063: InsecureRequestWarning: Unverified HTTPS request is being made to host 'gfm-mlflow-internal-nasageospatial-dev.cash.sl.cloud9.ibm.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
/opt/miniforge/lib/python3.12/site-packages/urllib3/connectionpool.py:1063: InsecureRequestWarning: Unverified HTTPS request is being made to host 'gfm-mlflow-internal-nasageospatial-dev.cash.sl.cloud9.ibm.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
/opt/miniforge/lib/python3.12/site-packages/urllib3/connectionpool.py:1063: InsecureRequestWarning: Unverified HTTPS request is being made to host 'gfm-mlflow-internal-nasageospatial-dev.cash.sl.cloud9.ibm.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
/opt/miniforge/lib/python3.12/site-packages/urllib3/connectionpool.py:1063: InsecureRequestWarning: Unverified HTTPS request is being made to host 'gfm-mlflow-internal-nasageospatial-dev.cash.sl.cloud9.ibm.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
🏃 View run Test tune 1 at: https://gfm-mlflow-internal-nasageospatial-dev.cash.sl.cloud9.ibm.com/#/experiments/1230/runs/878463fbe8a042888e30197ebc075053
🧪 View experiment at: https://gfm-mlflow-internal-nasageospatial-dev.cash.sl.cloud9.ibm.com/#/experiments/1230
/opt/miniforge/lib/python3.12/site-packages/urllib3/connectionpool.py:1063: InsecureRequestWarning: Unverified HTTPS request is being made to host 'gfm-mlflow-internal-nasageospatial-dev.cash.sl.cloud9.ibm.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
Traceback (most recent call last):
File "/opt/miniforge/bin/terratorch", line 8, in
sys.exit(main())
^^^^^^
File "/opt/miniforge/lib/python3.12/site-packages/terratorch/main.py", line 9, in main
_ = build_lightning_cli()
^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniforge/lib/python3.12/site-packages/terratorch/cli_tools.py", line 457, in build_lightning_cli
return MyLightningCLI(
^^^^^^^^^^^^^^^
File "/opt/miniforge/lib/python3.12/site-packages/lightning/pytorch/cli.py", line 396, in init
self._run_subcommand(self.subcommand)
File "/opt/miniforge/lib/python3.12/site-packages/lightning/pytorch/cli.py", line 706, in _run_subcommand
fn(**fn_kwargs)
File "/opt/miniforge/lib/python3.12/site-packages/lightning/pytorch/trainer/trainer.py", line 749, in test
return call._call_and_handle_interrupt(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniforge/lib/python3.12/site-packages/lightning/pytorch/trainer/call.py", line 47, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniforge/lib/python3.12/site-packages/lightning/pytorch/trainer/trainer.py", line 789, in _test_impl
results = self._run(model, ckpt_path=ckpt_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniforge/lib/python3.12/site-packages/lightning/pytorch/trainer/trainer.py", line 944, in _run
call._call_setup_hook(self) # allow user to set up LightningModule in accelerator environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniforge/lib/python3.12/site-packages/lightning/pytorch/trainer/call.py", line 103, in _call_setup_hook
_call_callback_hooks(trainer, "setup", stage=fn)
File "/opt/miniforge/lib/python3.12/site-packages/lightning/pytorch/trainer/call.py", line 222, in _call_callback_hooks
fn(trainer, trainer.lightning_module, *args, **kwargs)
File "/opt/miniforge/lib/python3.12/site-packages/terratorch/cli_tools.py", line 318, in setup
shutil.copyfile(self.config_path_original, self.config_path_new)
File "/opt/miniforge/lib/python3.12/shutil.py", line 240, in copyfile
raise SameFileError("{!r} and {!r} are the same file".format(src, dst))
shutil.SameFileError: '/geotunes/tune-tasks/geotune-cwtestft/config-train.yaml' and '/geotunes/tune-tasks/geotune-cwtestft/config-train.yaml' are the same file
An error occurred. Sending error notification...

[Joao-L-S-Almeida]

It happens when, at the end of a training loop, the input YAML file is copied to the log dir. The users have requested it since the YAML file saved by Lightning is not exactly equal to the input one. Maybe #481 will address this issue.

[romeokienzler]

fixed in #480 and #481