Hardcoded 'preprocessed_2500' in file path ignores '--segment_len' argument

[akurukunda01]

The base preprocessing stage writes its output to a folder derived from the --segment_len argument (default 2500, defined in src/configs/configs.py):

self.save_dir = f"{DATA_DIR}/{self.args.base}/preprocessed_{self.args.segment_len}"

So --segment_len 1250 writes to {DATA_DIR}/mimic_iv/preprocessed_1250/.

Problem

The tokenizer reads from a path with 2500 hardcoded, in two places:

# src/ecg_tokenizer/build_ecg_tokenizer.py

#Line 69 
file_paths = glob.glob(f"{DATA_DIR}/mimic_iv/preprocessed_2500/*.npy")

# Line 96 
file_paths = glob.glob(f"{DATA_DIR}/mimic_iv/preprocessed_2500/*.npy")

With any non-default --segment_len, the folder the tokenizer reads from does not exist:

ValueError: No file paths found to process.

Solution

Build the path from the argument, as base_dataset.py already does in both methods:

file_paths = glob.glob(f"{DATA_DIR}/mimic_iv/preprocessed_{self.args.segment_len}/*.npy")