bug: Error with botocore dependency, not reproducible

[D3SL]

Describe the bug

I have a flow that repeatedly runs 7 python scripts in parallel (branch-to-all inside of a for-loop). Today 2/7 scripts gave errors related to botocore during a single loop iteration.

One error was ModuleNotFoundError for botocore.vendored, the other was DataNotFoundError for endpoints. When I tried to rerun the exact same scripts, inputs, and source data manually everything worked fine. Some googling suggests the former is tied to the installation method of boto3 and aws-cli, and the latter may have something to do with json files boto3 uses in its directory.

To reproduce

The endpoints error was triggered by this code:

    bucket = wmill.get_resource("f/storage/garage_s3")["bucket"]
    s3 = s3fs.S3FileSystem(**wmill.polars_connection_settings().s3fs_args)
    storage_options = wmill.polars_connection_settings().storage_options

    dataframe= pl.read_parquet("s3://{}/{}".format(bucket, input_s3_obj["s3"]),
        storage_options=storage_options
    )

I'm not certain but I think the ModuleNotFoundError was caused by importing s3fs.

Expected behavior

The script to run and be able to read and write parquet files from an S3 compatible system.

Screenshots

No response

Browser information

No response

Application version

CE v1.458.3

Additional Context

Logs:
botocore error unable to load endpoints.json
windmill_logs_0194f3ea-cdba-ce82-9f9a-02649c7cf265.txt