Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem downloading S3 bucket #746

Closed
mahmoodn opened this issue Jun 5, 2024 · 0 comments
Closed

Problem downloading S3 bucket #746

mahmoodn opened this issue Jun 5, 2024 · 0 comments

Comments

@mahmoodn
Copy link

mahmoodn commented Jun 5, 2024

Hi,
I followed the instructions in the large language model to download the S3 bucket, but apparently that doesn't work.

$ rclone config create mlc-training s3 provider=Cloudflare access_key_id=76ea42eadb867e854061a1806220ee1e secret_access_key=a53625c4d45e3ca8ac0df8a353ea3a41ffc3292aa25259addd8b7dc5a6ce2936 endpoint=https://c2686074cb2caf5cbaf6d134bdba8b47.r2.cloudflarestorage.com
[mlc-training]
type = s3
provider = Cloudflare
access_key_id = 76ea42eadb867e854061a1806220ee1e
secret_access_key = a53625c4d45e3ca8ac0df8a353ea3a41ffc3292aa25259addd8b7dc5a6ce2936
endpoint = https://c2686074cb2caf5cbaf6d134bdba8b47.r2.cloudflarestorage.com

$ rclone copy mlc-training:mlcommons-training-wg-public/gpt3/megatron-lm/dataset_c4_spm.tar ./ -P
2024/06/05 15:34:21 NOTICE: Time may be set wrong - time from "c2686074cb2caf5cbaf6d134bdba8b47.r2.cloudflarestorage.com" is 1h52m37.145836609s different from this computer
2024/06/05 15:34:21 ERROR : S3 bucket mlcommons-training-wg-public path gpt3/megatron-lm/dataset_c4_spm.tar: error reading source root directory: RequestTimeTooSkewed: The difference between the request time and the server's time is too large.
        status code: 403, request id: , host id:
2024/06/05 15:34:21 ERROR : Attempt 1/3 failed with 1 errors and: RequestTimeTooSkewed: The difference between the request time and the server's time is too large.
        status code: 403, request id: , host id:
2024/06/05 15:34:21 ERROR : S3 bucket mlcommons-training-wg-public path gpt3/megatron-lm/dataset_c4_spm.tar: error reading source root directory: RequestTimeTooSkewed: The difference between the request time and the server's time is too large.
        status code: 403, request id: , host id:
2024/06/05 15:34:21 ERROR : Attempt 2/3 failed with 1 errors and: RequestTimeTooSkewed: The difference between the request time and the server's time is too large.
        status code: 403, request id: , host id:
2024/06/05 15:34:21 ERROR : S3 bucket mlcommons-training-wg-public path gpt3/megatron-lm/dataset_c4_spm.tar: error reading source root directory: RequestTimeTooSkewed: The difference between the request time and the server's time is too large.
        status code: 403, request id: , host id:
2024/06/05 15:34:21 ERROR : Attempt 3/3 failed with 1 errors and: RequestTimeTooSkewed: The difference between the request time and the server's time is too large.
        status code: 403, request id: , host id:
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         1.3s
2024/06/05 15:34:21 Failed to copy: RequestTimeTooSkewed: The difference between the request time and the server's time is too large.
        status code: 403, request id: , host id:

I used the same rclone config command as the readme file says. I am not sure if I have to do anything more or not. Any thoughts on that?

@mahmoodn mahmoodn closed this as completed Jun 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant