Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More intensive tests for fit/inference #30

Closed
Joao-L-S-Almeida opened this issue Jun 29, 2024 · 1 comment · Fixed by #31
Closed

More intensive tests for fit/inference #30

Joao-L-S-Almeida opened this issue Jun 29, 2024 · 1 comment · Fixed by #31
Assignees

Comments

@Joao-L-S-Almeida
Copy link
Member

The problem with the tests using fit and inference pipelines is that not all the architectures have versions available on HuggingFace or can be slow to download during user-side tests. In order to circumvent that I suggest:

  • To automatically create manufactured (fake) checkpoints to be used together with template YAML configs.
  • These checkpoints store randomly generated weights, just used to test the execution via command line.
  • The checkpoints generation and command line tests can be automatically executed via GitHub Actions.
  • The manufactured checkpoints can be used to perform equally manufactured fine-tuning/prediction processes for a few epochs, just to guarantee the toolkit general usability for those applications.

Any suggestion @CarlosGomes98 and @biancazadrozny ?

@Joao-L-S-Almeida Joao-L-S-Almeida self-assigned this Jun 29, 2024
@Joao-L-S-Almeida Joao-L-S-Almeida linked a pull request Jul 1, 2024 that will close this issue
@Joao-L-S-Almeida
Copy link
Member Author

Please check the PR attached to this issue in order to see if the proposal was covered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant