Update hf-bitsandbytes-integration.md #1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Great work, as always, Younes! I really like the additions. It makes the procedure and how you integrated it into the HF libraries very tangible. One major change I think needs to be made is the example about how to use a PyTorch module with bitsandbytes in Int8. One thing is, that one can train in any precision and also store the checkpoint in any precision. FP16 is not required. What is required is to use FP16 inputs into the module once it is converted with bitsandbytes. This is so, because bnb casts any model to FP16, but it will not cast the inputs to FP16 (the user has to do this). Otherwise, I made some small edits here and there. I will add the link to the paper once it is announced in the evening.
|
Thanks a lot Tim for your comments and just integrated the majority of them! |
younesbelkada
pushed a commit
that referenced
this pull request
Nov 14, 2022
* Empty-Commit * Review (#1) * add: initial post. * add: colab notebook. * add: rest of the files including figures. * fix wrong image3 * pr feedback. * remove > character. * add: thumbnail Co-authored-by: Chansung <[email protected]> Co-authored-by: Chansung Park <[email protected]> * Update deploy-vertex-ai.md * remove gcp project id * minor updates to the notebook and draft. * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * Update deploy-vertex-ai.md Co-authored-by: Steven Liu <[email protected]> * pricing: purpose -> estimate the cost of usage * minor changes in the conclusion. * change: publication date. * addresses matt's feedback shared via Slack * address matt's feedback on Colab (shared via Slack) * make pricing section shorter * update publication date * turn steps into list form * update _blog.yml Co-authored-by: Chansung Park <[email protected]> Co-authored-by: Steven Liu <[email protected]>
younesbelkada
pushed a commit
that referenced
this pull request
Mar 8, 2023
… (huggingface#761) * added article first pass iterated on article added images smaller images smaller image smaller image 2 added embedded video expanded article simplified text small tweaks small tweaks next article pass another pass, added images updated images added thumbnail to article removed embedded thumbnail small tweaks tweaks typo fix grammar Update _blog.yml Co-authored-by: Thomas Simonini <[email protected]> Update gamedev-5days-1.md Co-authored-by: Thomas Simonini <[email protected]> Update gamedev-5days-1.md Co-authored-by: Thomas Simonini <[email protected]> revisions from thomas suggestions Change title Co-authored-by: Omar Sanseviero <[email protected]> Grammar Co-authored-by: Omar Sanseviero <[email protected]> Change Title Co-authored-by: Omar Sanseviero <[email protected]> Added context sentence Co-authored-by: Omar Sanseviero <[email protected]> Typo Co-authored-by: Omar Sanseviero <[email protected]> * renamed to ml-for-games, added diffusers reference moved images to documentation-images fixed broken image links resolved conflict, updated _blog.yml fixed spacing fixed broken image links * updated _blog title * updated tiktok link Co-authored-by: Omar Sanseviero <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Great work, as always, Younes!
I really like the additions. It makes the procedure and how you integrated it into the HF libraries very tangible.
One major change I think needs to be made is the example about how to use a PyTorch module with bitsandbytes in Int8. One thing is, that one can train in any precision and also store the checkpoint in any precision. FP16 is not required. What is required is to use FP16 inputs into the module once it is converted with bitsandbytes. This is so, because bnb casts any model to FP16, but it will not cast the inputs to FP16 (the user has to do this).
Otherwise, I made some small edits here and there. I will add the link to the paper once it is announced in the evening.