Saves Quantized Model to disk for loading next time via "--int8"#61
Open
petermg wants to merge 5 commits intobytedance:mainfrom
Open
Saves Quantized Model to disk for loading next time via "--int8"#61petermg wants to merge 5 commits intobytedance:mainfrom
petermg wants to merge 5 commits intobytedance:mainfrom
Conversation
saves quantized model to disk when using "python app.py --int8", then on next load it will load the quantized model from the disk so that it no longer has to quantize it every launch. You still need to use "--int8" when running to tell it to use the quantized version.
Modified from original: Saves quantized models to disk Loads quantized models from disk if found, so no need to quantize every run. support for LoRAs Added the ability to specify number of images to generate per run. Exposed and/or added the following options in the UI: "Face Upscale Factor", "Face Crop Size", "resolution for ref image", "Neg Prompt", and some others that were previously hidden in the "Advanced Options" accordion.
This was referenced May 24, 2025
|
I gave this a go on a hosted VM but the code still beligerantly tries to download full Flux.1.Dev models from HuggingFace. I tried to remove the I tried to use a pretrained int8 model using |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
saves quantized model to disk when using "python app.py --int8", then on next load it will load the quantized model from the disk so that it no longer has to quantize it every launch. You still need to use "--int8" when running to tell it to use the quantized version.
Also opened up a few other options in the UI.