underutilizing GPU?

[ppbrown]

Apparently taggui is only using 25% of my 4090 GPU. (and half the ram)

Wouldnt allowing some kind of batch mode speed things up?
But I see no option for that currently.

I would really like to make my 5 day run take 1 day instead.

This is with internlmvl-7b-4bit, if it makes a difference.

[jhc13]

Wouldnt allowing some kind of batch mode speed things up?

Yes, but it is difficult to make it work for all models, so it has not been added.

[ppbrown]

Some models dont really need it anyway since they are fast already, like moondream and blip/clip. Pllleeease add it for the ones people are most likely to use for large scale at home? You do have certain other features that only work for specific models now anyway.... ? Message ID: ***@***.***>

…

[LoFiApostasy]

I just launch a 2nd instance of the app and process a different folder.

[ppbrown]

but that would spawn an entirely new instance which would require loading the full model TWICE. some people dont even have enough ram for that. imagine having to load two separate instances of comfyui or A111, on the same model, instead of just telling the programs “batch size =2”?? comfyui at least, makes it so that you get faster renders if you specify batch size >1

…

On Mon, Feb 3, 2025 at 9:37 PM LoFiApostasy ***@***.***> wrote: I just launch a 2nd instance of the app and process a different folder. — Reply to this email directly, view it on GitHub <#328 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANEV6JZYMP45L7BTH7X5NL2OBGZFAVCNFSM6AAAAABV3UVBY6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMZSHEZDGNBZHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>