Noticable slow down with recent code update

[deepfree2023]

GPU: 4070 ti super
Before: about 1.0s/it, higher vram usage, possible vram allocation error
After: about 1.4s/it, lower vram usage, no vram allocation error

[smthemex]

I forcibly enabled CPU offload. If necessary, I will change it back to optional.