Can u give some toy data demo to run training scripts?

[xiao2mo]

Thanks a lot.

[yujincheng08]

if u need a quick trial, try to use the diffusers pipeline for training.

[kopyl]

Why you can't load a dataset from Hugginface? In the same format which would be needed for SD 1.5 fine-tune.

[lawrence-cj]

May I have a doc for what u said so that I can follow up?

[kopyl]

@lawrence-cj are you replying to me?

Check this:
https://github.com/huggingface/diffusers/tree/main/examples/text_to_image

You can train a model like this:

export MODEL_NAME="CompVis/stable-diffusion-v1-4"
export DATASET_NAME="lambdalabs/pokemon-blip-captions"

accelerate launch --mixed_precision="fp16"  train_text_to_image.py \
  --pretrained_model_name_or_path=$MODEL_NAME \
  --dataset_name=$DATASET_NAME \
  --use_ema \
  --resolution=512 --center_crop --random_flip \
  --train_batch_size=1 \
  --gradient_accumulation_steps=4 \
  --gradient_checkpointing \
  --max_train_steps=15000 \
  --learning_rate=1e-05 \
  --max_grad_norm=1 \
  --lr_scheduler="constant" --lr_warmup_steps=0 \
  --output_dir="sd-pokemon-model" 

with a dataset created like this:
https://huggingface.co/docs/datasets/image_dataset

Which looks like this (my dataset): https://huggingface.co/datasets/kopyl/833-icons-dataset-1024-blip-large

[lawrence-cj]

@lawrence-cj are you replying to me?

Oh yah, I thought you are talking to me. lol

[kopyl]

@lawrence-cj yeah, to you. But I'm not sure who are you replying to without mentioning anyone.

[kopyl]

@wen020 i also want to try to fine-tune the 512 model

[lawrence-cj]

Toy images added. closed

[kopyl]

@lawrence-cj thank you very much :)