https://github.com/konstmish/prodigy and based on our 2.1 alpha FFusion/di.FFUSION.ai-v2.1-768-BaSE-alpha in under 15 min on 4090 with just 1 rep and 2 epoch… ha :D… 🤟 🥃

–v2 --v_parameterization --enable_bucket --pretrained_model_name_or_path=“FFusion/di.FFUSION.ai-v2.1-768-BaSE-alpha” --resolution=“1024,1024” --network_alpha=“64” --training_comment=“di.FFusion.ai” --save_model_as=safetensors --network_module=networks.dylora --network_args conv_dim=“64” conv_alpha=“64” unit=“2” rank_dropout=“0.04” module_dropout=“0.02” --text_encoder_lr=1.0 --unet_lr=1.0 --network_dim=64 --output_name=“cyberbunnyfusion11” --lr_scheduler_num_cycles=“2” --scale_weight_norms=“1” --network_dropout=“0.1” --learning_rate=“1.0” --lr_scheduler=“linear” --lr_warmup_steps=“20” --train_batch_size=“4” --save_every_n_epochs=“1” --mixed_precision=“fp16” --save_precision=“fp16” --caption_extension=“.txt” --cache_latents --optimizer_type=“prodigy” --optimizer_args safeguard_warmup=True --max_token_length=225 --clip_skip=2 --keep_tokens=“2” --vae_batch_size=“2” --bucket_reso_steps=64 --min_snr_gamma=4 --save_state --xformers --bucket_no_upscale --scale_v_pred_loss_like_noise_pred --noise_offset=0.12 --adaptive_noise_scale=0.01

planning to test this one on a larger scale soon along with CosineAnnealingLR