OneTrainer is a comprehensive training solution catering to stable diffusion models, including Stable Diffusion 1.5, 2.0, 2.1, SDXL, Würstchen-v2, and inpainting models. Boasting diverse features such as masked training, automatic backups, image augmentation, and multi-resolution training, OneTrainer simplifies the fine-tuning process. The platform offers a user-friendly interface with preset configurations for quick setup, separate workspaces for distinct training runs, and advanced settings for caching, tensorboard integration, and model specifications. Its flexibility extends to input/output model settings, data configurations, and training parameters, making it a versatile tool for efficient and tailored diffusion model training. Additionally, the backup and continuation features facilitate a seamless training experience, allowing users to regularly back up and convert model data for continuous progress.

IP-Adapter, an effective and lightweight adapter to achieve image prompt capability for the pre-trained text-to-image diffusion models. An IP-Adapter with only 22M parameters can achieve comparable or even better performance to a fine-tuned image prompt model. IP-Adapter can be generalized not only to other custom models fine-tuned from the same base model, but also to controllable generation using existing controllable tools. Moreover, the image prompt can also work well with the text prompt to accomplish multimodal image generation.

https://github.com/cubiq/ComfyUI_IPAdapter_plus

ProFusion is a new framework for customized text-to-image generation that preserves fine-grained image details without using regularization, as proposed in the paper. ProFusion includes PromptNet, an encoder network, and Fusion Sampling, a method that generates customized images based on a single user-provided image and text requirements. The paper explains how ProFusion works and provides experiments demonstrating its superior performance compared to existing approaches, while still meeting additional user-defined requirements.

StyleDrop is a technology that generates images in any desired style using text-to-image transformer, Muse. The technology captures nuances of user-provided styles such as design patterns and colour schemes. StyleDrop works by fine-tuning a few trainable parameters and improves the quality of generated images via iterative training with human or automated feedback. The technology can generate high-quality images from text prompts, and style descriptors are added during training and synthesis to improve the results. StyleDrop is easy to use and can be trained with brand assets. It can be used to generate alphabets with consistent styles in a single reference image. StyleDrop on Muse outperforms other methods in style-tuning for text-to-image models.

QLoRA allows fine-tuning of large language models on a single GPU. Using this method, they trained Guanaco, a family of chatbots based on Meta's LLaMA models, achieving over 99% of ChatGPT's performance. QLoRA reduces the memory requirement by quantizing models to 4 bits and adding low-rank adaptive weights. The team found that data quality is more important than quantity for fine-tuning, with models trained on OpenAssistant data performing better. Even the smallest Guanaco model outperformed other models, and the team believes that QLoRA will make fine-tuning more accessible, bridging the resource gap between large corporations and small teams. They also see potential for private models on mobile devices, enabling privacy-preserving fine-tuning on smartphones.

Demo
Another article about it

BLIP-Diffusion is a new model for generating and editing images based on text prompts and subject images. Unlike previous models, it uses a pre-trained multimodal encoder to represent the subject, allowing for efficient fine-tuning and better preservation of subject details. The model enables the generation of new images based on text prompts and subject images, even without prior training on specific subjects. It also supports image manipulation, style transfer, and editing guided by subject images. The model is trained in two stages to learn subject representation and can be combined with other techniques for more control over the generation and editing process. Overall, BLIP-Diffusion provides a flexible and efficient approach to generate and edit images with specific subjects.

Simple LLM Finetuner is a beginner-friendly interface designed to make it easy to fine-tune various language models using LoRA method via the PEFT library . With this intuitive UI, users can easily manage datasets, customize parameters, train, and evaluate the model's inference capabilities. Users can paste datasets in the UI and specify the new LoRA adapter name in the "New PEFT Adapter Name" textbox, adjust the max sequence length and batch size to fit their GPU memory, and click train. The model will be saved in the lora/ directory. The UI is beginner-friendly and has explanations for each parameter. The requirements are Linux or WSL and modern NVIDIA GPU with >= 16 GB of VRAM.

Extensive wiki about Stable diffusion.

The text introduces Civitai, a platform that allows users to share and discover resources for creating AI art. Civitai provides users with custom models that they can train using their own data or download models created by other users. Models are machine learning algorithms that have been trained to generate art or media in a particular style. Users can use these models with AI art software to create unique works of art. Civitai's community is constantly sharing new and interesting models, making it a vibrant and supportive community of AI artists.

Another HuggingFace DreamBooth library browser :

To extract the LoRA model from the base model using Kohya_ss, users can install the repository from GitHub and navigate to the Utilities tab, then select the Extract LoRA tab. The user must select the finetuned model to extract LoRA from and the Stable diffusion base model, then choose the path to save the output LoRA file and the precision. The Network Dimension scrollbar can be left at the default value of 8, and the user can click on the "Extract LoRA model" button.

Alternatively, users can download two files from kohya_ss\networks: lora.py and extract_lora_from_models.py, and make a .bat file to execute them. They need to change the file names and dimensions to fit their case and install a few requirements if their SD distribution doesn't have them already. They can also use qwerty.py to check which models fit their new LORA best, which is also useful for Textual Inversion embeddings.