Collage Diffusion is a novel interface for interacting with image generation models. It allows you to specify the composition of an image in a familiar Photoshop-like interface. Our modified version of Stable Diffusion takes the layers in and produces a harmonized image, ensuring that everything from perspectives to lighting are plausible. Unlike text prompting supported by traditional diffusion interfaces, Layered Diffusion allows you to precisely outline how a scene should be composed—from where objects are relative to each other to what they look like.

Stability AI has released an open-source version of its DreamStudio text-to-image consumer application called StableStudio. The company intends to work with the broader community to create a world class user interface for generative AI that users fully control. DreamStudio was first conceived as an animation studio that shifted its focus to image generation with the arrival of Stable Diffusion in the summer of 2022. StableStudio enable local-first development through WebGPU and a desktop installation of its Stable Diffusion tool. It is also compatible with ControlNet tools and local inference through AUTOMATIC1111 stable-diffusion-webui tool.

aiNodes is a Python-based AI image/motion picture generator node engine that facilitates creativity in the creation of images and videos. The engine is fully modular and can download node packs on runtime. It also features RIFE and FILM interpolation integration, coloured background drop, and node-creation with IDE annotations. The installation process requires Python 3.10, Git, and an nVidia GPU with CUDA and drivers installed. AiNodes engine is an open-source desktop AI-based image/motion generator that supports various features such as Deforum, Stable Diffusion, Upscalers, Kandinsky, ControlNet, LORAs, Ti Embeddings, Hypernetworks, Background Separation, Human matting/masking, and Compositing, among others.

ComfyUI is a powerful and modular stable diffusion GUI and backend that enables users to design and execute advanced stable diffusion pipelines using a graph/nodes/flowchart-based interface. This software supports SD1.x and SD2.x, and provides an asynchronous queue system with many optimizations. It can be used to load ckpt, safetensors, and diffusers models/checkpoints, as well as standalone VAEs and CLIP models. Other features include embeddings/textual inversion, area composition, inpainting with both regular and inpainting models, ControlNet and T2I-Adapter, upscale models, unCLIP models, and more. ComfyUI starts up quickly and works fully offline without downloading anything. Users can also save and load workflows as Json files, and the nodes interface can be used to create complex workflows.

https://github.com/BlenderNeko/ComfyUI_Cutoff

The strength of modern generative models lies in their ability to be controlled through text-based prompts. Typical "hard" prompts are made from interpretable words and tokens, and must be hand-crafted by humans. There are also "soft" prompts, which consist of continuous feature vectors. These can be discovered using powerful optimization methods, but they cannot be easily interpreted, re-used across models, or plugged into a text-based interface.
We describe an approach to robustly optimize hard text prompts through efficient gradient-based optimization. Our approach automatically generates hard text-based prompts for both text-to-image and text-to-text applications. In the text-to-image setting, the method creates hard prompts for diffusion models, allowing API users to easily generate, discover, and mix and match image concepts without prior knowledge on how to prompt the model. In the text-to-text setting, we show that hard prompts can be automatically discovered that are effective in tuning LMs for classification.

This space can either generate a text fragment that describes your image, or it can shorten an existing text prompt. This space is using OpenCLIP-ViT/H, the same text encoder used by Stable Diffusion V2. After you generate a prompt, try it out on Stable Diffusion here, here or on Midjourney. For a quick PEZ demo, try clicking on one of the examples at the bottom of this page.

Note: Generation with 1000 steps takes ~60 seconds with a T4. Don't want to wait? You can also run on Google Colab. Or, you can reduce the number of steps.

More example here : https://bird.trom.tf/tomgoldsteincs/status/1623358917110538240#m

Code on Github

The Stable Diffusion WebUI offers a range of features for generating and processing images, including original txt2img and img2img modes, outpainting, inpainting, color sketch, prompt matrix, stable diffusion upscale, attention, loopback, X/Y/Z plot, textual inversion, extras tab with GFPGAN, CodeFormer, RealESRGAN, ESRGAN, SwinIR, Swin2SR, LDSR, and more. The tool also offers various functionalities such as resizing aspect ratio options, sampling method selection, interrupt processing, 4GB video card support, live prompt token length validation, generation parameter saving, negative prompt, styles, variations, seed resizing, CLIP interrogator, prompt editing, batch processing, Img2img alternative, high-res fix, reloading checkpoints, checkpoint merger, custom scripts, composable-diffusion, deepdanbooru integration, xformers, history tab, generate forever option, training tab, clip skip, hypernetworks, Loras, API support, and more.