This paper aims to understand how neural networks work by breaking them down into smaller components that are easier to comprehend. However, it turns out that individual neurons are not easy to understand because they respond to a mix of unrelated inputs. One reason for this is superposition, where a neural network represents more features than it has neurons. The paper explores different approaches to finding interpretable features hidden by superposition, such as sparse autoencoders. By using sparse autoencoders, the researchers were able to extract meaningful features from a neural network and analyze its behavior. The paper provides detailed investigations, global analyses, and visualizations to support their findings.

NeoPrompt is a tool designed to make the creation of AI art more accessible and less time-consuming by providing a comprehensive framework for generating unique prompts. Catering to both beginners and expert artists, it features a repository of imaginative words and phrases and offers explanatory insights into its prompt generation methods. It's compatible with popular AI art platforms like Midjourney and Stable Diffusion, and the team is continuously working on enhancing its compatibility and accuracy.

The gpt-prompt-engineer tool is a powerful solution for prompt engineering, enabling users to experiment and find the optimal prompt for GPT-4 and GPT-3.5-Turbo language models. It generates a variety of prompts based on the provided use-case and test cases, and then tests and ranks them using an ELO rating system. Additionally, there is a specific classification version that evaluates test case correctness and provides scores for each prompt. The tool also supports optional logging to Weights & Biases, allowing for tracking of configurations and prompt performance.

Promptsandbox.io is a node-based visual canvas used for creating chatbots powered by OpenAI APIs. The platform has an intuitive drag-and-drop interface for creating dynamic chains of nodes that perform specific operations as part of the workflow. It is built using React and has various features such as integration with OpenAI APIs, document upload and retrieval, support for various block types, debugging tools, a chatbot gallery, and extensibility with additional node types. Promptsandbox.io provides a seamless experience for users to work with OpenAI APIs and build more complex chatbots.

Token Merging (ToMe) is a technique used to speed up transformers by merging redundant tokens, which helps reduce the workload for the transformer without compromising quality. The technique is applied to the underlying transformer blocks in Stable Diffusion, minimizing quality loss while preserving the speed-up and memory benefits. It works without training and can be used for any Stable Diffusion model, reducing the workload by up to 60%. ToMe for SD is not another efficient reimplementation of transformer modules, but an actual reduction of the total workload required to generate an image. The results of ToMe for SD show that it produces images similar to the originals, while being faster and using less memory, making it an efficient tool for image generation.

Awesome resources for in-context learning and prompt engineering: Mastery of the LLMs such as ChatGPT, GPT-3, and FlanT5, with up-to-date and cutting-edge updates.

Prompt engineering is the process of structuring words that can be interpreted and understood by a text-to-image model. This prompt book by OpenArt aims to help users learn essential building blocks and techniques to master Stable Diffusion. The book emphasizes the importance of a well-written prompt consisting of modifiers and good sentence structure, as well as well-adjusted Stable Diffusion parameters. The book also introduces the prompt format and various modifiers and magic words that can be used. The goal is to help users bring their creative dream images to life.

Want to figure out what a good prompt might be to create new images like an existing one?
The CLIP Interrogator is here to get you answers!
This version is specialized for producing nice prompts for use with Stable Diffusion 2.0 using the ViT-H-14 OpenCLIP model!

The strength of modern generative models lies in their ability to be controlled through text-based prompts. Typical "hard" prompts are made from interpretable words and tokens, and must be hand-crafted by humans. There are also "soft" prompts, which consist of continuous feature vectors. These can be discovered using powerful optimization methods, but they cannot be easily interpreted, re-used across models, or plugged into a text-based interface.
We describe an approach to robustly optimize hard text prompts through efficient gradient-based optimization. Our approach automatically generates hard text-based prompts for both text-to-image and text-to-text applications. In the text-to-image setting, the method creates hard prompts for diffusion models, allowing API users to easily generate, discover, and mix and match image concepts without prior knowledge on how to prompt the model. In the text-to-text setting, we show that hard prompts can be automatically discovered that are effective in tuning LMs for classification.

This space can either generate a text fragment that describes your image, or it can shorten an existing text prompt. This space is using OpenCLIP-ViT/H, the same text encoder used by Stable Diffusion V2. After you generate a prompt, try it out on Stable Diffusion here, here or on Midjourney. For a quick PEZ demo, try clicking on one of the examples at the bottom of this page.

Note: Generation with 1000 steps takes ~60 seconds with a T4. Don't want to wait? You can also run on Google Colab. Or, you can reduce the number of steps.

More example here : https://bird.trom.tf/tomgoldsteincs/status/1623358917110538240#m

Code on Github

The clip-retrieval package allows for easy computing of clip embeddings and building of a clip retrieval system. It can be used to quickly compute image and text embeddings, build efficient indices, filter data, and host these indices with a simple Flask service. The package also includes a simple UI querying system. The clip-retrieval package has been used by cah-prepro to preprocess 400M image+text for the dataset and by other projects such as autofaiss and antarctic-captions. ClipClient allows remote querying of a clip-retrieval backend via python. The package is installable with pip.

https://github.com/rom1504/clip-retrieval

Prompt engineering is a relatively new discipline for developing and optimizing prompts to efficiently use language models (LMs) for a wide variety of applications and research topics. Prompt engineering skills help to better understand the capabilities and limitations of large language models (LLMs). Researchers use prompt engineering to improve the capacity of LLMs on a wide range of common and complex tasks such as question answering and arithmetic reasoning. Developers use prompt engineering to design robust and effective prompting techniques that interface with LLMs and other tools.

Motivated by the high interest in developing with LLMs, we have created this new prompt engineering guide that contains all the latest papers, learning guides, lectures, references, and tools related to prompt engineering.

The random drawing prompt generator provides users with easy drawing ideas by generating a stream of random prompts. The generator is not based on AI technology, but rather on a list of creative drawing ideas that include universal prompts suitable for easy or detailed observational drawing, as well as cool anime drawing ideas. The purpose of the generator is to help artists overcome creative blocks by providing them with a starting point for their artwork. The prompts are intended to disrupt the artist's mind and help ideas start flowing.

Get an approximate text prompt, with style, matching an image. Optimized for stable-diffusion (clip ViT-L/14)). The resource is an adapted version of the CLIP Interrogator notebook by @pharmapsychotic, which uses OpenAI CLIP models to analyze an image's content and suggest text prompts to create more similar images. The results are combined with BLIP caption to provide suggested text prompts.

Example :

a cat wearing a suit and tie with green eyes, a stock photo by Hanns Katz, pexels, furry art, stockphoto, creative commons attribution, quantum wavetracing