AI Art Generation Handbook/Stable Diffusion settings

In Stable Diffusion, prompting is still considered as witchcraft at the moment as this field is currently very new (as in 2022)

However, we will be exploring some of Stable Diffusion settings

In this case, we will be using the following prompts:

Standing rhinoceros wearing business suit screaming aloud with hands on the cheek while seeing the stock price chart down

We will modify each parameters to see how it changes

For ease of understanding, we will explain the concepts thru the painter robot analogy.

Seeds

"Seed" refers to a fixed starting point or initial condition that is used to generate new pictures .

Seeds are often used to ensure that the generated data is reproducible and can be used for testing and evaluation purposes.

In Stable Diffusion , value "-1" is randomly assigned seed number

CFG Scales (Classifier Free Guidance)

CFG can be considered as guidance of how the diffusion models should follow your prompt .It can be considered as direction from external system to influence the output of an AI system. The higher the value, the higher the AI Art will resemble the text input. The lower the value, it will be more creative in generating AI art.

P.S: Take note that, higher CFG doesn't necessarily meant better art. See images as below.

Weightage (Emphasis)

In Stable Diffusion , there are applications of word emphasis (This is applicable on certain Stable Diffusion Web UI , most notably by Automatic1111.)

The word(s) ... are just needed to be enclosed with semi bracket to let the AI Art Generation model to know which words that you are emphasizing on.

The semi square bracket [ ... ] is usually to denote negative emphasis / do not emphasis

The semi round bracket ( ... ) is usually to denote positive emphasis / please emphasis

The more amount of brackets it had, the models will add more emphasis weightage of the included word to the generated images as shown in the generated images below .

However, too much of the weightage/emphasis will drown out the image subject / background

Steps

Sampler

There are many types of sampler

"Sampler" is a type of machine learning algorithm that is used to generate new data samples based on a given dataset. It is used in combination with machine learning algorithms, such as generative models, to generate new content that is similar to the data in the dataset. For example, a sampler could be used to select a sample of latent space, and a generative model could be trained on this sample to generate new images that are never seen before.

Here are some of the types of sampler used in Stable Diffusion (as of Jan 2023):

Euler , Euler a, LMS, Heun, DPM2, DPM2 a, DPM++ 2S a, DPM++ 2M, DPM++ SDE, DPM fast, DPM adaptive, LMS Karras, DPM2 Karras, DPM2 a Karras,DPM++ 2S a Karras, DPM++ 2M Karras, DPM++ SDE Karras, DDIM, PLMS

This is another examples of generating realistic human figures :

Negative prompts

Negative prompts is the keyword/modifier that are undesirable in the AI generated picture and it is better to include this to gain better quality.

3d, 3d render, 3dcg, abhorrent, abominable, amateur, anatomical nonsense, anime, asymmetrical, awful, awkward, b&w, bad anatomy, bad animal ears, bad anus, bad art, bad asshole, bad breasts, bad camel toe, bad clit, bad collarbone, bad crotch, bad crotch seam, bad cum, bad digit, bad ears, bad eyes, bad face, bad feet, bad gloves, bad hairs, (bad hands), bad knee, bad mouth, bad nipples, bad panties, bad proportions, bad pussy, bad shadow, bad shoes, bad tails, bad teeth, bad tentacles, bad thigh gap, bad tongue, bar code, basic, beard, big face, big mouth, big muscles, black and white, black clit, black nipples, black tongue, black-white, (blur), (blurred), (blurry), blurry eyes, body out of frame, boring, botched, broken legs censor, cartoon, censor bar, censored, cgstation, cloned face, (close up), colorful camel toe, colorful clit, colorful nipples, colorful tongue, contemptible, contorted, cracked mouth, crayon, (cropped), (cropped body), (cut off), decorating, decoration, ((deformed, deformed body, deformed glasses, deformed legs)), detestable, dick, different nipples, dirty face, dirty pantie, dirty teeth, disappearing arms, disappearing calf, disappearing legs, disappearing thigh, disconnected limbs, disfigured, disgusting, distasteful, distorted, distorted eyes, draft, drawing, duplicate, error, execrable, extra animal ears, extra arms, extra breasts, extra calf, extra digit, extra ears, extra eyes, extra feet, (extra fingers), extra heads, extra knee, extra legs, extra limb, extra limbs, extra shoes, extra thighs, extra limb, failure, fake, fake face, fat roll, fewer digits, floating limbs, frightful, furnishing, furniture, fused animal ears, fused anus, fused asshole, fused breasts, fused calf, fused clit, fused cloth, fused collarbone, fused crotch, fused cum, fused digit, fused ears, fused eyes, fused face, fused feet, (fused fingers), fused gloves, fused hairs, fused hand, fused mouth, fused nipples, fused pantie, fused pussy, fused seam, fused shoes, fused tentacles, fused thigh gap, ghastly, grainy, grayscale, grain, gross, gross proportions. short arm, grotesque, hateful, heavy animal ears, heavy ears, head out of frame, hideous, huge haunch, illustration, image corruption, irregular, jpeg artifacts, label, liquid animal ears, liquid body, liquid breasts, liquid clit, liquid collarbone, liquid digit, liquid ears, liquid tentacles, liquid thigh gap, liquid tongue, loathsome, logo, long body, long body, long face, long neck, long teeth, lopsided, low, low quality, low res, low resolution, low-res, lowers, lowres, malformed, malformed feet, (malformed hands), malformed limbs, mangled, messy drawing, misshapen, missing animal ears, missing arms, missing asshole, missing breasts, missing calf, missing clit, missing collarbone, missing digit, missing ears, missing feet, (missing fingers), missing hand, missing legs, missing limb, missing nipples, missing thigh gap, missing thighs, monochrome, morbid, more than 1 left hand, more than 1 right hand, more than 2 legs, more than 2 nipples, more than 2 thighs, more than two nipples, more than two shoes, mosaic, multiple, multiple breasts, mutated, (mutated hands), (mutated hands and fingers), mutated vagina, mutation, mutilated, no color, normal quality, obesity, obnoxious, odious, offensive, oil, old, one hand with less than 5 digit, one hand with less than 5 fingers, one hand with more than 5 digit, one hand with more than 5 fingers, (out of focus), (out of frame), oversaturated, penis, pony, poor, poorly drawn, poorly drawn, poorly drawn animal ears, poorly drawn anus, poorly drawn asshole, poorly drawn breasts, poorly drawn cloth, poorly drawn crotch, poorly drawn crotch seam, poorly drawn cum, poorly drawn ears, poorly drawn eyes, poorly drawn face, poorly drawn feet, poorly drawn gloves, poorly drawn hairs, (poorly drawn hands), poorly drawn mouth, poorly drawn nipples, poorly drawn pantie, poorly drawn pussy, poorly drawn shoes, poorly drawn tentacles, poorly drawn thigh gap, portrait, (pubic hair), qr code, red eyes, repellent, repugnant, repulsive, revolting, sex, sickening, signature, skin defect, soft light, split tentacles, surreal, terrible, testis, text, text font ui, thousand hands, three arms, tiling, twisted, ugly teetch, ugly, ui, unappealing, uncoordinated body, uneven, unnatural, unnatural body, unprofessional, unsightly, username, vile, watermark, watermarks, weird, weird colors, worst, worst quality, yellow teeth

Correlations between settings

The picture shows the correlations between CFG and Sampling Steps.

It is note worthy to look that steps that are too low and with high CFG values will almost caused the images to become distorted.

The CFG that are too low will make the picture appeared washed out.