Merge branch 'main' into moonride-main

Author: MoonRide303

fooocus_version.py

@@ -1,2 +1,2 @@
-version = '1.0.51 MRE'
+version = '1.0.61 MRE'
 full_version = 'Fooocus ' + version

modules/async_worker.py

@@ -130,7 +130,8 @@ def handler(task):
             n_cond = pipeline.process_prompt(n_txt, base_clip_skip, refiner_clip_skip, negative_prompt_strength)
 
             outputs.append(['preview', (9, 'Encoding positive text ...', None)])
-            p_txt = apply_style_positive(style, prompt)
+            p_txt_a, p_txt_b = apply_style_positive(style, prompt)
+            p_txt = p_txt_a + p_txt_b
             p_cond = pipeline.process_prompt(p_txt, base_clip_skip, refiner_clip_skip, positive_prompt_strength, revision_mode, revision_strengths, clip_vision_outputs)
 
             for i in range(image_number):
@@ -149,10 +150,12 @@ def handler(task):
                 outputs.append(['preview', (5, f'Preparing positive text #{i + 1} ...', None)])
                 current_seed = seed if same_seed_for_all else seed + i
 
-                p_txt = pipeline.expand_txt(prompt, current_seed)
-                print(f'Expanded positive prompt: {p_txt}')
+                p_txt_a, p_txt_b = apply_style_positive(style, prompt)
+                p_txt_e = pipeline.expand_txt(p_txt_a, current_seed)
+                print(f'Expanded positive prompt: {p_txt_e}')
+
+                p_txt = p_txt_e + p_txt_b
 
-                p_txt = apply_style_positive(style, p_txt)
                 tasks.append(dict(
                     prompt=prompt,
                     negative_prompt=negative_prompt,

modules/expansion.py

@@ -3,6 +3,22 @@
 from modules.path import fooocus_expansion_path
 
 
+fooocus_magic_split = [
+    ', extremely',
+    ', trending',
+    ', perfect',
+    ', intricate',
+    '. The',
+]
+
+
+def safe_str(x):
+    x = str(x)
+    for _ in range(16):
+        x = x.replace('  ', ' ')
+    return x.rstrip(",. \r\n")
+
+
 class FooocusExpansion:
     def __init__(self):
         self.tokenizer = AutoTokenizer.from_pretrained(fooocus_expansion_path)
@@ -15,9 +31,12 @@ def __init__(self):
         print('Fooocus Expansion engine loaded.')
 
     def __call__(self, prompt, seed):
-        prompt = str(prompt).rstrip('\n')
         seed = int(seed)
         set_seed(seed)
+
+        prompt = safe_str(prompt) + fooocus_magic_split[seed % len(fooocus_magic_split)]
+
         response = self.pipe(prompt, max_length=len(prompt) + 256)
-        result = response[0]['generated_text'].rstrip('\n')
+        result = response[0]['generated_text']
+        result = safe_str(result)
         return result

modules/sdxl_styles.py

@@ -64,7 +64,10 @@ def load_styles(filename=None, base_dict=None):
 
 def apply_style_positive(style, txt):
     p, n = styles.get(style, default_style)
-    return p.replace('{prompt}', txt)
+    ps = p.split('{prompt}')
+    if len(ps) != 2:
+        return txt, ''
+    return ps[0] + txt, ps[1]
 
 
 def apply_style_negative(style, txt):

readme.md

@@ -100,10 +100,10 @@ Coming soon ...
 
 Below things are already inside the software, and **users do not need to do anything about these**.
 
-1. [Prompt expansion and the "Raw mode" to turn it off](https://github.com/lllyasviel/Fooocus/discussions/331) (similar to Midjourney's hidden pre-processsing and "raw" mode, or the LeonardoAI's Prompt Magic).
+1. [Prompt expansion and the "Raw mode" to turn it off](https://github.com/lllyasviel/Fooocus/discussions/117#raw) (similar to Midjourney's hidden pre-processsing and "raw" mode, or the LeonardoAI's Prompt Magic).
 2. Native refiner swap inside one single k-sampler. The advantage is that now the refiner model can reuse the base model's momentum (or ODE's history parameters) collected from k-sampling to achieve more coherent sampling. In Automatic1111's high-res fix and ComfyUI's node system, the base model and refiner use two independent k-samplers, which means the momentum is largely wasted, and the sampling continuity is broken. Fooocus uses its own advanced k-diffusion sampling that ensures seamless, native, and continuous swap in a refiner setup. (Update Aug 13: Actually I discussed this with Automatic1111 several days ago and it seems that the “native refiner swap inside one single k-sampler” is [merged]( https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12371) into the dev branch of webui. Great!)
 3. Negative ADM guidance. Because the highest resolution level of XL Base does not have cross attentions, the positive and negative signals for XL's highest resolution level cannot receive enough contrasts during the CFG sampling, causing the results look a bit plastic or overly smooth in certain cases. Fortunately, since the XL's highest resolution level is still conditioned on image aspect ratios (ADM), we can modify the adm on the positive/negative side to compensate for the lack of CFG contrast in the highest resolution level. (Update Aug 16, the IOS App [Drawing Things](https://apps.apple.com/us/app/draw-things-ai-generation/id6444050820) will support Negative ADM Guidance. Great!)
-4. We implemented a carefully tuned variation of the Section 5.1 of ["Improving Sample Quality of Diffusion Models Using Self-Attention Guidance"](https://arxiv.org/pdf/2210.00939.pdf). The weight is set to very low, but this is Fooocus's final guarantee to make sure that the XL will never yield overly smooth or plastic appearance (examples [here](https://github.com/lllyasviel/Fooocus/discussions/117)). This can almostly eliminate all cases that XL still occasionally produce overly smooth results even with negative ADM guidance. (Update 2023 Aug 18, the Gaussian kernel of SAG is changed to an anisotropic kernel for better structure preservation and fewer artifacts.)
+4. We implemented a carefully tuned variation of the Section 5.1 of ["Improving Sample Quality of Diffusion Models Using Self-Attention Guidance"](https://arxiv.org/pdf/2210.00939.pdf). The weight is set to very low, but this is Fooocus's final guarantee to make sure that the XL will never yield overly smooth or plastic appearance (examples [here](https://github.com/lllyasviel/Fooocus/discussions/117#sharpness)). This can almostly eliminate all cases that XL still occasionally produce overly smooth results even with negative ADM guidance. (Update 2023 Aug 18, the Gaussian kernel of SAG is changed to an anisotropic kernel for better structure preservation and fewer artifacts.)
 5. We modified the style templates a bit and added the "cinematic-default".
 6. We tested the "sd_xl_offset_example-lora_1.0.safetensors" and it seems that when the lora weight is below 0.5, the results are always better than XL without lora.
 7. The parameters of samplers are carefully tuned.

update_log.md

@@ -1,3 +1,19 @@
+### 1.0.60
+
+* Tune the balance between style and Prompt Expansion.
+
+### 1.0.56
+
+* Begin to use magic split.
+
+### 1.0.55
+
+* Minor changes of Prompt Expansion.
+
+### 1.0.52
+
+* Reduce the semantic corruption of Prompt Expansion.
+
 ### 1.0.51
 
 * Speed up Prompt Expansion a bit.

update_log_mre.md

@@ -1,6 +1,7 @@
 ### 1.0.51 MRE
 
 * Added support for adjusting text prompt strengths (useful in Revision mode).
+* Reduced allowed random seed range to match limits in Prompt Expansion and transformers (trainer_utils.py).
 * Updated Comfy.
 
 ### 1.0.50 MRE