sd_hassan
patreon
SDXL Training NSFW Samples
🕑 Added 2023-09-14 10:55:34 +0000 UTCHowdy all,
Since the last update I've been toying around with SDXL NSFW training.
Here's what I've found so far and there's some examples of the results below:
LEARNINGS
1: SDXL training learns NSFW concepts much better than SD2.1. It takes less images and less time for SDXL to learn the concepts which means the base model already has a good understanding of most of the anatomy with the exception of specific granular details
2: The resources needed to train SDXL are higher than SD2.1 but with the above points it balances out as we don't need to spend as much time training it. I'm currently training on runpod as the 12GB card isn't sufficient enough to train at a decent speed and config.
3: I've tried a couple of methods when it comes to data for SDXL:
- a) I've batch scraped 20k+ NSFW images and auto captioned with WD tagging, the result of the training was that it grabbed some concepts good but the overall quality isn't great
-b) I've tried using normal BLIP auto captioning and found results worse than WD tagging as the NSFW concepts weren't grasped
-c) I've tried reducing my dataset to hand selected high quality images, example that some of them are 5760x3840 etc, as high quality as I was able to find that were not huge file sizes. I manually captioned every image and found the concept was learned much better than both A and B. Manual captioning + very focused high quality dataset is key.
Yes it seems obvious this is a common known when it comes to previous versions of the model however for SDXL I wanted to run those experiments again to see if we can get away with a shortcut of training a huge dataset with auto captions.
USING THE MODEL
Then I took method C and trained a LORA also so we have both a finetuned SDXL model based on NSFW Female Anatomy and we have a LORA trained on the same.
The LORA is performing just as good as the SDXL model that was trained.
Here's what I've noticed when using the LORA
1: The standard workflows that have been shared for SDXL are not really great when it comes to NSFW Lora's. The workflows often run through a Base model, then Refiner and you load the LORA for both the base and refiner model. I've found that the refiner tends to negatively impact the results because it's not really capable of full NSFW.
2: I've found running through high res fixes with upscalers can work as long as the denoise is low to maintain the detail of the NSFW anatomy.
3: The detail custom enhancers work well to try and rectify issues that the refiner cannot fix. In my current workflow I've got the refiner only doing very little so I'm relying on the detailer enhancers to pick up where the refiner is left off, this is using the models for hand/face/person found in the adetailer repo's https://huggingface.co/Bingsu/adetailer
I'm still working out the details of the workflow but in parallel I'm expanding my dataset to now keep manually captioning really high quality images.
RELEASE PLAN
I'm taking a different approach for SDXL releases than I did for previous training. For SD2.1 it took so long and many hours and $ to try get it to the stage where I just let you all use it. That's because it struggled with all the concepts and I tried training all the concepts into the model to make it well rounded.
For SDXL I will release it in incremental stages, either as a model or a lora or both.
Some of these will only be for patrons, some will be released publicly
Having a LORA for each concept will be handy and then when all the concepts are learned I may have a one4all finetune to release.
This means my patrons will not have to wait long for each release and the first release of Female NSFW is almost ready to release already.
SAMPLES
These samples are from using a comfyUI workflow I've modified. They are not perfect but the focus here is to include accurate female anatomy, there are issues I'm seeing with hands/eyes etc but I can put extra effort into my prompts to fix these.
They are all called inpaint hands as the last preview I save is the detail hands enhancer. Still tweaking the settings though
