Source
Slumber_watcherOP•

Made a script that lets you give a range of weights to tokens in the prompt.

This example is from the prompt: "photo of (cat:10) or (dog:01)"

The script will then generate a sequence of images, slowly lowering the weight of "cat" from 1 to 0 and increasing the weight of "dog" from 0 to 1.

https://github.com/yownas/shift-attention

6
Cap10B9•

This is great, I can create the images but get the following error when checking off the save results to video checkbox

​

I tried installing moviepy but I guess that I still have something missing.

1
Slumber_watcherOP•

Oh, that is right. I forgot to add that to the readme.

I assume you are using AUTOMATIC1111/stable-diffusion-webui. Just add moviepy==1.0.3 to requirements_versions.txt and it should be installed next time you start it.

2
haemtim•

i love this, but when i try this, even with the excact same prompt and parameters, i only get either the cat or the dog throughout, depending on which one i set to start.
Have you had the same issue or have a workaround / fix in mind?

1
Slumber_watcherOP•

Maybe, a friend of mine had a similar problem, but that was because he had made a typo and used a "-" instead of a "~".

The script in itself isn't that advanced, it search for a pattern that match ":<number>~<another number>)" and then changes it to ":<new number>)". What you describe does sound like what happens when it fails to find a match.

If you check in outputs/txt2img-images/shift/<number>, you should see the filename of the images change. If they are called something like "...(cat_0_1)..." something is wrong with the script.

(But, for example, "...(cat_0.5)..." is ok)

2
haemtim•

im extremely confused now, according to filenames and AUTOMATIC1111's PNG Info the weights are correct, but stable diffusion makes it weird. so instead of starting with a cat and morphing into a dog, it starts with a dog, throws some cat photos in between the dog photos, and in general they are almost all dog photos with only few cat images

1
Slumber_watcherOP•

I'm no expert, that cat2dog video I posted was the second video I've made. :)

But, yes, it is very temperamental. I think the main problem is that even if you say "(cat:0)" you are still telling the AI that there is a cat, just don't put much emphasis on it. I've had some luck with adding synonyms, or things that are close in the negative prompt. For example, if you put "kitten" in the negative prompt and maybe "(cat:0~1.2)" in the prompt. It will avoid "kittens" until it really wants to make "cats".

EDIT: I noticed now that in the webui settings there is "Eemphasis: use (text) to make model pay more attention to text and [text] to make it pay less attention" that has to be checked, think it should be by default but I don't know if you need to update the webui to get it.

I guess CFG scale and other tokens also affect what happens...I'm still experimenting. :)

1
haemtim•

thanks, i probably will have to do a lot of testing haha

1