Full pose control for generative art

Stable Diffusion v2 is here.

There are many comparisons between v1.5 and 2 already so we skip that part.

Instead, in this article we focus on a new feature called depth2img which is v.15's img2img feature on steroids.

According to the official release notes "depth2img infers the depth of an input image (using an existing model), and then generates new images using both the text and depth information."

This very powerful new feature lets you create new images from reference pictures and it opens up new ways to control pose on an image generated by Stable Diffusion.

‍

In this article we show you how you can combine pose maker software with Stable Diffusion v2 and have full control over the generated image.

Alright, let's get started with...

Pose Makers

Pose maker tools are very useful for making digital drawings and many of them are available for free. So we decided to use them for generative art and combine it with Stable Diffusion v2.

We decided to use Magic Poser because you can download it for free from the App Store.

After downloading the app open it and you should see something like this.

This is great, but the background color is very similar to the color of the 3D person. So let's change it to black by opening the menu in the top left corner and clicking on the Views button. Scroll down and change the environment settings according to the screenshot below.

Once you're done, you should see something like this.

After this you can change the pose of the 3D person and take a screenshot. We cropped the screenshot and filled in the blank parts such that we have a 512 x 512 px image.

Using Stable Diffusion v2 depth2img

It's a real pain to set up Stable Diffusion v2 locally but fortunately a reddit user u/backnotprop shared a colab notebook that we can run in the cloud for depth2img.

Open the notebook here and run the cells.

Finally you can upload your pose maker as input and generate picture with your prompts.

We used pretty basic prompts to generate the images below, e.g. stormtrooper, realistic, 4k

And we're done.

If you liked this tutorial and want to read more make sure you subscribe to our newsletter below (we send out one issue per week and never spam).

‍