In the last month multiple AI art generators have come into the news. It looks like every large tech company has their own variant of diffusion art, and the list of generators is ever growing. I would like to share some of the clips I have put together and scrape the surface of what is possible with the current state of art generation.
Below is an example of what is possible when you combine generative art with large frame renders. The basics of how this works is given a handful of prompts a base image is generated. The next image/frame is generated and interpolated (mixed) with the original frame. This cascades forward to create the ever-transforming image. The base image is generated off of a number called a “seed” this seed value can be referenced again to generate the same image again.
This video was created using Google Colab and the Deforum notebook.
Notebook Link: Deforum_Stable_Diffusion.ipynb – Colaboratory (google.com)
How to make your own:
- Launch the Google Colab Notebook.
- Download the Stable Diffusion model and upload it to your google drive. sd-v1-4.ckpt – shared with FileRun (yerf.org)
- Place the model in the following folders AI/models (You will need to create these folders)
- Click on the “play” button on the first 2 code blocks in the colab notebook. You will have to allow google colab to interact with your gdrive. This will load the model into the notebook.
- When the execution has completed run all the remaining blocks up to “Settings”
- In settings you will be able to start building the type of animation you want. If you do not have google colab pro, keep the frames under 1000. Start with 2d and test the different camera settings to get a desired effect.
- When happy with the settings, click the run button to save them
- Moving onto the prompts this is where you will create the images you want. Each prompt will have the following format: ### : “Prompt words”. The number at the front is the weight of the prompt and the higher the number the more your video will use that prompt to influence the generation. If you need help coming up with ideas you can look at the bottom of the article for some helpful links.
- When creating prompts the syntax needs to be correct. Make sure you have quotes around each prompt and a comma after each prompt.
- Run the prompts cell once you are done with them.
- In the Run cell, increase the steps to 200 and the scale to 7.1.
- Click run, this will take a long time to generate the images.
- Once complete run the last cell and the mp4 should be avalible in your google drive Ai folder.
Helpful inspiration for prompts:
- Buying Google Colab Pro for $10 a month significantly speeds up the render process.
- If you have a timeout, you may just want to re-render.
- Keep your browser open and computer awake while rendering is happening. Good to leave in the background.