[ISSUE #4] It's raining creativity!
Stable diffusion enabled creative projects - watching audibooks 📖, generating music videos 🎞️ and generating music 🎶!
Twitter is a gold mine of information. You can see people using it to test their ideas and showcase their projects. I spend an ungodly amount of time on Twitter and bookmark links for my future self to review. But as we all know, that never happens, so I decided I’ll share them here with you. You remain up to date on the AI proceedings, and my bookmarks don’t end up in the unreachable ether - win-win for both of us 😬 And you never know; just keeping up with the news might lead you to connect a few dots and spark a new idea in you.
In this issue -
Watching audiobooks
Generating music videos
Generating royalty-free music from textual descriptions
Watching Audiobooks 📖
Half of the fun in reading a fictional book is in imagining the vivid sceneries, compelling characters, and impressionable moments created by the author. But what if we could complement that with AI-generated videos right from the textual content of the book? It’s often said, “the movie adaptation of the book sucked!”. With this technology, we could perhaps have short movies with visual aspects as close to the book as possible.
The images were generated using Lexica powered by a stable diffusion model. The audio is pulled from a youtube video narration of the book using youtube transcript API.
Github link: https://github.com/danielgross/videobook
Automatically generating music videos 🎞️
In the same realm, here’s a music video for the song “Killer Queen” by compiling AI-generated images. The peculiar images personify the music and weirdly bring it to life.
Even though it looks promising, the image selection is a well-curated and time-consuming process (for now). It involves multiple prompt iterations, and thousands of outputs from a particular prompt, out of which a few are selected, color-corrected, upscaled, and polished.
Generating royalty-free music from textual descriptions 🎶
Mubert wants to empower creators by redefining the way music is created and licensed. You can generate royalty-free music and use it in your youtube videos without the fear of getting copyright-strike!
Here are some examples from Mubert’s demo notebook -
Prompt: calm music
Prompt: vladimir lenin smoking weed with bob marley
Mubert API: https://mubert.com/blog/introducing-mubert-api-2-0
Mubert Colab Notebook Demo: https://github.com/MubertAI/Mubert-Text-to-Music
That was quite a visual and musical update this week. Almost all the projects showcased in this issue use stable diffusion in some way or another. It won’t be an overstatement to say that over the period of 3 months, stable diffusion has equipped us with creative power limited only by our imagination.