DEC 2023
Every year, R/GA Studios is tasked with creating animated films to announce each of our Global Cube Award winners. This year we decided to leverage that ubiquitous new tech we've all been digging into. We created 11 unique films, built from the ground up using generative AI. Every shot in every film was created with AI (no stock footage, no CG etc.).
The process went from short scripts to Midjourney for storyboarding/shot planning/style frames. These still frames were then fed to Runway Gen 2 video generator to create the animated shots. The low res outputs were then run through Topaz Labs upscaler and interpolation tool, bringing them up to 4K 60fps. They were edited and fixed up with minor compositing in Adobe After Effects. Audio and motion graphics type animation was added at the end, using traditional methods. Could this be the first AI-generated award show package in history?
During this process I generated 2984 Midjourney images of which 40 were selected to be the final shots (1.3%). Those 40 selects were run through RunwayML 1097 times to arrive at the final 40 animated shots (3.6%).
As you can see, the success rate is very low and we are (currently) forced into this brute force method of generating repeatedly until something decent comes out. I can see this only improving with time as the language model gets better at interpreting the user prompts and the quality of the video generation improves.
The quality isn't photoreal or production ready but it will be interesting to use this as a benchmark to see where video generation tools are at the end of 2024. I have high hopes.
Below is a montage of some of my favorite shots. I will post all 11 films somewhere soon.
Cheers to all the Cube Award winners!