5 lessons I learned from making an AI Film for the Gen:48 competition

I participated in the Runway’s Gen:48 AI film competition

Sep 19, 2024

Last week, I took part in the third edition of Runway’s Gen:48 AI film competition. With over 3,500 participants, the challenge was to craft a 1–4 minute AI-generated video over the course of a weekend, using 300,000 temporary credits provided by Runway.

I have to say it: it was an amazing experience.

While the time constraints were intense, I came away with valuable lessons that can benefit anyone working with AI tools in creative filmmaking.

1. Plan Ahead

Even though Gen:48 is a weekend event, and you get the Brief descriptions one day before the event, I think successful execution starts months in advance.

Having a story or at least a handful of story options ready is crucial.

Don’t start brainstorming during the event.

The more prepped you are, the easier it is to jump into the actual production. Brainstorm potential ideas ahead of time, including key scenes, characters, and visuals. This allows you to focus on refining the story instead of scrambling to come up with a concept under pressure.

2. Character Consistency is Still a Work in Progress

One of the biggest challenges in AI filmmaking is maintaining character consistency. As amazing as AI tools have become, creating a character that looks and feels the same across different scenes can still be tricky.

Each AI generator has its quirks, and subtle differences in lighting or angles can throw off the uniformity. This can result in characters appearing slightly different in each shot, which can be distracting to viewers.

But this is where we are. You can have great initial shots in Midjourney using the cref parameter, but you still have to animate those images with Runway, and in that process your character will shift.

So, don’t worry too much about it. Embrace it, use few characters with defined traits, and move on.

Midjourney’s SREF Codes Help, But They Aren’t Perfect

To maintain stylistic consistency in a film, I turned to Midjourney’s SREF codes, which allow you to lock in specific styles and attributes.

However, these codes have limitations.

Some shots are simply out of reach for certain SREF presets, and you may need to adjust your expectations when working with them. In my case, the SREF Code just couldn’t do macro close ups.

So you adjust.

My workaround was to experiment blending two similar SREF styles and adjusting their weight when a specific shot was causing trouble.

4. Adaptation is Key

When things don’t go as planned, adaptability becomes your greatest asset. During the competition, I found that certain AI-generated scenes or character designs didn’t align with my initial vision.

Instead of forcing the AI to conform to my original ideas, I adapted my story and visuals to better fit what the AI was capable of delivering. This flexibility allowed me to work within the limitations while still achieving a polished final product.

5. Focus on Delivering, Not Perfection

Perhaps the most important lesson I learned was the value of completion over perfection. In such a tight timeframe, it’s easy to get bogged down trying to perfect every scene.

However, time is your enemy in Gen:48, and focusing on delivering a complete, coherent story is more important than getting every detail exactly right. Polish can come later, but finishing the project is the top priority during the competition.

Here is my entry to the competition, if you want to take a look. I would certainly appreciate any feedback:

Participating in Gen:48 was a fast-paced, thrilling experience that tested my creativity and technical skills. Looking forward for the next one.

Thanks for reading!

Hi👋 I’m Erik Knobl. I talk about Generative AI on X/Twitter, and I‘m sharing my learnings in this newsletter. It would be amazing if you could join me on this journey.