Creative storytelling with AI: The making of Ancestra

Speaker A: What I think about most in terms of the future of the filmmaking industry is what was, uh, inspiring for me to hear and to understand as someone who is so technical and working so deeply with the research and the technology, is that it still always boils down to storytelling.

Speaker B: I find myself being an optimist. So I would like to say the future is bright. And I would challenge, uh, all the people that are in the film industry to, uh, look at this as a moment of responsibility and ensure that it is bright.

Speaker C: As a filmmaker, I've spent a lot of time asking for permission to be able to tell my stories. And the beauty of this technology is that you can just go do it. And I think that's what's really, really exciting for me about the future with these tools.

Speaker B: Whoa.

Speaker D: What an amazing conversation we have today. Today, Christina, we were talking to three incredible people in the film industry. They were involved in the recent release of Ancestra, a film created in partnership with Google DeepMind and Primordial Soup, uh, which is owned by Darren Aronofsky. And we brought the three of them together to talk not just about this film, but so many other amazing things.

Speaker E: Yeah, no, it was so interesting to me because the film, as you mentioned, like, it debuted, uh, back in June at the Tribeca Film Festival. It's available online. We will have it linked. Um, all of you should go and watch it because it's fantastic. But what was so interesting to me was hearing about the process of how this film was made and how the, you know, I was in VO was used to get some of the shots. And, and I think that just how quickly, uh, went from kind of, you know, ideation to being at, like, one of the most prestigious film festivals in the world, uh, blew my mind. And, and this is a fantastic conversation. We go into a lot of really interesting things about, I think kind of the future of the film industry and creativity and the important role that humans play in everything that we're doing. And we can't wait for you to listen to it.

Speaker D: Yes. Let's jump right in. This podcast is sponsored by Google. Any remarks made by the speakers are their own and are not endorsed by Google. We are in the studio today with three incredible guests who have come together to create the latest short film. Ancestra, premiered at the Tribeca Film Festival this past summer. It is an absolute honor to introduce them. We are joined by Eliza McNitt, who is an Emmy nominated writer and director pioneering the fusion of storytelling and emerging technology, notably with her VR film Spheres, executive produced by Darren Aronofsky. Her work explores the intersection of science and art with her latest short, Ancestra. Combining live action and generative AI to tell a powerful story of maternal love. Eliza's films have been showcased at prestigious film festivals like Sundance, south by Southwest and Venice. We're also joined in the studio by Ben Wiley, who is a creative director and AI first writer at Google Creative Labs. With a background in storytelling and filmmaking, he's worked on everything from music videos and album launches to TED talks and documentaries, to big brand commercials, weird experiments and apps. Lately he has been dedicated to making a dent in the universe of AI, creatively hacking models and trying to shape them into tools for humans, human imagination. We're also joined by Corey Mathewson, who is a senior research scientist with Google DeepMind in Montreal, Canada, working on getting the best generative artificial intelligence into the hands of the best creative people in the world to tell otherwise impossible stories. Corey holds a PhD in Computing Science from the University of Alberta and is an associate industry member at the Quebec AI Institute. Corey improvises also with robots. Welcome to all of you.

Speaker A: Thanks, thanks.

Speaker C: Thank you for having us.

Speaker E: Ah, we're so glad to have you here. Um, so I wanted to, I guess, kind of start with you, um, Eliza, to ask a little bit about the origin, um, story behind Ancestra. So if you could give us a little logline for the film and we'll have this linked in our show notes so our listeners can watch it as well. Um, what's Ancestra about?

Speaker C: Great question. So what is Ancestra? Uh, during an emergency delivery, an expectant mother draws on the strength of all that came before, past matriarchs to dying stars, transforming her love into a cosmic force to save her daughter's life. And um, it is a film that is inspired by my story as, ah, a baby. Um, they discovered that I had a hole in my heart and had to be born immediately. And my mother went into the hospital for just a regular checkup and she had to have an emergency C section. And so this is really, um, this is that story and it's the story of my mother's courage and bravery bringing life into the world. And um, the story of every mother who goes through this in the universe.

Speaker D: Yeah, it sort of is such an amazing zoom in on the minute detail and also an incredible zoom out and bringing all of the elements together which represents the love that this mother feels for her child. I think it's such a beautiful, it's such a beautiful film and all three of you have worked on it. Eliza, you are the director and writer. Ben, you're the creative director and Corey you're the technical director. Um, how did you all come together? Was this uh, like a love ah at first sight, all of you coming together and working together or was this sort of inter, you know, first one of you started with someone and then the collaborative process sort of evolved. Or what was, how did you, you, what was your origin story of all working together?

Speaker C: Well, I showed up at Google after having 48 hours notice and I was suddenly sharpening a pencil and Corey was telling me he did improv and there was a camera in my face and then I walked into a room where Ben was sitting in there with a group of other people. And that was sort of the beginning of this process. Um, but I uh, had very little notice about this project. But I was just so excited about the opportunity to get to work with this team and to tell this story. And so that was my sort of my experience. Coming onto this was a bit like drinking from a fire hose. And then I sat in a room where people showed me the most mind blowing technology and I walked out of there and just had to stare at a wall for the next few days because I was so overwhelmed. But yeah, so that was my introduction to Ben and Corey, but I specifically remember Corey telling me he did improv. So that was really, really important part of my day.

Speaker A: Right on. I'm glad, I'm glad you said that. The, the, you know, it's, it's amazing how innovation goes slowly and then it goes really quickly all at once. And I think, I think the same is true with this sort of creative project that we had, you know, innovation slowly progressing along Google DeepMind has been at the forefront of a lot of generative sort of AI for a long time. Um, we've worked with the creative community for a long time and we think that their sort of feedback and engagement is super important. And then once it's like becoming real and we have the technology that is actually going to be super useful in the creative conversation, well then it moves very quickly all at once. And as Eliza says, like, you know, we found the right people who were ready and engaged to like grab hold of these generative media models and bring the stories to life that otherwise wouldn't have been possible. And it takes a great deal of collaboration and ad hoc collaboration relatively quickly and a lot of trust that needed to be built in the creative team. And you know, I think Ben and Eliza speak about, uh, yeah, just the trust that was Built so quickly in that moment that we were able to show you all of the generative media technology that we had and you had the story that you shared with us and then we were able to work together. But that, that trust building did happen relatively quickly. And Ben, maybe you can speak to sort of. Yeah. What it's like from your perspective.

Speaker B: Yeah, absolutely, yeah. I think we can also zoom back a bit too. Even before it was Eliza basically kind of like in Corey's intro, uh, putting the best AI in the hands of the best creatives, uh, we struck a partnership between Google, DeepMind and Darren and um, to create a series of short films. And Darren immediately was like, oh, well then I'm gonna need a, the first one. I'm gonna need a director who is very comfortable pushing technology, working with engineers in the unknown. Uh, and so he immediately put Eliza forward and on a plane, um, to show up. So I think that was ah, which is great. This only happens because Eliza basically was embedded with the team and, and there as if she was an employee of Google and a true collaborator in that sense. Um, yeah. And then it unfolded from there, um, and it was a success.

Speaker D: You have this amazing movie Ancestra on YouTube, which we'll link it in. But then you also have the making of Ancestra, which is also fascinating to watch. And the beginning of that is Darren Aronofsky talking about how technology has always played, played this role in filmmaking, in the creative process. And I'd love to know more about that moment when, like that aha. Moment when VO. Because you use VO2 for the making of this movie, correct?

Speaker C: Yeah, 2 and 3 and 3.

Speaker D: 2 and 3. And so when you were experimenting with VO2, what was that moment when both for you, Eliza, uh, as a director, but also for all of you in the room and also with Darren, what was that moment where you're like, whoa, this, there's a potential for this that is going to be taken to the next level that sort of kick started or was it sort of an evolutionary process of like, oh, we're playing with a couple of videos here and we can kind of push the boundaries. Or was there a moment where you saw something and you're like, okay, this, this is what we need to use and what we need to push forward?

Speaker C: Well, I just remember that first day when I came in and Ben and John, uh, sot, um, showed me, you know, videos that the team had put together and I was totally blown away by the possibilities, by the imagery. Um, I had, you know, pitched multiple ideas One of them was about a bee flying around Paris. Tragically, we didn't make that one. Um, but still, it's ideas on yet. I was gonna say the power of yet. Um, but, yeah, so that, you know, they showed me this incredible imagery of this honeybee and then a, you know, how they had mapped that using, um, an actor to map their body to it to create movement. And I just was so blown away that this was where we're at with technology. And it opened up so many ideas for me about what could be possible. And the first question I had is, because immediately I was thinking, this is the most incredible way to make one shot. And, you know, the first question I had is, how long are these clips? And that's when, you know, they told us eight seconds. And I think, you know, then it's like a lot of the limitations started to, um, come to light. And, um, you know, I suddenly realized, oh, okay, this, you know, this is an amazing tool, but I think there's so much I don't understand about it yet. So I was just really excited about rolling up my sleeves and getting in there with Ben and the team and figuring out how to break this open.

Speaker A: So for me, you know, VEO is a video generation model that takes the inputs from the user, the person, the creative person, and it translates them into pixels. And so it really relies on the creativity of an individual to be able to push the model into interesting places. And so what I find most, uh, inspiring and most exciting is once we get these models into the hands of really creative people, they can push them past where the researchers got them to, like, past the frontiers of what we thought were possible. So, for instance, some of the, um, some of the work in the film is this, this idea of matching motion across different natural scenes. And so imagine the motion of water swirling down a drain. Um, but then seeing that motion with leaves growing. The capability of the model, we sort of. We knew we had that capability, but it took someone's creative enlightenment and creative idea to sort of push the model to do something like that. That really becomes a surprising moment as to what the content can look like. So it's not, uh, us researchers, like, sitting in a room being like, I don't know, it's a cat and a dog walking together.

Speaker D: Yeah, we always have cats and dogs as examples.

Speaker A: You know, like, it's the first thing that I'm going to type in as my prompt. It's fast. I know what I want to type in. But it took really creative people to, like, push the model Past the frontiers.

Speaker B: I mean, I think for me, uh, like, I, I, I reflect on this because I'm, I'm so saturated in gen media. Like, I've been doing, working with these models for so long. So, like, my wow factor is. I have a high tolerance for a wow factor. But I think, like, um, I, by, like being a writer by trade, have always been a visual writer. And I think being able to get feedback on visuals that I have in my head in front of me and materialize allows me to push that. And then when you work with great visionaries, other incredible talented people like Eliza, it's a collective, uh, feedback loop that you're working on with the model and with great people, and you move way faster than you ever did before and in way more directions. And so I think, like, every creative process is iterative, and we've just seen that accelerate. And it's really exciting because you get to make stuff. You make stuff fast. And all creative people really like making things. Um, I'd say we'll get into this, but there were key moments on this film, specifically where I was like, oh, yeah, that was amazing. We got that. That's a beautiful shot. Or that actually worked. And those moments are the gold that any creative strikes. When you get in a flow state and it's like, yeah, we achieved the thing we hoped to achieve, or it was better than we hoped to achieve. And, uh, that's lightning. A bottle.

Speaker A: Yeah.

Speaker E: No, that's always the best part of art, right? Like when, when you have that, when everything comes together and you get what you didn't, it's better than what you anticipated. You could, like, that's, that's always really, really special. I was curious. Eliza, um, you worked, you've worked in VR filmmaking, uh, before this. And are there, uh, anything like you took, like, from your experiences using technology and some of your previous work, uh, within, uh, how you approached, um, using, uh, Gen AI Media. Um, were there any like. Or maybe not. Right? I'm just curious about that.

Speaker C: Yeah, no, great question. Um, I think, like, my other projects, um, specifically with Spheres, where it's a lot about envisioning the unknown. And in Spheres, you are thrust into the heart of a black hole, and nobody really knows what that looks like. Um, except for two professors at Columbia University who told me all about exactly what it looks like. Um, but, you know, I think that was really a, um, similar process with, uh, Ancestra is envisioning these impossible worlds where we've never been before, from the inner worlds within our bodies to uh, the cosmos. And so this was something I really wanted to use this tool for specifically was how do we visualize those things? But that was a huge challenge because there's not necessarily imagery that, you know, it would create from that. And so, you know, a lot of these more abstract images were hugely challenging to achieve and were part of those, like, wow. Moments on this project. Because when we began, for example, just try. And this is something I experienced too, with, you know, my work in virtual reality. We're so used to seeing these really scientific images of outer space. And how do you elevate that to make it feel more cinematic, to make it have, uh, you know, make it evoke a emotional response that tells a story and connects you deeper to the world. Um, and so I think that was something with Ancestra, we really wanted to have the images push the storytelling forward, which meant that it had to have this cohesion in aesthetic and visual style and in the, you know, overall approach that we took.

Speaker D: So.

Speaker C: So, you know, that was really this team of amazing artists that, uh, I had the opportunity to work with here, like Ben and John Sot, who, you know, were the connective tissue to really create that using the model.

Speaker E: That's so interesting. And that just made me think, too, when you're talking about having to kind of create these images of these things that kind of, you know, we don't know what they look like. That has to be a challenge, I guess, like, Corey, uh, to a certain extent, like, even from a model perspective of like, okay, we can. Obviously, you know, with prompts, you can get certain shots, but, like, a lot of the training that goes into these models is based on things that actually exist and imagery that already has, um, that things can be trained on. When there are things that we don't know what it looks like. Is that a challenge then, I guess, to, uh, have the model work so that you can get the output that you want that represents, um, what your vision for those things are?

Speaker A: Yeah, yeah, absolutely. It's definitely a challenge. It's much more of a challenge to visualize something that is much less represented in the data, much less represented in the model. And that's what requires the significant amount of effort and creates creativity from writers, from brilliant sort of like, creative writers who can bring their language and descriptive aesthetic to the visualization. Um, and so, I mean, Ben, you might even have an example of, like, sometimes you need to describe something somewhat kind of circuitously or a bit, um, tangentially to get the real visualization that you're going For.

Speaker B: Yeah, right. These, like, it's both an advantage and a weakness. These models are incredibly realistic and incredibly literal. So if you say you want the cosmos, it's going to look like Hubble space photography in all its full accuracy and brilliance. But then to do that as one choice, that's one creative choice. But to have an abstraction of that, to show the cosmos in a way that does emotionally resonate or even just aesthetically aligns with the rest of the film, you then have to come at it creatively like you would with any representation. Um, but we took a very much like a practical effects approach. Instead of just like abstractly describing the shapes and the blending, we actually, like, describe things that exist in the real world. Like what? Like, could you actually describe, like, microscopic photography and make it represent space? Uh, and it's. That's not necessarily an original idea. We actually took a page out of Darren's book. Darren did a lot of actual, like, practical special effects. Um, and it netted out in great results because the physics are so accurate and that you actually don't know what scale you're on. So with the context of the film, what is tiny and microscopic feels massive and cosmic.

Speaker C: And there was one artist in London who was assigned just to making tiny holes in the heart and creating a lot of visual nightmares there. So that was. There was, uh, a lot of abandoned images that go into our filed under our nightmares folder.

Speaker D: Excellent fodder for your next, uh, for your, uh, next horror movie.

Speaker C: Yeah, that's right, exactly.

Speaker B: Nightmare fuel.

Speaker C: Yeah. You never know what you're gonna get. Yeah, exactly.

Speaker D: Well, um, I'm loving these answers and I'd like us to dive a little bit deeper into the actual making of the film this, ah, season. The theme is we're focusing on builders, on developers and creatives and folks who are building and creating things. And so we'd love to get more into the nitty gritty of how you actually built it. I mean, you started off by saying that you had an artist specifically developing images, small, short clips of heart holes. Um, you know, what was. What was the process? Like, who sort of were the people involved? And, um. Yeah. And also, what were some of the biggest challenges along the way that you faced in putting this together?

Speaker C: It's a big question. Why don't we start with our team?

Speaker B: Yeah, go for it.

Speaker C: Yeah. So, um, we had an amazing team of people at the end of this process. There were about 200 artists who touched this project. And, um, the core team here at Google consisted, you know, was led by Ben and John Sot. And um, you know, then we had a team of artists who, um, were responsible for prompting and to, you know, create the images and let's see.

Speaker B: And then you have the technical side. That would be Corey as the lead over there.

Speaker A: Absolutely. And that represents, um, like the technical side represents a significant amount of effort from m. A bunch of people that have happened over the years. So you have, you know, a lot of people that are training these models and servicing capabilities of the models and we're getting access to them as soon as they become available and they're trying to sort of train us up on how to use them. And we're doing the handover from technical team to creative team. So we, you know, the team of collaborators is quite large and figuring out how to leverage these models as soon as possible is laborious. And you know, creating any sort of film is laborious and requires good collaboration. But especially when you have these deep technical experts working with deep creative experts and you're passing technology back and forth, this is like super, uh, challenging.

Speaker B: Yeah, yeah. We can maybe take you through like a bit of the journey of it, I guess. Like, Eliza showed up with, um, what is very traditional, had a treatment, uh, with an idea in mind of this story and brought that to us. The whole room lit up, which is like, not like. I think Eliza would probably attest to this. That's not like a common thing when like you have like unanimous like eyes wide, jaws drop, like, everyone's super excited. Um, and I think it was a credit to like you were thinking about, okay, how could this tech actually aid what kind of story would lend to this tech? Uh, but the directive wasn't make an AI film. It was make a film and incorporate VEO into your process. And I think that's important because this isn't just a pure AI film. There is a lot. And you should speak to maybe the lines you drew of like, okay, well, yeah, AI is going to be a part of this, but we're also going to have, uh, a lot of traditional filmmaking in here as well. Mhm.

Speaker C: Yeah. Actually when, um, I was in the room with you was when I had the idea for the film. I had a much broader idea before that, just about a mother and daughter. And it was just in the conversation, it was just in the conversation with everybody in the room when I suddenly had this aha moment and remembered my own story, um, about how I was born with a hole in my heart. And as somebody who, you know, has always had a fascination and love for black Hol. It was something I was, like, really drawn to because, um, it's just these images that you don't quite understand in the universe. And so I was really excited about exploring that. But, um, yeah, the team was set up both as, you know, I wanted to have a traditional live action component where we filmed actors. That was really important to me, to have human beings as the heart and soul of the film and grounding the characters as real humans. And, um, then we had this whole AI component, which was to craft these images that would be otherwise incredibly difficult to, um, capture otherwise of these inner worlds within the human body and these outer worlds into the cosmos. And so I think that was kind of how we organize this. You know, how are we going to tell the story? But for me, as a filmmaker and a storyteller and just also wrestling with, like, how do we use AI? It was really important for me that all heads of department were represented in this process. So, you know, I had a storyboard artist, I worked with a VFX supervisor. We brought on, you know, an AI unit. We had, you know, a cinematographer, I had composers. And, um, I really, you know, I really. With each of those people, I wanted to defer to their comfort level of how. How would they want to interact with these tools? How do they want to incorporate AI into the process? Because as a writer and director, this is, you know, I'm crafting this story, but I'm also, you know, this is a collaboration with hundreds of artists, and this is their film too. And so I felt it was really important, every step of the process to make sure that people were engaging with the tools in ways that they felt comfortable with as well.

Speaker E: I think that's so important and so interesting, and we want to touch on, um, I guess, you know, some of the concerns that happen in the industry, um, in a little bit. But I am just kind of curious at a high level, how long did this take? Um, I guess, you know, from kind of, you know, pre production, I guess, you know, you kind of coming in with the treatment to it premiering at Tribeca. Um, how long did this, uh, collaboration take?

Speaker C: Should have taken longer. M. It was much too fast. I don't think we slept for several months. I think I arrived May 26th and. No, no, no, sorry, let's go back. I arrived March 26th. We premiered the film June 13th of this year.

Speaker E: Oh, my God.

Speaker D: In the same year. My God.

Speaker C: Wow. Wow. Yeah. And by the way, that also, I arrived with no script, just an idea in my head. So we had to Spend like the first three weeks to the first month just ideating on the story, um, and then just launching into production. So I think this speaks to what Ben was saying as well. The beauty of this technology is the ability to visualize imagery as you are creating. So in essence you almost get to write with veo and that enables you and allows you the ability to, you know, be creating as you are imagining the project. And that helped my process so much as a director and a storyteller, just to be able to see what I was imagining in my head and to put that on paper, to be able to show my collaborators what I was thinking. And so, you know, when you write a script, it's a very abstract set of words, but to be able to also have this tool at my fingertips. And we were constantly iterating and creating based off of what I would write that night. And so it was a very collaborative and iterative process and I really enjoyed working that way. I would say with VR in the past and animation pipelines, that kind of flexibility has been often quite difficult. So I think having the ability to be so iterative was something really special about this new process for me.

Speaker D: Yeah, uh, yeah, well, I'd like to dive a little bit deeper into that aspect of it and maybe lead with a question for you, Corey. Veo, I mean, has been essentially the model has been trained on, on data that has been accessible, uh, on, on a large data set, essentially in the film, however, Eliza, you mentioned also in your interview that the images or the shots of the baby are inspired by pictures that your father took of you when you were a baby. And so I was curious about getting a little bit more deeper into, you know, how like from a layman's term, you know, did you, there was already the VO model, Did you use um, Eliza's pictures to train the model to generate the videos of her or was there like. I'd love to know a little bit more about, especially from a developer perspective, you know, how that sort of collaboration was.

Speaker A: Yeah, sure, happy to dig in. So, yeah, just to be clear, VEO has been trained on high quality video description pairings. Video descriptions are like pairs of videos and descriptions of what's happening in those videos. Um, and there's a lot that then gets compressed into the model that we can sort of get out of the model by prompting it in the right way. But as you say, kind of to get a very personal looking, uh, image or a personal looking video out, it's rather difficult because we don't necessarily have that Reference material. Um, so there's a few different ways that we can sort of push the model to generate things that we want. Um, especially when we want the model to be generating stuff that is as personal as Eliza's story. And we want it to match with the aesthetic and the vibe and the, uh, um, you know, the film as a whole. Um, one way to do that is to use Gemini's, um, image and video understanding capabilities. So we can input images and videos and get Gemini to describe them in, like, lots of words and lots of language, and then use those words and language to then prompt Veo to push it to generate things that look like that. More like mine. More like this. More like this. Um, we can also use images to be the first scene or the last frame of a particular video. So this is like first frame or last frame, sort of M generation. So say we want to start with this particular image, and then we want to generate a video as a continuation of that image. Um, other things that we can do are taking a video and then editing the video. So it has sort of live action aspects and generative aspects combined. So we say we want to mask out this section of the video and generate something within that. So there's a lot of ways that we can sort of, like, personalize the generated content, um, not just, um, in the ways that the models are trained. Because obviously training is very difficult with these models. And we want to sort of push prompting past where we think is possible. Um, and that means learning how to push these models with new language and with images and other ways that we can condition the model.

Speaker D: What is the framework or interface that you used for this?

Speaker A: Yeah, so we had lots of different artists working on this, and each of them have their own processes, right? Creative people have their own processes. So, uh, I know people were using Gemini to do image and video understanding. People were using Flow and the lead up to Flow, um, as they were sort of like leveraging new capabilities that were going to then land in Flow. Uh, Flow is accessible now for lots of people. Uh, creative people should check out Flow. Um, you know, I can push it as much as possible. Flow is awesome. And, uh, more capabilities will be landing always. But Flow is one way that you can be generating. Um, AI Studio is another. But most of the interactions that we did were sort of through custom, um, bespoke tooling that we build for the creative team. Uh, understanding that a creative team needs to collaborate in a very rich, um, sort of high information dense way. We need to be sharing clips and prompts very tightly with each other so that we can see what people are generating and how they're generating those things. And then we can remix their generations and maybe take something that we liked from their prompt and put it into our prompts. Those sorts of capabilities will find their way into Flow and the ecosystem of Flow will continue to grow. And I'm excited to see how filmmakers and creative people will use Flow in all sorts of different ways. But, uh, yeah, that's sort of the interface layer for how we interacted with these things. But all the way down to the sampling from the model, all the way up to, uh, looking at images and videos on a big screen all sitting together in a room.

Speaker B: And I think it's an important part of the impetus for this project, uh, is to be conduit between, uh, how the models are developing and ultimately the creatives and people that are going to use them. Uh, and just as much as that feedback is helpful for actually the models, also the quick interface tooling is just as valuable. And there's people like me who I don't have a lot of technical expertise beyond being able to maybe interact with these models through natural language. But there's people like Corey and Anthony Tripaldi and Michael Chang, other people on our team who are actually going to stand up these things with very simple UI that my simple brain can understand. Uh, and that's a feedback loop as well. Like, oh, well, I would love to leverage this control, but could we maybe allow me to, uh, modify it this way? Could that be a tool? Could I have a lever? Or can I be able to, uh, just give you a simple mask? Little things like that. That feedback of how we actually leverage the tools is just as important as how we actually evolve the models themselves.

Speaker C: And that was something also evolved throughout the process of making Ancestra as well.

Speaker B: Uh, yeah, absolutely.

Speaker C: It was going like image to video.

Speaker B: Oh, there's all kinds of. Yeah, there's all kinds of things that, like, we. I think one of the things here is like, we look to like, as Corey would send up a flare, like, new tool coming on, like, okay, great. How could we leverage that? Can that accomplish some kind of unique creative problem here? Maybe. Let's try it. Um, and I think that's a big piece here. And maybe even if it doesn't, it's like, okay, well, it could if we, you know, we could get it 20% further. And that 20% further is huge. That does. That's a lot. That's really valuable information. Um, and maybe that tool didn't pan out for a production ready use case. But now we have the feedback and we know the roadmap of how to get it there.

Speaker E: Now that you've had this experience, Eliza, is this, ah, a tool that you will consider using in future projects? Is this a way that you think that you will continue to make films in the future? Um, by using AI tools?

Speaker C: I mean, absolutely. I think just what we were able to do, for example, with the baby in the film where we actually, um, created my image as a newborn based off the images of my father and then placed that into the arms of the mother in the live action footage. Um, and that was something that Erin Raff, our VFX supervisor, um, really had to figure out throughout this process if that was even something that would be possible. Just the fact that that was something that we got to experiment with was so incredible and just created an entirely new option for having, uh, you know, a newborn in a film. That was something that's so exciting. And I am just really excited about the future of what we can continue to push and experiment with these tools to come up with solutions for things like that where, you know, it's a really difficult filmmaking challenge. You can't have a baby act. It's a challenge. And so I really wanted, um, I think it's something that I want to continue to explore and experiment with. And I'm super excited for as things evolve, to see, you know, and things are evolving so quickly. It was part of why we created this movie so fast because we wanted to showcase the technology and also not sleep for two and a half months. And you know, I think that that's, it's very exciting where things are heading. It's also the responsibility of filmmakers, artists and the people in charge to put boundaries around that as well.

Speaker E: Thank you for teaming me up. Like, you literally like, kind of got to my next question because I think it's great to hear about that. But obviously, like, we all, I think, see a lot of the potential or, um, I think many of us see the potential of this technology in art and in so many different, um, aspects of our lives. But it doesn't come without controversy. And this is obviously a very controversial topic in your industry, um, and in kind of the creative world at large. So talk to me about how you set those boundaries and how you think about how, you know, um, AI can be used as a tool rather than, um, hopefully my hope would be anyway, rather than a replacement for traditional, uh, jobs.

Speaker C: Yeah. It was so important for me in making this movie that we centered Humans at, you know, every aspect of the creation of the storytelling, uh, and the filmmaking process. Um, I also, you know, I, even though I was shown lots of incredible images of humans that AI had generated, I felt that it was critical to have, you know, a human actress and was so fortunate to have Audrey Corsa, who, um, played the role of my mother. And you know, before her audition, Audrey called my mom, whose name also is Audrey, that's just a strange coincidence, and asked her, you know, about the day that I was born and researched, you know, really researched from my mom what that was like and was able to really capture my mom's spirit and her, you know, her essence and what she was going through that day. And that's not something that, you know, an AI performance can generate. And so I think, you know, it's really important that we, you know, that we have humans at the helm of this process because it is ultimately the human soul that we are trying to capture. And I don't think that's something a computer can generate. Um, so I think that's a big part of it for me. And you know, I think otherwise, as the tools evolve, it is again, as I said before, it's the responsibility of artists and filmmakers and creators and creatives and the people who are creating these tools to, um, also put those, you know, those boundaries around them. But I'm actually, I'm really curious to hear from Ben and Corey because it's such a good question. I think everyone has a different perspective. So I'm really curious what you guys think.

Speaker B: Yeah, sure. Uh, I mean, I think, um, I've worked on a lot of different AI projects across mediums. And I think the biggest thing that. What I've really come down to and is these models are only going to output as good as what you put into them. And that takes human creativity, it takes human taste, it takes, ah, often experience that far exceeds AI, um, and then it really becomes a collaboration. It is this artificial intelligence times human imagination. That is a recipe that you need ultimately to create something that is going to impact people or at least impact, uh, even yourself. Um, and I think that that's not going away. Uh, I think there's also like, I don't think it will go anywhere. I think, matter of fact, it will be on creative individuals to even continue to harness what makes them human and what makes them creative. And how can they then influence these models and tools just like you would with any other tool. It's the same thing with all the other previous digital tools that we've seen the amazing thing is, is, yes, what can these tools enable? But the best part is, what do people do with them? And I think that we'll continue to see that. I don't think that that's ever gonna go away. And I think that we're gonna continue to value the rich things that truly are unique. Eliza's cries from her as a child are in this film. That is a rich thing that you can only have. And, yes, AI did help us, uh, put that in there, but that's still Eliza. And like she was saying about Audrey, the actress, Audrey called Eliza's mom, to learn about the experience of what that's like. That's a moment that is irreplaceable because that can only happen between, uh, Audrey. They're both named Audrey, actually.

Speaker C: Audrey and Audrey and Audrey.

Speaker E: Audrey and Audrey. Audrey's spirit.

Speaker B: So I think it's really important that. Yeah, I think I love the viewpoint of looking at these models in AI as a tool. And I like the idea that tools don't just make things on their own. And it takes humans with big ideas and a lot of bravery. I think it's very scary. It takes, um, courage to wade into this unknown because you have to let go of stuff and you have to be willing, uh, to see what's going to come out the other side. But, um, it is that it's on those brave individuals to then, uh, shape the thing for everyone in the best way possible.

Speaker C: I will say, though, walking into this process, uh, being able to work with people like Ben and John and Corey, who are so in this world and have such passion and excitement about it, even though I had an existential crisis on the first day, seeing a lot of the images I was shown and thinking, like, what is my role as a filmmaker in the future now? I think just seeing their excitement about the future of these tools and how we can actually use them to enhance our stories is the reason why I was so excited about working on this project.

Speaker A: I wanted to say we're focused on how these AI tools can unlock more opportunities for more people that would have maybe otherwise not had access to the tools or the knowledge or the skills. And I'm very optimistic that this kind of AI technology is going to, like, introduce new voices and new roles and new possibilities in filmmaking.

Speaker B: Right.

Speaker A: We had an AI unit on this film. I don't know if that was the first AI unit. I doubt that's going to be the last AI unit that we ever see.

Speaker E: Oh, I'm sure it's not right. I thought it was great, though, that there was an AI unit. I was like, okay, that's actually. That's great. It's like treating it like it's part of every other unit you would have.

Speaker A: Uh, absolutely. And it's like, this is the bridging of traditional filmmaking with new technologies and new tools. And by doing it early and by doing it often, it's. It's a bit challenging. Right. We need to be making a film with technology that is, quote, unquote, not ready for production yet, which is a challenge. But it does mean that we're, like, sort of writing some best practices or best, best yet practices of how we should be engaging. And I think it's. It's the right way that we should be doing technical innovation together, uh, so that we are understanding the challenges and empathizing with the creatives that are really the ones that are using them or not using them or choosing to use them, and how they're choosing to use them as early as possible so that we can truly understand them and work with them to find solutions or define responsible practices.

Speaker C: It felt a lot like the early days of ilm, just, you know, technologists, artists, engineers, just working together in, you know, trying to problem solve and figure things out, even though the technology is not there yet and it's not ready. And I think there was really such a magic to this process.

Speaker B: Yeah. I'd love to double click on one thing Cory said which really resonates with me. I do think as these tools democratize filmmaking, I do think there will be an incredible empowerment of personal storytelling. Ah. In a way that I think will just shape the stories we get to consume and share. Eliza, don't wanna put words in your mouth, but, um, I think you told me your story of your birth, obviously is family lore. You've been told that story for your entire life. But to actually be able to bring this caliber, the vastness of the story that stretches the cosmos and time to a very personal childhood story, that's a whole new realm. And there's a reason probably why you haven't actually made this film prior till now, because it was personal. And that is a hard thing to actually like. Okay, well, I love that story, but how could I bring it to the caliber and height of my filmmaking talent? And these tools, hopefully, to some degree, um, unlock that, um, and I think we'll see that across a variety of people with personal stories that just have not had the means to tell them, and now they can in a way that aligns with their vision.

Speaker C: One of my friends who was at the premiere, you know, afterwards, she said, like, wow, I feel like this technology has finally caught up to the ideas that I've had and has, you know, allowed me to create something that is so ambitious that otherwise, just on paper, someone would say, well, that's impossible. You can't do that. That's crazy, right? And I feel like, wow, actually, like, we can make these really ambitious, impossible stories now that are personal, that nobody would, you know, no one would greenlight this story. You know, it's just so special to be able to do that. And I do think it's amazing for individuals and, you know, storytellers, filmmakers, people who aren't even filmmakers, to be able to share their stories and have these tools at their fingertips. And I do think, you know, it's exciting to get to enter a new world now where everyone's going to be able to have access to this.

Speaker D: It is, it is. And it unlocks, as you said, so much opportunity and, um, almost makes, like, the impossible possible and it stretches it in a completely new way, which is what's so extraordinary. At the same time, though, coming back to one of the comments I made at the beginning of the episode was around this concept of technology and how technology has always been a part of the industry and filmmaking, but with every new introduction, there are these changes. And the big thing in film from a technology standpoint, most recently, before this was sort of vfx, I was curious as to your perspective on the evolution of one sort of technology merging into another. There are plenty of creatives out there who have spent a lot of time learning about, like, doing VFX videos or learning a specific technology. But with these new tools, like, things are. Things are definitely changing. And I'm curious to know all of you, kind of what your thoughts around

Speaker C: that were as far as it goes with, you know, VFX and these AI tools now being introduced. Um, there's a lot of fear around that and what that means. But I do think, you know, working with our VFX team, with Aaron Raff and his team of, you know, VFX artists, we still needed to have a full team of vfx, um, people. And it, you know, it just involved different kinds of problem solving now and using the same skill set, but to create things just, you know, and adapt to a new kind of a pipeline and workflow and, uh, you know, I think just for example, with like, the generation of the baby, that was something that would otherwise, you know, be incredibly difficult to create a CG Baby. But because of the tools, we came up with an entirely new method that still involved having to use, you know, artists to be able to perfect it. But, uh, getting to that point was a lot more efficient and much more realistic and I hope, ultimately enhance their process. So I do think it's all about, you know, adapting to the tools and using, um, your craft that you have honed over so many years to, uh, just, you know, implement that into your toolkit.

Speaker B: Yeah, I completely agree with that. The other thing I'd say is as. And maybe it's just about what I also just feel, um, is we as creative individuals, but also anyone that has exposure to these models and the privilege that we do to use them, I think we have a responsibility, uh, to use them in the right way and to answer the problems and how we're going to use them and say, okay, well, is generating this content the best solution? Uh, I think then that leads to a really cool and exciting challenge, which is, okay, well, if there's other solutions that maybe are better and are going to achieve that goal, what will I use these for? And can I push to do something that I can't do otherwise? Are there problems that I don't know how to solve that maybe this technology can? And then with that, don't just put out the output, but bring people along. That's the impetus for why we have a making of. The making of is even longer than the actual film itself. Uh, and I think that's. We hold that as just as much weight as the film, because that's important. We have to show why, not only how we did these, but why we made the decisions that we did. And, um, I think that's a big part of bringing people along the journey.

Speaker D: Definitely. And from this conversation in the making of, there is a very strong message that there's a collaborative effort across. Across the board. And it sounds like the integration of VEO and Generative AI really is just an addition. It's like an additional tool that you can use versus something that's replacing an already existing tool. I don't know. Corey, what do you think about that?

Speaker A: Yeah, absolutely. We see this as an augmentative tool as opposed to any sort of replacement. Right. VO is another way that you might bring an image in your head to life. But, like, more generally, the film industry has always been in a constant state of evolution, and that evolution has always been driven by technical breakthroughs. Right. Technological breakthroughs really defined a lot of Darren's early movies and reshaped how movies were made. And continue to reshape how movies are made. And that doesn't mean that we don't see like we still. What I mean to say is we still see hand drawn animation, but we also see full length feature films made in Blender. So we're seeing new technology used to tell new stories, which is rather interesting that maybe there's stories out there that haven't been able to be told until we have a technical innovation like generative AI. In the same way that the introduction of CGI and VFX effects kind of transformed, uh, the visual possibilities in storytelling and then led to things like Toy Story and Avatar or whatever your favorite CG world is.

Speaker B: One metaphor that we toss around internally is thinking of these generative models as a new type of camera. And that's not in replacement of old cameras. And it's a helpful mental model. You think of a GoPro camera, what did that do? It opened up a whole new genre of filmmaking. Or like the camera we all have in our pocket, it just changed the way we perceive and create. But not in replacement of any of the professional cameras or all the other cameras that are still used. And I think there's a new vector here and I think it is about actually looking at like, okay, how can we continue to push it into a new vector? What other lanes and opportunities does it open? Not necessarily. Oh, how can it replace the things we've previously done? So it's, it's also, there's an onus on how you use it.

Speaker C: Um, I think if, you know, if I was in film school right now and just learning how to tell stories, this tool would be so incredible because I would be able to actually hone my voice and experiment and be able to just create images to kind of figure out what is my vision as a filmmaker and have the ability to do that as opposed to going out on set and burning a bunch of time and money and making your friends, you know, stand out in the rain and, you know, all the things that of course are important to being a film student, but also, you know, but I think you could use this as a tool to really, like, help yourself prepare so much more and understand yourself as a filmmaker with, you know, less consequence early on. Um, and I think that would be super valuable. And I also think just for all filmmakers, it's an amazing tool to just visualize your, you know, your stories and your ideas before you even get on set so you have a better sense of, like, what is it you are trying to say?

Speaker E: I love that you said that I actually went to film school and. And, you know, now I'm a technologist. And it's interesting to me, you know, as you're talking, like, if we think about it, a. I think you're exactly right. Like, if I were. I wish that I were in film school now, you know, if I could, you know, if I were 18, 19, these things would be so interesting. And so I think, uh, would unlock so many creative possibilities in some ways, but also just makes me think just, just kind of as an aside, as a thought, like, if you look through, if you think about it like, the history of film and the evolution of technology, they're parallel stories. You know, the film industry has been shaped by technology literally from the beginning. You know, whether it's, you know, advancement of sound, you know, color, you know, going into imax, introduction of visual effects, you know, digital, uh, video cameras and so on and so forth. And so I think, uh, I appreciate you all kind of helping, I think maybe put it in that perspective, uh, a little bit, and also being committed to making sure that this remains artistic.

Speaker C: I mean, what's happening in the world with AI right now, it is a watershed moment where everything is changing and a lot of us don't even know what that's going to look like. And I think I just felt a responsibility in telling this story and making this film to try to really ground the use of AI in the most, um, you know, the most important ways to telling our story, uh, and the most personal ways that felt, you know, critical to what we were doing and, you know, not just trying to, like, show off a tool. So, you know, I think we'll look back at this in a couple years and be like, wow, can you believe that? We were writing text to prompt these things? And it wasn't just reading my mind, but, you know, I, I don't know. So I just think it was really exciting to kind of get to come in at this, at this moment.

Speaker A: Don't talk about the roadmap, Eliza.

Speaker C: You can't talk about the trail.

Speaker E: I love it. I love it. Well, I think that is a perfect way to end this conversation. Thank, um, you all so much. Before we end, um, we always have kind of like a little rapid fire, um, segment where we'll ask, um, each of you just a quick question and just come up with the first thing that comes to mind. So, um, Ben, I'm going to start with you. What was the last thing that you experimented or played around with when it comes to AI tools?

Speaker B: Um, my sister and I have been digitizing, uh, my grandma's Super 8 footage, and there is a 16 minute reel of Disneyland from 1960, and it is incredible. So I'm trying to figure out what I can do with that. Uh, and the first thing I tried to do was recreate some of the scenes using veo. Um, and so I'm currently doing that as kind of a extension, uh, of her history. I can't ask her about it, and I kind of wanted to play with that. So I'm generating 1960s Disneyland footage.

Speaker E: Amazing. Amazing.

Speaker D: Absolutely incredible. Incredible. All right, Eliza, what was the last thing you asked? Gemini or any AI?

Speaker C: Okay, this morning I did ask about, um, I asked about wormholes. I was curious about what, uh, about theories about wormholes. I think that's what I was doing in the cab, uh, this morning.

Speaker B: Casual, as one does in the cab.

Speaker E: Uh, Corey, um, what can you do now that you couldn't do, uh, six months ago?

Speaker A: Like, enjoy watching, uh, generated footage, like video that is generated by.

Speaker E: Yeah, there you go. You can actually watch people generate stuff.

Speaker A: Yeah, yeah. The stuff that people are generating is now much more interesting than it was six. Six months ago. I would say that that's a bit tongue in cheek, but it's really true. To think about where we were six months ago is actually quite bizarre. I mean, now you can generate it with audio and it just brings it to life. Like, video and audio that is so in sync is really super captivating and creative. People have, like, been stretching it to full. Full, you know, past eight minutes, past 15 minutes, past 20 minutes. And I find myself getting lost in some of these generated videos. Uh, and the other thing is scrolling through TikTok and watching all these, like, uh, you know, viral moments that are vo generated, which is also super fun, which wasn't a thing six months ago.

Speaker D: That's right. That's right. Two last questions, one for me, one from Christina to all of you. What are the next set of problems you're trying to solve? I know you kind of revealed a little bit of the roadmap, but any other. Any other things that you're trying to solve?

Speaker A: I'll start. I'll say, how do we bring in more of the personal context? Right. We talked about bringing in photos. Ben's talking about bringing in videos. Like, how do we bring in even more richness and depth of. Because people have archives of my grandmother's slide, uh, footage or video footage, or my grandfather's audio recordings of his personal journal when he was traveling through Europe or whatever these things are. How do we bring those in and then generate from that with such a, um, fidelity and a respect for the personal sort of content that is being brought in? That's. That's what I'm thinking about day and night these days.

Speaker C: I think, you know, Ben and John really taught me how there's such a poetry to the prompt, and it's something that I would really like to master myself.

Speaker B: Wow. Nice. Yeah. And the prompting strategy completely changed with VO3, so I'm, uh, overhauling my brain with that as well. I am also looking to other areas that generated video can apply to beyond filmmaking, uh, and how it might be a utility, um, in places that it wasn't before. Um, I'll leave it at that.

Speaker E: And final question, I guess, and this is kind of a big one. So what is the biggest learning that you've taken, I guess, in the last year about, like, the future of the film industry?

Speaker C: Oh, boy. This is the last question.

Speaker B: I can reuse my answer that I said on the Tribeca panel that. Corey.

Speaker C: Yeah. You start.

Speaker B: I mean, uh, when we premiered Tribeca, uh, the three of us and others, uh, were on a panel. Corey, uh, asked me, as the final question, he just threw a curveball and said what was the most powerful, most impactful moment of this process? And I still stand behind what I said then. The most impactful moment for me was watching Audrey, the actress, um, basically summon deep emotion and cry for nearly two hours on command with, uh, Eliza directing her. Um, and I have seen a lot of acting, I've directed acting, and I had never been moved like that nor seen that ability to channel that. So I think that that, uh, is a human experience that is going nowhere. It's only going to get deeper and more important and impactful. Emotional stories are only, uh, going to spread further and wider.

Speaker A: That's awesome.

Speaker C: That's a great answer, I feel. Could add no more there. That's right.

Speaker B: Maybe I can add one more. And you guys can use this if you want.

Speaker A: Sure.

Speaker B: Um, I find myself being an optimist, so I would like to say the future is bright, and I would challenge, uh, all the people that are in the film industry to, uh, look at this as a moment of responsibility and ensure that it is bright that it is being used. These tools are being, uh, shaped and developed in a way that is respectful of the humans that have laid the ground before, um, and will ultimately lay the ground for humans to come. Um, and so, from the optimist angle, yes, please, let's keep the future bright. Uh, and, um, yeah, that's good.

Speaker A: Yeah, I think I might use that. That's good. That's really good.

Speaker D: Yeah.

Speaker E: Yeah.

Speaker A: I'll say. What I think about most in terms of the future of the filmmaking industry is. Is the. What was in. You know, inspiring for me to hear and to understand as someone who is so technical and working so deeply with the research and the technology, is that it still always boils down to storytelling. Right. There's no innovation that can stand on its own. There's no technology that can stand on its own. It requires a storyteller. And that doesn't mean you have to be the best storyteller in the world. That just means that you have to be able to tell a. And hopefully these technologies can help more people tell more stories.

Speaker C: Yeah. I think to that end, as a filmmaker, I've spent a lot of time asking for permission to be able to tell my stories. And the beauty of this technology is that you can just go do it. And I think that's what's really, really exciting for me about the future. With these tools responsibly in our fingertips.

Speaker E: I love it.

Speaker D: What an incredible way to end. Thank you, all three of you, so much for your time. For such incredible answers. We will link the film in our show notes. We will link the making of the film in the show notes as well. Um, and we are just so honored to have you here and to share your stories.

Speaker B: Thank you guys so much.

Speaker C: Thank you, everybody, so much.

Speaker B: I really appreciate it.

Speaker D: Thanks for joining us. For more information about our guests today, please check out the description.

Speaker E: And for more great conversations like this, hit that subscribe button.

Speaker C: Until next time, thanks for listening.