Bad Decisions Podcast · 2026-10-31 · 1h 4m
Key moments - from our scoring
Substance score
33 / 100
Five dimensions, 20 points each
The Bad Decisions team explores whether AI can democratize game development in Unreal Engine for non-coders. While the hosts are experienced 3D artists and designers who've created Fortnite experiences using Blueprints and UEFN, they've historically hit walls when attempting full game creation without programming knowledge. Two experiments showcase AI as a potential co-pilot for bridging this gap - allowing artists to focus on creative direction while AI handles technical implementation. The episode opens with a detailed breakdown of legal tension between Higgs Field and ComfyUI: Higgs Field sent a cease-and-desist letter demanding ComfyUI remove its "Open Source Higgs Field" challenge, citing trademark violations and unauthorized use of the CEO's likeness. ComfyUI fired back, noting Higgs Field's own history of using celebrity likenesses and movie posters for marketing without apparent permission. The hosts argue that in the AI era, UI and features like Genjutsu aren't defensible moats, making such legal moves feel hypocritical given Higgs Field's own marketing practices. The episode concludes with a teaser about Google's Genie 4, a new world model worth watching.
Hosted by Bad Decisions Studio. The show scores 62.5/100 on substance across its analysed episodes. See the full show profile.
Higgs Field objected to ComfyUI's 'Open Source Higgs Field' challenge for two reasons: using the likeness of Higgs Field's CEO in promotional videos and using the trademarked 'Higgs Field' name to attract attention, arguing this goes beyond fair commentary into unlawful use.
A cease-and-desist letter is a formal warning demanding someone stop a specific action, typically sent before legal action. The receiving party can ignore it, negotiate, or accept it - ignoring it often leads to a lawsuit, but doesn't automatically force compliance.
No. ComfyUI's CEO responded that the challenge is staying up, arguing that Higgs Field had promised to open-source its software but failed to deliver, and that Higgs Field itself uses celebrity likenesses and famous movie posters in its own marketing without permission.
Yes, according to the episode's experiments. AI can act as a co-assistant alongside tools like Blueprints and UEFN (Unreal Editor for Fortnite) to help 3D artists and designers create full interactive game experiences without traditional coding knowledge.
Zoom API is a zero-markup routing platform for image and video generation that forwards requests to providers like Stable Diffusion and Minimax without the 50-414% markups charged by Higgs Field and Runway, replicating Higgs Field's UI while monetizing through feature add-ons instead.
Our reviewer’s read on each dimension, with quotes from the episode.
The Codex/Unreal Engine section contains genuinely substantive workflow detail - ragdoll physics, keyframe animation generation, mocap cleanup, cloth simulation - that an operator in game dev or creative production would find useful. However, the episode is heavily diluted by minutes of irrelevant filler (food stockpiling, food ordering, the 'orc debate'), and most analytical statements stay at the level of reaction rather than insight.
it goes back to the idea abundance of creation. Right. Yes, you can create. Create non stop. But not, not create non stop. Everybody can create.
if you have no knowledge of cloth simulation, you cannot make this at all. Even if you watch any tutorials. You know why? It's so complex. When you see the hands, it's not colliding through the hands.
The first-hand prototype walkthrough of Codex building a Superman game in Unreal Engine is a rare practical demonstration with some original workflow detail, but the broader analytical takes are recycled ('open source is good,' 'UI is not a moat,' 'world models = VR future') and the Genie 4 speculation never advances beyond surface-level brainstorming.
if Jack Dorsey is saying software and code is no longer a moat, UI definitely is not a mode. UI is definitely not a mode. These features like Genjutsu are not a moat.
Technically speaking. Why is lucid dreaming even a thing? It's fun because you can do whatever you want. And you can spontaneously decide on the spot and the world model in your mind reacts to it.
There are no external guests. The hosts are content creators and Unreal Engine educators with genuine hands-on experience, and Pratt provides useful technical commentary, but none are senior operators or practitioners at scale in games, enterprise software, or relevant B2B domains. A brief clip of an external founder (Zoom API) adds no interview depth.
we've been 3D artists for the longest time. We know how to design sets, we know how to create models, we know how to do texturing, how to make cinematic scenes.
we create Fortnite experiences. That's partially thanks to Epic Games and Tim Sweeney
The episode has notable pockets of concrete specificity: prize amounts, GPU model names, specific AI model versions, percentage markups on API costs, and a detailed first-hand account of the mocap cleanup ordeal with timeline and cost context. These are undercut by frequent vague exclamations and unverified claims (e.g., '80% of Roblox games are AI-made' with no source).
$10,000 for the best overall. We have RTX 5090. Two of them. No, uh, three of them. So we have the most innovative category, we have the most viral category
There were at least a handful of animators who rejected working on the project with us because the timeline was too short. We needed to fix it in a, uh, in one or two weeks.
The co-host format produces occasional productive pushback (e.g., challenging the 'orc' framing, pressing Pratt on model settings and token usage), but the conversation is mostly circular and self-reinforcing, with long tangents that go unresolved. There is no external guest to probe, and follow-up questions are mostly descriptive rather than analytical.
What was the reasoning level? Was it high or extra? High or extra.
Is this like one of your orc stories?
Computed from the transcript - who did the talking, and the words that came up most.
Higgsfield just sent ComfyUI a cease and desist over its Open Source Higgsfield challenge, and ComfyUI's CEO replied with two words: the irony. Then we show two experiments with Codex inside Unreal Engine. The first is a playable Superman prototype with guns, ragdolls, cape simulation and laser eyes, built with no templates. The second is a samurai mocap cleanup that cost us three weeks the first time. Finally, Google's Genie 4 world model, and why it feels like lucid dreaming in real life. Try Autodesk Flow Studio: Ready to level up with AI?
Transcribed and scored by The B2B Podcast Index.
Speaker A: What is uh up beautiful people? Welcome back to the Bad Decisions show where we bring you guys the latest and the greatest and the hottest in the world of tech. And damn, today we've got some amazing stuff happening.
Speaker B: You forgot to say every Tuesday and every Friday we come to you twice a week.
Speaker A: Every Tuesday and every Friday. That's right. Thank you, thank you for that reminder. Today is going to be really hot because we're going to start out by talking about some beef that is happening on the Internet between Hicksfield and Comfy ui. We're going to break it down. It's really fun. Next, we've got two hot experiments with AI and Unreal Engine. Essentially showing you guys that you can now make your own games with Unreal Engine by using AI as your co assistant. These two experiments are going to blow your mind, guaranteed. And last but not least, we're going to talk about Genie 4 which is Google's new world model. So a lot of stuff to uh, unpack. But before we begin talking about packing and unpacking, Farad, have you Stockpiled and packed 90 days worth of food surplus and drinks and batteries?
Speaker B: What is going on? The governments are asking people to stock up for water, food and batteries
Speaker A: without telling you why. What
Speaker B: is actually happening? They're
Speaker A: just like, hey, why don't you go ahead and stockpile 90 days worth of food and batteries? First of all, batteries. What? Power
Speaker B: banks. Power banks? You know, to charge your phone or charge your electric devices. I saw UK Prime Minister talk about it yesterday. Apparently Japan did it. Thank God nobody talked about it here in this region. So I think we are safe. We are not packing or stopping.
Speaker A: What's the reasoning behind this? Because our mom was the first person to warn us about it and me and Farah just brushed it over like what are you, what are you talking about?
Speaker B: What?
Speaker A: Yeah, this sounds like uh, we called it yellow news.
Speaker B: Yeah. Uh,
Speaker A: in Persian.
Speaker B: Like fake news. Yeah.
Speaker A: Which is like fake news. And then we went on to see major publications and news channels talk about the exact same thing the following day. And so it's just blowing my mind because it feels like it's something from a movie. Yeah. And it's always followed by a cataclysmic event. But
Speaker B: aliens, tornadoes, earthquake, cybersecurity attack, I don't know,
Speaker A: AI hacking? Judgment Day.
Speaker B: Judgment Day.
Speaker A: But again, have you found out anything about why?
Speaker B: Nothing.
Speaker A: We have to do this. All right, if you guys know,
Speaker B: let
Speaker A: us know in the comments. We ah, just told you the news. We don't know why it's happening. I've been trying to look into it. No one has answers. It just sounds really strange. But I'll tell you something. We're not stockpiling on
Speaker B: shit.
Speaker A: We're not. I hope I'm not going to regret saying this, by the way. I hope I'm not going to regret saying this.
Speaker B: Do we have power banks? Pratt in the studio?
Speaker A: Yeah, two.
Speaker C: We have two power banks, three packets of instant noodles, two Coke Zeros.
Speaker B: Okay, we're good. We're good.
Speaker A: What, for an hour? We're good for an hour. With
Speaker B: that, we will do two
Speaker A: zeros. Did you hear that? Three Cup Noodles. I
Speaker B: think we have some tequila.
Speaker A: Sure. That's gonna. Actually, you're gonna need that. If you've watched Pirates of the Caribbean, when you go out on long sales, the only thing that will keep it fun is just drinking alcohol, apparently.
Speaker B: So we have, we have enough. We have enough stock.
Speaker A: Yeah.
Speaker B: So are you ready, Faraz? For the first time,
Speaker A: I am, um, ready. Let's get started.
Speaker B: Last night, Higgs Field sent a cease and desist letter to ComfyUI demanding that they should take down a challenge that they previously launched called Open Higgs Field. So let's unpack this. What has happened? Hicksfield sent a letter to ComfyUI, the management team and the founders.
Speaker A: A cease and desist letter? What does that even mean?
Speaker B: So a cease and desist letter is a warning for, from someone to another party and asking them to stop and don't continue an action. So when they send this season, this is the. What they mean is stop the challenge and don't do it again.
Speaker A: So that is usually then followed by a lawsuit if the other party doesn't stop, usually.
Speaker B: And the other party has the option to ignore, negotiate or accept. So all the options are on the table. The other party doesn't have to accept it, they can just discard it. They can reply it or they can accept it and stop. To be
Speaker A: clear, this is not a lawsuit. No, this is
Speaker B: not a lawsuit.
Speaker A: It can turn into a lawsuit.
Speaker B: So this is a cease and desist letter for now.
Speaker A: So why are they sending this?
Speaker B: So let's go to ComfyUI's, uh, website previously, what they have launched, they have a challenge, uh, called. So on the left you will see Hicksfield, uh, CEO. On the right is Comfy UI CEO. So the challenge is called Open Source Higgs Field. What they are saying that three weeks ago hicksville promised they're going to open source their software. If you talk
Speaker A: about it, if you remember
Speaker B: we talked about it, we went to their GitHub. We said that they open sourced a few skills. It wasn't the full platform or program. They said since Hicksfield didn't open source the software, let's put a challenge on whoever can open source Hicksfield. You're going to have different templates, different skills. Of course you can have integration into open source models to create images and videos and, and the winner will get up to $25,000 if you go down. You would see that. And
Speaker A: there were GPUs as well, right?
Speaker B: Yeah, yeah. The prices are actually quite good. Can you, can you scroll down? Let's see the prices that they have. So $10,000 for the best overall. We have RTX 5090. Two of them. No, uh, three of them. So we have the most innovative category, we have the most viral category and we have the open weights only. So apparently you can even use your open source platform to connect it to Sitdance through API so that that is even allowable.
Speaker A: Are you supposed to remake the entire Higgs field or just specific parts of it? So for instance, the video we just saw, that was from Genjutsu's feature
Speaker B: Scroll down, you will see that they even called it out. So scroll down, scroll down. So for example, you can have openjutsu as a feature. So the best overall probably is the most complete. So you will have openjutsu, you have Open Tana or whatever features that they have. So. Or the most viral. Maybe somebody wants to just focus on, focus on Genjutsu and make the best version of Genjutsu. But with minimax, uh, H3.
Speaker A: Wow.
Speaker B: So. So there are different categories and I think it was a cool challenge. What they are actually demanding in the letter, if you go back to the letter, what they are saying is that actually, let's watch the, let's watch the video that they did so people will understand what Hicksfield is demanding. Can we watch the entire video? Yes. And you can play the song.
Speaker A: He said promise to open source.
Speaker B: They didn't. Let's help
Speaker A: them. Pause, pause, pause, pause, pause, pause. This song is going to be copyrighted, by the way. I'm just letting you know.
Speaker B: Did it got copyrighted last time when we did the show? Okay. Muted frat. You guys know this.
Speaker A: I'm just letting you know. Okay, play.
Speaker B: So basically what they're saying, uh, of course the song is going to play at the background, he Hicksfield CEO on the left and then you see comfy UIC on the right. So there are two areas that the Hicksfield team is highlighting. One is using the likeness of the Hicksfield CEO in the promo video.
Speaker A: So. Pause right there. Uh, so essentially one of the things they are saying you should stop doing is stop using
Speaker B: CEO. You are asking is you have to take down the challenge. Why? Two reason. You use the likeness of our CEO.
Speaker A: It's not the likeness. It is the CEO
Speaker B: likeness of the. No, they call it likeness of. When you use Matthew McConaughey. Is the likeness of Matthew M. McConaughey.
Speaker A: Sure, yeah.
Speaker B: So they use the likeness of the CEO and of course they name the challenge Open Hicks Field. So they use the name Hicks Field in the season. This is letter. They, they said that Hicksfield is trademark. This is the trademark number. You cannot use it. Right. And what they're asking is, oh, by the way, in the letter, something I, uh, found really interesting. They said, we are okay with commentary, we are okay with reviewing the product. But now you are using the name of Hicksfield to attract attention. That's the part we are not okay with. If you open the first page of the letter, Hicksfield does not object to open source software.
Speaker A: Where is that?
Speaker B: Yeah, yeah.
Speaker A: Uh, yeah.
Speaker B: So Higgs Field does not object to open source software, to competition, or to a commentary about Higgs Field. So what we are doing is actually perfectly fine.
Speaker A: Yeah. We take no issue with Comfy accurately naming Higgs Field, comparing products, or inviting developers to build their own tools.
Speaker B: The campaign goes beyond that and becomes unlawful.
Speaker A: Why?
Speaker B: Again, the same thing. Using
Speaker A: names. The competition, Open source Higgsfield. Yeah, but, I mean, yeah, of course,
Speaker B: but they are. Again, you're using the name of Higsfield. But what did the.
Speaker A: But if Higgsfield is using other names in their own videos, that's fine.
Speaker B: I mean, nobody sent them. Probably people have, uh, sent them letters.
Speaker A: Like, if they use After Effects or Premiere Pro in their videos.
Speaker B: That's a very valid point because. Yeah,
Speaker A: then After Effects can be like, why are you using After Effects in the title of your YouTube video?
Speaker B: That's why I think if you look at. So
Speaker A: using names is fine. That's not a problem.
Speaker B: Uh, if you look at Hicksfield CEO's reply. So you scroll up, scroll down.
Speaker A: Higgs Field CEO's reply.
Speaker B: No, sorry, comfy UI CEO's reply. So he mentioned that their argument is that our, uh, use of Hicksfield name and that using their CEO's likeness is an infringement violation. Man. The reason they are saying that is because Hicksfield has the habit of using celebrities, movies, uh, other creators. And
Speaker A: also, I will say this before I even say my statement. I made this Clear. Countless times before, I personally and the team here at Bad Decisions uses all of these tools. We use Higgsfield, we use Invideo, we use Comfyu, uh, I use all of them for the research purposes that we do for experiments, for our ultimate AI program. And the thing is, although I believe Higgs Field's team has an unmatched marketing capability compared to any other company in this space, we've seen this, we talked about this. In fact one of our first episodes in the show was about this, is that some of their marketing tactics has been in the gray area when it comes to what's right and what's wrong. And they've done this countless times. They've done it to motion designers, uh, they've done it to 3D artists, they've done it to 2D artists where they've uh, I've seen creators literally post that their work was shown on Higsfield's website, uh, without their consent. So they've used let's say an after effects template from somebody who's posted it on Envato or on their Instagram. And, and it's been shown as a marketing video on Higgsfield. And later I believe Higgsfield actually took it down. So it's not the reason why he's saying man. The irony is because this has happened so many times from Higgsfield that when someone else has done shouldn't sound so bad. But apparently to Higgsfield it does sound bad.
Speaker B: But this uh, Comfy UI guys keep going and saying that this couldn't be further from the truth which is that Higgs Field made a promise to open source this product and, and never did. For the past two weeks builders have built, have been recreating popular closed sourced app as open workflows anyone can run. And what they said is basically long story short, the challenge is going to stay up. They are not going to bring the challenge down. Game on. So you can go, but uh, scroll down to the comments people actually have been sharing. So this is the part that they were saying the irony. If you look, these are movie, famous movie posters that Hicks Field has used to promote their own products. But they of course they change. And
Speaker A: that's Dwayne Johnson.
Speaker B: That's Dwayne Johnson.
Speaker A: Akira. Uh,
Speaker B: yeah, we have
Speaker A: Spirited Away
Speaker B: and that's uh, the last one is the Spider man actor, what was his name?
Speaker A: Uh, Tom Holland and as Naruto.
Speaker B: Yes. So they have
Speaker A: Hypocrites Fields.
Speaker B: So again if you go through the comments and look at everything you would see that the community is actually supporting open source. And I think it has been clear in even outside of this case, the community is really supportive of open source. And I think open source is a force, uh, for good because it led people to do experiments, it led people to create on top of other models, on top of other softwares. And I don't think this would cause any harm to Hicksfield in the long run. I think, again, this is also another great marketing for both ComfyUI and Hicksfield because it's top on X. But, um, the challenge is going on. People have been submitting. Let's see what would be the next step.
Speaker A: Let me ask you guys this question again. Where I stand is if Hicksfield was absolutely clean, as in they have not done any of these marketing tactics themselves. Uh, I don't need to even mention. I think you guys have heard us talk about it. Uh, people have talked about it on X. You're seeing it here yourself. If they didn't do it themselves, I would understand why they'd be mad, but because there is proof from people. Again, my own work hasn't been on Hicksfield, so I cannot claim they took my work. But I've seen so many posts from other creators in different industries, uh, claiming that their work has been seen on Hicksfield. Uh, I'm not even talking about the films or the actors. I'm talking about creators in the industry. That's why I'm thinking it sounds a bit strange now that ComfyUI is doing the same thing. Uh, Higgs Field is getting mad,
Speaker B: but I don't think they would get anywhere with this case because again, ComfyUI, for those of you who don't know, is a platform which lets you create visuals generative AI with open source models. You can of course use closed source, but there's a huge library of open source models there. They've been doing this for the longest time and they are a big company. So I think the challenge will go on. Actually, we should look at the winner who wins. Let's see how. What?
Speaker A: No. What do you think about it though?
Speaker B: No, I think what Comfy UI did is perfectly fine. It's a challenge and I think we are going to have that in the future. Especially with the advent of AI. We are going to. How many YouTube videos are out there that the guy say, I rebuilt my Photoshop, I revealed
Speaker A: just not. There's another guy. I forgot his name. I sent it to you. Pratt. Maybe you can find it on the site in the messages I sent you. There's another Guy whose entire company is copying Hicksfield.
Speaker B: Oh, he the one that is, uh, sending the APIs with the
Speaker A: lowest markup. No, no, not, not lowest. No markup on APIs.
Speaker B: Can you scroll down? I think he commented, scroll down. No
Speaker A: markup on APIs.
Speaker B: He said, Time to use this now. Like he, he go
Speaker A: down. Maybe you can find it. Even if you don't find it, I have most definitely sent it to you. Uh,
Speaker B: I saw his comment, but
Speaker A: I'll tell you
Speaker B: anyways.
Speaker A: There's this guy, by the way, investment from founders fund, I think. So legit. Uh, it's not just a random person. And he literally created a platform, claims there is zero markup on the API costs and the entire interface is also like Higgs Field. And there's people asking him, hey, okay, this is really cool. I appreciate you doing this. I definitely want a cheaper price for my videos and images that I'm generating. But I'm just wondering what's the point of copying Higgsfield? He's like, that's exactly the point. It looks exactly like Higgsfield. And I'm showing you that someone can do this and that you don't have to pay so much more for the API cost of cdance and nanobanana and so on and so forth. And there's people rallying behind him too. The only thing is, I think perhaps the size of his company is still too small for Higgsfield to even consider, which. Whereas with ComfyUI, ComfyUI is a player that's been there, uh, perhaps even longer than Hicksfield.
Speaker B: Yeah. Oh, of course their valuation is not.
Speaker A: Oh yeah, David did. That's the guy. That's the guy. Yes, yes. So just play the video for us. Play the video for us, please. Thank you.
Speaker D: Video model providers like Hixfield API Fall and Runway mark up the models by 50% and even up to 414% by just routing the models. As a builder in the AI video space, I was so terrified. So today we're launching something really big that will be hated by the industry. Introducing Zoom API. Yeah, ever zero markup router API for image and video generation models with Zoom.
Speaker A: Let me know if you guys have used this. By the way, we haven't tried it ourselves.
Speaker D: Uh, but she's
Speaker A: very clear about the fact that the look of it is Hicks Field. It's working exactly the same way. And the way he believes they're going to make money is by selling you features. So you get to generate videos yourself. But things like genjutsu and many other features then you'll pay for that as an extra.
Speaker B: But you see, in the age of AI, and I think the rules and regulations has to change also. I remember previously, was it, uh, VOP that was suing another company because their UI looked the same? The tracking app that you have on your hand, there were a lot of this. I think Spotify sued another, uh, another
Speaker A: company. That's Whoop.
Speaker B: Whoop. Sorry, sorry. BOP is the editing app. Sorry. So Whoop was suing a company for their ui. Spotify was doing it with the age of AI. Anybody you
Speaker A: know, the problem is. Yeah, the problem is I think this is not happening at the top, top, top level, because they already know this. They're fine with it. For instance, we know we've covered this so many times. ChatGPT and Cloud practically look the same. They're copying each other's naming
Speaker B: of the features
Speaker A: and they're literally following each other's features back to back. And it goes both ways. It's like a highway. And they're not sending each other, uh,
Speaker B: lawsuits or season
Speaker A: letters. They know what they're doing. But here, Higgs Field is perhaps not. Perhaps they are the biggest player. Uh, of course, comfyui, if you're talking about open source, is the biggest player. But, uh, generally speaking, when it comes to pop culture and what I'm seeing on TikTok and Instagram, more people are using Higgsfield because of their powerful marketing machine.
Speaker B: User base is very different. People who are using Comfy probably very
Speaker A: different. Yes. Comfyui is purely artists.
Speaker B: Artists
Speaker A: and developers.
Speaker B: This, this fight won't even bring. For example, Sumo can even. You can sue Sumo or
Speaker A: Zoom.
Speaker B: Okay. You can go and sue Sum.
Speaker A: Yes.
Speaker B: And say that, hey, you stole my users. Higgsfield can do that. Right. But, uh,
Speaker A: stole my user. You cannot say you stole my users. No,
Speaker B: no. By copying, you know, I lost users, so I'm losing money. Don't copy me. Comfy UI users are very different. Like, people who are using KSPL are not going to go use ComfyUI. It's a very different user base.
Speaker A: My conclusion, my opinion is this. In summary, I will continue to use both platforms. I like both. Uh, I do think it is kind of hypocritical for Hicksfield to get mad at such a small, uh, thing. Again, in my opinion, they're bigger than this. They don't even have to care about this. They can just continue. And thirdly, as you mentioned, as we move forward, I think even in court cases, they're no longer going to accept such a thing. Because if Jack Dorsey is saying software and code is no longer a moat, UI
Speaker B: definitely is not a
Speaker A: mode. UI is definitely not a mode. These features like Genjutsu are not a moat. And so in my opinion, even courts themselves slowly won't pick up these cases. Like, don't even bother coming here. Yeah, we're not going to take it.
Speaker B: I agree, but
Speaker A: it's fun. We just wanted to share with you. Let us know what you guys think about it. Guys, one of the biggest problems people have when making AI videos right now is lack of control. You have an idea in your head, but what you get is, is something entirely different.
Speaker B: You can generate incredible looking shots with a video model like sea dance 2.5, but getting the exact camera movement, character position, animation, and even composition that you have in your head, it's still really difficult.
Speaker A: What creatives actually want is to be able to direct the scene. Imagine this. You have the idea for the world in which your story lives. Describe it and watch it turn into an explorable 3D environment within minutes.
Speaker B: You have the idea for the characters. Describe it and watch it turn from an image to a fully rigged character.
Speaker A: But wait, the character has no animation. That's fine. Just record yourself performing the motion and the character will copy you in minutes.
Speaker B: Now place those characters in the scene. Place your cameras in the scene. Position everything as you first imagined it in your mind, move your camera exactly the way you want it, and now you have full control.
Speaker A: And now it gets even more interesting. You can take that entire 3D setup and use it as a guiding reference for AI video generation. Instead of asking AI uh to guess what you mean, you're actually showing it. And then you let the AI render your scene into a fully cinematic experience exactly how you first imagined it.
Speaker B: These are the new 3D editor and canvas of Autodesk Flow Studio. If you want to try it for free, use the link in the description.
Speaker A: Next up, we're going to show you guys our experiments. We wanted to see, can you make games with Unreal Engine and AI Even if you don't have much of a, uh, coding background? This is the case for us. We've been 3D artists for the longest time. We know how to design sets, we know how to create models, we know how to do texturing, how to make cinematic scenes. But our abilities ended by creating cinematics. We could never really create video games. We could never. We did, though, as bad decisions. We create Fortnite experiences. That's partially thanks to Epic Games and Tim Sweeney and the team by Providing us with the tools that allow us to make games, even if you don't know how to code, such as blueprints and ah, uefn and uefn. But then that also had a cap, a limit. You could never really go and make your own Superman game or like Hogwarts Legacy or a, uh, Call of Duty, because you'd be limited by what the developers of Epic Games created for you. We know now, with the advent of AI, that we have so much more capability when it comes to modeling. You can 3D model things with AI. We've shown that before. You can texture things with AI, you can animate things with AI, but how much further can you push it? Uh, can you start building out a game? If you had the idea in your mind, can you start prototyping a game? And I think the term prototype is very important because when you're making games, if you've ever made games, you will know this. You start by building prototypes. So you start by setting up your level design, and you start testing features of the game. And you put one and two together like a puzzle, and you start putting three, four, five, until you come up with the basic features of your game. Then you start building the environments. Then your first level, your second level, we said, okay, let's just, uh. As that week is passing by, Pratt is going to have Codex on a side, right? And have codecs use computer use to control Unreal Engine and use MCB to control Unreal Engine. But by the way, it's free to use Blender or anything else. But ultimately we want to go ahead and start creating the idea that we have in our mind and turn it into a game piece by piece. We're going to show you guys what that experiment looks like and how it blew our mind. Because what we're showing you was not possible before. What you're looking at here is Unreal Engine 5, a, uh, 3D engine, a game engine. And almost everything you see was created by Codex. So what do we have here? First of all, we got a character. He's able to pick up a gun, shoot the gun. The gun will have recoil animation. There's bullets flying out. When you shoot at characters in the game, there is ragdoll physics. So Pratt is shooting the character there. The character falls on the floor with ragdoll physics.
Speaker B: Uh, question, guys. Did we use any template? No template, right?
Speaker A: No. No templates. No templates.
Speaker B: Everything is made from scratch.
Speaker A: Yes.
Speaker B: And before everybody asks in the comment, can you tell us what model you use?
Speaker A: Codecs. We're using, uh, Pratt, what are we using?
Speaker B: 6.1 SOL.
Speaker C: Yeah,
Speaker B: 6.1
Speaker C: SOL. I tried Terra. We know the answer. It's not that great.
Speaker B: Yeah, Terra didn't work. And what was the reasoning level? Was it high or extra? High or
Speaker C: extra. It's high.
Speaker A: It's high. Now, Pratt, can you try to kill one of those characters once again? I don't know if. Farad. If you picked up on the small details again, I will go down into the rabbit hole of telling you how much time these details would cost. But then look at the blood that seeps out from the wounded.
Speaker B: Yeah.
Speaker A: All of these things were coded by codex from the animation. So when it's picking up the gun, that animation, you had two options traditionally. One, you would go and buy an animation pack from the store. Or two, you would animate the character yourself.
Speaker B: Yourself? Yeah.
Speaker A: Either by hand, keyframing or motion capture.
Speaker B: Yes.
Speaker A: All of which cost time and money. Either time. Either a lot of time or a lot of money. If you compile all the animations for your game, if you're buying all of the animations, that would be a lot of money. You.
Speaker B: True.
Speaker A: If you're animating it yourself or doing mocap. We've done that before. A lot of time. Here. The animation again. Pick up the gun again, please. The animation. Picking up the gun and then pointing the gun, shooting it with the recoil of the character. All of it animated by Codex.
Speaker B: By keyframes.
Speaker A: By keyframes on its own. That is fucking insane. Okay, did you see the reload? Can we try to reload one more time? So if you notice, the reload is like how,
Speaker B: uh,
Speaker A: kind of pushed the gun in the air, pulled out the mag, put the new mag in. It's dependent on the gun because the pistol's animation would be different than the assault rifle. And. But, uh, Pratt, correct me if I'm wrong. The pistol and the rifle modeled by Codex.
Speaker C: Yeah, uh, it's modeled by Codex.
Speaker A: Uh, the table and the range and the characters.
Speaker C: Everything you see is modeled by Codex. The way they're walking. Okay, the walking animation, I think it's the unreal default, uh, animation. But there's like, it has adjusted it according to Superman. For example,
Speaker B: why we have a cape.
Speaker A: Okay, so that's the next part. That's the next part. Prad. You want to show him?
Speaker C: Why do we have a cape? Because you can float
Speaker A: and you can fly.
Speaker B: What the.
Speaker A: And there is cloth sim on your cape. And, Pratt, uh, your eyes work, too, right? Do you have lasers?
Speaker C: My eyes work, too.
Speaker A: So Pratt can shoot lasers and pay attention. He can burn anything. He can burn the floor, and the floor deforms. Of course, there's a bug there, if you notice really quickly. But we can fix all those things. Again,
Speaker B: we are doing also real time here, right? This is not a video real
Speaker A: time test.
Speaker B: Uh,
Speaker A: the laser's coming off his eyes. The floor kind of turns into lava. Molten lava deforms. And then over time, the. The deformation bounces back to normal, just like, uh, any other video
Speaker B: game.
Speaker A: And then. And then, moreover, if you hit the guys, can you hit one of the guys? He burns. He
Speaker B: burns.
Speaker A: And there's texture change, there's material change. Now, if you're a game developer or an artist, I want you, okay, I want you to understand one thing. I know at every step of the way, you're like, but wait a second. The lava looked ugly, or the material change looks ugly, or the gun animation is ugly. Understand something. Every one of these things, one can be manually changed, okay? And two can also be enhanced. We didn't try to make each of them perfect. We're just trying to test as many things as possible and see if all of these interactions work. The fact that Pratt could go in and punch a guy and their ragdoll worked, or the fact that he could burn one of the guys and then his body would be set on fire, all of these are interactions that you had to manually do through blueprints or code yourself. The fact that you grabbed a guy right now, did you grab one person? Yeah,
Speaker C: I grabbed him and I burned him.
Speaker A: Grab and burn. Uh, all. Any idea? So all of these things, by the way, are mini interactions of Pratt and Codex. It's not like, go and build me a Superman game that can shoot.
Speaker B: It's like working with someone and then they will do it.
Speaker A: Or when you're doing it yourself, you would create these features one by one. You were like, I want to be able to grab someone. And they would burn, burn
Speaker B: it.
Speaker A: And then the fire animation. If you hate it, because, honestly speaking, none of these things, by the way, in their first turn are good, you go ahead and enhance them even further. We'll show you guys our chat in a moment of how we achieve this. But fire, you understand we're trying to showcase all the different things that you would have to work with. Setting up the environment, testing the physics, setting up ragdoll, uh, creating material changes, switches. So for instance, in blueprints, it would be like, on hit, change material from normal to burnt, but then to coding. That would be a lot of lines. So you can then go ahead and now enhance these by buying packs from online or doing it yourself manually or having codecs do it for you.
Speaker B: Uh, I think I truly believe computer use and MCPS are the future for creation. Because when you see something like this, the biggest challenge with when you go and generate using Sitdance, or when you are generating using any generative tool. Right. You don't have any checkpoint. Your editings are limited here. At every single point, whatever Codex has done, you can go and change it without touching the other part. If you don't like the lava, just change the lava. If you don't like the animation, one part of the animation, just go and change that, but the rest stays the same.
Speaker A: This is while Pratt was working, this was happening on the side. And by the way, we just want to show you this as an experiment. Imagine if you have five, 10 people working, but religiously.
Speaker B: Yeah.
Speaker A: Working on this and changing things themselves while Codex is helping them. The amount of things you can achieve is.
Speaker B: That's why Team Sweeney was unbelievable that the games that they could never get made. These games. This is it. Uh,
Speaker A: when could we be working on this show and so many other works and have Pratt also make a Superman game and prototype it on the side? Never.
Speaker B: Never, never.
Speaker A: And half of these things, by the way, for instance, the deformation of the floor and it coming back up to this day, I still don't know how to do that. I can watch a tutorial learn it, but I'm saying there are things you can do that you also couldn't do yourself.
Speaker B: Makes sense.
Speaker C: It's. It's. You know, the deformation is just not. It's not visual. You can actually walk inside.
Speaker A: The character is going down.
Speaker B: The mesh has changed. Right?
Speaker C: Yeah. These things are really difficult. Like, I think, uh, we were really impressed when we watched the trailer by Unreal Engine. You know, the Captain America one?
Speaker A: Yes. Yes.
Speaker C: Oh, my God, that looks so cool.
Speaker A: Yes. You can do things like that.
Speaker C: Yeah, it's like just one prompt. Oh, hey, I, uh, want laser on the eyes. Can you make the ground deform when I'm focusing the laser on the ground? And it did.
Speaker A: You can probably even go ahead and do crazy things. Like, you know, you just try to burn the wall. Right. You can even save. So part of the mission is you have to burn a specific shape on the wall or burn the door to be able to go through it. So many things you can do.
Speaker B: Can you look at the. Behind the scene?
Speaker A: Uh, no, before the behind the scenes, I want to show you guys, the different state of the flying also. So it's not just fly. So, Pratt, if you try to fly, you'll see there's a smooth bend of the knee. And did you see how it started levitating? That was intentional. Pratt was telling it, hey, your first flight animation sucked. Can you ease in and out of the flight animation by bending the knee? And then when you go up, then when you continue to fly, you'll see there's the reaction to your cape. Then your hand goes further. I remember this was, I think two nights ago or yesterday night. I told Prad, prad, make sure when he's flying, his hand is up like Superman. And Pratt was like, don't worry, we're already doing that.
Speaker B: That's really cool. That's really cool.
Speaker A: By the way, is the cape animation great? No, but can you make it fucking amazing? Absolutely, yes.
Speaker B: Again, prototype,
Speaker C: if you have no knowledge of cloth simulation, you cannot make this at all. Even if you watch any tutorials. You know why? It's so complex. When you see the hands, it's not colliding through the hands. When it's flying in the air, there's actually wind simulation pushing the cloth up so it doesn't go inside the body.
Speaker B: Oh, I see. That's amazing. Can we look at the chat?
Speaker A: Let's go take a look at the behind the scenes, Pratt.
Speaker C: Okay. Well,
Speaker A: it's funny because where this all started, uh, is by Pratt just asking it to create the animation for the gun. So can you pick up the gun? Can you shoot it?
Speaker B: I never seen this viewer in Codex. It's
Speaker A: funny. I haven't seen it either.
Speaker B: Your eye is a keyframe.
Speaker A: Yeah. So this is it just trying to shoot. There's recoil, but you look at actually the top and the bottom. This one is better, right? This one look more natural.
Speaker C: No, because I asked it. I was like, uh, can we make the animation much better? My prompt is really basic.
Speaker A: That's a stupid prompt, Brad.
Speaker C: Uh, no, but what I'm trying to test is how intelligent AI is to understand, like, whatever you're saying. Because not everybody's going to type a whole paragraph.
Speaker A: Specific detail of the animation makes sense,
Speaker C: and it understands, like, oh, hey, okay, these are the issues that I have. Let me try to fix it.
Speaker A: But if you are an artist or a game dev yourself, you can guide it
Speaker B: in
Speaker A: much more detail to get it to a level of perfection. Whereas if Pratt is acting like a complete beginner here, he doesn't know how to make the animation. But he's just saying can you make it better? He's essentially pointing the AI towards a direction, but not with any specific direction.
Speaker B: Makes sense.
Speaker A: Not any specific detail, just the direction. It still worked, though. Yeah. It's not perfect. Again, I don't think the animation is that great yet, but. But it shows that you can still get somewhere.
Speaker B: Then you can also give references. Right. What you can do is, even if you're a beginner, you can say, hey, use this as a reference. Create an animation similar to that. What is this?
Speaker A: So this is Pratt asking it to put it inside Unreal Engine. Uh, because that was not Unreal Engine. It was just a character made of blobs.
Speaker C: So you see the left animation. That's what it did for the first time. So it's like,
Speaker A: it's not working right.
Speaker B: It's jittery. Sometimes
Speaker C: it's jittery. It's broken. Then I was like, oh, hey. I pointed out some things like, oh, hey, the hand gets messed up suddenly. The keyframes are not clean. It's not smooth enough.
Speaker A: Right.
Speaker C: Also, when it's reloading, like, uh, the mag, he's not getting it from the right part. And then it slowly started understanding it. Okay, let me fix it. You see? Look at the right one. It's
Speaker A: good. It's much better.
Speaker C: Left one is just random things happening.
Speaker A: Yeah. What else do we have?
Speaker C: We have. So
Speaker A: just let's go over the highlights, because I know this conversation was really long.
Speaker B: Oh, it modeled the gun.
Speaker C: It modeled the gun.
Speaker A: Just imagine for the gun itself, right? You can give it images. No, you can give it.
Speaker B: Generate the images
Speaker A: you want. Again, the point is, I don't want to direct people to buy the packs necessarily, because, uh, if you're trying to make a unique game,
Speaker B: your
Speaker A: guns have to be different. Right. So imagine
Speaker B: image
Speaker A: or. No, take a real gun and then modify it further.
Speaker B: Take a photo of a real gun.
Speaker A: Take a photo of a real gun. Modify
Speaker B: it. Take a real.
Speaker A: No, no. Yeah, yeah. Take a photo of the real gun. So what is this? This is the muzzle flash and the bullets.
Speaker C: Yeah, muzzle flash and the
Speaker A: bullets. Can you zoom in, please?
Speaker C: It was testing it out. Like, even testing, right? Yes. Of course, it doesn't look that great, but it tests everything on its own. Normally, you will have to go in, play the game. Here are the issues. So it plays on its own, records itself, checks each frame by frame. Oh, if the shell is falling on the floor, do I have muzzle flashes? Is the animation clean? Is the mag going correctly inside the gun? It checks everything
Speaker A: that's.
Speaker B: Did you have to reset your codex. Did you go out of credit?
Speaker C: Actually, uh, no. But they have been giving a lot of reset.
Speaker B: Yeah, they give reset every day. But then I'm
Speaker A: wondering how many percentage of your usage
Speaker B: throughout your day did you go below 10%?
Speaker C: No, no, not at all.
Speaker B: That's
Speaker A: actually really.
Speaker B: Imagine building this, going inside, testing. But the good thing about sol, the computer used tokens are very optimized.
Speaker A: Yeah.
Speaker B: So here we are not doing actually that much of coding. It's all computer use. Interesting.
Speaker A: No, but I think there's. There's coding behind the scenes. No,
Speaker B: there is but for testing.
Speaker A: For the logic of the
Speaker B: game. Yeah, yeah, but for testing.
Speaker A: For testing, no. Okay.
Speaker C: Like this is how it created the whole animation sequence. On the left you have
Speaker A: the base.
Speaker C: On the right, it replicated it. Then it was like, okay, I asked it, I want two guns, the SMG and the pistol. It got the SMG out, the assault rifle and. And then it's going to switch to pistol. There you have the pistol. Because
Speaker A: pistol has a different. Yeah. Uh, 2.1 second magazine change. So these are things.
Speaker B: That's cool. You can
Speaker A: change manually. Yes.
Speaker C: Movement for the gun and everything.
Speaker A: See
Speaker C: these videos, right? I haven't recorded it. It did everything on its own. I just asked it to. Oh, can you make it in a sequence so we can see here? Keyframes, individual keyframes. It did everything on its own. 481 key samples.
Speaker A: This is insane.
Speaker C: And then I asked it. Oh, build a shooting range here. It made the shooting range
Speaker A: here. You can make this sexy. You can give it assets you like. Build it with my assets
Speaker B: or give it reference. Make it like, make it futuristic.
Speaker A: The ragdoll. All of these, by the way, the interaction of the bullet with the person, the timing of it, all of this.
Speaker B: What does this mean? Like again, it goes back to the idea abundance
Speaker A: of creation.
Speaker B: Abundance of creation, right. Yes, you can create. Create non stop. But not,
Speaker A: not create non stop.
Speaker B: Everybody can create.
Speaker A: Creating non stop. That's uh.
Speaker B: Actually I agree with you. It's not the point. But when everybody can create, what would stand out is usually something that you have to spend 90
Speaker A: of your time. It's still. In my opinion, it's still about spending time because everyone will spit out many content pieces, whether it's video or game. But the. The difference will be when someone spends. When it is that easy, they still spend three years making something.
Speaker B: Yeah, that's. It's. You know, we were talking about it the other day how social media would be full of AI Videos. Just imagine how many AI games we would have.
Speaker A: There's already. I just saw a post the other day. 80% of, uh, the new games on Roblox are with AI. Made with AI, I think.
Speaker B: Uh, that's crazy. What do we have next?
Speaker A: Wait, uh, Prad is just showing you, like, the laser and everything. Oh,
Speaker C: here, See, when you're talking about the texture, look at the left side. That's what it gave me early on.
Speaker A: Oh, you asked it to change it,
Speaker C: asked it to change it. And then it slowly deforms back.
Speaker A: So you can continue to iterate on this until it's perfect. Yes. Wow. I love these product demos, by the way.
Speaker C: It's really cool. Even the cape, right. For example here, uh, when it was floating, the animation looked really, really bad. Wait, where is it? See, look at the animation. Look
Speaker B: at the hands.
Speaker C: Yeah, it gave some, um, AI slop on the bottom. I have some Niagara particle, some strips. Oh, you. What is this? So I had to go and say, like, oh, hey, this is. Yeah,
Speaker A: even the cape, it looks like a bra at the top. Yeah. So you manually fix all of those by telling it.
Speaker B: It's like a coworker, right? You are the boss and you have to guide.
Speaker A: No, but the beautiful thing is you can work at the same time. Yeah, you're working at the. You can create a new level. You can work on a different project inside Unreal Engine.
Speaker B: Yeah.
Speaker A: Uh, while this is working now, Prat, can we go to the other example? I want to show everyone this one is a completely different example. It's pretty important to us because this specific case is a stress test of AI. Now, a little backstory. When we were working on the ultimate Unreal Engine program, where we taught, uh, more than thousand of you guys how to use Unreal Engine in an advanced manner to create your own short films and cinematics. One of the biggest challenges for us was recording the motion capture for the samurai characters. So there was a fight scene at the end of it. Me and Farhad rented multiple cameras twice.
Speaker B: GoPros,
Speaker A: six GoPros recorded our motion and then had to transfer that motion to a 3D character through Move AI, which took a lot of time. And we also use Autodesk Flow Studio. It's a lot of time, a lot of effort, and even at the very end of it, because of some errors we made during our shoots, we ended up with an animation that was only 70% complete, which means there were lots of artifacts and, um, and jitters during the animation. So the foot wasn't properly landing on the floor, uh, or the hand and arms would cross over, which meant that the animation cannot be immediately used. We had to clean it up. Now, cleaning up of an animation, especially for a sword fight like that, requires an expert team of animators to be able to fix it and make sure that it works perfectly. That's not a job that is easily doable. It took us, uh, a month of contacting different people, and even at the end, trying to fix it ourselves, spending even more money. And I can confidently tell you, at the end of it, we were so desperate, we were trying every single method out there
Speaker B: because their course was launching.
Speaker A: Yes.
Speaker B: And then we had to get the animation to the students to try it out. Uh, uh, for themselves.
Speaker A: Yes. We had so many sleepless nights. And Pratt himself knows, because he tried to use every software out there possible to try to clean this up, even with AI and at the time, we couldn't. No,
Speaker B: because there was a sword fight, and we had our fingers around the sword, and we were fighting. You would fall down. There were so many complex elements into that animation.
Speaker A: So at the end of it, Pratt and a team of expert motion capture cleaners tried to fix it, and they did. But we were like, okay, you know what? Some time has passed. Can AI go ahead and fix this problem? Because if it can, it would have saved us a, uh, month of stress, pain, money, and time. So this is the test we're going to show you guys. The animation on the left is the raw motion capture data, and the one on the right is what Codex did, cleaning up the animation. Okay, so pay close attention to the left side, the foot. Can we go back to the beginning one more time, please? And then go closer on the foot. Look at the foot. Look at the foot jittering like this. And, uh, look at the hands. Look at the wrist. They're intersecting. Fingers are intersecting. He jittered again. And that was a big one. It's not even a jitter. It's a big one. The arms and the hands are all messed up. All of it. And when you put this on a samurai character, it just doesn't work. It just doesn't look good.
Speaker B: It jitters non stop.
Speaker C: Yeah, it's, like, constantly moving,
Speaker A: and at the end, the body is moving as well, like a ragdoll. Physics. Okay, now let's pay attention to the one on the right side, ladies and gentlemen.
Speaker B: Also 6.1. Sold. Pratt.
Speaker C: Yeah.
Speaker A: Okay,
Speaker C: same high.
Speaker A: Look
Speaker C: at
Speaker A: the placement of the wrist on the
Speaker B: sword, the fingers around the. Yeah, so good.
Speaker A: It's on the sword, guys. Okay. Look at the foot. Unbelievable.
Speaker B: Three weeks of non stop working with, uh.
Speaker A: Look at the contact of the swords. Pratt, you specifically asked for the contact of the swords, right? Yes.
Speaker B: How much time and money we would have saved.
Speaker A: I want to be clear about something. There was at least a handful of animators who rejected working on the project with us because the timeline was too short. We needed to fix it in a, uh, in one or two weeks. And they looked at the animation. Look at the end, the sword fell down. He's on the floor and
Speaker B: he's no jitters. Yeah.
Speaker A: We had many animators reject the project because they said they cannot be done. They can't clean it up in like two weeks or three weeks time. Uh, some of them we paid, they still couldn't get it to work. Right. And the reason why we're sharing this with you is because we know how much pain we had to go through and what this means for the future. Again, working on cinematic animation, working on a Hollywood film, a short film. Working on a video game, working on anything that requires complex animation. You now can solve your problems using a simple computer use with, with ChatGPT codecs while connecting it to your favorite tool, whether it's blender, Unreal Engine, Cinema 40. It's insane to me that this problem is now solved after it's been a year now.
Speaker B: It's more than a year now. And the most important part is our creation wouldn't change the final product, wouldn't change the pain that we went through, the time that we wasted, the money that we spent. That is the only part that is changing. Because the output, you still wanted the same output.
Speaker A: I think Pratt spent m, uh, three weeks,
Speaker B: six different codes.
Speaker A: Like two, three weeks. If I'm counting all the time compiled together on just this animation. Imagine all the other stuff we could have done to the program if, uh, this animation was done by Codex, as in cleaned up by Codex.
Speaker B: For people who are curious about the program, actually we have the link in the description so they can check it out. But what it has done.
Speaker A: And there's a special discount for you guys, right? For
Speaker B: both the program and the Unreal Engine program. Whoever watches the show, they get a very exclusive price.
Speaker A: If you guys are trying to learn whether it's Unreal Engine for your projects or AI in order to advance your careers, we got both programs and a special discount only for the viewers of this show. Uh, you guys can check out the link in the description below. That is it for the two experiments.
Speaker C: Let me, let me show you something cool. Okay.
Speaker A: Yes.
Speaker C: Here. Okay. It's. It's like you have the animation and everything. What I was surprised by is look at the sword. You know, you actually switch your hands in the actual mocap video.
Speaker A: Okay.
Speaker C: And it. It took the sword from the right hand, it placed it on the left, and it walks towards the phone and whatever you were doing.
Speaker B: Oh. So it's the end of the animation.
Speaker C: You see, the sword shifted. I didn't say anything.
Speaker B: Oh.
Speaker A: Because in the actual original video, I shifted the sword. It
Speaker C: did that. It did that. So it understands context really, really well.
Speaker A: Because you fed it the source video too.
Speaker C: Yeah, I fed it the source video too, in the end, because I wanted it to see the reference from like, like actual human reference on how it looks, how the sword is, how the way you're standing, the way you're moving. Because the messed up version, if I give the same reference, of course it's not going to understand, like, what's happening, but it did its best.
Speaker A: So technically, what you're suggesting, even if they cannot be done with this current version, in an upcoming version, we can just record ourselves and just submit that video to ChatGPT and have it animate a character based on our video?
Speaker C: Yeah, I think it
Speaker B: probably. Yeah. Uh, probably. Yeah. Uh, actually, we should give that a try now with Astra. Let's see how it's going to work. Maybe it works.
Speaker C: I mean.
Speaker A: Yeah, look at this. Left side is the original RAW capture. Right side is a fixed version.
Speaker C: Random jitters. It's. It's not clean here.
Speaker B: Wow. Uh, look at the hand.
Speaker A: Yeah, the wrist is like. It was doing this. And now it's just clean. Do you know the amount of keyframes he has to fix in order to get this right?
Speaker B: It's a long animation. It's a long fight. It was a long fight.
Speaker A: Long animation. And all the different bones have their own keyframes. Easy.
Speaker B: Wow. But that was cool. Before we go to the next topic, Prat, can you order food?
Speaker A: You're hungry.
Speaker B: I'm hungry.
Speaker A: Okay, so while Prat is ordering food, Farhat, what do we have on
Speaker B: Google? DeepMind released Genie 4, which is the most realistic and the most interactive word model to date. They released a few demo. A few people had special access to test it. So what do
Speaker A: you
Speaker B: do? Yes, because we didn't have access, we might have it from next week, but. Let's see. But what is going to happen is we are going to show you the demo. Sundar, Google CEO also posted a, uh, separate video. We are going to analyze that we also found videos from other creator who had access to Genie 4 to look at it. But what is different with this word model than from the previous?
Speaker A: What is a world model?
Speaker B: So the world model is an AI system that generates an environment, but it understands the scenes and the reactions and the
Speaker A: physics
Speaker B: and the physics and everything.
Speaker A: So like Unreal Engine, but without it being coded and created, it's just generating all the pixels in real time.
Speaker B: So the large language model, what they do is they predict the next word. The word model is trying to predict the next reaction, the next scene. What is going to happen next. For example, in a word model, if you have a car behind the red light and if your prompt is turn the light to green, not only will turn the light to green, the car will start moving too. Because what's the reaction to the green light is the moving car. So it understand the entire environment. What is it great for robotics when you want to train the robots? When you want to train the robots in a factory, if the robot's supposed to pick this from here and put it to here. You cannot just have a model that only understand this movement. Because when the robot is working in the factory, the. The safety of the surrounding is important. What is going to happen if the robot walks away? What will happen if the robot falls down? So the model have to have the understanding of the entire environment. Um, so it can predict what happens next. It can even predict what happens when they become knights. The lighting is going to change.
Speaker A: But robotics is definitely one of the cases for me.
Speaker B: Games.
Speaker A: I want to know what's more important for us. As in, okay, that's what the robot will see in his own world model. Want to know what does Genie 4 means for humans? Not robots coming to humans lives. But
Speaker B: what
Speaker A: can you do with a Genie 5
Speaker B: or
Speaker A: Genie 6? What does this mean? I
Speaker B: would say interactive entertainment.
Speaker A: Can. Can Pratt show the video while
Speaker B: they do this? Let's watch.
Speaker A: See, your food disrupted this entire thing. Sahad.
Speaker B: Let's. Let's watch the whole video.
Speaker E: Genius.
Speaker A: I'm curious what we can do with
Speaker E: it world model yet Build environments, simulate worlds on command, keep prompting and Genie 4 will work out what happens next, then explore and reshape them in real time.
Speaker B: And by the way, you have wasd. You are moving around in the scene.
Speaker A: What
Speaker E: transform a picture into a dynamic environment that's ready to explore.
Speaker B: Like one thing is museums. Just think about this. Like this is
Speaker E: Van Gogh
Speaker B: painting, right?
Speaker A: Yeah.
Speaker B: And by the way, Genie 4 has the ability to create the sound effect too. So it Creates the sound effect based on environment.
Speaker A: So the ambient sounds
Speaker E: create a scene, move the camera, direct every detail, And, um, watch the world change live.
Speaker F: Do you really think we're gonna make it?
Speaker E: What the fuck? Or describe an interactive experience. A few seconds later you're inside it with full control over your character. Hey, that's a. Attack the castle. No, it's a trap. Good luck, my friend. Genie4 can simulate scenarios like helping robots practice tasks virtually. So that one day they might be able to carry out work in the physical world. World. And during all the time we spent in other worlds, our, uh, first world kept evolving.
Speaker B: I'm thinking a few use cases here. Definitely. The first one is ready. Player 1. Video games. VR.
Speaker A: VR and video games.
Speaker B: Interactive experiences in a way that, like, for example, when you go into a museum,
Speaker E: the stories you
Speaker B: really going to. The. For example, if you go, you know, if you go to Louvre, there are different sections, right? There is Egyptian section, there is a Persian section, there is a Napoleon hall, uh, section. Imagine when you enter, you really get yourself immersed into that world, immersed into that painting. And you don't only see the painting, but you see the surrounding of it.
Speaker A: That's really cool, actually.
Speaker B: Uh, you put something on, you really experience what has happened in that environment. I think for educational purpose. That is really useful. Entertainment.
Speaker A: Video games. No, I'm just thinking video games. Okay. What is the. What is the thing that you do when you lucid dream? What's the point of lucid dream?
Speaker B: What is lucid dream?
Speaker A: Lucid dreaming is when you can get inside a dream and control your own dream. So you're inside your dream and you decide that you want to start flying. Then you start flying.
Speaker B: Lucid dreaming, then. Yeah.
Speaker A: Technically speaking. Why is lucid dreaming even a thing? It's fun because you can do whatever you want.
Speaker B: Yeah.
Speaker A: And you can spontaneously decide on the spot and the world model in your mind reacts to it.
Speaker B: Yeah.
Speaker A: So what I'm, um, seeing here is essentially lucid dreaming in real life.
Speaker B: True.
Speaker A: The moment you go inside this world, imagine with a VR headset and you don't even have to type it again. Typing is gonna go away. No,
Speaker B: no, just talk. You just say it. Just say it.
Speaker A: You just say it like, oh, I want a bird here flying. And I want to fly with the bird
Speaker B: on top of a massive.
Speaker A: I'm on. On top of a butterfly. I'm riding the butterfly and I'm flying with an eagle beside me. And it will create that. And then you'll feel it. That is insane to me.
Speaker B: What I could think that you can even monetize with is that you can sell movie experiences. Imagine you want to act as Legolas in Lord of the Rings. So there is a game which is not as open as this. I don't think being this open is great. You shouldn't be able to prompt. Somebody will build this. And now you're Legolas in Lord of the Rings in Two Towers and you're in the fight. You can look around it, you can go to different places, but everything is built. Or you are James Bond.
Speaker A: If everything is built, what's the point of the world model?
Speaker B: No, but the reactions, the changes. You can go and kill an orc. Not everybody will play the same game. You see, there are a thousand orcs.
Speaker A: You can kill an orc in a current Lord of the Rings game,
Speaker B: but you are not in that environment. When you put the VR, you are not in that environment.
Speaker A: You are in that environment. If someone builds. So this
Speaker B: is Wasds. What
Speaker A: do you mean
Speaker B: you put the VR? You're inside the experience. So you look.
Speaker A: That's what the game is.
Speaker B: No, but you're looking through a monitor.
Speaker A: No, inside the VR game.
Speaker B: If you don't have VR game for lore behavior,
Speaker C: you can make a plugin and go inside. Yeah,
Speaker A: yeah, no, we have.
Speaker C: We
Speaker B: have that.
Speaker A: Yeah,
Speaker B: monetize that.
Speaker A: He's like, we have that.
Speaker B: That idea is
Speaker A: lost. I loved your reaction to it. No, I think the whole point is that you get to change things on the spot. There
Speaker B: is no point then.
Speaker A: Yeah, I honestly. There's no point. Isn't. We have that. But there's Lord of the Rings VR. Of course there is.
Speaker B: Yeah. No, but what is the point if you can change Lord of the Rings to. I don't know. It's
Speaker A: not changing Lord of the Rings to Harry Potter. You keep going back to something like an existing ip. No, it's about exactly what they're showing. The limit is your imagination. You want to do most random things. You want to start flying. Rick
Speaker B: and Morty. Rick and Morty.
Speaker A: What's the ah. Like. I guess Roblox is the perfect explanation. It's like make whatever game you want. You come up with your own rules, you come up with your own design, your own concepts, and then you just make it. You want like one massive tree and everyone has to climb the tree. And you call the game climb the tree. And there's new things coming around you, trying to drop you off the tree. It's about. The whole idea is that you create worlds spontaneously. And new experiences spontaneously
Speaker B: That's. That, uh, makes sense. And if you look at the experience, if you think
Speaker A: about it. Sorry to cut you off. If you think about it as a kid when you're not. When you're not very conscious yet, where your subconscious is the one running thing
Speaker B: of that. How can you remember that part?
Speaker A: No, I, uh. Everyone remembers. Not every moment, but you remember roughly. As a kid, you don't understand anything about the world. You just have the wildest imagination about random things. Your, your imagination is just randomly running wild. For instance, I was thinking about this. You start randomly in the middle of the street, hopping and pretending like you're on a horse. There's no horse. But you feel like you're on a horse. And then suddenly you're like, woo. You start flying.
Speaker B: Sometimes you do that at this age too. But anyway, we have seen it. You
Speaker A: don't have to share that publicly. But point is the whole. I think the concept here is that imagine, it's like being a kid again, where you get to actually live those experiences.
Speaker B: Makes sense.
Speaker A: You know how we talk about how
Speaker B: imagine, imagine,
Speaker A: imagine, imagine. I'll give you an example. Imagine you're like, we've had this conversation in the past. You're like, what would it be like to be in like the year 17. 1780, and you're in the exact war where Napoleon won the biggest conquest.
Speaker B: Yeah. And
Speaker A: you want to be one of the. You want to be one of the arrow. Uh, you want to be one of the archers in the front lines. And then. No, let's reimagine that. You want to be a horde. Like one of the military leaders riding with Napoleon.
Speaker B: Yeah, yeah.
Speaker A: Or you want to be one of the 10.
Speaker B: That's what I was pointing. Or you want to be
Speaker A: one. You
Speaker B: want to be in love. No,
Speaker A: you're like, go and kill an orc.
Speaker B: I'm like, you can already do
Speaker A: that.
Speaker B: No, he just stole my idea. That was my idea. That was
Speaker A: my
Speaker B: idea. I just presented it wrong. That was my idea.
Speaker A: That
Speaker B: was my idea.
Speaker A: Uh, so you claim there's no Lord of the Rings VR and you said, go and kill an orc. You can already do that. You don't need all of these compute and data centers to kill an orc.
Speaker B: That was, by the way,
Speaker A: glad you can vibe code kill an orc right now and
Speaker B: give him
Speaker A: the meta quest.
Speaker B: Let
Speaker A: the man kill an orc.
Speaker B: He stole the idea. But anyhow, uh, but what I
Speaker A: said is fascinating.
Speaker B: It is. You
Speaker A: want to. Go ahead. Sorry again, I'm cutting you off
Speaker B: so many Times this round,
Speaker A: live as a pharaoh.
Speaker B: Yeah.
Speaker A: For, like, a whole day. You can be a pharaoh. All of the experiences around the pharaoh. No one
Speaker B: cannot. I cannot eat like a pharaoh.
Speaker A: You can.
Speaker B: I cannot. You
Speaker A: can.
Speaker B: What? How so
Speaker A: you can eat. Imagine, there's. There's
Speaker B: imaginary food.
Speaker A: There is. There is men and women coming in with giant bowls of fruit.
Speaker B: Yeah. And I cannot eat them. Oh, my God, that's so hard. Do
Speaker A: you understand what I'm
Speaker B: saying?
Speaker A: There's no developer who can code you an entire experience of a pharaoh for 30 days. Living as a pharaoh for 30 days.
Speaker B: What are you doing in that 30 days?
Speaker A: Have you not seen the Mr. Beast challenges? Like, I'm gonna do something.
Speaker B: Uh,
Speaker A: this person is gonna do something for 30 days in a shopping
Speaker B: mall. True. Actually, he might do that with all of this. Do
Speaker A: you
Speaker B: see? Yeah, it does. It does. The quality is actually so good. Can we look at the other video from Theoretically media? He will go through the tv. So he made a horror. Japanese horror movie version of the Ring. And then, uh, what happens is he goes into the tv, and the word inside the TV instantly becomes the new word. Just. Let's just watch this.
Speaker G: This was, uh, you know, kind of this idea of like, what if David lynch directed a Japanese horror movie? You know, kind of obviously very, like, ring inspired. But the idea being that, you know, we move, uh, from television set into another television set. Uh, and, you know, these world models, obviously, are very good at this sort of thing, because the moment that we enter the tv, it's like, oh, well, you're
Speaker B: like the vfx, you know, in the movies, right? Yeah.
Speaker G: Pulling something off like this practically. Uh, well, I mean, I don't think you could pull it off practically, but even digitally or with a. Like a. Like a traditional video model would be pretty difficult. Um, here we get a pretty, like, awesome technique, in my opinion. Look. Yeah, you can get super weird with this. I was actually generating this up, uh, late last night, and I will admit it kind of scared myself. This was.
Speaker B: But that was really cool, too. So hopefully we get access next week
Speaker A: where
Speaker B: there's
Speaker A: more explorations below. Is there anything? No,
Speaker B: no. He has a YouTube video where I think he tried different things.
Speaker A: Okay.
Speaker B: You look at, uh, Sundar's video. This is the one I talked about. So the prompt is only turn the light green, and lights will become m green, and he will go,
Speaker A: oh. Uh, so why is he playing pressing W so many times? What is W doing?
Speaker C: I think he's controlling the bike or maybe the camera.
Speaker A: Yeah, I think he's controlling the bike, right?
Speaker B: No, no, no, no, no. Uh, go. Go up. Read his prompt.
Speaker C: Steer dynamic environments in real. Yeah, I
Speaker A: think he moved.
Speaker B: No, no, no, he didn't move.
Speaker C: I'm
Speaker B: pretty sure he didn't move.
Speaker A: Is this
Speaker B: like one of your
Speaker A: orc stories?
Speaker B: Look at the comments. First comment,
Speaker A: environment A. I think you have to show more.
Speaker B: Yeah, no, no, it's there. No, it's the same comment. Yeah. Ah, yeah. So you see traffic light green and Gemini reasons through what happens next as the signal changes, the engine revs and the motorcycle accelerates. See, that's not an old story.
Speaker A: I'm sorry. I'm sorry for that. Uh,
Speaker B: thank you guys for watching. It's
Speaker A: trusting you, but I. I figured the W is pushing him forward.
Speaker B: You figured wrong. Just like a stories.
Speaker A: What's Pratt. What's the point of W?
Speaker C: I don't know.
Speaker B: Is the camera.
Speaker A: Why is it? No, but the camera's not moving.
Speaker B: I don't know. As, uh, Sundar DM him.
Speaker A: Or maybe Sundar is telling us another orc story. It could be Sundar. Are you telling us the truth? Tell us the truth, man.
Speaker B: But when we get who's
Speaker A: controlling the motorcycle,
Speaker B: next time we can give it a try. Yes, we'll bring it back to the show. And I think this. What
Speaker A: are you gonna create with it?
Speaker B: I don't know. I have to think about it. The orc Lord of Rings? No, no, no, I'm not gonna do that.
Speaker A: You're gonna get copyright infringement.
Speaker B: No. From who?
Speaker A: From, uh,
Speaker B: Hicksfield. Okay, by the end of this interview, Google, Hicksville, they all can send their cease and desist letters. Well, thank you guys for watching this episode. Have a great Halloween weekend ahead of you.
Speaker A: May I ask a few guys for a favor? If you guys enjoyed this episode, please don't forget, subscribe to the channel if you haven't already, by subscribing
Speaker B: beginning of the show next time.
Speaker A: I always forget. I always forget. But now you shouldn't forget because we didn't forget. And you shouldn't forget, subscribe to the channel if you already haven't, because that helps you stay notified when we upload an episode like this. Also, it helps other people find the video that they need to. What else do they need to do?
Speaker B: Leave a five star review on Spotify.
Speaker A: Yeah,
Speaker B: why
Speaker A: does that, uh, help?
Speaker B: That helps grow the audience of the show on the Spotify so a lot more people can see the show. And the show will grow,
Speaker A: AKA Fathod will be more rich.
Speaker B: Do
Speaker A: you not want to be more rich
Speaker B: doesn't pay.
Speaker A: No, I just, um. I'm testing to see if they are.
Speaker B: If they want me to
Speaker A: be rich. Yes.
Speaker B: I live that. Five star.
Speaker A: Yes. All right.
Speaker B: See you guys on Tuesday.
Speaker A: See you guys on Tuesday. Until then, ciao.
Other episodes covering the same guests and topics, from across The B2B Podcast Index.
