Franz Tschimben (ALLSIDES) & Michael Brehm & Ben Scheidt (Redstone): Why physical AI needs 3D data

Speaker A: We're now setting out to build the largest ever created data set and data infrastructure to then let anyone train. On top of that, uh, within our

Speaker B: platform, you use some of the larger models for coding, you need other ones more for, for the physical world. So that's a, uh, huge trend where you, where it becomes pretty obvious that one size fits all doesn't. It doesn't work.

Speaker C: I love this thesis about converging from the bits level to the atom level, which is basically enabling AI to really do stuff in the physical world. This will leap humanity forward, but we are not there yet. For these robots to work, they basically use 3D data as their native language.

Speaker B: The core bottleneck today is not more parameters, it's better reality grade data.

Speaker A: And while we get this quality that is unprecedented, at the same time these machines crank out these assets at five minutes per object, which is unheard of.

Speaker C: A lot of these companies who are now hyped in the VC field will have troubles keeping up the high valuations that they raised money for. And what will really count in future is still defensible.

Speaker D: USPs welcome everyone back to the European Podcast Today. I want to say that we've spent so many years obsessing over models. Who has the best one? How big is it? What, what benchmark it tops. And I think that that conversation is more or less over because models are becoming commodity and that's happening fast. The real question now is what do you actually do with them? And more importantly, what do they still fundamentally lack? The answer, at least for anyone building in the physical world, such as robotics simulation, autonomous system is. We need grounding. We need to ground the models in real world data. We need models that are not just scraped from the Internet, but that are captured from reality itself in 3D. That's what today is all about. Joining me for this conversation is Michael Brehm, managing partner at Redstone, one of the earliest investors in deep tech infrastructure across Europe. Ben Scheidt, partner at Redstone, focus also on AI and emerging data platforms. And of course Franz Shimpin, founder of All Sites, building what might become the next 3D data layer for physical AI.

Speaker A: This show is not investment advice and the hosts of this episode may be invested in the funds and companies featured.

Speaker D: Michael, I want to ask you, we've been talking so much about AI being a horizontal layer. Why are we now shifting towards more verticalized systems?

Speaker B: Well, like, uh, first of all, thanks for having us on euvc, probably one of the best places to talk about technology, uh, in Europe and uh, yeah, I'm very excited to talk about that. We have seen this kind of horizontal layer and as you said this rat race to the best models. And what has been very interesting is that especially a lot of the Chinese open source models have been catching up so fast with uh, the top US models, the more closed ones, uh, so the question is uh, where does the real defensibility come from but also where does uh, the need come from? And what now becomes more and more obvious that depending on the use case, depending on the application, you have m very different requirements for the type of compute, for the type of chips, interference, uh, for the type of data you need, for the type of labels. And there's more and more specialization even with the larger models. You use some of the larger models for maybe uh, for coding, you need other ones more for the physical world. So that's a uh, huge trend where it becomes pretty obvious that one size fits all doesn't work. And I think that's the really big trend that we are seeing. And if you look like it started and also like at Redstone, we've been probably active for over 10 years now in the AI space. I myself co founded an AI company 10 years ago in the voice space. And at the time also the thesis was oh, there's one voice company that will cover everything for every use case. Uh and then we've invested in the security stack, uh xane, one of the world market leaders for cybersecurity for AI chips for xai as a generalistic model, uh, that has been a tremendous success. And even there now you see new developments and the recent investment has been AMI as a world model with a very different thesis from France. So it's pretty exciting to see based on this compute explosion with more specialized hardware, augmented reality for the industrial application is getting dramatically better. Virtual reality in video games is getting uh, better and better. But in all these spaces you need really entirely different vertical experience and data than you needed before. You need specialized infrastructure systems that perform reliably also in a messy physical world, something very different from a purely digital world.

Speaker D: And Franz, this is exactly what you're building. So maybe this is the perfect time to bring you in, tell everyone exactly what Allsights is, what the problem that you're set out to solve is, and then from there we can expand back into the more intellectual conversation around AI.

Speaker A: Absolutely. And it's a pleasure to be on uh, euvc. Uh, so thanks for having me. I think Allsights is um, a groundbreaking company. We're a deep tech company Between Europe, uh, and the US and we are building the data layer, data infrastructure and platform for the applications of physical AI and let's say generative 3D AI. Um, Michael, you mentioned world models which falls under this category. Ultimately I think these two um, new frontier AI models, I think they will come together um, over the next decade or so. Um, but right now to get them off the ground it's quite uh, difficult. If you compare these types of applications of AI in the real world, let's say that has to do with um, physical properties. Uh, mostly it's not like we used to see from text or image or video based model, um, where of course fantastic developments have uh, been made since. Let's call them the, call it the ChatGPT moment. But the data was just available in abundance and various different quality levels. So it was never really a question what you use to train your models on because you could just let's say scrape the Internet. Now this has shifted already. We see this in text, um, image and videos, um, and audio, uh, as you might mentioned in Michael as well. But it becomes more and more obvious in the physical world where um, the data is just scarce but it's also very, very difficult to create. Right. And so this is what All Sides is covering. So we are really enabling anyone to build in this space to build faster, to build quicker and to build more verticalized and specialized AI models.

Speaker D: Maybe. I tend to say that I really love talking to you because you're really good at translating the technical to the more concrete. For someone like my, maybe you can talk a bit about what is it that breaks when we are trying to build robotic solutions without having data that comes from the real world, but Instead is more 2D Internet data as you might call it.

Speaker C: Yeah. So I really love what Franz is doing with All Sides. So at Redstone we think about um, the markets and how to develop in the future. And um, definitely AI is changing the world. You can feel it everywhere where you look at. But at the certain point, at the, at the current point AI is scaling a lot of logic and a lot of text based stuff that you see on the Internet. And mostly two dimensional and not three dimensional. And what is interesting is that if you look at the global GDP growth, it's actually not so well at the moment even though everyone is acknowledging hey, AI is changing the world. But so, so how, why is this the case? And here I love this thesis about converging from the bits level to the atom level, which is basically enabling AI for example to control robots. To really do stuff in the physical world, maybe to even work in hospitals and take care to do their own surgeries. This will leap humanity forward, but we are not there yet. And for these robots to work, they basically use 3D data as their native language. And yeah, we can dive deeper into this. Why specifically 3D data and what these robots need. But um, I leave it for now. Back to you, Frans.

Speaker D: I want to ask you to make it super concrete. When you're building these data sets, how are you doing it? Are you setting up uh, an iPhone in uh, lab and then it's recording a bunch of robots doing stuff or how are you doing it?

Speaker A: Yeah. So maybe just to go back to the origins of the company when we started a few years ago, um, we already had imagined that of course AI will leave a mark also uh, on the physical world, but, but there was really limited means out there to really capture it at the high quality and at scale. So I want to stress those two points. Quality matters in any AI model, right? So um, the quality in means very likely quality out in terms of output of these big AI slash machine learning models. And of course uh, the size of the data set matters as well. I mean you can train on 2 data points or you can train on 2 billion data points. It's quite different. And I think by now everyone has understood uh, that. So those were our guiding principle as well. You, to your example, you could of course record um, data, uh, with an iPhone and take a bunch of images and then there's open source software and some more dedicated software out there that reconstructs, that's the correct technical term, a 3D model based on a bunch of images and other information. So that exists today, but there's inevitably a quality bar. Right. And so as we started out the company, we tried to really solve this fundamental problem that has been existing for the past 20, 30 years in various industries. Right? So if you look at 3D overall, and I think we were mentioning video games before, we were mentioning industrial processes with CAD models, um, CAD models, let's say, and um, E Commerce for instance, with their own 3D models to promote some products. 3D is not something new, has been existed, um, has been around for quite some time already. But there was always a glass ceiling in terms of how many high quality assets you could produce automatically and at scale. It was mainly a manual process.

Speaker B: Right.

Speaker A: And so we set out as companies to solve that. And we solved this by developing something that is called a um, fully automated 3D scanner. So we have Various products of that. And you can imagine that to be a machine of 2 by 2 by 2 meters, lots of lights, lots of cameras, lots of other sensors that in about five minutes capture all the data you need to then derive. And there is a lot of AI technology on our end as well. Under the hood then to derive a high quality 3D model, which in our case means a mesh plus PBR, which stands for physically based rendering. So really capturing the physical attributes of that object, which means besides the weight, the exact measurements, we get the collision, we get very, very detailed geometry, sub millimeter geometry. And while we get this quality that is um, unprecedented at the same time these machines crank out these assets at five minutes per object, which is unheard of when a human or manual process takes uh, about two days to create these types of assets. So all of a sudden we have the quality plus the quantity scale combination. And this of course in this day and age when everyone is craving this type of data to train their robots or to develop their generative 3D world models is of course, uh, fantastic, uh, start for us and that's what we set out to do. Uh, one more thing, I will say we're now setting out to build the largest ever created data set and data infrastructure to then let anyone train on top of that, uh, within our platform. And this is the mission of the company right now. We work with customers such as Meta, Amazon, Nike, Adidas for various applications. This to the point of verticalized AI like Michael and Ben were um, introducing at the beginning of the podcast, AI is getting more verticalized, especially in tough applications like dealing with uh, robots or generating new 3D from, from, from a bunch of data inputs. We're quickly onboarding both on the US as well as on the, on the Chinese side, the leading AI and robotics companies. And we have a very deep relationship also with Nvidia.

Speaker D: Ben and Michael, I want to ask you, in Venture we always talk about the total addressable market. That's kind of uh, the thing that any wanting to be, uh, founder hears when they talk about Venture is that you got to have a huge tam. Let me hear from you. How have you thought, when you looked at all sides in the beginning, how did you think about the market sizing here? The potential for a technology like what France is bringing to market.

Speaker C: You can do either bottom up analysis where you really look at what customers are existing and how large can these projects be. The market for the core business was already quite large. But then we thought, hey, this could also be quite a unicorn, if not Even a Decacorn if they monetize the database. When we invested, the tipping point at the market wasn't reached. So we needed also to check a little bit how this is developing. In the meantime, Nvidia has also created a standard for 3D assets which is called SIM ready. So simulation ready data, huh? And Allsights is also living up to the standard and can create it. It's getting bigger and bigger and also reusable for new kind of clients. So that's what's making the whole thing super attractive. And yeah, we believe that we are building a Decacorn here based from South Tyrol and uh, want to bring it around the globe.

Speaker B: I mean if you're looking at 3D, what you're looking at in two areas. One is you want to have this immersive experience where a digital uh, 3D experience is indistinguishable for a real world experience. And we're not yet there. We're close, super close, but we are not yet there. And what's missing is super high quality reality data. And then obviously you have all this physical AI where you also really need super high fidelity 3D understanding. And the core bottleneck today is not more parameters, it's better reality grade data. And that was one of the core thesis. And we said if you solve that, it's basically that in touchpoint with every human nearly every day for that type of technology and they would build basically the back end for that. So it's one of the largest TAMS that we have ever seen in terms of addressable market. And we've looked in depth into that space and the question was, how could you tackle that? And what Franz said is what's interesting, you can get 3D data, but the quality even of things you film with iPhone or with cameras ultimately is really bad. What was super impressive that they solved the problem from the very ground and said what type of data do we need? And build a whole machine. It's like they have huge machines that basically scan and that are like 1000x better in terms of quality and time and cost than anything that's out in the market. And now building that, I think it came a little bit shorter. It was a huge anatomy. You're building these huge data factories, like real factories with dozens of scanners and like automation. It's enormous. It's one of the most important projects for AI, I think globally right now for the advancement.

Speaker D: Let me ask you a question, Franz. Is this not a temporary problem? As in we have LLMs when we needed to build LLMs, we said, okay, we've got 28 letters in the Alphabet, at least we do in Denmark. And then we've got this many numbers and we've got this pattern. Once that's mapped out, you're all done. Isn't it like that, that once you've taken your big machines, you've kind of defined what a shoe look like, what a hammer looks like, and then in two years you're done, you've kind of mapped the world. You don't need a 3D model anymore because you've got all the data. This is obviously a provocative way.

Speaker A: It's a fantastic question that, by the way, uh, you're not the only one to ask. And of course it's a recurring theme also here at the company. First of all, I think we need to lift the entire, uh, physical AI space off the ground, right? I mean of course what is available, uh, on the tool side and what Nvidia is putting out there helps a lot. But ultimately you will always have this huge, huge bottleneck in what you're able to do just because there's no data there. So we need to lift this entire trillion dollar industry up by providing the data. And there's already a humongous effort that will take the next five, six, seven years, I would say, in order to really, really see this, um, all come together. Um, I compare this sector almost like to the beginning of the, of the self driving car industry, right? So there's lots and lots of stuff, um, to do. Although of course AI is at a different level and can help speed up many of these processes. So that's the first step and along the way I think we can build an amazing, amazing business both from a technological level, uh, in terms of just size and on the revenue side. By doing so, we're not just a company who says, oh, we're building the data, here's the data, license the data and then bye bye onto the next one. There's deep integration with the big AI labs in the world, right? And so this deep integration first of all will teach us a lot, will hopefully teach us on the AI labs a lot too. And I believe you look at, uh, the next five to 10 years, there will be applications built on top of it that we don't even know can exist yet. So if you compare this to maybe like the smartphone moment at the beginning, nobody knew that maybe an Airbnb, an Uber or a Instacart might be killer applications and worth in their own regards several, dozens of billions of ah, Dollars. Right. And I think we can be in the driver's seat in the pole position to then ourselves either help build these applications or build them ourselves. Much like to the verticalized AI topic that uh, the Redstone gentlemen were mentioning. And so I think it's, it's even more exciting what we can do afterwards

Speaker D: I would say, am I right? Funds to also say that yes, while you might get to a point where you've mapped the world, so to say, you will also, by having been one of the core players mapping the world, be the ones that own that data, which then means that you can monetize the ownership of basically the 3D mapped world.

Speaker A: Absolutely. That's what we plan to do from the beginning by the way. And maybe to stress the point on if we are ever done mapping the world. I mean there's so many products and objects that uh, come out each year. I mean we live in a consumer uh, based society. Right. It's like billions and billions of different items each year. And it will be a recurring theme to map them all out. Then again on top of it again, as you develop the applications, even more specialized data is needed. Um, they might not be then publicly available to everyone, but maybe just tailored to different um, customers. And that is an exciting prospect as well.

Speaker B: I think what it also one of the, and we had the same discussion and I think it's one of the core question uh, about temporary. But what you're building, if you have this large database and if you're understanding about you can also build more and more tools to kind of interact with them, to use them, to manipulate them, to combine them, uh, build them. And we believe that uh, one of the best companies in that position will be allsights because of the competence in data. And then you're becoming more a kind of 360 solution provider, uh, than just a data uh, source.

Speaker D: Let me ask you, your core customers right now, who are they and who do you expect them to evolve to be?

Speaker A: Yeah, so right now, um, the customers that we can publicly mention are meta, Amazon, Nike, Adidas, Zara. We're starting to do a lot with also um, the large Chinese players, which I cannot mention yet, and also with the large AI labs out of uh, the US And I always have to mention the deep integration on the Nvidia side of things as they are shaping the space with their tooling. We want to shape it together with them on the data side of things. And that has been really working well for us. And uh, those are the ones that uh, right now either license the data or they buy the technology to produce data for the very specific applications themselves.

Speaker D: To all three of you, what does it say about the current state of the market that your core customers today are the hyperscalers, the known entities? Because one would think that there is a long list of rest of world that would also be using data like yours and wanting to help build and benefit from the 3D world's data. But I'm guessing that they're just not there yet from an adoption perspective.

Speaker A: Maybe just to answer this question. So we foresaw that as uh, well not everyone might be in a position to do the scale that the hyperscalers are needing and doing, but there's lots of other companies out there that need 3D assets for their own applications, AI and not. And for that what we did in the go to market effort, we partnered with the leading content creation and 3D companies in Europe and the US to stand up so called 3D tech centers or 3D scanning centers, where basically companies who just want to get started, they don't have the large, large volumes LED or the large needs yet, they can just ship their products and get them done kind of as a service model. And this is for us an easy way to uh, track the market to get lots of uh, prospective customers hooked before they then really um, roll it up and go into other dimensions in terms of scale of data. But the technology should be accessible to anyone and we're working hard to make this really um, available across the globe. Um, in fact we are also continuously developing better technology just more suitable to different use cases so that everyone can get started um, at all times.

Speaker D: Ben and Michael, I would love to ask both of you from investing in France on all sides, what has this taught you? What has it taught you about how to think about AI in the physical world? You're investing both in the pure software layer, but you're also investing in hardware. I'm super curious to hear what is an investment like allsights doing for you when it comes to thinking about how AI is evolving.

Speaker B: I mean when we invested actually it was a hardware company and it was a quite controversial one because it was just producing hardware and we're investing and say like look, it's so we couldn't find a good solution to create the data model and said you first have to build the hardware to do that and then you're able to build the data layer. And that was not an easy one because you had complex hardware problems before you could build that. But now it gives it A lot of defensibility. And it was, let's say a little controversial, uh, a little bit contradictory, uh, out of the box investment. Uh, if you look at the people who are involved, from amazing universities, amazing places, all the big centers, but it's also based in South Tyrol. So people said like that's strange. Pure hardware in South Tyrol. What the heck do you want to do there? Uh, this doesn't make any sense. And now they have uh, a lot of the biggest brand names, the largest companies in the world by any metrics as customers. It has encouraged us to think controversial and do kind of contradictory, uh, uh, investments or at least on my side, I don't know about Ben, what he has taken away from so far.

Speaker C: Yeah, I think it's also a little bit like if there's a gold rush, you should invest into the shovels. You have a lot of fast growing AI companies now on the software side, but at the same time most of them don't really have a usp. So you can even copy it with a team of students over a weekend and then you have the whole process up and running. So I think that a lot of these companies who are now hyped in the VC field will have troubles keeping up the high valuations that they raised money for. And what will really count in future is still defensible USPs. And this you can get from this combination of hardware, software, technical knowledge, real scientific knowledge, a great team behind it. And this is all what we combine with all sides. And it really turns out well like with these customers we don't have any churn like KPIs. You can only dream of as an investor normally. So yeah, it taught me that you should really think about the USPs uh, in the current world.

Speaker D: Fanta, I'd love to ask you. Michael mentioned this. You grew out of Tyrol, but then in the beginning you also said you're, you're now building out both off here in Europe, but also in the States. I'd love to hear your experience from going to the States, when you did it, why you did it, what you learned from that, what you'd say to founders tuning in about making that move.

Speaker A: I think, I mean first of all it is a great question. The reason we built it out of South Tyrol. I mean I'm South Tyroldian, um, one of my co founders, South Tyroldian. The other one is uh, a uh, German who wants to live in South Tyrol. So this is how we came to together. And uh, my experience in the past was shaped by spending considerable time of my life in, in San Francisco, in Silicon Valley, I was working computer vision, machine learning. Industry taught me a lot and I wanted to build a global company from the get go. And this is how we approached the uh, market we, how we approached uh, talents here. So we have a team of um, close to 45 people right now from 16 different nationalities, um, between here and then also the subsidiary in New York City uh, which has always been around. Not that prominent of course from the beginning was more kind of an access to the market with the right people. But now we're really trying to scale that up because of course when it comes to our customer base and the customer base that we want to grow with in terms of physical AI and generative 3D, a lot of uh, the big AI labs are um, in the US that's just how it is. Would be maybe another topic to discuss this on a EUBC podcast, what that means over the next decade for uh, Europe and the other big area that we are also uh, working with is on the Asian side and we have built internally a small team to handle that as well. And that's how we're set up.

Speaker D: France. I'm super curious because you've done it quite well in bridging Europe and the US I'd love to ask you what has it taught you this internationalization process and what would you say to founders that are tuning in, who are standing in front of that move?

Speaker A: I would tell founders, um, especially those who work in the AI space to think globally from the beginning. You can make deals anywhere. Even though you're based out of South Tyrol or any other place, uh, initially it doesn't really matter. All that matters is that you have great uh, great product and you solve a pain point for your customers. But to have immediately this, this global mindset in terms also of concrete go to market efforts and then in terms of internalization, just building up the squad or the team in our case in New York City, it's very important to have trusted personnel there. So to not just treat this as, oh, you know, we just hire a bunch of people and it will work because it's different and really depends also where the founders spend their time in. So my way is always to spend time in the European office, but also in the uh, in the US office and really start working with trusted people that I have known from the past and that I know we identify with uh, the culture of the company and can identify with the vision medium to long term. Now this might not be true for Everyone that we hire as we grow fast and as we grow big. But that's sort of the mantra, especially for the first, I would say dozen hires who form then the core team on which everything else is built. That to me, people is at the core of this, uh, internationalization strategy. But the beginning, the global mindset has to be prevalent.

Speaker D: You obviously have lived in the States for a long time and been part of building companies there. Now you're doing it in South Turo. Is it all bull when people are talking about culture and how hard people work in the States versus in Europe?

Speaker A: It's tough for me to answer this because I only live my own realities, which was basically working for technology companies, startups there, started my own company also in the US and then doing immediately afterwards here. So I only know my own culture. And it's just about getting stuff done pragmatically and do what we have to do. Whether this is then, uh, with, you know, 12 hours of work, 10 hours of work or even more doesn't really matter. It's just about the attitude to not bullshit around and just get stuff done.

Speaker D: I couldn't agree more. And I set you up for an easy one because I think anyone who's been building in Europe and who know European founders and early hires know that they work just as hard as anyone in the States. Michael, Ben, do you agree, uh, you've invested a lot across the pond as well.

Speaker B: I would actually say if you look at the hours, people work harder in Europe than in the US the thing is that that's interesting and I had a lot of discussions around that. What kind of in the US is faster? The transaction speed and the circularity of things, of contracts, uh, is faster to get a deal done. You need to spend less time in the US than you need to in Europe. And I think that's the big difference. And that's where we need to work on. We need to be faster in kind of turning things around and getting to the next step. We are always, we want to probably like have very, very concrete or very diligent in every step. And in the US a lot of things are just faster moving, which means that they effectively work less time by getting more done.

Speaker D: I think you're right, Michael. I think that this speaks to the, uh, to the risk mindset also, especially as a startup that's trying to secure contracts with larger customers. But it also speaks of course to the mindset of early hires and people to join startups that in Europe it can be harder to get people to make that leap. I think it's more about making the leap of, of, of joining a startup in Europe versus uh, versus whether when you do it, you'll work hard.

Speaker A: Yes, no, I agree on the whole conversation. And yeah, it's just I would just laser focused on we have to do and uh, try to execute as hard as we can.

Speaker B: Even though I have to say that that fronts and all sides is one of the things where I say they work incredibly hard and get even more done. Which is astonishing even by any standard, be it European or us.

Speaker D: I would wish we could talk forever about the productivity of startups today, because building as an AI native startup today is just a completely different thing than anything we've seen before. But we're up on time guys, so I just want to thank all three of you for joining me. M on the podcast today.