dbt Charts: The Markdown of dashboards

Speaker A: Hi, this is Dan Poppy, producer of the Analytics Engineering podcast Quick psa. DBT State just went ga and it's one of our favorite things we've shipped as a combined DBT and fivetran team. Once it's on, it checks what's changed upstream and if nothing has, it skips rebuilding those models. So, yeah, less compute. But what we've heard is the dev experience is really underrated. No cognitive overload on what you're building, no accidental expensive builds because you forgot some syntax. It's like guardrails for your dev environment. It works wherever you run DBT and with orchestrators like Airflow. So give it a try. There's a 30 day free trial. Just search DBT State.

Speaker B: Welcome to the Analytics Engineering Podcast, featuring conversations with practitioners inventing the future of, uh, analytics engineering.

Speaker A: In this episode, Justin talks with Dave Fowler, founder of Chartio and the creator of DBT Charts. Dave started Chartio back in 2010 and sold it to Atlassian. Now he's back with DBT Charts, building what he calls the markdown of dashboards, a human readable spec that AI can write and people can actually review. They discuss why Dave thinks AI generated charts need to be auditable. They dig into what AI means for the BI bottleneck, and they explore where dashboards go when agents start spinning them up on demand. Dave is also the author of the book the Informed how to Build Modern Agile Data Stacks that Drive Winning insights. You can learn more about DBT charts@, uh, DBT charts dot com. This is a fun one, as Tristan says. Let's get into it.

Speaker B: Dave Fowler, welcome to the Analytics Engineering Podcast. Great to have you.

Speaker C: Thank you, Tristan. So pumped to be here.

Speaker B: You and I have known each other for a little bit now, uh, going back to the days of a company that you founded called Chartio. Um, give our listeners some of the background. Where did you come, uh, from?

Speaker C: Yeah, I started chario in, um, 2010. We did y commander in 2010. Uh,

Speaker B: I actually didn't realize it was that early. That's cool.

Speaker C: Yeah. Um, and, uh, when I launched it, I built this tool first. It was called SQL charts. And I'd done Y Combinator in 2008, uh, with a different startup in the social space. And I just saw all my friends were, um, uh, making their own dashboards and I was making my own dashboards. Just try to see what was going on and coding it ourselves. And they were all really atrocious dashboards. We would compare them, uh, and then my roommates at the time were working on The Facebook data team and they were building the Facebook dashboards internally. And I was like everybody I know is spending 30 to 100% of their time kind of coding these dashboards and uh, making tools for that. And there needed to be something in the cloud for that. And so I made this little tool called SQL Charts that some people started using and then it was just type SQL and get charts. And then um, got more obsessed with making it so anyone could work with data and started chartio and focused on the drag and drop and SQL kind of combo. At the time I launched it summer 2010, there's really only like um, you guys out there. RJ Metrics. RJ

Speaker B: Ah, metrics

Speaker C: in the cloud space. Uh, and we didn't know each other then at all. Um, and there was also like, who else was there? There's like burst and good data.

Speaker B: Oh yeah, I haven't heard those names in a

Speaker C: minute. They were kind of different. You know, I didn't run into them all that much. They were kind of like

Speaker B: super enterprise Y.

Speaker C: They were a full stack super enterprise Y focus. So they were like some of the first people going after cloud bi but like uh, and then about a year after that Looker started. Uh, and so we really head to head with Looker. And then after that

Speaker B: looker was 2011 or 2012.

Speaker C: Something around there. Yeah. Uh, um, and then we went out and then kind of like four or five years after that there was like the Mode and Periscope generation kind of came out. Uh, and so Chartier was profitable. We had um, some great customers. Doordash, Postmates, New York Times, lots of wonderful customers. Uh, um, and eventually I sold it to Atlassian, uh, and it's called Atlassian analytics now. And still it's like a bigger team. There's like three or four times as many engineers on it. But it's now it's just like it's like the same product with a big focus now on like just Atlassian data. Uh, and it drives a lot of the enterprise adoption and sales, uh, inside of Atlassian. So uh, that's really cool. Like that acquisition went really well. That was like five or six years ago now. And uh, most of the team's still there. It's like over half the people are still there six years later.

Speaker B: Wow, that's a huge win. That uh, shows the kind of affinity that the team had for the product and for each other.

Speaker C: Yeah, it's a good company. It's a great company. Uh, um, and so it was a good place. It was A good home for it. Uh, and then I thought I would get out of data for I thought I was done with data. Uh, and

Speaker B: wait, wait, don't, don't go forward in the story quite yet. Um, I think it is very interesting uh to think about the vintages of different data companies because

Speaker C: ah,

Speaker B: especially like in the pre 2020era because when you started the company was what technology existed for you at the time. So for example um, you mentioned RJ Metrics. Uh, um RJ was a, was uh, ultimately turned into a thing called Stytch which uh, built a bunch of integrations. Um

Speaker C: and it was uh, who bought it again? It became a big E commerce

Speaker B: Magento bought the BI part of it. Yes.

Speaker C: Right. The

Speaker B: bi part of it is much less.

Speaker C: Yeah three ways really if you count.

Speaker B: Well, come on. Um, uh but RJ was founded in I think late 2008 and as a result was not really that cloud oriented for a long time. It was very browser oriented but uh, didn't have presence ah at AWS for a bunch of years because AWS was still very, very nascent. Um and so I'm just curious like what uh, because of when Chartio was founded what did the world look like and what kind of decisions did you have to make in product design as a result?

Speaker C: Yeah, so I got a really. I think I had a really nice view of the future and I was really just designing for my friends uh, you know and that we would meet up all the time and or you know is living with some of the friends and uh, and just everybody was using the cloud uh and no one was really using none of. They were all startups so they weren't like, they didn't have like huge data yet, you know. Uh, and so I always kind of figured there would be something. This was pre redshift, right. And so I, people were just directly connecting to like a read replica or like a read only user of their postgres database, you know. Uh, and I thought there would be you know eventually some kind of like cloud warehouse but that was just a hunch, you know. Um, and I thought cloud based would be the way to go. Uh, and the other was like I was just really, you know I wanted this. We eventually called it Visual SQL but we had a drag and drop thing that like would write SQL for you in any of these dialects but then you could go into SQL mode and edit and change that SQL all you wanted as well. And eventually we had all these pipeline features and things as well. Um, but it really needed that flexibility of being both kind of like user friendly but also under the hood. Like you could really change and go to Raw SQL if you really needed it. Um, and again, in the early days there was no cloud warehouse. This was before like the C Store, uh, data, but I figured that that was going to come. Uh, and I don't remember. So

Speaker B: your early users were just like plugging in on top of MySQL and

Speaker C: totally postgres

Speaker B: and whatever. Yeah.

Speaker C: And it was all just startups, you know, and it was like friends of mine and. Or people read about a TechCrunch doing startups and they, you know, had some, some database in the cloud they were looking at, um, and still all the way through. Honestly, it was a lot of like founders and product managers that were like our number one users. They would just like connect us up to a database a lot of times, like live databases, you know, with a read only. It was like pretty good for a little while, you know what I mean? Um, before it got like easier to make the warehouse and uh. But even after it was easier to make the warehouse and stuff, that's often a good way to start. It's like you're just kind of looking around the regular database or a copy of it or something like that.

Speaker B: So early days, uh, very like early adapter space. There were a lot of like very um, I would even say rabid Chario fans. Um, and you eventually sold to Atlassian and there was this time period where, I don't know, do you have a name for this time period? I think of you as like, uh, you know. Okay, so this is maybe a little bit of a diversion, but um. You ever watched Avatar the Last Airbender?

Speaker C: Yeah,

Speaker B: yeah, yeah, yeah. So my kids are 6 and 8 and they just ripped through the entire four season original, like great show, great show. And uh, he essentially for the first three seasons needed to learn to bend all the other elements and he went on like a. I don't. There's a name for these journeys, like journey of self realization. Uh, and that's how I think of you. Like this wandering monk. Uh, after you sold Chardio, uh, kind of. I think you spent some time in India, right? Like what, what uh, was this, this experience that you had?

Speaker C: Yeah, yeah. Um, I was really nervous. You know, I've had a number of friends just living here in Silicon Valley, a number of friends who have like retired early and got really depressed. And actually this is one of the reasons I took a long time to sell Chartio. We had quite a few offers over the years, and I was just like, no, I don't want to go. What would I do with myself? I don't know. And, uh, uh, you know, it's a place of privilege. But I was also aware of, like, that, you know, people have a tough time with that struggle, and I was worried about it for me. So what I did is I actually, like, just made a workshop here in San Francisco. That's where I'm sitting in today. It's a bunch of woodworking equipment, CNCs and 3D printers and stuff like that. And then here behind me, you can see a few. There's, like, just, like, people in the neighborhood and friends and I teach some workshops, and we just, like, build stuff with friends here. And, uh, there's a bunch of pictures of people who build things in the shop. Uh, and so I spend a decent amount of time doing that. Um, and then I also got really worried about AI safety and spent quite a bit of time in kind of the. Kind of an AI doomer, frankly. Uh,

Speaker B: what time period is this?

Speaker C: This is like, you know, basically right when ChatGPT came out, I just started being, like, pretty worried. Doomer.

Speaker B: You were a, uh, real AI safety hipster.

Speaker C: Yeah. And, uh, and really spent a while. You know, I know some of the people who started these companies, and, uh, I don't know, it just makes me really nervous to make something smarter than us. I'm like, I don't think we should make something smarter, more capable than us. I think that's pretty risky thing to do. We should do so with a lot more care than we currently do. Anyway, uh, and so I went into that space for a while, like, seeing what I could do. Trying to do that

Speaker B: is, again, total diversion. And sorry for everybody who has to listen to me and Dave shoot the shit. This is what happens when you do an episode where the people know each other well. But, um, you mentioned woodworking. Uh, you train some people to woodwork. Uh, do you have a 3D printer?

Speaker C: Yeah. Yeah. Yeah.

Speaker B: Okay.

Speaker C: Yeah.

Speaker B: Do you ever combine your 3D printing with woodworking? I recently, uh, have been learning about how you can make things in woodworking that you previously were, like, almost unmakeable by printing out really cool jigs on your 3D printer.

Speaker C: Yeah. Yeah, that's really common. Um.

Speaker B: Really? Oh, you're way ahead of me.

Speaker C: And I like, um. Yeah, we either make jigs, actually. That's one of the nicer things I like to do. Or, like, there's all kinds of, like, in the shop. You need all Kinds of like, oh, I need a shop vac adapter from like this machine to this hose, you know, and like there's like, there's lots of little learning things. That was good when I was learning fusion. Um, uh, but then you, uh, know, we also have a couple of CNC machines and uh, that's another way to cut out and make these bigger jigs or just make the whole part. Honestly, at the end of the day I like some of that stuff. But I find the whole point of making the shop was to make stuff with your hands. And when you go that route, you end up

Speaker B: the more technical it gets. You destroy the whole point.

Speaker C: Yeah, uh, I'm like, oh, instead I just spent several hours on the computer modeling something out and printing it three different times to get the tolerance right. And, and it kind of takes some of the joy out of it in some of the way. So it's like kind of the point is to go make some stuff by hands and have some scuffs in it and the jig, you know, um, so it can be a trade off. It can be a battle to figure out which way to do it.

Speaker B: Despite, uh, your concerns about retiring early, you, uh, post Chardio established about a thousand hobbies. You, you were all in on hobby life and um, I think you did some charitable work in there too. Whatever, uh, as one does. Um, but then, um, fast forward, what, two years, three years, four years and uh, George Fraser knocks on your front door.

Speaker C: Yeah, well, actually it was when I saw the announcement of you guys merging. Uh,

Speaker B: uh,

Speaker C: and it was that email that I saw that announcement and some other history way back in the day. I was really trying to make Chardio this full stack product. So when we were like 20 people and Fivetran was like five people, I kind of went to George and Tristan said, hey, why don't we join together? We were both these cash flow positive companies that were having trouble raising. Uh, it's like, why don't we join together and make this full stack experience? And um, no. They said no and became worth a ton of money. Uh, and I really wanted to deliver that full stack experience. So we built some stuff in to make it so you could just log into Chartio with Salesforce and whatever and get your modern data stack just auto set up for you. Um, but it was really clunky to get the tools together while they were still really at different companies and three different APIs and different. It was really hard to assemble that easy to start stacking and I really wanted to deliver that to people. Why can't you just log into Salesforce, get all that data automatically, have all these dashboards? Um, so I really want to do that about. I don't know, that was like maybe almost 10 years ago now. And, um, and Charto again was like the, I think the first partner of fivetran, the first fivetran thing was to make, um, a bunch of data for Zenefits, which was they were using us as for bi, they were using Chartio. Uh, and so, you know, I knew those guys from the very beginning. And, uh, so I want to make that full stack experience. And then when you guys were merging, I was like, hey, you're making that full stack. That's awesome. And I darted that email to you guys saying, congrats, you're going to do great together. And if you ever want to go up the stack, I said, hit me up, I'll come out of retirement, put a dream team together. And George responded right away and he's like, that's exactly what I want to do. And I was kind of like, oh, I don't know. I wasn't quite serious in that. I was like, would I do that? Uh, and I just started thinking about it and thinking about what was needed in the data space. I advise and invest in a number of data companies still. Uh, and the idea of ever starting my own, going into bi, uh, going to data again, uh, my own just sounded really daunting. All the go to market, just the noise out there is really difficult. Uh, but also I see all these product needs. Uh, and so I built a few prototypes and George got excited about them, we got excited about it, and then I was excited to talk to you about it, but then I couldn't for a while. Uh,

Speaker B: oh, yeah, geez, I've already blocked all that out. Yeah. What Dave is referring to is that, uh, there was a little dust up that we had with the US Department of Justice, uh, that there was a little hold put on executing, uh, our merger, which I think ended up lasting about six months, something like that.

Speaker C: Yeah. And

Speaker B: during that period, we were not allowed to, uh, do a bunch of stuff together. And that included, you know, comparing notes on new products.

Speaker C: And so I couldn't talk to you until, you know, right when the DOJ said we could. And, uh, and I'd been building away these prototypes and putting a team together. Um, and like we said, it was kind of in the BI space, not really in the bi. I think it's kind of different. I think of it more in the artifact space. Uh, it's kind of this new space coming out with AI, uh, but there's definitely a lot of overlap with the BI space. Um, and yeah, it was exciting to show it to you for the first time. Uh, and you kind of seemed to get it right away. You were nervous as well to see what I was building. Uh, but then you were like, oh, this isn't bi. This is like exactly what we need. This is like something I was kind of vibe coding up on the side. You were already thinking about this and then you published that.

Speaker B: I don't know if we came to it from the exact same place, but I thought about it as, um, if you're going to create high quality data visualizations that are checked into Git, um, you want to think about it as having a spec and a renderer. And, uh, a lot of intelligence gets put into the renderer. So that, you know, I had been playing around with early versions of, you know, hey, Claude, make this dashboard for me. And the code that it output on the other side worked, but it was slow and it was different every time and, uh, very expensive from a token perspective. And I was just like, you want a very straightforward way, uh, to think of, um, how you want to describe a chart or dashboard. And, uh, then it can be much more concise and consistent. Um, and it turned out that I was able to build absolute garbage version of this in. In. I don't know, I think it was like two days or something, um, which was enough to make me think like, yeah, the ergonomics of this are really good, but there's really gonna be work involved in making the thing useful.

Speaker C: There has been. You know, I also made like, a pretty functional prototype in a month or so that was actually pretty good. And I felt like, man, I'm almost done with this thing. And like, months, months later, I'm like, oh, my gosh, there's still, there's still this depth, especially with a VIZ tool and libraries. Just kind of like each new chart setting is almost like another feature. Uh, and it gets deep and it gets crazy. Uh, but we're using a lot of AI to make all this and we've got some really great people. The first person I went and recruited, um, uh, when I had these prototypes was RJ Andrews. And he is, I think, the most passionate. You should totally have him on this podcast. He's the most passionate about charts person I've ever met in my life. He's very talented. Chartographer, designer, historian. He has a huge collection of, like, ancient, uh, data viz Books. Uh, when you're in sf, you got to come check out his library. It's like he's got a first edition of the first printing of a pie chart and the first Venn diagram. It's like an 1850s book, you know. Uh, and he, and he was just like, he's like a professor on all this stuff. And he both is most knowledgeable and passionate about it. He named his son Atlas. That's how passionate about charts this guy is. And so he was the first person I went, rj, come. And I'm making this thing that I think is the next gen of visualization and data tools, but it still kind of looks like it was made out of Vega Lite. And I'm like, it looks like BI has looked for the past decade. Can you help make it look next gen? And we've been working on that hard ever since. Uh, uh, and it's been an honor and privilege. Um, but what you were saying is I start calling this the chart effect. And what is that thing that AI creates for you? And there's so far been kind of two main paths. One has been, uh, it can tell a BI tool to make something and then you've got the traditional kind of BI chart. Uh, or it can tell, you know, it can make a bunch of JavaScript and a Python and combination of all these things that you can't really read, you know. And so, um, Exactly. I think that the right seam is this middle ground where you, if it's more of this spec, so it's still dashboards as code, but it's a spec that you can audit, that you can read easily, that you can run tests against, that you can validate. That's really important. So data teams, um, can give this to their users inside the company and say instead of having Claude just make random messes of code and then coming and asking us, please check if this is right and trace that query through three different layers. Um, uh, they can instead just keep these artifacts, these traceable chart effects, uh, to and be able to audit and test those and validate those at a much better spot. So it's a scene where I really think a lot about. Again, I worry about AI in the future, but it's here, it's coming. Uh, and how do we make sure that people and AI can work well together? Uh, and having that right seam where it's still this thing that you can audit and test and humans can still have an eye on it and monitor and govern that data, I think is going to be really really important.

Speaker B: Having something that is, um, agent writable and human reviewable is actually not a super easy design constraint to satisfy. Um, but I think so, so important, um, I have definitely spoken, uh, to folks on data teams where they've gone so far down the rabbit hole of having agents write all of their code that they've lost the ability to review it. And that gets to this. You know, I'm not saying that humans necessarily need to review all code forever, but I think it's important for it to be reviewable if you want to review it. Um, because otherwise the structure of the conversation, uh, of like, hey, are you sure that this is correct? Is just like, I cannot tell you on my own authority that is correct. I'm going to turn around to my agent and ask it that question and I'm then going to have to rely on its answer. Um, so I think that that design criteria is exactly the right one.

Speaker C: I think even in a future, I think about a future where like we're not even looking at the code, you know, it's still important to have those right level of abstractions, even for the AI. Like markdown is inherently valuable to AI. It's like abbreviates like it's stuff, it uses less tokens, it's a better storage format, it's more parsable. The AI can understand that versus like if it was instead talking in just either raw text or raw HTML, it would be kind of a mess and it would just take up more time and confuse the AI more. So the markdown is a really nice abstraction and that's what with uh, DBT charts we're basically trying to be like the markdown of dashboards, of charting and create that, what's that Minimum spec. Ah, to define a dashboard that's interactive and that can have all these settings, um, and still be very free and flexible. Uh, and so that's what it is. I think even a world where we don't actually look at those, it'll still be really valuable. And also it has all these benefits of being something again that you can go, you know, where all the query logic is, you can go test those and you can go check those. It's all organized and in good spots. The other thing, and you brought this up too of like giving advice to the AI, having smart things built in. And so, um, the AIs are kind of, they're kind of blind. Um, you can give them images and stuff like that, but they're not very good yet at just knowing how to look at something and Say, what's wrong with this? So we automatically build in a lot of smart things and uh, give those errors and warnings very quickly to the AI. So things like, hey, you only gave this much width for the chart, but you put 100 bars in here in the data set, it's going to overlap, stuff's going to get cut out and we just kind of give those signals as warnings back to the AI all over the place. Uh, so that the AI can very quickly not take eight iterations to get there and not have to look at PNGs, but can just actually quickly get to that end result for much less tokens. So

Speaker B: as in everything else, um, a new technology paradigm changes uh, the way that we think about constructing our tools. Um, the, you know, we talked about the transition of the cloud before and how Chartio and a bunch of other companies were, you know, responses to the rise of the cloud. And this is dbt. Uh, Charts is an answer to um, how might we do data visualization in the age of AI? Um, there were there. It is not uh, the first attempt at, at doing something code first in the viz space. There have been a couple of companies over the years that have done this, um, with some niche success but not kind of mainstream success. But my read is that that's because humans didn't really want to hand write dashboard and chart specs. It's kind of onerous. Yeah, you'd actually prefer to do that in a point and click way. It's actually faster. Is that your take on it as well? Is now the appropriate time? Because humans don't have to write all this stuff.

Speaker C: You know, in hindsight, you know, you know, in the early days of Chartio, like my big competitor was looker and you know, I kind of disliked its whole philosophy in hindsight, you know, a lot of the code first, they were really trying to do this code first. You could even like do these kind of do these YAML based uh, dashboards and stuff. Um, and it was a first attempt.

Speaker B: Almost nobody actually built them from scratch that way. You would like exported them to YAML.

Speaker C: Nobody did. Uh, uh, and that's how a lot of the kind of dashboards code things too are still mostly like export things.

Speaker B: I

Speaker C: really did look at like what exists in the industry and like should we just kind of merge with one of those teams or something like that first? Uh, and I think there's a lot of great work that's been done out there, but it did just seem easier and better to like design and build this language from scratch. Um, uh, fresh and new with that goal of being the markdown of dashboards, make it as easy and simple to read as possible. Um, uh, uh, but a lot of people have attempted this and I think it was really an awesome start. Uh, and I think some have done really well. I uh, think it is the future. I think it's kind of like building. I um, think of it as building bi backwards. When you're language first, when you're really truly built on the language first. Like in uh, dbtcharts.com, our hosted version we have all these uh, this panel that you can uh, UI wise change. You can change text on the board, you can change background colors, you can change all these things. That's all built by the spec, that's all like generatively built ui, uh, that when we add a new field into our language like that just automatically builds the right widgets you need in that panel. Uh, and it's cool, it's like really fast to build these things and keep it in sync. And a language first approach I think is the right approach. Uh, and people had just started a few years ago, the timing on it might have been a little early and doing it in a post AI world. I think there are some advantages where we really see who is the user and market of this kind of thing. Um, but kudos to those who have come before, like huge props.

Speaker B: I want to take this one step further because uh, as much as I like what you're building, uh, I think it's possible that you could level the criticism of it that it's attempting to create, recreate the, the prior like the pre AI world. And it's not actually, it's, it's innovating in technology but it may not be innovating in process or workflow relative to uh, you know, what, what AI makes possible. So here's what I mean by this. Um, humans have used dashboards as kind of these authoritative artifacts and you know, people spend a lot of time doing, building them. They, they code review them, they check them into source control or at least maybe they should check them into source control. And I think that DBT ah, charts is a great answer for that version of the world. Now imagine that uh, agents are actually the ones that are monitoring our businesses in the future and humans are not waking up on Monday morning and checking their you know, new user dashboard and you know, coming up with a diagnosis for why the line isn't going up the way that they want it to go up. Instead agents uh, are kind of like using a semantic layer and kind of monitoring the entire system of metrics. And they're saying like, actually today, here are the six things that you need to look at. And they build a dashboard for you from scratch to interact with it. There, um, is one, like, do you think that's a crazy scenario? Like, do you think that that's not the way the world is going to go? And two, do you think that, like, do we need this type of a thing in the world where like all dashboards are kind of these like ephemeral artifacts that are spun up and spun down as needed?

Speaker C: I go on the theory that like, you, you might want to save these at some point.

Speaker B: Uh,

Speaker C: and a lot of it's ephemeral, like, and, But I think it's like what we've built is again, very much like Markdown. And I think a lot of Markdown's ephemeral. Like the AI is just like showing you markdown. Sometimes you save it, sometimes you don't. You know, know, like most of the time you don't. It just goes through. And so this is the same thing. It's like, um, you know, you can also think of it like mermaid charts inside of Markdown.

Speaker B: Like

Speaker C: we, we now have fenced thing to. If you put just like you want to put regular charts inside of Markdown, like, this is, this is the language to do that. And so it's like it, it should be this ephemeral language. Like you can just quickly make things. But it's the easiest, quickest way for AI to, to make and show a chart. Uh, is it needed? Like AI can just go pull a charting library off the shelf. It does make a bit of a mess if you ever do want to save it. Uh, so this should save you tokens and it should be something you could save. And so we hope it has some lasting place in that and becomes this big standard. Uh, um, and

Speaker B: you're investing in making it, uh, standard. It's currently what, MIT Apache 2. What's the.

Speaker C: It's Apache 2. It's just like DBT Core. Yeah,

Speaker B: yeah. And, uh, so what people can. Does it download with DBT today? Is it a separate download?

Speaker C: It's a separate install today with the beta. We really hope to bring it in together. Uh, that would be awesome. But while it's in beta, it's called the cli, is called DCT Data Chart Tool. Uh, and, uh,

Speaker B: that's very creative.

Speaker C: It's very much like dbt. Uh, but it's DCT at the moment. And I'd love to just like, you know, eventually we'll be planning on bringing into DBT charts. That's still got to be decided. Got to be. You know, the beta's got to go well, but so far it's going quite well. Like, especially in the open source. Like people are using the open source like a lot. We had that great Hacker News launch and got us a lot of attention at press and we hope to continue to improve and like try to get this standard really adopted by lots of people. Um, I've been having a lot of customer meetings, people showing us like the things that they're creating. Um, and uh, it's really great to see. So people are really using it and liking it and giving some great feedback.

Speaker B: I love to see all the traction when uh, it hit uh, Hacker News. That was really cool. Um, the benefit of uh, having a uh, spec for how to define visualizations is you get Rails, everything looks a little more consistent, you don't have to spend as many tokens. Um, the, the drawback is that you're operating in a box a little more. Uh, you can't just like randomly create whatever the hell you want. Um, is there, is there extensibility? Like can you uh, create kind of custom artifacts?

Speaker C: Yeah, we're about to release custom. Ah, we had it built before, uh, and we took it back because the AI was abusing the crap out of it. You have to really, the old AI is just like wanted to reinvent every chart type, you know, and it was like, come on, stop. Uh, the new models really don't do that near as much. Uh, and so, um, we're bringing it back. Uh, it was just in the early days we wanted to just stop custom creating. Um, but the nice thing about our custom, so there's a custom chart type and you make it with SVG and Jinja and AI can really quickly make a template. It's impressive how good they can make these custom chart types. Um, we have Sankey diagrams and we have um, old diagram like information graphics like you can really do. We've recreated a bunch from RJ's Old Historic Books of population things using people, uh, as the bars. Uh, uh, and so we can make all kinds of things with this custom chart type. Uh, but what's so cool, it's still within the rails, um, but at the right seams. Again if you have the right seams, you still have all this freedom. But, but the right. So it, so it still gets all the like all your themes. So everything's extensible and you Inherit a theme. Uh, so it gets all your colors, it gets all the choices you have, um, in your styles. Uh, and that gets applied to your custom charts as well. And so it can fully use and extend that versus, you know, that that becomes. That's such a tough part in Vibe coding something too. You kind of get something in Claude style in their, like, unique way. And trying to get it to change and, and to use yours is like a lot of work to like, massage it into the version that you want. Uh, where with this you just like extend your theme file and it works even for the custom charts. And so also that, you know, we have these very interactive. We have a variable system. You just define up top. And it's just a Jinja thing that gets put into all your queries. So when you change your date range or your region or whatever, just change that URL re renders the dashboard and that also wires into these custom charts. So everything still has that same flow and system for getting the data. Um, but now it's just like the render part is what's totally custom. And that's a really nice place to make that custom, where the whole thing's not custom, but just that render part. You get like 95% of the freedom. But all the benefits of those Rails

Speaker B: tell me that you can take the custom chart types that you've designed and distribute them as DBT packages.

Speaker C: Totally, yeah.

Speaker B: Ha. Uh, uh,

Speaker C: already done.

Speaker B: That's cool.

Speaker C: Well, not released yet, but already done.

Speaker B: Dashboards and even interactive dashboards are, uh, information retrieval tools, historically at least. And there is, with the ability to create data apps, uh, more on the fly, uh, there's this graying boundary between something, uh, that is read only versus something that is read and write. Um, and, you know, whatever. We don't have to have this arcane conversation about what makes a data app. But, um, I think at least one of the things that causes people to stop calling something a dashboard and start calling it a data app is this ability to. For a user to interact with the artifact and for those interactions to actually create action in the real world, um, in other systems at least. Um, do you imagine at some point enabling this type of thing

Speaker C: as, you know, like. One of the optional names of dbdchart's internal project name was dataface. Dataface is a domain I bought 10 years ago. It means database plus interface. A lot of people can't stop seeing the face part. So people didn't like this. People think like you put a pie chart on your face or something like this. Uh, so we didn't go with Dataface as a name, um, but I bought this about 10 years ago wanting to pivot Chartio, uh into also being right. I realized bi tool is basically you're making these apps and it's basically a SaaS application builder but it doesn't have the right access. But if you just add the right um, you basically have built this app builder. And so I was considering moving us into that space and bought that domain database. Um, uh, but this language is intentionally dashboard focused, not data app focused. But honestly especially with the custom charts, it's all URL based. You can put in your own form in there, you can put links obviously in there. Links are really easily in there. Uh, and so there's all kinds of things you can get into that app esque space. Uh but I think I intentionally do keep it um, kind of artifact versus app focused uh as that's its main use case. There are a number of people out there trying to make these generic do anything and everything app builder type things and I think when you lose some vertical focus there, uh, it can take away a lot. So this is intentionally trying to be somewhat of a specialized tool but it has every ability to expand uh, from that. If users take it in that direction

Speaker B: I fully believe that uh, the wall between uh, read only and acting is going to increasingly fall. But at the same time I don't think we have enough of an idea of what the primitives are for um, data apps, action taking things, uh, that we could make a good spec for them yet there's a lot of um, it's a bigger topic of how does the uh, data stack even become more action oriented because there is often going to be a real need for closer uh, to real time data flows in that world too. So there's a lot to go there.

Speaker C: You know, when George and I first started talking he was really thinking this would take shape of like a streamlit thing. He was, you know, and that is more like app builder focus. Um, and I really considered that and went down that T pole too. And then I was like, you know, YAML's like easier and just like you can parse it and run tests against it and your logic's all in the right spot in the specs. So those are the advantages to YAML. Um, but then yeah, streamlit is a little bit more like also write back esque, um and app builder esque, um, yeah, so it could go that direction. One other, one other thing that besides charting and dashboarding, you know and like DBT Charts is kind of named almost humbly on purpose of being, you know it's, it's just charts, you know, uh, but there's a lot more to it than that. There's like, you know, a whole big plan and strategy behind this. But I think um, I think and hope it really changes the data stack, uh, and the modern data stack. And I've had for years something I've called the BI bottleneck. And I think that uh, you know I gave a talk at Coalesc a number of years ago about Kimball data modeling and remember

Speaker B: it was very uh, spicy.

Speaker C: It was very spicy. And I was like, whether that's still like something you should do. And I think more than we like to admit, one of the main reasons we still do dimensional modeling is because BI tools really require it. BI tools require us to make some sort of pivot table. Whether you call it a data set or an explorer, it can't really work with relational data. You have to kind of go and make it this two dimensional data set uh, so that the BI tool UI can work and so you can hand it to business users. And the BI tools kind of blame this on business users don't get joins and they can't do joins and we shouldn't let them do joins. So you have to come to us with everything pre joined and um, I think that makes this huge bottleneck. I think it makes all this work for analytics engineers to prepare these data sets for people and it's making all these copies of the data and then the business users get in the stance that of course they're like okay, here's the user data mixed with the finance data. And of course they come back and say well now I need also this data and also that data. And then they have to make, either expand it or make a new one. And there's all this because you're giving them these limited sets. Uh, and if instead you could do it in a relational model, if the BI tool could do joins, you could not have to do that layer. You could work at that relational normalized, uh, or slightly denormalized layer, uh, which is much easier to make, which is much more uh, uh, structural to make. Um, and it could really unlock a lot of things. And anthropic in its latest blog post, uh, about how they enable self serve analytics really go through, they found that they really needed a relational database, that those dimensional models confused the crap out of AI because the user's data was in five different tables joined in different ways. And the AI Ah was like where's the actual, where's the actual data? And so having a relational one keeps it to one entity and I think it's going to change that stack. It's going to change the way we do modeling. Uh, and you and I should chat about it more. I'd love to get your thoughts. I think that's going to really change the data stack and how we do data modeling uh, for AI and what AI really needs because AI doesn't make fan out mistakes. We don't have to be as worried about that. And it does joins so it can for business users do those joins. And uh, there's, there's lots of advantages of where we don't have to do that, that dimensional modeling spec. And so again that's kind of a spicy take but I think that's the future and anthropic showing that's what worked for them. And I, and I think that's like something we should, we should test more and like really get the word out on that more.

Speaker B: Yeah. Clean gold layer and uh, clearly defined semantic model of how the gold layer joins together. I

Speaker C: totally agree. What most people call the gold layer today it's like most people call that silver layer. It's kind of like the AI might just need that silver layer. It depends on what you call the gold layer. Right. But do you need it brought into the marts or the dimensional form uh, or is a clean relational layer better?

Speaker B: So you mentioned Anthropic, um, both uh, Anthropic and OpenAI have been iterating on their own tooling around um dashboards or data visualization and uh, which, which makes sense because I think that's, I think somebody from the company shared with me that data uh work is in the top three workloads, uh enterprise workloads for uh AI.

Speaker C: Totally.

Speaker B: Yeah. Not surprising. Um, there are some interesting user experiences that have been shipped. The one that I think caught both my attention and your attention um, is uh, the ability to. So in Google Docs you've been able to leave comments, start comment threads on the right hand side for I don't know, 15 years, like whatever. Like a fricking long time.

Speaker C: Yeah.

Speaker B: Um and I are there. Is this a common design pattern in BI tools right now? I don't think it is.

Speaker C: No. There's a little bit like Chardio released that like chats on charts. Uh yeah, um, but it's not that common. Uh we're about to release it here for DB charts.

Speaker B: So yeah I love the idea of comment threads um, on a dashboard mostly not for, I mean humans consuming comment threads. Fine, like that, whatever. That's somewhat useful but I love the idea of ah, little trigger points or like, like anchor points inside of a dashboard where you could create a comment and then start an AI conversation which maybe leads to changing uh, the dashboard, but maybe just helps you interpret the dashboard.

Speaker C: Totally. Um, and we go a step further than that. Even in DBD charts of you know, we, when you go visit the dashboard there's like, there's inside there's like a chat thread that you can do that's just like Claude or, or ChatGPT. And then there's also when you click on a dashboard, say you start making a chart or dashboard, you want to go drill into it and like change specific things. Then there's more. The, like the artifact view where you're in editing mode and you can go change the text directly on the page. You don't have to tell the AI to change it. Uh, and you can go change colors and backgrounds like there's a whole GUI just like Figma. Or you can tell the chat, you know and you can also like select charts to give the context of what chart you're talking about to the chat. So there's, there's all kinds of that but, but having both where humans so many things are just easier to just do yourself than, than tweak with the AI. Like if you're going to edit a section and Claude did just start releasing this stuff just a month ago with documents and now with Claude dashboards where you can also now kind of edit some of those documents not near to the extent that you can um, with DBD charts and us having this, this stronger chart effect uh, gives us more power to have these um, generatively built UI elements so that each thing has adjustable settings. You can't quite do that as well when um, the artifacts are really random. It made it with different react libraries or different charting libraries. You can't then expose all the settings near as well. Uh, I think there's some nice advantages to the chartifact in it being something that you can collaboratively edit and work with. I also similar like uh, before working on this I was making a notes tool I called Anote. Uh and I similarly built. I was frustrated that AI working with it. You can't um, like if you're trying to work on a document like help me write this or proofread it, it would just give you another version. Like it would rewrite everything and you'd be like well what did you even change, like, quit rewriting my stuff? Like, I kind of wanted in that voice. And so, you know, I added this Open Sources feature that was like one of the number one requested features on Prose Mirror, which is the number, number one text editor online that both ChatGPT and Claude are built on. Uh, and it basically, um, makes. Lets you redline and green line, um, documents. So the AI would work collaboratively with you on documents. Uh, and that's called, um, prosemere suggestion mode. Uh, and so I really care about that space of like, how do you not have the AI just like redo your work and you have no say or power. It just like kind of undid your document, uh, and made a different copy of it instead. I want it to be like, tweaking and editing and you can work collaboratively with it. Uh, and I think that's such a better experience.

Speaker B: A lot of. And you know this better than I do, but a lot of the work historically in building data visualization has been the long feature checklist of enterprise companies. Um, and every executive wants to see something slightly differently. Um, every it, uh, organization wants slightly different, uh, security controls. Uh, there's just like a lot of checklisty stuff to build. And so I think that even for people who started out with, uh, a desire to really think from first principles about data visualization, it almost gets hammered out of you because you're just like, well, what are the things I need to build next month to close the deals that I need to hit? Um, and a lot of that stuff in the current era kind of goes away. It becomes easier. And so, uh, you can get back to focusing on what are the experiences we actually need as humans to intermediate with this massive volume of data.

Speaker C: Basically, because the AI can do so much and be so flexible, you don't want to put too many rails on it. You want to make sure, uh, that any of the Rails you give it. And that's how we think about it. These rails are additive. They're just kind of like systems, like a framework that makes it easier. Uh, and they're good building blocks to build on, but they're all escapable and they're all things you can really customize because everybody does have these unique requests and there's this long tail of requests in bi. It's just like, never ending. That's something I really love about the space, actually. Tableau. Sorry. Um, Excel is like 40 years old now, but they're still actively developing it. There's still active feature requests on Excel, uh, and because in the data space there's this kind of like this never ending set of problems and use cases and, and it's interesting to me, like, I love solving those problems. Um, uh, and so in this space it's more about not restraining the AI, giving, giving it more power and not restraining it so uh, they can do all those things that different people ask for. What we're seeing related to that, what we're seeing so quickly, I'm amazed. At DBT Summit, that was a big takeaway at our customer advisory board. A bunch of them said, oh, uh, we already told our BI products we're leaving after a year. Nobody even uses those tools anymore. And we're like, what? And it turns out like as soon as you kind of give people this ability for AI to make their dashboards, that's all people use. Because it doesn't make sense. If you could just go get exactly the dashboard you want and ask for tweaks and ask to keep going. You don't go back to the tableau dashboard that some analyst made for you. That thing becomes just so quickly obsolete. You've moved on and you've made your own version of this. Uh, and it was much easier to just do that than even go ask the analyst, you know? Um, and so, uh, I think that that really is this, like people are just like really quickly moving to. I think the big, biggest threat for SaaS companies out there is this artifact space and cloud making. The artifacts of being like, it makes custom UIs, custom interfaces, custom reports, uh, people just, you know, do you go into Salesforce anymore? You just say like, what are the top five people I should call this week? And the AI just tells you, um, and that's your interface more and more. And the artifacts are like a way to save and collaborate on those decisions and documents. Um, and so I think that's the big competitive space these days is that artifact space. Uh, and we're trying to carve out, is there a unique space for the chartifact, uh, versus just general artifacts? And I think so. I think the data tools can be unique and even more powerful and a little bit differentiated. Uh, and there would be a standalone interface for that that would really thrive, that would work with these tools, but also warrants its own interface.

Speaker B: All right, last question for you. Uh, for anyone who's curious, uh, how do they get started and what do they get?

Speaker C: So you go to dbtcharts.com, and there is a, uh, message to copy and paste to your Claude and you just say, basically start Using DBT charts. And what will happen is your token costs will go down and the artifacts that are created will be prettier and governable, readable, governable. Uh, and then if you choose our hosting as well, uh, it's that more interactive where humans can work with it. You can go and edit things, uh, and edit documents and share them and collaborate with your team. Uh, much more easily, uh, than something just kind of vibe coded for you. Uh, I

Speaker B: love

Speaker C: that

Speaker B: the uh, paste these instructions into claude installation method is uh, ever more common. And I think it's such a great experience.

Speaker C: It was fun to actually test that. I had like, um, a user tested that by launching a hundred different instances and scenarios of like here we have uh, just a CSV file. Here we have a postgres plus dbt. Here we have redshift, but no DBT set up. And it created all these different scenarios on different instances and went and tried installing and then interviewed each AI, like what was frustrating about that experience? And then it came back and then we iterated instead of like, oh, it's lacking this and this and this. And we did this a bunch of times and we basically user tested this on a hundred different agents, I don't know, dozens of times, uh, to get that streamlined, nice experience. So you basically our CLI can just do everything. It will connect to your bigQuery, it will connect to your. If it finds the CLI, it will connect to your GitHub, it will do all that. There's a very minimal amount of actually logging into to the places and giving that access. Um, makes it quite streamlined. It's nice.

Speaker B: I love that. Uh, Dave Fowler, thanks for everything you're doing with DBT charts. Uh, enjoyed catching up with you.

Speaker C: Pleasure man. See you soon.

Speaker B: The Analytics Engineering podcast is sponsored by DBT Labs. I'm your host. Email us@podcastbtlabs.com with comments and guest suggestions. Our producers are Jeff Fox and Dan Poppe. If you enjoyed the show, drop us a review or share with a friend. Thanks for listening.