Linting Isn’t Governance

Ikenna: Hello everyone and welcome to the API governance for scale podcast where we discuss about API platforms, API governance and delivering APIs in the enterprise. I'm your host, Ikenna Wee Woo. I am the principal consultant at Ikenna Consulting. And today, you know, we want to talk about API linting or idea that API linting by itself is not API governance.

I have a lot of people who talk to me about we're doing API linting, so you know what? We're doing governance. Yeah, we're fine. Well, we want to really explore that idea, you know?

And with me today is Alistair Parker, who is the founder and creator of Jagon.sh, which is a collaborative platform for, you know, editing models and working ⁓ with APIs. And yeah, I'd like just welcome you to the show, Alistair. How are you?

Alastair: Good, you. Ken, thank you very much. Happy to be here. Ikenna: Cool, awesome, awesome.

Alisa, can you just tell the audience a bit about yourself and what you do and how you kind of got into the whole API space? Alastair: Yeah, sure. Look, I've been involved in kind of data exchange for a very long time. Well before APIs were kind of the current vogue way of doing things well back in SOAP and even sometimes pre-SOAP days.

so moving data between systems has been a thing I've been involved with for a very long time. And what's interesting is that many of the difficulties that existed over the last 20 years, we still kind of exist with today. And that's been a large part of kind of where we are and what we're doing is trying to look at why is it? that we're still doing the same things that we've done and getting the same outcomes.

Why aren't we doing different things and getting better outcomes than we've done before? And so I'd be happy to kind of dive into that and unpack that a little bit. From a personal perspective, these things became problematic enough for us in work that we do. We decided to do something about it and try and see if there's some things that we can do to improve how this works and try and improve some community practice.

And that largely led us to do it at jargon.sh. We didn't set out to become a software platform. That wasn't our goal as a group of people, but...

we realized that in order to do the work that we wanted to do with our clients, we weren't kind of supported the way we wanted to be supported by what was out there. And so we decided to give it a go and Jogging is the result of that. Ikenna: Yeah, yeah. Really interesting.

I mean, I've known you for a while, Alistair, and I know a bit about the platform. But you know what really inspired this podcast is I came across your presentation at API Days Australia last year, where you had this idea that why API governance fails in the enterprise. And from the slide, I think it was a great talk. And just for the audience, can you just give us some ideas of why you think API governance kind of fails, really, in you know?

in the enterprise. Alastair: Yeah, it's, it's, it's obviously a loaded kind of click baity title, but, I think there is some, reality there that's worth talking about. And so in the, in the work that we do with, with customers, and I've got to preface this, that when, when I say customers, we work with large organizations and large enterprises, people who do large amounts of data exchange between them and other large organizations. And so many of these problems that I'm talking about are somewhat specific to the work we've seen in those spaces.

But I have a feeling that they probably applied all levels of the scale across any number of organizations. But I think the big thing that we observed in doing work was that there's a lot of focus on API governance. But when you look at it a little bit more detail, what that really means is the API consumption side of governance. We care a lot about security practices.

We care a lot about monitoring. We care a lot about API gateways. And I don't want to stress for a second that that's not what we should do. We should care about those things quite a lot.

They're very important concerns and considerations. But why caring about those things to the extent that we care about them, we tend to starve other things of oxygen. And we don't really focus on the production side of these things. And I think when we look at where governance goes wrong, it's really because our gateway fell over.

It's really because we haven't got our security monitoring set up. That they have problems and they have failure modes and we know what those are. But I wouldn't really classify them as governance problems. Governance problems are more that We built an API and then we lost track of it.

We didn't really focus on what it did until we built a different API. So now we've got two APIs. So you have this kind of API scroll concept. And so from our little corner of the world, we're focused a lot on looking at the production side of API governance rather than the consumption side of API governance.

And so in the talk, I focused quite a lot on this this battle. I don't think it's meant to be a battle, but it always comes out as being a battle between production and consumption of that. how we focus so much of our efforts and our attention on scaling to have them win users, which is good, but we don't focus on what the API does. We focus a lot on picking one of kind of 10 or 15 API gateways, but those API gateways in and of themselves don't really provide much business differentiation to me versus my competitors.

Whereas the design of my APIs, the way I evolve them over time, the way I describe them to my customers is what gives me business benefit. Ikenna: Yeah. Yeah. Alastair: Certainly no one comes to my customers because they use gateway number one or gateway number two.

They come to them because they've lacked the business value that they're providing to them. And so by focusing so much on the governing side of production, sort of consumption of APIs and not focusing very much on how they're produced and how they're governed, that's where we see the failures happen around the business outcomes, not necessarily the platform values. And it's a bit unfair and crass to say so, but our view, a lot of the work and effort around... Ikenna: Mm.

Alastair: governing of platforms is more about risk mitigation. We're doing this because we don't want to be embarrassed. We're doing this because we don't want to fall over from production. We're doing this because we don't want to leak customer data.

Again, all things that we care about and we want to make sure we keep doing, but there's more to it than that. And if we only focus on those things and ignore everything else, we tend to miss the valuable things, the things that actually give us business differentiation that make us be successful businesses and have customers want to come and consume our product. Ikenna: Yeah, that's really interesting, Alistair, really interesting because, yeah, I see that pattern as well. You know, there's a lot of focus on those kind of technologies, know, or technology solutions and a bit sometimes driven by those technology solutions instead of actually thinking of the API, you know, as a product, what's the value is delivering, how we make sure, you know, delivering that great experience and that great value to the of the API.

But ⁓ me a bit, Alistair, because I like to explore this more like Why do you think, because what you're talking about is almost like either some sort of cultural thing in the organization or some, either they need some guidance on like how to build APIs right, right? Like, ⁓ what do you think, obviously there's kind of systemic issues here, know, what do you think makes people or organizations focus more on those kind of what you call the consumption side, you know, the API gateways, the security side and all those.

Alastair: Yeah. Ikenna: rather than the value side, the value proposition of the API. Alastair: Yeah. So we've done a lot of thinking about this as well.

And I'm going to give you my honest heartfelt answer, which is probably a bit unfair. think we focus a lot ⁓ the consumption side because it's easier. It's much easier to go and spend a million dollars on a piece of platform than it is to actually examine the root challenging things within my business and with my organization. It's much easier to say, ⁓ throw some money at the problem and the problem kind of goes away.

Again, that's not fair, but I think that that kind of lends a lot to why we are where we are. It's in a world where if are spending millions of dollars on something, then that comes with a variety of organizational controls. ⁓ And controls take time and those controls take effort. And so naturally, ⁓ we starving the things that don't cost millions of dollars.

And so there's just ⁓ an of because it's expensive, and rightly so, they do a very important job, but because they're expensive, they consume a lot more time. So things that are typically ⁓ free or just people time, we don't necessarily focus on so very much. And so I think... I think it's inescapable that because of the quantum of money we're spending, it takes up a large amount of time.

⁓ if we're not spending ⁓ equivalent amount of money on other areas, and I'm not statistically sure, but if we're not, it's easy to see why the focus goes to where the money gets spent, because we're all meaningful sized organizations, we've all got appropriate controls and balances and governance about how we spend our money, ⁓ want to make sure we get good value from that money. And so there's a lot of focus spent on doing that. ⁓ equally, as I said, I think it ⁓ somewhat easy to do that.

It's easy to say, great, let's go and buy an API gateway. Let's do all the platform work required to get set up. There are things that we know. There are things that we know we need to do.

There are things that if you went and Googled, how do I run an API program, those things would be there. Things that are less talked about and less described are, how do you make sure you're exposing business value? What are you doing about actually making sure that the things that you produce, the APIs that you run, are actually doing something worthwhile for you as an organization and for you as your customers or your consumers? I think you're right.

think it falls down to a variety of things. Money is inescapable. think that there's a large chunk of why we focus so much on these things. But I think equally because they're very expensive, they're very visible, they're very obvious.

Things that are less obvious are things like enterprise consistency and making sure that do we have one common definition of address for our human sake across all of our channels, across all of our exchanges? Do we make sure that we can use that common definition of address to do things like match customers with one another? Ikenna: Mm-hmm. Alastair: Those things are far less obvious than big heavyweight platforms.

And again, it's not just one platform. We've got platforms for gateways. We've got platforms for monitoring. We've got platforms for security.

We've got any number of platforms. So they're all quite large expensive things. And so it's understandable that those take up a lot of the oxygen in the room, which is again understandable, but also a shame because there are a lot of business differentiators that don't exist in that space. If you were a bank, Ikenna: Yeah.

Alastair: you're not going to be successful because you picked gateway one or the gateway to obviously if they're both reasonably appropriate gateways and done your due diligence, they're probably both excellent at what they do. You've probably can make either of them work. So by picking any particular gateway that is suitable for its use case, but ignoring kind of what my API's do, how they work, how they evolve, you're not really getting the best out of API. You're probably not delivering good outcomes to your customers.

Ikenna: Absolutely, Alistair, in your talk in API Days, you talked about this idea of shifting left on API governance. Could you tell us a bit about more and what you mean by that? Alastair: Yeah. So shifting left is a movement, I guess, in the moment across a variety of IT disciplines, largely stemming from the fact that we all exist in a value chain.

Nowadays, there's someone in front of us who pushes stuff to us that we improve and then we push it further down. Shifting left is all talking about what can we do to push some of the value further up the chain so that way I can rely on good quality things coming to me so I can do my work and move it further down. I don't want to deal with problems that really should have been figured out earlier up the track. And so how that applies to APIs, especially in the talk that we were talking about is predominantly things that we think are governance related, things that we should have figured out before we got there.

so as an example, if being asked to design an API that talks about customer information, it really shouldn't be my job when I'm designing the API to go and harmonize customers across enterprise. That's really an upstream thing that I shouldn't have to worry about or concern about. ⁓ And someone before they comes to me should say, here is the enterprise agreed version of what customer is and what a customer means. That doesn't mean we have to have one side.

It's all a model. I'm a big believer in demand and design, but someone beyond me upstream should have made those decisions. It's not up to me to figure those things out and how they work for the enterprise. And if it is up to me, then it's also probably up to other people doing similar APIs.

And it's highly unlikely we're going to do the same thing the same way. so pushing value left, shifting things left means there's an activity before we design APIs to think about how do we talk about our customers? How do we talk about our clients? How do we talk about our users?

Ikenna: Yeah. Yeah. Alastair: upstream of this conversation and figure it out. And again, I'm a big believer in domain design, so I don't want to impress for a second, I'm talking about one big model to rule them all.

But that's not what this is about. But someone upstream should have figured some of those things out. And so that's an example of kind of what shifting things left means. ⁓ what's interesting, when we think about governance in that lens is that, and this ties into ⁓ I'm saying that linting is in governance.

There's very little you can do from linting one API document that tells you is this definition of customer correct? How many definitions of customers do I have? someone else have a competing but slightly different definition of customer? These things you can't figure out yes or no by looking at one particular API document.

⁓ got to have a look at a swarm of things, take a forest from the trees kind of view and look at the sweep of these artifacts. And so that's kind of where we were leaning into when we saying that that linting is in governance. Again, linting is an important thing to do and it does provide a lot of value to how we make sure that our APIs meet some level of requirement that we care about. But there's a variety of other ones that we simply can't get.

And in the API Dice talk, I had this picture of kind of someone comparing a linting checklist with a kind of a business value checklist and linting checklist. And again, it was somewhat tongue in cheek and provocative. Was talking about things like, oh, you've got to have some number of tags or you've got to have a 200 response or a 400 response. And again, there is some value in those things.

But on the flip side, from a business value perspective, it talks about things like let's not expose duplicate records. Let's expose one view of customer and one view only of customer. Let's make sure that we are exposing things in the language of the domains who our customers understand what these things are rather than using our internal language. Again, those aren't things that you can link in ⁓ kind of linking process by looking at one document.

You've got to have a look at the broader context. to work out how we conform into these things or not. So again, I'm not saying linting is bad, linting is excellent, but I think there are constraints and the limits to what can be done by looking at one API document in isolation at a time. And so that's kind of where we got to by thinking about what does this mean in correlation of shifting left?

And so we want to make sure that by the time I am running linting on my API design document, someone upstream of me has already figured some of those things out or some processes figured out that. we've already got some of these design things nailed down and these governance things nailed down so that I can genuinely lint the things that I should be linting for and not expecting it's some sort of panacea or helm area that solves all the problems for me and call it governance because it's not.

There are things you simply cannot govern by looking at one document at a time or one design at a time. You've got to look at a suite in the broader context of what your enterprise is aiming to do and what decisions they've made. Ikenna: Yeah, absolutely. Absolutely.

In my book, you know, I write about linting, breaking chain checks, all these automated checks, you know, whether we're using spectral or vacuum and all those kinds of things. you know, from my perspective, they're like almost like easy to implement, right? It's it's easy, almost easy. Okay, depending on the context, sometimes it's complex in organizations, but it's relatively straightforward to roll out linting for your...

you know, open API specs to roll out breaking chain checks in your CI CD pipeline, kind of the engineering, the API platform team kind of rolled that out. What you're talking about in terms of going upstream and talking about domain driven design, getting multiple teams, multiple stakeholders to agree on what, you know, what ⁓ data models mean, what the, the, ⁓ the, the, the concepts are there, the, the domain objects or entities or however you describe them, what they mean. Alastair: Mm-hmm.

Ikenna: and how they make sense in that, know, that customer, that journey, ⁓ workflow, whether you're doing event storming or all those kinds of things. It's very like intense, ⁓ it's very hard, you know, to get all those people in the room talking to each other, defining those things, to come up with that data model of customer that you can then build an API on and get that consistent, ⁓ know, those consistent concepts across. You know, when you talk to your customers, look, how do you find people, know, organizations fair in their different maturity of actually doing that hard work, that heavy lifting of domain-driven design?

You know, maybe they're not calling it domain-driven design, maybe they're calling something else, but the whole architectural rigor and discipline of defining ⁓ bounded context, defining... Alastair: Yeah. Ikenna: defining what the the concepts are, the models are that should be reused. Tell us a bit about that.

⁓ Alastair: Yeah, it, yeah, as always, it varies, right? But it's rare that we've seen someone doing it in a way that delivers the outcomes they wanted to deliver. So I think people are always looking for ways to improve. And so I'll give you, I'll give you some kind of differentiating examples that we've seen.

think everyone sets out with an intent to do it, but the way in which they set out and the way in which they go about it has a material impact on where they land and how they succeed. So on one hand, you've got a group who draws a bunch of diagrams, whether it's on a whiteboard or a mirror board in PowerPoint, whatever it is, right? They approach it upfront saying, we want to do the right thing. We want to know, taking the customer example, that a customer has these characteristics, which is similar to a client in these ways, which is similar to a user in some other ways, but we recognize that they are different for different roles.

So they share this common kind of kernel of or addresses or whatever the rule may be. And so they start out with those things as diagrams. And then they typically say, all right, team one, two, and three, each of you pick each of these things and go onto an API and then come back to me. from which point those diagrams are now useless because individual teams come across different problems and different challenges and they have to adjust to those ⁓ in best way that they can ⁓ without a lot of concern of how it marries back to what this big design was.

⁓ so to those teams and those teams APIs work quite well, but at an enterprise level, we start seeing creep in and we start seeing discordance creep in where ⁓ can no longer get a single view of customer because we've diverged from what this customer meant. so client information can't be married up to user information, can't be married up to customer information because they're different. ⁓ And that's one world where starting with kind of all the best intentions starting with design, you lose of that consistency because we change kind of model about how we talk about these things from an enterprise design to individual teams building design and APIs.

And there's quite a lot of people in that space. And on the flip side, there's quite a lot of people who don't even start there. They say, great, we need to go and build some APIs, go and figure out what data you require. Or leave it up to individual teams to go and look at some large enterprise data model and say, make sure you align with this, but there's no real way of thinking or maybe checking that you are just do a bit of review and have a look at it.

And so I think we haven't really seen someone nail this, which is what I said at the beginning, right? We didn't set out to want to build jargon. We want to set out to help people build better quality APIs. And we, by looking around, we didn't really find much of a middle ground that says, how do we enable enough of that high level upfront design, but make sure it flows all the way through?

Where is the CI CD for designing these processes? What can we do about bridging the gap between PowerPoint and GitHub? What can we do about making sure that we identify problems and challenges upstream and flow them all the way through the code and the testing. And if something goes wrong, we flow them all the way back up again.

And so I think that's, For us, where that kind of missing thing is, that gap between how do we take and make all that designs that we do focusing on things like enterprise consistency, have real automated consequences in the implementation of these things. And how do we do a feedback loop that says, by going through this process, we've realized that we've got the design wrong, click a button, update the design, update the code and test it see if we've improved it. That's the kind of missing feedback loop that we weren't able to see, we weren't able identify.

And so obviously it's not gonna work for everyone. There are still people who want to go off and just. hammer one of these things out by hand and do the best that they can. But we're trying to find an easy way for them to say, let's make it easier for you to get governance outcomes without having to have a big heavyweight process, whilst at the same time saying, if you are going to do big heavyweight process, get some value from that process.

Make sure that you are actually investing upfront, shifting the value left, but in a way that you can draw upon that value later on, not just by doing diagrams, but designing things in a way that those designs are live and executable. And so that's kind of where we're seeing people at the moment. lot of good intentions, but not a loss of necessarily automatable execution that that kind of flows through that process. But similarly, a lot of people who just want to get on with it and do things quite quickly, don't really have lot of opportunity to push things further up and say, Hey, can we have a conversation about my customer versus your client versus your user, and look for ways to get some enterprise consistency across those things.

So that's, kind of the two extremes that we've seen in almost anything in between. But I think in all those circumstances, people ⁓ to do better, there is a and identified failure mode that aren't necessarily know what it is. Things like API sprawl pop up as kind of terminology to describe some of these things, but I don't think that really reflects the problem. sprawl is more about we've got no idea what APIs we've got, not we don't know what our APIs are doing.

How do we make sure that these APIs are consistent and delivering enterprise value rather than just product by product value? But I think there is a desire to do better, but a bit of a challenge for them in how they actually do their work and how they improve in that space. Ikenna: Yeah, yeah. kind of mentioned jargon ⁓ some of my listeners are going to be wondering what's jargon?

You know, and you've given us kind of the ⁓ of what it does, I think, in what you've explained, the of big ideas. But can you just tell us a bit more about just for someone who's never heard of jargon before? Yeah. Alastair: Yeah.

Yeah. And so without, without turning this into a sales process, I'll try and keep it as, kind of technology agnostic, but, intent driven as possible. for us, before got into kind of consulting around API design, we were IT architects. And so we spent a lot of time ⁓ thinking systems and doing systems as well as then building those systems.

And so for us in our work, ⁓ we quite a large disconnect between The types of activities that we do from a DevOps sense, and DevOps is quite good and quite mature and very sick with a lot of feedback loops. But those things were often very, very disconnected from the architecture work that we did to thinking and the designs. And so we'd go through this process of getting around whiteboards or talking to groups of people and figuring some of these things out. And then we'd stop that activity.

And then we'd go and start DevOps activity. And we do DevOps for a while. And if we had to go back to the thinking activity, we'd stop the DevOps activity and go back into the thinking activity. And so in our minds, we said, well, why is this divide?

Why does DevOps start when we write code when we start thinking about systems and how we start operating the systems, why there any work about how we think about these systems, how we design these systems, how we govern these systems? ⁓ so for us is our realization of bringing those things closer together. What can we do about bringing some of the DevOps mentality ⁓ and DevOps rigor into how we think about these things and design these things. And that largely came about when we thought about where do we get these things wrong?

It's not because we build the systems wrong. It's not because we operate them wrong. It's because we thought about them wrong, because we designed them wrong. It's because we diverged from where we were.

And so that's an age old thing. That wasn't a new thing for us to discover, but certainly when we were doing some feedback on what can we do to improve the quality of work we provide for our customers, where have we gone wrong and where has things tripped up and had bad consequences? Yeah, of course, sometimes it happens in the build because software has bugs and you fix those things. But more of it was because we didn't necessarily do the work we needed to do ahead of that, the shifting left of that to make sure that we're doing the right thing, not just API by API, but at an enterprise level.

And that becomes quite difficult because enterprises are very long lived things. You make a decision today that is going to impact you 10 years in the future, but you don't know about it until 10 years time. And so we're trying to look at ways we can say, how can we bring the kind of design and architecture thinking closer to DevOps? And so jargon for us was about saying, Well, here's how we do.

We turn designs into artifacts that can be executed in the government. We turn designs into things that can kick off and be part of DevOps pipelines. It seemed to make sense. So jargon for us is largely applying what we think the enterprise and the industry's learned from open source software and DevOps, but pushing it further left upstream into the thinking and the governance and the design of these things to try and get better quality of outcomes and try and get the value from the DevOps thinking process.

how we govern and design these things. That's for us what kind of jargon is and where we saw a gap and said well let's have a crack and see how goes. Ikenna: Really cool. Really cool.

You know, going to just pause here and give a kind of a disclaimer my listeners to say, look, ⁓ I'm an independent consultant. You know, this is not a sponsored podcast or anything, but look, you know, ⁓ I come across Jagan. I thought like, wow, this is really interesting because ⁓ yeah, love domain design. I'm a big fan of domain driven design and that whole, the rigor of that approach, right?

Solving the kind of problems we're talking about. And, and Jagan was something that brought that kind of DDD view and API design and CICD. I cannot try to really merge those things. And I thought, you know what, it would be good to have a great, have a conversation with Alistair.

So, so, so big disclaimer, you know, this is not sponsored. I'm not being paid for this. I just, I just love kind of the approach that the platform is bringing, which is why we're having this conversation. But can I just, yeah, can I just dial back a bit because I've used Jagan a bit and in there, I saw this kind of.

Alastair: Absolutely. Yeah, absolutely. ⁓ Ikenna: This metaphor of like a town ⁓ almost like a town plan. So you have town planning metaphor and then ⁓ and then go, that's like at the macro level and then going down into kind of more objects ⁓ entities, kind of data models.

⁓ you just tell us about that kind of framing ⁓ that? Yeah. Alastair: Yeah, absolutely. And again, a lot of this was in the talk that we talked about how do we get better at governing these things?

And I think what we've learned is that you kind of can't govern what you can't see. And so unless you know how one API has relationships to another API, how can you govern the relationship between those two things? How can you govern the dependency between those two things? Again, even though we're not talking about linting, you can't answer that question just by looking at one API at a time and saying, are these things going to work together?

And so in order to do governance, you need to think above individual APIs. You need to talk about how one API interacts with another API. You need to talk about how this data is related to some other piece of data. so for us, we said, well, what do we do about that?

And so town planning is a, as you say, a metaphor, but it's also a practice from other forms of engineering. If you're a civil engineer, it's quite often that you've got a plan and you talk about, ⁓ I a four lane road between this place and that place? Or will a dirt road do? We think about these things as both individual chunks as, and having individual lives on their own, but also the interrelationships between these things and how they work between one another.

And so town planning for us is, I just say a metaphor, but also our first class citizen where we say, how do we talk about more than one API? How do we talk about kind of sweets of APIs, the sweets of domains and how they interact with each other. And all of that led then to thinking about, well, we need to make sure that APIs aren't just documents, whether it's a a live document in some sort of API development portal or a PDF document or a Word document or however you describe it, they should be packages of information which should encapsulate a lot of things around how it works and what it does.

And again, this is exactly what happens with open source software. If you look at a GitHub repo or an NPN package, there's an awful lot more information than just here's the code or here's a binary. You learn an awful lot more about it. And so what can we do about it saying let's add more of that information into APIs and data in ways that allow people to self-serve and make decisions and look at relationships and navigate those relationships.

I should better click through and say, if this API depends on that data, I'm going to go look at that data. I want to work out what that depends on. so, town planning for us is a way of saying, how do we rise above looking at just one API in isolation? Like we said about linting, you can look and learn a lot from one API, but there are things that you can't learn by looking at one API.

So let's go up a layer and look. at a collection of these things and talk about that collection of things and figure out how they work and what they do. And again, as a consequence of that, we needed to have more information about these individual things to help people figure out what they do and how they work, what their dependencies are. And much like any other kind of DevOps platform or process, you should be able to have observability of those things.

You should be able to look at these things and say, well, who is using that? When was it last changed? What breaking changes does it have? All the things that we expect day to day from our kind of DevOps platforms and pipelines.

We don't have how we design these things. so, planning for us as way of saying, how do we bring some of that rigor, some design process to thinking about suites of APIs. And again, if you're an enterprise, care an awful lot more about suites of APIs than you do about individual APIs. Sure, if you're Netflix, you've got one API for streaming and that's obviously like a cash cow.

But if you're a bank, you don't care about retail banking versus wholesale banking. You care about consistency across banking across my enterprise, because you care about money laundering or you care about any number of things. And so. Ikenna: Yeah.

Alastair: by looking at the forest from the trees and having it be an activity that's not just on PowerPoint or a mural, you turn it into an artifact that you can play with that you can put on front of your DevOps pipeline and have it live together with that process, rather than being a standalone thing that as I said before, you do and then you stop and you pick up something else and you keep going. You start having these disconnect, you don't feedback and kind of correct your designs as you go.

Town planning for us is way of saying what to join those dots and have kind of the thinking and design be part of the DevOps process. Ikenna: Yeah, yeah. You you mentioned that, you you can't govern what we can see, you know. so, you know, that that seeing is important.

⁓ know, popular people say is, you know, a picture is worth a thousand words, isn't it? So, ⁓ you can you give us a screenshot? We've talked about this town plan and everything. What does it look like?

You know, can you can you can you share screen? You show us a bit of Jagan and this town planning in Jagan for for the listeners. Alastair: Yeah. Sure.

Yeah, absolutely. Let me... Ikenna: People watching this on YouTube, maybe on the audio podcasts, we'll be able to see if on YouTube, yeah. Alastair: Yep, so I've got, as everyone does, handling something prepared earlier.

Let me share my screen and we can have look at it. That one. All right. So this is what we think a town plan is.

And so for us, a town plan is something that groups things together into different roles. so, again, without going into the jargon, is there any way to do this? For us, we want to say, Ikenna: Mm-hmm. Alastair: What are the things on a town plan?

so each of these boxes or buildings is a domain, is an API, something that kind of describes information. And they've got metadata about them. We want to know, does this one have life cycles? Does this one have events that get thrown from it?

What are the events? How do I consume those events? How do we layer and structure these things together? Which ones are customer facing?

Ikenna: Yeah. So just to interrupt, so in kind of DDD language, would that be like a bounded context for each of those layers? Alastair: Very much so. we, it's interesting you bring that up.

We, we focus very much on, on DDD, but we've found that DDD isn't necessarily practiced by the people who need to make these decisions. And so we try and use DDD language where it makes sense and where it kind of is hard around aggregates and entities and things like that. And we are deliberately a bit more vague on some DDD context, not because we disagree with them, because we found that the people who make the decisions around here who are really impacted by things like bounded context.

typically haven't got the familiarity or understanding with them. yes, absolutely. So these be things like a bounded context. We call them, again, town plans and domains because they're far more familiar terminology.

We look at all of our large enterprise customers and they've got domain architects, someone who's responsible for retail banking or a domain architect for wholesale banking. And so we tend to use that language and that terminology because it marries up to what they expect. But absolutely, these would be bounded context. And then we group and name those bounded context by their role or by their...

innate characteristics to kind of lay it out and group it. And so for us, town planning doesn't have to be really any more complicated than that. We care lot more about how it kind of lives and works at scale, but if nothing else, we should rather lay out what are our APIs and how do they work. And so for us, we can click on some of these APIs and see that they are linked together with one another.

And we can then traverse those relationships and work out where they are and what they do. Similarly, we care a lot about the non-functional aspects of these things about life cycles, you know, are all these things at the same level of maturity across an organization? Which ones are in progress? Which ones aren't in progress?

Which ones have value that haven't been taken up by other groups? And so it could be as simple as having an online mural board where you keep track of these things and then have a look at it. And I suspect many organizations do, but for us, the difference is that we need to go a bit more than just having a picture of these things. wanted it to be a...

an interactive process that's intimately tied to the design of how these things work and how they operate rather than as being a standalone picture. So if you'll, you'll, I'm happy to dive into kind of what each of these domains and APIs has a look and a feel to it and about how we actually then turn those into, into real things to help try and bring some life to some of the, the shift left governance kind of topics that we've been talking about and seeing how there's an opportunity for not just linking on some of these things.

Ikenna: Yeah, absolutely. Absolutely. Because, know, so it's great to see this view as an architect, you know, this kind of town plan, but also from the dev who's I'm the dev, I'm designing an API, you know, I need to do all those other things, like what, yeah, tell us a bit more what that looks like. Alastair: Yeah.

So let's have a look. we're looking at, like said, this, this town plan view and we're looking at a forest from the trees view. But if we, if we kind of dive in, we can have a look at, one of these things. And so if we have a look at the order domain, we can go and have a look about, about order.

so we can see that there's firstly a model that describes kind of what it looks like and how it works. And so we can start straight away to see the data in these things. And so I've got, I've got the editor for that opening and I've preloaded it with some. design time problems that typically are invisible to how we govern APIs typically.

And I'll go through them and we'll talk a little bit about how, how, all we do is link individual documents at a time, it's very difficult, if not impossible to kind of find some of these things. But I think what's, what's interesting to talk about is that it's at this point when governance is typically the most invisible when we're designing the API and when we're thinking about it and talking about it. And so, like I said, I've preloaded this with, with some deliberate problems so we can talk a bit about how we can think about them from a governance perspective and specifically about shifting them left.

And so the first one here is that we've got some information about a customer and we've got information about their address. We know, you know, know, humans know that if we're talking about sharing data with people and we're talking about things like addresses, well, there's a good chance that's got personal information about it. And so we should know. Is this personal information that is private or is it public?

it things that we are as enterprise allowed to share or do we need to lock down and control? And so we can have governance rules that say anytime anything looks like it might be personally identifying information, we need to come along and tell ⁓ our company, our organization, what do we do with it? Is it public? Is it private?

And so I can come along and say that the data classification of this is private. And so that should inform downstream processes. If we've shifted this ⁓ value to the left, then we should realize that benefit to the right. And so by saying this information is private, we should expect something to fail if we design an API that doesn't have security around a path that interacts or changes this information.

so by us describing to the enterprise that this information is private, we should get some value from that. so again, if we can't have making, if changing this to private doesn't say great, throw some error downstream when you've got an API that doesn't protect it, then that's not governance. That's not shifting things left. And so even at that level, it'll be very, very difficult to write a linting rule that detects and identifies that.

Aside from the fact that you'd have to hard code into your linting rule, all of the things that could possibly be personally identifying, you'd then have to write a linting rule that can handle all the possible ways that information is structured in your API. If you've got a whole bunch of DEFs or REFs, or even if you've got inline stuff, You'd have to write a linting rule that can handle all the possible permutations where this data could sit in any number one of your schemas or your responses to work out does the API path talking this thing have a security header or does it not have a security header?

And so even if you could do it by linting, the difficulty would be intractable and you'd have to hard code into all your linting rules the things that you care about from a design perspective. so. Ikenna: Mm-hmm. Alastair: There is a variety of things again that we simply can't look at one document and say, this correct?

And like, you've got to look at the context of the organization in that vein. There's another area here saying that, customers already been declared somewhere else across your organization. So you probably shouldn't make your own one. Or if you are going to make your own one, maybe subclass from some common thing instead.

And so again, it's looking at how do we know that we're doing things the right way? How do we let people know when they're designing things? How do we give them the guardrails to help them? stay on the right track rather than telling them when they get some design review three weeks later that, hey, we've already got a custom model and you haven't used it.

What can we do upfront to let them know that we do have a customer model? It looks like this. Go and have a look at all the ways that's being used. Have a look at all the different comments there are and why we've got it ⁓ designed this way versus some other way and learn a little bit about why we've done it.

So you can make better choices as well. so again, a lot of it's about saying what can we do to bring the wider enterprise context into the design process to make sure that people designing these things. have as much help and assistance as possible to do it the right way. Another example down here is that we've got a payment status.

And so the area here is that, status looks like a life cycle. And I think it usually is, right? If you're talking about status, you're probably following some sort of design process. And the area here is that saying, well, at the moment, you've got status as just an enumeration of values, ⁓ whether it's been purchased or rejected or something.

⁓ And that's rarely the case. We probably have some sort of life cycle that describes. how these things work and how they operate. Something that says, well, here are all the possible states these things can be in.

Here is how we can move between those states. And here's how we can explain to people what those things are. And so again, yes, it might materialize in an open API specification as a list of canceled or placed or preparing, but from a governance perspective, we want to make sure that this thing follows some lifecycle and some process. And so even if you could write, and I'm sure you can write a linting rule that says, please check this thing as the appropriate states, how would you know what those states are?

How would you tie that back to a government-agreed model that describes the business life cycle of this thing? And secondly, if you have got a model that describes the business life cycle of this thing, it's probably going to appear in your documentation for API. And so what can we do about making sure that the internal rules that say what the valid states are line up with the documentation that we tell people about, how they can expect these things to flow and operate? And so that's another example where saying, There are things we can check for by linting and there are things that we can't check for by linting.

Even if a list of states or a state machine values both look the same in an open-air guide specification to a linter, they're not necessarily the same from a governance perspective. And so we should be able to say, we care about these things at this layer and then push that value further downstream so that we know anytime that we have things like status codes for argument's sake, they are backed by governed agreed designs for how these things work. And so for us, that's, that's kind of the key of what we mean by, by adding guardrails to things and shifting things left.

Even at this stage, when we're just nodding around the design of kind of what the data looks like in this API, we can start running governance checks of have I done this in a way that's consistent across enterprise? Have I done this in a way that's consistent with how we design things? Have I done this in a way that's consistent with our enterprise rules for personal identifying information for argument's sake? All of those things.

are burned in at design time. And again, like I said, this design time is often where governance is the most invisible. Sure, you might be running a linter in the background or in parallel as you're doing these things, but as we've talked about, linter can only check so much. Certainly you can't link for a variety of those enterprise context things that I talked about.

And if we can't link for them, then you don't see them. And as we said before, if you can't see it, you can't govern it. ⁓ It's very rare for organizations to do these things and care about them until they work out that they're wrong. And so shifting left in our world view is what can we do to bring forward things that we would normally learn about later on when we've deployed and released these things right back when we were designing these things so we can work out early and soon what the consequences of these things are.

More than that, we should then learn a little bit more about how these things operate and live. And so let's say for argument's sake, rather than me fixing it the right way, I just go through and delete stats code or... Ikenna: Yeah. Alastair: I might want to change the name of ⁓ ID to order ID.

I've now made a variety of changes that seem innocuous at the moment. Everything still looks kind of okay, but probably it's broken. And probably we've made changes that we care about that other people should know about. And so part of this as well is not just knowing about what works for me and how do I make sure that I apply my governance early on.

A lot of it's a bit about how do I make sure that I do think in a way that make it clear to other people. And so I'll quickly go through and have a look at that. I'll just save out of this to update the API design based on what I've done. If we go back to here, we should now say that, well, we should know, waste on what we did previously, what we should be doing now.

And so by looking at the changes we've made, there's no longer a property called auto ID and we've added some new things. This is now a breaking change. As a consequence it being a breaking change, we should tell other people that there are things that you need to do. You can no longer consume this as you were consuming it before because something has been broken.

That's the entire intent behind semantic versioning. And if you've made a bunch of changes to the design of your thing, the system should tell you that you've made these changes and the system should tell you that these changes have these consequences. And so if I go through and make this release, if we now go back to the town plan, we should now see. that something has changed.

And that's exactly what it's telling us. It's now telling us that that order domain on our town plan that we were using that was linked in these ways to these things is now breaking. And so if you were relying on it, or if you were depending upon it in some way, the next time you update the newer version, it's going to break. so again, that's another example of just at design time, we should better know what the consequences of these things are.

And so we talked about that quite a lot in our talk about having blast radiuses of change, we should know in advance of making a change. consequences are going to be, who's going to be impacted so we can go and talk to them and so we can work with them around what the right thing to do now is. Typically you find that out when you have a problem in production. You typically find out during testing or integration testing that these things aren't working the right way.

And that's, that's great, but it's really quite late and we've spent a bunch of money and time getting to that point. so shifting things left and in our world API governance is about knowing those things before you write the code. It's about knowing those things as you design for these things. And so I don't, don't want to go into a huge map more of jargon.

Again, this is, as you said, this is not a sales picture of sales call or sponsored content. Really all I want to do is help people visualize how some of these views and concepts around shifting governance left and, what you can and what you can't govern and APIs can look like. And hopefully start making people a bit aware about how they do work for themselves in their own organization, irrespective of what tool they're using. And if nothing else, at least know the limitations.

What can you check with linting? What can't you check with linting? And if you can't check some things with linting, what other checks are you doing? What other processes do you have in place to know that, Hey, we've just introduced a breaking change.

And that breaking change is going to impact these teams in these ways. And so a lot of this is about trying to say as a profession, as people who care about APIs, what can we do to learn from how we design software, how we build software, how we test software. and apply many of those same DevOps principles, platforms and processes to how we design and govern the design of APIs, the design of exchanging data between two groups of people or across enterprise. And so for us, that's kind of where we focused on, about making things observable, making things ⁓ easy to look at and easy to see.

As I said earlier, you can't govern what you can't see. And if all of this information only exists within a couple of API specifications, it's going to be very difficult for the right people to see that and make a change. Sure, a developer might go through and have a look, but what about the enterprise architect? How do they then know what the consequences of this change were?

Because there's absolutely going to be some consequences. Otherwise it wouldn't be a breaking change. That's what Semblor gives us. And so again, without diving too far into into jargon about how it works and what it does, hopefully this has given a bit of a view of why we should care about some of these things and about what observing some of these things can tell us.

versus just linting individual documents one after the other. Ikenna: Yeah, it's just really powerful to see that, Alistair, really powerful. You know, as we wrap up, know, imagine I'm the director of engineering, I'm the head of APIs or head of architecture in an organization. And I've listened to this, you know, I love that idea of shifting left on API governance of all these good things we've talked about.

But where do I begin? know, where do I start? What are the principles? What are the tips you would give me to get started in applying the ideas you've talked about my, know, if I platform teams, to my, you know, software teams, to the architectural teams ⁓ doing ⁓ Alastair: Yeah, look, absolutely.

again, I think, I think the key theme through all this is observability. It's about knowing what you've got and finding ways to, to extract from individual things, some sort of structure, some sort of pattern, some sort of design of what these things are how they work. And so I think if described earlier in our chat kind of two groups, people who start with PowerPoint and then kind of lose control and people who don't even start with PowerPoint at all. ⁓ I think depending on who you are, you obviously have two different perspectives on how to approach this.

If you start with PowerPoint, you clearly care about these things at an enterprise level. You probably find it difficult to flow that design and that value all the way through and down. I think for them, it's about saying, what can we do to make sure that we bring that PowerPoint, those diagrams closer to the teams? How do we make sure that we keep those things in sync and up to date?

Yes, you're quite right, a picture is worth a thousand words, but if it's wrong, it's not worth very much at all. In fact, it can be hurtful and harmful if it's wrong. And so if you are gonna draw diagrams, make sure you get value out of those diagrams. Make sure you get the value that you've invested into that time of doing it.

It doesn't have to be heavyweight. We're not talking about going to incredibly detailed model-driven architecture or you're not a designer for these things. If that's what you want to do, great. If you don't want to do that, also great.

But if you're going to invest time doing some sort of design, get the value from that designer. So find ways to bring those things closer together. Find ways to go and check, is this still what the design was? And again, you can't just rely on linting.

You can't run a linting check that says, is my API in alignment with his PowerPoint presentation and how the model works on the Fortune. So you've got to do something else. And so if you've got to go and check, if you've got to put things in your calendar every couple of weeks to go and do a difference check or a gap analysis between what we designed, then that's what you've got to go and do. Find some way to make sure that you can bring your designs into the DevOps process because DevOps is going to leave you behind.

At the end of the day, they're going to do a lot of work and a lot of value. They're going to release to production and now it's too late. Once it's in production, you don't go back, right? You don't take things away from people.

And so You've got to make sure that you get in front of that process. if it has gone to production, what can you do that dealing with the consequences of what we've done and where we've been? And so ⁓ got to bring the design to DevOps DevOps isn't going to go to you. They're going to keep doing their work and having value to the organization, even if it doesn't add value to the entire organization.

So you've got to meet them where they are. If you're in the other category of, we're, too focused on individual platforms and individual channels to worry about the enterprise. You've got to start worrying about the enterprise. Sooner or later, it's going to bite you.

Like I said earlier on, you're probably making decisions now that are going to bite you five, 10, 15 years down the track. You might not even think it's going to be a problem down the track. But things change quite rapidly in business, enabled by technology. Things that we didn't realize were going to be possible are now becoming quite possible.

And if you've got a whole bunch of lines of business that aren't aligned and aren't joined up and can't be joined up because we've made deliberate decisions or we've made deliberate omissions, that mean they can't be joined up. That's going become a problem for you. So what can you do about trying to extract from these individual APIs some enterprise design and looking at these things and again, so I do some gap analysis. Again, I think a lot of it is around observability.

You've got to find some ways to see what these things are. Sure, you might not have an automated process that says as I'm designing these things, I'm breaking these rules or what are the consequences of me changing that? That's cool. You've got to find some way to supplement that.

You certainly can't just say, I run linting, so everything's okay. That's not the outcome. And hopefully, For some of this conversation, we've given some examples of why linting isn't governance and what governance starts to look like in these spaces. And so if I was a technology leader coming across this for the first time saying, hey, my teams are telling me we're doing governance.

I'd like to think that they're saying, well, maybe there's more to this than my teams are telling me. Not that the teams are lying, the teams are governing within their worldview of Skype and what they're looking at, but there's more to it than that. And so you should go and see, go and check. What are we looking, what are we checking?

And if we're not checking these things in governance, how are we checking these things? And I suspect for a great number of your listeners, they're probably not checking these things. And it's not doom and gloom, right? This is not enterprise is going to fail tomorrow, but it does mean there are some things you probably start having a look at to make sure they are observable so you can make better decisions around this information.

Ikenna: Yeah, yeah. Thanks, thanks Alisa. It's been great just getting this, know, getting your view on this and seeing this all come together, you know, with both the principles and seeing this in practice as you've shown us. If my listeners wanted to find out more about you, kind of what you do, maybe about Jagan, where would you ask them to go?

Alastair: Yeah. Yeah, the best place to go and have a look at is jargon.sh. There's an awful lot of documentation there.

There's links to a whole bunch of articles that we write around doing these things. There are links that you can find to the API days presentation on how do we shift things left. And so aside from being a platform that we run, we are genuinely trying to improve practice for these things and make it easier for people. So jargon.

sh is the best place to find information about this kind of stuff. Otherwise, if you're interested in me specifically. I'm on LinkedIn, I write a lot of articles around governance, APIs, around data governance, because I think these things need to be a lot closer together than they currently are. And so I'm findable, if you want to find me.

But otherwise, jargon.sh is the best place to start for looking at our view on how some of these things can be handled. And like we showed in the demo, it's filled with lots of interesting visuals to help you think for yourself. How do I do this for myself in my organization?

What aren't I doing? Whether you use jargon or not use jargon, I hope there's something that helps people guide some decisions around. finding better ways to govern these things rather than just focusing all the money and all the attention on the big platform. Ikenna: Cool.

Thank you very much, Alistair. It's been great. It's been absolutely great and enlightening talking to you. And I look forward to seeing you again soon somewhere, maybe in some API Days conference or something like that somewhere.

Okay, thank you. Thank you very much. Bye. Alastair: Fantastic.

Thanks for having me.