DevOps Daily with Fexingo · 2026-08-31 · 10 min
Key moments - from our scoring
Substance score
38 / 100
Five dimensions, 20 points each
Resource quotas in Kubernetes enforce limits based on pod resource requests - not actual consumption - which creates a gap between perceived and real cluster capacity. Lucas walks through a real scenario where a platform team's namespace deployment mysteriously failed during a traffic spike, only to discover that a cron job requesting sixteen cores was consuming quota all day despite using just two cores. The episode breaks down how quotas operate at the API server level, why misconfigured workloads with inflated requests can lock out entire namespaces, and the dangers of scope selectors creating overlapping quota restrictions. Luna and Lucas emphasize that quotas apply to counts as well (pods, services, PVCs), multiplying the failure modes, and that adding quotas retroactively to running namespaces can trap you in an over-quota state. The practical guidance focuses on treating quotas as living governance objects: baseline them against actual usage patterns, monitor via kube-state-metrics or custom scripts at 80% thresholds, and conduct periodic reviews since request sizes drift over time. This episode is essential for platform engineers managing multi-tenant clusters and operators implementing namespace-level resource governance.
Quotas enforce limits on pod requests, not actual usage. If your namespace quota of ten cores already has eight cores of requests from another pod (even if that pod uses only two cores), a new pod requesting three cores will be rejected because 8+3 exceeds ten.
Run 'kubectl get resourcequota -n yournamespace' to see aggregate usage by resource type, then inspect individual pod specifications with 'kubectl get pods -n yournamespace -o custom-columns=NAME:.metadata.name,CPU:.spec.containers[*].resources.requests.cpu' to identify which workloads request the most.
The quota applies only to new pod creations. Existing pods are unaffected, but if the namespace is already using more than the quota limit, you cannot create any new pods until you delete enough running pods to drop below the limit.
Yes, using scope selectors in the ResourceQuota spec. You can create quotas that apply only to pods with certain labels (e.g., batch jobs) and separate quotas for other pods, but overlapping scopes can create surprising failures.
Use kube-state-metrics to expose quota status or write a script checking 'kubectl get resourcequota' and alerting when usage reaches 80% of the limit, so you have time to adjust requests or quotas before hitting the hard limit.
Our reviewer’s read on each dimension, with quotes from the episode.
The episode delivers a reasonable number of concrete technical points for a 10-minute runtime - requests-vs-actual-usage enforcement, scope selectors, retroactive quota inapplicability - but most of these are well-documented Kubernetes behaviors rather than hard-won insights. There is mid-episode filler including a donation plug that interrupts the technical flow.
quotas don't apply to existing pods. If you create a resource quota on an existing namespace, the pods that are already running are not affected
if you have a quota that applies to all pods, and another that applies to a subset, the API server will check both
Every point made - requests vs. actual usage, scope selectors, kubectl debugging commands - is canonical Kubernetes documentation content presented conversationally. There is no contrarian framing, no first-principles reasoning, and the closing takeaway is a generic platitude about monitoring and capacity planning.
Treat resource quotas as a dynamic part of your cluster health. Set them based on real usage, monitor them
The key thing to understand is that a ResourceQuota object doesn't just limit what you can create. It's actively enforced by the API server
There is no external guest; the episode is two hosts whose credentials and seniority are never established. The only firsthand practitioner evidence is attributed to an unnamed 'friend who runs a mid-size platform team,' making it impossible to assess the depth of operational experience behind the claims.
I've got a story from a friend who runs a mid-size platform team
I've seen a case where a team had a quota for 'batch' jobs and a global quota
The episode does use concrete numbers throughout its examples - specific CPU core counts, memory figures, and pod limits - and names real tooling like kube-state-metrics and actual kubectl commands. However, all war stories are anonymised and unverifiable, limiting evidential weight.
They had a namespace with a resource quota that looked generous: twenty CPU cores and forty gigabytes of memory. But they had a bunch of stateful services that requested way more than they used, and they had a cron job that only ran for a few minutes but requested sixteen cores
you can run 'kubectl get resourcequota -n yournamespace' and it shows the current usage and hard limits
Luna's questions are functional connectors ('Can you give me an example?', 'So what's the best practice?') that keep the explainer moving but never probe assumptions, challenge framing, or create productive tension. The format is essentially a structured lecture with affirmative back-channelling rather than genuine dialogue.
So, what's the best practice? How do you avoid this kind of surprise?
Ah, so it's a classic case of 'quota says no' even though you have headroom in practice
Computed from the transcript - who did the talking, and the words that came up most.
In Episode 172 of DevOps Daily, Lucas and Luna drill into a deceptively simple Kubernetes feature: namespace resource quotas. When a namespace hits its quota, pods fail to schedule and existing workloads can be evicted. Using a real-world example of a production outage caused by a misconfigured ResourceQuota, they explain how Kubernetes calculates quota usage, why the system doesn't always behave the way you expect, and how to avoid common pitfalls. They discuss the difference between requests and limits, the role of scope selectors, and why monitoring quota usage is often overlooked. If you run Kubernetes in production, this episode will save you from a painful 2 a.m. page. #Kubernetes #DevOps #CloudNative #ResourceQuota #NamespaceQuotas #Infrastructure #SRE #TechOperations #Reliability #CloudComputing #ContainerOrchestration #ClusterManagement #ProductionIncident #SecretsManagement #FexingoBusiness #BusinessPodcast #TechnologyPodcast #DevOpsDaily Keep every episode free: buymeacoffee.com/fexingo
Transcribed and scored by The B2B Podcast Index.
Lucas: So, you and I have spent a lot of episodes talking about Kubernetes edge cases. Pod disruption budgets, finalizers, metrics server latency. But there's one feature that feels so simple on the surface, we barely give it a second thought. I'm talking about namespace resource quotas.
Luna: Resource quotas. I mean, you set a number, and Kubernetes makes sure the namespace doesn't use more than that. What could go wrong? Lucas: Right, that's the theory.
But in practice, quotas can be the source of some of the most confusing failures in a cluster. I've got a story from a friend who runs a mid-size platform team. They had a namespace that was quietly failing to deploy new pods, and the only clue was a cryptic event message about 'exceeded quota'. Luna: And of course, no one was looking at quota usage.
I think I see where this is going. Lucas: Exactly. And it's a great example because it shows how a seemingly straightforward feature can bite you if you don't understand how it actually calculates usage. But before we dive into the weeds, I want to mention something.
You know, a lot of listeners have asked how to support the show, and honestly, the best way is just a small monthly donation. A couple of dollars a month is genuinely what keeps these going - buy me a coffee dot com slash fexingo, if you've gotten something out of them. Luna: Yeah, it really does make a difference. Even a few dollars helps keep the episodes ad-free and the research deep.
Lucas: Absolutely. So, back to quotas. The key thing to understand is that a ResourceQuota object doesn't just limit what you can create. It's actively enforced by the API server, and if you hit the limit, the API server rejects the pod creation outright.
But the trick is that quota is calculated on requests, not on the actual usage. Luna: So if you set a quota on CPU with a request of 100 millicores, you're not going to be able to create a pod that requests 200, even if the node has plenty of free CPU. Lucas: Precisely. And that's usually fine, because you're expected to set requests that match your actual usage.
But the problem comes when you have workloads that are misconfigured, or when you have a mix of workloads with different request patterns. Luna: Can you give me an example of that? Like, what's a typical scenario where this becomes a real problem? Lucas: Sure.
Let's say you have a namespace with a quota of ten CPU cores. You have a batch job that requests eight cores, and it's running. Now you want to deploy a small web service that only requests half a core. That should fit, right?
Four point five cores total. But the batch job is actually using only two cores because it's waiting on I/O. The quota doesn't care about actual usage. It cares about the requests.
So the web service pod gets rejected. Luna: Ah, so it's a classic case of 'quota says no' even though you have headroom in practice. Lucas: Exactly. And this is where my friend's story comes in.
They had a namespace with a resource quota that looked generous: twenty CPU cores and forty gigabytes of memory. But they had a bunch of stateful services that requested way more than they used, and they had a cron job that only ran for a few minutes but requested sixteen cores. When the cron job fired, it basically locked out the namespace for the rest of the day. Luna: Wow.
So the cron job wasn't even running all day, but its request was occupying quota the whole time. Lucas: Right. And the team only discovered it when they tried to scale up their web tier during a traffic spike, and pods just wouldn't schedule. They saw the 'exceeded quota' event, but they didn't immediately know what was consuming the quota.
They had to dig into each workload's requests and manually calculate. Luna: That sounds painful. Is there a way to see quota usage without manually summing up? Lucas: There is.
You can run 'kubectl get resourcequota -n yournamespace' and it shows the current usage and hard limits. But that only shows the aggregate. To find which workloads are using the most, you need to look at each pod's requests. And that's tedious.
That's why it's so easy to overlook. Luna: So, what's the best practice? How do you avoid this kind of surprise? Lucas: The first thing is to set quotas that are realistic for your workloads.
That means you need to know what your typical requests are. And you should monitor quota usage over time, not just set it and forget it. There are tools that can alert you when you're approaching a quota limit. Luna: So, you're saying you should treat quotas like you treat capacity planning for nodes?
Lucas: Yes, exactly. And there's another subtlety: quota can be applied to more than just CPU and memory. You can also limit the number of pods, services, and persistent volume claims. That can be useful, but it adds another layer of complexity.
Luna: Oh, I've seen that. A team set a pod count quota to avoid namespace sprawl, and then they couldn't deploy anything because they had a bunch of jobs that each created many pods. Lucas: Right. And that's often the cause of mysterious failures.
The pod count quota is usually set to, say, fifty, and you have a deployment with a high replica count, and suddenly you're blocked. But the error message can be misleading because it says 'exceeded quota' but doesn't tell you which type. Luna: So, what's the best way to debug that? You have the event, and it says 'exceeded quota', but you're not sure which quota.
Lucas: The first thing is to look at the resource quota object itself. It will show you the current usage for each resource type. If you see that pod count is at, say, forty-nine out of fifty, that's your problem. But it's also important to understand that quotas are namespace-scoped, so you can have different quotas for different namespaces.
Luna: And that's where scope selectors come in, right? I've heard about those but never dug in. Lucas: Yes. Scope selectors allow you to apply different quotas to different categories of pods within the same namespace.
For example, you can have a quota for pods that have a certain label, and a different quota for the rest. Luna: That sounds powerful, but also potentially confusing if you have overlapping scopes. Lucas: Absolutely. And that's another place where people get burned.
If you have a quota that applies to all pods, and another that applies to a subset, the API server will check both. So you might have plenty of headroom in the global quota, but the scoped quota is full. Luna: That's a classic gotcha. I've seen a case where a team had a quota for 'batch' jobs and a global quota, and the batch quota was set too low, causing all their nightly jobs to fail silently.
Lucas: Exactly. And the worst part is, those failures often happen at night, and no one notices until the morning. That's why it's so important to have good monitoring and alerting on quota usage. Luna: So, what's a good metric to monitor?
Is it the 'used' field in the resource quota status? Lucas: Yes, you can scrape that. There are exporters, like the kube state metrics, that expose quota usage. But you can also write a simple script that checks the quota and sends an alert when usage is above, say, eighty percent.
That way you have time to adjust before a full block. Luna: And what about the other side? Some people set quotas too low by accident, and they end up blocking everything. How do you recover from that quickly?
Lucas: The simplest way is to edit the resource quota and increase the limit. You can do that with 'kubectl edit resourcequota', and the change is immediate. But you have to be careful because if you increase the limit, and the namespace is already using more than the new limit, the API server will start rejecting new pods. So you need to either delete some pods or increase the quota enough.
Luna: So it's kind of like a safety valve. You have to be careful not to blow it. Lucas: Exactly. And there's one more thing I want to mention: quotas don't apply to existing pods.
If you create a resource quota on an existing namespace, the pods that are already running are not affected. The quota only applies to new pod creations. Luna: So, if you add a quota after the fact, you might have a namespace that's already over the quota, and you won't be able to do anything until you delete some pods. Lucas: Right, which can be a real issue if you're trying to enforce quotas across many namespaces as part of a governance initiative.
You have to plan for that transition. Luna: Let's go back to your friend's story. How did they resolve it? Did they just increase the quota?
Lucas: They did, but it was a temporary fix. They also had to look at the workloads and reduce the requests on some services that were overprovisioning. It was a wake-up call for them to do proper capacity planning. Luna: So, what's the one takeaway you want listeners to remember?
Lucas: Treat resource quotas as a dynamic part of your cluster health. Set them based on real usage, monitor them, and don't assume that just because a namespace has quota, you're safe. The safest thing is to review your quotas regularly, just like you review your node capacity. Luna: And maybe schedule a periodic review of your request sizing too, because those can drift.
Lucas: Definitely. And if you have a story about quota surprise, we'd love to hear it. But for now, I think that's a good place to wrap up. Thanks for listening.
Other episodes covering the same guests and topics, from across The B2B Podcast Index.
Why Your Kubernetes Cost Reports Are Lying To YouHow Kubernetes Node Pools Can Cost You More Than You ThinkHow Kubernetes Pod Overhead Changes Node Capacity CalculationsHow Kubernetes Pod Security Admission Blocks Risky WorkloadsHow Kubernetes Service Meshes Hide Network Latency
