Introduction Containerd is a popular container runtime used in Kubernetes. It serves as the underlying engine responsible for managing low-level tasks, such as creating and managing container sandboxes, starting processes within those sandboxes, and handling image pulling. As Containerd plays a crucial role in the smooth operation of containerized applications, monitoring its performance and health […] The post How to monitor Containerd appeared first on Povilas Versockas.| Povilas Versockas
Introduction PrometheusMissingRuleEvaluations is an alert coming from Prometheus Monitoring Mixin. Monitoring Mixins are a bundle of Grafana Dashboards, Prometheus Alerts, and Recording rules. Check out my Getting Started With Monitoring Mixins blog post to learn more about Monitoring Mixins. You typically get this alert automatically if you use kube-prometheus-stack or kube-prometheus solutions. Typically, the configuration for the […] The post How to fix Prometheus Missing Rule Evaluati...| Povilas Versockas
Introduction Kubelet is a Node daemon that registers itself with the Kubernetes API Server and manages Pods for that Node. It’s important to know that, Kubelet works in terms of Pods rather than Deployments or Containers. It watches the Pods assigned to it and ensures they are healthy and running. There are many things Kubelet […] The post How to monitor Kubelet appeared first on Povilas Versockas.| Povilas Versockas
Introduction Missing logs can be a thorn in the side for many Kubernetes users. In this article, we dive into why this happens and how to prevent it. I have been investigating the case of missing Kubernetes logs in Elasticsearch, which in my case, aggregates logs for Kubernetes pods. I’m running a standard setup of […] The post Troubleshooting Missing Kubernetes Logs in Elasticsearch appeared first on Povilas Versockas.| Povilas Versockas
Introduction A while back, my colleagues and I, ran a Kubernetes cluster with large nodes with about 300-400 containers running on each node. It was running on Linux CentOS 7, with a linux-3.10.0-1160.88.1.el7 kernel. And after about three years of mostly stable cluster, our nodes started randomly freezing. It usually started in the morning, and […] The post Debugging a Random Node Lock Up in a Linux Kernel appeared first on Povilas Versockas.| Povilas Versockas
What are Kubernetes Controllers? Kubernetes Controllers are designed to manage the lifecycle of certain resources, like Deployments and Services. They are responsible for keeping the cluster’s desired state in sync with its actual state. For example, the following code snippet shows a simple Controller loop: Note: this example is taken from the following documentation. So the […] The post A Technique To Monitor Kubernetes Controller Latency appeared first on Povilas Versockas.| Povilas Versockas
Introduction The Four Golden Signals and the RED method are fantastic for monitoring user-facing request-driven services. But how do you apply The Four Golden Signals to non-request-driven services, such as – Kubernetes Controllers? What are Kubernetes Controllers? Kubernetes Controllers are designed to manage the lifecycle of certain resources, like Deployments and Services. They are responsible for […] The post How to monitor Kubernetes Controllers appeared first on Povilas Versockas.| Povilas Versockas
Did you know that Kubernetes Controller Manager can draw graphs? But first, let’s start from the beginning. The Kubernetes Controller Manager is a daemon that embeds the core Kubernetes controllers that act independently. Controllers watch the cluster’s shared state through the Kubernetes API Server and command changes to move the current state towards the desired […] The post Exploring the Debugging Features of Kubernetes Controller Manager appeared first on Povilas Versockas.| Povilas Versockas
This is a complete guide to Kubernetes API Server SLO Alerts. In this new guide, you’ll learn: Kubernetes official Service Level Objectives (SLOs). What are Error Budgets? How to turn Error Budgets into alerts? What are Multiwindow, Multi-Burn-Rate alerts? What is this KubeAPIErrorBudget alert? Mixin’s Kubernetes API Server SLO alerts. Lots more. Kubernetes Service Level […]| Povilas Versockas
This blog explores Kubernetes DNS, focusing on service discovery. It covers differences between DNS implementations such as musl and glibc.| Povilas Versockas
