Cluster autoscaler is a popular tool for automatically adjusting the size of a Kubernetes cluster based on the current workload. It helps ensure that your applications have enough resources to run efficiently while minimizing costs by scaling down unused nodes. However, monitoring the cluster autoscaler is crucial to ensure that it is functioning correctly and that your applications are running smoothly.

Keda is a tool that provides event-driven autoscaling for Kubernetes, allowing you to scale your applications based on external metrics. It uses the Kubernetes Horizontal Pod Autoscaler (HPA) to adjust the number of pods in a deployment based on metrics like CPU usage, memory usage, or custom metrics from external sources. It also supports scaling based on event sources like message queues, databases as a job and defines a new Custom Resource Definition (CRD) called ScaledJob to configure the scaling behavior. Monitoring Keda effectively is crucial to ensure that your autoscaling policies are working as expected and that your applications are performing optimally.

Not long ago, I shared how to monitor Kubernetes costs using OpenCost, Prometheus, and Grafana. In this follow-up, I want to highlight a specific feature: setting up proactive budget alerts using OpenCost and Prometheus. It’s a simple but powerful way to catch cost overruns early.

Kubernetes events offer valuable insights into the activities within your cluster, providing a comprehensive view of each resource’s status. While they’re beneficial for debugging individual resources, they often face challenges due to the absence of aggregation. This can lead to issues such as events being garbage collected, the necessity to view them promptly, difficulties in filtering and searching, and limited accessibility for other systems. The blog post explores configuring Loki with Alloy to efficiently scrape Kubernetes events and visualize them in Grafana.

With the release of Karpenter v1 we have stable Prometheus metrics, but the Grafana dashboards are not that great and there are no open source alerts. Therefore, I decided to create a monitoring-mixin that provides a set of Prometheus rules and Grafana dashboards for Karpenter. This blog post will introduce the kubernetes-autoscaling-mixin - a set of Prometheus rules and Grafana dashboards for Kubernetes autoscaling, but we will only write about Karpenter monitoring in this blog post.

The kube-prometheus-stack Helm chart, which deploys the kubernetes-mixin, is designed for standard Kubernetes setups, often pre-configured for specific cloud environments. However, these configurations are not directly compatible with k3s, a lightweight Kubernetes distribution. Since k3s lacks many of the default cloud integrations, issues arise, such as missing metrics, broken graphs, and unavailable endpoints (example issue). This blog post will guide you through adapting the kube-prometheus-stack Helm chart and the kubernetes-mixin to work seamlessly in k3s environments, ensuring functional dashboards and alerts tailored to k3s.

OpenCost is an open-source tool designed to help you monitor and understand the cost of your cloud infrastructure. As a project under the Cloud Native Computing Foundation (CNCF), OpenCost offers a transparent and powerful solution for cloud cost management. It provides both a user-friendly interface for visualizing cloud costs and Prometheus metrics, enabling you to query and visualize these costs using Grafana. The popular tool KubeCost is built on top of OpenCost, offering an enhanced feature set and user experience. However, KubeCost is not open-source, and its free plan has limitations on data retention and storage. Given these constraints and a preference for consolidating data visualization within Grafana, I opted to use OpenCost. This blog post will introduce the opencost-mixin - a set of Prometheus rules and Grafana dashboards for OpenCost.

The kubernetes-mixin is a popular resource for providing excellent dashboards and alerts for monitoring Kubernetes clusters. However, it lacks comprehensive support for autoscaling components such as Pod Disruption Budgets (PDB), Horizontal and Vertical Pod Autoscalers (HPA, VPA), Karpenter, and the Cluster Autoscaler. These essential components are commonly deployed in Kubernetes environments but do not have standardized, open-source monitoring solutions. This blog post aims to solve that by introducing the kubernetes-autoscaling-mixin - a set of Prometheus rules and Grafana dashboards for Kubernetes autoscaling.

The Vertical Pod Autoscaler (VPA) can manage both your pods’ resource requests but also recommend what the limits and requests for a pod should be. Recently, the kube-state-metrics project removed built-in support for VPA recommendation metrics, which made the VPA require additional configuration to be valuable. This blog post will cover how to configure the VPA to expose the recommendation metrics and how to visualize them in Grafana.

Ingress-nginx provides an easy integration with Prometheus for monitoring. However, it can be challenging to get started with monitoring Ingress-nginx and creating dashboards and alerts. Therefore, I’ve created a monitoring mixin for Ingress-nginx which will provide Prometheus alerts and Grafana dashboards focusing on Ingress-nginx.