Cloud-Native Microservices With Kubernetes - 2nd Edition

DaemonSet: How to Use It

If you know how to create a Deployment, you already know how to create a DaemonSet. The manifest structure is very similar; the main difference is the kind field, which should be set to DaemonSet instead of Deployment.

kubectl apply -f - <
# Namespace creation
apiVersion: v1
kind: Namespace
metadata:
  name: monitoring
# DaemonSet creation
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
  namespace: monitoring
  labels:
    app: node-exporter
spec:
  selector:
    matchLabels:
      app: node-exporter
  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      hostPID: true
      hostNetwork: true
      containers:
      - name: node-exporter
        image: prom/node-exporter:v1.8.1
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9100
          hostPort: 9100
        args:
        - "--path.rootfs=/host"
        volumeMounts:
        - name: rootfs
          mountPath: /host
          readOnly: true
      volumes:
      - name: rootfs
        hostPath:
          path: /
EOF

The purpose of this DaemonSet is to deploy the Prometheus Node Exporter on every node in the cluster to collect system metrics. Note that if you are using a managed Kubernetes service, such as GKE, EKS, or AKS, you might not have permission to deploy Pods on the control plane nodes. The DaemonSet will only execute on the worker nodes.

If you are not familiar with Prometheus and the Node Exporter, here is a brief explanation:

Prometheus is a monitoring system that pulls metrics from various targets at specified intervals and stores the results in a time-series database. Among these targets, we can find exporters, which are components that expose metrics from different systems in a format that Prometheus can scrape. The Node Exporter is one of the most popular exporters since it exposes hardware and OS metrics from the node where it is running, such as CPU usage, memory usage, disk I/O, network statistics, and tens of other system-level metrics.

We are not going to cover Prometheus here, but we will see the exported metrics.

The DaemonSet should be used for this case to get accurate metrics from each node in the cluster. In the YAML, we used some specific configurations:

• hostNetwork & hostPID: Allow direct access to the node’s system metrics. This will help the Node Exporter collect accurate data.
• hostPort 9100: Exposes metrics on the same port across all nodes for Prometheus to scrape.
• hostPath /: Grants read-only access to the node’s filesystem so the exporter can collect metrics safely.

These configurations are not required for every DaemonSet; they are simply tailored for our exporter example.

The key difference from a Deployment is the use of kind: DaemonSet. Note that the replicas field is omitted because a DaemonSet automatically manages the number of Pods.

Now, confirm that each node has one Node Exporter Pod:

# Check the DaemonSet status
kubectl get daemonset node-exporter -n monitoring

# Find the Pods created by the DaemonSet
kubectl get pods -n monitoring -o wide -l app=node-exporter

Using kubectl port-forward, you can access the Node Exporter metrics from any node in your cluster:

kubectl port-forward -n monitoring daemonset/node-exporter 9100:9100 > /dev/null 2>&1