gemail/gmkrestarter repository overview

⁠GMKRestarter is a Kubernetes operator designed to automatically restart pods after changing secrets

This operator is especially useful when using cert-manager to generate secrets with new certificates.

For each secret you need to set a restart-at label:

kubectl label secret <name_of_secret> restart-at=HH.MM 

For each such secret, the operator calculates the checksum and adds the annotation secret-checksum

Every change to the secret is monitored and if any changes are detected (the checksum changes) new checksum is written to the secret. Two annotations are added to each deployment where the secret is used:

• pending-restart-{name_of_secret}: epoch_time
• restart-at-{name_of_secret}: HH.MM

Deployments are also monitored and pods are restarted in due time according to the restart-at annotation.

The following annotation is added to the deployment:

• restart-ts: epoch_time_of_restart

and the two previous ones (pending-restart-{secret_name} and restart-at-{secret_name}) are deleted.

Thanks to this mechanism restart of the pods does not occur immediately after the change, but at the most convenient time specified by the restart-at label in the secret.

⁠Installation in kubernetes cluster

We assume that you know Kubernetes quite well and we will not go into unnecessary details in this description.

⁠Add service account and define role and role binding

apiVersion: v1
kind: ServiceAccount
metadata:
  name: gmkoperator
  namespace: name_of_namespace

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: gmkoperator-role
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "list", "watch","patch"]
  - apiGroups: [""]
    resources: ["namespaces"]
    verbs: ["get", "list","watch"]
  - apiGroups: ["apps"]
    resources: ["deployments","deployments/status"]
    verbs: ["get", "list", "watch", "patch"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: gmkoperator-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: gmkoperator-role
subjects:
  - kind: ServiceAccount
    name: gmkoperator 
    namespace: name_of_namespace

⁠Add deployment for gmkrestarter

Create deployment in namespace name_of_namespace. Of course, you also need to change the value of the TZ variable.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: gmkrestarter
spec:
  replicas: 1
  selector:
    matchLabels:
      app: gmkrestarter
  template:
    metadata:
      labels:
        app: gmkrestarter
    spec:
      serviceAccountName: gmkoperator
      containers:
        - name: gmkoperator
          image: gemail/gmkrestarter:1.0.4
          command: ["kopf", "run", "--standalone","--namespace=name_of_namespace", "/operator/handler.py"]
          env:
            - name: TZ
              value: "Europe/Warsaw"

⁠Monitoring secrets

For each secret we want to monitor, we add a label restart-at:

kubectl label secret <name_of_secret> restart-at=HH.MM