Kubernetes cluster on a RPi (Part 2)

In the previous part of this post I stood up a basic 2 node k8s cluster using this excellent tutorial. I now need to learn how to use k8s and decide what I want and can use it for.

The last part of the tutorial is around validation of the cluster, and a good starting point to follow. As expected I followed the instructions and had 3 pods up and running in no time at all.

I then created a service to access, which created fine, however I hit an issue as I could not curl the ClusterIP from the main “master” node, but Ii could from the worker node that the pod was running on. After several rebuilds and troubleshooting later I discovered that I needed to run the command on all nodes:

sudo iptables -P FORWARD ACCEPT

With this I could then curl the ClusterIP and retrieve the web page from any node in the k8s cluster.

Persistent Volumes

I think the thing that has always alluded me apart from networking in k8s is how persistent storage is handled. Obviously containers are designed to be ephemeral, but coming from a sysadmin background I want storage and configuration files to be there. This is handled with persistent volumes in k8s.

For my k8s cluster I added a higher end flash drive to my k8s main node, one which is rated for both read and write, rather than just read. This hopefully will be temporary until I can invest in an M.2 NVMe caddy/card.

I created a basic NFS server on the main node, exporting the entire flash drive as an NFS mount. I then followed these instructions on how to deploy and use an NFS-client provisioner within my k8s setup.

To test this out I wanted to create a persistent volume on my NFS storage, and then use a persistent volume claim within the deploy script to mount this as a filesystem within the container. I thought the best way to try this would be to mount the NFS share into /usr/share/nginx/html on the containers and share webpages from there.

First I will create the persistent volume claim:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: nginx-webpage
  namespace: kube-verify
  annotations:
    volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Mi

Next I want to edit my deploy file to use the PVC and mount it inside the container at the specified mountpoint:

apiVersion: v1
kind: Pod
metadata:
  name: kube-verify
  namespace: kube-verify
  labels:
    app: kube-verify
spec:
  volumes:
    - name: task-pv-storage
      persistentVolumeClaim:
        claimName: nginx-webpage
  containers:
    - name: task-pv-container
      image: quay.io/clcollins/kube-verify:01
      ports:
        - containerPort: 8080
          name: "http-server"
      volumeMounts:
        - mountPath: "/usr/share/nginx/html"
          name: task-pv-storage

The pod starts up:

We create a service for the pod using the following:

apiVersion: v1
kind: Service
metadata:
  name: kube-verify
  namespace: kube-verify
spec:
  selector:
    app: kube-verify
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080

And we can connect to the ClusterIP and see the index.html webpage which has been created on the NFS exported flash drive:

Just to prove a point, I installed firefox on the main k8s node and ran it over a X11 session and can browse the webpage:

I can then edit the index.html file directly on the mounted flashdrive

and the change is reflected through the webpage on an nginx pod on my k8s cluster running on my worker node.

In the next part I will expose this with an ingress controller.