k3s: Lightweight Kubernetes Distribution first experience

2019-03-06 about 998 words 5 min

Rancher introduced alsmost week ago k3s, a lightweight Kubernetes Distribution. In the YouTube video below you hear Shannon Williams and Darren Shepherd from Rancher talk about K3S, what it is, the usecases and demo of K3S.

https://youtu.be/5-5t672vFi4

k3s is a fully compliant, production-grade Kubernetes distribution that maintains an absolutely tiny footprint. Weighing in at less than 40 MB, it only needs 512 MB of RAM to run. This means it’s perfect for all kinds of computing that requires a minimal about of memory and space.

k3s is designed for Edge computing, IoT, CI, and ARM. Even if you’re working with something as small as a Raspberry Pi, k3s allows developers to utilize Kubernetes for production workloads. It simplifies operations, reducing the dependencies and steps needed to run a production Kubernetes cluster.

Installation is a breeze, considering that k3s is packaged as a single binary with less than 40 MB. However, security isn’t an afterthought, since TLS certificates are generated by default to make sure that all communication is secure by default.

Installation

As k3s is built for running on hardware like the Raspberry Pi,I thought it would be interesting to take a closer look at the product and install it on my Raspberry Pi.

My home setup

My home setup is not where it should be yet but two Pi’s is enough to start with. My plan is to add some more PI’s in the future so I will be able to have multiple nodes and when k3s also supports HA, I can also add an extra master.

Currently I use the following equipment for my k3s environment

Raspberry PI 3B+ (2 at the moment. 2 other will be added later)
16 GB SD card (2)
TP-Link TL-SG105 - Switch
Anker PowerPort+ 5 Binnen Zwart

So I started with preparing the Raspberry Pi’s. First I downloaded raspbian-stretch-lite from Raspbian.org . Then I used Etcher from Balena to flash the SD-cards with the image I downloaded. I used 2018-11-13-raspbian-stretch-lite.

Before I powered on the Raspberry Pi’s I mounted the sd-cards again and created a file in the root of the boot volume so that I could ssh to the PI’s

The following steps should be run on all Raspberry Pi’s that will be part of the cluster.

1
touch ssh

After that I powered on the Raspberry Pi’s and connected to them using ssh. To make it myself easy I assigned ip-addresses based on the mac-addresses of the Raspberries in my router. The default password for the PI user is raspberry.

1
ssh pi@192.168.2.100 #this is the ip-address of my Raspberry that will run as master

The next step was setting up the host name, changing the password and setting the ip configuration. The
changing of the host name and password can be done by raspi-config.

1
sudo raspi-config

After changing the host name, choose Finish and reboot the Pi. Next stepping was setting up the network configuration. The network configuration can be configured in /etc/dhcpcd.conf

1
2
3
4
profile static_eth0
static ip_address=192.168.2.100/24 # replace this with your node's ip-address
static routers=192.168.2.254 # replace this with the router address
static domain_name_servers=8.8.8.8 

Next step is turning off swap.

1
2
3
dphys-swapfile swapoff && \
dphys-swapfile uninstall && \
update-rc.d dphys-swapfile remove

Next step is adding the following line to /boot/cmdline.txt.
Don’t add any new lines! After saving the file reboot and login.

1
cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory

These step should only be run on the Pi that will have the role of master

1
2
3
curl -sfL https://get.k3s.io | sh -
# Check for Ready node, takes maybe 30 seconds
k3s kubectl get node

Default k3s doesn’t assign roles to the nodes and allows for pods to be scheduled on the master. If you want you can change that with the following commands

1
2
3
4
5
# label node as master
kubectl label node mymasternode kubernetes.io/role=master
kubectl label node mymasternode node-role.kubernetes.io/master=""
# exclude master from scheduling pods
kubectl taint nodes mymasternode node-role.kubernetes.io/master=effect:NoSchedule

On the node run the following commands

1
2
3
curl -fSL "https://github.com/rancher/k3s/releases/download/v0.1.0/k3s-armhf" \
  -o /usr/local/bin/k3s && \
chmod +x /usr/local/bin/k3s

After that you start the agent

1
2
# NODE_TOKEN comes from /var/lib/rancher/k3s/server/node-token on the master
sudo k3s agent --server https://myserver:6443 --token ${NODE_TOKEN} &

Optionally you can also set a label for the node. The commands should be run from the master node

1
2
kubectl label node mynode kubernetes.io/role=node
kubectl label node mynode node-role.kubernetes.io/node="" 

You are now ready to run a pod. As first pod to run I chose Nginx. Create a file at /home/pi/nginx-test.yaml with the following content

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-unprivileged-test
  namespace: default
spec:
  type: NodePort
  selector:
    app: nginx-unprivileged-test
  ports:
  - protocol: TCP
    nodePort: 30123
    port: 8080
    name: http
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-unprivileged-test
  namespace: default
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx-unprivileged-test
    spec:
      containers:
      - image: nginxinc/nginx-unprivileged
        name: nginx-unprivileged-test
        ports:
        - containerPort: 8080
          name: http
        livenessProbe:
          httpGet:
            path: /
            port: http
          initialDelaySeconds: 3
          periodSeconds: 3

Next step is deploying to the cluster

1
kubectl apply -f /home/pi/nginx-test.yaml

Since this is a NodePort service, k3s will open a port on the Raspberry Pi at 30123. On my local network, the Raspberry Pi is located on 192.168.2.100

A lot of more possibilities and stuff to find and try out. It is very easy to install Kubernetes and get a pod running. Looking forward to the upcoming releases.

If you want to hear more about k3s, you can attend the online meetup k3s: The Lightweight Kubernetes Distribution Built for the Edge. You can register here

As input for the post I used the following sites:

Another interesting blog is K3S sur un cluster de Raspberry Pi, Blog Zenika