Home Lab: Lightweight Kubernetes Cluster

While I did have knowledge and some projects that used containers, I was on the skeptical side of the aisle when it came to implementing them in production environments. Working for a FinTech company made me realize how important containers really are, especially how big of an impact they can have, as long as you understand how to configure them with YAML files. Based on this experience, I decided to build a small cluster to experiment with containers at home, where the stakes were much lower to learn fast.

The Hardware

I wanted to keep this project as low cost as possible, while offering great computing power when clustered together. Luckily for me a few cluster boards were on the rise, like the TuringPi 2. In their words, “The Turing Pi 2 is a 4-node mini ITX cluster board with a built-in Ethernet switch that runs Raspberry Pi CM4 or Nvidia Jetson compute modules in any combination” (Kickstarter.com, 2022). I unfortunately had just missed their Kickstarter campaign, and I had to wait a few months until they had their official launch before purchasing a TuringPi 2, which also gave me some time to purchase 4 Raspberry Pi Compute Modules. At the time, the Raspberry Pi Foundation had a few supply chain issues that led to some serious price gouging by scalpers (Raspberry Pi, 2022). I also purchased a 1TB SSD and got a case from Amazon for the project, so it could sit nicely next to my desk once complete.

My patience was definitely tested, but it also gave me a chance to see how others in the community were building their clusters and what things they were running. Once I got my hands on the hardware, it was time to get hands on with running it.

The Software

Getting it running took quite a few tries, but in the process, I learned and put into practice some skills like flashing the compute modules that had eMMC flash memory and some that did not, all through the Turing Pi I/O. I ran into a few issues here, especially because to communicate with the boards from the TuringPi I/O, I needed a USB A to USB A cable. I thought I could outsmart it by using adapters I already had with a USB C to USB A or a USB C to USB C cable.I was wrong and it only caused delays.

Once I had flashed the compute modules with Ubuntu Server, I configured some SSH security roadblocks in case anyone tried to ssh into any of the nodes. I also changed some user permissions to not have to use the root user. Once I felt comfortable with the initial security configuration, it was time to get these modules to talk to one another.

I chose to follow the TuringPi docs on running the Lightweight Kubernetes (K3s) version on the cluster. I picked and assigned a master role to one of the nodes, and the rest were worker nodes. I also ran into a few DNS issues here, which I had to resolve by logging into my router and assigning permanent IP addresses to the nodes, as well as IP ranges. Understanding this step was crucial when I moved to a new apartment and I no longer had access to the router, the IP address range, and had to reconfigure everything with my new ISP. As Jeff Geerling says, “It was DNS” (Jeff Geerling, 2022).

Once I got the nodes configured to talk to one another, I continued with the documentation and installed Helm and Arkade, which was a straightforward process. With the package managers configured, I moved on to installing MetalLB for load balancing, configuring my SSD with Longhorn, and installing ArgoCD for a nice GUI to manage my containers.

With the full configuration complete, I was now free to configure some YAML files for my projects like my website, and an auto backup process to ensure I don’t lose my data in case the SSD stops working.

The Convenience

Configuring my projects into containers was fairly easy. I must confess, I did use AI tools like Codeium, Cursor and ChatGPT for help, but they also ran into some limits that began taking me in loops. At the end of the day, I still had to read through some documentation to get through some hiccups. Once I began getting the hang of it all, running other projects became easier and easier.

It is not just my projects, as I can also quickly spin up open source software to enhance my home lab experience and turn it into a great developer learning hub.

Conclusion

I understand that my configuration will not be running enterprise solutions, but working on this project allowed me to learn many things about containers and how they actually work under the hood on bare metal. I no longer need to imagine how it all looks; I instead go back to this experience to try to understand any roadblocks I come across. Furthermore, building this cluster allowed me to be curious at every step of the way, learning without being afraid of making mistakes. Now, I can take all the skills I have learned and use them at the enterprise level. At the end of the day, it is all computers talking to each other through text files.