Cluster Update

My kubernetes cluster runs on a mix of RaspberryPI 4s, and x86(AMD64) machines, but because I built it originally with only the raspberrys, my control plane was on a PI

Well that was working fine for nearly 2 years, but the SD card on the control plane died, which wasn’t a good time

So I decided to what I always do, and go nuts

Spending spree

So I purchased a few things to tackle this issue

ArchLinux NerdCTL

Arch is a lot like Slackware and Gentoo, whilst it does have a package manager, you have to know what you want before things work

so NerdCTL has the ability to use buildkit to install stuff on multi-arch but that isn’t as easy as you would like and as such there are a few things that need installing to make it work

Steps

This will need YAY (well there are others but if you copy paste then its yay)

Mistakes were made

Well I’ve had a fun weekend, on one side I “finally” caught covid and as such my concentration levels were very low, now that doesn’t make up for what I did but at least I have 1/2 an out

What happened

I decided to do a little cleaning up of my kubernetes cluster, and decided that I should remove some deployments that weren’t in use, that was going fine, but then I decided to try and move some of the things that were more “central” to their correct namespace,
correct is a probably the wrong word, since I do kind of follow the principle of a namespace for a service rather than a namespace for a project (e.g. all of chewedfeed's stuff goes in the chewedfeed one, some things should be seperate if they are used by multiple things)

K3OS

K3OS is kinda self explanitory if you know what K3s is, if not then k3OS is Rancher’s k3s which is a kubernetes compatible version but a lot smaller

Why did I tell you that

Well now that bit is out of the way, I upgraded ubuntu on one of my raspberry pi 3’s, and at the same time upgraded k3s to the latest version, and that pi became almost impossible to use with kubernetes, pods failing all the time,
since I needed to wipe it anyway, and possibly replace with a pi4 (which are harder to find than rocking-horse-{redacted}) why not give k3os a shot, this is where the fun began

Issue

So you are already using RancherDesktop and NerdCTL from the betas congrats, but you want to use the multi-platform build

so you need to update Rancher and Nerd, well thats where the issue exists, Lima doesnt update NerdCTL automatically so you need to update your instances version

Fix

First you need to get into the instance

LIMA_HOME=/Users/<username>/Library/Application\ Support/rancher-desktop/lima \
	/Applications/Rancher\ Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl \
	shell 0 sudo --preserve-env=CONTAINERD_ADDRESS sh

Now that your in the instance, you need to update NerdCTL

Integrating Vault with 1Password in Kubernetes

I decided that I should probably get a real handle on my passwords and secrets in kubernetes, since kubernetes is not very good at keeping secrets

So I don’t generally take a helm chart or kube-schema at face value, this has more to do with I prefer to keep things in their own namespace rather than in “default”, because of this I tend to clone the chart first

Well I made Kubernetes cluster using Raspberry PIs, I largely followed the guide RasperryPI 4 Cluster

It has the following features

• 5 Raspberry Pi 4s, 4Gb RAM
• 1 DualCore x64, 16Gb RAM
• 1 DualCore x64, 4Gb RAM
• 4Tb storage distributed over the whole cluster
• ClusterCTRL, this is so power goes through the backplane rather than having 5 seperate power sources

Things that are using the cluster, well this site for one

This project is so that I can deploy projects into my cluster using github pipelines

now that sounds simle enough, except there aren’t any (from my limited research) deployment agents for kubernetes that work on arm

So whats my plan, well this system works thus

Method

There is a github action that is triggered in your pipeline, this talks to an orchestrator on k8sdeploy

the orchestrator then sends a message to a queue with a topic for an agent that has been registered by you for your cluster

Well I’ve decided that I should probably learn kubernetes, so I built a raspberry pi cluster

Home