diff --git a/README.md b/README.md index 316e041..794d867 100644 --- a/README.md +++ b/README.md @@ -155,7 +155,8 @@ these should be handled by `tls:` section in Ingress. ## Cluster formation -Create Ubuntu 20.04 VM-s on Proxmox with local storage. +Created Ubuntu 22.04 VM-s on Proxmox with local storage. +Added some ARM64 workers by using Ubuntu 22.04 server on Raspberry Pi. After machines have booted up and you can reach them via SSH: @@ -193,14 +194,8 @@ nameserver 8.8.8.8 EOF # Disable multipathd as Longhorn handles that itself -systemctl mask multipathd -systemctl disable multipathd -systemctl stop multipathd - -# Disable Snapcraft -systemctl mask snapd -systemctl disable snapd -systemctl stop snapd +systemctl mask multipathd snapd +systemctl disable --now multipathd snapd bluetooth ModemManager hciuart wpa_supplicant packagekit # Permit root login sed -i -e 's/PermitRootLogin no/PermitRootLogin without-password/' /etc/ssh/sshd_config @@ -209,16 +204,15 @@ cat << EOF > /root/.ssh/authorized_keys sk-ecdsa-sha2-nistp256@openssh.com AAAAInNrLWVjZHNhLXNoYTItbmlzdHAyNTZAb3BlbnNzaC5jb20AAAAIbmlzdHAyNTYAAABBBD4/e9SWYWYoNZMkkF+NirhbmHuUgjoCap42kAq0pLIXFwIqgVTCre03VPoChIwBClc8RspLKqr5W3j0fG8QwnQAAAAEc3NoOg== lauri@lauri-x13 EOF userdel -f ubuntu -apt-get remove -yq cloud-init - - +apt-get install -yqq linux-image-generic +apt-get remove -yq cloud-init linux-image-*-kvm ``` -Install packages, for Raspbian set `OS=Debian_11` +Install packages: ```bash -OS=xUbuntu_20.04 -VERSION=1.23 +OS=xUbuntu_22.04 +VERSION=1.24 cat < /etc/apt/sources.list.d/kubernetes.list apt-get update -apt-get install -yqq apt-transport-https curl cri-o cri-o-runc kubelet=1.23.5-00 kubectl=1.23.5-00 kubeadm=1.23.5-00 +apt-get install -yqq apt-transport-https curl cri-o cri-o-runc kubelet=1.24.8-00 kubectl=1.24.8-00 kubeadm=1.24.8-00 sudo systemctl daemon-reload sudo systemctl enable crio --now apt-mark hold kubelet kubeadm kubectl -sed -i -e 's/unqualified-search-registries = .*/unqualified-search-registries = ["docker.io"]/' /etc/containers/registries.conf ``` On master: @@ -254,7 +247,7 @@ for j in $(seq 1 9); do kubectl label nodes worker${j}.kube.k-space.ee node-role.kubernetes.io/worker='' done -for j in $(seq 1 3); do +for j in $(seq 1 4); do kubectl taint nodes mon${j}.kube.k-space.ee dedicated=monitoring:NoSchedule kubectl label nodes mon${j}.kube.k-space.ee dedicated=monitoring done @@ -265,15 +258,26 @@ for j in $(seq 1 4); do done ``` -On Raspberry Pi you need to take additonal steps: - -* Manually enable cgroups by appending - `cgroup_memory=1 cgroup_enable=memory` to `/boot/cmdline.txt`, -* Disable swap with `swapoff -a; apt-get purge -y dphys-swapfile` -* For mounting Longhorn volumes on Rasbian install `open-iscsi` - For `arm64` nodes add suitable taint to prevent scheduling non-multiarch images on them: ```bash kubectl taint nodes worker9.kube.k-space.ee arch=arm64:NoSchedule ``` + +For door controllers: + +``` +for j in ground front back; do + kubectl taint nodes door-${j}.kube.k-space.ee dedicated=door:NoSchedule + kubectl label nodes door-${j}.kube.k-space.ee dedicated=door + kubectl taint nodes door-${j}.kube.k-space.ee arch=arm64:NoSchedule +done +``` + +To reduce wear on storage: + +``` +echo StandardOutput=null >> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf +systemctl daemon-reload +systemctl restart kubelet +```