k-space/kube
10
1
Fork 3
Code Issues 32 Pull Requests Activity

Update README

Browse Source
This commit is contained in:
Lauri Võsandi 2022-12-24 21:01:49 +02:00
parent 0a40686c16
commit 3458cbd694
1 changed files with 29 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

54
README.md
View File

@ -155,7 +155,8 @@ these should be handled by `tls:` section in Ingress.
## Cluster formation ## Cluster formation
Create Ubuntu 20.04 VM-s on Proxmox with local storage. Created Ubuntu 22.04 VM-s on Proxmox with local storage.
Added some ARM64 workers by using Ubuntu 22.04 server on Raspberry Pi.
After machines have booted up and you can reach them via SSH: After machines have booted up and you can reach them via SSH:
@ -193,14 +194,8 @@ nameserver 8.8.8.8
EOF EOF
# Disable multipathd as Longhorn handles that itself # Disable multipathd as Longhorn handles that itself
systemctl mask multipathd systemctl mask multipathd snapd
systemctl disable multipathd systemctl disable --now multipathd snapd bluetooth ModemManager hciuart wpa_supplicant packagekit
systemctl stop multipathd
# Disable Snapcraft
systemctl mask snapd
systemctl disable snapd
systemctl stop snapd
# Permit root login # Permit root login
sed -i -e 's/PermitRootLogin no/PermitRootLogin without-password/' /etc/ssh/sshd_config sed -i -e 's/PermitRootLogin no/PermitRootLogin without-password/' /etc/ssh/sshd_config
@ -209,16 +204,15 @@ cat << EOF > /root/.ssh/authorized_keys
sk-ecdsa-sha2-nistp256@openssh.com AAAAInNrLWVjZHNhLXNoYTItbmlzdHAyNTZAb3BlbnNzaC5jb20AAAAIbmlzdHAyNTYAAABBBD4/e9SWYWYoNZMkkF+NirhbmHuUgjoCap42kAq0pLIXFwIqgVTCre03VPoChIwBClc8RspLKqr5W3j0fG8QwnQAAAAEc3NoOg== lauri@lauri-x13 sk-ecdsa-sha2-nistp256@openssh.com AAAAInNrLWVjZHNhLXNoYTItbmlzdHAyNTZAb3BlbnNzaC5jb20AAAAIbmlzdHAyNTYAAABBBD4/e9SWYWYoNZMkkF+NirhbmHuUgjoCap42kAq0pLIXFwIqgVTCre03VPoChIwBClc8RspLKqr5W3j0fG8QwnQAAAAEc3NoOg== lauri@lauri-x13
EOF EOF
userdel -f ubuntu userdel -f ubuntu
apt-get remove -yq cloud-init apt-get install -yqq linux-image-generic
apt-get remove -yq cloud-init linux-image-*-kvm
``` ```
Install packages, for Raspbian set `OS=Debian_11` Install packages:
```bash ```bash
OS=xUbuntu_20.04 OS=xUbuntu_22.04
VERSION=1.23 VERSION=1.24
cat <<EOF | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list cat <<EOF | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/ / deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/ /
EOF EOF
@ -232,11 +226,10 @@ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list
apt-get update apt-get update
apt-get install -yqq apt-transport-https curl cri-o cri-o-runc kubelet=1.23.5-00 kubectl=1.23.5-00 kubeadm=1.23.5-00 apt-get install -yqq apt-transport-https curl cri-o cri-o-runc kubelet=1.24.8-00 kubectl=1.24.8-00 kubeadm=1.24.8-00
sudo systemctl daemon-reload sudo systemctl daemon-reload
sudo systemctl enable crio --now sudo systemctl enable crio --now
apt-mark hold kubelet kubeadm kubectl apt-mark hold kubelet kubeadm kubectl
sed -i -e 's/unqualified-search-registries = .*/unqualified-search-registries = ["docker.io"]/' /etc/containers/registries.conf
``` ```
On master: On master:
@ -254,7 +247,7 @@ for j in $(seq 1 9); do
kubectl label nodes worker${j}.kube.k-space.ee node-role.kubernetes.io/worker='' kubectl label nodes worker${j}.kube.k-space.ee node-role.kubernetes.io/worker=''
done done
for j in $(seq 1 3); do for j in $(seq 1 4); do
kubectl taint nodes mon${j}.kube.k-space.ee dedicated=monitoring:NoSchedule kubectl taint nodes mon${j}.kube.k-space.ee dedicated=monitoring:NoSchedule
kubectl label nodes mon${j}.kube.k-space.ee dedicated=monitoring kubectl label nodes mon${j}.kube.k-space.ee dedicated=monitoring
done done
@ -265,15 +258,26 @@ for j in $(seq 1 4); do
done done
``` ```
On Raspberry Pi you need to take additonal steps:
* Manually enable cgroups by appending
`cgroup_memory=1 cgroup_enable=memory` to `/boot/cmdline.txt`,
* Disable swap with `swapoff -a; apt-get purge -y dphys-swapfile`
* For mounting Longhorn volumes on Rasbian install `open-iscsi`
For `arm64` nodes add suitable taint to prevent scheduling non-multiarch images on them: For `arm64` nodes add suitable taint to prevent scheduling non-multiarch images on them:
```bash ```bash
kubectl taint nodes worker9.kube.k-space.ee arch=arm64:NoSchedule kubectl taint nodes worker9.kube.k-space.ee arch=arm64:NoSchedule
``` ```
For door controllers:
```
for j in ground front back; do
kubectl taint nodes door-${j}.kube.k-space.ee dedicated=door:NoSchedule
kubectl label nodes door-${j}.kube.k-space.ee dedicated=door
kubectl taint nodes door-${j}.kube.k-space.ee arch=arm64:NoSchedule
done
```
To reduce wear on storage:
```
echo StandardOutput=null >> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
systemctl daemon-reload
systemctl restart kubelet
```