Updates and cleanups

This commit is contained in:
Lauri Võsandi 2023-08-29 09:29:36 +03:00
parent 4fa554da57
commit b11ac8bcae
42 changed files with 254 additions and 297 deletions

View File

@ -2,21 +2,8 @@
## Introduction
This is the Kubernetes manifests of services running on k-space.ee domains:
- [Authelia](https://auth.k-space.ee) for authentication
- [Drone.io](https://drone.k-space.ee) for building Docker images
- [Harbor](https://harbor.k-space.ee) for hosting Docker images
- [ArgoCD](https://argocd.k-space.ee) for deploying Kubernetes manifests and
Helm charts into the cluster
- [camtiler](https://cams.k-space.ee) for cameras
- [Longhorn Dashboard](https://longhorn.k-space.ee) for administering
Longhorn storage
- [Kubernetes Dashboard](https://kubernetes-dashboard.k-space.ee/) for read-only overview
of the Kubernetes cluster
- [Wildduck Webmail](https://webmail.k-space.ee/)
Most endpoints are protected by OIDC autentication or Authelia SSO middleware.
This is the Kubernetes manifests of services running on k-space.ee domains.
The applications are listed on https://auth2.k-space.ee for authenticated users.
## Cluster access
@ -27,7 +14,7 @@ General discussion is happening in the `#kube` Slack channel.
For bootstrap access obtain `/etc/kubernetes/admin.conf` from one of the master
nodes and place it under `~/.kube/config` on your machine.
Once Authelia is working, OIDC access for others can be enabled with
Once Passmower is working, OIDC access for others can be enabled with
running following on Kubernetes masters:
```bash
@ -120,7 +107,7 @@ Our self-hosted Kubernetes stack compared to AWS based deployments:
| AWS Route53 | Bind and RFC2136 | DNS records and Let's Encrypt DNS validation |
| AWS S3 | Minio Operator | Highly available object storage |
| AWS VPC | Calico | Overlay network |
| Dex | Authelia | ACL mapping and OIDC provider which integrates with GitHub/Samba |
| Dex | Passmower | ACL mapping and OIDC provider which integrates with GitHub/Samba |
| GitHub Actions | Drone | Build Docker images |
| GitHub | Gitea | Source code management, issue tracking |
| GitHub OAuth2 | Samba (Active Directory compatible) | Source of truth for authentication and authorization |
@ -129,7 +116,6 @@ Our self-hosted Kubernetes stack compared to AWS based deployments:
External dependencies running as classic virtual machines:
- Samba as Authelia's source of truth
- Bind as DNS server
@ -137,13 +123,13 @@ External dependencies running as classic virtual machines:
Deploy applications via [ArgoCD](https://argocd.k-space.ee)
We use Treafik with Authelia for Ingress.
We use Treafik with Passmower for Ingress.
Applications where possible and where applicable should use `Remote-User`
authentication. This prevents application exposure on public Internet.
Otherwise use OpenID Connect for authentication,
see Argo itself as an example how that is done.
See `kspace-camtiler/ingress.yml` for commented Ingress example.
See `camtiler/ingress.yml` for commented Ingress example.
Note that we do not use IngressRoute objects because they don't
support `external-dns` out of the box.

View File

@ -1,6 +1,7 @@
# Workflow
Most applications in our Kubernetes cluster are managed by ArgoCD.
Most notably operators are NOT managed by ArgoCD.
# Deployment
@ -20,8 +21,6 @@ kubectl -n argocd rollout restart statefulset/k6-argocd-application-controller
kubectl label -n argocd secret oidc-client-argocd-owner-secrets app.kubernetes.io/part-of=argocd
```
Note: Refer to Authelia README for OIDC secret setup
# Setting up Git secrets
@ -50,3 +49,32 @@ rm -fv id_ecdsa
Have Gitea admin reset password for user `argocd` and log in with that account.
Add the SSH key for user `argocd` from file `id_ecdsa.pub`.
Delete any other SSH keys associated with Gitea user `argocd`.
# Managing applications
To update apps:
```
for j in asterisk bind camtiler drone drone-execution etherpad freescout gitea grafana hackerspace nextcloud nyancat rosdump traefik wiki wildduck woodpecker; do
cat << EOF >> applications/$j.yaml
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: $j
namespace: argocd
spec:
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: $j
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: $j
syncPolicy: {}
EOF
done
find applications -name "*.yaml" -exec kubectl apply -n argocd -f {} \;
```

View File

@ -3,6 +3,7 @@ apiVersion: codemowers.io/v1alpha1
kind: OIDCGWClient
metadata:
name: argocd
namespace: argocd
spec:
displayName: Argo CD
uri: https://argocd.k-space.ee
@ -19,4 +20,18 @@ spec:
- openid
- profile
pkce: false
---
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
namespace: argocd
name: k-space.ee
spec:
clusterResourceWhitelist:
- group: '*'
kind: '*'
destinations:
- namespace: '*'
server: '*'
sourceRepos:
- '*'

View File

@ -1,14 +1,16 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: prometheus-operator
name: asterisk
namespace: argocd
spec:
project: default
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: prometheus-operator
path: asterisk
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: prometheus-operator
namespace: asterisk
syncPolicy: {}

View File

@ -1,17 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: authelia
namespace: argocd
spec:
project: default
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: authelia
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: authelia
syncPolicy:
syncOptions:
- CreateNamespace=true

View File

@ -1,17 +1,16 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: keel
name: bind
namespace: argocd
spec:
project: default
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: keel
path: bind
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: keel
syncPolicy:
syncOptions:
- CreateNamespace=true
namespace: bind
syncPolicy: {}

View File

@ -1,10 +1,11 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: camtiler
namespace: argocd
spec:
project: default
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: camtiler
@ -12,6 +13,4 @@ spec:
destination:
server: 'https://kubernetes.default.svc'
namespace: camtiler
syncPolicy:
syncOptions:
- CreateNamespace=true
syncPolicy: {}

View File

@ -1,10 +1,11 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: drone-execution
namespace: argocd
spec:
project: default
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: drone-execution
@ -12,6 +13,4 @@ spec:
destination:
server: 'https://kubernetes.default.svc'
namespace: drone-execution
syncPolicy:
syncOptions:
- CreateNamespace=true
syncPolicy: {}

View File

@ -1,10 +1,11 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: drone
namespace: argocd
spec:
project: default
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: drone
@ -12,6 +13,4 @@ spec:
destination:
server: 'https://kubernetes.default.svc'
namespace: drone
syncPolicy:
syncOptions:
- CreateNamespace=true
syncPolicy: {}

View File

@ -1,22 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: elastic-system
namespace: argocd
spec:
project: default
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: elastic-system
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: elastic-system
syncPolicy:
syncOptions:
- CreateNamespace=true
ignoreDifferences:
- group: admissionregistration.k8s.io
kind: ValidatingWebhookConfiguration
jqPathExpressions:
- '.webhooks[]?.clientConfig.caBundle'

View File

@ -1,10 +1,11 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: etherpad
namespace: argocd
spec:
project: default
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: etherpad
@ -12,6 +13,4 @@ spec:
destination:
server: 'https://kubernetes.default.svc'
namespace: etherpad
syncPolicy:
syncOptions:
- CreateNamespace=true
syncPolicy: {}

View File

@ -1,17 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: external-dns
namespace: argocd
spec:
project: default
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: external-dns
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: external-dns
syncPolicy:
syncOptions:
- CreateNamespace=true

View File

@ -1,17 +1,16 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: harbor
name: freescout
namespace: argocd
spec:
project: default
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: harbor
path: freescout
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: harbor
syncPolicy:
syncOptions:
- CreateNamespace=true
namespace: freescout
syncPolicy: {}

View File

@ -0,0 +1,16 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: gitea
namespace: argocd
spec:
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: gitea
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: gitea
syncPolicy: {}

View File

@ -1,10 +1,11 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: grafana
namespace: argocd
spec:
project: default
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: grafana
@ -12,6 +13,4 @@ spec:
destination:
server: 'https://kubernetes.default.svc'
namespace: grafana
syncPolicy:
syncOptions:
- CreateNamespace=true
syncPolicy: {}

View File

@ -0,0 +1,16 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: hackerspace
namespace: argocd
spec:
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: hackerspace
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: hackerspace
syncPolicy: {}

View File

@ -1,17 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: kubernetes-dashboard
namespace: argocd
spec:
project: default
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: kubernetes-dashboard
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: kubernetes-dashboard
syncPolicy:
syncOptions:
- CreateNamespace=true

View File

@ -1,17 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: logging
namespace: argocd
spec:
project: default
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: logging
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: logging
syncPolicy:
syncOptions:
- CreateNamespace=true

View File

@ -1,17 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: logmower
namespace: argocd
spec:
project: default
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: logmower
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: logmower
syncPolicy:
syncOptions:
- CreateNamespace=true

View File

@ -1,17 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: members
namespace: argocd
spec:
project: default
source:
repoURL: 'git@git.k-space.ee:k-space/kube-members.git'
path: .
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: members
syncPolicy:
syncOptions:
- CreateNamespace=true

View File

@ -1,22 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: metallb-system
namespace: argocd
spec:
project: default
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: metallb-system
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: metallb-system
syncPolicy:
syncOptions:
- CreateNamespace=true
ignoreDifferences:
- group: apiextensions.k8s.io
kind: CustomResourceDefinition
jqPathExpressions:
- '.spec.conversion.webhook.clientConfig.caBundle'

View File

@ -1,17 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: mysql-operator
namespace: argocd
spec:
project: default
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: mysql-operator
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: mysql-operator
syncPolicy:
syncOptions:
- CreateNamespace=true

View File

@ -0,0 +1,16 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: nextcloud
namespace: argocd
spec:
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: nextcloud
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: nextcloud
syncPolicy: {}

View File

@ -0,0 +1,16 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: nyancat
namespace: argocd
spec:
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: nyancat
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: nyancat
syncPolicy: {}

View File

@ -1,17 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: phpmyadmin
namespace: argocd
spec:
project: default
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: phpmyadmin
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: phpmyadmin
syncPolicy:
syncOptions:
- CreateNamespace=true

View File

@ -1,17 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: reloader
namespace: argocd
spec:
project: default
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: reloader
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: reloader
syncPolicy:
syncOptions:
- CreateNamespace=true

View File

@ -1,10 +1,11 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: rosdump
namespace: argocd
spec:
project: default
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: rosdump
@ -12,6 +13,4 @@ spec:
destination:
server: 'https://kubernetes.default.svc'
namespace: rosdump
syncPolicy:
syncOptions:
- CreateNamespace=true
syncPolicy: {}

View File

@ -0,0 +1,16 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: traefik
namespace: argocd
spec:
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: traefik
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: traefik
syncPolicy: {}

View File

@ -0,0 +1,16 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: wiki
namespace: argocd
spec:
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: wiki
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: wiki
syncPolicy: {}

View File

@ -1,10 +1,11 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: wildduck
namespace: argocd
spec:
project: default
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: wildduck
@ -12,6 +13,4 @@ spec:
destination:
server: 'https://kubernetes.default.svc'
namespace: wildduck
syncPolicy:
syncOptions:
- CreateNamespace=true
syncPolicy: {}

View File

@ -0,0 +1,16 @@
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: woodpecker
namespace: argocd
spec:
project: k-space.ee
source:
repoURL: 'git@git.k-space.ee:k-space/kube.git'
path: woodpecker
targetRevision: HEAD
destination:
server: 'https://kubernetes.default.svc'
namespace: woodpecker
syncPolicy: {}

11
asterisk/README.md Normal file
View File

@ -0,0 +1,11 @@
# Asterisk
Asterisk is used as
This application is managed by [ArgoCD](https://argocd.k-space.ee/applications/argocd/asterisk)
Should ArgoCD be down manifests here can be applied with:
```
kubectl apply -n asterisk -f application.yaml
```

View File

@ -9,9 +9,9 @@ spec:
type: LoadBalancer
externalTrafficPolicy: Local
selector:
app: asterisk
app: sip
ports:
- name: sip
- name: asterisk
protocol: UDP
port: 5060
---

View File

@ -1,9 +1,10 @@
# Bind setup
The Bind primary resides outside Kubernetes at `193.40.103.2` and
it's internally reachable via `172.20.0.2`
it's internally reachable via `172.20.0.2`.
Bind secondaries are hosted inside Kubernetes and load balanced behind `62.65.250.2`
Bind secondaries are hosted inside Kubernetes, load balanced behind `62.65.250.2` and
under normal circumstances managed by [ArgoCD](https://argocd.k-space.ee/applications/argocd/bind).
Ingresses and DNSEndpoints referring to `k-space.ee`, `kspace.ee`, `k6.ee`
are picked up automatically by `external-dns` and updated on primary.

9
freescout/README.md Normal file
View File

@ -0,0 +1,9 @@
# Freescout
This application is managed by [ArgoCD](https://argocd.k-space.ee/applications/argocd/freescout)
Should ArgoCD be down manifests here can be applied with:
```
kubectl apply -n freescout -f application.yaml
```

View File

@ -1,5 +1,9 @@
# Gitea
This application is managed by [ArgoCD](https://argocd.k-space.ee/applications/argocd/gitea)
Should ArgoCD be down manifests here can be applied with:
```
kubectl apply -n gitea -f application.yaml
```

View File

@ -61,6 +61,7 @@ metadata:
labels:
app.kubernetes.io/name: gitea
spec:
revisionHistoryLimit: 0
serviceName: gitea
replicas: 1
selector:

View File

@ -1,5 +1,9 @@
# Grafana
This application is managed by [ArgoCD](https://argocd.k-space.ee/applications/argocd/grafana)
Should ArgoCD be down manifests here can be applied with:
```
kubectl create namespace grafana
kubectl apply -n grafana -f application.yml
@ -8,8 +12,4 @@ kubectl apply -n grafana -f application.yml
## Grafana post deployment steps
* Configure Prometheus datasource with URL set to
`http://prometheus-operated.prometheus-operator.svc.cluster.local:9090`
* Configure Elasticsearch datasource with URL set to
`http://elasticsearch.elastic-system.svc.cluster.local`,
Time field name set to `timestamp` and
ElasticSearch version set to `7.10+`
`http://prometheus-operated.monitoring.svc.cluster.local:9090`

View File

@ -185,3 +185,11 @@ spec:
tls:
- hosts:
- "*.k-space.ee"
---
apiVersion: codemowers.cloud/v1beta1
kind: MysqlDatabaseClaim
metadata:
name: grafana
spec:
capacity: 1Gi
class: shared

View File

@ -1,21 +0,0 @@
To deploy components:
```
kubectl create namespace members-site
kubectl apply -n members-site -f doorboy.yml
```
# Doorboy
Set up Doorboy UID hashing salt:
```
kubectl create secret generic -n members-site doorboy-api \
--from-literal=DOORBOY_SECRET=hg2NmVlf6JcS3w237ZXn
kubectl create secret generic -n members-site doorboy-uid-hash-salt \
--from-literal=KDOORPI_UID_SALT=hkRXwLlQKmCJoy5qaahp
kubectl create secret generic -n members-site mongo-application-readwrite \
--from-literal=connectionString.standard=mongodb://kspace_accounting:dBDCS21pHlZAd5isyfBI@mongodb.infra.k-space.ee:27017/kspace_accounting?replicaSet=kspace-mongo-set
```

View File

@ -1,5 +1,7 @@
# Logging infrastructure
Note: This is deprecated since we moved to [Logmower stack](https://github.com/logmower)
## Background
Fluent Bit picks up the logs from Kubernetes workers and sends them to Graylog

View File

@ -1,4 +1,9 @@
Traefik Ingress Controller:
# Traefik Ingress Controller
This application is managed by
[ArgoCD](https://argocd.k-space.ee/applications/argocd/traefik)
Should ArgoCD be down manifests here can be applied with:
```
kubectl create namespace traefik