forked from k-space/kube
53 lines
3.9 KiB
Markdown
53 lines
3.9 KiB
Markdown
# k-space.ee infrastructure
|
|
Kubernetes manifests, Ansible [playbooks](ansible/README.md), and documentation for K-SPACE services.
|
|
|
|
<!-- TODO: Docs for adding to ArgoCD (auto-)sync -->
|
|
- Repo is deployed with [ArgoCD](https://argocd.k-space.ee). For `kubectl` access, see [CLUSTER.md](CLUSTER.md#kubectl).
|
|
- Debugging Kubernetes [on Wiki](https://wiki.k-space.ee/en/hosting/debugging-kubernetes)
|
|
- Need help? → [`#kube`](https://k-space-ee.slack.com/archives/C02EYV1NTM2)
|
|
|
|
Jump to docs: [inventory-app](hackerspace/README.md) / [cameras](camtiler/README.md) / [doors](https://wiki.k-space.ee/en/hosting/doors) / [list of apps](https://auth.k-space.ee) // [all infra](ansible/inventory.yml) / [network](https://wiki.k-space.ee/en/hosting/network/sensitive) / [retro](https://wiki.k-space.ee/en/hosting/retro) / [non-infra](https://wiki.k-space.ee)
|
|
|
|
Tip: Search the repo for `kind: xyz` for examples.
|
|
|
|
## Supporting services
|
|
- Build [Git](https://git.k-space.ee) repositories with [Woodpecker](https://woodpecker.k-space.ee).
|
|
- Passmower: Authz with `kind: OIDCClient` (or `kind: OIDCMiddlewareClient`[^authz]).
|
|
- Traefik[^nonginx]: Expose services with `kind: Service` + `kind: Ingress` (TLS and DNS **included**).
|
|
|
|
### Additional
|
|
- bind: Manage _additional_ DNS records with `kind: DNSEndpoint`.
|
|
- [Prometheus](https://wiki.k-space.ee/en/hosting/monitoring): Collect metrics with `kind: PodMonitor` (alerts with `kind: PrometheusRule`).
|
|
- [Slack bots](SLACK.md) and Kubernetes [CLUSTER.md](CLUSTER.md) itself.
|
|
<!-- TODO: Redirects: external-dns.alpha.kubernetes.io/hostname + in -extras.yaml: IngressRoute and Middleware -->
|
|
|
|
[^nonginx]: No nginx annotations! Use `kind: Ingress` instead. `IngressRoute` is not used as it doesn't support [`external-dns`](bind/README.md) out of the box.
|
|
[^authz]: Applications should use OpenID Connect (`kind: OIDCClient`) for authentication, whereever possible. If not possible, use `kind: OIDCMiddlewareClient` client, which will provide authentication via a Traefik middleware (`traefik.ingress.kubernetes.io/router.middlewares: passmower-proxmox@kubernetescrd`). Sometimes you might use both for extra security.
|
|
|
|
### Network
|
|
|
|
All nodes are in Infra VLAN 21. Routing is implemented with BGP, all nodes and the router make a full-mesh. Both Serice LB IPs and Pod IPs are advertised to the router. Router does NAT for outbound pod traffic.
|
|
See the [Calico installation](tigera-operator/application.yml) for Kube side and Routing / BGP in the router.
|
|
Static routes for 193.40.103.36/30 have been added in pve nodes to make them communicating with Passmower via Traefik more stable - otherwise packets coming back to the PVE are routed directly via VLAN 21 internal IPs by the worker nodes, breaking TCP.
|
|
|
|
<!-- Linked to by https://wiki.k-space.ee/e/en/hosting/storage -->
|
|
### Databases / -stores:
|
|
- KeyDB: `kind: KeydbClaim` (replaces Redis[^redisdead])
|
|
- Dragonfly: `kind: Dragonfly` (replaces Redis[^redisdead])
|
|
- Longhorn: `storageClassName: longhorn` (filesystem storage)
|
|
- Mongo[^mongoproblems]: `kind: MongoDBCommunity` (NAS* `inventory-mongodb`)
|
|
- Minio S3: `kind: MinioBucketClaim` with `class: dedicated` (NAS*: `class: external`)
|
|
- MariaDB*: search for `mysql`, `mariadb`[^mariadb] (replaces MySQL)
|
|
- Postgres*: hardcoded to [harbor/application.yml](harbor/application.yml)
|
|
|
|
\* External, hosted directly on [nas.k-space.ee](https://wiki.k-space.ee/en/hosting/storage)
|
|
|
|
[^mariadb]: As of 2024-07-30 used by auth, authelia, bitwarden, etherpad, freescout, git, grafana, nextcloud, wiki, woodpecker
|
|
|
|
[^redisdead]: Redis has been replaced as redis-operatori couldn't handle itself: didn't reconcile after reboots, master URI was empty, and clients complained about missing masters. ArgoCD still hosts its own Redis.
|
|
|
|
[^mongoproblems]: Mongo problems: Incompatible with rawfile csi (wiredtiger.wt corrupts), complicated resizing (PVCs from statefulset PVC template).
|
|
|
|
***
|
|
_This page is referenced by wiki [front page](https://wiki.k-space.ee) as **the** technical documentation for infra._
|