migrate workers to infra vlan, use bgp for calico, use calico for lb service annoucements

This commit is contained in:
Erki Aas 2024-08-14 18:15:04 +03:00
parent 351f0ae746
commit 9bf5e2408a
7 changed files with 80 additions and 37 deletions

View File

@ -26,6 +26,13 @@ To find usage examples in this repository use
# For administrator # For administrator
Ingresses and DNSEndpoints referring to `k-space.ee`, `kspace.ee`, `k6.ee`
are picked up automatically by `external-dns` and updated on primary.
The primary triggers notification events to `172.21.53.{1..3}`
which are internally exposed IP-s of the secondaries.
# Secrets
To configure TSIG secrets: To configure TSIG secrets:
@ -61,7 +68,7 @@ zone "foobar.com" {
file "/var/lib/bind/db.foobar.com"; file "/var/lib/bind/db.foobar.com";
allow-update { !rejected; key foobar; }; allow-update { !rejected; key foobar; };
allow-transfer { !rejected; key readonly; key foobar; }; allow-transfer { !rejected; key readonly; key foobar; };
notify explicit; also-notify { 172.20.53.1; 172.20.53.2; 172.20.53.3; }; notify explicit; also-notify { 172.21.53.1; 172.21.53.2; 172.21.53.3; };
}; };
``` ```

View File

@ -116,7 +116,7 @@ metadata:
spec: spec:
type: LoadBalancer type: LoadBalancer
externalTrafficPolicy: Local externalTrafficPolicy: Local
loadBalancerIP: 172.20.53.1 loadBalancerIP: 172.21.53.1
selector: selector:
app: bind-secondary app: bind-secondary
statefulset.kubernetes.io/pod-name: bind-secondary-0 statefulset.kubernetes.io/pod-name: bind-secondary-0
@ -138,7 +138,7 @@ metadata:
spec: spec:
type: LoadBalancer type: LoadBalancer
externalTrafficPolicy: Local externalTrafficPolicy: Local
loadBalancerIP: 172.20.53.2 loadBalancerIP: 172.21.53.2
selector: selector:
app: bind-secondary app: bind-secondary
statefulset.kubernetes.io/pod-name: bind-secondary-1 statefulset.kubernetes.io/pod-name: bind-secondary-1
@ -160,7 +160,7 @@ metadata:
spec: spec:
type: LoadBalancer type: LoadBalancer
externalTrafficPolicy: Local externalTrafficPolicy: Local
loadBalancerIP: 172.20.53.3 loadBalancerIP: 172.21.53.3
selector: selector:
app: bind-secondary app: bind-secondary
statefulset.kubernetes.io/pod-name: bind-secondary-2 statefulset.kubernetes.io/pod-name: bind-secondary-2

View File

@ -150,7 +150,7 @@ metadata:
spec: spec:
type: LoadBalancer type: LoadBalancer
externalTrafficPolicy: Local externalTrafficPolicy: Local
loadBalancerIP: 172.20.51.4 loadBalancerIP: 172.21.51.4
ports: ports:
- name: filebeat-syslog - name: filebeat-syslog
port: 514 port: 514
@ -169,7 +169,7 @@ metadata:
spec: spec:
type: LoadBalancer type: LoadBalancer
externalTrafficPolicy: Local externalTrafficPolicy: Local
loadBalancerIP: 172.20.51.4 loadBalancerIP: 172.21.51.4
ports: ports:
- name: filebeat-syslog - name: filebeat-syslog
port: 514 port: 514

View File

@ -26,19 +26,7 @@ metadata:
namespace: metallb-system namespace: metallb-system
spec: spec:
addresses: addresses:
- 172.20.51.0/24 - 172.21.51.0/24
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: zoo
namespace: metallb-system
spec:
ipAddressPools:
- zoo
- bind-secondary-external
- bind-secondary-internal
- wildduck
--- ---
# Slice of public EEnet subnet using MetalLB L3 method # Slice of public EEnet subnet using MetalLB L3 method
apiVersion: metallb.io/v1beta1 apiVersion: metallb.io/v1beta1
@ -67,7 +55,7 @@ metadata:
namespace: metallb-system namespace: metallb-system
spec: spec:
addresses: addresses:
- 172.20.53.0/24 - 172.21.53.0/24
--- ---
apiVersion: metallb.io/v1beta1 apiVersion: metallb.io/v1beta1
kind: IPAddressPool kind: IPAddressPool
@ -99,13 +87,3 @@ spec:
passwordSecret: passwordSecret:
name: mikrotik-router name: mikrotik-router
namespace: metallb-system namespace: metallb-system
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: public
namespace: metallb-system
spec:
ipAddressPools:
- eenet
- elisa

View File

@ -33,9 +33,9 @@ spec:
static: static:
- 193.40.103.2 - 193.40.103.2
- 62.65.250.2 - 62.65.250.2
- 172.20.53.1 - 172.21.53.1
- 172.20.53.2 - 172.21.53.2
- 172.20.53.3 - 172.21.53.3
--- ---
apiVersion: monitoring.coreos.com/v1 apiVersion: monitoring.coreos.com/v1
kind: Probe kind: Probe

View File

@ -2,7 +2,7 @@
Calico implements Calico implements
[container network interface plugin](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/) [container network interface plugin](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/)
which enables pods to talk to eachother. which enables inter-pod network with BGP, also advertising Service LB IPs.
# For user # For user
@ -13,7 +13,8 @@ Nothing specific to point out, this is standard Kubernetes feature
Tigera operator was used to deploy Calico: Tigera operator was used to deploy Calico:
``` ```
curl https://raw.githubusercontent.com/projectcalico/calico/v3.28.0/manifests/tigera-operator.yaml -O curl https://raw.githubusercontent.com/projectcalico/calico/v3.28.1/manifests/tigera-operator.yaml -O
kubectl apply --server-side --force-conflicts -f tigera-operator.yaml kubectl apply --server-side --force-conflicts -f tigera-operator.yaml
kubectl apply -f application.yaml kubectl apply -f application.yaml
kubectl -n calico-system create secret generic bgp-secrets --from-literal=password=...
``` ```

View File

@ -12,7 +12,7 @@ spec:
- blockSize: 26 - blockSize: 26
cidr: 10.244.0.0/16 cidr: 10.244.0.0/16
encapsulation: VXLANCrossSubnet encapsulation: VXLANCrossSubnet
natOutgoing: Enabled natOutgoing: Disabled
nodeSelector: all() nodeSelector: all()
--- ---
# This section configures the Calico API server. # This section configures the Calico API server.
@ -22,3 +22,60 @@ kind: APIServer
metadata: metadata:
name: default name: default
spec: {} spec: {}
---
apiVersion: crd.projectcalico.org/v1
kind: BGPPeer
metadata:
name: mikrotik-router
spec:
peerIP: 172.21.255.254
asNumber: 64567
keepOriginalNextHop: true
password:
secretKeyRef:
name: bgp-secrets
key: password
---
apiVersion: crd.projectcalico.org/v1
kind: BGPConfiguration
metadata:
name: default
spec:
serviceLoadBalancerIPs:
- cidr: 172.21.51.4/32
- cidr: 172.21.53.1/32
- cidr: 172.21.53.2/32
- cidr: 172.21.53.3/32
- cidr: 193.40.103.36/32
- cidr: 193.40.103.37/32
- cidr: 193.40.103.38/32
- cidr: 193.40.103.39/32
- cidr: 62.65.250.36/32
- cidr: 62.65.250.37/32
- cidr: 62.65.250.2/32
- cidr: 193.40.103.25/32
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: secret-access
namespace: calico-system
rules:
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["bgp-secrets"]
verbs: ["watch", "list", "get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: secret-access
namespace: calico-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: secret-access
subjects:
- kind: ServiceAccount
name: calico-node
namespace: calico-system