Cloud native log shipping daemon written in Go

Go to file

rasmus 4573618440 watcher: mv utils to util.go		2022-11-11 12:17:17 +02:00
k8s/dev	k8s: remove NoExecute	2022-11-11 12:15:41 +02:00
pkg	watcher: mv utils to util.go	2022-11-11 12:17:17 +02:00
vendor	update deps	2022-11-09 23:45:58 +02:00
.drone.yml	add drone?	2022-11-07 00:26:12 +02:00
.gitignore	it is shipping and parsing	2022-11-06 03:43:52 +02:00
Dockerfile	don't copy . in Dockerfile	2022-11-09 21:55:36 +02:00
go.mod	update deps	2022-11-09 23:45:58 +02:00
go.sum	update deps	2022-11-09 23:45:58 +02:00
main.go	change metrics port to 8000	2022-11-10 00:03:59 +02:00
README.md	yet an another namespace swap	2022-11-09 23:57:47 +02:00
skaffold.yaml	add skaffold and fix bugs	2022-11-09 23:27:38 +02:00

README.md

logmower-shipper

Watches log directory for logs, and ships them to mongo.

Running: go run .
Building: go build .

Files

pkg/globals defines globals. This is used for Prometheus namespace, and for forwarding static global CLI configuration with less clutter.
main.go serves /metrics and runs watcher.go.
pkg/watcher main routine; initializes (including mongo.go) and watches for new log files.
- pkg/mongo provides statically safe and central tools to interact with the database.
- pkg/util provides additional utility functions.
pkg/file handles file lifecycle; watches files and tails them, streaming lines to pkg/lines.
pkg/lines processes lines and streams them to pkg/sender.
pkg/sender batches lines and ships them to mongo.

Skaffold

export NS=gitdbd-5cym5
kubectl apply -f k8s/dev/mongodb.yaml -n "$NS"
kubectl apply -f k8s/dev/prom.yaml -n "$NS"
skaffold dev --namespace "$NS"

hwatch kubectl get pods --namespace "$NS"