Restore syslog ingestion support #1

Closed
opened 2022-09-17 05:18:19 +00:00 by lauri · 1 comment
Owner

Graylog/ElasticSearch was replaced with ZincSearch in c4208037e2

Filebeat ingests logs from Kubernetes workers and submits them to ZincSearch but there is currently no way to ingest logs using Syslog

The Mikrotik network gear is currently configured to send Syslog over TCP to 172.20.51.4 see here

ZincSearch suggests using syslog-ng to forward logs
https://www.syslog-ng.com/community/b/blog/posts/another-use-for-the-syslog-ng-elasticsearch-http-destination-zinc

Filebeat also has Syslog input support https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-syslog.html

Add Deployment into namespace logging for either syslog-ng or Filebeat to forward Syslog to Zinc

Graylog/ElasticSearch was replaced with ZincSearch in c4208037e2515b45b003d5141b2e0f58700942ff Filebeat ingests logs from Kubernetes workers and submits them to ZincSearch but there is currently no way to ingest logs using Syslog The Mikrotik network gear is currently configured to send Syslog over TCP to 172.20.51.4 see [here](https://git.k-space.ee/k-space/kube/src/commit/d1cb00ff83e264438e5b91af699b692de0ab3bd3/logging/application.yml#L43) ZincSearch suggests using syslog-ng to forward logs https://www.syslog-ng.com/community/b/blog/posts/another-use-for-the-syslog-ng-elasticsearch-http-destination-zinc Filebeat also has Syslog input support https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-syslog.html Add `Deployment` into namespace `logging` for either syslog-ng or Filebeat to forward Syslog to Zinc
dos4dev was assigned by lauri 2022-09-17 05:19:05 +00:00
Author
Owner

Zinc is still missing lots of features so this issue is now irrelevant.

Current plan is to implement homebrew logging system with Mongo.

Log shipping prototype is live in playground namespace
https://git.k-space.ee/k-space/log-shipper and it's being rewritten in Go at https://git.k-space.ee/k-space/logmower

Frontend is being developed at https://git.k-space.ee/eaas/log-viewer-microfrontend

Zinc is still missing lots of features so this issue is now irrelevant. Current plan is to implement homebrew logging system with Mongo. Log shipping prototype is live in `playground` namespace https://git.k-space.ee/k-space/log-shipper and it's being rewritten in Go at https://git.k-space.ee/k-space/logmower Frontend is being developed at https://git.k-space.ee/eaas/log-viewer-microfrontend
lauri closed this issue 2022-10-11 06:26:07 +00:00
Sign in to join this conversation.