k-space

mongo-logger

Docker Fluent logging driver compatible piece for submitting Docker logs to Mongo

You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Go to file

Madis Mägi b7b36548f4 gitea-kspace/mongo-logger/pipeline/head This commit looks good Details Add Jenkinsfile		2 years ago
.flake8	Formatting fixes	2 years ago
.gitignore	Add missing bits	2 years ago
.pre-commit-config.yaml	Formatting fixes	2 years ago
Dockerfile	Initial commit	2 years ago
Jenkinsfile	Add Jenkinsfile	2 years ago
README.md	Fix link in README	2 years ago
docker-compose.yml	Add missing bits	2 years ago
logger.py	Formatting fixes	2 years ago
overnode.yml	Add missing bits	2 years ago
tailer.py	Formatting fixes	2 years ago

Mongo logger

This is Mongo backed logging system that makes use of the fluentd driver found in Docker.

Running in production

To collect container logs on a Docker host populate .env with MONGO_URI and deploy docker-compose.yml on all Docker hosts.

In Docker host adjust /etc/docker/daemon.json

{
  "log-driver": "fluentd",
  "log-opts": {
    "fluentd-address": "127.0.0.1:24224",
    "fluentd-sub-second-precision": true
  }
}

Proceed to reload Docker daemon.

Enable syslog forwarding:

echo '*.* @127.0.0.1:5140' | tee /etc/rsyslog.d/fwd.conf
systemctl restart rsyslog

To dump logs on console:

pip3 install motor termcolor
source .env
export MONGO_URI
python3 tailer.py

Loki is ridiculously complex to set up for HA setup.

After spending great deal of time working around bugs and design flaws in Fluentd we gave up. For reasoning see here