Docker Fluent logging driver compatible piece for submitting Docker logs to Mongo
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
Madis Mägi b7b36548f4
gitea-kspace/mongo-logger/pipeline/head This commit looks good Details
Add Jenkinsfile
2 years ago
.flake8 Formatting fixes 2 years ago
.gitignore Add missing bits 2 years ago
.pre-commit-config.yaml Formatting fixes 2 years ago
Dockerfile Initial commit 2 years ago
Jenkinsfile Add Jenkinsfile 2 years ago
README.md Fix link in README 2 years ago
docker-compose.yml Add missing bits 2 years ago
logger.py Formatting fixes 2 years ago
overnode.yml Add missing bits 2 years ago
tailer.py Formatting fixes 2 years ago

README.md

Mongo logger

This is Mongo backed logging system that makes use of the fluentd driver found in Docker.

Running in production

To collect container logs on a Docker host populate .env with MONGO_URI and deploy docker-compose.yml on all Docker hosts.

In Docker host adjust /etc/docker/daemon.json

{
  "log-driver": "fluentd",
  "log-opts": {
    "fluentd-address": "127.0.0.1:24224",
    "fluentd-sub-second-precision": true
  }
}

Proceed to reload Docker daemon.

Enable syslog forwarding:

echo '*.* @127.0.0.1:5140' | tee /etc/rsyslog.d/fwd.conf
systemctl restart rsyslog

Dumping logs

To dump logs on console:

pip3 install motor termcolor
source .env
export MONGO_URI
python3 tailer.py

Why not Loki?

Loki is ridiculously complex to set up for HA setup.

Why not Fluentd?

After spending great deal of time working around bugs and design flaws in Fluentd we gave up. For reasoning see here