add project structure ascii

README.md

@@ -15,6 +15,32 @@ Watches log directory for logs, and ships them to mongo.
 4. `pkg/lines` processes lines and streams them to `pkg/sender`.
 5. `pkg/sender` batches lines and ships them to mongo.
 
+```txt
+┌──────────────────────────────────────────────────────────────┬─────────┐
+│ List directory      New files created                        ) watcher (
+│ │                    │                                       └─────────┤
+│ └► Simulate initial ─┴─► File watcher ──► Parse filename (k8s)         │
+│     create events                              ┌─◄───┘                 │
+│                                 Ignore files   │─◄─┐Namespace filters  │
+│ ┌─ Launch file collector ◄──── based on filter ┘─◄─┘Pod prefix filters │
+└─▼──────────────────────────────────────────────────────────────────────┘
+  │                                         ┌──────┐
+  │ Retry any downstream error with backoff ) file ( ┌──────────┬────────┐
+┌─▼─────────────────────────────────────────┴──────┤ │  Submit  ) sender (
+│┌┴─► mongo: get offset     create channels  ┌─────► │ to mongo └────────┤
+││      │                 + chan processors  │     │ ├────▲────┬─────────┘
+│└──┐───┘      ┌─────────────┘           └───┼─────► ) batcher (
+└───▼──────────▼──────────────────────────┐  │     │ ├────▲────┤
+┌───┴────┐ ┌───────┬────────────────────┐ │  └─────► │  queue  │
+) tailer ( ) lines (      combine       │ └────────┘ └────▲────┘
+└───▼────┘ ├───────┘   partial lines    │                 │
+    │      │            ▲       │       │   {possible processing plugins}
+    └──────► parse k8s  │       ▼       │                 ▲
+           │ line metadata ─────┴───────►─────────────────┘
+           │                            │
+           └────────────────────────────┘
+```
+
 ## Skaffold
 ```bash
 export NS=gitdbd-5cym5