Commit Graph

82 Commits

Author SHA1 Message Date
Mehran Kholdi
be2cd1b72c Expose inode-related metrics
Test Plan: - Check `:9100/metrics` for existence of inode-related metrics

Reviewers: h.marvi, mhyousefi

Reviewed By: h.marvi

Differential Revision: https://phab.hamravesh.ir/D816
2020-06-14 03:33:33 +04:30
Mehran Kholdi
8e5cb5de78 Rewrite: Change the way we're exposing volume metrics
Summary:
Formerly, we were updating the metrics every 15 seconds. We were facing a couple of issues doing it manually:

- Outdated metrics in case of a one-time crash
- Metrics getting exposed for deleted PVs

Instead of fixing the bugs, I preferred to do it the right way. As per `python-prometheus` docs:

>  Sometimes it is not possible to directly instrument code, as it is not in your control. This requires you to proxy metrics from other systems. To do so you need to create a custom collector...

Test Plan:
- Deploy on a cluster with existing rawfile PVs
- Send request to `:9100/metrics` and assert that metrics are exposed
- Delete a PV, and assert that its metrics disappear

Reviewers: h.marvi, bghadiri, sina_rad, mhyousefi

Reviewed By: h.marvi, bghadiri, sina_rad

Differential Revision: https://phab.hamravesh.ir/D815
2020-06-14 03:31:21 +04:30
Mehran Kholdi
523ccd510f Use slim base image to reduce the resulting image size 2020-06-13 18:25:28 +04:30
kmova
fd7db09a0c chore(docs): add contributor guidelines
Signed-off-by: kmova <kiran.mova@mayadata.io>
2020-06-13 06:08:25 +00:00
Mehran Kholdi
5397774ce6 Publish under Apache License 2.0 2020-06-12 02:31:02 +04:30
Mehran Kholdi
b2c4b77911 Implement STAGE_UNSTAGE_VOLUME capability
Summary: Before this, we directly mounted the the rawfile on the mountpoint. In this revision the `STAGE_UNSTAGE_VOLUME` capability is implemented, meaning that the volume is first mounted to a staging path, and then `bind`-mounted to the actual path. This way we can free up loopback devices when they are not needed.

Test Plan:
- Create a pvc, and use it inside a pod
- Run `losetup -l` on the node running the pod, and assert the creation of a loop device
- Delete the pod, but not the pvc
- Run `losetup -l` on the same node, and assert the removal of the loop device

Reviewers: h.marvi, bghadiri

Differential Revision: https://phab.hamravesh.ir/D806
2020-05-31 13:23:10 +04:30
Mehran Kholdi
f557aef8ec Pin base image version
We rely on the output format of some command line tools, and hence
need to know we're getting the same version of them.
2020-05-31 13:20:15 +04:30
Mehran Kholdi
46bd3565a0 Use immutable tags for running tasks
This way we can run tasks with `IfNotPresent` and be sure it works as intended.
2020-05-29 21:04:40 +04:30
Mehran Kholdi
0ab1e393fc Fix typo 2020-05-29 21:04:20 +04:30
Mehran Kholdi
a51305e878 Push official images to dockerhub 2020-05-29 21:04:20 +04:30
Mehran Kholdi
b49f8304f5 Add component label to controller service 2020-05-03 19:28:55 +04:30
Mehran Kholdi
87dd918a74 Update ServiceMonitor to chart 2020-04-28 22:54:38 +04:30
Mehran Kholdi
a6a859fc0e Add installation and usage instructions to readme 2020-04-28 22:54:38 +04:30
Mehran Kholdi
a2fdcb9756 Deploy controller using statefulset instead of deployment
So that no two controllers are running simultaneously.
2020-04-28 22:54:38 +04:30
Mehran Kholdi
420b0f4ff7 Create helm chart 2020-04-28 22:54:38 +04:30
Mehran Kholdi
336f72a3c9 Cleanup manifests 2020-04-26 02:02:00 +04:30
Mehran Kholdi
974ad321a9 Hardcode block size as 512 instead of using blksize!
See: https://stackoverflow.com/a/3212102/585678
See: https://stackoverflow.com/a/12952830/585678
2020-04-26 02:02:00 +04:30
Mehran Kholdi
9deaefc203 Implement basic metrics 2020-04-26 02:02:00 +04:30
Mehran Kholdi
c9da83b19b Setup loopback devices using Direct I/O 2020-04-26 02:01:42 +04:30
Mehran Kholdi
dcc2210b3c Handle attaching loop devices instead of handing it to mount
* So that we can pass extra loop options like Direct I/O
* To workaround an issue with loop device files not getting
automatically created
2020-04-26 02:01:42 +04:30
Mehran Kholdi
c500df8ca2 Delete task pods upon successful completion 2020-04-26 01:37:23 +04:30
Mehran Kholdi
4ec6acf731 Store metadata file 2020-04-26 01:37:23 +04:30
Mehran Kholdi
373c43fa0e Add simple validations to CreateVolume request 2020-04-26 01:37:23 +04:30
Mehran Kholdi
9de82a9b47 Allocate new volumes remotely 2020-04-26 01:37:23 +04:30
Mehran Kholdi
73a618228b Temp: Disable python buffering
So that we can debug based on stdout
2020-04-26 01:37:23 +04:30
Mehran Kholdi
da2856237b Switch from alpine base to debian
So that python requirements could be installed from wheels.
2020-04-26 01:37:23 +04:30
Mehran Kholdi
0dec2f269b Add README 2020-04-26 01:37:22 +04:30
Mehran Kholdi
18abfa1ec4 Configure CI 2020-04-24 19:35:37 +04:30
Mehran Kholdi
6f23dbe067 Initial PoC of the csi driver 2020-04-24 19:35:37 +04:30
Mehran Kholdi
98139d428e Autogen csi grpc interface
Proto taken from:
https://github.com/container-storage-interface/spec/blob/master/csi.proto

Compiled using:
`python -m grpc_tools.protoc -I./protos --python_out=./csi/ --grpc_python_out=./csi/ protos/csi.proto`
2020-04-24 00:08:36 +04:30
Mehran Kholdi
6ac237abb8 Autogen python gitignore 2020-04-23 04:18:53 +04:30
Mehran Kholdi
7df7dc3ee5 Initial commit 2020-04-23 04:13:33 +04:30