Go to file
Mehran Kholdi 0095c0e83a Implement metadata schema migration
Summary: Trying to add custom `fsType`s, new metadata fields need to be stored. We need a mechanism to migrate existing volume metadata.

Test Plan:
- Install the chart using an older image tag like `36fc480`
- Create and use a pvc
- Verify that the volume's metadata file, located at `/var/csi/rawfile/pvc-.../disk.meta` does not contain the `schema_version` field
- Upgrade the chart to use the image tag `feature-schema-migration`
- Wait until all node pods are upgraded
- Verify that the volume's metadata file contains the new `schema_version` field

Reviewers: bghadiri, h.marvi, mhyousefi, sina_rad

Reviewed By: bghadiri, h.marvi, mhyousefi

Differential Revision: https://phab.hamravesh.ir/D832
2020-07-18 09:41:24 +04:30
.ci [skip-e2e] Use slugified branch name as docker image tag 2020-07-13 21:18:09 +04:30
csi Autogen csi grpc interface 2020-04-24 00:08:36 +04:30
deploy/charts/rawfile-csi [skip-e2e] Use openebs's docker image in helm chart 2020-07-13 20:05:40 +04:30
orchestrator Use immutable tags for running tasks 2020-05-29 21:04:40 +04:30
protos Autogen csi grpc interface 2020-04-24 00:08:36 +04:30
templates Use immutable tags for running tasks 2020-05-29 21:04:40 +04:30
.dockerignore Configure CI 2020-04-24 19:35:37 +04:30
.gitignore Autogen python gitignore 2020-04-23 04:18:53 +04:30
.travis.yml Setup CI: Build, run e2e tests, and push images to docker hub 2020-07-11 23:14:13 +04:30
CODE_OF_CONDUCT.md chore(docs): add contributor guidelines 2020-06-13 06:08:25 +00:00
consts.py Use immutable tags for running tasks 2020-05-29 21:04:40 +04:30
declarative.py Hotfix: Make side effects idempotent 2020-06-14 05:10:32 +04:30
Dockerfile Use slim base image to reduce the resulting image size 2020-06-13 18:25:28 +04:30
GOVERNANCE.md chore(docs): add contributor guidelines 2020-06-13 06:08:25 +00:00
LICENSE Publish under Apache License 2.0 2020-06-12 02:31:02 +04:30
MAINTAINERS chore(docs): add contributor guidelines 2020-06-13 06:08:25 +00:00
metrics.py Expose inode-related metrics 2020-06-14 03:33:33 +04:30
rawfile_servicer.py Hotfix: Make side effects idempotent 2020-06-14 05:10:32 +04:30
rawfile_util.py Implement metadata schema migration 2020-07-18 09:41:24 +04:30
rawfile.py Implement metadata schema migration 2020-07-18 09:41:24 +04:30
README.md Add "Motivation" section to README 2020-07-11 19:03:23 +04:30
remote.py Implement metadata schema migration 2020-07-18 09:41:24 +04:30
requirements.in Implement basic metrics 2020-04-26 02:02:00 +04:30
requirements.txt Implement basic metrics 2020-04-26 02:02:00 +04:30
SECURITY.md chore(docs): add contributor guidelines 2020-06-13 06:08:25 +00:00
util.py Handle attaching loop devices instead of handing it to mount 2020-04-26 02:01:42 +04:30
volume_schema.py Implement metadata schema migration 2020-07-18 09:41:24 +04:30

FOSSA Status

RawFilePV

Kubernetes LocalPVs on Steroids

Install

helm install -n kube-system rawfile-csi ./deploy/charts/rawfile-csi/

Usage

Create a StorageClass with your desired options:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: my-sc
provisioner: rawfile.hamravesh.com
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true

Features

  • Direct I/O: Near-zero disk performance overhead
  • Dynamic provisioning
  • Enforced volume size limit
  • Thin provisioned
  • Access Modes
    • ReadWriteOnce
    • ReadOnlyMany
    • ReadWriteMany
  • Volume modes
    • Filesystem mode
    • Block mode
  • Volume metrics
  • Supports fsTypes
  • Online expansion: If fs supports it (e.g. ext4, btrfs)
  • Online shrinking: If fs supports it (e.g. btrfs)
  • Offline expansion/shrinking
  • Ephemeral inline volume
  • Snapshots: If the fs supports it (e.g. btrfs)

Motivation

One might have a couple of reasons to consider using node-based (rather than network-based) storage solutions:

  • Performance: Almost no network-based storage solution can keep up with baremetal disk performance in terms of IOPS/latency/throughput combined. And youd like to get the best out of the SSD youve got!
  • On-premise Environment: You might not be able to afford the cost of upgrading all your networking infrastructure, to get the best out of your network-based storage solution.
  • Complexity: Network-based solutions are distributed systems. And distributed systems are not easy! You might want to have a system that is easier to understand and to reason about. Also, with less complexity, you can fix unpredicted issues more easily.

Using node-based storage has come a long way since k8s was born. Right now, OpenEBSs hostPath makes it pretty easy to automatically provision hostPath PVs and use them in your workloads. There are known limitations though:

  • You cant monitor volume usage: There are hacky workarounds to run “du” regularly, but that could prove to be a performance killer, since it could put a lot of burden on your CPU and cause your filesystem cache to fill up. Not really good for a production workload.
  • You cant enforce hard limits on your volumes size: Again, you can hack your way around it, with the same caveats.
  • You are stuck with whatever filesystem your kubelet node is offering
  • You cant customize your filesystem:

All these issues stem from the same root cause: hostPath/LocalPVs are simple bind-mounts from the host filesystem into the pod.

The idea here is to use a single file as the block device, using Linuxs loop, and create a volume based on it. That way:

  • You can monitor volume usage by running df in O(1) since devices are mounted separately.
  • The size limit is enforced by the operating system, based on the backing file size.
  • Since volumes are backed by different files, each file could be formatted using different filesystems, and/or customized with different filesystem options.

License

FOSSA Status