Add (image) caching microservice #27

New Issue

lauri · 2023-08-30T07:12:18Z

lauri commented

2023-08-30 07:12:18 +00:00

The cached content could be placed in S3 bucket, if it exists there redirect to signed URL otherwise attempt to fetch from original URL and upload it to S3 bucket.

The microservice could keep in-memory lookup table of items it knows are in present on S3.

The cached content could be placed in S3 bucket, if it exists there redirect to signed URL otherwise attempt to fetch from original URL and upload it to S3 bucket. The microservice could keep in-memory lookup table of items it knows are in present on S3.

lauri commented

2023-08-30 07:14:58 +00:00

This setup could/should somehow support workers that preemptively get the content cached, maybe by watching events on the original bucket a'la:

with client.listen_bucket_notification(
    "my-bucket",
    prefix="my-prefix/",
    events=["s3:ObjectCreated:*", "s3:ObjectRemoved:*"],
) as events:
    for event in events:
        print(event)

Note that the content on the URL should be immutable, so best candidate would be to use hash of the content in the original URL

This setup could/should somehow support workers that preemptively get the content cached, maybe by watching events on the original bucket a'la: ``` with client.listen_bucket_notification( "my-bucket", prefix="my-prefix/", events=["s3:ObjectCreated:*", "s3:ObjectRemoved:*"], ) as events: for event in events: print(event) ``` Note that the content on the URL should be immutable, so best candidate would be to use hash of the content in the original URL

rasmus added the

new app

label 2024-08-16 21:38:21 +00:00

Sign in to join this conversation.

No Label

new app

greenlit

No Milestone

No project

No Assignees

1 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: k-space/kube#27