Add (image) caching microservice #27

New Issue

lauri · 2023-08-30T07:12:18Z

lauri commented

2023-08-30 07:12:18 +00:00

The cached content could be placed in S3 bucket, if it exists there redirect to signed URL otherwise attempt to fetch from original URL and upload it to S3 bucket.

The microservice could keep in-memory lookup table of items it knows are in present on S3.

The cached content could be placed in S3 bucket, if it exists there redirect to signed URL otherwise attempt to fetch from original URL and upload it to S3 bucket. The microservice could keep in-memory lookup table of items it knows are in present on S3.

lauri commented

2023-08-30 07:14:58 +00:00

This setup could/should somehow support workers that preemptively get the content cached, maybe by watching events on the original bucket a'la:

with client.listen_bucket_notification(
    "my-bucket",
    prefix="my-prefix/",
    events=["s3:ObjectCreated:*", "s3:ObjectRemoved:*"],
) as events:
    for event in events:
        print(event)

Note that the content on the URL should be immutable, so best candidate would be to use hash of the content in the original URL

This setup could/should somehow support workers that preemptively get the content cached, maybe by watching events on the original bucket a'la: ``` with client.listen_bucket_notification( "my-bucket", prefix="my-prefix/", events=["s3:ObjectCreated:*", "s3:ObjectRemoved:*"], ) as events: for event in events: print(event) ``` Note that the content on the URL should be immutable, so best candidate would be to use hash of the content in the original URL

Sign in to join this conversation.