Here's why you need a cloud consultant (even for small microservices)

A client this week had implemented a document upload microservice.

Uploads were being made to S3 with a Lambda function.

Since the files were a little large (several dozen MBs on average), Lambda couldn't process them because of the 6MB limit.

He decided to use S3's multi part upload feature to bypass this limit.

So documents would be uploaded into 5MB chunks at a time.

When I saw this I immediately knew it didn't make sense.

S3's multi part upload is designed for much larger file sizes.

When you've got files in the GB range that's a good use case.

When you've got files larger than 5GB - S3's file size limit - multi part upload is what you need.

But for my client I simply refactored the code to generate a presigned URL and used that URL to upload files directly to S3.

Quick and easy - but doing the wring thing could have cost him in performance in the long run.

Have you ever come across either use case of having to use either Multipart uploads or presigned URLs for large files?

If you're not sure how well designed your cloud infrastructure don't hesitate to ask,

I'm here to help you!

-—

📬 You can subscribe to my newsletter for weekly posts on saving money in the cloud.

You can also check out my blog posts here: Medium