Optimizing Costs On AWS CloudWatch

Welcome back to The Cloud Economist!

Last week we explored how serverless services can reduce cloud costs, emphasizing that the benefits depend heavily on the use case.

I shared strategies for optimizing serverless costs, such as choosing the right provider, monitoring resource usage, optimizing code, and leveraging AWS cost management practices, as well as a few curated articles helping you with these strategies.

If you missed that you can grab it here.

In the question of the week, I asked “How can you minimize the impact of cold starts in a serverless function to reduce costs?”

To reduce AWS Lambda cold starts, use provisioned concurrency (most effective) to keep functions warm, optimize your code by minimizing dependencies and using lightweight packages, reduce function package size, and choose a runtime with fast startup times (like Node.js or Python).

This week I have a wealth of knowledge to share with you on optimizing and reducing costs in CloudWatch - a very widely used service.

This one is very important to start reducing your cloud costs!

Here are the best articles I’ve found on cloud cost savings this week, summarized.

Article 1

Why are my CloudWatch costs so high on AWS? by Thiago Salvatore

The team working at Alby faced unexpectedly high costs associated with CloudWatch.

After detailed analysis using AWS Cost Explorer and CloudWatch Metrics, the problem was traced to an excessive number of logs being sent to CloudWatch from their EKS cluster.

The high data processing costs were driven by the PutLogEvents operation, which uploads logs to log groups or streams. By identifying the specific log group responsible for this spike, they disabled the unnecessary logging, which immediately dropped the CloudWatch costs.

Here’s the 5 step process they used:

1. Use AWS Cost Explorer to analyze expenses

2. Investigate Data processing in CloudWatch

3. Analyze CloudWatch metrics (they searched for IncomingBytes metric)

4. Pinpoint the source of high data ingestion

5. Disable unnecessary logging

The root cause of the high cloudwatch costs: excessive logs being sent by an EKS cluster.

TLDR: Analyze and delete all unnecessary logs in CloudWatch.

Article 2

AWS Costs: An Important Service To Watch Out For To Reduce Costs by Uriel Bitton

AWS CloudWatch is a powerful and versatile logging service, but developers often overlook the cost implications.

The problem is that CloudWatch stores logs indefinitely by default.

CloudWatch charges for both data storage and ingestion, which can accumulate over time. To control these costs, you need to set explicit log retention periods or delete unneeded log groups to prevent excessive long-term storage fees.

At scale this can save you hundreds every month.

Here’s a guide on how to do this:

1. Navigate to Logs: In the left sidebar, click on Logs, then select Log Groups.

2. Select Log Group: Choose the log group you want to modify.

3. Edit Retention: Click on the Actions dropdown, then select Edit retention settings.

4. Set Retention Period: Choose your desired log retention period from the modal.

TLDR: change the retention period of cloudwatch logs to a shorter date and delete unnecessary logs to reduce CloudWatch monthly costs.

Article 3

Cost Optimization Strategies for AWS CloudWatch Logs and Metrics by Yash Mittal

Mittal shares some valuable strategies that you can implement around CloudWatch logs and metrics to reduce your monthly bill.

1. Optimize Log Group Structure: Organize logs by application, environment, and component to improve manageability and cost efficiency.

2. Set Appropriate Retention Policies: Adjust log retention periods based on compliance and business needs to avoid unnecessary storage costs.

3. Enable Log Data Compression: Compress logs to reduce storage size and costs.

4. Use Logs Insights Sparingly: Analyze only essential logs with CloudWatch Logs Insights to control query-related costs.

Check out the article for a deeper dive on these cost-saving strategies.

One Tip on Cloud Cost Savings

One company’s CloudWatch bill was costing them over $20 per month, here’s some valuable tips they use to reduce it significantly:

• Ensure that custom metrics are essential and remove any that are not needed.

• Consolidate CloudWatch alarms where possible to reduce the number of alarms.

• Use log filters to reduce the amount of data ingested into CloudWatch Logs.

• Set appropriate retention policies for logs to avoid unnecessary storage costs.

• Regularly delete logs that are no longer needed.

This Week’s Question

How does optimizing log ingestion patterns impact CloudWatch costs?

I encourage you to send in your answer, or you can tune in next week for the answer.

Until next week.

The Cloud Economist