AWS 3rd party monitoring tool may be costing you an arm🦾 and leg 🦿

Photo from Unsplash by @moneyphotos

There are a lot of external monitoring tools for AWS like Lumigo, Epsagon, DataDog, New Relic, Sumologic, etc. They provide extra features in comparison to AWS cloudwatch service.

Background: Last year our team started using one of these tools to get more efficient and real-time monitoring alerts and detailed dashboards. During that time we put a lot of configuration and custom dashboards to segregate monitoring.

We focus a lot of energy on utilizing most of the features provided by the tool for which we were paying monthly. Because these features are the selling point for that tools.

The thing that most people miss: is “Exactly how the tool is getting the data from AWS so that it can utilize it in some way or the other”

To get the metrics data of the AWS environment like lambda invocations, Instance details, etc these tools utilized under the hood are cloudwatch API calls.

Cloud watch APIs are costly eg. GetMetricData, GetMetricStatistics is $0.01 per 1,000 requests.

After using these tools the AWS bill of a cloudwatch service will increase depending on the usage.

I recently checked how much the cloudwatch bill is because of the API calls from the monitoring tool.

And the reality was shocking almost 60% of cloudwatch cost is due to the API calls that the monitoring tool was making. Averaging around 2 million requests per day

The actual cost of 3rd party monitoring tools should be calculated as : Monitoring tool (or license) bill + cloudwatch(API request) bill

We optimized some configurations and removed some of the dashboards which are unutilized.

Manage to reduce the AWS bill by 25% which is a significant percentage.

Learning and takeaways:

• Monitoring tools will always have an impact on the cloudwatch bill
• Learn about the tool as much as possible, because tag lines like minimal/no configurations required can be misleading.
• Constantly revisit your monitoring tool configuration and dashboards periodically, since some have not been updated or unutilized. Also, aws might have created something similar under the AWS Management and governance services umbrella which might have the same features and less cost.
• Always do a value-to-cost analysis for where to do monitoring of new applications, there might be cases where AWS cloudwatch might be more cost-effective than 3rd party monitoring.
• Monitoring tools are required because they reduce a lot of effort on the operations side. The only concern is they should not increase the overall cost against the value they are generating.