AWS Cost Explorer - A window into Cost Optimization

The AWS Cost Explorer, part of the AWS Cost Management Service is one of the best options if you want to manage your AWS costs and have precise awareness of the line items in your monthly bill. It allows you to envision your daily, monthly, or estimated spending with multiple filters, dashboards, and insight options.

AWS’s Pay-as-you-go model has attracted many large-scale enterprises to the cloud. The ability to provision infrastructure within minutes and keep it only for as long as you need provides a massive business value where an organisation does not need to invest as much capital as before to purchase, host, store, and manage big data centres.

It is no surprise that more than 74% of companies surveyed claim to already be in the process or have plans to migrate to the cloud.

However, with ease of operation comes additional responsibilities as well, when it’s easy to provision resources and infrastructure with a push of a button it becomes easy to be done with it and never terminate those resources.

This may be the reason why almost 62% of cloud users report “Feeling like they are paying more” for their cloud accounts.

The total price for this waste is astronomical. Jay Chapel, CEO of ParkMyCloud, a company that provides cost optimization solutions anticipates almost $21 Billion worth of wastage simply because of idle, underutilized resources or architectures that were created without a sound financial model in place.

So, how do you combat cloud usage wastage in your organization? The most important thing for a cost-optimized operation is key resources having insight and visibility into what’s going on in the account financially. That is where the AWS Cost Explorer comes in.

Cost explorer immediately gives a complete oversight of the cost incurred over the last six months. Costs are colour-coded and presented in a simple, easy-to-consume way that immediately lets the right stakeholders understand where the most cost is going and address whether that is as expected or if there is an anomaly at play that should be addressed immediately.

The above example explains one such scenario. A company that provides sandbox accounts to developers to test out code on AWS infrastructure started operations in May. The main usage on the account that was decided as part of the business ideas and cost analysis was to be majorly EC2 focused. However, within a month, the organization saw a sharp increase in cost.

Luckily, the AWS cost explorer presents all information about usage down to which service costs how much. A quick look into the cost increase lets Solution Architects know that the cost increase was because of services other than AWS EC2.

Using AWS Cost Explorers’ cost breakdown and visibility over what services were contributing to the invoice cost, Solution Architects quickly understood that the bulk of the extra cost was coming from an exponential increase in cloudtrail. Identifying an issue where cloud trails were being created for each sandbox session and addressing a fix allowed them to curb the cost by half for the month of august and down to zero for September.

Without the quick insight and accurate visibility of Cost Explorer, it might have taken a long effort to investigate and analyze usage patterns and access logs on an account that is active in multiple regions.

Similar to how this use case worked out, here are some of the leading causes of unoptimized cost patterns and how AWS Cost Explorer can help identify and provide visibility into the situation.

Autoscaling:

Autoscaling refers to systems created that can scale out or in depending on the load on the instances. Ideally, you should have instances that operate on the average network/resource usage and have autoscaling policies set up to kick in when the load increases.

Taking a closer look at the graph of the last 7 days, it seems like there is a constant cost to EC2 instances.

This might be a bad practice which means that there is just a fixed number of EC2 instances that are running the entire time. Ideally, you should have a lower number of instances which can autoscale out either horizontally or vertically depending on the usage. So that the cost graph would be on the lower end with spikes in usage throughout the day.

AWS STORAGE OPTIMIZATION:

There are multiple Storage options on AWS (read more about storage optimization in some of the blogs published here).

Glacier is an AWS Offering that provides around 90% cost savings as compared to Standard S3 and is a perfect fit for long-term archiving. Setting up lifecycle policies so that objects automatically move to Glacier is simple to set and is automatic going forward. You can read detailed instructions and more on our detailed blog.

Ideally, S3 costs should decrease and Glacier costs should increase to signify that objects are moving to a lower-cost storage solution. The following snapshot shows two services filtered, S3 and Glacier. However, we can see that the S3 cost is constant and there is no rising cost for Glacier which indicates that efficient cost optimization for storage is not being implemented.

VPC to Mitigate Data Transfer Costs:

A massive hidden cost of AWS is DataTransfer costs. Data transfer within regions and within the VPC is free while data traffic between regions is costly and will add considerable weight to your AWS Monthly bill.

The following dashboard makes use of the UsageType filter available in the AWS Cost explorer tool to see what costs are directly associated with data transfer.

Using the findings, you can identify if any region is communicating outside its zone and make changes to the architecture.

To understand the costs associated with the hundreds of AWS Services available, one must have visibility and an operational overview of the cost of running services. AWS Cost Explorer provides valuable tools and insights to see the cost of running infrastructure in AWS.

With the data available with filters and flexibility to view, it provides cloud admins with all the tools to improve the architecture and make their accounts cost-optimized and adhere to the founding pillar of the AWS Well-Architected Framework.