Utilize Data

Utilizing Your Data

Now you have pulled together optimization data there different ways in which you can analyze and visualize it and use to make infrastructure optimization decisions

Visualization of Trusted Advisor data with Amazon QuickSight

You can visualize Trusted Advisor Data with TAO Dashboard. To deploy TAO Dashboard please follow TAO Dashboard deployment steps and specify organizational data collection bucket created in this lab as a source

Visualization of Compute Optimizer data with Amazon QuickSight

You can visualize Compute Optimizer Data with Compute Optimizer Dashboard. Compute Optimizer Dashboard also deliver Athena Tables and Views.

AWS Organization Data and The Cost Intelligence Dashboard

This video shows you how to use the Optimization Data Collection Lab to pull in AWS Organization data such as Account names and Tags into the Cost And Usage report so it can be used in the CID.

You can also use the below query to update your account_map table in athena to read from this data

New Account Map Query

Join with Cost and Usage Report

Example query on how you can connect your CUR to this Organizations data as a one off. In this query you will see the service costs split by account names.

Steps

Snapshots and AMIs

When a AMI gets created it takes a Snapshot of the volume. This is then needed to be kept in the account whilst the AMI is used. Once the AMI is released the Snapshot can no longer be used but it still incurs costs. Using this query we can identify Snapshots that have the ‘AMI Available’, those where the ‘AMI Removed’ and those that fall outside of this scope and are ‘NOT AMI’. Data must be collected and the crawler finished running before this query can be run.

Optimization Data Snapshots and AMIs Query

There is an option to add pricing data to this query. This assumes you have already run the accounts collector lambda.

Optimization Data Snapshots and AMIs with OD pricing data
Optimization Data Snapshots and AMIs with CUR data

EBS Volumes and Trusted Advisor Recommendations

Trusted advisor identifies idle and underutilized volumes. This query joins together the data so you can see what portion of your volumes are flagged. Data must be collected and the crawler finished running before this query can be run.

This section requires you to have the Inventory Module and the Trusted Advisor Module deployed.

Optimization Data EBS Volumes and Trusted Advisors Query

There is an option to add pricing data to this query.

Optimization Data EBS Volumes and Trusted Advisor with pricing data

The section below will bring in opportunities to move EBS volumes to gp3

EBS Volumes and Trusted Advisor moving to gp3

EBS Volumes and Trusted Advisor Recommendations

Trusted advisor identifies idle and underutilized volumes. This query joins together the data so you can see what portion of your volumes are flagged. Data must be collected and the crawler finished running before this query can be run.

This section requires you to have the Inventory Module and the Trusted Advisor Module deployed.

Optimization Data EBS Volumes and Trusted Advisors Query

AWS Budgets into Cost Dashboard

In these labs we have a couple of amazing cost dashboards that can be found here. If you would like to add your budget data into these dashboard please follow the below steps. Data must be collected and the crawler finished running before this query can be run. There is a saved query called aws_budgets created in the CloudFormation. This is used when connecting to dashboard.

Guide to add AWS Budgets into Cost Dashboard

AWS EBS Volumes and Snapshots

If you wish to see whats volumes have what snapshots attached to them from a holistic view then this query can combine these two data sources. This could provide information into which snapshots you could archive using Elastic Block Storage Snapshots Archive

Optimization Data Snapshots with EBS

If you wish to connect to your Cost and Usage report for snapshot costs please use the below:

Optimization Data Snapshots with EBS and CUR

ECS Chargeback

Report to show costs associated with ECS Tasks leveraging EC2 instances within a Cluster

Athena Configuration

AWS Transit Gateway Chargeback

AWS Transit Gateway data transfer cost billed at the central networking account is allocated proportionally to the end usage accounts. The proportion is calculated by connecting with AWS CloudWatch bytes in bytes out data at each Transit Gateway attachement level. The total central data transfer cost is calculated at the central networking account with Cost and Usage Report. The chargeback amount is the corresponding proportional cost of the total central amount.

Athena Configuration

RDS Graviton Eligibility

Graviton2 instances provide up to 35% performance improvement and up to 52% price/performance improvement for open source databases depending on database engine, version, and workload. You can easily determine what databases in your account can take advantage of Graviton using the RDS Graviton Eligibility query. This query will output all existing RDS databases, if they are eligible for graviton, if it requires any version upgrades in order to to migrate, the target graviton instance, as well as estimated savings.

This section requires you to have the RDS Module deployed.

Optimize RDS with Graviton