Lab complete!
Now that you have completed this lab, make sure to update your Well-Architected review if you have implemented these changes in your workload.
Click here to access the Well-Architected Tool
The FAQ for this dashboard is here.
The Data Transfer Dashboard is an interactive, customizable and accessible QuickSight dashboard to help customers gain insights into their data transfer. It will analyze any data transfer that incurs a cost such as outbound internet and regional data transfer from all services.
This dashboard contains data transfer breakdowns with the following visuals:
Ensure you have requested access to the Cost Intelligence template here.
The data source for the dashboard will be an Athena view of your existing Cost and Usage Report (CUR).
Login via SSO in your Cost Optimization account, go into the Athena console:
Create the Data Transfer view by modifying the following code, and executing it in Athena or using aws cli:
The Athena Views are updated to reflect any additions in the cost and usage report. If you created your dashboard prior to July 26, 2020 you will want to update to the latest views.
We will now create the data sets in QuickSight from the Athena views.
Go to the QuickSight service homepage
Click on the Datasets and then click on New dataset
Click Athena
Enter a data source name of DataTransfer_Cost_Dashboard and click Create data source:
Select the costmaster database, and the data_transfer_view table, click Edit/Preview data:
If you have large data for data transfer in CUR, we do NOT recommend using SPICE when setting up your data set in QuickSight, you can quickly use up the 10GB/user allocation and start to incur additional charges. Please use your judgment before enabling it.
Select SPICE to change your Query mode:
Hover over linked_account_id to get the drop down arrow and click on it, then hover over Change data type then select # Int:
Repeat step 7 for:
Hover over region to get the drop down arrow and click on it, then hover over Change data type then select # String
Hover over blended_cost to get the drop down arrow and click on it, then hover over Change data type then select # Decimal:
Ensure the following fields are all # Decimal, repeat step 10 if necessary for:
Select Save:
Select the data_transfer_view Data Set:
Click Schedule refresh:
Click Create:
Enter a schedule, it needs to be refreshed daily, and click Create:
Click Cancel to exit:
Click the x in the top corner:
You now have your data set setup ready to create a visualization.
We will now use the CLI to create the dashboard from the Data Transfer Cost and Usage Analysis Dashboard template, then create an Analysis you can customize and modify in the next step.
If you have not requested access, go to this we page to request access to the template: Template Access
Edit the following command, replacing AccountID with your account ID, and region with the region you are working in, then using the CLI list the QuickSight datasets and copy the Name and Arn for the dataset: data_transfer_view:
aws quicksight list-data-sets --aws-account-id (AccountID) --region (region)
{
"Arn": "arn:aws:quicksight:us-east-1:<your account id>:dataset/fc0cf1eb-173b-4aca-93b6-f58784637732",
"DataSetId": "fc0cf1eb-173b-4aca-93b6-f58784637732",
"Name": "data_transfer_view",
"CreatedTime": "2020-08-09T23:06:41.666000-04:00",
"LastUpdatedTime": "2020-08-11T23:15:35.438000-04:00",
"ImportMode": "SPICE"
}
Get your users Arn by editing the following command, replacing AccountID with your account ID, and region with the region you are working in, then using the CLI run the command:
aws quicksight list-users --aws-account-id (AccountID) --namespace default --region (region)
{
"Arn": "arn:aws:quicksight:us-east-1:<your account id>:user/default/<your user>",
"UserName": "<your user>",
"Email": "<your user email>",
"Role": "ADMIN",
"Active": true,
"PrincipalId": "<principal id>"
}
Create a local file create-data-transfer-dashboard.json with the text below, replace the values (Account ID) with your account ID on line 2 and line 25, (User ARN) with your user ARN on line 7, and (DataTransfer view Dataset ARN) with your dataset ARN on line 25:
{
"AwsAccountId": "(Account ID)",
"DashboardId": "data_transfer_cost_analysis_dashboard",
"Name": "DataTransfer Cost Analysis Dashboard",
"Permissions": [
{
"Principal": "(User ARN)",
"Actions": [
"quicksight:DescribeDashboard",
"quicksight:ListDashboardVersions",
"quicksight:UpdateDashboardPermissions",
"quicksight:QueryDashboard",
"quicksight:UpdateDashboard",
"quicksight:DeleteDashboard",
"quicksight:DescribeDashboardPermissions",
"quicksight:UpdateDashboardPublishedVersion"
]
}
],
"SourceEntity": {
"SourceTemplate": {
"DataSetReferences": [
{
"DataSetPlaceholder": "data_transfer_view",
"DataSetArn": "arn:aws:quicksight:us-east-1:(Account ID):dataset/(DataTransfer view Dataset ARN)"
}
],
"Arn": "arn:aws:quicksight:us-east-1:869004330191:template/data-transfer-cost-analysis-template"
}
},
"VersionDescription": "1"
}
To create the dashboard from the template, edit then run the following command, replacing (region) with the region you are working in, and you should receive a 202 response:
aws quicksight create-dashboard --cli-input-json file://create-data-transfer-dashboard.json --region (region)
After a few minutes the dashboard will become available in QuickSight under All dashboard, click on the Dashboard name:
Click Share, click Share dashboard:,
Click Manage dashboard access:
Add the required users, or share with all users, ensure you check Save as for each user, then click the x to close the window:
Click Save as:
Enter an Analysis name and click Create:
Perform steps 11 and 12 above to create additional analyses for other teams, this will allow each team to have their own customizable analysis.
You will now have an analysis created from the template that you can edit and modify:
To view data transfer charts for desired period change From and To date filters. Dashboard will automatically refresh upon date changes.
You have successfully created the analysis from a template. For a detailed description of the dashboard read the FAQ
Now that you have completed this lab, make sure to update your Well-Architected review if you have implemented these changes in your workload.
Click here to access the Well-Architected Tool