CUDOS Dashboard

Create Athena Views

The data source for the dashboard will be an Athena view of your existing Cost and Usage Report (CUR). The default dashboard assumes you have both Savings Plans and Reserved Instances, if not you will need to create the alternate views.

1. Login via SSO in your Cost Optimization account, go into the Athena console:

2. Modify and run the following queries to confirm if you have Savings Plans, and Reserved Instances in your usage. If no lines are returned, you have no Savings Plans or Reserved Instances. Replace (database).(tablename) and run the following:

   Savings Plans:

    select * from (database).(tablename)
    where savings_plan_savings_plan_a_r_n not like ''
    limit 10
   

   Reserved Instances:

    select * from (database).(tablename)
    where reservation_reservation_a_r_n not like ''
    limit 10
   

3. Create the account_map view by modifying the following code, and executing it in Athena:

   • View0 - account_map

4. Create the Summary view by modifying the following code, and executing it in Athena:

   • View1 - Summary View

5. Create the EC2_Running_Cost view by modifying the following code, and executing it in Athena:

   • View2 - EC2_Running_Cost

6. Create the Compute savings plan eligible spend view by modifying the following code, and executing it in Athena:

   • View3 - compute savings plan eligible spend

7. Create the s3 view by modifying the following code, and executing it in Athena:

   • View4 - s3

8. Create the RI SP Mapping view by modifying the following code, and executing it in Athena:

   • View5 - RI SP Mapping

9. Create the customer_all view by modifying the following code, and executing it in Athena:

   • View6 - customer_all

Create QuickSight Data Sets

Create Datasets

1. Go to the QuickSight service homepage inside your account. Be sure to select the correct region from the top right user menu or you will not see your expected tables

   

2. From the left hand menu, choose Datasets

   

3. Click New dataset displayed in the top right corner

   

4. Choose Athena as your Data Source

   

5. Enter a data source name of Cost_Dashboard and click Create data source

   

6. Select the database which holds the views you created (reference Athena if you’re unsure which one to select), and the summary_view table, then click Edit/Preview data

   

7. Select SPICE to change your Query mode

   

8. Click Add Data

   

9. Select Data source

10. Choose your Cost_Dashboard view and click Select

    

11. Select the database which holds the CUR views you created

12. Choose your account_map view and click Select

13. Click the two circles to open the Join configuration, then select Left to change your join type

    

14. Configure the join clause to linked_account_id = account_id, then click Apply

    

15. Select Save

    

16. Select the summary_view dataset

    

17. Click Schedule refresh

    

18. Click Create

    

19. Enter a daily schedule, in the appropriate time zone and click Create

    

20. Click Cancel to exit

    

21. Click x to exit

    

22. Repeat steps 3-21, creating data sets with the remaining Athena views. You will reuse your existing Cost_Dashboard data source, and select the following views as the table:

• s3_view

• ec2_running_cost

• compute_savings_plan_eligible_spend

  NOTE: Make sure to reuse the existing Athena data source by scrolling to the bottom of the Data source create/select page when creating a new Dataset instead of creating a new data source

  When this step is complete, your Datasets tab should have 4 new SPICE Datasets

23. Select the summary_view dataset

    

24. Click Edit Data Set

    

25. Click Add Data

    

26. Select Data source

27. Choose your Cost_Dashboard view and click Select

    

28. Select the database which holds the CUR views you created

29. Choose your ri_sp_mapping view and click Select

    

30. Click the two circles to open the Join configuration, then select Left to change your join type

    

31. Click Add a new join clause twice so you have 3 join clauses to configure in total. Configure the 3 join clauses as below, then click Apply

    • ri_sp_arn = ri_sp_arn_mapping
    • payer_account_id = payer_account_id_mapping
    • billing_period = billing_period_mapping

    

32. Click Save

    

33. Click New dataset displayed in the top right corner of QuickSight

    

34. Select your existing Cost_Dashboard as your Data Source

29. Select Create dataset

30. Select the database which holds your customer_all view and select that view then click Edit/Preview data

31. Keep the Query mode as Direct query

    

32. Repeat steps 8-15, but configure this join clause in step 14: line_item_usage_account_id = account_id

    

Import Dashboard Template

We will now use the CLI to create the dashboard from the CUDOS Dashboard template.

1. Edit and Run list-users and make a note of your User ARN:

aws quicksight list-users --aws-account-id <Account_ID> --namespace default --region <Region>

2. Edit and Run list-data-sets and make a note of the Name and Arn for the 5 Datasets ARNs:

aws quicksight list-data-sets --aws-account-id <Account_ID> --region <Region>

3. Create an import.json file using the below sample

{
    "AwsAccountId": "<Account_ID>",
    "DashboardId": "cudos",
    "Name": "CUDOS",
    "Permissions": [
         {
                "Principal": "<User ARN>",
        "Actions": [
                     "quicksight:DescribeDashboard",
                     "quicksight:ListDashboardVersions",
                     "quicksight:UpdateDashboardPermissions",
                     "quicksight:QueryDashboard",
                     "quicksight:UpdateDashboard",
                     "quicksight:DeleteDashboard",
                     "quicksight:DescribeDashboardPermissions",
                     "quicksight:UpdateDashboardPublishedVersion"
            ]
        }
    ],
"DashboardPublishOptions": {
"AdHocFilteringOption": {
"AvailabilityStatus": "DISABLED"
}
},
    "SourceEntity": {
        "SourceTemplate": {
            "DataSetReferences": [
                {
                    "DataSetPlaceholder": "summary_view",
                    "DataSetArn": "arn:aws:quicksight:<Region>:<Account_ID>:dataset/<DatasetID>"

                },
                                         {
                    "DataSetPlaceholder": "ec2_running_cost",
                    "DataSetArn": "arn:aws:quicksight:<Region>:<Account_ID>:dataset/<DatasetID>"

                },
                                         {
                     "DataSetPlaceholder": "compute_savings_plan_eligible_spend",
                     "DataSetArn": "arn:aws:quicksight:<Region>:<Account_ID>:dataset/<DatasetID>"

                 },
                                         {
                     "DataSetPlaceholder": "s3_view",
                     "DataSetArn": "arn:aws:quicksight:<Region>:<Account_ID>:dataset/<DatasetID>"

                 },
                              {
                 "DataSetPlaceholder": "customer_all",
                 "DataSetArn": "arn:aws:quicksight:<Region>:<Account_ID>:dataset/<DatasetID>"
             }
             ],
                     "Arn": "arn:aws:quicksight:us-east-1:223485597511:template/cudos_dashboard_v3"
         }
     },
     "VersionDescription": "1"
}

4. Update the import.json to match your details by replacing the following placeholders:

   Placeholder	Replace with
   <Account_ID>	AWS Account ID where the dashboard will be deployed
   <Region>	Region Code where the dashboard will be deployed (Example eu-west-1)
   <User ARN>	ARN of your user
   <DataSetId>	Replace with Dataset ID’s from the data sets you created in the Preparing Quicksight section NOTE: There are 5 unique Dataset IDs

5. Run the import

aws quicksight create-dashboard --cli-input-json file://import.json --region <Region> --dashboard-id cudos

6. Check the status of your deployment

aws quicksight describe-dashboard --dashboard-id cudos --region <Region> --aws-account-id <Account_ID>

If you encounter no errors, open QuickSight from the AWS Console, and navigate to Dashboards. You should now see CUDOS available. This dashboard can be shared with other users, but is otherwise ready for viewing and customizing.

If something goes wrong in the dashboard creation step, correct the issue then delete the failed deployment before re-deploying

aws quicksight delete-dashboard --dashboard-id cudos --region <Region> --aws-account-id <Account_ID>

1. Navigate to the Cloud Intelligence Dashboards automation repo and follow the instructions to run the command line tool. You will have the option of deploying the CUDOS Dashboard from the list of supported dashboards.

Follow the remaining steps if you have not already deployed the Cost Intelligence Dashboard in the previous step .

2. Once complete, visit the account mapping page and follow the steps there to get your account names into the dashboard.

Schedule dataset refresh in QuickSight.

1. Select the summary_view dataset

   

2. Click Schedule refresh

   

3. Click Create

   

4. Enter a daily schedule, in the appropriate time zone and click Create

   

5. Click Cancel to exit

   

6. Click x to exit

   

7. Repeat these steps with any remaining Athena datasets except customer_all.

Create the Cost Intelligence Dashboard using a CloudFormation Template

1. Login via SSO in your Cost Optimization account

2. Click the Launch CloudFormation button below to open the pre-populated stack template in your CloudFormation console and select Next

   • Launch CloudFormation Template

3. Enter a Stack name for your template such as Cost-Intelligence-Dashboard-QuickSight

4. Review 1stReadMe parameter to confirm prerequisites before specifying the other parameters

5. Update your AthenaQueryResultsBucket with the Athena results location where your CUR table is To validate your Athena primary workgroup has an output location by

   • Open a new tab or window and navigate to the Athena console
   • Select Workgroup: primary
   • Click the bubble next to primary and then select view detail
   • Confirm your Query result location is configured with an S3 bucket path.
     • If configured, add the location to the AthenaQueryResultsBucket in your CloudFormation Template.
     • If not configured, continue to setting up by clicking Edit workgroup
   • Add the S3 bucket path you have selected for your Query result location and click save
   • Add the location to the AthenaQueryResultsBucket in your CloudFormation Template.

6. Update your BucketFolderPath with the S3 path where your year partitions of CUR data are stored To validate the correct path for your year partitions of the CUR data follow the tasks below:

   • Open a new tab or window and navigate to the S3 console
   • Select the S3 Bucket your CUR is located in
   • Navigate your folders until you find the folder with the year partitions of the CUR
     • Tip: Your yearly partitions folder is located in the folder with your .yml file, monthly folders, and status report
   • Add the identified BucketFolderPath to the CloudFormation parameter making sure to not add trailing / (eg - BucketName/FolderName/…/FolderName)
     • Tip: copy and paste the S3 URI then remove the leading ‘s3://’ and the ending ‘/'

7. Update your CURDatabaseName and CURTableName with the name of the CUR Athena Database and Table To validate the Athena Database and Table of the CUR data follow the tasks below:

   • Open a new tab or window and navigate to the Glue console
   • Select the Athena Table your CUR is located in
   • Find your Database CURDatabaseName and Table CURTableName
   • Add the identified CURDatabaseName and CURTableName to the CloudFormation parameter

8. Update your QuickSightUser with your QuickSight username To validate your QuickSight complete the tasks below:

   • Open a new tab or window and navigate to the QuickSight console
   • Find your username in the top right navigation bar
   • Add the identified username to the CloudFormation parameter

9. Update your QuicksightIdentityRegion with your QuickSight region

   • Optional add a Suffix if you want to create multiple instances of the same account.

10. Select Next at the bottom of Specify stack details and then select Next again on the Configure stack options page

11. Review the configuration, click I acknowledge that AWS CloudFormation might create IAM resources, and click Create stack.

12. You will see the stack will start in CREATE_IN_PROGRESS

12. Once complete, the stack will show CREATE_COMPLETE

13. Navigate to Dashboards page in your QuickSight console, click on your Cost Intelligence Dashboard name or your CUDOS Dashboard name

14. Set up scheduled SPICE refresh for SPICE Datasets manually or click the Launch Spice Refresh CloudFormation button below to open the pre-populated stack template in your CloudFormation console and select Next

    • Launch Spice Refresh CloudFormation Template

If you are tracking our Changelog , you already know that we continue improving Cloud Intelligence Dashboards.

To pull the latest version of the dashboard from the public template please use the following steps.

Option 1: Command Line Tool

Visit the GitHub repository to download and install the CID Command Line Tool and follow the instructions for running the update command.

Option 2: Manual Update

1. Create an update.json file by removing permissions section from the import.json file. Sample for CUDOS Dashboard update.json file below:

{
    "AwsAccountId": "<Account_ID>",
    "DashboardId": "cudos",
    "Name": "CUDOS",
    "DashboardPublishOptions": {
      "AdHocFilteringOption": {
        "AvailabilityStatus": "DISABLED"
      }
    },
    "SourceEntity": {
      "SourceTemplate": {
        "DataSetReferences": [
          {
            "DataSetPlaceholder": "summary_view",
            "DataSetArn": "arn:aws:quicksight:<region>:<Account_ID>:dataset/<DatasetID>"
          },
          {
            "DataSetPlaceholder": "ec2_running_cost",
            "DataSetArn": "arn:aws:quicksight:<region>:<Account_ID>:dataset/<DatasetID>"
          },
          {
            "DataSetPlaceholder": "compute_savings_plan_eligible_spend",
            "DataSetArn": "arn:aws:quicksight:<region>:<Account_ID>:dataset/<DatasetID>"
          },
          {
            "DataSetPlaceholder": "s3_view",
            "DataSetArn": "arn:aws:quicksight:<region>:<Account_ID>:dataset/<DatasetID>"
          },
          {
            "DataSetPlaceholder": "customer_all",
            "DataSetArn": "arn:aws:quicksight:<region>:<Account_ID>:dataset/<DatasetID>"
          }
        ],
        "Arn": "arn:aws:quicksight:us-east-1:223485597511:template/cudos_dashboard_v3"
          }
      }
}

2. If needed update the update.json to match your details by replacing the following placeholders:

   Placeholder	Replace with
   <Account_ID>	AWS Account ID where the dashboard will be deployed
   <Region>	Region Code where the dashboard will be deployed (Example eu-west-1)
   <DatasetID>	Replace with Dataset ID’s from the datasets you created in the Preparing Quicksight section NOTE: There are 5 unique Dataset IDs

3. Pull the latest published version of the dashboard template. Example for CUDOS Dashboard below:

aws quicksight update-dashboard --cli-input-json file://update.json --region <region>

4. Query the version number of the published dashboard. Example for CUDOS Dashboard below:

aws quicksight list-dashboard-versions --region <region> --aws-account-id <Account_ID> --dashboard-id cudos --query 'sort_by(DashboardVersionSummaryList, &VersionNumber)[-1].VersionNumber'

5. Apply the latest pulled changes to the deployed dashboard with this CLI command. Example for CUDOS Dashboard below:

aws quicksight update-dashboard-published-version --region <region> --aws-account-id <Account_ID> --dashboard-id cudos --version-number <version>