Deploy Data Collection Modules

Data Modules

For every module, there are three steps to complete:

  1. Grant additional permissions to the IAM roles created by OptimizationManagementDataRoleStack or OptimizationDataCollectionStack so they can access the relevant data. For our pre-made modules, it will specify which stack will need to be updated.
  2. Update OptimizationDataCollectionStack with the module to retrieve the data.
  3. Test the deployed Lambda function to confirm it is working as expected

These modules templates are managed in AWS owned buckets. If you do not wish to have updates on them from AWS then please save a copy to a bucket in your account and use instead.

Cost Explorer Rightsizing Recommendations
Inventory Collector
Trusted Advisor
Compute Optimizer Collector
ECS Chargeback Data
RDS Utilization Data

How to Update your CloudFormation

To add your selected modules from above please follow the steps specified in the module section.

How to Update OptimizationDataCollectionStack

  1. Login via SSO in your Cost Optimization account and search for CloudFormation. Images/cloudformation.png

  2. In your Cost Account under CloudFormation select your OptimizationDataCollectionStack

  3. Click Update Images/Update_CF.png

  4. Choose Edit template in designer then click View in Designer Images/update_in_designer.png

  5. In the template box copy your module code and past at the bottom of the template. When using Designer we recommend deleting the first space of the first line of the module. This will help with formatting the code all in the same style. Then Click the upload button on the top left hand corner.

Images/designer_view.png

  1. This will take you back to the upload section. Click Next and follow the same process you did on the initial setup. Images/Update_stack.png

How to Update IAM Policies in OptimizationManagementDataRoleStack

The IAM Roles created in the previous section need to be updated with the relevant permissions.

Depending on the module, you will need to add the permissions to either the management role or the role created in the stack set. In the pre-made modules this will be specified.

These instructions are for updating the roles in OptimizationManagementDataRoleStack. This stack is deployed in the management account.

  1. Login via SSO in your Management account and search for CloudFormation Images/cloudformation.png

  2. Select the OptimizationManagementDataRoleStack and click Update
    Images/Update_man_role.png

  3. Select Edit template in designer then View in Designer
    Images/Edit_template_man_role.png

  4. Copy the IAM permission code from the module section above. In Designer in the template section paste the code at the bottom. Click the Upload button in the top corner.
    Images/Update_man_role_design.png

  5. Click Next and keep everything to default till deployed

How to Update IAM Policies in OptimizationDataRoleStack

The IAM Roles created in the previous section stack set need to be updated with the relevant permission.

Depending on the module, you will need to add the permissions to either the management role or the role created in the stack set. In the pre-made modules this will be specified.

These instructions are for updating the roles in OptimizationDataRoleStack. This stack is deployed in each of the linked accounts.

  1. Copy the IAM permission code from the module section above. Using the file you downloaded in the Role for Read Only Data Collector step (called optimisation_read_only_role.yaml), paste the additional policy at the bottom and save.

  2. Login via SSO in your Management account and search for CloudFormation Images/cloudformation.png

  3. Copy the IAM permission code from the module section above. In your local copy of the optimisation_read_only_role.yaml file add it to the bottom.

  4. In CloudFormation click on the hamburger icon on the side panel on the left hand side of the screen and select StackSets.

  5. Select the OptimizationDataRoleStack. Click Actions, Edit StackSet Details Images/Update_SS.png

  6. Select Replace current template, Upload a template file then Choose file with you local copy of the optimisation_read_only_role.yaml file Images/Update_SS_File.png

  7. Click Next and keep everything to default till deployed

Testing your deployment

Once you have deployed your modules you will be able to test your Lambda function to get your first set of data in Amazon S3.

  1. The updated CloudFormation will have created a Nested stack. By clicking on your stack and selecting Resources find your lambda function and click the hyperlink.

  2. To test your lambda function click Test Images/lambda_test_cf.png

  3. Enter an Event name of Test, click Create:

Images/Configure_Test.png

  1. Click Test

  2. The function will run, it will take a minute or two given the size of the Organizations files and processing required, then return success. Click Details and view the output.

  3. Go to the Athena service page

Images/Athena.png

  1. You will be able to see your data in the Optimization_Data Database

Images/Optimization_Data_DB.png

  1. If your module has a saved query you will be able to see it in the Saved queries section. Images/Saved_queries.png

If you would like to make your own modules then go to the next section to learn more on how they are made!

Now you have your data in AWS Athena you can use this to identify optimization opportunities using Athena Queries or Passing into Amazon QuickSight.