FAQ

NOTE: These steps assume you’ve already setup the CUR to be delivered in each payer (management) account.

Setup S3 CUR Bucket Replication

1. Create S3 bucket with enabled versioning in the region where QuickSight is available.

2. Open S3 bucket and apply following S3 bucket policy with replacing respective placeholders {PayerAccountA}, {PayerAccountB} and {BucketName}. You can add more payer accounts to the policy if needed.

    {
    "Version": "2008-10-17",
    "Id": "PolicyForCombinedBucket",
    "Statement": [
        {
            "Sid": "Set permissions for objects",
            "Effect": "Allow",
            "Principal": {
                "AWS": ["{PayerAccountA}","{PayerAccountB}"]
            },
            "Action": [
                "s3:ReplicateObject",
                "s3:ReplicateDelete"
            ],
            "Resource": "arn:aws:s3:::{BucketName}/*"
        },
        {
            "Sid": "Set permissions on bucket",
            "Effect": "Allow",
            "Principal": {
                "AWS": ["{PayerAccountA}","{PayerAccountB}"]
            },
            "Action": [
               "s3:List*",
               "s3:GetBucketVersioning",
               "s3:PutBucketVersioning"
            ],
            "Resource": "arn:aws:s3:::{BucketName}"
        },
        {
            "Sid": "Set permissions to pass object ownership",
            "Effect": "Allow",
            "Principal": {
                "AWS": ["{PayerAccountA}","{PayerAccountB}"]
            },
            "Action": [
                "s3:ReplicateObject",
                "s3:ReplicateDelete",
                "s3:ObjectOwnerOverrideToBucketOwner",
                "s3:ReplicateTags",
                "s3:GetObjectVersionTagging",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::{bucket name}/*"
        }
    ]
    }
   

This policy supports objects encrypted with either SSE-S3 or not encrypted objects. For SSE-KMS encrypted objects additional policy statements and replication configuration will be needed: see https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication-config-for-kms-objects.html

Set up S3 bucket replication from each Payer (Management) account to S3 bucket in Governance account

This step should be done in each payer (management) account.

1. Open S3 bucket with CUR
2. On Properties tab under Bucket Versioning section click Edit and set bucket versioning to Enabled
3. On Management tab under Replication rules click on Create replication rule.
4. Specify rule name

5. Select Specify a bucket in another account and provide Governance account id and bucket name in Governance account
6. Select Change object ownership to destination bucket owner checkbox
7. Select Create new role under IAM Role section

8. Leave rest of the settings by default and click Save.

Copy existing objects from CUR S3 bucket to S3 bucket in Governance account

This step should be done in each payer (management) account.

Sync existing objects from CUR S3 bucket to S3 bucket in Governance account

aws s3 sync s3://{curBucketName} s3://{GovernanceAccountBucketName} --acl bucket-owner-full-control

After performing this step in each payer (management) account S3 bucket in Governance account will contain CUR data from all payer accounts under respective prefixes.

Prepare Glue Crawler

These actions should be done in Governance account

1. Open AWS Glue Service in AWS Console in the same region where S3 bucket with aggregated CUR data is located and go to Crawlers section
2. Click Add Crawler
3. Specify Crawler name and click Next
4. In Specify crawler source type leave settings by default. Click Next

5. In Add a data store select S3 bucket name with aggregated CUR data and add following exclusions **.zip, **.json, **.gz, **.yml, **sql, **csv, **/cost_and_usage_data_status/*. Click Next

6. In Add another data store leave No by default. Click Next

7. In Choose an IAM role select Create an IAM role and provide role name. Click Next

8. In Create a schedule for this crawler select Daily and specify Hour and Minute for crawler to run

9. In Configure the crawler’s output choose Glue Database in which you’d like crawler to create a table or add new one. Select Create a single schema for each S3 path checkbox. Select Add new columns only and Ignore the change and don’t update the table in the data catalog in Configuration options. Click Next

Please make sure Database name doesn’t include ‘-’ character

10. Crawler configuration should look as on the screenshot below. Click Finish

11. Resume deployment methodoly of choice from previous page.

Considerations:

• The permissions dataset can’t contain duplicate values. Duplicates are ignored when evaluating how to apply the rules.

• Each user or group specified can see only the rows that match the field values in the dataset rules.

• If you add a rule for a user or group and leave all other columns with no value (NULL), you grant them access to all the data.

• If you don’t add a rule for a user or group, that user or group can’t see any of the data.

  • If the userbase of QuickSight will be changing frequently, consider storing your csv in S3 rather than a local file.

• The full set of rule records that are applied per user must not exceed 999. This applies to the total number of rules that are directly assigned to a user name plus any rules that are assigned to the user through group names.

• If a field includes a comma (,) Amazon QuickSight treats each word separated from another by a comma as an individual value in the filter. For example, in (‘AWS’, ‘INC’), AWS,INC is considered as two strings: AWS and INC. To filter with AWS,INC, wrap the string with double quotation marks in the permissions dataset.

Create a CSV file defining users and Linked Account IDs

Create a CSV file that looks something like this:

username,account_id
user1@amazon.co.uk,"123456123456"
user1@amazon.co.uk,"987654987654"
user2@amazon.fr,"123456123456"
user3@amazon.com,"789123456123"

Any Account IDs that you wish the given user to see should be defined in the account_id field of the CSV. Create a separate row for a single username having access to multiple account IDs. Ensure there are no spaces after your final quote character. Name this file something similar to CUDOS_Dataset_rules.csv

If you want to use QuickSight groups the CSV input file is slightly different. Instead of UserName as the initial field you have to use GroupName. Also, you can only use Users or Groups in the input file, not both. This page provides more details. QuickSight groups can only be created and managed via the QuickSight CLI. There is no UI for this in the QuickSight console.

You now have 2 options on how to proceed:

1. Keep your CSV file local and create a new Dataset by Uploading a File as the Data Source
2. Upload your CSV to S3 and create a new Dataset with an S3 Manifest file as the Data Source

Using a local CSV file as the data source

Create a new Dataset using the CSV file above as the Data Source: Click New Dataset and select Upload a file. Locate your CUDOS_Dataset_rules.csv and a Preview will appear.

Click Edit settings and prepare data. Verify that the account_id field is a String data type. If it appears as an Integer, change the data type for account_id to String.

The reason we need to do this is that we will lose any leading or trailing zeroes on an Account ID if it remains as an Integer value. This is also the same Data Type used for the account_id field in all the CUDOS Datasets.

Save the CUDOS_Dataset_rules.csv dataset.

1. Select the dataset name
2. Click Row-level security
3. Select CUDOS_Dataset_rules.csv
4. Click Apply dataset
5. Click Apply dataset again
6. Wait until you see Selected dataset rules populate with CUDOS_Dataset_rules
7. Click x to exit
8. Click x to exit
9. Repeat steps for all CUDOS-related datasets (See below for a list of current Dataset Names)

Once you have applied the CUDOS_Dataset_rules S3 Dataset to all your CUDOS datasets, visit the CUDOS Dashboard as a user who is defined in the csv file, and confirm the Account IDs shown are only the ones specified in that file.

Use S3 as the data source

Upload your csv file to the Athena query location bucket. eg. aws-athena-query-results-123456123456-us-east-1

Create an S3 manifest file that looks something like this:

{
"entries": [
	{"url":"s3://aws-athena-query-results-123456123456-us-east-1/CUDOS_Dataset_rules.csv", 	"mandatory":true},
 ]
}

This manifest file can be saved locally, or uploaded to the same S3 bucket where the csv file is stored. Save this file as something similar to CUDOS_manifest.json.

Back in the QuickSight Admin Console, click New Dataset and select S3. Name the Dataset CUDOS_Dataset_rules and locate your CUDOS_manifest.json file either by entering the S3 URL where it is stored, or choosing to upload from your local machine.

Click Edit/preview data.

Verify that the account_id field is a String data type. If it appears as an Integer, change the data type for account_id to String.

Save the CUDOS_Dataset_rules dataset.

On each of the datasets that the CUDOS dashboard is using, define Row Level Security by following these steps:

1. Select the dataset name
2. Click Row-level security

3. Select CUDOS_Dataset_rules

4. Click Apply dataset
5. Click Apply dataset again
6. Wait until you see Selected dataset rules populate with CUDOS_Dataset_rules

7. Click x to exit
8. Click x to exit
9. Repeat steps for all CUDOS-related datasets (See below for a list of current Dataset Names)

Once you have applied the CUDOS_Dataset_rules S3 Dataset to all your CUDOS datasets, visit the CUDOS Dashboard as a user who is defined in the csv file, and confirm the Account IDs shown are only the ones specified in that file.