Enable VPC Flow Logs

Central AWS Account: AWS account which you want to designate for storing VPC Flow Logs centrally. This account will also contain Athena DB, table and QuickSight Dashboard.

Additional Accounts: These are other accounts that you own and has VPCs that you wish to enable Flow Logs and have an ability to push it to Central AWS Account’s S3 bucket.

QuickSight: To manage VPC Flow Logs and QuickSight dashboard in central account please make sure you create resources for the central account in the region supported by QuickSight. Refer to this link to see supported regions.

VPC

If you already have VPC and other resources running your AWS account continue with next section “Enable VPC Flow Logs” otherwise click on below link to deploy VPC and a toy webapp into your account.

Click here for instructions how to deploy a VPC to your AWS account:

Enable VPC Flow Logs

QuickSight dashboard provided in this lab requires all the fields mentioned in the Introduction section are required. If you already have enabled VPC Flow logs with those fields (with CSV format, Hive partition enabled and delivered to S3) then you can skip this section and proceed to "Create Athena resources, Lambda function and CloudWatch rule" section to continue. If you do not have VPC flow logs enabled or existing VPC Flow logs does not have all the required fields then this section will help you in enabling vpc flow logs for existing VPC(s) in your account. Repeat all the steps from this section for each VPC in case you want to enable VPC Flow logs in respective account to visualize them in QuickSight dashboard under central account.

  1. Login to your central AWS account.

  2. Run CloudFormation stack to enable VPC Flow Logs.

  • Download CloudFormation Template:
    • vpc-flow-logs-custom.yaml

      • This CloudFormation template enables VPC Flow Logs in the account you run it. You will need to run it per VPC.
    • From AWS Console navigate to CloudFormation. Then click on Create stack Images/quicksight_dashboard_dt-8.png

    • Create stack page:

      1. In Create stack page Specify template select Upload a template file.
      2. Then Choose File and upload the template vpc-flow-logs-custom.yaml (you have downloaded previously)
      3. Click Next Images/quicksight_dashboard_dt-9.png
  1. Provide name for the stack e.g., “vpc-flow-logs-stack” and values for the stack parameters and then click Next

    • TrafficType (ACCEPT, REJECT, ALL): Type of traffic you wish to record

      • ACCEPT — The recorded traffic was permitted by the security groups and network ACLs.
      • REJECT — The recorded traffic was not permitted by the security groups or network ACLs.
      • ALL - The recorded traffic that was permitted (ACCEPT) and was not permitted (REJECT) by the security groups or network ACLs.
    • VpcFlowLogsBucketName (Optional): S3 bucket name where VPC flow logs will be stored.

      • If you specify the bucket name then it is assumed that the bucket already exists. If you want to centralize the storage of the logs create the bucket before and specify the bucket name here.

      • If you leave it blank CloudFormation template will create a bucket for you.

      If you are enabling VPC Flow Logs in additional account then please make sure to modify S3 bucket’s policy from the central account to grant access to additional account and provide the name of the central bucket to this parameter.

      VpcFlowLogsBucketName - This bucket will be used to gather vpc flow logs for all of your vpcs from one or more accounts. So please make sure this is the central account where you want your VPC flow logs to be collected and QuickSight dashboard to be hosted.

    • VpcFlowLogsFilePrefix (Optional): VPC Flow logfile prefix in S3 bucket. See bold text in below example

      e.g., bucket_name/vpc-flow-logs/AWSLogs/aws_account_id/vpcflowlogs/region/year/month/day/

    • VpcId: You can find the VPC ID in console

    Images/quicksight_dashboard_dt-2.png

  2. In Configure stack options page, add below tags and click on Next

    • Name=VPCFlowLogs-CFN
    • Purpose=WALabVPCFlowLogs

Images/quicksight_dashboard_dt-3.png

  1. On Review screen verify the inputs you have provided

Images/quicksight_dashboard_dt-4.png

  1. Last click on Create stack Images/quicksight_dashboard_dt-5.png

  2. As shown below you will see progress of the stack creation under Events tab. Please wait for the stack to complete the execution. Once complete it will show the status CREATE_COMPLETE in green then proceed to the next step. Images/quicksight_dashboard_dt-6.png

  3. To verify, navigate to VPC service, click on vpc link and then click on Flow Logs tab at the bottom part of the screen. You will see a line with flow logs you just created now. Images/quicksight_dashboard_dt-7.png

Delete older VPC Flow Logs from S3 bucket (Optional)

We recommend you to create a life cycle policy to delete logs older than 90 days or lesser as per your requirement to save cost. All the steps from this section are required to execute one time in central account.

  • Click here to see the steps to Delete older VPC Flow Logs from S3 bucket

Enable VPC Flow Logs in additional accounts and store it in central bucket (Optional)

  • Click here to see the steps to enable VPC Flow logs in additional accounts