[AWS] CloudFormation (CFN)

CFN is an Infrastructure as Code (IAC) product – you can create, manage, and remove infrastructure automatically.


Features

Templates

  • A CFN template is JSON or YAML. It contains logical resources and configuration.
  • A template can create up to 200 resources.
  • The Resource section is mandatory.
  • Optional sections: Metadata, Parameters, Mappings, Conditions, Transform, Outputs

Resources

  • Resources are identified with Resource type identifiers.
  • AWS::product::type
    • e.g.) AWS::EC2::Instance, AWS::IAM:Role, AWS::S3::Bucket

Stacks

Stacks are created and modified based on templates, which can be changed and used to update a stack.

  • Stacks take logical resources from a template and create, update, or delete physical resources in AWS.
    • Nested stacks provide the ability to configure multiple elements within your environment while reducing duplication of code.
  • If a stack is deleted, any resources it has created are also deleted.
  • A stack can be updated by uploading a new version of a template.
  • New logical resource -> New physical resource
  • Removed logical resource -> causes the stack to delete physical resources.
  • Changed logical resources -> some disruption or replace physical resources.

Template File

  • Structure
AWSTemplateFormatVersion: "version date" # optional

Description: # optional
  String

Metadata: # optional - additional information 
  template metadata

Parameters: # optional - input values at runtime
  set of parameters

Rules: # optional - validate parameters
  set of rules

Mappings: # optional - key/value pairs as lookup table
  set of mappings

Conditions: # optional - controls resource creation 
  set of conditions

Transform: # optional - import snippets of code
  set of transforms

Resources: # required - specify stack resources
  # set of resources
  {Logical Name}:
    Type: {type name}
    Properties:
      set of properties

Outputs: # optional - return values
  set of outputs


Examples

  • Create an EC2 instance and enable SSH
AWSTemplateFormatVersion: 2010-09-09

Description: Create an EC2 instance and enable SSH

Parameters: 
  KeyName:
    Description: Name of SSH KeyPair
    Type: 'AWS::EC2::KeyPair::KeyName'
    ConstraintDescription: the name of an existing SSH key pair

Resources:
  MyEC2Instance:
    Type: 'AWS::EC2::Instance'
    Properties:
      InstanceType: 't3.micro'
      ImageId: 'ami-0d5eff06f840b45e9'
      KeyName: !Ref KeyName
      SecurityGroups:
        - !Ref InstanceSecurityGroup
      Tags:
        - Key: Name
          Value: My CF Instance
  InstanceSecurityGroup:
    Type: 'AWS::EC2::SecurityGroup'
    Properties:
      GroupName: MySecurityGroup
      GroupDescription: Enable SSH via port 22
      SecurityGroupIngress:
      - IpProtocol: tcp
        FromPort: 22
        ToPort: 22
        CidrIp: 0.0.0.0/0

Outputs: 
  InstanceID:
    Description: The Instance ID
    Value: !Ref MyEC2Instance

Benefits

  • Quick deployment and Easy cleanup
  • Allows to prepare for disaster recovery
  • Infrastructure version control

Error Handling

You can check the stack status in the CloudFormation console.

Common Errors

  • IAM: Insufficient permissions
  • Resource Limit: ex. 20 EC2 instances per region
    • Request a limit increase
  • Failed Rollback
    • You need to fix resources manually.

Pseudo Parameters

Pseudo parameters are predefined references to AWS objects. You can use them with the “Ref” function.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html

  • AWS::AccountId
    • the AWS account ID of the account
  • AWS::Region
    • a string representing the Region
  • AWS::NotificationARNs
    • the list of notification Amazon Resource Names (ARNs) for the current stack
  • AWS::NoValue
    • Removes the corresponding resource property when specified as a return value in the “Fn::If” intrinsic function
  • AWS::StackId, AWS::StackName
    • the id or name of the stack
Resources:
  MyEc2Instance:
    Type: "AWS::EC2::Instance"
    Properties:
      InstanceType: t2.micro
      ImageId: ami-0b5eea76982371e91 # us-east-1
      Tags:
        - Key: "Name"
          Value: !Join [ " ", [ "Instance", "for", !Ref AWS::Region ] ] 
        - Key: "Account"
          Value: !Ref AWS::AccountId  

Intrinsic Functions

CloudFormation provides a couple of built-in functions that help you manage your stack. Please refer to the following reference page.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html

!Join [ ":", [ a, b, c ] ]    # a:b:c

!Split [ "|" , "a|b|c" ]  # ["a", "b", "c"]

!Select [ "1", [ "apples", "grapes", "oranges" ] ] # grapes

# creates 6 CIDRs with a subnet mask "/27" 
# inside from a CIDR with a mask of "/24"
!Cidr [ "192.168.0.0/24", 6, 5 ]

Mappings

Mappings associate input values to another values.

  • Define the mappings inside the “Mappings” section
  • Use the “FindInMap” function to retrieve the value in the map
!FindInMap [MapName, TopLevelKey, SecondLevelKey]
Mappings:
  RegionMap:
    us-east-1:
      AMI: ami-0b5eea76982371e91
    us-west-1:
      AMI: ami-bf5540df
    eu-west-1:
      AMI: ami-3bfab942
Resources:
  MyEc2Instance:
    Type: "AWS::EC2::Instance"
    Properties:
      InstanceType: t2.micro
      ImageId: !FindInMap ["RegionMap", !Ref AWS::Region, "AMI"]
      Tags:
      - Key: "Name"
        Value: !Join [" ", ["My Instance for", !Ref AWS::Region]]

Input Parameters

  • You can pass custom values into the template at run time.
  • You can specify the default values.
  • Parameters are defined in the top level “Parameters” section.
  • Parameter Data Types
    • String
    • Number
    • List
    • CommaDelimitedList
    • AWS-Specific Types such as AWS::EC2::Image::Id
Parameters:
  InstanceName:
    Description: Name of your new instance
    Type: String
  KeyName:
    Description: Name of an existing EC2 KeyPair
    Type: AWS::EC2::KeyPair::KeyName
Resources:
  MyEc2Instance:
    Type: "AWS::EC2::Instance"
    Properties:
      InstanceType: t2.micro
      ImageId: ami-0b5eea76982371e91
      Tags:
        - Key: "Name"
          Value: !Ref InstanceName
      KeyName: !Ref KeyName

Outputs

You can access information about resources in a stack.

  • For example:
    • public IP
    • DNS name
Resources:
  MyEc2Instance:
    Type: "AWS::EC2::Instance"
    Properties:
      InstanceType: t2.micro
      ImageId: ami-0b5eea76982371e91
      Tags:
        - Key: "Name"
          Value: "My Public Instance"
Outputs:
  PublicIp:
    Value: !GetAtt ["MyEc2Instance", "PublicIp"]

EC2 User Data

Resources:
  MyEc2Instance:
    Type: 'AWS::EC2::Instance'
    Properties:
      InstanceType: t2.micro
      ImageId: ami-0b5eea76982371e91
      Tags:
      - Key: 'Name'
        Value: 'My httpd Server Instance'
      SecurityGroupIds:
        - !Ref MyHttpSecurityGroup
      UserData:
        !Base64 |
            #!/bin/bash -xe            
            yum update -y
            yum install -y httpd
            service httpd start
  MyHttpSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Open Ports 80
      SecurityGroupIngress:
      - IpProtocol: tcp
        FromPort: '80'
        ToPort: '80'
        CidrIp: 0.0.0.0/0
Outputs:
  Website:
    Description: The Public DNS for the EC2 Instance
    Value: !Sub 'http://${MyEc2Instance.PublicDnsName}'

Helper Scripts

CloudFormation provides Python-based helper scripts to optimize the process.

  • Helper scripts are preinstalled on Amazon Linux images.
  • cfn-init
    • reads and execute Metadata
    • defined in the “AWS::CloudFormation::Init:” section
  • cfn-singal
    • sends signal CFN when resources are ready
  • cfn-get-metadata
    • retrieve metadata based on a specified key

The following template will install and run the http service through the init script.

Resources:
  MyEC2Instance:
    Type: AWS::EC2::Instance
    Metadata: 
      AWS::CloudFormation::Init:
        config: 
          packages: 
            yum:
              httpd: []
          services: 
            sysvinit:
              httpd:
                enabled: true
                ensureRunning: true
    Properties:
      InstanceType: t2.micro
      ImageId: ami-0b5eea76982371e91
      Tags:
      - Key: 'Name'
        Value: 'My httpd Server Instance'
      SecurityGroupIds:
        - !Ref MyHttpSecurityGroup
      UserData: # call the init script
        'Fn::Base64':  !Sub | 
          #!/bin/bash -xe
          /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource MyEC2Instance  --region ${AWS::Region} 
  MyHttpSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Open Ports 80
      SecurityGroupIngress:
      - IpProtocol: tcp
        FromPort: '80'
        ToPort: '80'
        CidrIp: 0.0.0.0/0
Outputs:
  Website:
    Description: The Public DNS for the EC2 Instance
    Value: !Sub 'http://${MyEC2Instance.PublicDnsName}'

Change Sets

Updating a stack can be risky, especially for the production environment. For example, renaming an SQS queue will result in deleteing the old one and creating a new one.

Using Change Sets, we can preview how changes will impact to the resources.

Operations

  • Create
    • create a Change Set by updating existing template
  • View
    • check the proposed changes
  • Execute
    • apply changes to the existing stack
  • Delete
    • delete the Change Set without updating the stack

StackSets

A StackSet lets you create, update, and delete CloudFormations stacks across multiple AWS accounts and regions in a single operation.

You need to set up permissions using cross-account roles.

  • For the administrator account, use AWSCloudFormationStackSetAdministrationRole, which allows to assume AWSCloudFormationStackSetExecutionRole to provision resources in the target accounts.

In your administrator account:

  • Create a role: AWSCloudFormationStackSetAdministrationRole
    • allows the admin account to assume the stack set execution role
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "sts:AssumeRole"
            ],
            "Resource": [
                "arn:aws:iam::*:role/AWSCloudFormationStackSetExecutionRole"
            ],
            "Effect": "Allow"
        }
    ]
}
  • Setup the Trust Relationship (Admin account)
    • The CloudFormation service is trusted to assume the stack set admin role
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudformation.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

In each of your target accounts

  • Create a role: AWSCloudFormationStackSetExecutionRole
    • defines the permissions that need in the target account
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": 
               [
                 "cloudformation:*",
                 "s3:*",
                 "sns:*",
 				 "ec2:*"
               ],
            "Resource": "*"
        }
      ]
}
  • Setup the Trust Relationship (Target account)
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::{admin_account_id}:root"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Stack Policy

  • You can prevent stack resources from being unintentionally updated or deleted during a stack update by using a stack policy.
  • A stack policy is a JSON document that defines the actions that can be performed on designated resources.
{
  "Statement" : [
    {
      "Effect" : "Allow",
      "Action" : "Update:*",
      "Principal": "*",
      "Resource" : "*"
    },
    {
      "Effect" : "Deny",
      "Action" : "Update:*",
      "Principal": "*",
      "Resource" : "LogicalResourceId/ProductionDatabase"
    }
  ]
}

Best Practices

  • Control access to CloudFormation using IAM
  • Be aware of Service Limits
  • Avoid Manual Updates – mismatch between your stack template and the current stack state.
  • Use CloudTrail to track all changes

CDK vs. CloudFormation vs. SAM

CDK

AWS CDK (Cloud Development Kit) is an open-source development framework to enable Infrastructure as code through imperative programming.

  • CDK models your cloud infrastructure of your application imperatively using familiar programming languages.
    • You can write CDK applications using JavaScript, Python, Java, or C#.
  • CDK applications define one or more stacks.
  • CDK provisions resources with CloudFormation.
    • AWS CDK Toolkit (CDK CLI) is a command line tool to convert CDK stacks to CloudFormation templates and deploy them to an AWS account.

CloudFormation

  • When AWS CDK applications are run,
    • the CDK applications compile down to fully formed CloudFormation templates
    • then, the templates are submitted to the CloudFormation service for provisioning

SAM

  • AWS Serverless Application Model (SAM) allow you to define your serverless infrastructure in concise declarative templates.

CDK Workflow

  1. Create the app from a template (from CDK)
    • Use “cdk init” command with a desired template and programming language
  2. Add custom code to the app
  3. Build the app if required
  4. Synthesize stacks in the app to create an AWS CloudFormation template
  5. Deploy the stacks in your AWS account
    • Use “cdk deploy” command

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s