[AWS] CloudFormation (CFN)

CFN is an Infrastructure as Code (IAC) product – you can create, manage, and remove infrastructure automatically.

CloudFormation (CFN)

Templates

A CFN template is JSON or YAML. It contains logical resources and configuration.

  • A template can create up to 200 resources.
  • The Resource section is mandatory.
  • Optional sections: Metadata, Parameters, Mappings, Conditions, Transform, Outputs

Resources

Resources are identified with Resource type identifiers.
AWS::product::type
e.g.) AWS::EC2::Instance, AWS::IAM:Role, AWS::S3::Bucket


Stacks

Stacks are created and modified based on templates, which can be changed and used to update a stack.

  • Stacks take logical resources from a template and create, update, or delete physical resources in AWS.
  • If a stack is deleted, any resources it has created are also deleted.
  • A stack can be updated by uploading a new version of a template.
  • New logical resource -> New physical resource
  • Removed logical resource -> causes the stack to delete physical resources.
  • Changed logical resources -> some disruption or replace physical resources.

Template Files

  • Structure
AWSTemplateFormatVersion: "version date" # optional

Description: # optional
  String

Metadata: # optional - additional information 
  template metadata

Parameters: # optional - input values at runtime
  set of parameters

Rules: # optional - validate parameters
  set of rules

Mappings: # optional - key/value pairs as lookup table
  set of mappings

Conditions: # optional - controls resource creation 
  set of conditions

Transform: # optional - import snippets of code
  set of transforms

Resources: # required - specify stack resources
  # set of resources
  {Logical Name}:
    Type: {type name}
    Properties:
      set of properties

Outputs: # optional - return values
  set of outputs

  • Example – Create an EC2 instance and enable SSH
AWSTemplateFormatVersion: 2010-09-09

Description: Template to create an EC2 instance and enable SSH

Parameters: 
  KeyName:
    Description: Name of SSH KeyPair
    Type: 'AWS::EC2::KeyPair::KeyName'
    ConstraintDescription: Provide the name of an existing SSH key pair

Resources:
  MyEC2Instance:
    Type: 'AWS::EC2::Instance'
    Properties:
      InstanceType: t2.micro
      ImageId: ami-0d5eff06f840b45e9
      KeyName: !Ref KeyName
      SecurityGroups:
        - !Ref InstanceSecurityGroup
      Tags:
        - Key: Name
          Value: My CF Instance
  InstanceSecurityGroup:
    Type: 'AWS::EC2::SecurityGroup'
    Properties:
      GroupName: MyDMZSecurityGroup
      GroupDescription: Enable SSH access via port 22
      SecurityGroupIngress:
      - IpProtocol: tcp
        FromPort: 22
        ToPort: 22
        CidrIp: 0.0.0.0/0

Outputs: 
  InstanceID:
    Description: The Instance ID
    Value: !Ref MyEC2Instance

Benefits

  • Quick deployment and Easy cleanup
  • Allows to prepare for disaster recovery
  • Infrastructure version control

Error Handling

You can check the stack status in the CloudFormation console.

Common Errors

  • IAM: Insufficient permissions
  • Resource Limit: ex. 20 EC2 instances per region
  • Failed Rollback

Pseudo Parameters

Pseudo parameters are predefined references to AWS objects. You can use them with the “Ref” function.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html

  • AWS::AccountId
    • the AWS account ID of the account
  • AWS::Region
    • a string representing the Region
  • AWS::NotificationARNs
    • the list of notification Amazon Resource Names (ARNs) for the current stack
  • AWS::StackId
    • the name of the stack
Resources:
  MyEc2Instance:
    Type: "AWS::EC2::Instance"
    Properties:
      InstanceType: t2.micro
      ImageId: ami-0b5eea76982371e91 # us-east-1
      Tags:
        - Key: "Name"
          Value: !Join [ " ", [ "Instance", "for", !Ref AWS::Region ] ] 
        - Key: "Account"
          Value: !Ref AWS::AccountId  

Intrinsic Functions

CloudFormation provides a couple of built-in functions that help you manage your stack. Please refer to the following reference page.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html

!Join [ ":", [ a, b, c ] ]    # a:b:c

!Split [ "|" , "a|b|c" ]  # ["a", "b", "c"]

!Select [ "1", [ "apples", "grapes", "oranges" ] ] # grapes

# creates 6 CIDRs with a subnet mask "/27" 
# inside from a CIDR with a mask of "/24"
!Cidr [ "192.168.0.0/24", 6, 5 ]

Mappings

Mappings associate input values to another values.

  • Define the mappings inside the “Mappings” section
  • Use the “FindInMap” function to retrieve the value in the map
!FindInMap [MapName, TopLevelKey, SecondLevelKey]
Mappings:
  RegionMap:
    us-east-1:
      AMI: ami-0b5eea76982371e91
    us-west-1:
      AMI: ami-bf5540df
    eu-west-1:
      AMI: ami-3bfab942
Resources:
  MyEc2Instance:
    Type: "AWS::EC2::Instance"
    Properties:
      InstanceType: t2.micro
      ImageId: !FindInMap ["RegionMap", !Ref AWS::Region, "AMI"]
      Tags:
      - Key: "Name"
        Value: !Join [" ", ["My Instance for", !Ref AWS::Region]]

Input Parameters

  • You can pass custom values into the template at run time.
  • You can specify the default values.
  • Parameters are defined in the top level “Parameters” section.
  • Parameter Data Types
    • String
    • Number
    • List
    • CommaDelimitedList
    • AWS-Specific Types such as AWS::EC2::Image::Id
Parameters:
  InstanceName:
    Description: Name of your new instance
    Type: String
  KeyName:
    Description: Name of an existing EC2 KeyPair
    Type: AWS::EC2::KeyPair::KeyName
Resources:
  MyEc2Instance:
    Type: "AWS::EC2::Instance"
    Properties:
      InstanceType: t2.micro
      ImageId: ami-0b5eea76982371e91
      Tags:
        - Key: "Name"
          Value: !Ref InstanceName
      KeyName: !Ref KeyName

Outputs

You can access information about resources in a stack.

  • For example:
    • public IP
    • DNS name
Resources:
  MyEc2Instance:
    Type: "AWS::EC2::Instance"
    Properties:
      InstanceType: t2.micro
      ImageId: ami-0b5eea76982371e91
      Tags:
        - Key: "Name"
          Value: "My Public Instance"
Outputs:
  PublicIp:
    Value: !GetAtt ["MyEc2Instance", "PublicIp"]

EC2 User Data

Resources:
  MyEc2Instance:
    Type: 'AWS::EC2::Instance'
    Properties:
      InstanceType: t2.micro
      ImageId: ami-0b5eea76982371e91
      Tags:
      - Key: 'Name'
        Value: 'My httpd Server Instance'
      SecurityGroupIds:
        - !Ref MyHttpSecurityGroup
      UserData:
        !Base64 |
            #!/bin/bash -xe            
            yum update -y
            yum install -y httpd
            service httpd start
  MyHttpSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Open Ports 80
      SecurityGroupIngress:
      - IpProtocol: tcp
        FromPort: '80'
        ToPort: '80'
        CidrIp: 0.0.0.0/0
Outputs:
  Website:
    Description: The Public DNS for the EC2 Instance
    Value: !Sub 'http://${MyEc2Instance.PublicDnsName}'

Helper Scripts

CloudFormation provides Python-based helper scripts to optimize the process.

  • Helper scripts are preinstalled on Amazon Linux images.
  • cfn-init
    • reads and execute Metadata
    • defined in the “AWS::CloudFormation::Init:” section
  • cfn-singal
    • sends signal CFN when resources are ready
  • cfn-get-metadata
    • retrieve metadata based on a specified key

The following template will install and run the http service through the init script.

Resources:
  MyEC2Instance:
    Type: AWS::EC2::Instance
    Metadata: 
      AWS::CloudFormation::Init:
        config: 
          packages: 
            yum:
              httpd: []
          services: 
            sysvinit:
              httpd:
                enabled: true
                ensureRunning: true
    Properties:
      InstanceType: t2.micro
      ImageId: ami-0b5eea76982371e91
      Tags:
      - Key: 'Name'
        Value: 'My httpd Server Instance'
      SecurityGroupIds:
        - !Ref MyHttpSecurityGroup
      UserData: # call the init script
        'Fn::Base64':  !Sub | 
          #!/bin/bash -xe
          /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource MyEC2Instance  --region ${AWS::Region} 
  MyHttpSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Open Ports 80
      SecurityGroupIngress:
      - IpProtocol: tcp
        FromPort: '80'
        ToPort: '80'
        CidrIp: 0.0.0.0/0
Outputs:
  Website:
    Description: The Public DNS for the EC2 Instance
    Value: !Sub 'http://${MyEC2Instance.PublicDnsName}'

Change Sets

Updating a stack can be risky, especially for the production environment. For example, renaming an SQS queue will result in deleteing the old one and creating a new one.

Using Change Sets, we can preview how changes will impact to the resources.

Operations

  • Create
    • create a Change Set by updating existing template
  • View
    • check the proposed changes
  • Execute
    • apply changes to the existing stack
  • Delete
    • delete the Change Set without updating the stack

StackSets

A StackSet lets you create, update, and delete CloudFormations stacks across multiple AWS accounts and regions in a single operation.

You need to set up permissions using cross-account roles.

In your administrator account:

  • Create a role: AWSCloudFormationStackSetAdministrationRole
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "sts:AssumeRole"
            ],
            "Resource": [
                "arn:aws:iam::*:role/AWSCloudFormationStackSetExecutionRole"
            ],
            "Effect": "Allow"
        }
    ]
}
  • Setup the Trust Relationship (Admin account)
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudformation.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

In each of your target accounts

  • Create a role: AWSCloudFormationStackSetExecutionRole
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": 
               [
                 "cloudformation:*",
                 "s3:*",
                 "sns:*",
 				 "ec2:*"
               ],
            "Resource": "*"
        }
      ]
}
  • Setup the Trust Relationship (Target account)
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::{admin_account_id}:root"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Best Practices

  • Control access to CloudFormation using IAM
  • Be aware of Service Limits
  • Avoid Manual Updates – mismatch between your stack template and the current stack state.
  • Use CloudTrail to track all changes

AWS CDK

AWS CDK (Cloud Development Kit) enables Infrastructure as code through imperative programming.

  • CDK models your application imperatively using programming languages.
    • Write the CDK application using JavaScript, Python, Java, or C#.
  • CDK applications define one or more stacks.
  • CDK provisions resources with CloudFormation.
    • AWS CDK Toolkit (CDK CLI) is a command line tool to convert CDK stacks to CloudFormation templates and deploy them to an AWS account.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s