[AWS] CloudFormation (CFN)

CFN is an Infrastructure as Code (IAC) product – you can create, manage, and remove infrastructure automatically.

Templates

Read more about Templates

Templates

• JSON or YAML format.
• A template contains logical resources and configuration.
• A template can create up to 200 resources.
• A template has to be uploaded to S3.
• Sections
  • The “Resources” section is mandatory.
  • Optional sections
    • Metadata, Parameters, Mappings, Conditions, Transform, Outputs

Resources

• Resources are identified with Resource type identifiers.
• AWS resource and property types reference – AWS CloudFormation (amazon.com)
  • ! The resource reference pages are really important for dev-ops engineers.
• Resource type identifier
  • service-provider::service-name::data-type-name
    • AWS::product::type
    • e.g.) AWS::EC2::Instance, AWS::IAM:Role, AWS::S3::Bucket

Permissions

• Create an AWS CloudFormation service role that has the required permissions.
• Grant the “iam:PassRole” permission to the user who runs the template.

Stacks

Read more about Stacks

Stacks are created and modified based on templates, which can be changed and used to update a stack.

• Stacks take logical resources from a template and then create, update, or delete physical resources in AWS.
  • Nested stacks provide the ability to configure multiple elements within your environment while reducing duplication of code.
• If a stack is deleted, any resources it has created are also deleted.
• A stack can be updated by uploading a new version of a template.
  • You can preview the ChangeSet, which shows what will be changed by a new template.
• How does the changes of template work with stacks:
  • New logical resources -> create new physical resources
  • Remove logical resources -> delete physical resources
  • Change logical resources -> cause some disruption or replace physical resources.

Benefits

• Infrastructure as Code (IaC)
  • Infrastructure version control and review
  • Leverage existing templates and documentation
• Cost Management
  • Easy estimation of the cost of the resources
• Productivity
  • Quick deployment and Easy cleanup
  • Allows to prepare for disaster recovery

Error Handling & Troubleshooting

You can check the stack status in the CloudFormation console.

Common Errors

• IAM: Insufficient permissions
• Resource Limit: ex. 20 EC2 instances per region
  • Request a limit increase
• UPDATE_ROLLBACK_FAILED
  • You need to fix resources manually and then call the “ContinueUpdateRollback” API
• DELETE_FAILED
  • Some resources just cannot be deleted.
    • Non-empty S3 buckets
    • Security groups that are associated with EC2 instances
  • Use Custom Resources
  • Use “DeletionPolicy = Retain” to skip the deletion

Rollbacks

• Stack Creation Fails:
  • Stack Failure Options
    • Rollback all stack resources (default)
      • Check the logs
    • Reserve successfully provisioned resources

• Stack Update Fails
  • The stack rolls back to the previously known working state. (default)

Drift

What if you manually modify resources that were created by CloudFormation? How do we know whether the resources have changed (called drift) ?

• CloudFormation provides the feature to detect drift in your stack.
  • Stack Actions => “Detect Drift” or use CLI
  • Check the drift status

# get the stack drift detection id
$ aws cloudformation detect-stack-drift --stack-name <stack-name> 

# get the stack drift detection status via id
$ aws cloudformation describe-stack-drift-detection-status --stack-drift-detection-id <id>

Rolling back when drifted

• A stack goes into the “UPDATE_ROLLBACK_FAILED” state when a resource can not return to its original state during an update.
• Solutions
  • Fix the errors manually and then continue the stack update
  • Skip the failed resources

Helper Scripts

CloudFormation provides Python-based helper scripts to optimize the process.

• Helper scripts are preinstalled on Amazon Linux images.

cfn-init

• You can define instructions in the “Metadata” and then executes it in the “UserData”.
• “Metadata” configuration:
  • “AWS::CloudFormation::Init:” section
  • Multiple “config” sections can be created and they will executed in order.
  • A “config” contains the following:
    • packages: download and install packages
    • groups: define user groups
    • users: define users
    • sources: download files to be placed on EC2 instances
    • files: create files
    • services: launch a list of system services
    • commands: run commands
• You can find the logs in the “/var/log/cfn-init.log“

Resources:

  MyEC2Instance:
    Type: AWS::EC2::Instance
    Metadata: 
      AWS::CloudFormation::Init:
        config: 
          packages: 
            yum:
              httpd: []
          services: 
            sysvinit:
              httpd:
                enabled: true
                ensureRunning: true
    Properties:
      InstanceType: t2.micro
      ImageId: ami-0b5eea76982371e91
      Tags:
      - Key: 'Name'
        Value: 'My httpd Server Instance'
      SecurityGroupIds:
        - !Ref MyHttpSecurityGroup
      UserData: # call the init script
        'Fn::Base64':  !Sub | 
          #!/bin/bash -xe
          /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource MyEC2Instance  --region ${AWS::Region} 

  MyHttpSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Open Ports 80
      SecurityGroupIngress:
      - IpProtocol: tcp
        FromPort: '80'
        ToPort: '80'
        CidrIp: 0.0.0.0/0

Outputs:
  Website:
    Description: The Public DNS for the EC2 Instance
    Value: !Sub 'http://${MyEC2Instance.PublicDnsName}'

cfn-singal

• You can run the “cfn-signal” script right after “cfn-init”
  • sends a signal to CloudFormation whether the resource creation is successful or not
• You need to set the “WaitCondition” to block the stack until it receives a signal from “cfn-signal”
  • For EC2 and Auto Scaling Groups, you need to use the “CreationPolicy” attribute instead of the “AWS::CloudFormation::WaitCondition” resource type.
• Troubleshooting
  • If the “WaitCondition” does not receive the required number of signals
    • Check whether the helper scripts are installed in EC2 instances
      • Check the AMI
    • Check the logs in the “/var/log/cfn-init.log” file
      • You need to disable the rollback to keep the instance alive.
    • Verify that EC2 instance has connection to the internet
      • If your instances are in the private subnet, you need to check the NAT Gateway is properly configured.

Resources:

  MyEC2Instance:
    Type: AWS::EC2::Instance
    Metadata: 
      AWS::CloudFormation::Init:
        ...
    Properties:
      ...
      UserData: # call the init script
        'Fn::Base64':  !Sub | 
          #!/bin/bash -xe
          /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource MyEC2Instance  --region ${AWS::Region} 
          /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource MyEC2Instance  --region ${AWS::Region}
    CreationPolicy:
      ResourceSignal:
        Timeout: PT5M
        Count: 1
  

Resources:

  MyEC2Instance:
    Type: AWS::EC2::Instance
    Metadata: 
      AWS::CloudFormation::Init:
        ...
    Properties:
      ...
      UserData: # call the init script
        'Fn::Base64':  !Sub | 
          #!/bin/bash -xe
          /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource MyEC2Instance  --region ${AWS::Region} 
          INIT_SIGNAL = $?
          /opt/aws/bin/cfn-signal -e INIT_SIGNAL --stack ${AWS::StackName} --resource MyWaitCondition --region ${AWS::Region}

  MyWaitCondition:
    Type: AWS::CloudFormation::WaitCondition
    CreationPolicy:
      ResourceSignal:
        Timeout: PT5M
        Count: 1
  

cfn-get-metadata

• retrieves metadata based on a specified key

cfn-hup

• A daemon checks CloudFormation template metadata changes (every x minutes) and applies the metadata configuration again.
• configuration files
  • /etc/cfn/cfn-hup.conf
  • /etc/cfn/hooks.d/cfn-auto-reloader.conf
• execute custom hooks when the changes are detected

ChangeSets

Updating a stack can be risky, especially for the production environment. For example, renaming an SQS queue will result in deleting the old one and creating a new one.

Using ChangeSets, we can preview how changes will impact to the resources. Note that ChangeSets do not say if the update will succeed.

Operations

• Create
  • creates a ChangeSet by updating existing template
• View
  • checks the proposed changes
• Execute
  • applies changes to the existing stack
• Delete
  • deletes the ChangeSet without updating the stack

Best Practices

• Control access to CloudFormation using IAM
• Be aware of Service Limits
• Avoid Manual Updates
  • Mismatch between your stack template and the current stack state.
• Reuse Templates in multiple environments
• Do NOT embed credentials
  • Use input parameters
    • Set “NoEcho” to true
• Use Parameter Constraints
  • AllowedValues or AllowedPattern
  • MinLength & MaxLength
  • MinValue & MaxValue
• Validate Templates before using them
  • Use CloudFormation Designer
• Used nested Stacks

Parameters:
  DBPassword:
    Type: String
    Description: DB Password
    MinLength: 8
    MaxLength: 40
    NoEcho: true
    AllowedPattern: ^[a-zA-Z0-9]*$

CDK

AWS CDK (Cloud Development Kit) is an open-source development framework to enable Infrastructure as code through imperative programming.

• CDK models your cloud infrastructure of your application imperatively using familiar programming languages.
  • You can write CDK applications using JavaScript, Python, Java, or C#.
• CDK applications define one or more stacks.
• CDK provisions resources with CloudFormation.
  • AWS CDK Toolkit (CDK CLI) is a command line tool to convert CDK stacks to CloudFormation templates and deploy them to an AWS account.

CloudFormation

• When AWS CDK applications are run,
  • the CDK applications compile down to fully formed CloudFormation templates
  • then, the templates are submitted to the CloudFormation service for provisioning

CDK Workflow

1. Create the app from a template (from CDK)
   • Use “cdk init” command with a desired template and programming language
2. Add custom code to the app
3. Build the app if required
4. Synthesize stacks in the app to create an AWS CloudFormation template
5. Deploy the stacks in your AWS account
   • Use “cdk deploy” command