[AWS] AWS CodeBuild

CodeBuild is a managed CI/CD (Continuous Integration/Continuous Delivery) build service that compiles the code, runs unit tests, and produces deployment artifacts.

Features

“CodeBuild” automatically scales to meet your build volume, immediately processes each build you submit, and can run separate builds concurrently.

• On-demand Scaling, Pre-configured environments for many programming languages
• CodeBuild is the part of enabling Continuous Integration (CI) and Continuous Delivery (CD).
• Security
  • IAM can be used to control permissions.
    • The “CodeBuild Service Role” should have enough permission to access resource during the build.
  • KMS is used to encrypt build artifacts
  • CloudTrail is used for API calls
• Logging
  • CloudWatch Logs
  • S3
• Local Build
  • You can run CodeBuild locally on your desktop when you need deep troubleshooting.
  • You need to install Docker and CodeBuild agent.
• Environment Variables
  • AWS Provided
    • AWS_DEFAULT_REGION, CODEBUILD_BUILD_ID, CODEBUILD_BUILD_IMAGE …
  • Custom
    • Static
    • Dynamic – Secrets Manager and Parameter Store

Build Environments

Leverages Docker under the hood for reproducible builds.

• Pre-configured
  • Various versions of Windows and Linux
• Custom
  • Docker Images

Build Project and Buildspec

A “Build Project” includes information about how to build.

• Where to get the source code
  • CodeCommit, S3, GitHub, Bitbucket, …
• Which build environment to use
• How to get dependencies
• Which build command to execute
  • Or use the build spec file (buildspec.yml)
• How to run tests
• Where to store build outputs (artifacts)

Steps (Phases) in a Build Project

• Submitted
• Provisioning
• Download Source
• Install
• Pre-Build
• Build
• Post-Build
• Upload Artifacts
• Finalizing
• Completing

A “Buildspec” is a YAML file for CodeBuild to run a build.

• A collection of build settings and commands.
• You can include a buildspec file as part of the source code.
  • Must be named “buildspec.yml” and placed in the root folder of your source code.
• Or you can define Buildspec when you create a build project.

https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html

version: 0.2

run-as: Linux-user-name

env:
  shell: shell-tag
  variables:
    key: "value"
  parameter-store:
    key: "value"
  exported-variables:
    - variable
  secrets-manager:
    key: secret-id:json-key:version-stage:version-id
  git-credential-helper: no | yes

proxy:
  upload-artifacts: no | yes
  logs: no | yes

batch:
  fast-fail: false | true
  # build-list:
  # build-matrix:
  # build-graph:
        
phases:
  install:
    run-as: Linux-user-name
    on-failure: ABORT | CONTINUE
    runtime-versions:
      runtime: version
    commands:
      - command
    finally:
      - command
      - command
    # steps:
  pre_build:
    run-as: Linux-user-name
    on-failure: ABORT | CONTINUE
    commands:
      - command
    finally:
      - command
    # steps:
  build:
    run-as: Linux-user-name
    on-failure: ABORT | CONTINUE
    commands:
      - command
    finally:
      - command
    # steps:
  post_build:
    run-as: Linux-user-name
    on-failure: ABORT | CONTINUE
    commands:
      - command
    finally:
      - command
    # steps:

reports:
  report-group-name-or-arn:
    files:
      - location
    base-directory: location
    discard-paths: no | yes
    file-format: report-format

artifacts:
  files:
    - location
  name: artifact-name
  discard-paths: no | yes
  base-directory: location
  exclude-paths: excluded paths
  enable-symlinks: no | yes
  s3-prefix: prefix
  secondary-artifacts:
    artifactIdentifier:
      files:
        - location
      name: secondary-artifact-name
      discard-paths: no | yes
      base-directory: location
    artifactIdentifier:
      files:
        - location
      discard-paths: no | yes
      base-directory: location

cache:
  paths:
    - path
    - path

• Version
• Env
  • Environment variables
• Phases
  • Install
    • Commands that CodeBuild runs during installation
    • Runtime versions
  • Pre-Build
    • Commands to run before the build
  • Build
    • Commands to run during the build
  • Post-build
    • Commands to run after the build
  • Finally blocks (each phase)
    • These can be called after each section’s command blocks run even if a command in the command block fails.
• Artifacts
  • Artifacts represent a set of build artifacts that CodeBuild uploads to the output bucket.
  • discard-paths:
    • specifies if the build artifact directories are flattened in the output
    • No: The build artifacts are output with their directory structure intact.
    • Yes: The build artifacts are placed in the same directory.
  • base-directory:
    • represents one or more top-level directories relative to the original build location. CodeBuild uses this to determine which files and subdirectories to include in the build output artifact.
• Reports
  • Location where the test reports are stored
• Cache
  • Location where you can cache dependencies for reuse between projects
    • Can be an S3 location or local
  • Improves the build time by reducing the amount of dependencies downloaded
  • Can be invalidated

Caching

S3 Cache

• Stores the cache content in an S3 bucket
• good for small-sized build artifacts
  • not good for large artifact because it takes long time to transfer the content from S3 to the build environment
• does not support the Docker Layer cache mode

Local Cache

• Stores the cache content locally on a build host
• Source Cache mode
  • caches Git metadata
  • After the cache is created, subsequent builds pull only the change between commits.
• Docker Layer mode
  • caches existing Docker layers
  • good when the application is running in a large Docker image

Integration

• CodeCommit
  • You can run the CodeBuild project after the successful merge to CodeCommit via EventBridge.
• GitHub
  • In the CodeBuild, set the sources as GitHub
  • You can use webhook events:
    • Rebuild every time a code change is pushed to the repository
  • Customize the buildspec file

Build Notification

• EventBridge has built-in support for AWS CodeBuild.
• Use EventBridge and an SNS topic to notify subscribers of build events.

CodeBuild Inside VPC

By default, a CodeBuild container is launched outside of a VPC and you cannot access resources in a VPC.

You can launch a build container in a VPC by specifying:

• VPC ID
• Subnet IDs
• Security Group IDs

Build Badges

• A badge is a dynamically generated image that shows the status of a build.
• A badge is the branch-level.
• A badge can be accessed through a public URL of the CodeBuild project.

Validating the Pull Request

Once you integrate the CodeCommit and CodeBuild, you might want to validate your Pull Request (PR) before completing the merge.

1. A user creates a Pull Request
2. An event “pullRequestCreated” or “pullRequestSourceBranchUpdated” is passed to EventBridge.
3. (Optional) EventBridge Invokes a Lambda function to update the PR
   • Set the comment
4. EventBridge starts the CodeBuild Project
5. A build event (Success/Fail) or a test event (Success/Fail) is passed to EventBridge
6. EventBridge Invokes a Lambda function to update the PR
   • Set the PR completed or failed

Test Reports

• You can run tests during the build process.
• Create your test cases and create report files 
  • The following test report file formats are supported
    • Cucumber JSON (.json)
    • JUnit XML (.xml)
    • NUnit XML (.xml)
    • NUnit3 XML (.xml)
    • TestNG XML (.xml)
    • Visual Studio TRX (.trx)
• Add a Report Group in buildspec.yaml

reports:
 new-report-group: #report-group name for surefire junit reports
   files:
     - '**/*'
   base-directory: 'surefire/target/surefire-reports'
   file-format: 'JUNITXML'

• Test Report Metrics
  • Pass Rate %
  • Test Run Duration
  • Number of Passed versus Failed/Error test cases