[AWS] Step Functions

“Step Functions” is a fully managed long-running, serverless workflow service that provides state machines. It provides a visual interface for you to build and run serverless applications as a series of steps.

Features

AWS Step Functions is a fully managed service that makes it easy to coordinate the components of distributed applications and microservices using visual workflows. It helps you building application from individual components that each perform a discrete function, which lets you scale the applications easily.

• Step Functions is a reliable way to coordinate components and step through the functions of your application. Step Functions provides a graphical console to arrange and visualize the components of your application as a series of steps.
• Step Functions logs the state of each step, so when things do go wrong, you can diagnose and debug problems quickly.

Using Step Functions

Step Functions automatically triggers and tracks each step, and retries when there are errors, so your application runs in order and as expected.

• Each step in your application executes in order, as defined by your business logic.
• Each function can be implemented as a Lambda function.
• The output of one step may act as an input to the next.
• Step Functions ensures your application to be executed in order the sequential logic by using sequencing, error handling, and retry logic.
  • The state of each step is logged, so you can track what/where went wrong.

Workflows

• Standard
  • can run for up to one year
    • used for asynchronous processing
  • exactly-once execution
    • Non-idempotent
  • Useful for long-running works that need to have an auditable history

• Express
  • at-least-once execution
    • No state management – should be idempotent
  • can run for up to only five minutes
  • useful for high-event-rate workloads such as IoT data streaming
  • Synchronous Express Workflow
    • Begins a workflow, waits until it completes, and returns a result
  • Asynchronous Express Workflow
    • Begins a workflow and confirms it has started
    • The result can be found in CloudWatch Logs

State Machine

Workflow steps are known as states, and they can perform work via tasks.

A state machine can be defined using JSON-based Amazon States Language (ASL). State machines maintain states (Lambda is a shot-running stateless function) and allow longer running processes.

• Tasks are performing actions.
• Activity is a program code that interacts with Step Functions using API actions
  • CreateStateMachine
  • StartExecution
  • StopExecution
  • GetActivityTask
  • SendTaskSuccess
  • SendTaskFailure
• Lambda function responds to the state machine tasks.

States

Each state in a state machine makes decisions based on inputs, perform actions, and passes output.

State Types

• Pass
  • passes any input directly to its output with no work done
• Task
  • represents a single unit of work performed, such as Lambda or SNS
• Choice
  • adds a branching decision logic
• Wait
  • creates a specified time delay by pausing the process
  • use the “Wait” state instead of a Lambda function to wait for a work to complete
• Succeed
  • completes the execution successfully
• Fail
  • stops the execution and marks it as failed
• Parallel
  • runs parallel branches of executions
  • waits until all branches terminate
• Map
  • runs a set of steps based on elements of an input array

You can retry the task by specifying the “Retry” configuration of the state.

"Get Job Status": {
  "Type": "Task",
  ...
  "Retry": [
    {
      "ErrorEquals": ["States.ALL"],
      "IntervalSeconds": 1,
      "MaxAttempts": 3,
      "BackoffRate": 2
    }
  ]
},

Step Functions vs. Simple Workflow

Step functions replace SWF (Simple Workflow) with a serverless version.

• To make a decision, SWF uses a decider program, and Step Functions use a JSON-based state machine.
• “Step Functions” provides the visual workflows.
• “Step Functions” orchestrates multiple AWS resources.
• SWF provides complete control over the workflow but increases the complexity.