Step Functions

Step Functions orchestrates workflows as state machines. Choose between Standard and Express based on your needs.

Standard workflows: Run up to one year, support approximately 4,000 executions/second, and are priced per state transition. They are ideal for long-running, auditable workflows like order processing, ETL pipelines, or approval flows. Every execution is logged in CloudWatch and has a unique execution history.

Express workflows: Run up to 5 minutes, support up to 100,000 executions/second, and are priced per execution plus duration. They are ideal for high-volume, short-duration workloads like IoT data ingestion, API orchestration, or real-time stream processing.

# Create a Standard workflow
aws stepfunctions create-state-machine \
  --name order-processing \
  --definition file://workflow.asl.json \
  --role-arn arn:aws:iam::123456789012:role/StepFunctionsRole

# Create an Express workflow
aws stepfunctions create-state-machine \
  --name data-ingestion \
  --definition file://workflow.asl.json \
  --type EXPRESS \
  --role-arn arn:aws:iam::123456789012:role/StepFunctionsRole

Key differences: Standard workflows give you full execution history and audit trails. Express workflows support higher throughput but sample logs. Use Standard for compliance-critical flows, Express for high-volume data processing.

A state machine consists of states connected by transitions. Each state has a type that determines its behavior.

Pass states pass input to output, optionally injecting fixed values. Useful for testing or setting default values.

Wait states pause execution for a fixed duration, until a specific timestamp, or until a timeout:

{
  "WaitForApproval": {
    "Type": "Wait",
    "Seconds": 3600,
    "Next": "CheckApproval"
  }
}

Choice states branch execution based on conditions:

{
  "CheckOrderTotal": {
    "Type": "Choice",
    "Choices": [
      {
        "Variable": "$.order.total",
        "NumericGreaterThan": 1000,
        "Next": "RequireManagerApproval"
      },
      {
        "Variable": "$.order.total",
        "NumericLessThanOrEqual": 1000,
        "Next": "AutoApprove"
      }
    ],
    "Default": "RequireManagerApproval"
  }
}

Choice states support string, numeric, boolean, and timestamp comparisons. Use And, Or, and Not for complex conditions. The Default transition handles unmatched cases.

Succeed and Fail states terminate execution with success or failure. Fail states include an error code and cause for debugging.

Parallel and Map states enable concurrent execution within a state machine.

Parallel state runs multiple branches simultaneously and waits for all to complete:

{
  "ProcessMultipleChannels": {
    "Type": "Parallel",
    "Branches": [
      {
        "StartAt": "SendEmail",
        "States": {
          "SendEmail": {
            "Type": "Task",
            "Resource": "arn:aws:lambda:...:send-email",
            "End": true
          }
        }
      },
      {
        "StartAt": "SendSMS",
        "States": {
          "SendSMS": {
            "Type": "Task",
            "Resource": "arn:aws:lambda:...:send-sms",
            "End": true
          }
        }
      }
    ],
    "Next": "LogCompletion"
  }
}

Each branch runs independently. Results are collected as an array. Use Catch on the Parallel state to handle failures in any branch.

Map state iterates over a list and runs the same workflow for each item:

{
  "ProcessAllItems": {
    "Type": "Map",
    "ItemsPath": "$.items",
    "MaxConcurrency": 10,
    "Iterator": {
      "StartAt": "ProcessItem",
      "States": {
        "ProcessItem": {
          "Type": "Task",
          "Resource": "arn:aws:lambda:...:process-item",
          "End": true
        }
      }
    },
    "Next": "AggregateResults"
  }
}

MaxConcurrency limits parallel iterations. Without it, all items process concurrently. The Map state output is an array of results from each iteration.

Step Functions provides built-in error handling with Retry and Catch clauses on any task or parallel state.

Retry automatically re-executes the failed state with exponential backoff:

{
  "ProcessPayment": {
    "Type": "Task",
    "Resource": "arn:aws:lambda:...:process-payment",
    "Retry": [
      {
        "ErrorEquals": ["States.TaskFailed"],
        "IntervalSeconds": 2,
        "MaxAttempts": 3,
        "BackoffRate": 2
      }
    ],
    "Catch": [
      {
        "ErrorEquals": ["States.ALL"],
        "ResultPath": "$.error",
        "Next": "HandleFailure"
      }
    ],
    "Next": "ShipOrder"
  }
}

Retry is attempted first. If all retry attempts fail, Catch takes over. ErrorEquals matches error names, use States.ALL to catch everything. ResultPath stores the error in the state output.

Lambda error handling: Lambda functions return custom errors by throwing exceptions with specific error names. Step Functions matches these against ErrorEquals:

class InsufficientFundsError(Exception):
    pass

def handler(event, context):
    if event['amount'] > event['balance']:
        raise InsufficientFundsError('Balance too low')

Set MaxAttempts to 0 for errors that should never be retried like validation errors. Use ResultPath to preserve the original input when transitioning to error states.

Task states perform work by invoking AWS services. Step Functions has native integrations that avoid writing wrapper code.

Lambda task: Invokes a Lambda function. The task result is the function return value:

{"Type": "Task", "Resource": "arn:aws:lambda:...:function-name", "End": true}

SQS task: Sends a message to an SQS queue:

{"Type": "Task", "Resource": "arn:aws:states:::sqs:sendMessage", "Parameters": {"QueueUrl": "...", "MessageBody": "$.data"}}

DynamoDB tasks: Put, get, update, or delete items:

{"Type": "Task", "Resource": "arn:aws:states:::dynamodb:putItem", "Parameters": {
  "TableName": "orders",
  "Item": {"orderId": {"S": "$.orderId"}, "status": {"S": "PROCESSING"}}
}}

ECS tasks: Run containers for long-running work. Use .sync to wait for the task to complete:

{"Type": "Task", "Resource": "arn:aws:states:::ecs:runTask.sync", "Parameters": {
  "Cluster": "my-cluster",
  "TaskDefinition": "process-order",
  "LaunchType": "FARGATE"
}}

Use .sync for services that have a completion state like Lambda, ECS, and Glue. Use .waitForTaskToken for callback-based integrations where the task explicitly signals completion.