AWS Step Functions AI Orchestration

Overview

This project demonstrates an intelligent AI model orchestration system using AWS Step Functions. It automatically selects the most cost-effective AI model based on input complexity, providing an optimal balance between performance and cost.

Key Features

• Smart model selection based on input complexity
• Integration with multiple AI models:
  • Amazon Bedrock (Claude Instant and Sonnet)
  • OpenAI GPT-3.5
• Comprehensive error handling and retries
• Serverless architecture

Prerequisites

• AWS Account with appropriate permissions
• OpenAI API key
• Terraform >= 1.0
• Python 3.12
• AWS CLI configured

Installation

1. Clone the repository:

git clone https://github.com/your-repo/aws-step-functions-ai-orchestration
cd aws-step-functions-ai-orchestration

2. Set up OpenAI API key in AWS Systems Manager Parameter Store:

aws ssm put-parameter \
    --name "/ai-orchestration/openai-api-key" \
    --type "SecureString" \
    --value "your-openai-api-key"

3. Deploy the infrastructure:

cd terraform
terraform init
terraform plan
terraform apply

Usage

Execute the Step Functions workflow using AWS CLI:

# Get the state machine ARN
export STATE_MACHINE_ARN=$(terraform output -raw state_machine_arn)

# Execute with a test input
aws stepfunctions start-execution \
    --state-machine-arn $STATE_MACHINE_ARN \
    --input '{"input": "What is the capital of France?"}'

Example Inputs

1. Simple Query:

{
  "input": "What is the capital of France?"
}

2. Calculation:

{
  "input": "Calculate 15% of 850 and add 100"
}

3. Complex Analysis:

{
  "input": "Analyze the impact of artificial intelligence on healthcare"
}

Monitoring

• Monitor execution costs per model
• Track performance metrics
• View execution history in Step Functions console

Documentation

• Architecture Guide
• Cost Optimization
• Manual Execution Guide

License

This project is licensed under the MIT License - see the LICENSE.md file for details.