Skip to content

Installation

This guide covers the installation of Redshift Spectra for both development and production environments.

Prerequisites

Before installing Redshift Spectra, ensure you have:

Requirement Version Purpose
Python 3.11+ Runtime environment
uv Latest Package management
Docker Latest LocalStack & Lambda layer building
Terraform >= 1.5 Infrastructure as Code
Terragrunt >= 0.99 DRY Terraform configuration
AWS CLI 2.x AWS credential management (for AWS deployments)

Installation Methods

For local development using LocalStack (no AWS account required):

# Clone the repository
git clone https://github.com/zhiweio/redshift-spectra.git
cd redshift-spectra

# Install all dependencies (including dev tools)
task setup:install-dev

# Configure environment (defaults work for LocalStack)
cp .env.example .env

# Start LocalStack and deploy infrastructure
task local:deploy

# Verify deployment
task local:status

See LocalStack Setup for detailed local development instructions.

AWS Development Setup

For deploying to AWS:

# Clone the repository
git clone https://github.com/zhiweio/redshift-spectra.git
cd redshift-spectra

# Install all dependencies
task setup:install-dev

# Configure for AWS
cp .env.example .env
# Edit .env with your AWS Redshift settings

# Build and deploy
task build:all
task infra:apply-dev

Production Setup

For production deployments, install only runtime dependencies:

# Install production dependencies only
task setup:install

# Build Lambda packages
task build:all

AWS Credentials

Configure AWS credentials with sufficient permissions:

# Configure AWS CLI
aws configure

# Or use environment variables
export AWS_ACCESS_KEY_ID=your_access_key
export AWS_SECRET_ACCESS_KEY=your_secret_key
export AWS_DEFAULT_REGION=us-east-1

Required IAM Permissions

Your AWS credentials need permissions for:

  • Lambda (create, update, invoke)
  • API Gateway (create, manage)
  • DynamoDB (create tables, read/write)
  • S3 (create bucket, read/write)
  • Secrets Manager (read secrets)
  • Redshift Data API (execute statements)
  • CloudWatch (logs, metrics)
  • IAM (create roles, policies)

Use Least Privilege

For production, create a dedicated IAM user/role with only the permissions needed. See Security Best Practices.

Environment Configuration

Create a .env file from the template:

cp .env.example .env

For LocalStack (Local Development)

The default values work out of the box:

# Environment is set to local by default
ENVIRONMENT=local

# LocalStack endpoint
LOCALSTACK_ENDPOINT=http://localhost:4566

# Credentials (any non-empty value works)
AWS_ACCESS_KEY_ID=test
AWS_SECRET_ACCESS_KEY=test

For AWS (Dev/Prod)

Edit the .env file with your AWS configuration:

# Required settings
ENVIRONMENT=dev
SPECTRA_REDSHIFT_CLUSTER_ID=my-redshift-cluster
SPECTRA_REDSHIFT_DATABASE=mydb
SPECTRA_REDSHIFT_SECRET_ARN=arn:aws:secretsmanager:...
SPECTRA_S3_BUCKET_NAME=my-spectra-bucket

# Optional settings (with defaults)
SPECTRA_AWS_REGION=us-east-1
SPECTRA_LOG_LEVEL=INFO

See Configuration Reference for all available options.

Verify Installation

Run the test suite to verify everything is working:

# Run all tests
task test:all

# Run with coverage
task test:cov

# Run linting
task lint:check

Next Steps