Creating a Multi-Region AWS S3 Storage Setup with Terraform

In this article, we will dive into creating a multi-region AWS S3 storage setup using Terraform. AWS S3 is widely used for its scalable storage solution, and deploying a multi-region setup can help improve redundancy, reduce latency, and enhance performance. By leveraging Terraform, we can automate the creation and management of AWS resources in a consistent and repeatable manner.

Prerequisites

Before you proceed with the setup, ensure you have the following:

A Terraform environment set up on your local machine or CI/CD pipeline.
An active AWS account with the necessary permissions to create S3 buckets and configure cross-region replication.
Terraform AWS provider configured with access credentials.

Setting Up Terraform AWS Provider

The first step is to define the AWS provider in your Terraform configuration. The provider will allow Terraform to interact with AWS and create the necessary resources.

hcl
provider “aws” {
region = “us-east-1”
access_key = “”
secret_access_key = “”
}

Creating S3 Buckets in Multiple Regions

To create an S3 bucket in multiple regions, define multiple bucket resources in your Terraform configuration. The following example shows the configuration for two buckets: one in the US-East-1 region and another in the EU-West-1 region.

hcl
resource “aws_s3_bucket” “us_east_1_bucket” {
bucket = “my-unique-bucket-us-east-1”
region = “us-east-1”
acl = “private”
}

resource “aws_s3_bucket” “eu_west_1_bucket” {
bucket = “my-unique-bucket-eu-west-1”
region = “eu-west-1”
acl = “private”
}

Enabling Cross-Region Replication

Cross-region replication (CRR) enables the automatic, asynchronous copying of objects across AWS S3 buckets in different regions. Here, we’ll configure replication from the US-East-1 bucket to the EU-West-1 bucket. First, enable versioning on both the source and destination buckets, as CRR requires versioned buckets.

hcl
resource “aws_s3_bucket_versioning” “us_east_1_versioning” {
bucket = aws_s3_bucket.us_east_1_bucket.bucket
versioning_configuration {
status = “Enabled”
}
}

resource “aws_s3_bucket_versioning” “eu_west_1_versioning” {
bucket = aws_s3_bucket.eu_west_1_bucket.bucket
versioning_configuration {
status = “Enabled”
}
}

Configuring Replication Rule

Next, define the replication rule for the source bucket to automatically replicate objects to the destination bucket. The rule will specify the source and destination bucket, along with any filters, and the IAM role required for the replication task.

hcl
resource “aws_s3_bucket_replication_configuration” “replication” {
bucket = aws_s3_bucket.us_east_1_bucket.bucket

role = “arn:aws:iam::123456789012:role/replication-role”

rules {
id = “ReplicationRule”
status = “Enabled”
filter {
prefix = “docs/”
}

destination {
bucket = aws_s3_bucket.eu_west_1_bucket.arn
storage_class = “STANDARD”
}
}
}

Creating IAM Role for Replication

For replication to work, you need to create an IAM role with the appropriate permissions. This role must allow S3 to read from the source bucket and write to the destination bucket.

h
resource “aws_iam_role” “replication_role” {
name = “replication-role”

assume_role_policy = jsonencode({
Version = “2012-10-17”
Statement = [
{
Action = “sts:AssumeRole”
Principal = {
Service = “s3.amazonaws.com”
}
Effect = “Allow”
Sid = “”
},
]
})
}

resource “aws_iam_policy” “replication_policy” {
name = “replication-policy”
description = “Allow S3 replication”
policy = jsonencode({
Version = “2012-10-17”
Statement = [
{
Action = [
“s3:GetObjectVersion”,
“s3:ReplicateObject”,
“s3:ReplicateDelete”
]
Effect = “Allow”
Resource = “arn:aws:s3:::${aws_s3_bucket.us_east_1_bucket.bucket}/*”
},
{
Action = [
“s3:PutObject”,
“s3:PutObjectAcl”
]
Effect = “Allow”
Resource = “arn:aws:s3:::${aws_s3_bucket.eu_west_1_bucket.bucket}/*”
},
]
})
}

resource “aws_iam_role_policy_attachment” “replication_policy_attachment” {
policy_arn = aws_iam_policy.replication_policy.arn
role = aws_iam_role.replication_role.name
}

Deploying the Configuration

Once your Terraform configuration is set up, you can deploy the resources using the following commands:

terraform init – Initializes the Terraform working directory.
terraform plan – Displays the execution plan, showing what changes will be made.
terraform apply – Applies the configuration to create the resources in AWS.

Verifying the Replication

After applying the Terraform configuration, upload an object to the source bucket (US-East-1), and verify that it is automatically replicated to the destination bucket (EU-West-1). You can check the replication status in the AWS Management Console or by using the AWS CLI.

aws s3 cp myfile.txt s3://my-unique-bucket-us-east-1/docs/
aws s3 ls s3://my-unique-bucket-eu-west-1/docs/

Conclusion

With this Terraform configuration, you can easily set up a multi-region AWS S3 storage solution with cross-region replication. By utilizing versioned S3 buckets and defining appropriate IAM roles and policies, you ensure that your data is replicated seamlessly across different regions.

We earn commissions using affiliate links.