Store Terraform State in S3 (with DynamoDB Locking)

Why S3 for Terraform State

Terraform supports multiple backend options, but S3 is the natural choice for AWS-based teams. S3 provides 99.999999999% durability (11 nines), meaning the probability of losing your state file due to storage failure is effectively zero. It's fully managed, globally available, and integrates natively with AWS security services.

S3 offers features purpose-built for state management:

Versioning keeps every state file update as a separate object version. If someone breaks state with a bad apply, you can restore the previous version. No data loss from overwrites.

Server-side encryption protects state files at rest. Terraform state contains resource IDs, configuration details, and sometimes secrets. Encryption isn't optional.

Access control via IAM lets you restrict who can read or write state. Terraform processes get exactly the permissions they need. Everyone else gets denied.

CloudTrail integration logs every access to the state bucket. You can audit who read state, when they wrote it, and trace infrastructure changes back to specific users or CI jobs.

For AWS workloads, S3 is the preferred backend. It's already in your account, costs pennies per month for state storage, and handles all the durability and availability concerns that make local state unreliable.

State Locking with DynamoDB

State files are mutable. If two terraform apply runs execute concurrently without coordination, both read the same starting state, make different changes, then write back their versions. The second write wins, silently discarding the first. Resources get lost. State becomes corrupted. Recovery is painful.

State locking prevents this by allowing only one Terraform process to modify state at a time. Before any write operation, Terraform acquires a lock. Other processes attempting to acquire the lock wait or fail. After completion, Terraform releases the lock.

DynamoDB provides the distributed lock mechanism for S3 backends. It's a NoSQL database with conditional writes, which Terraform uses to implement a lock table. The setup is simple: create a table with a primary key named LockID, and Terraform handles the rest.

The DynamoDB table stores a single item per concurrent operation (usually just one item most of the time). Costs are negligible with on-demand billing. The lock item contains metadata about who holds the lock and when it was acquired, making stuck locks easy to debug.

Note on S3 native locking: Terraform 1.10+ supports native S3 locking via use_lockfile = true, which creates a .tflock object instead of using DynamoDB. This guide covers DynamoDB locking as it remains the proven, widely-deployed method. Both work fine; DynamoDB has broader adoption in existing infrastructure.

Step 1: Create the S3 Bucket

First, create an S3 bucket to store state files. Choose a unique name following your organization's naming convention. The bucket should live in the region where your Terraform runs execute.

This configuration creates a bucket with three critical features:

Versioning preserves every state file update. Each terraform apply creates a new object version instead of overwriting. If you need to roll back, retrieve the previous version from S3's version history.

Encryption at rest via SSE-S3 (AES-256). All state files stored in the bucket are encrypted automatically. For additional control, use SSE-KMS with a customer-managed key instead of AES256.

Public access block ensures the bucket and its objects are never publicly accessible, regardless of bucket policies or ACLs. This prevents accidental exposure of state files.

Optional: Enforce HTTPS-Only Access

Add a bucket policy that denies any non-HTTPS requests, ensuring encryption in transit:

Step 2: Create the DynamoDB Table

Create a DynamoDB table with a single attribute: LockID as the partition key. Terraform uses this to coordinate state locks across concurrent operations.

Key configuration details:

Table name: terraform-state-locks (can be anything, but make it descriptive).

Billing mode: PAY_PER_REQUEST is ideal for state locking. Lock writes happen infrequently (only during applies), so you pay pennies per month instead of provisioning fixed read/write capacity.

Primary key: Must be named LockID with type String. Terraform expects exactly this attribute name. No sort key or secondary indexes needed.

That's it. DynamoDB will store one item when a lock is held, zero items when no operations are running. Terraform handles all the conditional write logic to acquire and release locks.

Step 3: Configure IAM Permissions

The IAM role or user running Terraform needs specific permissions to the S3 bucket and DynamoDB table. Follow least-privilege principles: grant only the exact actions required, scoped to the specific resources.

IAM Policy for S3 Backend

Replace ACCOUNT_ID with your AWS account ID and us-east-1 with your region.

S3 permissions breakdown:

s3:ListBucket on the bucket resource lets Terraform check if state files exist.

s3:GetObject and s3:PutObject on state file objects allow reading and writing state. The resource ARN restricts access to terraform.tfstate files only, not the entire bucket.

DynamoDB permissions breakdown:

dynamodb:DescribeTable lets Terraform verify the table exists.

dynamodb:GetItem checks if a lock exists.

dynamodb:PutItem acquires a lock (conditional write).

dynamodb:DeleteItem releases a lock.

Attach this policy to the IAM role your CI/CD pipeline assumes, or to the IAM user developers use for Terraform operations. Scope permissions as narrowly as possible.

Step 4: Configure the Terraform Backend

With infrastructure in place, configure Terraform to use the S3 backend. Add a backend block to your Terraform configuration (typically in a file named backend.tf or within your main configuration).

Configuration parameters:

bucket — The S3 bucket name created in Step 1.

key — The path and filename for the state file within the bucket. Use descriptive paths like environment/component/terraform.tfstate to organize multiple states.

region — AWS region where the bucket exists.

dynamodb_table — The DynamoDB table name created in Step 2.

encrypt = true — Enables SSE-S3 encryption for state files uploaded to S3.

Using KMS for Encryption

For customer-managed encryption keys, specify the KMS key ARN:

The IAM role running Terraform needs kms:Encrypt, kms:Decrypt, and kms:GenerateDataKey permissions on the specified KMS key.

Migrate Local State to S3

After adding the backend configuration, initialize Terraform to migrate existing local state to S3:

Terraform uploads your local terraform.tfstate to S3 at the specified key. After migration, delete your local state files—they're no longer the source of truth. All future operations use the remote backend.

Step 5: Verify Locking Works

Test that state locking prevents concurrent modifications. Open two terminals and run terraform apply simultaneously in both:

The second apply blocks until the first releases the lock. This serialized execution prevents state corruption from concurrent writes.

If a lock gets stuck (process crashed, CI job killed), force-unlock using the lock ID:

Only use force-unlock after confirming no other Terraform process is actually running. The lock ID acts as a safety check to prevent unlocking the wrong lock.

Security Best Practices

S3 backends store sensitive infrastructure data. Securing the state bucket and DynamoDB table is not optional for production environments.

Enable CloudTrail Logging

CloudTrail logs all API calls to S3 and DynamoDB. Enable data events for your state bucket to audit who accessed state files:

CloudTrail records GetObject and PutObject calls on the state bucket. Query logs to determine who read or modified state, when they did it, and from what source IP. This audit trail is invaluable for compliance and incident investigation.

Separate State per Environment

Use different S3 keys (or separate buckets) for dev, staging, and production environments. This isolation prevents a mistake in one environment from affecting another.

Example key structure:

Never Commit Credentials to Code

Backend configuration should not contain AWS access keys. Use IAM roles in CI/CD, AWS CLI profiles for local development, or environment variables. Terraform authenticates to S3 and DynamoDB using the ambient AWS credentials.

For CI/CD pipelines, configure the job to assume an IAM role with permissions to the state bucket and lock table. Supply the role ARN via environment variables or pipeline configuration, not in Terraform code.

Restrict Access to State Bucket

Only Terraform processes should have access. Developers don't need direct S3 access to the state bucket—all operations go through Terraform CLI, which handles state reads and writes.

Lock down the S3 bucket and DynamoDB table to specific IAM principals. Use SCPs or permission boundaries if needed to prevent privilege escalation.

Common Issues and Solutions

Error: Failed to Get State Lock

This occurs when another Terraform process holds the lock. Wait for the other operation to complete, or if it's stuck (crashed process), use terraform force-unlock with the lock ID from the error message.

Error: AccessDenied on S3 Bucket

The IAM role/user running Terraform lacks required S3 permissions. Verify the policy includes s3:ListBucket on the bucket and s3:GetObject/s3:PutObject on the state object path.

Error: ResourceNotFoundException on DynamoDB Table

The DynamoDB table specified in the backend config doesn't exist, or it exists in a different region. Verify the table name and region match your backend configuration.

State File Corrupted

If state becomes corrupted, retrieve a previous version from S3's version history. Navigate to the bucket in the AWS Console, select the state object, and restore a prior version. Versioning must be enabled for this to work (which is why Step 1 enabled it).

Wrong Region Configuration

Terraform can't find the S3 bucket because the region in the backend config doesn't match where the bucket exists. S3 bucket names are globally unique, but buckets themselves live in specific regions. Ensure region in your backend block matches the bucket's region.

The Bottom Line

Local state files don't survive team collaboration, CI/CD pipelines, or laptop failures. S3 with DynamoDB locking provides the durable, versioned, encrypted remote state backend that production Terraform requires.

The setup is straightforward: create an S3 bucket with versioning and encryption, create a DynamoDB table with LockID as the primary key, configure IAM policies following least privilege, and add a backend block to your Terraform configuration. After terraform init -migrate-state, your local state moves to S3 and all future operations use the remote backend.

Enable CloudTrail logging for audit trails. Separate state by environment using different S3 keys. Never commit AWS credentials to code. Restrict bucket access to Terraform processes only.

Teams that configure remote state properly gain reliable collaboration, protection against concurrent modifications, and recovery options when things go wrong. Teams that skip it deal with corrupted state, lost infrastructure mappings, and 2 AM recovery efforts.