from-terraform-workspaces.md7.9 KB

Migrating from Terraform Workspaces

This reference is the agent's decision guide for users coming from a terraform.workspace-driven setup. For the full user-facing prose tutorial, see atmos.tools/migration/terraform-workspaces.

The Two Migration Paths

There are two paths, and choosing wrong costs the user state. Identify which one fits their situation before proposing anything:

Situation	Path
User wants to keep existing workspace state intact, migrate gradually	Path 1: Keep workspaces
User explicitly wants separate backend configuration, or policy requires it	Path 2: Move to separate backends

Default guidance: Path 1 first to get the user running on Atmos with zero state risk, and only discuss Path 2 if the user asks for separate backends or has an operational policy that requires them. Do not make state migration a prerequisite for adopting Atmos.

Mapping `terraform.workspace` → Atmos Stack

The cleanest mapping is: one workspace → one stack file. The workspace name typically becomes the stack name (dev, staging, prod).

If the user has a workspace naming convention you need to preserve (e.g., the upstream state was written with workspaces named tenant-environment-stage-component), use metadata.terraform_workspace on the component to override the workspace name Atmos derives:

components:
  terraform:
    vpc:
      metadata:
        terraform_workspace: '{{ .vars.tenant }}-{{ .vars.environment }}-{{ .vars.stage }}-{{ .atmos_component | regexp.ReplaceLiteral "\\W" "-" }}'

This is critical when the legacy workspace name does not match Atmos's default workspace derivation. Without it, Atmos will create a new (empty) workspace and the user will think their state vanished.

Replacing `terraform.workspace` Ternaries

The single largest source of code in workspace-based repos is conditional logic keyed on terraform.workspace. Replace it with stack-level vars.

Before (workspace logic in .tf):

locals {
  instance_type     = terraform.workspace == "prod" ? "m5.large" : "t3.small"
  enable_monitoring = terraform.workspace == "prod" ? true : false
  backup_retention  = terraform.workspace == "prod" ? 30 : 7
}

After (config moves to stack YAML, code stays generic):

variable "instance_type"     { type = string }
variable "enable_monitoring" { type = bool }
variable "backup_retention"  { type = number }

# stacks/prod.yaml
components:
  terraform:
    app:
      vars:
        instance_type: m5.large
        enable_monitoring: true
        backup_retention: 30

# stacks/dev.yaml
components:
  terraform:
    app:
      vars:
        instance_type: t3.small
        enable_monitoring: false
        backup_retention: 7

Once the conditionals are gone, the Terraform code is generic and reusable across any number of stacks without further changes.

Path 1: Keep the Workspace State (Easiest)

For users who already have valuable state in workspaces and don't want to migrate it, Atmos can read and write to the existing workspace structure unchanged.

# stacks/prod.yaml
terraform:
  backend_type: s3
  backend:
    s3:
      bucket: terraform-state           # Same bucket as before
      key: vpc/terraform.tfstate        # Same key
      region: us-east-1
      workspace_key_prefix: env         # Matches existing workspace convention

components:
  terraform:
    vpc:
      metadata:
        terraform_workspace: prod       # Selects the existing workspace
      vars:
        cidr_block: "10.100.0.0/16"
        environment: prod

Critical: The user's metadata.terraform_workspace value must exactly match the existing workspace name in their state backend. If their workspaces are named env:prod vs prod vs production, the value here must match exactly or Atmos will operate against an empty workspace.

After this works for one stack, repeat for dev, staging, etc. The user has zero state risk and can now use atmos terraform plan/apply against their existing infrastructure.

Path 2: Move to Separate Backends (Optional)

This is an optional operational change for teams that explicitly want independent backend configuration per environment, or have an internal policy requiring separate state backends. Atmos does not require this layout, and users do not need it to migrate to Atmos.

This requires a one-time state migration per workspace. Walk the user through it carefully -- it is the highest-risk step in the whole migration.

Export state from each workspace:

terraform workspace select prod
terraform state pull > prod.tfstate

Configure the new per-stack backend in Atmos:

# stacks/prod.yaml
terraform:
  backend_type: s3
  backend:
    s3:
      bucket: terraform-state-prod   # Dedicated bucket
      key: vpc.tfstate
      region: us-east-1

Initialize the new backend and push state:

atmos terraform init vpc -s prod
terraform state push prod.tfstate

Verify with a plan -- it must show zero changes:

atmos terraform plan vpc -s prod

Repeat for each workspace.

If the team moves all environments to separate backends, they can delete the old workspaces from the original backend after confirming no other tooling references them.

Real Environments vs Per-Developer Sandboxes

Users sometimes conflate two distinct uses of workspaces:

Environment workspaces (dev, staging, prod) -- these become Atmos stacks. Use either migration path above.
Per-developer sandbox workspaces (alice-test, bob-experiment) -- these are usually short-lived and should not become long-lived stacks. Convert them to ephemeral stacks created on demand (e.g., via atmos terraform apply <comp> -s sandbox --var "owner=alice") or have developers create per-feature stack files (stacks/sandbox-alice.yaml).

If the user has hundreds of sandbox workspaces from drift over years, treat them as state to audit-and-delete, not state to migrate.

Reading State from Un-migrated Workspaces

If the user is migrating component-by-component, they will often need a new Atmos component to read outputs from a TF root module that still uses workspaces. The remote-state-bridge.md pattern handles this -- specifically Variant A with metadata.terraform_workspace set to the legacy workspace name and backend.s3 pointing at the legacy state file.

CI/CD Update

Replace workspace selection in CI with stack arguments:

Before:

terraform workspace select $ENV
terraform plan
terraform apply -auto-approve

After:

atmos terraform plan $COMPONENT -s $STACK
atmos terraform apply $COMPONENT -s $STACK -auto-approve

For the broader CI/CD setup (affected detection, native Atmos CI container, GitHub Actions patterns), route to the atmos-ci skill.

Common Mistakes

Wrong workspace name in metadata.terraform_workspace -- silently operates against an empty workspace. Always cross-check against terraform workspace list output.
Migrating state without a backup -- always terraform state pull > backup.tfstate before any push or backend change.
Removing workspace logic from .tf before stacks are ready -- the user's existing terraform apply will break. Add the variable declarations alongside the locals, then remove locals only after stacks are populated.
Treating Path 2 as required -- Path 1 unblocks Atmos adoption without state migration. Only move to separate backends when the user explicitly wants that operating model.