# Component Retry Configure per-component retry behavior so transient failures — provider download 502s, S3 backend timeouts, registry rate limits — recover automatically without manual re-runs. Terraform, Helmfile, Packer, and Ansible commands often fail with transient infrastructure errors that have nothing to do with the component's own configuration. The most common is a 502 Bad Gateway during `terraform init`: ```text Error: Failed to install provider … 502 Bad Gateway returned from https://github.com/opentofu/terraform-provider-local/releases/... ``` Configure a `retry` block on the component and Atmos will retry each subprocess invocation (init, workspace, plan/apply, sync, build, playbook run — whatever the component type shells out to) when the captured output matches one of the regex patterns you list as recoverable. ## Supported component types `retry` works the same way for every component type that shells out to a binary: **Terraform, Helmfile, Packer, and Ansible**. Native **Kubernetes** and native **Helm** components do not shell out — they call Go SDKs directly, with no subprocess to capture and pattern-match. `retry` is not wired up for them yet. ## Usage ```yaml components: terraform: vpc: retry: max_attempts: 5 backoff_strategy: exponential initial_delay: 2s max_delay: 30s conditions: - /Bad Gateway/ - /GOAWAY/ - /5\d\d / - /connection reset/ - /TLS handshake timeout/ - /could not query provider registry/ ``` ## How it works When `retry.conditions` is configured, each call Atmos makes to the component's underlying binary (terraform/tofu, helmfile, packer, or ansible-playbook) is wrapped in an independent retry loop: 1. The subprocess runs as usual; its stdout and stderr stream to the user **and** are captured into an in-memory buffer. 2. When the subprocess exits with a non-zero status, the captured output is matched against every regex in `conditions`. 3. If at least one pattern matches, Atmos waits for the configured backoff and retries. 4. If no pattern matches, the error is returned immediately — real failures (`terraform plan` exit-code 2, schema errors, permission denials) are never silently retried. Each subprocess invocation has its own retry loop. Retrying `terraform init` does not consume the `apply` retry budget — the same isolation applies across any two invocations for any component type. ## Provider registry failures Provider downloads can fail during `init` in more than one way. The example above shows a 502 Bad Gateway. The registry can also close the HTTP/2 connection with a GOAWAY error: ```text Error: Failed to resolve provider packages Could not resolve provider cloudposse/awsutils: could not query provider registry for registry.opentofu.org/cloudposse/awsutils: http2: server sent GOAWAY and closed the connection; LastStreamID=5, ErrCode=NO_ERROR, debug="" ``` Both errors come from a temporary connection problem with the registry. They are not code problems. The `conditions` list above already matches both errors, through `/could not query provider registry/` and `/GOAWAY/`. This type of error can fail many components at the same time. A drift-detection run, a bulk `apply`, or a CI job can query the registry for many components in a short time window. One registry connection problem can then fail the whole batch. You have four ways to share one retry policy across those components: 1. Configure `retry` once at the stack-manifest root — see [Stack-Level Defaults](#stack-level-defaults). This applies to every supported component in the stack. No mixin, no shared base component, no per-component copies. 2. Configure `retry` on each affected component. 3. Configure `retry` on one shared abstract base component — see [Inheritance](#inheritance). This only works when the components already share a base. 4. Configure `retry` once in a [mixin](/howto/mixins), using `overrides.retry`. This works even when the components do not share a base component, and lets you scope the policy to part of a stack instead of all of it. Any of the four lets each component recover on its own, independently, within its own `max_attempts` or `max_elapsed_time` budget. A component recovers only if the registry error clears before its budget runs out. If the outage lasts longer than that, the component still fails even with `retry` configured — a long enough outage can still fail part or all of the batch. You do not need to start a manual re-run for any component that recovers within its budget. [`atmos terraform cache`](/cli/commands/terraform/cache) reduces the number of calls Atmos makes to the upstream registry. The cache does not retry a failed call. A cache miss still sends one call to the registry and returns any error from that call. Retry and the cache work together: the cache reduces how often you meet a registry problem, and `retry` recovers when you do. ## Arguments - **`conditions`** A list of regex patterns matched against captured stdout/stderr. Only errors whose output matches at least one pattern are retried. Patterns may be wrapped in /.../ for readability. Without `conditions`, no retry is attempted — this is a safety default so a typo never silently retries real failures. - **`max_attempts`** Maximum number of attempts, including the first. Omit for unlimited; defaults to 1 (no retry) when `retry` is unset entirely. - **`backoff_strategy`** One of constant , linear , or exponential . Defaults to exponential when initial_delay is set. - **`initial_delay`** Delay before the second attempt as a Go duration string (e.g. "2s" , "500ms" ). - **`max_delay`** Upper bound on any single delay; caps exponential growth (e.g. "30s" ). - **`max_elapsed_time`** Total time budget across all attempts (e.g. "5m" ). After this the most recent error is returned regardless of `max_attempts` . - **`multiplier`** Growth multiplier for exponential backoff. Defaults to 2.0 . - **`random_jitter`** Fractional jitter applied to each delay (0–1). Useful to avoid thundering-herd retries from CI fleets. ## Inheritance The `retry` block participates in component inheritance. Abstract components can define a default retry policy that concrete components inherit and override: ```yaml components: terraform: base/network: metadata: type: abstract retry: max_attempts: 3 initial_delay: 2s backoff_strategy: exponential conditions: - /Bad Gateway/ - /GOAWAY/ - /5\d\d / vpc: metadata: inherits: - base/network # vpc inherits the base retry block. `metadata.inherits` pulls in # base/network's config; `metadata.component` alone does not — it only # selects which Terraform source to use, not which config to inherit. transit-gateway: metadata: inherits: - base/network retry: # Scalars are replaced: max_attempts becomes 5. `conditions` is a # list, so it follows settings.list_merge_strategy — by default # (replace) this list replaces the base's conditions entirely, # leaving only /TLS handshake timeout/. Set # `settings.list_merge_strategy: append` to accumulate conditions # across layers instead of replacing them. max_attempts: 5 conditions: - /TLS handshake timeout/ ``` ## Stack-Level Defaults Configure `retry` once at the root of a stack manifest to apply the same policy to every supported component in that stack. No mixin, no shared base component, no per-component copies: ```yaml title="stacks/deploy/prod.yaml" retry: max_attempts: 5 backoff_strategy: exponential initial_delay: 2s max_delay: 30s conditions: - /Bad Gateway/ - /GOAWAY/ - /could not query provider registry/ components: terraform: vpc: # ... transit-gateway: # ... ``` Precedence, lowest to highest: stack-root `retry` → abstract base component `retry` → concrete component `retry` → `overrides.retry`. A more specific layer wins on a conflicting key. Non-conflicting keys from lower layers still apply — a component that only sets its own `max_attempts` still inherits the stack-root `conditions` list. Stack-root `retry` is scoped to the manifest file, and to any file that imports it — the same scoping rule as global `vars`, `metadata`, and `hooks`. Use a [mixin](/howto/mixins) instead when the policy should apply to only some components in a stack, not the whole stack. ### Opting a component out `retry` merges as a deep merge, like every other section. An empty override does not turn retry off: ```yaml components: terraform: vpc: retry: {} # This does NOT disable an inherited retry policy. An empty map # contributes no keys to the merge, so the component still gets the # full stack-root or base-component retry policy unchanged. ``` To opt a single component out of an inherited policy, set `max_attempts: 1` on that component. A budget of one attempt means the first failure is returned immediately, regardless of `conditions`: ```yaml components: terraform: vpc: retry: max_attempts: 1 ``` ## When NOT to use retry - **Real Terraform failures.** A `plan` that exits with code 2 because of a configuration error or a `validate` that catches a typo should fail loudly. Keep `conditions` narrowly scoped to transient infra patterns. - **Stateful mutations with non-idempotent side effects.** If a third-party API is invoked from a `local-exec` provisioner and is not safe to retry, do not configure retry conditions that could match its error output. - **Hooks.** Atmos hooks have their own configuration and are not affected by component retry. ## See also - [Workflow retry](/workflows/steps/retry) — retry configuration for multi-step workflows. - [Vendor retry](/vendor/config/sources) — retry configuration for component vendoring. - [Terraform Registry Cache](/cli/commands/terraform/cache) — reduces calls to the registry. It does not retry on its own. - [Mixins](/howto/mixins) — share a retry policy across components that do not have a common base component.