Skip to main content
yaml-functions.md14.5 KB
View on GitHub

YAML Functions Complete Reference

YAML functions are the recommended way to add dynamic behavior to Atmos stack configurations. They use YAML explicit tags (the ! prefix) and execute after YAML parsing, making them type-safe, predictable, and unable to break YAML syntax.

All YAML functions support Go template expressions in their arguments. Atmos processes templates first, then executes the YAML functions.

!labels — Component metadata lookup

vars:
labels: !labels # Complete metadata.labels map
runner: !labels runner # One literal, case-sensitive key
owner: !labels owner "Platform Team" # Fallback only when the key is absent
optional: !labels missing "" # Explicit empty fallback

The lookup uses resolved metadata after defaults and inheritance. Bare !labels returns {} when labels are absent; a missing key without a fallback is an error. An existing empty string counts as present. Keys with dots, slashes, and hyphens are literal, not nested paths. The function accepts at most two arguments; quote defaults containing spaces.

For templates, use {{ .metadata.labels.runner }} or {{ index .metadata.labels "cost-center" }}. There is no built-in .labels shortcut. Functions also work in settings.pro workflow inputs; use !labels runner ubuntu-latest for per-component runner selection with a fallback. This does not provision GitHub runners.

!terraform.state (Recommended)

Read Terraform outputs directly from the state backend without initialization. This is the recommended way to read remote state in Atmos -- it reads the state file directly from the backend, skipping terraform init and provider downloads entirely.

Syntax

# Current stack
!terraform.state <component> <output>
!terraform.state <component> <yq-expression>

# Specific stack
!terraform.state <component> <stack> <output>
!terraform.state <component> <stack> <yq-expression>

Supported Backends

  • s3 (AWS)
  • local
  • gcs (Google Cloud Storage)
  • azurerm (Azure)

Examples

vpc_id: !terraform.state vpc vpc_id
subnet_ids: !terraform.state vpc plat-ue2-prod private_subnet_ids
vpc_id: !terraform.state vpc {{ .stack }} vpc_id
first_subnet: !terraform.state vpc .private_subnet_ids[0]
username: !terraform.state config .config_map.username
# Default value for unprovisioned components
vpc_id: !terraform.state vpc .vpc_id // "default"
# YQ string concatenation
url: !terraform.state 'aurora-postgres .master_hostname | "jdbc:postgresql://" + . + ":5432"'
# Bracket notation for keys with special characters
key: !terraform.state security '.users["github-dependabot"].access_key_id'

Notes

  • 10-100x faster than !terraform.output (no init, no provider download)
  • Results are cached per execution
  • Supports YQ expressions and defaults
  • Same syntax as !terraform.output -- easy to migrate

!terraform.output

Read Terraform outputs by running terraform output. This works but is significantly slower than !terraform.state because it requires Terraform initialization (downloading providers). Use !terraform.state instead when your backend is supported.

Syntax

# Current stack
!terraform.output <component> <output>
!terraform.output <component> <yq-expression>

# Specific stack
!terraform.output <component> <stack> <output>
!terraform.output <component> <stack> <yq-expression>

Examples

vpc_id: !terraform.output vpc vpc_id
vpc_id: !terraform.output vpc plat-ue2-prod vpc_id
vpc_id: !terraform.output vpc {{ .stack }} vpc_id
first_subnet: !terraform.output vpc .private_subnet_ids[0]
username: !terraform.output config .config_map.username
# Default value for unprovisioned components
vpc_id: !terraform.output vpc .vpc_id // "fallback-id"
# YQ string concatenation
url: !terraform.output 'aurora-postgres .master_hostname | "jdbc:postgresql://" + . + ":5432"'
# Bracket notation for keys with special characters
key: !terraform.output security '.users["github-dependabot"].access_key_id'

Notes

  • Requires Terraform initialization (downloads providers) -- slow
  • Results are cached per execution
  • Prefer !terraform.state for supported backends (s3, local, gcs, azurerm)

!store

Read values from configured stores following the Atmos component/stack/key convention.

Syntax

# Current stack
!store <store_name> <component> <key>
!store <store_name> <component> <key> | default <default-value>
!store <store_name> <component> <key> | query <yq-expression>

# Specific stack
!store <store_name> <stack> <component> <key>
!store <store_name> <stack> <component> <key> | default <default-value>
!store <store_name> <stack> <component> <key> | query <yq-expression>

Examples

sg_id: !store prod/ssm security-group/lambda id
sg_id: !store prod/ssm plat-ue2-prod security-group/lambda id
sg_id: !store prod/ssm {{ .stack }} security-group/lambda id
kms_arn: !store prod/ssm kms config | query .arn
api_key: !store prod/ssm config api_key | default "not-set"

Notes

  • Constructs store keys from stack/component/key pattern
  • Use !store.get for arbitrary keys

!store.get

Retrieve arbitrary keys from stores without following the Atmos naming convention.

Syntax

!store.get <store_name> <key>
!store.get <store_name> <key> | default <default-value>
!store.get <store_name> <key> | query <yq-expression>
!store.get <store_name> <key> | default <default-value> | query <yq-expression>

Examples

# SSM Parameter Store
db_password: !store.get ssm /myapp/prod/db/password
feature_flag: !store.get ssm /features/new-feature | default "disabled"

# Redis
global_config: !store.get redis global-config
api_version: !store.get redis global-config | query .version
regional_config: !store.get redis "config-{{ .vars.region }}"

# Azure Key Vault
api_secret: !store.get azure-keyvault external-api-key
ssl_cert: !store.get azure-keyvault ssl-certificate | default ""

# Google Secret Manager
client_id: !store.get gsm oauth-config | query .client_id

Key Differences from !store

Feature!store!store.get
Key formatConstructs from stack/component/keyExact key as provided
Use caseAtmos component outputsArbitrary external values

!secret

Resolve a declared secret from its configured secret backend. The secret must be declared under the component's secrets.vars before it can be referenced.

components:
terraform:
api:
secrets:
vars:
DATADOG_API_KEY:
store: app-secrets
required: true
DATABASE_CONFIG:
store: app-secrets
required: true
vars:
datadog_api_key: !secret DATADOG_API_KEY
db_host: !secret DATABASE_CONFIG | path ".host" | default "localhost"

Use !secret instead of !store for sensitive values. Secret values are registered with the I/O masker and are redacted in output.

!append and !unset

Use !append to concatenate lists during stack inheritance instead of replacing them:

components:
terraform:
eks:
dependencies:
components: !append
- component: rds
- component: elasticache

Use !unset to delete inherited keys:

components:
terraform:
vpc:
vars:
enable_vpn_gateway: !unset

!emulator

Resolve connection details from an emulator component. Use this when local containers or components need endpoints for stack-scoped emulators instead of hardcoded localhost values.

Git Functions

Git functions derive repository metadata from the current Git repository:

vars:
git_host: !git.host
git_owner: !git.owner
git_name: !git.name
git_repo: !git.repository
git_url: !git.url

!env

Read environment variables from stack manifest env: sections or OS environment.

Syntax

# Read env var (null if not found)
!env <env-var-name>

# Read env var with default
!env <env-var-name> <default-value>

# Default with spaces
!env '<ENV_VAR> "default with spaces"'

Resolution Order

  1. Stack manifest env: sections (merged via inheritance)
  2. OS environment variables
  3. Default value (if provided)

Examples

api_key: !env API_KEY
app_name: !env APP_NAME my-app
description: !env 'APP_DESC "my application"'
region: !env AWS_REGION us-east-1

!exec

Execute shell scripts and assign the output.

Syntax

# Single-line command
!exec <command>

# Multi-line script
!exec
<line1>
<line2>
...

Examples

timestamp: !exec date +%s
result: !exec echo 42
config: !exec get-config.sh --format json

# Multi-line
computed: |
!exec
foo=0
for i in 1 2 3; do
foo+=$i
done
echo $foo

# With template for dynamic args
output: !exec atmos terraform output component1 -s {{ .stack }} --skip-init -- -json test_map

Notes

  • Uses the interp Go package (POSIX-compatible, Bash-like)
  • Complex types (lists, maps) must be returned as JSON strings
  • Atmos automatically decodes JSON output into YAML types
  • Prefer !terraform.output or !terraform.state over !exec atmos terraform output

!include

Include local or remote files, parsed by extension.

Syntax

# Include entire file
!include <file-path>

# Include with YQ query
!include <file-path> <yq-expression>

Supported Formats

ExtensionFormat
.jsonJSON
.yaml, .ymlYAML
.hcl, .tf, .tfvarsHCL
.txt, .md, othersRaw text

Supported Sources

ProtocolExample
Local (relative)!include ./config.yaml
Local (absolute)!include /path/to/file.yaml
Local (base_path)!include stacks/catalog/vpc/defaults.yaml
HTTPS!include https://example.com/config.yaml
GitHub!include github://org/repo/main/path/file.yaml
S3!include s3::https://bucket.s3.amazonaws.com/file.yaml
GCS!include gcs::gs://bucket/file.yaml
SCP/SFTP!include scp://user@host:/path/file.yaml
OCI!include oci://registry/image:path/file.yaml

Examples

# Local files
config: !include ./config.yaml
cidr: !include ./vpc_config.yaml .vars.ipv4_primary_cidr_block
vars: !include config/prod.tfvars
description: !include ./description.md

# Remote files
region_config: !include https://raw.githubusercontent.com/org/repo/main/config.yaml .vars

# Paths with spaces
values: !include '"~/My Documents/config.yaml"'

# YQ with bracket notation
key: !include ./config.yaml '.security.users["github-dependabot"].key'

!include.raw

Include files as raw text regardless of file extension. Useful when you want to treat a .json or .yaml file as a plain string.

Syntax

!include.raw <file-path>

!template

Evaluate Go template expressions and convert JSON results to proper YAML types.

Syntax

!template '<go-template-expression>'

Examples

# Complex outputs from atmos.Component
subnet_ids: !template '{{ toJson (atmos.Component "vpc" .stack).outputs.private_subnet_ids }}'
config_map: !template '{{ toJson (atmos.Component "config" .stack).outputs.config_map }}'

# Using settings
cidrs: !template '{{ toJson .settings.allowed_ingress_cidrs }}'

# Appending to lists
all_cidrs: !template '{{ toJson (concat .settings.allowed_ingress_cidrs (list "172.20.0.0/16")) }}'

Notes

  • Essential for handling lists and maps from atmos.Component
  • Converts JSON strings to proper YAML types
  • Prefer !terraform.output or !terraform.state over !template + atmos.Component

!literal

Preserve values exactly as written, bypassing all template processing.

Syntax

!literal "<value>"
!literal |
multi-line value

Examples

# Helm templates
annotation: !literal "{{ .Values.ingress.class }}"

# Terraform interpolation
user_data: !literal "${var.hostname}"

# ArgoCD
config: !literal "{{external.config_url}}"

# Inline arrays
users: [!literal "{{external.email}}", !literal "{{external.admin}}"]

# Regex patterns
pattern: !literal "^[a-z]+\\d{3}$"

!random

Generate cryptographically secure random integers.

Syntax

!random # 0 to 65535
!random <max> # 0 to max
!random <min> <max> # min to max (inclusive)

Examples

port: !random 1024 65535
id: !random 1000 9999
default_random: !random
small: !random 100

Notes

  • Values are NOT persisted -- regenerated each time Atmos processes config
  • Uses crypto/rand for cryptographic security

!cwd

Get the current working directory where Atmos is executed.

Syntax

!cwd

!repo-root

Get the root directory of the Atmos repository.

Syntax

!repo-root

!aws.account_id

Get the current AWS account ID using STS GetCallerIdentity.

Syntax

!aws.account_id

!aws.organization_id

Get the current AWS Organization ID using the AWS Organizations DescribeOrganization API.

Syntax

!aws.organization_id

Notes

  • Requires organizations:DescribeOrganization IAM permission
  • The AWS account must be a member of an AWS Organization
  • Results are cached per auth context for the duration of the Atmos execution
  • Uses a separate cache from the identity functions (!aws.account_id, etc.)

!aws.caller_identity_arn

Get the full ARN of the current AWS caller identity.

Syntax

!aws.caller_identity_arn

!aws.caller_identity_user_id

Get the unique user ID of the current AWS caller identity.

Syntax

!aws.caller_identity_user_id

!aws.region

Get the current AWS region from SDK configuration.

Syntax

!aws.region

YQ Expression Syntax

Several YAML functions accept YQ expressions for querying complex data:

# Array index
!terraform.output vpc .private_subnet_ids[0]

# Map key access
!terraform.output config .config_map.username

# Default values (// operator)
!terraform.output vpc .vpc_id // "fallback"

# String concatenation
!terraform.output 'db .hostname | "jdbc://" + . + ":5432"'

# Bracket notation for special characters
!terraform.output security '.users["github-dependabot"].key'

Quoting Rules

  • Use a single-quoted YAML scalar when an expression contains readable JSON or YQ string literals
  • Use double quotes inside brackets for string keys
  • Compact JSON such as {"key":"value"} may be a plain scalar; JSON containing : needs YAML quoting
  • Double a single quote only when it appears inside a single-quoted YAML scalar