Performance Optimization

Slow plans are almost never one problem. A run that takes 45 minutes is usually running more directories than it needs to, running them one at a time, and paying the same setup cost in each one.

This page walks through the levers, in the order that usually pays best:

Measure first — profile the run before changing anything.
Cut the fixed startup cost — every run pays it, whatever the change.
Run fewer dirspaces — the cheapest work is work you skip.
Run dirspaces concurrently — use the runner you are paying for.
Make each dirspace cheaper — cut per-directory setup, and the plan itself.
Audit your workflow steps — third-party tools are often the largest remaining cost.

Measure first

Do not tune from a guess. Every Terrateam operation writes a full step trace to the GitHub Actions log, and each step is timestamped. The runner logs one line per step, per directory:

2026-08-19T05:17:37.8221006Z INFO:root:STEP : RUN : /github/workspace/live/dev/lambda : {'type': 'init', 'extra_args': []}
2026-08-19T05:17:49.5175714Z INFO:root:STEP : RUN : /github/workspace/live/dev/lambda : {'cmd': ['terragrunt', 'validate'], 'type': 'run'}
2026-08-19T05:17:57.0497046Z INFO:root:STEP : RUN : /github/workspace/live/dev/lambda : {'type': 'plan', 'extra_args': ['-input=false']}

Subtracting consecutive timestamps per directory gives you the cost of every step. That is the number you are optimizing.

Profile a run

Rather than doing that by hand, open the slow Terrateam job on GitHub, copy the URL out of your browser, and hand it to the profiler:

curl -sO https://docs.terrateam.io/terrateam-run-profile.py
python3 terrateam-run-profile.py https://github.com/OWNER/REPO/actions/runs/12345678901/job/97731319483

If you would rather grab the logs yourself, the Download log archive button on the run page gives you a zip. Pass the zip as-is. There is no need to unpack it or work out which file inside it matters:

python3 terrateam-run-profile.py logs_32825125903.zip

A single log file and an unpacked log directory both work too. Only the URL form needs the GitHub CLI, to fetch the log for you. The profiler itself needs Python 3 and nothing else.

Output looks like this. This is a real three-directory Terragrunt plan:

Terrateam run 9d9e3c58  |  3 dirspaces  |  terragrunt
=====================================================
  wall clock                78.9s
  step time, summed         53.0s  across all directories
  measurement           3 at least (>); 2 at most (<)
  dirspace concurrency       2.8x  (35.0s of step time in a 12.6s window)

WHERE THE WALL CLOCK WENT
----------------------------------
  runner startup, before Terrateam      40.2s   50.9%  #########.........
  work manifest + repo config            6.5s    8.2%  #.................
  pre hooks                             14.4s   18.3%  ###...............
  dirspace steps                        12.6s   16.0%  ###...............
  post hooks                             3.6s    4.5%  #.................
  teardown after completion              1.6s    2.0%  ..................

STEP TIME BY TYPE
----------------------------------
  init                               26.4s   49.8%  #########......... x3
  infracost_setup                    10.3s   19.5%  ####.............. x1  (1 not exact)
  plan                                6.6s   12.5%  ##................ x3  (3 not exact)
  update_terrateam_github_token       6.1s   11.5%  ##................ x4
  drift_create_issue                  3.6s    6.8%  #................. x1  (1 not exact)

STEP TIME BY DIRSPACE
----------------------------------
  . (hooks, repo root)         18.0s   34.0%  ######............ (2 not exact)
  live/us-east-1/prod/ecs      12.6s   23.8%  ####.............. has changes (1 not exact)
  live/us-east-2/prod/rds      12.2s   23.0%  ####.............. has changes (1 not exact)
  live/us-east-1/prod/rds      10.2s   19.2%  ###............... has changes (1 not exact)

TERRATEAM API TIME  (15.6s, 19.7% of wall clock)
------------------------------------------------
  POST /api/github/v1/work-manifests/{id}/initiate           6.2s   40.0%  #######........... x2
  POST /api/github/v1/work-manifests/{id}/access-token       4.9s   31.7%  ######............ x3
  PUT /api/github/v1/work-manifests/{id}                     2.6s   16.6%  ###............... x1
  POST /api/github/v1/work-manifests/{id}/plans              1.2s    7.8%  #................. x3
  GET /repos/terrateam-demo/bh-demo/issues                   0.6s    3.9%  #................. x1

SLOWEST INDIVIDUAL STEPS
----------------------------------
  <   10.3s  infracost_setup            .  (at most)
      10.2s  init                       live/us-east-1/prod/ecs
       8.8s  init                       live/us-east-2/prod/rds
       7.4s  init                       live/us-east-1/prod/rds
       4.1s  update_terrateam_github_token .
  <    3.6s  drift_create_issue         .  (at most)

SIGNALS
----------------------------------
  * create_and_select_workspace is on for 3 of 3 dirspaces. If every
    directory uses the default workspace, setting it to false removes one
    tool invocation per dirspace.
  * Terrateam API calls are 20% of the run. That is latency to the Terrateam
    server, not your Terraform.
  * Runner startup is 40s before Terrateam does anything, 51% of the run.
    That is queue, action image build, and checkout.
  * cost estimation (infracost) is enabled in this run.

Read it in this order:

Where the wall clock went first, because it decides which section of this page you need. Time in runner startup is the fixed startup cost and has nothing to do with your Terraform. Time in dirspace steps is everything below that.
Dirspace concurrency against your parallel_runs setting. If you set parallel_runs: 5 and this says 1.4x, the directories are not overlapping, so raising the setting will not help until you find what is serializing them.
Step time by type, to pick the target. init at the top points at the provider cache. run(...) steps at the top point at your own workflow steps. plan at the top, with everything else small, points at state size.
Step time by dirspace, where NO CHANGES marks a directory that planned and produced nothing. Those are pure waste and belong in running fewer dirspaces.
Terrateam API time, which is latency to the Terrateam server rather than anything in your configuration. Raise it with support if it is a large share.
Signals, which calls out specific settings worth changing, based on what the run actually did.

A duration is only reported plainly when the log proves it: steps inside one directory run back to back, so the gap between two starts is the exact cost of the first. Where the log cannot prove an end, the number is marked rather than guessed. < means at most, > means at least, and ? means the end is unknown. The measurement line counts them.

Ask an LLM what to change

Once you have the profile, feed it and your configuration to an LLM and let it map the numbers onto the fixes on this page. This prompt works well:

You are optimizing a Terrateam run. I will give you:

1. The output of terrateam-run-profile.py for a slow run. It reports the run's
   phases, step time by type and by dirspace, Terrateam API time, and signals.
2. My .terrateam/config.yml.

Use these sources, and only these, for what Terrateam actually supports.
The JSON Schema is the authority; where prose and schema disagree, the schema wins.

- https://raw.githubusercontent.com/stategraph/stategraph/main/api_schemas/terrat/config-schema.json
    The config JSON Schema. Every valid key, its type, and its default.
    Most objects set additionalProperties:false, so an unknown key is an error.
- https://docs.terrateam.io/workflows/performance   (this tuning guide)
- https://docs.terrateam.io/reference/configuration (config reference prose)
- https://github.com/terrateamio/action             (the runner: how steps execute)
- https://github.com/stategraph/stategraph          (the server)

Produce:
- A ranked list of changes, each with the seconds it should save, taken from the
  profile's own numbers rather than guessed. Say when you cannot estimate one.
- Attribute every second to a phase from "WHERE THE WALL CLOCK WENT", so it is
  clear whether a change targets startup, hooks, the dirspace steps, or the
  Terrateam API.
- The exact YAML diff for each change.
- The risk of each change and how to verify it did not break anything.
- Anything the profile suggests is NOT a Terrateam configuration problem
  (runner sizing, a slow third-party tool, provider API latency, repository
  size) stated plainly as such.

Rules:
- Validate every key you propose against the JSON Schema before you emit it.
  Cite its schema path and the default it declares, for example
  definitions/version-1/properties/parallel_runs, default 3.
- If a key is not in the schema, do not propose it. Say it does not exist.
- One change per branch, so each saving is attributable.
- Ranked by seconds saved, not by how easy the change is.

<paste the profiler output and your config.yml here>

The value of the LLM step is the ranking and the YAML, not the diagnosis — the profiler already did the diagnosis.

Pointing the model at the schema is what keeps it honest. It is the same file Terrateam validates your configuration against, so a key that is not in it does not exist, whatever the model claims. If your model cannot fetch URLs, download the schema and paste it alongside your configuration:

curl -sO https://raw.githubusercontent.com/stategraph/stategraph/main/api_schemas/terrat/config-schema.json

If init or plan is slow under Terragrunt and the log does not say why, set both variables and re-run. Output is verbose but it shows every dependency resolution and every provider call.

hooks:
  all:
    pre:
      - type: env
        name: TG_LOG_LEVEL
        cmd: ["echo", "debug"]
      - type: env
        name: TF_LOG
        cmd: ["echo", "DEBUG"]

Keep each change on its own branch so you can attribute the saving. Two changes landed together tell you nothing about which one worked.

Cut the fixed startup cost

Before Terrateam plans a single directory, every run pays for the queue, the action image, and the checkout. Note every run: the tree builder, config builder, and indexer are separate work manifests, so each one that fires is another Actions run paying this cost again, in series, before your plan starts. The profiler cannot see any of it — it only covers the action step — so read these from the job timings in the GitHub Actions UI. On a small change, this fixed cost can be most of the run.

Do not rebuild the action image on every run

The Terrateam action is a Docker container action. Its action.yml declares image: 'Dockerfile', so uses: terrateamio/action@v1 makes GitHub build the image from source on every run, in every repository, for every job.

Terrateam also publishes the same image prebuilt:

ghcr.io/terrateamio/action:v1

Pulling the prebuilt image instead of building it is reported to save 30–45 seconds per run. The exact edit depends on your workflow file, so make the change, then compare the job timings before and after to confirm the saving on your own setup.

Cache the image on self-hosted runners

If you run Actions Runner Controller, start each job as a container and cache the images on the node, so the runner does not pull them again on every job. That is reported to save a further 5–15 seconds per run.

The floor for startup cost is a long-lived self-hosted runner on a VM that stays up between runs. It removes queue and provisioning time entirely. The trade-off is that the runner is no longer ephemeral, which matters if you rely on a clean environment per run for isolation.

Watch the checkout on a large repository

actions/checkout runs against your whole repository, not just the directories Terrateam plans. In a large monorepo this is a real line item, and it is one the profiler will never show you. If the job timings put meaningful time in checkout, that is a repository-size problem rather than a Terrateam configuration problem, and the fixes live in your workflow file.

Run fewer dirspaces

Runtime scales with the number of dirspaces in the operation. Before making each dirspace faster, check that every one of them needs to run.

Confirm what Terrateam thinks it should run by commenting terrateam repo-config on a pull request. That prints the fully evaluated configuration, including anything a config builder or the indexer generated.

Shared files that fan out

A shared file — a provider definition, a common parent, a root variable file — that appears in the file_patterns of every directory will trigger every directory when it changes. This is the single most common cause of a run that is ten times larger than the change.

Decide deliberately what a shared file should trigger, then narrow the pattern:

dirs:
  live/dev/lambda:
    when_modified:
      file_patterns:
        - "${DIR}/terragrunt.hcl"
        - "${DIR}/*.tf"
        - "${DIR}/*.tfvars"

See when_modified for the full pattern syntax, including ! exclusions.

Directories that should never run on their own

Module directories, templates, and scratch directories should not be dirspaces. Give them an empty pattern list:

dirs:
  modules:
    when_modified:
      file_patterns: []

See Ignoring a Directory.

Dependencies that pull in unchanged directories

By default, when a dependency changes, everything that depends on it is pulled into the run even if it has no changes of its own. If you want a directory to keep its place in the layering order but only run when it changes itself, use prune_on_no_change:

dirs:
  database:
    when_modified:
      depends_on:
        tag_query: 'dir:network'
        prune_on_no_change: true
      file_patterns: ["${DIR}/*.tf"]

Be aware that depends_on also serializes: a dependent directory cannot start until its dependency finishes. Dependencies you do not actually need cost you both extra dirspaces and lost concurrency.

The indexer: a trade, not a free win

If you are hand-maintaining file_patterns so that a module change triggers its consumers, the indexer does that for you. It maps module blocks and symlinks, marks module directories as not directly runnable, and marks the directories that reference them as runnable.

indexer:
  enabled: true

It is not free. Indexing is its own work manifest, peer to plan and apply, so it runs as a separate GitHub Actions run that must finish before your plan is evaluated. You pay the whole fixed startup cost a second time, in series, ahead of the run you actually wanted.

The cost is bounded. The index is stored against the commit, so it is built once per commit rather than once per operation. A re-plan on the same SHA reuses it, and only the first operation on a new commit waits.

So decide it on your own numbers:

Enable it when	Skip it when
Hand-written patterns are missing module consumers, so changes ship unplanned	Your patterns are already tight and correct
A shared module change over-triggers because patterns are broad and defensive	Profiles show few dirspaces and a large fixed startup cost
You have enough directories that a wrong dirspace set costs minutes	One extra serialized job is a large share of your total run

The break-even is simple: the indexer is worth it when the dirspaces it removes cost more than the extra run it adds. Profile before and after and compare the WHERE THE WALL CLOCK WENT totals across the whole pull request, not one job.

Run dirspaces concurrently

`parallel_runs`

Within a single GitHub Actions job, parallel_runs controls how many dirspaces run at once. The default is 3.

parallel_runs: 5

One important detail: init is serialized and plan is not. The runner wraps every init in a flock so that concurrent inits cannot corrupt a shared provider plugin cache. Plans then run at parallel_runs concurrency.

    
flowchart LR
subgraph serial["init (serialized by flock)"]
  i1["dir 1"] --> i2["dir 2"] --> i3["dir 3"] --> i4["dir 4"]
end
subgraph par["plan (parallel_runs)"]
  p1["dir 1"]
  p2["dir 2"]
  p3["dir 3"]
  p4["dir 4"]
end
serial --> par

So raising parallel_runs speeds up the plan phase and the steps around it, but not init. If init dominates your run, go to the provider cache instead.

Raise the value gradually and watch runner CPU, memory, and disk while you do. There is a point where more concurrency makes things slower, and where it sits depends on your runner size and how heavy your plans are. Measure it; do not assume it.

`batch_runs`

parallel_runs is bounded by one job on one runner. batch_runs splits the operation across multiple GitHub Actions jobs, which can run on separate runners:

parallel_runs: 10
batch_runs:
  enabled: true
  max_workspaces_per_batch: 50

With batch_runs enabled, tree builder, config builder, and the indexer each get their own run, and workspaces beyond max_workspaces_per_batch spill into additional runs.

See batch_runs.

Runner size and routing

Larger runners are worth more once parallel_runs is above the default, because concurrent plans compete for CPU and disk. Use runs_on to route heavy workflows to a bigger pool:

workflows:
  - tag_query: "production"
    runs_on: [self-hosted, linux, x64, large]

Self-hosted ephemeral runners have a specific trap: because the pod is thrown away after every run, every run re-downloads providers over your NAT gateway. That shows up on your cloud bill as well as your plan times. Fix it with a persistent cache volume — see below.

Make each dirspace cheaper

By default every dirspace installs providers into its own .terraform directory, so an operation over N dirspaces downloads the same provider N times. Point them at one shared cache with an all.pre hook:

hooks:
  all:
    pre:
      - type: env
        name: TF_PLUGIN_CACHE_DIR
        cmd: ["bash", "-c", "mkdir -p /tmp/tf-plugin-cache && echo /tmp/tf-plugin-cache"]

This is safe because the runner already serializes init.

Measured on terraform 1.5.7 with hashicorp/aws 5.31.0, random 3.6.0 and null 3.2.2 (96 MB of downloads, 381 MB installed):

Scenario	Without cache	With cache
3 dirspaces, sequential `init`	34.9 s, 288 MB downloaded	19.4 s, 96 MB downloaded
4 dirspaces at `parallel_runs: 3`	47–52 s, 1523 MB on disk	~24 s, 382 MB on disk

The same shape holds for terraform 1.6.6 and 1.7.0 and for tofu 1.6.3 and 1.9.0.

On ephemeral self-hosted runners, put the cache on a persistent volume (EBS, EFS, or an equivalent) so it survives between runs rather than only within one. A cache keyed on GITHUB_RUN_ID still helps across the dirspaces of a single run, but pays full price on the first directory of every run.

Drop a redundant `validate` step

A terraform validate or terragrunt validate step before plan is usually dead weight. plan surfaces the same configuration errors, so validate buys you nothing in CI while costing a full tool invocation in every dirspace. In one Terragrunt repository the validate step cost 14 s per dirspace against a 17 s plan — close to doubling the run.

Run validate in a pre-commit hook or a local check instead, and remove it from the workflow:

workflows:
  - tag_query: ""
    plan:
      - type: init
      - type: plan
        extra_args: ["-input=false"]

Skip workspace selection if you only use `default`

After init, Terrateam runs workspace select and, if that fails, workspace new. If you isolate environments by directory rather than by Terraform workspace, that is a wasted tool invocation per dirspace. Under Terragrunt it is worse than it sounds, because each invocation walks the whole dependency graph.

dirs:
  live/dev/lambda:
    create_and_select_workspace: false

See dirs.

If your dirs entries are generated by the bundled Terragrunt config builder, set it for every generated entry at once:

config_builder:
  enabled: true
  script: terragrunt-config-builder --no-create-and-select-workspace

Terragrunt: fetch dependency outputs from state

On a large dependency graph, Terragrunt resolves each dependency block by running tofu output (or terraform output) against that dependency — a separate process per dependency, per unit. Reading the state file directly removes that:

hooks:
  all:
    pre:
      - type: env
        name: TG_DEPENDENCY_FETCH_OUTPUT_FROM_STATE
        cmd: ["echo", "true"]

This is frequently the largest single win on a Terragrunt monorepo.

When the plan itself is the cost

Everything above trims the setup around the plan. If the profiler shows plan at the top while init and your own steps are already small, then the plan itself is the cost — and plan time tracks the size of the state, not the size of the change. Terraform and OpenTofu load and evaluate the whole state on every run, so a directory holding thousands of resources is slow even for a one-line diff. This is the case that none of the levers above will fix: the graph got too big.

There are two ways out.

Stategraph replaces the flat state file with a database, so a plan runs against a much smaller working set than the whole state, and independent root modules can plan and apply concurrently against the same server. The state stays whole; the plan stops reading all of it.

You keep your current module layout, and plan time stops tracking total state size, so the state can keep growing without ever needing the split.

engine:
  name: stategraph

Each directory is onboarded once, and you need a Stategraph server you operate. See the Stategraph integration for setup.

Audit your workflow steps

Once init and plan are under control, the remaining time is usually yours, not Terrateam’s. Every step you add to workflows.plan runs once per dirspace. A 60-second security scan across 10 dirspaces is 10 minutes of wall clock.

Three things to check.

Do not regenerate the plan JSON. If a step needs the plan as JSON, and an earlier step already produced it, reuse the file. Re-running terraform show -json — worse, terragrunt show -json, which re-walks the dependency graph — is pure duplication. Terrateam exposes the plan file path as $TERRATEAM_PLAN_FILE. A duplicated show is easy to miss, because each one looks cheap on its own. The profiler totals it across every dirspace, which is where it stops looking cheap.

Check whether the tool actually parallelizes. A tool that queues internally gets slower as you feed it more concurrent work, so raising parallel_runs makes it worse rather than better. One measured example: a scan that took 76 s when it ran alone took about 220 s per dirspace when three overlapped — of 2010 s of total scan time, only about 760 s was real work and the rest was queueing. If that is what you see, the tool belongs outside the plan path.

Move non-blocking tools out of the operation. Security scans, cost annotations, and AI summaries do not have to run inside the Terrateam action. Running them as a separate GitHub Actions workflow triggered on the same pull request keeps plan feedback fast, and they still have access to the branch, the diff, and the plan artifacts. Anything that does not gate the apply is a candidate.

Checklist

Symptom	Look at
Small changes still take minutes	Fixed startup cost: the action image is rebuilt every run
Job timings show time before the action step	Prebuilt image, node-level image cache, or a long-lived runner
One small change plans dozens of directories	`file_patterns`, shared parent files, `depends_on`
Module or template directories get planned	Ignoring a directory, then weigh the indexer
`init` dominates the run	Shared provider cache, committed lock files
Ephemeral runners, high NAT gateway egress	Provider cache on a persistent volume
Directories run one after another	`parallel_runs`, `batch_runs`
Runner is saturated	`runs_on` with a larger pool
Terragrunt is slow before Terraform even starts	`TG_DEPENDENCY_FETCH_OUTPUT_FROM_STATE`, `create_and_select_workspace: false`
`plan` dominates and setup is already small	The state is too large: splitting buys time, Stategraph removes the refactor
Third-party tools dominate	Reuse `$TERRATEAM_PLAN_FILE`, or move the tool to its own workflow
You do not know which of these it is	Measure first