Debugging terraform init: 401, 403, 502, and What Your Headers Are Telling You

terraform init failed. You got one cryptic line. You don't know if the problem is authentication, network, the registry, or your configuration. The error message tells you almost nothing.

Here is how to find out what went wrong in 30 seconds using curl.

soc2.compliance.tf is used as an example

All examples in this post use soc2.compliance.tf as the registry host. The same debugging steps apply to any compliance.tf framework endpoint: cis.compliance.tf, hipaa.compliance.tf, pcidss.compliance.tf, and others. Replace the hostname to match whichever framework you are using.

How terraform init talks to registries

Most Terraform users never see the HTTP requests that terraform init makes. Once you see the protocol, you can replay each step with curl and find the failure immediately.

The flow has three stages:

Service discovery. Terraform fetches /.well-known/terraform.json from the registry host. This JSON file tells Terraform where the module and provider APIs live.
Version listing. Terraform calls /v1/modules/{namespace}/{name}/{provider}/versions to get the list of available versions that match the version constraint in your configuration.
Download redirect. Terraform requests the specific version. The registry responds with a 204 No Content status and an X-Terraform-Get header pointing to the actual module archive. Terraform downloads the archive from that URL.

When terraform init fails, it could be failing at any of these three stages. The error message rarely tells you which one. Curl does.

Replace terraform init with curl

Instead of running terraform init and staring at a one-line error, replay the protocol steps with curl and read the actual HTTP responses.

Use curl -sI for the download endpoint

When debugging the download endpoint (Step 3), use curl -sI to send a HEAD request. The -I flag fetches headers only, which avoids downloading the module archive and shows you the status code and response headers that matter for debugging. For the discovery and versions endpoints (Steps 1-2), use curl -s with | jq to inspect the JSON response body.

Step 1: Service discovery

bash

curl -s "https://soc2.compliance.tf/.well-known/terraform.json" | jq

Expected response:

json

{
  "modules.v1": "/v1/modules/"
}

If this fails, the problem is DNS, network, or the registry host itself. Nothing Terraform-specific yet.

Step 2: Version listing

bash

curl -s "https://soc2.compliance.tf/v1/modules/terraform-aws-modules/s3-bucket/aws/versions" \
  -H "Authorization: Bearer YOUR_TOKEN" | jq

Expected response:

json

{
  "modules": [
    {
      "versions": [
        { "version": "5.0.0" },
        { "version": "4.0.0" }
      ]
    }
  ]
}

This is how Terraform decides which version to download. It matches the version constraint in your configuration (e.g., version = "~> 5.0") against this list and picks the highest matching version. If this call returns an empty list, Terraform cannot resolve the module and fails with a "no available releases match" error.

This endpoint requires authentication. A 401 here means the same thing as on the download endpoint: missing or invalid token.

Step 3: Module download

bash

curl -sI "https://soc2.compliance.tf/v1/modules/terraform-aws-modules/s3-bucket/aws/5.0.0/download" \
  -H "Authorization: Bearer YOUR_TOKEN"

A successful response looks like:

text

HTTP/2 204
x-terraform-get: https://prod-ctf-registry.s3.amazonaws.com/...signed-url...
cache-control: private, no-store, max-age=0, must-revalidate
pragma: no-cache
vary: Authorization

If you see anything other than 204, the status code tells you exactly what went wrong. The rest of this guide covers each failure case.

401 Unauthorized

Symptoms

text

Error: Failed to query available provider packages

Could not retrieve the list of available versions for module
"terraform-aws-modules/s3-bucket/aws" from soc2.compliance.tf:
error querying module registry: 401 Unauthorized.

Causes

No token configured. Terraform has no credentials for this registry host.
Expired token. On compliance.tf, tokens issued by terraform login are JWTs that expire after 24 hours. Static API tokens created from the dashboard do not expire automatically but can be revoked.
Wrong host in credentials. You have a token for registry.compliance.tf but the module source uses soc2.compliance.tf.
Missing credentials file. The credentials.tfrc.json file doesn't exist or isn't where Terraform expects it.

Debug

Check whether Terraform can find your credentials:

bash

# Check the credentials file exists
cat ~/.terraform.d/credentials.tfrc.json | jq

# You should see an entry like:
# {
#   "credentials": {
#     "soc2.compliance.tf": {
#       "token": "ctf_..."
#     }
#   }
# }

Verify the token is valid:

bash

curl -sS "https://registry.compliance.tf/whoami" \
  -H "Authorization: Bearer YOUR_TOKEN" | jq

The /whoami endpoint only supports GET requests (not HEAD), so use -sS instead of -sI here. The -sS flag keeps curl silent for progress output but still shows errors if the request fails.

A valid response returns your account details, plan tier, and which frameworks you have access to. A 401 here confirms the token itself is bad.

Fix

Re-authenticate with the registry:

bash

terraform login soc2.compliance.tf

If this is your first time setting up the registry, follow the getting started guide which covers account creation, token configuration, and your first terraform init.

Or manually create ~/.terraform.d/credentials.tfrc.json:

json

{
  "credentials": {
    "soc2.compliance.tf": {
      "token": "ctf_your_token_here"
    }
  }
}

For the full specification of credentials.tfrc.json, including environment variable overrides and per-host configuration, see the Terraform CLI credentials documentation.

One host, one token

Each compliance.tf subdomain (soc2.compliance.tf, hipaa.compliance.tf, pcidss.compliance.tf) shares the same authentication backend. A single token works for all of them, but credentials.tfrc.json needs a separate entry for each host you use. Wildcard entries are not supported — you must add each subdomain explicitly.

Two types of tokens

Compliance.tf supports two authentication methods:

JWT tokens — issued when you run terraform login against a compliance.tf host. Stored in ~/.terraform.d/credentials.tfrc.json. Valid for 24 hours. After expiration, run terraform login again to get a fresh token.
Static API tokens — created from the compliance.tf dashboard. Prefixed with ctf_. Long-lived with no automatic expiration. Can be revoked from the dashboard at any time.

The /whoami endpoint returns a token_type field (jwt or custom) and token_expires timestamp so you can check which kind of token you are using and when it expires.

.netrc vs credentials.tfrc.json

Which credentials file Terraform reads depends on how you reference the module:

credentials.tfrc.json — used when modules are sourced via the Terraform Registry protocol (e.g., source = "soc2.compliance.tf/terraform-aws-modules/s3-bucket/aws"). Terraform sends the token as a Bearer header.
.netrc — used when modules are sourced via HTTPS URLs (e.g., source = "https://soc2.compliance.tf/..."). The token is sent as HTTP Basic authentication. Note that .netrc does not support wildcards, so you need a separate entry per host — just like credentials.tfrc.json.

Both formats use the same token value. The registry authorizer accepts both Bearer and Basic authentication.

403 Forbidden

Symptoms

text

Error: Failed to query available provider packages

Could not retrieve the list of available versions for module
"terraform-aws-modules/s3-bucket/aws" from soc2.compliance.tf:
error querying module registry: 403 Forbidden.

Causes

Free tier accessing a paid framework. Your account has access to CIS but you are requesting from soc2.compliance.tf.
Trial expired. Your trial period ended and the account reverted to the free tier.
Module not available. The module exists in the catalog but is not yet available for the requested framework.

Debug

The error response tells you exactly what happened:

bash

curl -s "https://soc2.compliance.tf/v1/modules/terraform-aws-modules/s3-bucket/aws/5.0.0/download" \
  -H "Authorization: Bearer YOUR_TOKEN" | jq

json

{
  "error": "Framework 'soc_2' requires a paid subscription. Your free tier includes CIS v6.0 at cis.compliance.tf. Upgrade at compliance.tf/pricing",
  "code": "FRAMEWORK_NOT_ENTITLED",
  "request_id": "3f9c2d1e-8a4b-4c7d-9e2f-1a5b6c7d8e9f",
  "timestamp": "2026-08-20T12:00:00Z",
  "remediation": "Upgrade at compliance.tf/pricing or use cis.compliance.tf (free tier)"
}

The code field is machine-readable. The remediation field tells you what to do next, and request_id is what support will ask for if you open a ticket.

Fix

If you need SOC 2, HIPAA, or PCI DSS modules, upgrade your plan. If you are evaluating, switch to the CIS framework which is available on all tiers:

terraform

module "s3_bucket" {  source  = "cis.compliance.tf/terraform-aws-modules/s3-bucket/aws"  version = "5.0.0"  bucket  = "my-data"}

503 Service Unavailable (and the rare 502)

Symptoms

text

Error: Failed to query available provider packages

Could not retrieve the list of available versions for module
"terraform-aws-modules/s3-bucket/aws" from soc2.compliance.tf:
error querying module registry: 503 Service Unavailable.

Causes

First-time module build. The specific module + version + framework combination is being generated on demand: the registry fetches the upstream module, applies compliance patches, packages the result, and uploads it to storage. Only one request wins the build; concurrent requests for the same combination wait briefly and, if the build has not landed yet, get a 503 with a Retry-After header.
Build failure. If generation itself fails, the registry returns a 502 instead. That one does not fix itself with a retry.

Debug

bash

# Check the response headers
curl -sI "https://soc2.compliance.tf/v1/modules/terraform-aws-modules/s3-bucket/aws/5.0.0/download" \
  -H "Authorization: Bearer YOUR_TOKEN"

A 503 carries a Retry-After header that tells you how many seconds to wait before retrying, and a body with "code": "MODULE_BUILD_IN_PROGRESS". A 502 carries "code": "MODULE_GENERATION_FAILED" and a request_id to send to support.

What happens behind the scenes

The first download of any module + version + framework combination triggers an on-demand build behind a short lease. A typical cold build completes in well under a minute; while it runs, concurrent requests for the same combination return 503 with Retry-After set. Once the build lands, the patched module is cached in S3 and every later request for that combination is served directly from cache.

You usually do not have to do anything. Terraform's registry client honors Retry-After on 429 and 503 responses and sleeps for the advertised interval before retrying, so a plain terraform init normally rides out the build on its own. If you script the protocol with curl, honor Retry-After yourself:

bash

# First attempt may return 503 while the module variant is built
terraform init

# Terraform retries automatically after the advertised delay.
# If it still fails, re-run once the Retry-After window has passed:
terraform init

A persistent 502 with MODULE_GENERATION_FAILED means the build itself failed. Retrying will not help; contact support with the request_id from the response body.

CI pipelines

Terraform retries 503 responses automatically, so most CI runs need no special handling. If your pipeline calls the registry API directly (curl, custom tooling), honor the Retry-After header on 429 and 503. The first run in a new environment triggers the build; subsequent runs are fast.

Reading response headers

The compliance.tf registry includes debugging headers in every response. Here is what they mean.

bash

curl -sI "https://soc2.compliance.tf/v1/modules/terraform-aws-modules/s3-bucket/aws/5.0.0/download" \
  -H "Authorization: Bearer YOUR_TOKEN"

text

HTTP/2 204
cache-control: private, no-store, max-age=0, must-revalidate
pragma: no-cache
x-terraform-get: https://prod-ctf-registry.s3.amazonaws.com/...
vary: Authorization

Header	Meaning
`Cache-Control: private, no-store, max-age=0, must-revalidate` + `Pragma: no-cache`	The response is per-user. Do not cache it in a shared proxy, corporate forward proxy, or registry mirror. Each user gets a unique presigned download URL. The hash of the operational rules and controls applied to a module no longer travels in a response header; it is recorded in the module manifest inside the artifact, where tooling can actually read it.
`X-Terraform-Get`	The actual download URL. This is a time-limited, presigned S3 URL pointing to the patched module archive. Terraform follows this URL automatically.
`Vary: Authorization`	The response changes based on the `Authorization` header. Two different tokens may get different presigned URLs.

Quick troubleshooting with curl

Two requests can rule out the most common problems before you dig deeper.

Check if the registry is reachable:

bash

curl -s "https://registry.compliance.tf/.well-known/terraform.json" | jq

If this returns a valid JSON response, the registry is up. If it fails, the problem is DNS, network, or the registry itself.

Verify your token:

bash

curl -sS "https://registry.compliance.tf/whoami" \
  -H "Authorization: Bearer YOUR_TOKEN" | jq

This returns your account tier, token type, expiration, and available frameworks. A 401 means the token is bad. A successful response with a different tier than you expected means your plan changed.

Quick reference

Status Code	Cause	Fix
`401`	Missing or invalid token	Run `terraform login` or check `credentials.tfrc.json`
`403`	Plan does not include this framework	Switch to CIS (free) or upgrade your plan
`404`	Module or version does not exist	Check the module catalog for supported modules and versions
`429`	Rate limited	Wait and retry; reduce parallelism in CI
`502`	Module generation failed	Contact support with the `request_id` from the response body
`503`	Module variant being built	Retry after the `Retry-After` delay (Terraform does this automatically)
`504`	Gateway timeout	Retry; if persistent, contact support

Next steps

The debugging loop is always the same: replace terraform init with a curl request, read the HTTP response, fix the problem.

Browse the module catalog to confirm which modules and versions are available.
Review the getting started guide for initial setup and authentication.
Check the frameworks page to see which frameworks your plan includes.

Debugging checklist

Run curl -s against /.well-known/terraform.json to verify the registry is reachable.
Run curl -sI with your token against the module download endpoint to get the real status code.
Check the quick reference table above to match the status code to a fix.
Verify your credentials.tfrc.json has an entry for the correct host.

Continue the conversation

Discuss this post with the community or share it with your network.

Discuss

Discusson Hacker News

Discusson r/terraform

Discusson r/devops

Shareon X

Shareon LinkedIn