Debugging terraform init: 401, 403, 502, and What Your Headers Are Telling You

terraform init failed. You got one cryptic line. You don't know if the problem is authentication, network, the registry, or your configuration. The error message tells you almost nothing.

Here is how to find out what went wrong in 30 seconds using curl.

soc2.compliance.tf is used as an example

All examples in this post use soc2.compliance.tf as the registry host. The same debugging steps apply to any compliance.tf framework endpoint: cis.compliance.tf, hipaa.compliance.tf, pcidss.compliance.tf, and others. Replace the hostname to match whichever framework you are using.

How terraform init talks to registries

Most Terraform users never see the HTTP requests that terraform init makes. Once you see the protocol, you can replay each step with curl and find the failure immediately.

The flow has three stages:

Service discovery. Terraform fetches /.well-known/terraform.json from the registry host. This JSON file tells Terraform where the module and provider APIs live.
Version listing. Terraform calls /v1/modules/{namespace}/{name}/{provider}/versions to get the list of available versions that match the version constraint in your configuration.
Download redirect. Terraform requests the specific version. The registry responds with a 204 No Content status and an X-Terraform-Get header pointing to the actual module archive. Terraform downloads the archive from that URL.

When terraform init fails, it could be failing at any of these three stages. The error message rarely tells you which one. Curl does.

Replace terraform init with curl

Instead of running terraform init and staring at a one-line error, replay the protocol steps with curl and read the actual HTTP responses.

Use curl -sI for the download endpoint

When debugging the download endpoint (Step 3), use curl -sI to send a HEAD request. The -I flag fetches headers only, which avoids downloading the module archive and shows you the status code and response headers that matter for debugging. For the discovery and versions endpoints (Steps 1-2), use curl -s with | jq to inspect the JSON response body.

Step 1: Service discovery

bash

curl -s "https://soc2.compliance.tf/.well-known/terraform.json" | jq

Expected response:

json

{
  "modules.v1": "/v1/modules/"
}

If this fails, the problem is DNS, network, or the registry host itself. Nothing Terraform-specific yet.

Step 2: Version listing

bash

curl -s "https://soc2.compliance.tf/v1/modules/terraform-aws-modules/s3-bucket/aws/versions" \
  -H "Authorization: Bearer YOUR_TOKEN" | jq

Expected response:

json

{
  "modules": [
    {
      "versions": [
        { "version": "5.0.0" },
        { "version": "4.0.0" }
      ]
    }
  ]
}

This is how Terraform decides which version to download. It matches the version constraint in your configuration (e.g., version = "~> 5.0") against this list and picks the highest matching version. If this call returns an empty list, Terraform cannot resolve the module and fails with a "no available releases match" error.

This endpoint requires authentication. A 401 here means the same thing as on the download endpoint: missing or invalid token.

Step 3: Module download

bash

curl -sI "https://soc2.compliance.tf/v1/modules/terraform-aws-modules/s3-bucket/aws/5.0.0/download" \
  -H "Authorization: Bearer YOUR_TOKEN"

A successful response looks like:

text

HTTP/2 204
x-terraform-get: https://prod-ctf-registry.s3.amazonaws.com/...signed-url...
cache-control: private, no-store
x-ctf-rules-hash: none
vary: Authorization

If you see anything other than 204, the status code tells you exactly what went wrong. The rest of this guide covers each failure case.

401 Unauthorized

Symptoms

text

Error: Failed to query available provider packages

Could not retrieve the list of available versions for module
"terraform-aws-modules/s3-bucket/aws" from soc2.compliance.tf:
error querying module registry: 401 Unauthorized.

Causes

No token configured. Terraform has no credentials for this registry host.
Expired token. On compliance.tf, tokens issued by terraform login are JWTs that expire after 24 hours. Static API tokens created from the dashboard do not expire automatically but can be revoked.
Wrong host in credentials. You have a token for registry.compliance.tf but the module source uses soc2.compliance.tf.
Missing credentials file. The credentials.tfrc.json file doesn't exist or isn't where Terraform expects it.

Debug

Check whether Terraform can find your credentials:

bash

# Check the credentials file exists
cat ~/.terraform.d/credentials.tfrc.json | jq

# You should see an entry like:
# {
#   "credentials": {
#     "soc2.compliance.tf": {
#       "token": "ctf_..."
#     }
#   }
# }

Verify the token is valid:

bash

curl -sS "https://registry.compliance.tf/whoami" \
  -H "Authorization: Bearer YOUR_TOKEN" | jq

The /whoami endpoint only supports GET requests (not HEAD), so use -sS instead of -sI here. The -sS flag keeps curl silent for progress output but still shows errors if the request fails.

A valid response returns your account details, plan tier, and which frameworks you have access to. A 401 here confirms the token itself is bad.

Fix

Re-authenticate with the registry:

bash

terraform login soc2.compliance.tf

If this is your first time setting up the registry, follow the getting started guide which covers account creation, token configuration, and your first terraform init.

Or manually create ~/.terraform.d/credentials.tfrc.json:

json

{
  "credentials": {
    "soc2.compliance.tf": {
      "token": "ctf_your_token_here"
    }
  }
}

For the full specification of credentials.tfrc.json, including environment variable overrides and per-host configuration, see the Terraform CLI credentials documentation.

One host, one token

Each compliance.tf subdomain (soc2.compliance.tf, hipaa.compliance.tf, pcidss.compliance.tf) shares the same authentication backend. A single token works for all of them, but credentials.tfrc.json needs a separate entry for each host you use. Wildcard entries are not supported — you must add each subdomain explicitly.

Two types of tokens

Compliance.tf supports two authentication methods:

JWT tokens — issued when you run terraform login against a compliance.tf host. Stored in ~/.terraform.d/credentials.tfrc.json. Valid for 24 hours. After expiration, run terraform login again to get a fresh token.
Static API tokens — created from the compliance.tf dashboard. Prefixed with ctf_. Long-lived with no automatic expiration. Can be revoked from the dashboard at any time.

The /whoami endpoint returns a token_type field (jwt or custom) and token_expires timestamp so you can check which kind of token you are using and when it expires.

.netrc vs credentials.tfrc.json

Which credentials file Terraform reads depends on how you reference the module:

credentials.tfrc.json — used when modules are sourced via the Terraform Registry protocol (e.g., source = "soc2.compliance.tf/terraform-aws-modules/s3-bucket/aws"). Terraform sends the token as a Bearer header.
.netrc — used when modules are sourced via HTTPS URLs (e.g., source = "https://soc2.compliance.tf/..."). The token is sent as HTTP Basic authentication. Note that .netrc does not support wildcards, so you need a separate entry per host — just like credentials.tfrc.json.

Both formats use the same token value. The registry authorizer accepts both Bearer and Basic authentication.

403 Forbidden

Symptoms

text

Error: Failed to query available provider packages

Could not retrieve the list of available versions for module
"terraform-aws-modules/s3-bucket/aws" from soc2.compliance.tf:
error querying module registry: 403 Forbidden.

Causes

Free tier accessing a paid framework. Your account has access to CIS but you are requesting from soc2.compliance.tf.
Trial expired. Your trial period ended and the account reverted to the free tier.
Module not available. The module exists in the catalog but is not yet available for the requested framework.

Debug

The error response tells you exactly what happened:

bash

curl -s "https://soc2.compliance.tf/v1/modules/terraform-aws-modules/s3-bucket/aws/5.0.0/download" \
  -H "Authorization: Bearer YOUR_TOKEN" | jq

json

{
  "code": "FRAMEWORK_NOT_AVAILABLE",
  "message": "Your current plan does not include access to the SOC 2 framework.",
  "remediation": "Use cis.compliance.tf (included in all plans) or upgrade at https://compliance.tf/pricing"
}

The code field is machine-readable. The remediation field tells you what to do next.

Fix

If you need SOC 2, HIPAA, or PCI DSS modules, upgrade your plan. If you are evaluating, switch to the CIS framework which is available on all tiers:

terraform

module "s3_bucket" {  source  = "cis.compliance.tf/terraform-aws-modules/s3-bucket/aws"  version = "5.0.0"  bucket  = "my-data"}

502/504 Gateway Timeout

Symptoms

text

Error: Failed to query available provider packages

Could not retrieve the list of available versions for module
"terraform-aws-modules/s3-bucket/aws" from soc2.compliance.tf:
error querying module registry: 502 Bad Gateway.

Or the command hangs for up to 60 seconds and then returns a 504.

Causes

First-time module generation. The specific module + version + framework combination is being generated on demand. This involves cloning the upstream module, applying compliance patches, packaging the result, and uploading it to storage.
Lambda cold start. The backend function that generates modules had no recent traffic and is initializing.

Debug

bash

# Check the response headers
curl -sI "https://soc2.compliance.tf/v1/modules/terraform-aws-modules/s3-bucket/aws/5.0.0/download" \
  -H "Authorization: Bearer YOUR_TOKEN"

Look for a Retry-After header in the response. If present, it tells you how many seconds to wait before retrying.

What happens behind the scenes

The first download of any module + version + framework combination triggers on-demand patching. This process can take up to 60 seconds (the backend Lambda timeout). During that time, the registry may return a 502 because the backend has not finished generating the artifact.

The second request for the same combination is instant. The patched module is cached in S3 and served directly.

The fix is simple: wait 30-60 seconds and run terraform init again.

bash

# First attempt may fail with 502
terraform init

# Wait for generation to complete
sleep 60

# Second attempt succeeds (cached)
terraform init

CI pipelines

If you hit 502s in CI, add a retry with backoff to your terraform init step. The first run in a new environment triggers generation; subsequent runs are fast. Most CI systems have built-in retry support.

Reading response headers

The compliance.tf registry includes debugging headers in every response. Here is what they mean.

bash

curl -sI "https://soc2.compliance.tf/v1/modules/terraform-aws-modules/s3-bucket/aws/5.0.0/download" \
  -H "Authorization: Bearer YOUR_TOKEN"

text

HTTP/2 204
cache-control: private, no-store
x-ctf-rules-hash: none
x-terraform-get: https://prod-ctf-registry.s3.amazonaws.com/...
vary: Authorization

Header	Meaning
`Cache-Control: private, no-store`	The response is per-user. Do not cache it in a shared proxy or CDN. Each user gets a unique presigned download URL.
`X-CTF-Rules-Hash`	Fingerprint of the operational rules applied to this module. Currently returns `none` for all responses. When rule-hash tracking ships, this header will contain a hash of the specific rules that were applied during module patching, letting you verify exactly which rule version produced the artifact you downloaded. A value of `none` means the module was served without rule modifications (compliance controls only).
`X-Terraform-Get`	The actual download URL. This is a time-limited, presigned S3 URL pointing to the patched module archive. Terraform follows this URL automatically.
`Vary: Authorization`	The response changes based on the `Authorization` header. Two different tokens may get different presigned URLs.

Quick troubleshooting with curl

Two requests can rule out the most common problems before you dig deeper.

Check if the registry is reachable:

bash

curl -s "https://registry.compliance.tf/.well-known/terraform.json" | jq

If this returns a valid JSON response, the registry is up. If it fails, the problem is DNS, network, or the registry itself.

Verify your token:

bash

curl -sS "https://registry.compliance.tf/whoami" \
  -H "Authorization: Bearer YOUR_TOKEN" | jq

This returns your account tier, token type, expiration, and available frameworks. A 401 means the token is bad. A successful response with a different tier than you expected means your plan changed.

Quick reference

Status Code	Cause	Fix
`401`	Missing or invalid token	Run `terraform login` or check `credentials.tfrc.json`
`403`	Plan does not include this framework	Switch to CIS (free) or upgrade your plan
`404`	Module or version does not exist	Check the module catalog for supported modules and versions
`429`	Rate limited	Wait and retry; reduce parallelism in CI
`502`	Module being generated on demand	Wait 60 seconds and retry
`504`	Generation timed out	Retry; if persistent, contact support

Next steps

The debugging loop is always the same: replace terraform init with a curl request, read the HTTP response, fix the problem.

Browse the module catalog to confirm which modules and versions are available.
Review the getting started guide for initial setup and authentication.
Check the frameworks page to see which frameworks your plan includes.

Debugging checklist

Run curl -s against /.well-known/terraform.json to verify the registry is reachable.
Run curl -sI with your token against the module download endpoint to get the real status code.
Check the quick reference table above to match the status code to a fix.
Verify your credentials.tfrc.json has an entry for the correct host.

Continue the conversation

Discuss this post with the community or share it with your network.

Discuss

Discusson Hacker News

Discusson r/terraform

Discusson r/devops

Shareon X

Shareon LinkedIn